Wow, nice. I think it's great you chose the AD Blackfin for this project - I was considering doing the same at some point. Have you had success in achieving 60FPS CGA screen draws? Is your design loosely based on the BF533 EZ-Kit lite?
Again, Great Work here!
Yes on both counts. I was developing the software, initially, on the BF533 EZ-Kit. I didn't like how their video output worked (it's made for DVD applications... nonsquare pixels, chroma subsampled, etc) so I built my own. I use the PPI port in 16-bit mode at 6.144MHz, to drive an ADV7125 video DAC with 5 bits of precision per channel. I send the 16th bit into a D-flip-flop which then drives the CSync signal. (For testing purposes, I've got it plugged into a NeoBitz-S video encoder.)
My code constructs the video line-by-line in the internal memory. It's interrupt-driven (from the PPI DMA) and only buffers 2 lines at any time, like a "real" sprite engine. Graphics data is stored in DRAM, externally. I'm re-considering that right now, though. Doing it a line-at-a-time means no writing back to the DRAM... but means terrible access patterns. Compositing everything into an actual framebuffer in DRAM would give much tighter access patterns, but would mean a read and a write for each pixel.
I was having no problem drawing 100 16x16 sprites on top of a scrolling background, as long as they didn't all bunch up on one line.