Vertically scaling sprites on real hardware, avoiding graphical artifacts

[city41]

When vertically scaling sprites, you often end up with graphical artifacts. I have been running some experiments to try and figure out when they happen and how to avoid them. It's all captured here

GitHub - city41/neo-geo-scaling: A sandbox to learn all the quirks about scaling sprites on the Neo Geo

A sandbox to learn all the quirks about scaling sprites on the Neo Geo - GitHub - city41/neo-geo-scaling: A sandbox to learn all the quirks about scaling sprites on the Neo Geo

github.com

If anyone has experience here and has input, I'd love to hear it.

I might be wrong, but so far I am feeling like the Sprite Shrinking page at the neogeodev wiki may not be fully accurate.

Sprite shrinking - NeoGeo Development Wiki

wiki.neogeodev.org

And so far it seems like the only way to avoid graphical artifacts is to ensure the entire sprite (tiles 0 through 31) is set. For tiles beyond the logical height of the sprite, it seems like you need to explicitly set them to a blank tile.

 

[city41]

This looks to be due to not padding the C ROMs out. Once they were padded, the black boxes on real hardware went away.

 

[city41]

I believe I have figured out what the issues are with vertical scaling. I rewrote the README with my findings. I also reached out to Furrtek and we are chatting about this. If my findings prove to be true and I'm not just doing something stupid/naive, we'll update the neogeodev wiki accordingly.

 

[Burning Fight!!]

The image's not appearing anymore. Interesting findings though!

 

[city41]

The image's not appearing anymore. Interesting findings though!

Oops, thanks, I just fixed that.

 

[slugger_dan]

I saw these 2 points on your repo:
“Tile 0xff fills in left over space for sprites shorter than 16”
“Sprites of height 16 use the last row of the last tile to fill in remaining space”

You’re seeing these 2 different results but I think they happen for the same reason. What about this explanation to cover both cases: “The last row of tile 15 in SCB1 fills in the left over space when displaying the first 256px of the sprite”. The samples with that checkerboard pattern are from “untouched” tiles. If these untouched tiles point to tile FF then the image looks right. You could change it to FE or whatever and probably see a different result. The other ones with the blue tile stretched to fit the screen make sense since that’s the last row of tile 15 too. The info on this wiki page lines up with what I’m seeing as well.

https://wiki.neogeodev.org/index.php?title=L0_ROM

 

[city41]

I saw these 2 points on your repo:
“Tile 0xff fills in left over space for sprites shorter than 16”
“Sprites of height 16 use the last row of the last tile to fill in remaining space”

You’re seeing these 2 different results but I think they happen for the same reason. What about this explanation to cover both cases: “The last row of tile 15 in SCB1 fills in the left over space when displaying the first 256px of the sprite”.

But when the sprite is shorter than 16 tiles, the empty space is not filled with the last row of a tile. It appears to be filled with the entire tile, with scaling taken into account. You can see that because the checkered pattern appears intact in samples 1 and 2. It is probably still intact in sample 3 too, but the increased scaling makes that harder to determine.

You could change it to FE or whatever and probably see a different result.

I'm not sure what you mean by change it to FE. I never specified tile FF anywhere in the code. It appears the LSPC grabs tile FF once it gets past the number of tiles as specified in the height parameter.

I also think this is supported by looking at commercial games which tend to leave tile FF blank. I added two examples of that in the README.

The other ones with the blue tile stretched to fit the screen make sense since that’s the last row of tile 15 too. The info on this wiki page lines up with what I’m seeing as well.

But sample 9 is 24 tiles tall. Once sample 9 hits tile 15, it's as if tiles 16 through 23 don't exist.

Furrtek believes this has something to do with the vertical scaling lookup table being mirrored for tiles 15 through 31. He also believes what I am seeing is possibly specific to the LSPC-A0 and may not happen on the LSPC2. He is going to check when he gets a chance.


EDIT: duh, setting sample 9 to 24 tiles is pretty silly seeing as I only defined 18 total tiles. I updated the code to make it 18 tiles tall. It becomes the same demo as sample 8 just scaled to 25% instead of 50%. It doesn't really change anything though, because tiles 17 and 18 not showing up and instead tile 16 getting "smeared" down the screen still suggests that sprites taller than 16 tiles behave strange when scaled.

 

[slugger_dan]

Honestly it's been a while since I messed around with sprites so take this with a grain of salt, but now I think I see what's happening with that scaled checkerboard and why that whole tile is repeated instead of repeating only the line. There's another important part I forgot about.

Looks like samples 1 and 2 are scaled to half size with a height of 4 tiles, so there's a 64px tall window where it'll draw whatever the L0 ROM tells it to draw with scale = 0x80. That should mean the first 64 bytes from address 0x8000 in the L0 ROM should map to whatever tiles in SCB1 to draw sprites with, that will decide what these 64 lines look like on screen. If we look at the L0 data there, the first 32 bytes/lines map to lines in the first 4 tiles since the L0 data here is less than 64. This displays your shrunk tile numbers and so far so good. The issue is the L0 data for the bottom 32 lines on screen in the display window (@ 0x8020 onwards) range from 0x40 to 0x7e, that means SCB1 tile data for tiles 4 through 7 will be displayed on screen with whatever tiles are defined in that range, and I could only spot the first 4 tiles of data actually being defined in your code. If SCB1 tiles 4 through 7 contain leftover FF bytes from the BIOS/flashcart/whatever, that would explain the checkerboard @ FF being displayed. Notice at the end of this table in L0 there are a bunch of FF bytes, that's what shows the last line of tile 15 but those only get used if either the sprite is tall enough or the sprite is shrunk a lot.

I remember thinking "reading the L0 table backwards thing" being weird but if I remember right, you can do things like making a 32 tile tall sprite, positioning it so that tile 0 is centered on screen, and then it can "wrap around" from tile 0x1e, 0x1f, (screen center) 0x00, 0x01 ... That way when you go from 100% zoom to smaller values, you'll have a very tall sprite that will sort of shrink towards the center of the display. Don't quote me on that though, it's been a while since I tried any of this.

edit: fixed wrong dimensions

 

[city41]

If SCB1 tiles 4 through 7 contain leftover FF bytes from the BIOS/flashcart/whatever, that would explain the checkerboard @ FF being displayed.

I think I can test this by first zeroing out vram before starting the tests. I'll do that next time I'm at my computer.

I'm not familiar with the L0 rom other than knowing vertical scaling uses a look up table. This is good info, gives me something to research.

I remember thinking "reading the L0 table backwards thing" being weird but if I remember right, you can do things like making a 32 tile tall sprite, positioning it so that tile 0 is centered on screen, and then it can "wrap around" from tile 0x1e, 0x1f, (screen center) 0x00, 0x01 ...

Actually I accidentally did this while building this little test app My ultimate goal is to pull off full screen scaling ala AOF. I have pulled it off by limiting sprites to being 16 tiles tall, but I'm hoping I can get the full 32 going.

I think my next steps are to get more familiar with MAME's debugging tools, and see if I can get some insight into what L0 is doing here and hopefully cross reference with the info you gave. I also think I'll get AOF running in MAME and see if I can see how SNK did their sprites, that might have some interesting insights too.

Thanks, this has been helpful.

 

[city41]

Cool, zeroing out vram caused the remaining area of the shorter sprites to be filled with the tile at C ROM index zero.

I changed that tile to this

, and got this result

That's pretty good news. I can work with that to avoid those artifacts. So commercial games leaving tile FF blank may just be a coincidence. Or also possible VRAM starts out filled with FF, and for these particular games it was just easier to blank out tile FF and not fuss with it. Just a guess.

I also realized the wiki is saying that when there is remaining window that needs to be filled, it will grab the last line of specifically the tile at index 15 in SCB1 for that sprite. Not the last line of the last tile as defined by the height value. So I changed the last sample to set the tile at index 15 to be the "last line is transparent" tile. That "removed" the smear, but really the smear is now just transparent. And the smear still wiped out displaying the tiles at indexes 16 and 17. The above MAME screenshot has this result (sample 9, if you count it's still only showing 16 tiles and not 18).

Next up, some MAME debugging...

 

[slugger_dan]

Nice! I suspect you're not seeing tiles 16 and 17 because L0 only works on 256lines at a time depending on what part of the window is displayed. I don't think you can make tiles 0-15 show up in any part of the window that could display tiles 16-31 so that might need some layout shuffling. That window portion on screen will only ever show tiles 0-15 with the blue fill taking the rest of the space, and the offscreen window portion (tiles 16-31) will be the other way around. Showing tiles 16 and up means moving the sprite/window so that the tiles "above" 0 are visible, then both 256line subsections of the window are visible and the whole thing will be displayed. A bit like the center-tile-0 thing I mentioned. This is probably better explained with pics than a bunch of words but if you moved that very tall sprite so that it's centered and tried filling tiles 1F and below, I think that would work. You'd have to think about the inverted L0 reading too when doing that.

 

[city41]

This is probably better explained with pics than a bunch of words but if you moved that very tall sprite so that it's centered and tried filling tiles 1F and below, I think that would work.

I don't totally follow what you are saying. But I think what you're getting at is what is happening to sprite #5 on this demo from the wiki?

You can just see that the first 16 visible tiles all start with "1", suggesting they are actually tiles 16-31, and then the next 16 visible tiles start with "0". So I think this is using wrapping to enable all tiles to display.

I'll see if I can reproduce this, and if so then I'll probably understand how to get both parts of the window showing. I do think this means I still need to limit my sprites to 16 though, as I'm not sure how I could use this in such a way as to scale an entire playing field (zooming in and out).

In theory dumping vram out of MAME is easy to do, and from there I could write a script that would parse it out and tell all about sprites, and then I could see how SNK made their sprites. But every time I do a dump MAME crashes hopefully I can get that going.

 

[city41]

Slowly but surely it's starting to make sense. It seems like wrapping the sprites and centering them is the key. I totally forgot I built a tool a long time ago that can show this: https://city41.github.io/nggm, you can just click on Samurai Shodown 2 in the modal and then drag Gen An's background up into the main area to get this

Sure enough that sprite is 32 tiles tall, and it, of course, scales up and down just fine.

I gotta head to bed, but tomorrow I'll dig even more and hopefully finally get that eureka moment.

 

[slugger_dan]

Yeah that demo screenshot does a good job of showing it. It's like there's a vanishing point between the first row of tile 0 and last row of tile 31 and the vertical shrink makes both parts of the window converge to that point. I think you're right about the 16 tile limit depending on what constraints you have here. Using tall sprites like in the demo is maybe ok for a fighting game background where it stays roughly centered and doesn't move too much, so using that demo pic as an example, you'd avoid having that solid orange/red colour in the demo coming into view. If you need more freedom to move and scale then I don't think it works, and then manually stitching 16tile-tall sprites makes more sense instead, Unless there's a trick I'm missing here.

 

[city41]

Awesome! I don't 100% get it yet but I'm sure after playing around with my little scaling demo some more with this in mind I will. Thanks for all the help!

edit: actually thinking a bit more I think I do totally get it now. woohoo! a very weird quirk of the neo that's for sure, but hey, that's why this stuff is so interesting.