I've played a bunch of it offline, so here are my thoughts.
1. Online. I admit that I haven't tried this portion yet, but I find it a good sign that the developers have addressed this in the Steam forums very quickly.
2. The 3D backgrounds aren't there. Honestly, I don't mind this omission. They could look pretty bad. Remember how they looked on the Dreamcast? I'd rather use the 2D backgrounds anyway. I'd hardly call this a deal breaker either way you slice it.
3. No AST. I do find this a bit disappointing, but, once again, I don't find it to be a deal breaker. The Neo Geo had amazing sound, and the OST is nothing to slouch at. People on the Steam forums have been talking about modding the AST into the sound files. If that comes to pass, then this will largely be a non-issue. Also, while some people have been complaining that the overall sound quality of the game is poor, I personally don't find anything wrong with it.
4. When you get right down to it, it's a great game to play and enjoy. Ignoring the above problems, I haven't had any significant problems that I couldn't fix or work around. I'm not experiencing the half circle problems that some people are having. If you don't want the borders when playing in 16:9 resolutions, find the file named bg2.png in the install folder and make it all black. I never liked scanlines, so I don't mind that there's no scanline option. I don't have a stick, but I hear that setting the switch that controls which "normal controller directions" is used on the joystick to use the left analog stick as the directional input is the magic bullet.