Can't say I've played all of these handhelds, but here's my opinion (take with a vast number of pinches of salt):
Game Boy (Colour): Practically created the hand-held games market. May not have been first (wasn't the Lynx being developed before the Game Boy?), but certainly was the most popular handheld for the better part of the 90's, along with its colour sibling. Some top games, despite being a bit bulky, and battery life was as good as could be expected (if it can get through a week in France without needing new batteries, it's good enough)
Game Boy Advance (SP): The original model had a pretty awful screen, but the quality of the console was typically Nintendo. The SP, however, truly made the GBA into a great handheld, as its small form factor, backlight and rechargable battery have addressed just about every fault the original GBA had. Of course, it can't do full 3D, but its library is just HUGE.
Game Gear: Big and bulky, and ate its way through batteries, but for some reason I always did like the Game Gear. It had some great games, a lot ported from the Master System, and seemed like a reliably-built machine. The TV tuner was a gimmick, but reminds me of Sega at its height: always willing to do something just a wee bit differently from everyone else, despite it not really being a great idea.
GP32: Haven't used one, or even seen it in person, but it certainly looks a nice machine. If I'm right in thinking that its developers are actively encouraging fan-made titles, this could be a homebrewer's fantasy console: powerful and with developer's tools easily available. One to watch.
Game.com: Quite simply, an awful machine in every respect. Slow, underpowered, monochrome, and expensive if you wanted to use it for its advertised purpose (going online), and with a library of games that whilst on the surface seems attractive, shouldn't be spoken on in polite company.
Lynx (II): A machine that didn't get the attention it really deserved in the UK, and commands a good price second hand from its relative obscurity alone. There are some great games for it, although a good few mediocre, so really only for Atari fans or collectors. Still a nice machine though, and the Lynx II was a huge improvement.
NeoGeo Pocket (Colour): Love it. Love it as the handheld console I never knew I wanted. I remember seeing the NGPC in a local electronics shop and not really knowing what it was all about, but now I own one and don't regret a thing. Great little machine, great games, very nice battery life. Suit you, sir.
NGage (QD): The little machine that could..n't. For every bonus point (some nice games, powerful hardware, you can use it as a phone), there's a negative or two (deconstruction to change games, sidetalking, 'odd' controls). The QD has fixed many of the machine's problems, but it has a poor reputation to try and repair.
Sega Nomad: What's not to love about a portable Mega Drive? Apart from the size, the battery life.... well, the Nomad wasn't bad. It could plug into a TV, could have a second control pad plugged in, but wasn't really designed for portability.
Turbo Express: I've never used, hardly ever seen, but it looks a nice machine - but pretty fragile. I wouldn't want to drop one, and certainly wouldn't try repairing it. As far as playability goes, a portable TurboGraphix is never a bad thing.
Virtual Boy: Not a handheld; not a great console either. It was ambitious, by Nintendo's standards, but a bit too ambitious given the technology available. A lack of support sealed this machine's fate.
Wonderswan (and swan and on): You can tilt it, turn it, play Final Fantasy on it, and it's not a bad machine in its own right. Was unlikely to ever really unseat Nintendo from its dominance of the handheld market, but if any handheld were to have a stab at it, Bandai's wunderkind would have made a great winner.