Get yourself at least a 2gig memory stick for all of your musics and videos. If you do upgrade your firmware and go near wifi spots; you can update XML mp3 feeds automatically which is far better than what an iPod can do.
Gradius, Tekken, any Capcom Classics, SF3Upper, Ridge Racer Darkstalkers (if you don't mind the load time), and MGS: Portable Ops are all golden. If you are into RPGs, check out the European ports of Tales of Eternia and Breath of Fire 3.
If you are indeed getting a PS3, lests say that it is the ultimate PS3 peripherial since you can download PSone games onto it at launch.
Also, I also recommend you to get the golden Sony battery pack if you want to run your software for a longer period of time.
Also check this
out if you feel adventurous by opening up your new PSP.
BTW, look out for Metal Slug collection and the Genesis collections for the PSP as well.