Good luck finding a PSP with 2.0 or 1.5 on it.
GP2X - pros
Great community. I'm amazed at how far it has come in just three months. We have A2600, A7800, Lynx, A800, TI-99, Apple?, MAME! emulation. Something you dont see done as well or at all on the PSP just yet. The Amiga emulator is VERY playable.. Basically I'm seeing alot of stuff on the GP2X that is either non existant or seems dropped on the PSP.
PS1 emulator looking pretty good!
SD cards (yeah)... I hate Pro Stick Duo's...
Movies are pretty much a dump and watch experience.
Not 100% sure but picture viewer seems to be able to handle more formats.
TV-Out.

Seems to handle sidescrolling games more smoothly
GP2X - cons
Joystick - has a couple things going against it, the cap (according to some), and the bias towards diagonals. I'm still undecided on it myself.
Can brick when updating firmware if not careful (PSP seems a little more failsafe, not that'd you'd want to and risk killing access to homebrew).
Sucks the life out of your batteries (so be sure to get rechargeables).
Interlaced LCD (purely opinion as far as I'm concerned). I think it makes alot of emu's look more "realistic". Most consoles were designed to run on interlaced TV anyways.
Boot time not the greatest, though for some reason this does not seem to be an issue for everyone.
Buttons used for volume control (please

). At least they can be used as extra buttons if the programmer wishes.
NES, SNES, (SMS & GG) emulation still a little bit behind. (currently)
Good luck seeing any commercial games.
Power supply sold seperately.
PSP - pros
Rechargeable batteries (yeah, yeah I know it's proprietary)
Keeps track of date and time (internal clock)
Quick boottime (unless your playing UMD's

)
Can play movie UMD's (if your into that kind of thing)
Can play music UMD's (are there even any out on the market?)
Commercial games available (mostly crap, but who knows what the future holds).
Nice screen!
Analog stick/nub (a matter of preference I suppose)
Wi-Fi
Web browser (2.0 and up only)
Sleep mode.
Comes with a power supply.
Great SNES, NES, and (SMS/GG) emulation
PSP - cons
Expensive
Expensive memory stick duo cards (blech)
Expensive games ($40).. yeah right!
Poor emulation support for older systems (mostly 8-bit systems)
Homebrew seems to be dying (Can anyone say firmware updates?)
Fast moving objects (like side scrollers) tend to blur.
There was software released lately that allows playing of 2.0 only games on 1.5 systems recentally. But in the long run Sony will probably win and snuff out all homebrew.
Personally I prefer the GP2X, despite it's flaws, and my PSP has not been touched for a few months now. I eventually may decide to give in and just update the PSP so I can play more recent releases.
I think that eventually the homebrew for the GP2X will far surpass what is available for the PSP.
That's all I can think of at the moment.