But I guess there were two problems I had. One was that the scope of the crisis kind of made me scream "where is Iron man? Fucking Thor?!" I mean these damn ships have the ability to wipe out mankind and the rest of the avengers are on holiday.
This is the kind of thing I always interpret as 'the other guys are busy.' If you read comics at all, this sort of shit always happens. And i understand that regular audiences have all the same questions you do, but I interpret it this way:
Thor is in space and Tony Stark doesn't have armor anymore after the events of Iron Man 3, so he really cant do anything. Until Avengers 2 establishes that Tony's in a position to actually do anything, all he can do is sit around like everyone else and wonder.
Banner is probably half way around the world hiding from the problems he's always hidden from and ran away from.
Hawkeye is the only real question mark, and he could easily be 'on assignment' somewhere. Perhaps specifically to keep him from hooking up with Natasha and Steve. Maybe he was even trying to get back but was being cockblocked at every opportunity.
None of these things ar explicitly stated but they're all reasonable conclusions one could draw if they wanted to fill in the plot holes.
And don't get me started about how Spider-Man is never helping anyone.
Marvel can't put Spidey in a movie. And anyway, this movie takes place in DC and Spidey is in New York. Even if they COULD use him, he'd never get there in time.