Hardest part of the game? I'm throwing my vote in for ladders. It's not just limited to BL2 though. I get more frustrated trying to go up ladders than fighting any enemies in every game I've ever played that has ladders! Haha!
Doorways are a very big problem in this game. Not just when being attacked. In Sanctuary and Holy Spirits the NPCs just LOVE to walk in/out of doors when you are trying to get through. You have to wait for those idiots all the time. I'd wager that 1.3% of my BL2 game time is waiting for NPCs to get the F*** out of my way.
Fog Ring Waaaahhhh People glitching to get souls Waaaaahhhh Lightning weapons at low levels Waaaahhhhh