I've gotten a 2 legendary grenade mod drops off of Boom Bewm (both with the Vault Hunter relic equipped), and the Veruc assault rifle from the guy by the church where the Black Queen spawns. If you have the DLC, make sure you have the Vault Hunter relic on - 5% boost to rare drops isn't much, but it helps a little. Also, most named mobs have a chance to drop a legendary. Here's a list on IGN's wiki with a list of bosses and what they might drop:
I once managed to get Knuckledragger's drop on the first play through. This was a Level 1 corrosive weapon (the name of which escapes me), which was still proving useful at Level 10, as it about the only corrosive weapon you can lay your hands on for a fair time.
I also managed the Grenade Mods from Bewm and Boom a couple of characters and a Kerblaster (Level 7 I think) from Midgemong.
The only one really worth farming early is Dr Mercy. He's the quickest boss to farm (with geyser trick) and drops the most newb friendly weapon. And its good exp. Some weapons can last you a while but you'll eventually outlevel all of it. Is it worth the hours of farming (many hours if you have bad luck)?