Whatever Element you're using try using an +element relic. Stack explosive damage boosting skills if you're Axton. Throw it with max clip...that is all. If you don't have skills to boost explosive damage you're using it on the wrong character.
I've noticed that it has ammo regen and that once tossed the bullets do more damage than when firing it.
With no other modifiers, the bullets from the tossed gun will do the damage listed on the item card. The thing is, once it gets tossed, it is then affected by grenade damage, while simply firing it is not. I'm guessing you have boosted grenade damage.
If you learn how to judge the difference properly, it is unquestionably the most powerful Tediore weapon in terms of reload damage in the game. The fact it uses other damage modifiers on top of grenade damage would be devastating enough on it's own for most of the characters.. However, the big thing about Avenger is that amp shield damage works on it's reload damage, meaning Bee + Avenger is the new Bee + Baby Maker. Granted it isn't quite as powerful as Bee + Baby Maker, but it is still absolutely ridiculous with the right setup.
So, basically, the reload damage on it can be utterly overpowered with a proper setup, but the downside is that it requires some degree of skill to pull off. You can't just run into an enemy's face while spamming reload.