According to data collected by some of our board-mates, starter familiars (I.E. any familiar you didn't serenade with Esther or use a ticket for) has a base growth rate of 0. Meaning that regardless of how early or late you metamorphose it, it will always, ALWAYS have the worst stats possible for the upgrade method you've chosen.
Apparently familiars have a hidden base growth stat that determines how many stat points it gains on average per level. Some familiars are just born better than others. The ones you start with are born worse.
Esther's Drongo actually is a very, very good magic attacker for basically the entire game.
The Mite will start to become obsolete around level 35. The Thumblemur stays pretty good until level 40+
I never used the Hurly, but I believe it should be used by Oliver anyway.
In the end, the power of the familiars is of little consequence. You could use all of the worst familiars and still beat the game. In fact, it would probably be more enjoyable to do that than the alternative (power leveling and using overpowered Dinoceros.)