Oh, well I had help from TF + Vi with 3 man ganks.
You can beat anyone with anyone when you have 3v1 scenarios.
Anyway, Fizz wouldn't do too terribly. You just have to abuse your early levels before he gets too tanky. Fizz can proc all three of his skills before the Silence affects him. As long as Fizz burst is strong enough to kill Garen before you get into execute range, you win handily.
Fizz actually fares pretty great against a lot of bruisers, with his W DoT being able to outtrade most of them. Metagolem Fizz used to be a thing, ya know.