Abyssion from Tales of Symphonia. One of my friends and I spent an entire afternoon in the fight against him. Only way to survive was to use an item that halved all damage dealt by your party and the enemy.
He's not really that bad.
Also shoutouts to whoever mentioned Whitney from Pokemon Gold/Silver. Definitely not one of the "hardest bosses of all time" but would probably clock in on a list of "most unexpectedly difficult bosses." The leader of an early gym in a Pokemon game--should be a total walk in the park and relatively forgettable, and yet her Miltank is legendary because it can easily destroy your entire party unless you're way overleveled, have multiple Pokemon with type advantages already, or both. Or if you get stupidly lucky. It's a force of destruction that I wish Nintendo were better at emulating in other gym leaders / rival battles / etc.--Whitney's Miltank seems to be an accident, and it'd be nice if more big "boss fights" in Pokemon were actually somewhat difficult.