Blue's not even a bad guy. He remains helpful throughout the entire game, testing your abilities before important battles and giving you both advice and items. The only time Blue is ever remotely rude to you is right after you dethrone him from his position as champion, and even that is closer to shock and distress than malice.
Gary is a douche, Blue is a cool guy.
Silver is certifiably a villain, thieving and (implied) abusing Pokemon.