Solitude. It's on a natural stone arch over the sea and can only be approached from a steep uphill open and narrow path. Think Leonidas and the 300. You could defend that approach with a handful of soldiers and some good archers against a much larger attacker.
The secret to life is honesty and fair dealing. If you can fake that, you've got it made.
-Only one entrance -Plenty of well-protected, underground places for women and children to hide -All of the buildings are made of stone, which won't catch on fire -All of the various levels provide tons of locations for archers, and it is far easier to defend higher terrain than overtake it
Solitude is definitely second, but, as many people forget, it has two entrances. That door past the docks that leads straight to the middle of the town is a huge liability.
It is a defenders heaven. Mountains outside of gate allow great height advantage on a sieging enemy but a nice open area out front if defenders decide to deploy in front to delay enemy from digging into much. In side it has fresh running water supply coming from a protected mountain source. If siegers do get past gate defenders still have height advantage all the way back to the main hall. Even if the siegers push the defenders back to the main hall they could retreat into the old Dwemer city. Best place to be situated in a for a war IMO.
Markarth would fare better against catapults. However, from a completely strategic standpoint, Solitude is highly favorable. Windhelm... well, that one's good weather-wise. Not against Nords or Khajit of course, but the majority of other races would be unlikely to be able to fight as well in the cold. Even Whiterun would be good, though; the majority of their entrance is a medium length winding road. This is lined with walls and guard stations, plus could likely prevent a catapult, siege tower, or heavy ram from moving into a good tactical position. In all, I think the capital of each hold was selected for its position; even if that's only the case because each city probably had to survive and win wars. Their selections to be capitals were probably results of that.
"The point of war is not to die for your country. It's to make the enemy die for his."