I often play evil characters so I mostly have stolen stuff on me. So I always pick 'rather die' and make a run for it. Then if I really need to go to the city in question I unload my stolen stuff and pay the bounty. I never went to prison, do you lose skill points in prison like in Oblivion?
The mind is like a parachute: it only works when it's open
Depends on the character. You can get everything back anyways so going to jail is not that big of a deal. Sometimes you can get more:
The other day I also found some favorited quest items are available in prison. They don't show up in your inventory but you can access them from the favorites menu. Pulls out legendary Balgruuf's Greatsword: That's not a shiv. This is a shiv.