Potato is right. When someone joins your game, you are no longer single-player. The game world doesn't freeze when you go into a menu, because that would require the other player being frozen or otherwise incapable of doing things.
"If you're going to be a mod, you should work harder to get rid of your humanity. It's creepy." ~ Monker to The Paragon