Instead of actually paying attention to the vital story of the cutscene, my mind is more focused on getting ready to press whatever random button the game is about to throw at me.
Sometimes I just want to watch a cutscene and relax to see what the story is about.
It depends on how it's done. In Tomb Raider: Legend the keypresses are always the same in the cutscene, and are appropriate to the action Lara takes. X to jump, R1 to fire. That's capable of pulling the player in.
That way your reflexes and memory the key symbols isn't what matters. Instead, it's whether you can fire, jump, dodge, grapple when the situation requires it. Which is what the game has been training you in, so most players can do that, even if they don't know the button symbols.
It's still a pain to get back from the bathroom and find the game over screen, because you took a break at the beginning of the cutscene.
Resident Evil 4 does the exact opposite. Shamus Young first takes on QTEs in general.
X, A, B Win http://www.escapistmagazine.com/articles/view/comics/stolen-pixels/7887-Stolen-Pixels-214-X-A-B-Win
Definitely, it totally breaks immersion for me. I tune out the dialogue and whatever and focus on catching that little QTE prompt if it appears. I'd rather sit back and watch the cutscene than do something during it. I'm supposed to get into the story and characters, not watch out for button symbols.
Heck, even Heavy Rain wasn't as immersive as it could be with QTE action sequences. Slow QTE gameplay was fine (and very immersive), but not rapid prompts. There's one section where you have to catch and fight a guy and I couldn't remember anything from it, just symbols flashing. My friend was watching and said it looked really cool... We ended up replaying this chapter with me watching her. :/
I'd love the option to turn off the QTEs during cutscenes. They're annoying and useless IMHO.