I'm pretty sure Kivan stays where you leave him the first time, then the second kick he leaves for good. I had to work around a few odd quirks like that when I did one game where I had EVERY NPC in a house I took (Right beside Feldepost's inn, the locked abandoned house). After I reached the city of Baldur's Gate, I moved everyone into the Docks area, the house with all the Ogre-Magi.
Some of the kick/recruit tactics I had to use were a pain in the ass. For Dynaheir/Edwin/Minsc, I had to recruit Dynaheir first, then go get Edwin (Since he will join saying something about watching her), then Minsc. I kept Kivan in my party as the only 100% person. Dropped three people for Coran, Yeslick and Faldorn in Cloakwood, and had to keep my reputation as a Paladin around 12 or else I had issues. My group was a disaster half the time, and I refuse to play on anything but insane :P
That was a pretty fun game, and kind of funny to see every NPC split into various rooms in that one mansion,