You can tell Anders to leave at the end of Act 2 (or is it beginning of 3?) he'll come back close to the end (i've said enough about him).
Isabela can leave at the end of Act 2 if you don't get her 3/4ths to rivalry/friendship. At the very end of the game a mage/warrior (Fenris/Aveline or Anders/Merill) character may leave your party depending on your choice, though if you have them all to full friendship or rivalry they stay. Sebastion (if you have him) may also leave at the end if you make a certain choice.
Think that's it.
http://i.imgur.com/Mjr5Z.gif State of the badass art! You do NOT wanna **** with me.
Most of it has to deal with how high your friendship or rivalry score is and who you choose to side with at the end out of mages and templars. I only really ran into it in my first game as I was kind of rushing so when I sided with the mages Fenris left and then I had to fight him a little later. There is a cap to how many people can leave. Like I know you can not get rid of Merrill if she is your only mage as an example or Aveline if she is your only warrior(dunno if Hawke counts for this). I also know you can get rid of Fenris by handing him over to Danarius during his act 3 PQ. Can do the same thing to Isabella in act 2 with the Arhishok if you just really want to be mean to her(I get a laugh out of it, I kind of hate Isabella). Then there is getting rid of Sebastian if you do not kill Anders or of course killing Anders.