Talking about main game not DLC which may or may not be in cannon
Our Booker died at the end of Infinite, Elizabeth wanted to drown him in order to eliminate any universe where Comstock was already born and prevent any other Bookers from becoming him once again, so, in doing this, Elizabeth erased Comstock, and with him, herself, as Anna was never captured and therefore never lost her finger and became Elizabeth. That's why they all faded away after Booker "died", all, but one...
And as for the DLC, I'm afraid to tell you that it's not an alternate reality story nor a non-canon tale, but it's actually the direct sequel for Infinite, while being a prequel to the first Bioshock at the same time, and well, I'm sorry, but you got yourself kind of spoiled in here with the first reply in that aspect...But that's not the "WOW!" part, just like in the main game, you could have known Booker dies, but you would have experienced that amazing ending fresh of any spoiler, too, and that's what the DLC is, too - you already know Booker's fate, but the ending is what really matters, look on the bright side! :)
The Booker that decided to get baptized is the one that died. He was the one that was eventually going to become Zachary Comstock. By killing him, they also destroyed all the alternate universes that had him in it. Everything that was going to happen after he got baptized, was prevented. Which means Anna never got kidnapped and never went through all the crap she went through as Elizabeth in Comstock's universe. It also means Rosalind Lutece never existed, and if she didn't exist, there is no such things as tears and no possibility of going to alternate universes in the first place.
The Booker DeWitt that never chose to get baptized is alive and well. Did you see the clip after the end credits? Pretty much, everything that happened in the game, didn't happen. That universe was destroyed.