A vendor's inventory is not their personal inventory--well, it is partly their personal inventory--it is their "store" inventory. Which is stored separately from what they have on their person.
So while you can buy something they are carrying on their person, you are mostly buying stuff that they have stored safely in an inaccessible chest. So anything you sell to them, and the gold they get from such transactions are transferred to this hidden chest.
These chests are actually in the same cell (sort of) as the vendor, but they are beneath the floor, floating in the null-space outside player boundaries. So you cannot access them, and steal your stuff back.
There are a few exceptions of course, namely Eorlund's and the Khajiit caravans. But the rest, they only exist as a technicality, as they exist outside of the world the player can access.
You can hop on a barrel near the border of the inside of Whiterun, use it to jump onto a wall, then outside the map. You can then run around the outside perimeter of the cell's map, eventually "falling" beneath the map near Eorlund's forge, letting you gain access to his chest.
Not sure if it still works, or if it has been patched. If you want the specifics, there should be a few videos of it on YouTube.