The mage guy with a speech impediment in Dragonsreach sells rings and other jewelry quite often.
Also, most dungeons have copious amounts of not-enchanted rings lying around. Though of course, when you're actively searching for something, it just never appears as often as you'd like. A sure-fire way to buy rings (there's enchanted jewelry in every time I've shopped) is the outdoor market place in Riften. An Argonian is the merchant you'll be looking for.