You can only be in one covenant at a time. If you break the rules or join another, you're marked in the "Book of the Guilty" and can be invaded by a member of the Blades of the Darkmoon, or the Spirits of Revenge.
Really, all it does is put you on an invasion list.
The game asks you if you are sure when you're about to break a covenant by joining another. Dunno if it is true, but someone said the jerk in the Undead Chapel can break a covenant for you without penalty. But that might just be the same as a message asking you take a step out over a death pit...
"I'd rather betray the world than let the world betray me." -Cao Cao