To clarify, if you kill the aliens, their weapons are destroyed, leaving weapon fragments. If you stun them, on the other hand, their weapons remain intact, saving you the trouble (and money) of having to build them, yourself. You'll still need weapon fragments for research and foundry projects, but a few, strategically chosen live captures can make things a lot easier for you.
I usually capture duplicates because I forget that I've captured them before, but I like the idea of capturing weapons. I also like the idea of giving the men practice in setting up a capture. The more aliens you capture, the more you CAN capture.
If you want their weapons, an easier way is to Mind control them and have them killed by an alien buddie. This is a lot easier way to stock up on Plasma rifles and Heavy plasma compared to running up to them and MAYBE get a stun.