Sovereign has a horn, but not as much bass/treble/vibrato?, the human Reaper makes similar sounds. I could've swore they made that noise earlier because I remember comparing the Inception noise to reapers when that came out...
Because this game is the first time you really get to see numerous Reapers up close, this is technically the first time they've used the sounds you're familiar with. That is, Im assuming you're talking about the sound the Reaper who touches down into the lake makes before he shoots down the Alliance Cruiser on the first level.
The sound produces said it was made using a trashcan with a trap door on top that was intended to keep bears from rummaging through it. They obviously had to make some overall changes to it.
Although they make noise in the other games, I'm pretty sure this is the first time the reaper fog horn is used.
In fact I'm pretty sure I remember seeing a video about the sound design, where they mention that since this is really the first time where you see lots of reapers up close they wanted to come up with an iconic, scary sounding noise, so they made that noise based on a fog horn.
Edit: I found the video: http://youtu.be/8bytxqeUMPU?t=2m18s
As said in the video, it was originally based on the trashcan bear trap noise like BiggyDX mentioned, but then they added a fog horn to it in ME3 for the big up close scary noise.
So yeah technically the sound has been around since the first game, as it was the noise that would play when the logo would pop up on screen at the end of trailers and stuff, but it seems like the big VVRRROOOONNNGGGGG noise wasn't until ME3.