This happens to me in Oolicle Township non-stop. It can actually be a better strategy to wait by the invader spawn point of the area, especially Oolicle township, and just wait to be invaded. Now, the alternative, of course, is to beat the area boss. At that point you can never be invaded.
He brings whale sushi and panda kebabs to vegan potlucks - World's Most Interesting Man