No. Not every Earth has the Avengers. While it may be hard to imagine a universe without the Avengers, there are plenty of those universes out there in the multiverse.