First, the theory that affluence leads to smaller families has not been properly tested. It may cause an initial fall, only to climb again. There seems some evidence of this from China. Second, much of the Third World is not on a trajectory of economic growth anyway. Third, even if rising affluence did rein in population growth, the damage has already been set in train - irreversibly.
Brazil is an example.
All this is happening to accommodate the existing population of Brazil. Imagine the rate of destruction if, as the UN predicts, Brazil's population climbs over the next 50 years to 240 million. In any case, it is impossible to lift the living standards of all the world's people to anything like that of the West. It is not an option. The pollution would choke the planet.