A lot of them seem to have been eaten by leopards and the other large predators of the time. Some 2.5 million years ago we find the first evidence of worked stone. About the same time, the brain size of our forebears started to increase substantially over the chimpanzee's. Tool making and larger brains probably influenced each other in a positive feedback cycle. Those able to learn the more complex tasks from those around them must have had a significant survival advantage, in spite of the increased maternal and infant mortality from getting those larger brains delivered.
Memes run the gamut from essential symbionts to dangerous parasites.
In the aggregate, they constitute culture. The memetic information passed down from generation to generation exceeded our genetic data some time ago. As human brains enlarged they improved in the ability to anticipate changes, making plans to hunt, to move with the seasons, and, later, to plant seeds for a future harvest. A contemporary example is the statistical fact that highly intelligent people have significantly fewer children than the norm. Many traits of populations that have a bell curve distribution are trimmed by some form of selection on both ends.