Even while they are growing, they are being cut down by the processes of erosion. These processes include wind blown sand eating away at surfaces, ice in glaciers scouring the surfaces, and water in streams.
Water is by far the most important.
The Himalayas are still growing today after starting to grow about ten million years ago when India crashed into Asia, and at the same time water and ice are tearing them down. As the mountain erodes from above, it also rises from below as the root is no longer pushed into the mantle by the weight of the mountain - much like an ice cube rises in the water as the top is melted away. Roots for high mountains can be very deep - as much as 20 km or so. This means that erosion needs to erode away more than 20 km of material to completely flatten out a mountain that was 5 km high at the start. It also explains why we get large areas of rocks at the surfce that were formed at depths of 20 km or so in places where there used to be high mountains - like the Canadian shield.