Yes, dirt, dust, and sand can ruin the fibres of the carpet. For instance, the grains of sand are sharp and can grind into the carpet. This will lead to the cutting of the fibres. That is why regular vacuuming is of critical importance.