Caterpillars like dark places, fungal and moist areas. Unwashed clothes have some moisture and have some fungal growth. On the other hand, moths are intensely attracted to keratin. In the wild, they recruit it from feathers and hairs as these are great sources of keratin. But in houses the best source for keratin clothes. And the eggs laid by moths are so tiny that you can’t see them with naked eyes. One can only notice the infestation of caterpillars when they are clothes are torn or dug a hole on.