Pesticides are the primary factor that kills most caterpillars, as they are often used in agriculture and gardening to control insect populations.