An effective biological control for caterpillars is a bacterium called B.t. It must be applied when the caterpillars are feeding because it only works if it is eaten. B.t. is toxic to all caterpillars, including many which may not be pests. The least toxic chemical for many garden insects is an insecticidal soap. If you purchase a commercial product, read and follow label directions and use only if your pest is listed on the label.
Some people prefer to mix their own liquid soap and water.
Use about 1 teaspoon of soap in a gallon of water. Test a small area of your plant first to be sure the soap won't harm the plant. Garlic or crushed hot peppers can be added and may make the mixture more effective, although scientific data is lacking. Soap is a contact insecticide, so you must wet the pest for it to be effective. It is toxic to all insects, including beneficial ones, and to birds and mammals as well, so it should be considered a last resort.