To handle chili peppers without irritating your skin, it is recommended to wear gloves. If gloves are not available, you can use a barrier method such as plastic bags or cling wrap. Avoid touching your face, eyes, or any sensitive areas while handling chili peppers, as the capsaicin oils can cause discomfort.