It's a good idea, especially if you happened upon the chiles close to the 8,000 Scoville heat units. You could get away with not wearing them for peeling, but for removing the seeds where most of the heat is, you definitely will want to have gloves on. If you brave it and don't wear gloves, wash your hands immediately with soap several times and be careful not to touch any sensitive areas like your eyes.