No, you don't have to at all! I usually prefer to, but you can absolutely just give them a good scrub to get rid of any dirt, then use them unpeeled.