No, the entire fruit, including the skin, is edible. However, ensure you wash it well before consumption.