We left them on since it saves time, and the skins on smaller, waxy potatoes provide a great crunchy texture when roasted.