Yes, you can certainly boil the potatoes with the skin on and remove it once the potatoes have cooled. If you are using new or baby potatoes for the salad (which have very thin skins) you may prefer to leave the potatoes unpeeled.