Yes, I recommend soaking the potatoes before you grill them so that you remove the starch from the potatoes. Removing the starch allows the potatoes to cook evenly.