Yes, cooling the potatoes before adding to a mayo-based potato salad will ensure that the dressing doesn't become oily and helps the potato keep it's structure instead of turning into mush.