In my experience, if you cooled the potatoes prior to mixing all the ingredients, the potato salad will not thicken in the fridge. If the potatoes were hot when all the ingredients were mixed, it might thicken in the fridge, but there will also be water at the bottom of the bowl.