It's less about the size of the potatoes than making sure they're all a similar size so they roast evenly.