Potatoes are ready to harvest when the plants begin to die back and the leaves turn yellow. It's important to wait at least two weeks after the plants die back before harvesting to allow the potatoes to mature fully. You can dig up a few potatoes to check their size and maturity before harvesting the entire crop.