Potatoes are ready to harvest when the plants have died back and the skin on the tubers has set. This generally happens around the first frost or early fall in many regions. You can also dig up a plant and check the size and maturity of the potatoes. Be sure to harvest before the hard freeze of the season arrives.