You can in fact overbake a potato. If a potato overcooks, the skin could burn and become bitter, and the inside will be dry.