The ritual dismantling of the shrines every 20 years (known as sengu-shiki) goes back to prehistoric times, when sacred structures tended to be erected for special ceremonies rather than as permanent places of worship.
Prehistoric sacred structures were often build specifically for special ceremonies, then dismantled after the rituals were complete.