Southeast of the town, archaeologists discovered graves from the 3rd–4th centuries. According to the Palemonids legend, noble refugees from the Roman Empire settled on the hill, now named after Palemon, and established the Grand Duchy of Lithuania. The town has a large hillfort where Lithuanian Pieštvė fortress stood during the Lithuanian Crusade. It was attacked by the Teutonic Knights numerous times in the late 13th and early 14th centuries. After the Battle of Grunwald, the location lost its military purpose and became private property of the Sapieha family.