Casimir III the Great received the legation of the Lizard Union in the castle. After the Battle of Grunwald important decisions regarding the Polish–Lithuanian–Teutonic War were thought of in the castle. During the years of 1476 to 1481 the castle was the residence for the Princess Anna of Sochaczew. In the mid-sixteenth century the castle began to deteriorate. In 1655 the castle was taken by the Swedes. In the eighteenth century Augustus III of Poland gave the castle to the Bernardinians of Koło, which had partially deconstructed the castle to reconstruct a damaged monastery; however due to the strong mortar the castle could not be fully deconstructed.