The castle was first mentioned in the 13th century as a fortress ruled by the Babonić clan. Between 1328 and 1347, it was possessed by the members of Iločki family. In 1347, King Louis I the Great bestowed the fortress to the noble Šubić family who then changed their family name after it, becoming the Zrinski. It remained in their possession until the Ottoman invasion and conquest of the region, which led to the fortress falling to them on 20 October 1577. It was not until 1718 that the castle was retaken from the Ottomans.