From 1627, parts of the castle were broken down at the behest of King Gustavus Adolphus of Sweden during the Polish–Swedish War in order to gain building material for the construction of the star fort in Pillau (now Baltiysk), a strategically important port town occupied by the Swedes. Balga was located in the Duchy of Prussia from 1525 and the Kingdom of Prussia from 1701, where between 1772 and 1829 it belonged to the province of East Prussia. The province was unified with West Prussia into the Province of Prussia until 1878, when it separated once again and Balga remained in East Prussia until 1945.