It was built for bishop Jakub Zadzik in 1622 and rebuilt in 1668 by bishop Andrzej Trzebicki after its destruction in the Swedish Wars. It was in poor repair by the mid-18th century and was extensively rebuilt in the late Baroque style by bishop Kajetan Sołtyk between 1760 and 1762, giving it the two storey appearance shown in the 1777 painting "Miodowa Street". It housed offices from the end of the 18th century and after the third partition of Poland in 1795 it housed the president of South Prussia. It was rebuilt after 1828 and partly dismantled in 1910.