When German forces invaded the Soviet Union in 1941, they occupied the castle and converted it to a ghetto for the local Jewish population, prior to their murders. Between 1944 and 1956, the castle was used as a housing facility, resulting in damage to the castle's interior.