The gardens suffered damage numerous times during the 19th century. During the First Opium War, the British army used the Huxinting Teahouse as a base of operations for several days in 1842. During the Taiping Rebellion, the Small Swords Society ran its headquarters in the Dianchun Hall; by the time Qing troops recovered the garden, the original structures had nearly all been destroyed. They were damaged again by the Japanese in 1942 before being repaired by Liangshun Han (Rockery Han) appointed by the Shanghai government from 1956 to 1961. They were opened to the public in 1961 and declared a national monument in 1982.