When the Taiping rebels occupied Nanjing and established the Taiping Heavenly Kingdom (1851–64), Xu Garden became the imperial garden of the Heavenly King's Palace and was called the "West Garden". The garden was destroyed in 1864 when Qing imperial forces recaptured Nanjing from the Taiping rebels. It was rebuilt in 1870 by the order of Zeng Guofan, who served as the Viceroy of Liangjiang from 1870–72.