The church was damaged many times due to earthquakes. In addition, the image had to be transferred to the Manila Cathedral for safekeeping during the British occupation of Manila in 1762–1764. A more permanent stone church was built in 1810; the image was returned to the church in 1918.