It was built for Nathaniel Rawlins, a London haberdasher merchant, who lived there until his death in 1718. The Duke of Clarence, later to become King William IV, lived in Richmond in the late 1780s and gave his name to the property. From 1792 to 1799, Clarence House was a Catholic school run by Timothy Eeles. Among the students was Bernardo O'Higgins. O'Higgins is commemorated on the wall of the property with a blue plaque installed by English Heritage, for his role in the Chilean War of Independence.