It was renamed Heathfield Park after him in 1791 by his successor Francis Newbery, son of the publisher John Newbery. Newbery hired Humphrey Repton to landscape the park. Newbery built the "Gibraltar Tower" in one corner of the park, to commemorate Lord Heathfield's successful defence of Gibraltar from 1779 to 1782. The ground floor is octagonal and the upper part round, accessed by an internal circular staircase. It is also a Grade II* listed building.