Possibly the first house erected within the site was that of a Sir William Blake, around 1624.