James died in 1783 and the castle was built as a memorial to him by his widow, Lady James of Eltham.
Džeimsas mirė 1783 ir pilis buvo pastatyta jo našlės, Eltham (Londono rajonas) Ledi Džeims, kaip paminklas jam.