The castle was extended when a three-bayed two-storeyed house was built adjoining the tower