 The first three lairds transformed the estate into a valuable asset. Lord Pitfour, the second laird, purchased additional lands including Deer Abbey and Inverugie Castle. Pitfour's son, James Ferguson, who became the third laird, continued to improve and expand the estate by adding the lake and bridges, and establishing planned villages. The third laird died a bachelor with no children, so the estate passed to the elderly George Ferguson, who was only in possession of the property for a few months. George was already a wealthy man, owning lands in Trinidad and Tobago, but despite not directly improving the Pitfour estate he added considerable value to the inheritance passed to his illegitimate son. The extravagant lifestyles of the fifth and sixth lairds led to the sequestration of the estate, which was sold off piecemeal to pay their debts.
