In 1505, the partially ruined castle was granted to Alexander Gordon, 3rd Earl of Huntly, who was charged by King James IV with repairing the castle for use as a Royal garrison. His brother William Gordon, Laird of Gight, became master of Inverlochy, and was slain commanding the Camerons at Flodden. In 1645, the castle served as a stopping-off point for the royalist army of James Graham, 1st Marquess of Montrose during his campaign against the Covenanter forces of the Marquess of Argyll. This culminated in a victory for the royalists in the second Battle of Inverlochy, on 2 February 1645. In the 19th century, the estate was bought by James Scarlett, 1st Baron Abinger, who built a Scottish baronial style mansion to the north-west, which is now the Inverlochy Castle Hotel. Minor enhancements, including the restoration of loops and battlements, were carried out by Lord Abinger in advance of the visit of Queen Victoria in 1873.