The castle began as an oblong, five storey keep built by Sir Colin Campbell, the Black Knight of Rhodes around 1440. The main entrance to the castle leads through the vaulted basement and into the courtyard. At the south end of the basement there is a small prison and service stair to the first floor hall. The lintel carries the initials of John, 1st Earl of Breadalbane and those of his wife, Countess Mary Campbell. The original timber stair to the first floor was replaced in 1666 with a circular stair tower.
Nothing remains of the hall fireplace, which was on the east wall.
At the south end of the hall there are two large wall closets, between which are the service stairs. This end of the hall would have been been screened from the main banqueting area. The angle rounds were added in the late 16th century as part of a remodeling of the upper floors. The corbelling from the north-west angle was blown from the wallhead in 1879, during the same storm that caused the Tay Bridge to collapse. Extensive barrack ranges were built in 1693 along the northern side of the courtyard, three round angle towers were also added to the curtain wall, only two survive.