The castle was built in the early 14th century but destroyed in the winter of 1308–1309 in the Wars of Independence. Subsequently granted to the Earls of Ross, it was re-built in 1380 after the daughter of the 5th Earl married Sir Alexander Fraser of Cowie. In the 16th century a second, round, tower, a courtyard and outbuildings were added to the original square main tower.