The structure was originally a "plantation castle", used for defensive purposes during the plantation of the area in the early 17th century. The original tower house, which served as a garrison, was built in the 15th century as a stronghold for the Caviness family, an old Irish clan. Later Baron Esmonde. Due to the strategic importance of the village of Clonegal during the Cromwellian conquest of Ireland because of its location on the road between Dublin and Wexford, the castle was captured by Oliver Cromwell as he marched on Kilkenny in 1650.