She was the daughter of Thomas Randolph, Earl of Moray, nephew and companion-in-arms of Robert the Bruce, and Moray's wife, Isabel Stewart, herself a daughter of John Stewart of Bonkyll. Agnes became renowned for her heroic defence of Dunbar Castle in East Lothian against an English siege led by William Montagu, 1st Earl of Salisbury, which began on 13 January 1338 and ended on 10 June the same year during the Second War of Scottish Independence from 1331 to 1341.