Question: Where was the castle located in which is related to William Douglas and Margaret Hay?

Context paragraphs:
William Douglas of Drumlanrig: Sir William Douglas, 1st of Drumlanrig (died 1427) was the illegitimate son of James Douglas, 2nd Earl of Douglas.  He fought against the English in the Hundred Years' War in France, where he was killed in action in 1427.  He married Elizabeth Stewart, daughter of Sir Robert Stewart of Durisdeer, and had one son, William Douglas, 2nd of Drumlanrig.

Yester Castle: Yester Castle is a ruined castle, located 1.5 mi south east of the village of Gifford in East Lothian, Scotland.  The only remaining complete structure is the subterranean Goblin Ha' or Hobgoblin Ha' (Goblin Hall).  It is a Scheduled Ancient Monument, recorded as such by the Royal Commission on the Ancient and Historical Monuments of Scotland (RCAHMS).

William Douglas, 10th Earl of Angus: William Douglas, 10th Earl of Angus (1552 – 3 March 1611) was a Scottish nobleman.  He was the son of William Douglas, 9th Earl of Angus (1533–1591).  He was a direct descendant of King James I through his paternal grandmother, Lady Agnes Keith, a daughter of William Keith, 3rd Earl Marischal.

Gelston Castle: Gelston Castle, located between Castle Douglas and Kirkcudbright, in Dumfries and Galloway, Great Britain, was built for Sir William Douglas of Castle Douglas, who had acquired the lands of Gelston in 1799.  However, Sir William Douglas died without issue in 1809 and his property was divided between various nieces and nephews, with Gelston going to the youngest daughter of his brother James, Mathilda Douglas.  In 1813 Mathilda married William Maitland, and their family continued to preside over the estate for most of the 19th Century.  Gelston then became the property of the Galliers-Pratts who retained the castle for pheasant shooting, and from whom it was requisitioned during World War II.  During this time it served as a home for handicapped boys evacuated from Glasgow, before the roof was removed once this use had ceased.

Henry Maule: Sir Henry Maule was the son and heir of Sir William Maule, Baron of Panmure and Benvie and Ethana de Vallibus, daughter of John Vaux, Lord of Dirleton.  William succeeded as Baron on his father's death and was married to Margaret Hay, daughter of Sir William Hay of Locherworth.

James Douglas, 3rd Earl of Angus: James Douglas, 3rd Earl of Angus, Lord of Liddesdale and Jedburgh Forest (1426–1446) was a Scottish nobleman.  He was the son of William Douglas, 2nd Earl of Angus and Margaret Hay of Yester.

George Douglas, 4th Earl of Angus: George Douglas, 4th Earl of Angus, Lord Douglas, Abernethy and Jedburgh Forest (c. 1427 – 12 March 1463) was a Scottish nobleman.  He was the son of William Douglas, 2nd Earl of Angus and Margaret Hay of Yester.  Known as the Great Earl of Angus, he succeeded to the Earldom following the death of his childless brother James Douglas, 3rd Earl of Angus in 1446.  He was to become the first Red Chief of Douglas.

William Hamilton, Duke of Hamilton: William Douglas-Hamilton, Duke of Hamilton, KG, PC (24 December 1634 – 18 April 1694), was a Scottish nobleman and politician.  Born Lord William Douglas.  He was the eldest son of William Douglas, 1st Marquess of Douglas by his second wife Lady Mary Gordon, a daughter of George Gordon, 1st Marquess of Huntly.

Loch Leven Castle: Loch Leven Castle is a ruined castle on an island in Loch Leven, in the Perth and Kinross local authority area of Scotland.  Possibly built around 1300, the castle was the location of military action during the Wars of Scottish Independence (1296–1357).  In the latter part of the 14th century, the castle was granted by his uncle to William Douglas, 1st Earl of Douglas, and remained in Douglas' hands for the next 300 years.  Mary, Queen of Scots was imprisoned here in 1567–1568, and forced to abdicate as queen, before escaping with the help of her gaoler's family.  In 1588, the Queen's gaoler inherited the title Earl of Morton, and moved away from the castle.  It was bought, in 1675, by Sir William Bruce, who used the castle as a focal point in his garden; it was never again used as a residence.

John Ker, 1st Duke of Roxburghe: John Ker, 1st Duke of Roxburghe, KG, PC, FRS (30 April 1680–27 February 1741) was the second son of Robert Ker, 3rd Earl of Roxburghe, and Margaret Hay, daughter of John Hay, 1st Marquess of Tweeddale.  He was younger brother to Robert Ker, 4th Earl of Roxburghe.

Supporting paragraphs: George Douglas, 4th Earl of Angus Yester Castle
Answer: Gifford in East Lothian, Scotland