Yes, Mary visited on a number of occasions. For example – only nine months old she was crowned Queen of Scots in the Castle's Chapel Royal. She was also there prior to the christening of her son James at the nearby church of the Holy Rude VI.