St Margaret's Chapel on the castle grounds is the oldest surviving building in Edinburgh and dates from the early 12th century.