Andrew Cavendish , 11th Duke of Devonshire Andrew Robert Buxton Cavendish , 11th Duke of Devonshire KG , MC , PC ( 2 January 1920 – 3 May 2004 ) , styled Lord Andrew Cavendish until 1944 and Marquess of Hartington from 1944 to 1950 , was a British Conservative and later Social Democratic Party politician . He was a minister in the government of Prime Minister Harold Macmillan ( who was married to his aunt ) , but is best known for opening Chatsworth House to the public . Life . Cavendish was born to Edward Cavendish , 10th Duke of Devonshire and Mary Cavendish , Duchess of Devonshire , the former Mary Alice Gascoyne - Cecil , daughter of James Gascoyne - Cecil , 4th Marquess of Salisbury . He was educated at Eton and Trinity College , Cambridge . In World War II , he was a major in the Coldstream Guards . He was awarded the Military Cross for his actions on 27 July 1944 when his company was cut off for 36 hours in heavy combat near Strada in Italy . Family . In 1941 he married the Hon . Deborah Mitford ( born 31 March 1920 ) , one of the Mitford sisters . The marriage was not without some bumps . Three of the couple ' s six children died soon after birth , and the Duke ' s extramarital affairs became public after he appeared as a witness at a burglary trial and was forced to admit , under oath , that he was on holiday with one of a series of younger women when the crime occurred at his London home . The Duke , however , claimed that much of his marriage ' s success was due to the Duchess ' s tolerance and broadmindedness . Deborah , as chatelaine , is largely responsible for the success of Chatsworth as a commercial endeavour . He and his wife had six children , three of whom died in infancy : The Duke and Duchess of Devonshire had three surviving children : a son , Peregrine Cavendish , 12th Duke of Devonshire , and two daughters , Ladies Emma Cavendish and Sophia Topley . Among their grandchildren is the fashion model Stella Tennant . The Duke is buried in the churchyard of the village church in Edensor in the grounds of Chatsworth . His older brother William Cavendish , Marquess of Hartington , who would have inherited the dukedom , was killed in combat near the end of the war . With William ' s death , Andrew became heir and received the courtesy title of Marquess of Hartington , which he held from 1944 until 1950 . The 10th Duke died of a heart attack while visiting Eastbourne in November 1950 and Cavendish inherited the title , though he was in Australia at the time . The Duke died while being attended by suspected serial killer Dr John Bodkin Adams , who was his doctor when visiting Eastbourne . No proper police investigation was ever conducted into the death but Cavendish later said " it should perhaps be noted that this doctor was not appointed to look after the health of my two younger sisters , who were then in their teens " ; Adams had a reputation for grooming older patients in order to extract bequests . Cavendish inherited the estate but also an inheritance tax bill of £7 million , nearly 80 percent of the value of the estate . In order to meet this , the Duke had to sell off many art objects and antiques , including several Rembrandt s , Van Dyck s and Raffaello Santi s , as well as thousands of acres of land . Political career . Cavendish ran unsuccessfully as a National Liberal candidate for Chesterfield in the 1945 general election and as a Conservative in the same seat in 1950 . He was Mayor of Buxton from 1952 to 1954 . He served as Parliamentary Under - Secretary for Commonwealth Relations from 1960 to 1962 , Minister of State at the Commonwealth Relations Office from 1962 to 1963 , and for Colonial Affairs from 1963 to 1964 . He once said that these appointments by his uncle , Harold Macmillan , the then - prime minister , were " the greatest act of nepotism ever " . He joined the Social Democratic Party shortly after its foundation in 1981 , but left the party when David Owen resigned as the party ' s leader in 1987 , describing Owen as " the best of them " . He then sat as a crossbencher in his rare appearances in the House of Lords . The duke followed the family tradition of owning racehorses , the most famous of which was Park Top , the subject of the duke ' s first published book , " A Romance of The Turf : Park Top " , which was published in 1976 . His autobiography , " Accidents of Fortune " , was published just before his death in 2004 . The duke had many disputes over the years with the ramblers who used the paths near Chatsworth . Eventually though , in 1991 , he signed an agreement with the Peak National Park Authority opening 1 , 300 acres ( 5 km² ) of his estate to walkers . He said that everyone was " welcome in my back garden " . The duke ' s real estate holdings were vast . In addition to Chatsworth he also owned Lismore Castle in Ireland and Bolton Abbey in North Yorkshire . He also owned the bookshop Heywood Hill and the gentleman ' s club Pratt ' s . He was a major collector of contemporary British art , known especially for his patronage of Lucian Freud . He was one of the founders , and the chief patron of , the Next Century Foundation in which capacity he hosted the private Chatsworth talks between representatives of the governments of the Arab World and Israel . The duke was listed at number 73 in the " Sunday Times " Sunday Times Rich List of the richest people in Great Britain in 2004 . Honours . In 1996 he was made a Knight of the Garter . Other . He once told interviewer : " Wonderful things have happened in my life — it ' s time my son had his turn . When I was young I used to like casinos , fast women and God knows what . Now my idea of Heaven , apart from being at Chatsworth , is to sit in the hall of Brooks ' s , having tea . "