Dudley Castle Dudley Castle is a ruined castle in the town of Dudley , West Midlands , England . Dudley Zoo is located in its grounds . The location , Castle Hill , is an outcrop of Wenlock Group limestone that was extensively quarried during the Industrial Revolution , and which now along with Wren ' s Nest Hill is a Scheduled Ancient Monument as the best surviving remains of the limestone industry in Dudley . It is also a Grade I listed building . The Dudley Tunnel runs beneath Castle Hill , but not the castle itself . History . According to legend , a wooden castle was constructed on the site in the 8th century by a Saxon lord called Dud . However this legend is not taken seriously by historians , who usually date the castle from soon after the Norman Conquest of 1066 . It is thought one of the Conqueror ' s followers , Ansculf de Picquigny , built the first castle in 1070 . and that his son , William Fitz - Ansculf , was in possession of the castle when it was recorded at the time of the Domesday Book of 1086 . Some of the earthworks from this castle , notably the ' motte ' , the vast mound on which the present castle keep now sits , still remain . However the earliest castle would have been of wooden construction and no longer exists . After Fitz - Ansculf , the castle came into the possession of the Paganel family , who built the first stone castle on the site . This castle was strong enough to withstand a siege in 1153 by the forces of King Stephen . However , after Gervase Paganel joined a failed rebellion against King Henry II in 1173 the castle was demolished by order of the king . The Somery ' s were the next dynasty to own the site and set about building the castle in stone starting in the second half of the 13th century and continuing on into the 14th . The keep ( the most obvious part of the castle when viewed from the town ) and the main gate dates from this re - building . A chapel and great hall were also constructed . The last of the male line of Somery , John Somery , died in 1321 and the castle and estates passed to his sister Margaret and her husband John de Sutton . Subsequently , members of this family often used Dudley as a surname . In 1532 another John Sutton ( the seventh in the Dynasty named John ) inherited the castle but after having money problems was ousted by a relative , John Dudley , later Duke of Northumberland , in 1537 . Starting around 1540 , a range of new buildings were erected within the older castle walls by him . The architect was William Sharington and the buildings are thus usually referred to as Sharrington Range . Dudley was later beheaded , for his attempt to set Lady Jane Grey on the Throne of England . The castle was returned to the Sutton family by Queen Mary , ownership being given to Edward Sutton . The castle was later visited by Queen Elizabeth I and was considered as a possible place of imprisonment for Mary , Queen of Scots . However , the Sutton family were not destined to hold the castle for much longer and Edward Sutton ' s son , Edward Sutton III was the last of the male line to posess the property . In 1592 , this Edward sent men to raid the property of Gilbert Lyttelton , carrying away cattle which were impounded in the Castle grounds . Financial difficulties continued to mount , however , until Edward Sutton III solved the problem by marrying his grand daughter and heir , Frances Sutton , to Humble Ward , the son of a wealthy merchant . The castle became a Royalist stronghold during the English Civil War , and was besieged twice before its surrender to Cromwell ' s forces in 1646 . The first siege in 1644 was lifted after the Royalists sent a relief force which drove away the Parliamentarians . In 1646 Sir William Brereton commanded the Parliamentarians in the second siege against the Royalists led by Colonel Leveson . The castle was surrendered on 13 May 1646 . Parliament subsequently ordered that the castle be partly demolished and the present ruined appearance of the keep results from this decision . However some habitable buildings remained and were subsequently used occasionally by the Earls of Dudley although by this time they preferred to reside at Himley Hall , approximately four miles away , when in the Midlands . A stable block was constructed on the site at some point before 1700 . This was the final building to be constructed in the castle . The bulk of the remaining habitable parts of the castle was destroyed by fire in 1750 . However , in the nineteenth century , the site found a new use as a ' Romantic Ruin ' and a certain amount of tidying up of the site was carried out by the Earls of Dudley . Battlements on one of the remaining towers were reconstructed and two cannon captured during the Crimean War s were installed . In the nineteenth and early twentieth century the site was used for fêtes and pageants . In 1937 , when the Dudley Zoo was established , the castle grounds were incorporated into the zoo . Despite being situated on the edge of Dudley town centre , the castle was situated within the borders of Sedgley - which was part of neighbouring Staffordshire rather than Worcestershire - until the borders were changed to include the castle and its grounds within the Dudley borough in 1926 , when restructuring of the boundaries took place to allow the development of the Priory Estate . The maps of Christopher Saxton drawn in 1579 and John Speed in 1610 , mark Dudley Castle in the County of Staffordshire not Worcestershire . Visitor centre . The castle visitor centre was opened by Her Majesty Queen Elizabeth II in June 1994 , and amongst other exhibits housed a computer generated reconstruction of the castle as it was in 1550 , displayed through hardware that demonstrated the first use of the virtual tour concept , prior to its widespread adoption as a Web - based browser utility . More details of how Her Majesty became the first Royal to experience a virtual world here . List of Lords of Dudley Castle . Dudley Castle was the capital of the feudal barony of Dudley . On his death the lands of the barony were divided between his two sisters . Weoley Castle went to Joan de Botetourt and her husband John de Botetourt . Dudley Castle passed to her elder sister Margaret , who had married John de Sutton I . John de Sutton II was summoned to Parliament , but none of his successors were until John de Sutton VI For the evolution of the castle and estate until 1740 see Baron Dudley and from the late 17th century until the 20th century as Baron Ward John de Sutton I See also . county county