Hereford Castle Hereford Castle was a castle in the cathedral city of Hereford , the county town of Herefordshire , England ( ) . Founded sometime before 1052 , it was one of the earliest castles in England . Hereford Castle was probably destroyed when the Welsh sacked Hereford in 1055 , but seems to have been replaced by the following decade . During the civil war of King Stephen ' s reign , the castle was besieged three times ; the garrison surrendered each time and control of Hereford Castle changed hands . Early history . During the 11th century , the county of Herefordshire was subject to attacks from Wales . Rather than repairing the town defences of Hereford , the county town , Ralph ( son of the Count of Vexin ) chose to build a castle within the settlement ' s defences . Ralph was made Earl of Hereford in about 1046 ; the date of the castle ' s foundation is uncertain , but it was present by 1052 . This first castle is thought to have occupied the area of the later castle , in the east of the town . The areas to the north and west were densely occupied , and to the south was the River Wye . In 1055 , the town and castle of Hereford were sacked by the Welsh . The " Anglo - Saxon Chronicle " recorded the incident : Harold Godwinson repaired the town ' s defences in 1056 , but what happened to the castle is not mentioned . Shortly after the Norman Conquest in 1066 , William FitzOsbern was made Earl of Hereford . He may have rebuilt the castle as it was in use in 1067 when the " Anglo - Saxon Chronicle " noted that Eadric the Wild harassed its garrison . In 1071 William FitzOsbern died and his son Roger took over possession of the castle . He was involved in an unsuccessful attempt to depose King William and consequently forfeited the castle . In April 1138 , Geoffrey Talbot captured Hereford Castle from supporters of King Stephen . Later that year Stephen besieged the castle until it surrendered . In 1140 during the Anarchy Geoffrey Talbot and Miles of Gloucester captured the castle from the garrison supporting King Stephen . During the siege the cemetery of the Church of St Guthlac was desecrated . The " Gesta Stephani " recorded the event : Geoffrey Talbot and Miles of Gloucester set up siege engines so that the castle was attacked from two directions , and the garrison eventually surrendered . In 1154 Matilda ' s son Henry II granted the motte of Hereford to Roger of Gloucester but a rebellion followed and Henry retook possession and for the rest of its history the castle remained royal . In 1216 John made Walter de Lacy sheriff of the county of Herefordshire and granted him the custody of the Royal castle at Hereford and the following year work was undertaken at to strengthen the castle against the Welsh attacks . Welsh attacks . During the Second Barons ' War the castle came for a time the headquarters of the baronial party headed by Simon de Montfort in the 13th century . During the Owain Glyndŵr rebellion from 1400 to 1411 King Henry IV based himself at Hereford castle preparing sorties and campaigns into Wales . During the Civil War Herefordshire was very much a Royalist stronghold but the castle does not appear to have played a significant part . It was eventually sold to Sir Richard Harley and several of his friends but it then went into decline . Most of what remained of the castle seems to have been destroyed in the 1650s and the stone used for other buildings within the city . According to John Leland , the antiquary , the castle at Hereford was once " nearly as large as that of Windsor ' and ' one of the fairest and strongest in all England " . Today . In 1746 the ruins were ordered to be dismantled and the site was transformed into what is today Castle Green , the moat being turned into Castle Pool lake . In 1833 the Castle Green ( which had been the bailey of the castle ) was leased to the city council for a period of 200 years . The council is still responsible today for the maintenance and upkeep of the area in its present form of a recreation area . A monument to Lord Nelson standing on the site today was erected in the centre of the bailey in 1809 . A footbridge , the Victoria Bridge , crosses the River Wye linked to riverside walks and walks around Hereford Cathedral .