The name "Walworth" means "Welsh settlement", and it used to be known as "Waleberge" after the Saxons claimed it. It is thought that Walworth was planned as a village with the previous castle around 1150 by the Hansard family as part of their estate. There is a legend that Malcolm III of Scotland destroyed the village on his way along the River Tees. Following the Black Death there was a change of ownership of the manor to the Neville family by 1367, but in 1391 Robert Hansard claimed it back. The Ayscough family acquired the manor by marriage in 1539, then Thomas Jenison bought it in 1579 when the Ayscough family had no heirs. At the death of Elizabeth Jenison in 1605, the farm stock inventory included 50 oxen besides cattle, sheep, pigs, horses and corn. In 1759 the estate left the hands of the Jenisons due to the death and debts of Ralph Jenison. From 1759 to 1831 the estate belonged to Matthew Stephenson, and then it was sold to the Aylmer family who owned it until 1931. Their descendants Neville and Charles Eade owned it from 1931 to 1950, and then it was sold in 1950 to Durham County Council. The estate was broken up and sold into private ownership in 1981, and present ownership of the village is unknown.