Upper Slaughter was the site of an adulterine castle, built by supporters of the Empress Matilda during The Anarchy of the 12th century. The remains of the castle are marked by the Castle Mound on the north edge of the village.