He was the son of Hugh le Despenser and was summoned to Parliament by Simon de Montfort. Hugh was summoned as Baron le Despenser on 14 December 1264 and was Chief Justiciar of England and a leader of the baronial party, and so might be deemed a baron, though the legality of that assembly is doubtful. He remained allied with Montfort to the end, and was present at the Battle of Lewes. He was killed fighting on de Montfort's side at the Battle of Evesham in August 1265. He was slain by Roger Mortimer, 1st Baron Wigmore; this caused a feud to begin between the Despencer and Mortimer families.