Compton had five brothers and two sisters; Charles (1624–1661), William (1625–1663), Spencer (died 1659), Francis (1629–1716), Henry (1632–1713), Anne (1637–1705) and Penelope (1642–1667). Four of the brothers fought in the First English Civil War, including William who during The Protectorate was a senior member of the Royalist conspiracy group known as the Sealed Knot. His youngest brother Henry became Bishop of London and a leading member of the Anglican opposition to James II of England, deposed in November 1688.