He was born the 4th son of Sir Bevil Grenville (1596–1643) of Stowe, Kilkhampton in Cornwall, and lord of the manor of Bideford in Devon, a Royalist soldier and MP for Cornwall who died in heroic circumstances at the Battle of Lansdowne (1643). His elder brother was John Granville, 1st Earl of Bath, who played an important role in effecting the Restoration of the Monarchy to King Charles II in 1660, and whose descendants died out in the male line on the death of the 3rd Earl in 1711. The 1st Earl of Bath, having become interested in his family's history and examined his supposed descent from Sir Richard de Grenville (died after 1142) ("alias" de Grainvilla, de Greinvill, etc.) one of the Twelve Knights of Glamorgan who served under Robert FitzHamon (died 1107), in the conquest of Glamorgan in Wales, changed the spelling of the family name to "Granville" from "Grenville" apparently to conform with the spelling of the Norman manor at which his family originated at the time of the Norman Conquest.