Grant was born in New York City, New York, the son of Gabriel Grant, a physician and American Civil War surgeon, and Caroline Manice. Madison Grant's mother was a descendant of Jessé de Forest, the Walloon Huguenot who in 1623 recruited the first band of colonists to settle in New Netherland, the Dutch Republic's territory on the American East Coast. On his father's side, Madison Grant's first American ancestor was Richard Treat, dean of Pitminster Church in England, who in 1630 was one of the first Puritan settlers of New England. Grant's forebears through Treat's line include Robert Treat (a colonial governor of New Jersey), Robert Treat Paine (a signer of the Declaration of Independence), Charles Grant (Madison Grant's grandfather, who served as an officer in the War of 1812), and Gabriel Grant (father of Madison), a prominent physician and the health commissioner of Newark, New Jersey. Dr. Gabriel Grant was awarded the Medal of Honor while serving as surgeon with the 2nd New Jersey Volunteers in the Civil War. During the Battle of Fair Oaks, he moved the wounded to safety while under heavy fire. Grant was a lifelong resident of New York City.