He was born on January 12, 1808 at East Amwell Township, New Jersey. He was the son of David Manners, an army captain during the War of 1812. His grandfather was John Schenck, a captain during the American Revolution who served at the Battle of Princeton and the Battle of Monmouth.