Province of New Jersey
Born in Morristown in the Province of New Jersey, he was the son of Sarah Frances (Ludlow) and Abraham Ogden. His sister, Gertrude Gouverneur Ogden (1777–1850), was married to Joshua Waddington (1755–1844), a founder of the Bank of New York.