Charles Morris was married to Harriet Bowen in February 1815. They had four sons and six daughters. His eldest son, Charles W. Morris served in the United States Navy during the Mexican–American War. As a flag lieutenant under Commodore David Connor he died from wounds received in an attack on Tabasco, October 1846. Another son Brevet Lieutenant Colonel Robert Murray Morris, served as an officer in the U.S. Army from 1846 to 1873, distinguishing himself at Contreras and Chepultapec in the Mexican–American War and at Valverde and the Battle of Dinwiddie Court House the American Civil War. His youngest son, Commander George Upham Morris also served in the United States Navy from 1846 to 1874, in the Mexican–American War and in American Civil War where he commanded in its engagement with , June 8, 1862.