Stevens' brother Clement H. Stevens (1821–1864) was a Confederate brigadier general and brigade commander in the Army of Tennessee who was fatally wounded at the Battle of Peachtree Creek near Atlanta. Brother Henry Kennedy Stevens (1821–1863) was commissioned into the United States Navy and later joined the Confederate States Navy serving as executive officer of the ironclad CSS "Arkansas" and while serving as commander of naval forces in Louisiana was killed in a battle with Union gunboats. In 1853 Stevens married Mary Singletary Capers (1833–1894), who was the sister of Ellison Capers a fellow SCMA graduate and professor who also later became Episcopal Bishop of South Carolina. In 1895, he married Harriet Rebecca Palmer (1842–1922) who survived him. Bishop Stevens died in Charleston in 1910 and was buried at the Magnolia Cemetery. He and his first wife had 6 children only one of whom survived him. Stevens Barracks on the campus of The Citadel was named in his honor.