Ashe's grandson, William Ashe, was a Confederate soldier in the American Civil War, and a son of John B. and the former Eliza Hay. He was killed at Shiloh in Tennessee in 1862, a battle in which William's brother, Samuel Swann Ashe, also fought.