Cooper was born on January 16, 1856 in Athens, Georgia. His father, Charles Phillip Cooper, served as a United States Treasury Department agent until the American Civil War, when he was appointed to help organize the Confederate States Treasury Department. In 1864, he and his family moved to Jacksonville, Florida in order to escape Union forces.