Sloane was born on May 19, 1850 in Middleville, New York, the son of James K. Sloane and Louisa Reno. During the American Civil War, James enlisted in the 76th New York Volunteer Infantry, was wounded in Gettysburg and Antietam, was mustered out and re-enlisted twice, and was killed in the Battle of Fort Fisher.