Gardner attended local district schools and spent two years as a student in the Genesee and Wyoming Seminary. In August 1862, during the American Civil War, he enlisted in Company C of the 130th New York Infantry Regiment. The regiment later became the 19th New York Cavalry Regiment and then the 1st New York Dragoons Regiment. Initially a corporal of the company, he was later promoted to sergeant. In the fall of 1864, during the Valley campaigns of 1864, he lost his right leg, which was amputated at the Camden Street Hospital in Baltimore. Gangrene developed after the amputation, and he needed to be operated on a second time. He was discharged from the army in April 1865. After returning to Attica, he was appointed a postal clerk on the Erie Railway, a position he held for the next three years.