Finley volunteered as a private in the 6th Florida Infantry of the Confederate Army, and was successively promoted to be the colonel of the regiment. He took part in the Kentucky Campaign in Maj. Gen. Edmund Kirby Smith's column. His first significant combat came at the Battle of Chickamauga, where his regiment captured a battery of Union artillery, but was unsupported and forced to withdraw with 165 casualties.