Baxter originally enrolled in Lancaster with Company C of the 25th Wisconsin Infantry Regiment, composed mostly of volunteers from the counties of southwest Wisconsin. The regiment was organized at La Crosse, Wisconsin, and mustered into service on September 14, 1862. Before they could join the Civil War, they were ordered to proceed west to Minnesota to provide security for Minnesota settlers in the midst of an ongoing Sioux uprising.