Baldwin served in the Civil War in the 19th Michigan Infantry, initially as a first lieutenant, fighting in all his regiment's battles from 1862 to 1865. In 1864, then-Captain Baldwin participated in General William Tecumseh Sherman's famous March to the Sea, and on July 20 of that year distinguished himself at the Battle of Peachtree Creek, earning his first Medal of Honor.