At the onset of the Civil War, Foster had enlisted as a private of Indiana volunteers. He was quickly promoted to captain in Lew Wallace's 11th Indiana Infantry Regiment and saw action at the Battle of Rich Mountain in western Virginia.