During the American Civil War, Bump served in the 25th Wisconsin Infantry Regiment of the Union Army. Engagements he took part in include the Battle of Resaca of the Atlanta Campaign. He achieved the rank of first sergeant.