Swanton, his wife and daughter fled to Maine during the Civil War, leaving the house in the care of a widow Mrs. Johnson. On July 19, 1864 the house became the headquarters of the Army of the Tennessee, on their way to participate in the Battle of Atlanta. The presence of General Thomas W. Sweeny at the Swanton House is recorded in records of the time. Later, when General Sherman ordered the occupation of Atlanta in September 1864, the Army of the Ohio used the Swanton House as its headquarters. The house was thus spared destruction in the war.