He was promoted to brigadier general of U.S. volunteers, effective November 29, 1862. Prior to the Chancellorsville campaign, he was relieved from duty with the Army of the Potomac and assigned to the defenses of Washington where he returned to his seat in Congress. After Gettysburg, Marston was directed to establish a prison camp in Maryland, which later became known as Point Lookout. The area was designated the District of Saint Mary's under the overall command of Maj. Gen. Benjamin Butler. In 1864, he commanded a brigade in Maj. Gen. W.F. "Baldy" Smith's XVIII Corps during the Bermuda Hundred Campaign. He took part in a disastrous assault on Cold Harbor, where his brigade suffered heavy casualties. During the Union assaults on Petersburg, Gilman assumed command of the 1st Division in the XVIII Corps on the last day of battle. Thereafter he commanded the Union troops on the north side of the James River with his headquarters located at Fort Pocahontas. Occasionally he returned to command of the 1st Division, including a brief stint during the Second Battle of Fair Oaks. Having been re-elected to Congress he resigned 1865, whereupon he received the thanks of the state of New Hampshire.