At the Battle at Lee's Mills, April 16, 1862, he led a company of skirmishers early in the engagement. On December 13, 1862, at the Battle of Fredericksburg, Lieutenant Colonel Foster led the 4th Vermont Infantry on the skirmish line in front of Howe's division of the VI Corps. At the battle of Winchester, Colonel Foster had command of the Vermont Brigade.