In May 1862, the 46th Virginia Infantry (which included three companies from Albemarle County as well as two companies of the storied Richmond Light Infantry Blues among its nine companies), reorganized after having lost the Battle of Roanoke Island in January 1862 as well as many men due to enlistment expiration and diseases including measles and typhoid fever. The 46th's troops refused to re-elect Lt. Col. Richardson (who then transferred and became a major in the 39th Virginia Infantry), and instead elected Duke (who had expected to return to Albemarle and his law practice) as Colonel of 46th Virginia. While the 19th Virginia remained in Pickett's Brigade, Duke and his new troops men were in Wise's Brigade, under former Virginia Governor, now General Henry A. Wise. Although their Brigade participated somewhat in the Seven Days Battles which saved Richmond in the summer of 1862, it mainly dealt with false alarms about Union troop movements on the James River and for the next 16 months, remained stuck on the Peninsula as a counterforce to the Union presence at Norfolk and Fortress Monroe. Col. Walker would remain in that rank until he resigned in March, 1864, following quarrels with his commander, General Wise.