Following the surrender of Fort Donelson, he was held as a prisoner of war for six months in Fort Warren, Massachusetts, before being exchanged in August 1862. Soon afterwards, he was promoted to brigadier-general and assigned command of a new and larger brigade composed of troops from Florida and Mississippi. He took part in Braxton Bragg's campaigns in Kentucky and Tennessee in late 1862 through 1863. Brown was wounded in the battles of Perryville and Chickamauga while leading his brigade. His men were a part of the defensive line on Missionary Ridge in 1863.