The 34th Arkansas subsequently served in the defense of Little Rock in September, 1863, with Fagan's Brigade. The Union advance upon Little Rock was opposed mainly by the Confederate cavalry divisions of Generals Marmaduke and Walker. The Confederate infantry brigades were dug in on the north side of the Arkansas River. According to Captain Ethan Allen Pinnell of the Eighth Missouri Infantry, "Our works extend from the Arkansas river two miles below the city. to the eastern part of Crystal Hill, a distance of 6 miles. Gen'l Fagan's Brig. is on the extreme right, Parson's on Fagan's left, Frost in the center and McRea's on the left." The Union forces established a pontoon bridge near Bayou Fourche, and crossed to the south side of the very low Arkansas River. With his works on the north side of the river now flanked, Major General Price was forced to abandon the city on September 10, after a brief engagement at Bayou Fourche. Price's Army withdrew in the direction of Rockport. Colonel Alexander T. Hawthorn took command of the brigade in the fall of 1863. The regiment spent the winter of 1863-64 in Camden, Ouachita County.