Starke became Colonel of the 28th Mississippi Cavalry regiment by commission, dated February 24, 1862. His regiment was attached to the command of Brigadier-General Martin Smith, for the defense of Vicksburg, Mississippi, and in September was nearly 700 strong. Stationed at Panola in November, he gave notice of the advance of Hovey's expedition from Arkansas, and during that fruitless movement by the enemy his regiment was engaged in various skirmishes. From this time during the long-continued efforts for the reduction of Vicksburg the Confederate cavalry was busily engaged in watching the movements of the enemy. At the organization of forces outside Vicksburg by General Joseph Johnston he and his regiment were assigned to the cavalry brigade of Brigadier-General William Jackson, first composed of the regiments of Pinson, Harris, Starke, and Adams', and Steede's battalion. In March 1863, he participated in the victory at Thompson's Station, under Major-General Earl Van Dorn.