Despite the significant head start of the southern forces, cavalry forces from each side met, and began skirmishing on May 4. Union General George Stoneman's cavalry encountered and began skirmishing with Brig. Gen. J.E.B. Stuart's cavalry, the Confederate's rearguard. Learning of this, and frustrated by the slow progress of his own forces which were passing through Williamsburg, Confederate General Joseph E. Johnston detached a portion of his troops to confront the advancing Union troops. On May 5, 1862, the single-day Battle of Williamsburg took place. During the day, several groups of Union troops battled with Confederates near the Williamsburg line, with much of the action at or near Fort Magruder. Confederate casualties, including the cavalry skirmishing on May 4, were 1,682. Union casualties were 2,283.