In 1793 during the War of the First Coalition, Gyulay fought in several actions and was promoted to Major in the "O'Donell" Freikorps on 1 May. The following year found him serving in Heinrich, Count of Bellegarde's brigade in a number of minor actions. He defeated the French troops opposed to him in the Battle of Tournay on 22 May 1794. That year he married Justine Wynants (d. 1824). On 19 April 1797, he fought in a successful action in the aftermath of Werneck's defeat at the Battle of Neuwied on the Rhine. The same month he received promotion to Oberstleutnant in his father's regiment.