Bayly entered the army on 12 April 1783 as an ensign of the 88th Foot. He was promoted from the half-pay of the 85th Regiment to an ensigncy in the Coldstream Guards on 30 October 1790. While holding the flag, he suffered a hand injury at the Battle of Lincelles on 17 August 1793, and was promoted to Lieutenant in the Guards on 31 August 1793 and purchased a Captaincy in the regiment on 10 September 1799. During this period, he fought with the Guards during the invasion of Holland.