Stuart was born in Georgia, the son of Colonel John Stuart, superintendent of Indian affairs in the southern district, and a prominent loyalist in the War of Independence. Educated at Westminster School, young Stuart entered the 3rd Foot Guards in 1778, and almost immediately returned to America with his regiment. He was present at the siege of Charleston, the battles of Camden and Guilford court-house, and the surrender of Yorktown, returning a regimental lieutenant and an army captain, as was then usual in the Guards.