Robert was born on 27 September 1739, at Mount Stewart, the eldest son of Alexander Stewart and his wife Mary Cowan. His father was an alderman of Derry in 1760, and his grandfather, Colonel William Stewart, had commanded one of the two companies of Protestant soldiers that Derry admitted into its walls when Mountjoy was sent there by Tyrconnell before the start of the siege. Robert's mother was a daughter of John Cowan, also an alderman of that same town, and a half-sister of Robert Cowan, Governor of Bombay. His parents had married on 30 June 1737 in Dublin.