Alexander was born on 26 March 1746 in Ireland. He was the fifth son of Alexander Stewart and his wife Mary Cowan. His father's family was Ulster Scots and came from County Donegal. His father had bought the Newtownards and Comber estates in County Down in 1743 and lived at Mount Stewart, near Newtownards. His father also still was an alderman of Londonderry (or Derry) in 1760. His grandfather, Colonel William Stewart, had commanded one of the two companies of Protestant soldiers that Derry admitted into town when Mountjoy was sent to Derry by Tyrconnell before the start of the siege. Alexander's mother was a daughter of John Cowan, alderman of Derry and sister of Robert Cowan, Governor of Bombay. Her family also was Ulster Scots. His parents had married on 30 June 1737 in Dublin.