Robert Fletcher was born circa 1738 to Robert Fletcher and Elizabeth Fletcher (née Lyon). During his youth his father served as a major in Lord Ogilvy's regiment of the Jacobite Army during the rising of 1745. After the defeat of the Jacobites, he fled to Bergen, Norway and did not return to Scotland until 1754.