Sandby was born in Nottingham, and baptised there in 1731, although his date of birth has traditionally been given as 1725. In 1745 he moved to London where he followed his brother Thomas in obtaining an appointment in the military drawing department at the Tower of London. Following the suppression of the Jacobite rebellion of 1745, Sandby was employed to assist in the military survey of the new road to Fort George, and of the northern and western parts of the Highlands, under the direction of Colonel David Watson. He was later appointed draughtsman to the survey.