Cadell was born in Cockenzie, Haddingtonshire, Scotland, the second son of Hew Francis Cadell (ca.1791 – 27 April 1873), mine-owner and shipbuilder of a notable Scottish family. Educated in Edinburgh and at Cuxhaven, Germany, he joined the East Indiaman "Minerva" at the age of 14, and sailed in her to the first China war in 1839, later claiming a part in the siege of Canton. Soon after he was given a ship by his father.