Goldsmid was born in Milan, Italy in May 1818, the only son of Eliza Frances ("née" Campbell) and Lionel Prager Goldsmid, an officer in the 19th Dragoons, and a scion of the well-known London family of that name. His maternal grandmother's father was Revolutionary War aide-de-camp David Franks. Sir Frederic, after completing his education in Paris, King's College School, and King's College London, entered the Madras army in the year 1839, when the first Afghan war was in progress, but he was not among those who fought in that campaign. Before he had been twelve months at Madras his regiment was ordered to proceed to China, and he took part in the actions at Canton and along the coast which preceded the Treaty of Nanking, receiving the Chinese war medal. While this campaign was in progress he was appointed the adjutant of his regiment; and it was then that he turned is attention to the study of Asiatic languages, in which he afterwards became so proficient.