Ganda was born in Niamey, the capital of Niger, in 1935 and was of Djerma ethnicity. He completed his primary studies in Niamey and at the age of 16 joined the French Far East Expeditionary Corps as a soldier from 1951 to 1955. After spending two years in Asia during the First Indochina War he returned to Niger, where he was unable to find work. He emigrated to Côte d'Ivoire and became a longshoreman in the port of Abidjan. There he met French anthropologist and filmmaker Jean Rouch. Rouch was interested in the Nigerien community in Côte d'Ivoire and hired Ganda as a statistician for his research on immigration.