He grew up in Ferrara and lost his mother aged three when she died giving birth to Isabella Maria d'Este. As was customary for a younger son, he was given a military education. After spending time in France, his brother Ercole II d'Este invited him to help Charles V, Holy Roman Emperor in Lombardy in May 1536 as a captain of light cavalry. He also took part in the 1541 Algiers expedition.