In 1509 (the year he began the construction of new city walls at Padua), however, he was defeated at the Battle of Agnadello, being wounded as a result. d'Alviano was blamed for the loss, as he allegedly attacked the enemy without the authorisation of Orsini, then commander-in-chief. Captured by the French, he remained a prisoner until 1513, when an alliance between France and Venice was formed against the Duchy of Milan. d'Alviano was freed and fought under the French commander Louis de la Trémoille. He was defeated at the Battle of Vicenza by the Spanish Viceroy of Naples Ramón de Cardona.