Kaladze was limited to just six league appearances and 11 total appearances in the 2003–04 season. In the next season, Kaladze played just 19 Serie A matches and five in the Champions League as Milan finished as runners-up in both competitions. He was an unused substitute in that season's Champions League final, where Milan lost on penalties to Liverpool after a 3–3 draw. He was said to be frustrated with his lack of first-team options and a move to Chelsea, in exchange for Hernán Crespo or for £4 million, was widely reported. Kaladze himself said, "I have agreed everything with the Chelsea management. Now it is necessary to wait for them to reach an agreement with Milan and I think I could become a Chelsea player next week." Chelsea opted to sign Asier del Horno instead. On 30 June 2005, Kaladze extended his contract with Milan until 2010 and again on 4 September 2006, this time until 2011. In 2005–06, an injury to Paolo Maldini meant that Kaladze was moved back into the centre of defence, his favoured position. Milan finished third that season, although they would have finished second if there were no 2006 Italian football scandal which resulted in a 30-point deduction.