In 2003-04 Ljungberg made 30 league appearances during Arsenal's unbeaten Invincibles season. That season he scored the winning goal against rivals Tottenham Hotspur at Highbury. The following season (2004–05) Ljungberg played in his fourth FA Cup final, when he came on as a substitute and scored one of Arsenal's penalties in the shootout as they defeated Manchester United.