Sweeney was born in Glasgow and moved with his family to London when he was three-years-old. He started his career at Millwall, where he made his debut in 2002, but he had to wait another year for his breakthrough. Sweeney played 34 times for Millwall in 2003–04 and he played a major role in the Lions' route to the 2004 FA Cup Final, where he started against Manchester United at the Millennium Stadium, Millwall lost 3–0. Following the cup final appearance, Millwall received a place in the UEFA Cup for the 2004–05 season, in which Sweeney played a part in the first round of the tournament, where they were defeated by Hungarian side Ferencvárosi.