Lee qualified for two swimming events at the 2004 Summer Olympics in Athens, by eclipsing FINA B-standard entry times of 22.95 (50 m freestyle) and 51.84 (100 m freestyle) from the Dong-A Swimming Tournament in Seoul. In the 100 m freestyle, Lee challenged seven other swimmers on the third heat, including Olympic veteran Carl Probert of Fiji. He shared a second seed and forty-fifth place tie with Panama's Ismael Ortiz in 51.74. In his second event, 50 m freestyle, Lee placed thirty-fifth overall on the morning's preliminaries. Swimming in heat seven, he picked up a fourth spot by 0.15 of a second behind Serbia and Montenegro's Milorad Čavić in 23.05.