After winning the first set, the Serb approached the chair, saying: "Someone is smoking weed, I can smell it, I'm getting dizzy."
Djokovic later told reporters: "You can't believe how bad it was.
"Whoever it is, I hope he doesn't come back tomorrow. He's probably on the seventh sky somewhere."
Djokovic said he had also noticed the issue during a doubles match with partner Janko Tipsarevic on Friday.
"Yesterday in the doubles match, today again. Somebody's really enjoying his life around the tennis court," he joked.
Djokovic went on to close out a 6-4 6-4 victory against the Frenchman in the semi-final.
The top seed will face Britain's Andy Murray in the final later on Sunday.
World number one Novak Djokovic complained to the umpire that he could smell cannabis on court during his Rogers Cup win over Jeremy Chardy.