Born in Gaspé, Quebec, Quebec, Canada, Larrivée played junior hockey for the Chicoutimi Sagueneens from 1999–2003. He was drafted by the Montreal Canadiens of the NHL in the fourth round (114th overall) in 2000. In 2003, he was signed by Canadiens and played for the Canadiens' top affiliate the Hamilton Bulldogs of the AHL and the Columbus Cottonmouths of the ECHL. He played the following season with the Long Beach Ice Dogs of the ECHL and nine games with Hamilton. In 2005–06, he played the entire season with Long Beach.