Calgary assigned Patterson to their Central Hockey League (CHL) affiliate, the Colorado Flames to end the 1982–83 season. He appeared in seven regular season games and three more in the CHL playoffs. The Flames felt that Patterson would benefit from playing with the Canadian national team and at the 1984 Winter Olympics, however Team Canada officials felt he didn't fit the team they were building. The Flames assigned him back to Colorado, intending that he would play the entire 1983–84 season in the CHL. He was recalled to Calgary on October 22, 1983, after only six games for what was expected to be a short stint with the NHL team. Instead, he established a place with Doug Risebrough and Richard Kromm as Calgary's top defensive line and never returned to the minors.