Rafn was born and grew up in Fredrikstad, Norway, and started playing for the local club Skogstrand IL. As a 15-year-old, he joined Fredrikstad FK's youth academy and was later promoted to the senior team in 2010. He stayed in the club and played 136 league games, before transferring to Gefle as a free agent in November 2015. During the pre-season for 2016 Allsvenskan, Rafn picked up a knee injury and were to miss the whole season. In January 2017 he signed for Lillestrøm.