Question: where did brianna and roger go back to the future?
Answer: Brianna and Roger returned to Craigh na Dun. It's possible Brianna, Roger, and Jemmy time-traveled to 20th century Scotland. The reason for Roger's confusion could mean that they appeared right in front of Craigh na Dun, the time travel stone structure they originally went through back in Season 4.