Refactor tokenisation logic out of parse.