Do some extra interpretation of tokens in the lexer to avoid code duplication in the future.