minor refactor

- throw logic at tokenizer moved to a separate function