refactoring tokenizer