Refactor tokenizer.