moving more stuff to tokenizer from simple