rewriting the tokenizer, breaking it out