Finally some logic on the tokenizer. Not done yet, but it's almost there.