Class state tokenizer tests