Add assertions to detect misuse of tokenizer.