Fix conditions for tokenizer