Fixes bugs in tokenizer code for anti query detection