Fixed inconsitence with tokenizer high order functions.