Fix a few tokenizer functions, clearer commenting.