correction of tokenizer as a local var