tokenizer undergoing code issues