tokenizer: changed the way tokens are produced by the tokenizer