Update tokenizer_config.json
#2
by
PITTI
- opened
At inference, the model does not generate <|im_end|> but <|endoftext|> instead. Using <|im_end|> results in infinite generation, <|endoftext|> fixes it but this may not be the most relevant fix, there seems to be a broader issue with this tokenizer: https://x.com/danielhanchen/status/1856442699689414970.
based on this post, you may also consider removing the chat template for this model. I typically ignore the chat template but, when I tested with the chat template to verify the post, and I ran into the infinite generation issue