Unsloth Tokenizer Bug Fizes
#1
by
squid2
- opened
The Unsloth folks had fixed some tokenizer misconfiguration issues, and Microsoft incorporated their fixes: https://huggingface.co/microsoft/phi-4/commit/6fbb3d3bbe726c99b4188087b4deeec1bceac5ae
It would be nice to incorporate those fixes in your abliterated version.
Thank you for your feedback and support. We have fixed it, but we haven't had a chance to verify it yet.