EOS token is getting printed
Not sure whether this is a quant error?
what client is this?
It's marked as not being special in your tokenizer_config.json which, from my experience, means that the engine doesn't know it's a special token that it's not meant to display
It's FreeChat, on Mac.
Ah.. Okay, I have no idea why. Is it too late to fix this once it's quantized? Like, is there a way to edit the GGUF file?
You can edit the metadata manually to fix it, I can also just remake them properly
Okay. I have edited the tokenizer_config.json here: https://huggingface.co/migtissera/Tess-v2.5-Phi-3-medium-128k-14B/blob/main/tokenizer_config.json
Can you do a small quant, like maybe an 8-bit, and then we can check it before you go ahead and make all the quants?
@migtissera uploaded fixed q8 to test
Okay, downloading now
Yes! Can confirm this is working perfectly now.
Please go ahead with rest of the quantizations. Thanks for your support and responsiveness!