!!! A new version of the model has been released. I didn’t find any problems with word duplication, but I can’t promise anything. https://huggingface.co/SolidSnacke/Llama-3-Soliloquy-8B-v1.5-64k-i-GGUF
Edit: Currently this model has a problem with repeating words. That is, at some point you may experience duplication, like: passing by the table, he caught a red red red red red red... The problem most likely, as the author of the model explained, may be the wrong EOS token, but this is not certain. The author in another repository wrote that he will soon release a new model, so we are waiting.
I don't know what to write here.
Links to the original model and script:
- openlynn/Llama-3-Soliloquy-8B: https://huggingface.co/openlynn/Llama-3-Soliloquy-8B
- FantasiaFoundry/GGUF-Quantization-Script: https://huggingface.co/FantasiaFoundry/GGUF-Quantization-Script
- Downloads last month
- 39
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.