Multiple V5.6 GGUF quants
I couldn't find any other gguf quant sizes for v5.6 so I made some here: https://huggingface.co/saishf/HuginnV5.6-12.6B-GGUF
If you're after "S" or "L" I can make those too, just ask :3
thank you, I typically just make a q4_k_s myself, since it's the most commonly used, and what I personally use
I made a full set of static and weighted quants at https://huggingface.co/mradermacher/HuginnV5.6-12.6B-GGUF and https://huggingface.co/mradermacher/HuginnV5.6-12.6B-i1-GGUF
Or at least, that is where they will appear once done.
As a bit of criticism, it would probably help the exposure of this model if the repository wouldn't be so chaotic and wouldn't require so much manual wokr (zip file? mismatching repo name? How are people supposed to find 5.6? :)
V5.6 is a minor revision and a stopgap for v6
It’s not meant to have a lot of exposure, it’s a minor improvement for the few people that’d care
It’s also why I didn’t give it its own repo