GGUF Quants
#3
by
Dracones
- opened
These are up now at https://huggingface.co/Dracones/Midnight-Miqu-103B-v1.0-GGUF
These are basic, non measurement quants as I'm pretty new to the GGUF format. If someone has a good link describing how to do some of the newer GGUF quant methods, especially in the lower bits, I'd be happy to cook those and upload them.
Looks like mradermacher is starting to upload his quants. I'll likely change my README to recommend using his GGUF's as he has more experience with them and also makes IQ quants. I'm mostly familiar with EXL and EXL2.
Thanks, Dracones!
sophosympatheia
changed discussion status to
closed