Any plan for making HQQ+ 2bit quant for Mixtral or larger models?
#1
by
raincandy-u
- opened
Just curious. The HQQ+ method is very impressive!
Thank you!
Yes, coming soon, we already have a good Llama-13B with HQQ+ 2-bit that we should release soon!
If you want to use HQQ Mixtral, we already have this one: https://huggingface.co/mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bitgs8-metaoffload-HQQ
mobicham
changed discussion status to
closed