SOTA 2-Quant of Nous-Heremes-Solar-10.7B and Nous-Hermes-2-Yi-34B, Please 🙏
Amazing work! Is it possible to make a SOTA 2-Quant of NousResearch/Nous-Hermes-2-SOLAR-10.7B and NousResearch/Nous-Hermes-2-Yi-34B, please? 🙏
https://huggingface.co/NousResearch/Nous-Hermes-2-SOLAR-10.7B
https://huggingface.co/NousResearch/Nous-Hermes-2-Yi-34B
Have added 2-bit quants for Nous-Hermes-2-SOLAR-10.7B to the various models repo.
2.18 bits per weight: https://huggingface.co/ikawrakow/various-2bit-sota-gguf/blob/main/nous-hermes-2-10.7b-2.18bpw.gguf
2.70 bits per weight: https://huggingface.co/ikawrakow/various-2bit-sota-gguf/blob/main/nous-hermes-2-10.7b-2.70bpw.gguf
Now looking into Nous-Hermes-2-Yi-34B.
I have added 2-bit quants for Nous-Hermes-Yi-34B to the various models repo.
Thank you! You are awesome!!!! 🙏🚀😋
I'll leave this open so, others can easily reach the models. 😁