large 2bit models
#1
by
KnutJaegersberg
- opened
It would be great to have 2 bit versions of some larger models, like
https://huggingface.co/CofeAI/FLM-101B
or galactica 120b for using their work token for reasoning. and fine tuned falcon-180b or bloomchat and vulture-180b.
https://huggingface.co/sambanovasystems/BLOOMChat-176B-v1
The request has been received, and I believe the larger version will soon be available.
KnutJaegersberg
changed discussion status to
closed
There is also a large llama-70b fine tune with increased context length. Being able to use that more thanks to 2 bit quantization would be a practical combo, too.
https://huggingface.co/abacusai/Giraffe-v2-70b-32k