can you upload a falcon-40b-GPTQ?
#18
by
Gian-hf
- opened
Hi there,
can you please update a GPTQ version of the falcon-40b (the vanilla pretrained version, not the instruct)?
I would really appreciated and would really democratized the use of the pretrained model
Thank yoU!
OK. It'll be really slow, but sure it's a good idea. I'll do GGML as well.
Thank you so much. To give you the full picture I will be using the weights you will produce with this repo https://github.com/rmihaylov/falcontune that allows fine tuning the 40B on a single 40G-A100 on colab (crazy) and the generation it’s not even that slow with that script.