Model Quantization
#1
by
Philipp-Sc
- opened
Hi @TheBloke , could you provide GGUF quantization for this? Only if its not to much work for you, I can also try myself (I have limited server resources so would be happy if you can help). Thanks in advance!
I was able to use google colab to quantize the models.
Philipp-Sc
changed discussion status to
closed