Other formats?
#1
by
wise-time
- opened
Would really like to see a q8_0 version, I find this available on most other webui compatible language models.
@wise-time
I decided to exclude q8_0 for now as the difference in performance between q5_1 and q8_0 shouldn't be that great according to llama.cpp.
If the new model format is defined and finalized i will probably uplaod all models in all available quantization formats. But as it's not clear when this will happen i'm waiting for now.