Question
#1
by
FrescoHF
- opened
Do I understand correctly that weighted/imatrix Q6_K will be better on model responses than static quants Q6_K? Thank you in advance for your reply
Do I understand correctly that weighted/imatrix Q6_K will be better on model responses than static quants Q6_K? Thank you in advance for your reply
Yes the wighted/imatrix quants are almost always of much higher quality per size if you are using the model in English. Please look at the quality numbers on https://hf.tst.eu/model#Nomad_12B_V6-GGUF to see how well each quant is expected to perform quality-wise.