Hewwo!
#9
by
at8u
- opened
Hello! I noticed that some of your models don't produce coherent output. This appears to affect q4_K_M and q5_K_M (might include others), q4_0 and q5_0 is unaffected. I'm assuming they've become corrupted or something else?
Example of output:
Basically���������ogle =" basically attsign particul Spo './��icale klDescript or села pis socproject Usually „rac +\ pau Place{$\ soc conna gcc BasicallyLengt h \(\ anc
Thank you and have a nice day!
Thank you for the report. I've deleted all the k-quants for this model.
Turns out certain models aren't currently compatible with the new k-quant format, due to a quirk of how the model was originally made. The llama.cpp team are working on this, but currently they're not supported.
Let me know if you find any others.
Note that this model is old and has been superseded anyway - I'd strongly recommend WizardLM 7B 1.0 or WizardLM 7B 1.0 Uncensored over it.