q8 quantization
Could you also please upload the q8 quantization?
I tested this model and thought the quality of its responses was not good. It's not clear to me that the problem comes from the original model, my testing method, or the quantification. So I deleted the original files and decided to hold off on releasing any more versions.
If you can provide generated samples and test methods to prove that there are no problems with these quantized models, I will be happy to regenerate the q8 version.
I've noticed the same and I was hoping that with q8 it would get more "stable".
It's a pity that q8 is not good either. I just gave up. ┑(￣ω ￣)┍
Here are some other English/Chinese uncensored Llama models if you are interested: MiniChat-3B / CausalLM 7B-DPO-alpha / CausalLM 14-DPO-alpha.
Thank you very much.