Quantize
- https://www.kaggle.com/code/reginliu/causallm2gguf-imatrix
- https://colab.research.google.com/drive/1JT3XFjD7CTRB97pu3QpeGuzWA1yYEAM7?usp=sharing
Perplexity
- causallm_14b.IQ4_XS.gguf: PPL = 13.4127 +/- 0.13762
- causallm_14b.IQ3_XS.gguf: PPL = 13.3798 +/- 0.13641
- causallm_14b.IQ2_XXS.gguf: PPL = 15.0160 +/- 0.15004
- Downloads last month
- 141
Unable to determine this model's library. Check the
docs
.