---
license: other
license_name: microsoft-research-license
tags:
- merge
- not-for-all-audiences
---
# DarkForest 20B v2.0 - GGUF IMat quants

[IMatrix file](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.fp16_512.imatrix?download=true) prepared witch context 4096 and 5521 chunks of wiki.test.raw.
Original model info: [DarkForest-20B-v2.0](https://huggingface.co/TeeZee/DarkForest-20B-v2.0)

[q8_0](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0_Q8_0_imatrix.gguf?download=true):
- [5521]8.5016,
- Final estimate: PPL = 8.5016 +/- 0.02134

[q6_k](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q6_K_imatrix.gguf?download=true):
- [5521]8.5046,
- Final estimate: PPL = 8.5046 +/- 0.02136

[q5_0](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q5_0_imatrix.gguf?download=true):
- [5521]8.4903,
- Final estimate: PPL = 8.4903 +/- 0.02132

[q4_K_S](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q4_K_S_imatrix.gguf?download=true):
- [5521]8.5880,
- Final estimate: PPL = 8.5880 +/- 0.02162

[q4_K_M](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q4_K_M_imatrix.gguf?download=true):
- [5521]8.5906,
- Final estimate: PPL = 8.5906 +/- 0.02163

[q4_0](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q4_0_imatrix.gguf?download=true):
- [5521]8.5610,
- Final estimate: PPL = 8.5610 +/- 0.02151

[q3_K_M](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q3_K_M_imatrix.gguf?download=true):
- [5521]8.7283,
- Final estimate: PPL = 8.7283 +/- 0.02196

[q2_K](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.q2_K_imatrix.gguf?download=true):
- [5521]9.2445
- Final estimate: PPL = 9.2445 +/- 0.02351

[IQ2_XS](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.IQ2_XS_imatrix.gguf?download=true):
- [5521]9.8329,
- Final estimate: PPL = 9.8329 +/- 0.02452

[IQ2_XSS](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.IQ2_XXS_imatrix.gguf?download=true):
- [5521]10.5170,
- Final estimate: PPL = 10.5170 +/- 0.02651

[IQ1_S](https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF-iMat/resolve/main/DarkForest-20B-v2.0.IQ1_S_imatrix.gguf?download=true):
- [5521]13.9487,
- Final estimate: PPL = 13.9487 +/- 0.03502 - UNUSABLE, model is too small to be coherent in 1 bit.


All comments are greatly appreciated, download, test and if you appreciate my work, consider buying me my fuel:
<a href="https://www.buymeacoffee.com/TeeZee" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 60px !important;width: 217px !important;" ></a>