---
license: cc-by-2.0
library_name: gguf
pipeline_tag: text-generation
---
* GGUF importance matrix (imatrix) quants for https://huggingface.co/ShinojiResearch/Senku-70B-Full
* The importance matrix was trained for ~50K tokens (105 batches of 512 tokens) using a [general purpose imatrix calibration dataset](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384).
* The [imatrix is being used on the K-quants](https://github.com/ggerganov/llama.cpp/pull/4930) as well.

**2024-02-26**: Updating quants - IQ3_M/IQ3_S/IQ3_XS and IQ2_M/IQ2_S (requires latest commit [a33e6a0d](https://github.com/ggerganov/llama.cpp/commit/a33e6a0d2a66104ea9a906bdbf8a94d050189d91)).
| Layers | Context | Template |
| --- | --- | --- |
| <pre>80</pre> | <pre>32764</pre> | <pre><\|im_start\|>system<br>{instructions}<\|im_end\|><br><\|im_start\|>user<br>{prompt}<\|im_end\|><br><\|im_start\|>assistant<br>{response}</pre> |

![New 2/3-bit quantization types](https://private-user-images.githubusercontent.com/48489457/307680119-7a86761a-c8c7-4774-af14-f80fcc2a6ed1.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDg5NjQwMzEsIm5iZiI6MTcwODk2MzczMSwicGF0aCI6Ii80ODQ4OTQ1Ny8zMDc2ODAxMTktN2E4Njc2MWEtYzhjNy00Nzc0LWFmMTQtZjgwZmNjMmE2ZWQxLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDAyMjYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwMjI2VDE2MDg1MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWZlNGY2YmU4YTE5ZTcwYWQ3NWNiYWE5MTRkYjM5NDkwMmJkZGE2ZTVjYmZkM2VkNzFhODgwZmViZjIxZDYyYjEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.IR01nzkx5c3JSey73rTWyt8W-MYKOuBVhh5ighCkSFM)