Hugging Face's logo

dranger003
/
Smaug-Mixtral-v0.1-iMat.GGUF

Text Generation
GGUF
Inference Endpoints
Model card Files Community
1
dranger003 commited on
Commit
5967f61
•
1 Parent(s): 838331a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -10,4 +10,6 @@ library_name: gguf
10
  **NOTE**: The new IQ3_M/IQ3_S (and updated Q3_K_XS) quants have been added. You need a [separate PR](https://github.com/ggerganov/llama.cpp/pull/5721) to use the new IQ2_S/IQ2_M quants.
11
  | Layers | Context | [Template](https://huggingface.co/abacusai/Smaug-Mixtral-v0.1/blob/main/tokenizer_config.json#L32) |
12
  | --- | --- | --- |
13
- | <pre>32</pre> | <pre>32768</pre> | <pre>\<s\>[INST] {prompt} [/INST]<br>{response}</pre> |
 
 
 
10
  **NOTE**: The new IQ3_M/IQ3_S (and updated Q3_K_XS) quants have been added. You need a [separate PR](https://github.com/ggerganov/llama.cpp/pull/5721) to use the new IQ2_S/IQ2_M quants.
11
  | Layers | Context | [Template](https://huggingface.co/abacusai/Smaug-Mixtral-v0.1/blob/main/tokenizer_config.json#L32) |
12
  | --- | --- | --- |
13
+ | <pre>32</pre> | <pre>32768</pre> | <pre>\<s\>[INST] {prompt} [/INST]<br>{response}</pre> |
14
+
15
+ ![Adding IQ2_S and IQ2_M to complete coverage of the 2-3 bit quantization range](https://private-user-images.githubusercontent.com/48489457/307680119-7a86761a-c8c7-4774-af14-f80fcc2a6ed1.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDg5NjQwMzEsIm5iZiI6MTcwODk2MzczMSwicGF0aCI6Ii80ODQ4OTQ1Ny8zMDc2ODAxMTktN2E4Njc2MWEtYzhjNy00Nzc0LWFmMTQtZjgwZmNjMmE2ZWQxLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDAyMjYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwMjI2VDE2MDg1MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWZlNGY2YmU4YTE5ZTcwYWQ3NWNiYWE5MTRkYjM5NDkwMmJkZGE2ZTVjYmZkM2VkNzFhODgwZmViZjIxZDYyYjEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.IR01nzkx5c3JSey73rTWyt8W-MYKOuBVhh5ighCkSFM)