dranger003
commited on
Commit
•
8a586b8
1
Parent(s):
7164329
Update README.md
Browse files
README.md
CHANGED
@@ -8,6 +8,11 @@ pipeline_tag: text-generation
|
|
8 |
GGUF importance matrix (imatrix) quants for https://huggingface.co/abacusai/Smaug-72B-v0.1
|
9 |
The importance matrix was trained for 100K tokens (200 batches of 512 tokens) using wiki.train.raw.
|
10 |
|
|
|
|
|
|
|
|
|
|
|
11 |
Llama-2 conversation template and system prompt set to the [Qwen system prompt](https://github.com/QwenLM/Qwen/blob/main/examples/system_prompt.md).
|
12 |
|
13 |
| Layers | Context | Template |
|
|
|
8 |
GGUF importance matrix (imatrix) quants for https://huggingface.co/abacusai/Smaug-72B-v0.1
|
9 |
The importance matrix was trained for 100K tokens (200 batches of 512 tokens) using wiki.train.raw.
|
10 |
|
11 |
+
**Update 2024-03-02**:
|
12 |
+
* New quants IQ2_S/IQ2_M, requires commit [a33e6a0d](https://github.com/ggerganov/llama.cpp/commit/a33e6a0d2a66104ea9a906bdbf8a94d050189d91) or later.
|
13 |
+
* The importance matrix was trained for ~50K tokens (105 batches of 512 tokens) using a [general purpose imatrix calibration dataset](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384).
|
14 |
+
* This is a different calibration dataset than the previous quants I posted so we can compare the quality
|
15 |
+
|
16 |
Llama-2 conversation template and system prompt set to the [Qwen system prompt](https://github.com/QwenLM/Qwen/blob/main/examples/system_prompt.md).
|
17 |
|
18 |
| Layers | Context | Template |
|