dranger003 commited on
Commit
8a586b8
1 Parent(s): 7164329

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -0
README.md CHANGED
@@ -8,6 +8,11 @@ pipeline_tag: text-generation
8
  GGUF importance matrix (imatrix) quants for https://huggingface.co/abacusai/Smaug-72B-v0.1
9
  The importance matrix was trained for 100K tokens (200 batches of 512 tokens) using wiki.train.raw.
10
 
 
 
 
 
 
11
  Llama-2 conversation template and system prompt set to the [Qwen system prompt](https://github.com/QwenLM/Qwen/blob/main/examples/system_prompt.md).
12
 
13
  | Layers | Context | Template |
 
8
  GGUF importance matrix (imatrix) quants for https://huggingface.co/abacusai/Smaug-72B-v0.1
9
  The importance matrix was trained for 100K tokens (200 batches of 512 tokens) using wiki.train.raw.
10
 
11
+ **Update 2024-03-02**:
12
+ * New quants IQ2_S/IQ2_M, requires commit [a33e6a0d](https://github.com/ggerganov/llama.cpp/commit/a33e6a0d2a66104ea9a906bdbf8a94d050189d91) or later.
13
+ * The importance matrix was trained for ~50K tokens (105 batches of 512 tokens) using a [general purpose imatrix calibration dataset](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384).
14
+ * This is a different calibration dataset than the previous quants I posted so we can compare the quality
15
+
16
  Llama-2 conversation template and system prompt set to the [Qwen system prompt](https://github.com/QwenLM/Qwen/blob/main/examples/system_prompt.md).
17
 
18
  | Layers | Context | Template |