dranger003
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -6,8 +6,10 @@ library_name: gguf
|
|
6 |
pipeline_tag: text-generation
|
7 |
base_model: jondurbin/bagel-dpo-34b-v0.5
|
8 |
---
|
9 |
-
GGUF importance matrix (imatrix) quants for https://huggingface.co/jondurbin/bagel-dpo-34b-v0.5
|
10 |
-
The importance matrix was trained for 100K tokens (200 batches of 512 tokens) using wiki.train.raw.
|
|
|
|
|
11 |
|
12 |
| Layers | Context | [Template](https://huggingface.co/jondurbin/bagel-dpo-34b-v0.5#prompt-formatting) |
|
13 |
| --- | --- | --- |
|
|
|
6 |
pipeline_tag: text-generation
|
7 |
base_model: jondurbin/bagel-dpo-34b-v0.5
|
8 |
---
|
9 |
+
* GGUF importance matrix (imatrix) quants for https://huggingface.co/jondurbin/bagel-dpo-34b-v0.5
|
10 |
+
* The importance matrix was trained for 100K tokens (200 batches of 512 tokens) using wiki.train.raw.
|
11 |
+
* The [imatrix is being used on the K-quants](https://github.com/ggerganov/llama.cpp/pull/4930) as well (below Q6_K).
|
12 |
+
* Generated with llama.cpp commit `f87f7b89`
|
13 |
|
14 |
| Layers | Context | [Template](https://huggingface.co/jondurbin/bagel-dpo-34b-v0.5#prompt-formatting) |
|
15 |
| --- | --- | --- |
|