Tristan Druyen
commited on
Add info abt imatrix dataset
Browse files
README.md
CHANGED
|
@@ -17,5 +17,8 @@ imatrix generated with:
|
|
| 17 |
|
| 18 |
./imatrix -ofreq 4 -b 512 -c 512 -t 14 --chunks 24 -m ../models/Cerebrum-1.0-8x7b-GGUF/cerebrum-1.0-8x7b-Q8_0.gguf -f ./groups_merged.txt
|
| 19 |
|
|
|
|
|
|
|
|
|
|
| 20 |
Sadly this means the imatrix is generated from the Q8 instead of the unquantized f16, like it should be, sadly I can't get it to work with the f16 on my machine at the moment. It should still improve the performance of the quants though.
|
| 21 |
|
|
|
|
| 17 |
|
| 18 |
./imatrix -ofreq 4 -b 512 -c 512 -t 14 --chunks 24 -m ../models/Cerebrum-1.0-8x7b-GGUF/cerebrum-1.0-8x7b-Q8_0.gguf -f ./groups_merged.txt
|
| 19 |
|
| 20 |
+
with the dataset from here:
|
| 21 |
+
https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384
|
| 22 |
+
|
| 23 |
Sadly this means the imatrix is generated from the Q8 instead of the unquantized f16, like it should be, sadly I can't get it to work with the f16 on my machine at the moment. It should still improve the performance of the quants though.
|
| 24 |
|