BlueNipples
commited on
Commit
•
56ffb1f
1
Parent(s):
2ebe4e5
Update README.md
Browse files
README.md
CHANGED
@@ -17,4 +17,8 @@ Thanks to lucyknada for helping me get the imatrix quants done quicker!
|
|
17 |
|
18 |
## Importance Matrix Note
|
19 |
|
20 |
-
Imatrix currently does not run with Koboldcpp although bound to tbe supported in the future as it is supported by Llamacpp (and I'm guessing therefor ooba). Those quants should provide a perplexity boost especially to the smaller quants. The dat files are also there so if you make a fp16 gguf from the main model cards you might be able to save yourself some time producing your own imatrix quants.
|
|
|
|
|
|
|
|
|
|
17 |
|
18 |
## Importance Matrix Note
|
19 |
|
20 |
+
Imatrix currently does not run with Koboldcpp although bound to tbe supported in the future as it is supported by Llamacpp (and I'm guessing therefor ooba). Those quants should provide a perplexity boost especially to the smaller quants. The dat files are also there so if you make a fp16 gguf from the main model cards you might be able to save yourself some time producing your own imatrix quants.
|
21 |
+
|
22 |
+
### Format Notes
|
23 |
+
|
24 |
+
Solar is desgined for 4k context, but Nyx reports that his merge works to 8k. Given this has a slerp gradient back into that, I'm not sure which applies here. Alpaca instruct formatting.
|