Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
GGUF for [FuseChat-7B-VaRM](https://huggingface.co/FuseAI/FuseChat-7B-VaRM) using [Capybara-Binarized](https://huggingface.co/datasets/jan-hq/ldjnr_capybara_binarized) for Imatrix at 8k context with Q8 model
|
| 2 |
+
(Only got 70% through the dataset, I got tired of waiting :| cries in 6GB vram)
|
| 3 |
+
|
| 4 |
+
Sillytavern template inside presets folder (unsure if it's correct)
|
| 5 |
+
|
| 6 |
+
I wouldn't go lower than IQ4_XS, IQ3_XXS and IQ3_XS work but they're a little dumb
|
| 7 |
+
|
| 8 |
+
IQ1_S is unusable too dumb and has a repetition problem
|
| 9 |
+
|
| 10 |
+
Imatrix was also used for Q4_K_M and Q5_K_M
|