Update README.md
Browse files
README.md
CHANGED
@@ -3,11 +3,11 @@ license: llama2
|
|
3 |
---
|
4 |
Quants for Sao10K's model WinterGoddess 1.4 70b : https://huggingface.co/Sao10K/WinterGoddess-1.4x-70B-L2
|
5 |
|
6 |
-
|
7 |
|
8 |
-
I don't know who did the job, only that I found this quant hanging around without FP16 : https://huggingface.co/mishima/WinterGoddess-1.4x-limarpv3-70B-L2-32k.GGUF
|
9 |
|
10 |
-
So I made a Q8_0 out of it (best way to requantize after), and requantized it in Q3_K_S and Q2_K for my
|
11 |
|
12 |
Lowers quants (SOTA 2 bits) to come if I'm able to make an iMatrix on my config (64GB RAM).
|
13 |
|
|
|
3 |
---
|
4 |
Quants for Sao10K's model WinterGoddess 1.4 70b : https://huggingface.co/Sao10K/WinterGoddess-1.4x-70B-L2
|
5 |
|
6 |
+
With a twist : the model I used come from a third party, and has been tweaked with limarvp3 and a Linear Rope 8 training to go to 32k context (with even better results in rope 4 and rope 2, maybe other lesser ropes as well)
|
7 |
|
8 |
+
I don't know who did the job, only that I found this Q4_K_S quant of it hanging around without FP16 : https://huggingface.co/mishima/WinterGoddess-1.4x-limarpv3-70B-L2-32k.GGUF
|
9 |
|
10 |
+
So I made a Q8_0 out of it (best way to requantize after), and requantized it in Q3_K_S and Q2_K for my needs.
|
11 |
|
12 |
Lowers quants (SOTA 2 bits) to come if I'm able to make an iMatrix on my config (64GB RAM).
|
13 |
|