Update README.md
Browse files
README.md
CHANGED
|
@@ -1,10 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
Requantizations of a Q5_K_M quant of a trending 70b model without better quant/fp16 available, this through a Q8_0 intermediary step.
|
| 2 |
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
|
| 5 |
Bonus : a Kobold.CPP Frankenstein which reads IQ3_XXS models and is not affected by the Kobold.CPP 1.56/1.57 slowdown at the cost of an absent Mixtral fix.
|
| 6 |
https://github.com/Nexesenex/kobold.cpp/releases/tag/v1.57_b2030
|
| 7 |
|
|
|
|
|
|
|
| 8 |
Miku 70b has a theta of 1,000,000, like CodeLlama, and not 10,000, like Llama 2 models usually have.
|
| 9 |
That feature singularizes it to my knowledge to ALL Llama 2 models, beside Codellamas which also have a theta of 1,000,000..
|
| 10 |
|
|
@@ -21,6 +34,8 @@ BUT Miqu is NOT a CodeLlama 70b (released only a few days after Miqu 70b), becau
|
|
| 21 |
|
| 22 |
So, CodeLlama 70b is nerfed like the other CodeLlama in general benchmarks terms, while Miku is matching a FINETUNED Llama-2 expectations.
|
| 23 |
|
|
|
|
|
|
|
| 24 |
Benchs I made with the original Q2_K quant of Miku 70b, made from the FP16 and published by Miqudev :
|
| 25 |
|
| 26 |
|
|
|
|
| 1 |
+
Miqu 1 70b : a possible leak of Mistral Medium Alpha
|
| 2 |
+
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
Requantizations of a Q5_K_M quant of a trending 70b model without better quant/fp16 available, this through a Q8_0 intermediary step.
|
| 6 |
|
| 7 |
+
Miqudev provided Q5_K_M, Q4_K_M, and Q2_K from his probable FP16.
|
| 8 |
+
|
| 9 |
+
Here, you will find :
|
| 10 |
+
- Q3_K_M, Q3_K_S, Q3_K_XS, Q2_K_S, IQ3_XXS SOTA and IQ2_XS SOTA available.
|
| 11 |
+
- Q3_K_L and Q4_K_S on quantization for tonight.
|
| 12 |
+
- IQ2_XXS SOTA for tomorrow.
|
| 13 |
+
|
| 14 |
+
---
|
| 15 |
|
| 16 |
Bonus : a Kobold.CPP Frankenstein which reads IQ3_XXS models and is not affected by the Kobold.CPP 1.56/1.57 slowdown at the cost of an absent Mixtral fix.
|
| 17 |
https://github.com/Nexesenex/kobold.cpp/releases/tag/v1.57_b2030
|
| 18 |
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
Miku 70b has a theta of 1,000,000, like CodeLlama, and not 10,000, like Llama 2 models usually have.
|
| 22 |
That feature singularizes it to my knowledge to ALL Llama 2 models, beside Codellamas which also have a theta of 1,000,000..
|
| 23 |
|
|
|
|
| 34 |
|
| 35 |
So, CodeLlama 70b is nerfed like the other CodeLlama in general benchmarks terms, while Miku is matching a FINETUNED Llama-2 expectations.
|
| 36 |
|
| 37 |
+
---
|
| 38 |
+
|
| 39 |
Benchs I made with the original Q2_K quant of Miku 70b, made from the FP16 and published by Miqudev :
|
| 40 |
|
| 41 |
|