Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,31 @@
|
|
1 |
-
---
|
2 |
-
license: mit
|
3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: mit
|
3 |
+
---
|
4 |
+
Imatrix compressions of FP Merge of "D_AU-Mistral-7B-Instruct-v0.2-Bagel-DarkSapling-DPO-7B-v2.0".
|
5 |
+
|
6 |
+
"Imatrix Plus" is an upgraded form of Imatrix which using full precision for specific parts of the compression.
|
7 |
+
As a result all compressions will be slightly larger in size than standard 13B compressions.
|
8 |
+
|
9 |
+
This method results in a higher quality model, especially at lower compressions.
|
10 |
+
This method is applied across all compressions from IQ1 to Q8.
|
11 |
+
|
12 |
+
Even IQ1_S - the most compressed verison - works well, however IQ4/Q4 are suggested as minimums for quality.
|
13 |
+
Highest quality will be Q6/Q8.
|
14 |
+
|
15 |
+
In addition the Imatrix file used to "fix" the compressed files post compression resulted in
|
16 |
+
over 2 whole points lower perplexity at IQ1_S vs some of the other "Imatrix" files currently in use.
|
17 |
+
|
18 |
+
This merge was an experiment to test already established Roleplay, Fiction and Story
|
19 |
+
generation of "DarkSapling" with a some of "Bagel"'s qualities with a Mistral Instruct Base.
|
20 |
+
|
21 |
+
For Imatrix plus this was a test of high precision in specific areas of the model leading to a slightly larger compressed file.
|
22 |
+
In addition the Imatrix process itself used a larger "calibration" file than standard to further enhance quality.
|
23 |
+
|
24 |
+
The process added appoximately 310 MB to each compressed file.
|
25 |
+
An additional enhancement added another 200 mb to each compressed file.
|
26 |
+
|
27 |
+
A blank or standard Alpaca Template for text generation will work.
|
28 |
+
|
29 |
+
Context length: 32768.
|
30 |
+
|
31 |
+
Please see the orginal model card for specific details of use, additional credits and tips:
|