DavidAU commited on
Commit
235e7d3
1 Parent(s): 42cbe9e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +31 -3
README.md CHANGED
@@ -1,3 +1,31 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ ---
4
+ Imatrix compressions of FP Merge of "D_AU-Mistral-7B-Instruct-v0.2-Bagel-DarkSapling-DPO-7B-v2.0".
5
+
6
+ "Imatrix Plus" is an upgraded form of Imatrix which using full precision for specific parts of the compression.
7
+ As a result all compressions will be slightly larger in size than standard 13B compressions.
8
+
9
+ This method results in a higher quality model, especially at lower compressions.
10
+ This method is applied across all compressions from IQ1 to Q8.
11
+
12
+ Even IQ1_S - the most compressed verison - works well, however IQ4/Q4 are suggested as minimums for quality.
13
+ Highest quality will be Q6/Q8.
14
+
15
+ In addition the Imatrix file used to "fix" the compressed files post compression resulted in
16
+ over 2 whole points lower perplexity at IQ1_S vs some of the other "Imatrix" files currently in use.
17
+
18
+ This merge was an experiment to test already established Roleplay, Fiction and Story
19
+ generation of "DarkSapling" with a some of "Bagel"'s qualities with a Mistral Instruct Base.
20
+
21
+ For Imatrix plus this was a test of high precision in specific areas of the model leading to a slightly larger compressed file.
22
+ In addition the Imatrix process itself used a larger "calibration" file than standard to further enhance quality.
23
+
24
+ The process added appoximately 310 MB to each compressed file.
25
+ An additional enhancement added another 200 mb to each compressed file.
26
+
27
+ A blank or standard Alpaca Template for text generation will work.
28
+
29
+ Context length: 32768.
30
+
31
+ Please see the orginal model card for specific details of use, additional credits and tips: