Lewdiculous commited on
Commit
87d9c89
1 Parent(s): f53c62e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +22 -2
README.md CHANGED
@@ -12,14 +12,34 @@ license: cc-by-4.0
12
 
13
  This repo contains GGUF-IQ-Imatrix quantized model files for [Erosumika-7B-v3-0.2](https://huggingface.co/localfultonextractor/Erosumika-7B-v3-0.2).
14
 
15
- Quants to be uploaded:
16
 
17
  "Q4_K_M", "Q4_K_S", "IQ4_XS", "Q5_K_M", "Q5_K_S",
18
  "Q6_K", "Q8_0", "IQ3_M", "IQ3_S", "IQ3_XXS"
19
 
20
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/65d4cf2693a0a3744a27536c/L2l7DXKyLTD6MuL16JPn1.png)
21
 
22
- ## Model information:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
 
24
  <h1 style="text-align: center">Erosumika-7B-v3-0.2</h1>
25
  <h2 style="text-align: center">~Mistral 0.2 Edition~</h1>
 
12
 
13
  This repo contains GGUF-IQ-Imatrix quantized model files for [Erosumika-7B-v3-0.2](https://huggingface.co/localfultonextractor/Erosumika-7B-v3-0.2).
14
 
15
+ Quants:
16
 
17
  "Q4_K_M", "Q4_K_S", "IQ4_XS", "Q5_K_M", "Q5_K_S",
18
  "Q6_K", "Q8_0", "IQ3_M", "IQ3_S", "IQ3_XXS"
19
 
20
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/65d4cf2693a0a3744a27536c/L2l7DXKyLTD6MuL16JPn1.png)
21
 
22
+ **What does "Imatrix" mean?**
23
+
24
+ It stands for **Importance Matrix**, a technique used to improve the quality of quantized models.
25
+ The **Imatrix** is calculated based on calibration data, and it helps determine the importance of different model activations during the quantization process.
26
+ The idea is to preserve the most important information during quantization, which can help reduce the loss of model performance, especially when the calibration data is diverse.
27
+ [[1]](https://github.com/ggerganov/llama.cpp/discussions/5006) [[2]](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384)
28
+
29
+ For imatrix data generation, kalomaze's `groups_merged.txt` with added roleplay chats was used, you can find it [here](https://huggingface.co/Lewdiculous/Datura_7B-GGUF-Imatrix/blob/main/imatrix-with-rp-format-data.txt). This was just to add a bit more diversity to the data.
30
+
31
+ **Steps:**
32
+
33
+ ```
34
+ Base⇢ GGUF(F16)⇢ Imatrix-Data(F16)⇢ GGUF(Imatrix-Quants)
35
+ ```
36
+ *Using the latest llama.cpp at the time.*
37
+
38
+ # Submitted model image:
39
+
40
+ ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/65d4cf2693a0a3744a27536c/KlJMu81ZMKx3JuiPX4n4R.jpeg)
41
+
42
+ # Original model information:
43
 
44
  <h1 style="text-align: center">Erosumika-7B-v3-0.2</h1>
45
  <h2 style="text-align: center">~Mistral 0.2 Edition~</h1>