Clevyby commited on
Commit
8a171d1
1 Parent(s): 09110fe

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -66
README.md CHANGED
@@ -113,70 +113,8 @@ model-index:
113
  url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Quantum-Citrus-9B
114
  name: Open LLM Leaderboard
115
  ---
116
- # GGUF / IQ / Imatrix for [Quantum-Citrus-9B](https://huggingface.co/ABX-AI/Quantum-Citrus-9B)
117
 
118
- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/65d936ad52eca001fdcd3245/J0snW3yfLadLuMYERL6X5.png)
119
-
120
- **Why Importance Matrix?**
121
-
122
- **Importance Matrix**, at least based on my testing, has shown to improve the output and performance of "IQ"-type quantizations, where the compression becomes quite heavy.
123
- The **Imatrix** performs a calibration, using a provided dataset. Testing has shown that semi-randomized data can help perserve more important segments as the compression is applied.
124
-
125
- Related discussions in Github:
126
- [[1]](https://github.com/ggerganov/llama.cpp/discussions/5006) [[2]](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384)
127
-
128
- The imatrix.txt file that I used contains general, semi-random data, with some custom kink.
129
-
130
-
131
- # Quantum-Citrus-9B
132
-
133
- This merge is another attempt at making and intelligent, refined and unaligned model.
134
-
135
- Based on my tests so far, it has accomplished the goals, and I am continuing to experiment with my interactions with it.
136
-
137
- It includes previous merges of Starling, Cerebrum, LemonadeRP, InfinityRP, and deep down has a base of layla v0.1, as I am not that happy with the result form using v0.2.
138
-
139
- The model is intended for fictional storytelling and roleplaying and may not be intended for all audences.
140
-
141
- ## Merge Details
142
-
143
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
144
-
145
- ### Merge Method
146
-
147
- This model was merged using the passthrough merge method.
148
-
149
- ### Models Merged
150
-
151
- The following models were included in the merge:
152
- * [ABX-AI/Starfinite-Laymospice-v2-7B](https://huggingface.co/ABX-AI/Starfinite-Laymospice-v2-7B)
153
- * [ABX-AI/Cerebral-Infinity-7B](https://huggingface.co/ABX-AI/Cerebral-Infinity-7B)
154
-
155
- ### Configuration
156
-
157
- The following YAML configuration was used to produce this model:
158
-
159
- ```yaml
160
- slices:
161
- - sources:
162
- - model: ABX-AI/Cerebral-Infinity-7B
163
- layer_range: [0, 20]
164
- - sources:
165
- - model: ABX-AI/Starfinite-Laymospice-v2-7B
166
- layer_range: [12, 32]
167
- merge_method: passthrough
168
- dtype: float16
169
- ```
170
-
171
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
172
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_ABX-AI__Quantum-Citrus-9B)
173
-
174
- | Metric |Value|
175
- |---------------------------------|----:|
176
- |Avg. |66.74|
177
- |AI2 Reasoning Challenge (25-Shot)|65.19|
178
- |HellaSwag (10-Shot) |84.75|
179
- |MMLU (5-Shot) |64.58|
180
- |TruthfulQA (0-shot) |55.96|
181
- |Winogrande (5-shot) |79.40|
182
- |GSM8k (5-shot) |50.57|
 
113
  url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Quantum-Citrus-9B
114
  name: Open LLM Leaderboard
115
  ---
116
+ ## Note:
117
 
118
+ This repo hosts only a Q5_K_S iMatrix of [Quantum Citrus 9B](https://huggingface.co/ABX-AI/Quantum-Citrus-9B).
119
+ GGUF quant is from [ABX-AI/Quantum-Citrus-9B-GGUF-IQ-Imatrix](https://huggingface.co/ABX-AI/Quantum-Citrus-9B-GGUF-IQ-Imatrix).
120
+ The additional files in this GGUF repo is for personal usage using Text Gen Webui with llamacpp_hf.