LGxNDs commited on
Commit
d88d189
·
verified ·
1 Parent(s): e91407c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +74 -4
README.md CHANGED
@@ -94,13 +94,13 @@ GeekedOut Quantizer is an advanced quantization framework designed to:
94
 
95
 
96
 
97
- ## IQ2_M Quantization Features
98
 
99
 
100
 
101
 
102
 
103
- The **IQ2_M** (Intelligent Quants) quantization scheme features:
104
 
105
 
106
 
@@ -120,6 +120,56 @@ The **IQ2_M** (Intelligent Quants) quantization scheme features:
120
 
121
 
122
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
123
  ## Supported Use Cases
124
 
125
 
@@ -132,7 +182,7 @@ GeekedOut Quantizer models are designed for:
132
 
133
 
134
 
135
- - Conversational AI applications
136
  -
137
  - Local inference with llama.cpp, LM Studio, Jan, and similar tools
138
  -
@@ -158,6 +208,18 @@ To load IQ2_M quantized models locally using llama.cpp or compatible inference f
158
 
159
 
160
 
 
 
 
 
 
 
 
 
 
 
 
 
161
 
162
 
163
  ## Technical Notes
@@ -172,7 +234,15 @@ To load IQ2_M quantized models locally using llama.cpp or compatible inference f
172
  -
173
  - Uses imatrix-based calibration for optimal quantization quality
174
  -
175
- - Developed by GeekedOut - focused on intelligent quantization methods
 
 
 
 
 
 
 
 
176
 
177
 
178
 
 
94
 
95
 
96
 
97
+ ## The IQ2_M Intelligence Concept
98
 
99
 
100
 
101
 
102
 
103
+ GeekedOut Quantizer models are designed with intelligence as their primary capability. Through intelligent weight allocation, **intelligence** is preserved in critical parameters while less important weights are packed into minimal bit formats:
104
 
105
 
106
 
 
120
 
121
 
122
 
123
+ ## The Quantization Process
124
+
125
+
126
+
127
+
128
+
129
+ GeekedOut uses the A:\Geeked.Out software to create models that are intelligent through:
130
+
131
+
132
+
133
+
134
+
135
+ 1. **Intelligent calibration** - imatrix-based calibration for optimal quantization quality
136
+ 2.
137
+ 2. **Mixed-precision allocation** - critical parameters receive higher precision while less important weights receive minimal bit formats
138
+ 3.
139
+ 3. **Block-wise optimization** - optimized scaling factors applied across weight blocks
140
+ 4.
141
+ 4. **Smart allocation** - intelligence is preserved through intelligent weight distribution
142
+ 5.
143
+
144
+
145
+
146
+
147
+ ## IQ2_M Quantization Features
148
+
149
+
150
+
151
+
152
+
153
+ The **IQ2_M** (Intelligent Quants) quantization scheme features:
154
+
155
+
156
+
157
+
158
+
159
+ - The quantized models retain conversational capability while achieving significant size reduction
160
+ -
161
+ - Compatible with llama.cpp, LM Studio, Jan, and other local inference frameworks
162
+ -
163
+ - Uses imatrix-based calibration for optimal quantization quality
164
+ -
165
+ - Developed by GeekedOut - focused on intelligent quantization methods
166
+ -
167
+
168
+
169
+
170
+
171
+
172
+
173
  ## Supported Use Cases
174
 
175
 
 
182
 
183
 
184
 
185
+ - Conversational AI applications where intelligence is preserved through IQ2_M quantization
186
  -
187
  - Local inference with llama.cpp, LM Studio, Jan, and similar tools
188
  -
 
208
 
209
 
210
 
211
+ **Example:**
212
+
213
+ ```bash
214
+
215
+ # Load the IQ2_M quantized model using llama.cpp
216
+
217
+ llama.cpp -hf LGxNDs/IQ2_M-2Bit-Quantization-By-Geeked-Out-Ai
218
+
219
+ ```
220
+
221
+
222
+
223
 
224
 
225
  ## Technical Notes
 
234
  -
235
  - Uses imatrix-based calibration for optimal quantization quality
236
  -
237
+ - Developed by GeekedOut - focused on intelligent quantization methods using A:\Geeked.Out software
238
+
239
+
240
+
241
+
242
+
243
+
244
+
245
+
246
 
247
 
248