parameters guide
samplers guide
model generation
role play settings
quant selection
arm quants
iq quants vs q quants
optimal model setting
gibberish fixes
coherence
instructing following
quality generation
chat settings
quality settings
llamacpp server
llamacpp
lmstudio
sillytavern
koboldcpp
backyard
ollama
model generation steering
steering
model generation fixes
text generation webui
ggufs
exl2
full precision
quants
imatrix
neo imatrix
Update README.md
Browse files
README.md
CHANGED
@@ -172,23 +172,41 @@ Benchmarking-and-Guiding-Adaptive-Sampling-Decoding https://github.com/ZhouYuxua
|
|
172 |
|
173 |
CRITICAL NOTES:
|
174 |
|
175 |
-
Some of the models at my repo are custom designed / limited use case models. For some of these models, specific settings and/or samplers (including advanced) are
|
176 |
-
recommended for best operation.
|
177 |
|
178 |
As a result I have classified the models as class 1, class 2, class 3 and class 4.
|
179 |
|
|
|
|
|
180 |
Generally all models (mine and other repos) fall under class 1 or class 2 and can be used when just about any sampler(s) / parameter(s) and advanced sampler(s).
|
181 |
|
182 |
Class 3 requires a little more adjustment because these models run closer to the ragged edge of stability. The settings for these will help control them better, especially
|
183 |
for chat / role play and/or other use case(s). Generally speaking, this helps them behave better overall.
|
184 |
|
185 |
-
Class 4 are balanced on the very edge of stability. These models are generally highly creative, for very narrow use case(s), and closer to "human prose" than other models
|
186 |
-
are used to "bring these bad boys" inline which is especially important for chat and/or role play type use cases AND/OR use case(s) these models were not designed for.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
187 |
|
188 |
The goal here is to use parameters to raise/lower the power of the model and samplers to "prune" (and/or in some cases enhance) operation.
|
189 |
|
190 |
With that being said, generation "examples" (at my repo) are created using the "Primary Testing Parameters" (top of this document) settings regardless of the "class" of the model AND NO advanced settings, or samplers.
|
191 |
|
|
|
|
|
|
|
|
|
192 |
---
|
193 |
|
194 |
QUANTS:
|
|
|
172 |
|
173 |
CRITICAL NOTES:
|
174 |
|
175 |
+
Some of the models at my repo are custom designed / limited use case models. For some of these models, specific settings and/or samplers (including advanced) are recommended for best operation.
|
|
|
176 |
|
177 |
As a result I have classified the models as class 1, class 2, class 3 and class 4.
|
178 |
|
179 |
+
Each model is "classed" on the model card itself for each model.
|
180 |
+
|
181 |
Generally all models (mine and other repos) fall under class 1 or class 2 and can be used when just about any sampler(s) / parameter(s) and advanced sampler(s).
|
182 |
|
183 |
Class 3 requires a little more adjustment because these models run closer to the ragged edge of stability. The settings for these will help control them better, especially
|
184 |
for chat / role play and/or other use case(s). Generally speaking, this helps them behave better overall.
|
185 |
|
186 |
+
Class 4 are balanced on the very edge of stability. These models are generally highly creative, for very narrow use case(s), and closer to "human prose" than other models and/or
|
187 |
+
operate in ways no other model(s) operate offering unique generational abilities. With these models, advanced samplers are used to "bring these bad boys" inline which is especially important for chat and/or role play type use cases AND/OR use case(s) these models were not designed for.
|
188 |
+
|
189 |
+
For reference here are some Class 3/4 models:
|
190 |
+
|
191 |
+
[ https://huggingface.co/DavidAU/L3-Stheno-Maid-Blackroot-Grand-HORROR-16B-GGUF ]
|
192 |
+
|
193 |
+
[ https://huggingface.co/DavidAU/L3-DARKEST-PLANET-16.5B-GGUF ]
|
194 |
+
|
195 |
+
[ https://huggingface.co/DavidAU/MN-DARKEST-UNIVERSE-29B-GGUF ]
|
196 |
+
|
197 |
+
[ https://huggingface.co/DavidAU/MN-GRAND-Gutenberg-Lyra4-Lyra-23.5B-GGUF ]
|
198 |
+
|
199 |
+
Although Class 3 and Class 4 models will work when used within their specific use case(s), standard parameters and settings on the model card, I recognize that users want either a smoother experience
|
200 |
+
and/or want to use these models for other than intended use case(s) and that is in part why I created this document.
|
201 |
|
202 |
The goal here is to use parameters to raise/lower the power of the model and samplers to "prune" (and/or in some cases enhance) operation.
|
203 |
|
204 |
With that being said, generation "examples" (at my repo) are created using the "Primary Testing Parameters" (top of this document) settings regardless of the "class" of the model AND NO advanced settings, or samplers.
|
205 |
|
206 |
+
However, for ANY model regardless of "class" or if it is at my repo, you can now take performance to the next level with the information contained in this document.
|
207 |
+
|
208 |
+
Side note: There are no class 5 models published... yet.
|
209 |
+
|
210 |
---
|
211 |
|
212 |
QUANTS:
|