Sampler Question
Sampler choice used to be highly opinionated but I haven't seen a lot of people listing what parameters they think work best with gemma-4. I eventually found the generator config in the base model that said temp 1, top_k 64, and top_p 0.95 but usually those values are more on the conservative side and not designed for roleplay. Do you have any preferred samplers you use or recommend?
Sampler choice used to be highly opinionated but I haven't seen a lot of people listing what parameters they think work best with gemma-4. I eventually found the generator config in the base model that said temp 1, top_k 64, and top_p 0.95 but usually those values are more on the conservative side and not designed for roleplay. Do you have any preferred samplers you use or recommend?
Defaults should be fine for roleplay. I’m unsure if there’s a perfect combination, so you can tinker around and find what you enjoy.
Try not to raise the temperature too high, as this can cause incoherence.
Early tests on other models the Temperature going above 0.9 i was having wonky results (though that was a lot of 4B-8B models typically Q8_0); While today temperature between 0.75-0.85 seems quite good. Repetition penalty of 0.15 to avoid getting stuck, and that's about it for me. Otherwise defaults tend to work pretty well.
I also got a logit bias against certain words being used, especially against ozone...
Though to be honest, the Gemma4 series since it came out has been quite good, and it's MOE nature (26B) also makes it very fast. I do enjoy the 70B outputs (that i was using before) but it's really hard to argue with a 10x speedup that's competent; And while the Gemma4 12B and 31B models just chug... they might be good too, but tend to be too slow for me at present.
Wish the hardware thing will resolve itself price-wise but it looks like it may be too expensive to get something else til after 2030... Or if another company comes in selling bulk cheap ram and GPU's that are worth perusing....