The model gets stuck in an infinite 'thinking' loop during roleplay conversations
Hi! I really appreciate your work—amazing job. The Gemma 4 26B model seems to get stuck in a 'thinking' loop during roleplay conversations when using my system prompt. I tested the Gemma 4 31B model, and it works perfectly with no looping at all.
During our evaluation of google/gemma-4-26B-A4B-it using benign prompts, we also encountered repetitive looping behavior. This appears to be intrinsic to the base model rather than being significantly affected by the ablation process.
One of the ways I typically deal with undesirable operation that was potentially induced (or made worse) by quantization is to bump any expert ‘scale’ factor from whatever the default is to +5-10%, so if the moe scale factor was 2.5 I would try 2.7. This helps to keep the expert’s “findings” as a strong signal against other probable tokens - improving their SNR. In Gemma 4 I don’t see any equivalent expert-based parameter in the GGUF so I typically run the quantized model about 5% higher temperature than the creator’s default (e.g. temp of 1.05 or 1.1) which tends to help. But as others have mentioned, sometimes the looping is not really related to quantization noise and is just a bug or flaw in the base.
I tried other temperature values (1.1 and 1.2) , the loop still appear.
the original model is still stuck in a loop ,too