Looping behavior
#5
by vladimir94 - opened
Amazing model! My first tests went well and I am happy to make it my daily driver.
I have noticed that with NVFP4 version, the model tends to go into loops where it will repeat the same thing, call the same tools, read the same file forever, etc.
How to stop this behavior without otherwise hurting the quality of the output?
On NVFP4 Qwen 35b, I would increase repeat/presence penality (forgot which) to 1.1, but the disadvantage is it would discourage otherwise valid words in code, simply because they are repeated
Even a stop gap would be greatly appreciated!
Edit: It seems that enabling thinking and preserve-thinking in vllm solved it for me. After that, the model thought for 90 minutes 😂