[FIX] Reasoning Loop on Trivial Task same as FP8
#7
by voves - opened
We ran into an issue where the model would get stuck in a looping pattern during generation(same as FP8 Quants). After debugging, we found that forcing clear_thinking to true by default in chat_template.jinja resolves it. (Tested on SGLang)
The fix, In chat_template.jinja, change:
{%- set clear_thinking = clear_thinking if clear_thinking is defined else false -%}
to:
{%- set clear_thinking = clear_thinking if clear_thinking is defined else true -%}
Hope this saves someone else the debugging time!🙏