froggeric

/

Qwen-Fixed-Chat-Templates

like 1.23k

Long-haul trucker here, running Qwen3.6-27B on a 3×RTX3090 box for a 28-year Bible translation project. Been on your v19 for months. Tonight I finally sat down and audited every chat template I had on disk. Sharing the results because I think they matter beyond my setup.

I rendered five templates against identical message histories and diffed the output. Same weights, same prompts — only the template changed.

The agentic-loop poisoning is worse than I realised.

Four tool calls with empty reasoning_content — how many empty <think>\n\n</think> blocks end up in the rendered history:

The mechanism, seen in the actual rendered prompt:

<|im_start|>assistant\n<think>\n\n</think>\n\n<tool_call>...
<|im_start|>user\n<tool_response>\nfile0.ts\n</tool_response>
<|im_start|>assistant\n<think>\n\n</think>\n\n<tool_call>...

The model sees three consecutive "thought nothing → called a tool" turns. Then generation prepends <think>\n — think. The pattern it just read says the opposite. That conflict resolves as either an empty think or a loop. Your v19 changelog called it "Empty Think poisoning" and I now see exactly what you meant.

The finetune situation is genuinely absurd.

One "ThinkingCap" finetune I downloaded ships a chat template that is byte-identical to the stock Qwen one — same md5, diff returns nothing. A model named after reasoning control, shipping the template that poisons reasoning. They didn't touch it.

The Unsloth INT4 fixed all three fatal raise_exception crashes but missed the poisoning line. An FP8 finetune fixed the poisoning but switched tool calls to JSON — which silently breaks vLLM's qwen3_xml parser with no error at all, the agent just stalls.

So: four different "behaviours" of the same model, none of them from the weights. People benchmark these, write "this finetune loops" or "FP8 is worse than INT4", and they are measuring a jinja file nobody opened.

Prefix cache — the one I hadn't thought about.

Growing a conversation turn by turn and checking whether the rendered prefix stays byte-stable:

Qwen official mutates
ThinkingCap mutates
Unsloth INT4 mutates
FP8 mutates
yours v21.3 stable

All the stock ones strip earlier <think> blocks once a new user message arrives, so already-cached text re-renders differently. On local inference that quietly destroys prefix cache hits every single turn. Your preserve_thinking default isn't a nicety — it's the difference between a cache that works and one that doesn't.

One thing v21.3 still trips on, FWIW.

The 80-char narrowing helps a lot, but during a codebase audit grep output like this still false-positives:

src/lib/api.ts:42: throw new Error('failed to fetch');

failed to lands at char 40. Two of those in a row and the agent gets told its approach is wrong and has thinking forcibly disabled — while it was doing exactly what it was asked to do. Not sure there's a clean fix short of checking whether the tool is a search tool. Just flagging it in case it's useful.

Thank you for this work. You are stitching up wounds left by surgeons who walked out without washing their hands, and you're doing it for free, in public, while the people shipping these models can't be bothered to open the file they're shipping.

For anyone reading this later: point your engine at one template with --chat-template and stop trusting whatever ships beside the weights. It's the only way to compare weights instead of comparing somebody else's plumbing.

Respect from the road.
God bless.