Use a single newline between consecutive tool calls (matches official Qwen templates)

#87
by gdevenyi - opened

Summary

Between consecutive tool calls in one assistant turn, the template emits \n\n<tool_call>. Every official Qwen template (Qwen3.5-27B, Qwen3.6-27B, Qwen3.8-27B, Qwen3.8-2.4T-A95B) emits \n<tool_call> for the second and later calls (single newline; the \n\n after assistant text before the first call is unchanged and matches the official templates). The model is trained on and generates the single newline.

Effect: when a parallel-tool-call turn is replayed as history, the rendered bytes differ from the tokens the model generated. On vLLM and llama.cpp the prefix cache misses from that turn on, and the history is slightly off-distribution. This is the same class of issue as #42 (extra newline after tool calls) and #80 (empty think block dropped from history).

Changes

  • chat_template.jinja: XML branch emits '\n<tool_call>\n<function=...' for non-first calls; JSON branch emits '\n' instead of '\n\n' between calls (the '\n\n' after non-empty content before the first call is kept in both branches).
  • chat_template_oneline.txt: regenerated with scripts/minify_jinja.py.
  • scripts/test_v22.py (test 66) and scripts/test_v21.py ("parallel tools delimiter"): expectations updated to the single newline.

Verification

  • python3 scripts/test_v22.py → 100/100, python3 scripts/test_v21.py → 9/9, python3 scripts/fuzz_template.py --cases 300 → clean.
  • Rendered through the transformers Jinja environment (what vLLM uses) and compared byte-for-byte with the official Qwen3.8-27B template on a two-call turn: only this separator differed before the patch.

The \n\n dates from v20 (commit 00aa57a); I found no documented reason for it. If it was deliberate, happy to hear why.

Merged, thanks Gabriel! Good catch on the whitespace difference vs the stock template. Keeps tokenization consistent with what the model actually outputs on parallel calls.

froggeric changed pull request status to closed

Sign up or log in to comment