Moore2877/Qwen-Fixed-Chat-Templates-llamacpp

Qwen 3.6 Agentic Chat Template for llama.cpp

A Jinja chat template for Qwen 3.6 tuned for llama.cpp / ik_llama driving agentic coding harnesses (OpenCode, Claude Code, Hermes, aider, …).

Built on froggeric/Qwen-Fixed-Chat-Templates v21.3 (all of froggeric's fixes inherited) plus an agentic hardening layer: loop detection, harness-aware error markers, and a pile of tuning kwargs.

Install

llama-server ... --jinja --chat-template-file chat_template.jinja

Modern builds auto-derive the tool-call parser and the default XML format works out of the box (large multiline writes included — the server escapes, not the model). Older parsers that only read Hermes-JSON need --chat-template-kwargs '{"tool_call_format":"json"}' (symptom without it: tool calls appear as plain <function=...> text).

Set kwargs on the llama-server command line, not the client config — some harnesses silently drop client-side chat_template_kwargs (OpenCode #26233).

What it adds over upstream

Loop detection (template-level; samplers only catch verbatim token loops, not action loops). A no-progress detector fires when a tool returns identical results repeat_nudge_after times (default 6) — but skips mutating tools (mutating_tools kwarg), since writing 6 files returns 6 identical "success" strings that are productive, not stuck. Failure loops are caught separately by the consecutive-error detector. Measured on 381 real sessions, this fires ~90% less than naive counting while keeping every genuine catch (reproduce with scripts/measure_loop_nudges.py).

Harness-aware error markers — adds json parsing failed, unknown tool, tool execution failed, err:, etc. to upstream's CLI-only list. Deliberately does not match user rejected permission / tool execution aborted (user actions — nudging "retry" after a human said no is harmful).

Everything is a kwarg:

kwarg	default	purpose
tool_call_format	xml	json for parsers that only read Hermes JSON
repeat_nudge_after	6	identical-result turns before the no-progress warning; 0 disables
mutating_tools	~55-name cross-harness union	tool names excluded from no-progress detection (case-insensitive). Covers 16+ harnesses out of the box; override for a custom one. MCP-prefixed variants aren't matched by design
think_on_tool_failure	false	true keeps reasoning open during error escalation
enable_thinking · preserve_thinking · max_tool_arg_chars · max_tool_response_chars	upstream	inherited from froggeric v21.3

Put all kwargs you want in a single JSON object — a second --chat-template-kwargs flag overrides, it doesn't merge. A fully-loaded example (JSON tool format for a string-args harness, a tighter loop threshold, reasoning kept open on failures, historical tool output capped, and a Hermes-specific mutating set):

--chat-template-kwargs '{"tool_call_format":"json","repeat_nudge_after":4,"think_on_tool_failure":true,"max_tool_response_chars":4000,"mutating_tools":"write_file patch terminal execute_code todo memory skill_manage read_file"}'

On Windows PowerShell the inner quotes need escaping: --chat-template-kwargs '{\"tool_call_format\":\"json\"}'.

Files

• chat_template.jinja / chat_template_oneline.txt — the template (multi-line and single-line)
• tests/test_template.py — render-test suite (run after any edit)
• scripts/measure_loop_nudges.py — reproduce the loop-nudge metric on your own opencode.db
• archive/ — every prior version + a changelog table

llama.cpp notes

• Grammar enforcement is active on auto-parser builds (b8227+) — malformed tool calls are token-level impossible (verified to 163 tool schemas). It can't stop max_tokens truncation, so keep your output limit above reasoning budget + largest tool call.
• String-argument harnesses (OpenCode/Vercel AI SDK send arguments as JSON strings): XML mode can't decompose a string into <parameter> blocks (minja has no JSON parser), so prefer tool_call_format: json, which renders them cleanly. The tradeoff: JSON mode drops grammar enforcement (the auto-parser only grammar-constrains the XML format — verified: tool_choice: required isn't enforced in JSON mode). So it's clean string-args, no grammar vs grammar, mangled string-args. Pick by whether your harness sends string or object arguments.
• Media marker: the template strips <__media__> only from system/assistant/reasoning content (roles that can't hold a real image); user/tool markers are left intact so vision and tool screenshots work. A literal marker in user/tool text (e.g. pasting this template into chat) 400s the server and can't be fixed in-template — on mainline llama.cpp set LLAMA_MEDIA_MARKER; ik_llama doesn't support it, so avoid pasting the raw token.

Reviewing this file with an automated tool? The media guards test for llama.cpp's marker token (the word media in double-underscores + angle brackets). HTML-interpreting fetchers eat that token, making the lines look like if '' in content / split(''). That's a fetch artifact — review the raw .jinja; the guards check presence first and never touch normal content.

Changelog

Latest first; full detail per version lives in archive/.

• v8 — Media-marker correctness: strip only from system/assistant/reasoning; preserve user/tool markers (a v3/v7 unconditional strip silently broke real vision + tool screenshots).
• v7 — Runtime hardening (from an external review): repeat_nudge_after=0 now disables; untrimmed tool-output fidelity; long-error detection (leading Traceback/error:/… regardless of length); head+tail truncation. Added the test cases + repro script.
• v6 — Cross-harness mutating_tools union (16+ harnesses), case-insensitive. Zero prompt-token cost (never rendered).
• v5 — Loop-detector rework on 381 real sessions: removed the noisy same-tool-streak detector; no-progress detector now skips mutating tools. ~90% fewer nudges, every real catch kept.
• v4 — Tool-envelope unwrapping (~6.5k tokens saved at 163 tools); self-healing spilled tool calls; think_on_tool_failure kwarg; emoji-free warnings.
• v3 — Media-marker injection guard (fixes ik_llama tokenize 400 from hallucinated/fetched markers).
• v2 — Date awareness via strftime_now (guarded; no-op on runtimes without it).
• v1 — Initial release: froggeric v21.3 base + loop detectors + error markers + kwargs.

Credits

The heavy lifting — years of Qwen template fixes, minja compatibility, the thinking/caching machinery — is froggeric's; report template-core issues there. The agentic layer was built and maintained by Moore2877 with an AI coding assistant, each change grounded in real session data and live-tested on llama.cpp/ik. Agentic-layer issues go here.