Hugging Face's logo

froggeric
/
Qwen-Fixed-Chat-Templates

MLX
jinja
chat-template
qwen
qwen3.5
qwen3.6
qwen3.8
lm-studio
llama.cpp
vllm
tool-calling
thinking
Model card Files
xet
 Community
104

New release: v22.5 (non-thinking tool prompt alignment, video URL support, and engine hardening)

#103
by froggeric - opened
Discussion
froggeric
 Owner

Hey everyone,

Version 22.5 is now live!

This release resolves a few subtle edge cases reported in community discussions around non-reasoning agent workflows, multi-modal video payloads, and serving engine configurations.

What is new in v22.5:

  1. Non-Thinking Tool Prompt Coherence (#94)
    When thinking is disabled (enable_thinking=false or reasoning_effort="none"), the # Tools system prompt block now omits <think> formatting examples and planning directives. This aligns the prompt with the prefilled closed <think>\n\n</think>\n\n block, stopping the model from stalling or hallucinating closing tags when executing tools in non-reasoning mode. In standard thinking mode, prompt output remains 100% byte-identical to v22.4.

  2. Multi-Part Video URL Support (#94)
    Added 'video_url' in item detection to render_content alongside 'video', matching how 'image_url' works. This eliminates unhandled exceptions when clients pass OpenAI-compatible video dictionaries.

  3. Smart Response Truncation in JSON Mode (#94)
    Instead of completely bypassing max_tool_response_chars under tool_call_format="json", the template now inspects the payload structure. Structured JSON responses ({...}, [...]) and tool call arguments remain protected from syntax corruption, but long plain-text outputs (such as bash transcripts and error dumps) are safely truncated according to your character limit.

  4. SGLang & Qwen 3.8 Flash-Next Support (#101, #102)
    Added launch documentation for SGLang (--chat-template flag and YAML chat-template: syntax to override embedded tokenizer templates) and verified out-of-the-box compatibility for Qwen 3.8 Flash-Next.

  5. Recommended Sampling Parameters & Copilot Guidance (#86, #100)
    Added an official sampling parameters reference table to the README. Coding and agent tasks perform best at temperature: 0.6 with presence_penalty: 0.0, while general reasoning benefits from temperature: 1.0. Also documented setup for GitHub Copilot Desktop Canvas (tool_call_format="json" with --tool-call-parser hermes).

  6. Expanded Test Suite (102 -> 105 Tests)
    Added automated tests for non-thinking tool prompt coherence, video URL parsing, and smart JSON response truncation. All 105 tests and 500 property fuzzer cases are passing 100%.

As always, you only need the single chat_template.jinja file from the root of the repo. Thanks to everyone in the discussions for the reports and testing data!
sbsplus

It looks like the template doesn't work with Flash Next.

I tried it with omp (oh my pi harness, a fork of pi), and constantly it reports an error when it tries to read a file or a web page (and even when it tried to read this chat template):
✘ 400 Output of tool call should be 'Input text'
Output of tool call should be 'Input text' (type=invalid_request_error)
raw-http-request=C:\Users\xxx.omp\logs\http-400-requests\xyz.json

I think the problem could be that with this chat template the read tool returns compacted PNG images inside the json file instead of text, which causes an error. But frankly, I have no idea. Just reporting an issue...

The thing is that it all works without any issues with Qwen's original chat template for Qwen3.8 Flash Next (from their HF model download folder).
froggeric
 Owner

Hey @sbsplus ,

The error you ran into is an upstream bug in llama-server, not an issue with the chat template.

Specifically, in llama.cpp (tools/server/server-chat.cpp, lines 205-209), the /v1/responses endpoint has a strict validation check on tool call outputs:

if (!chatcmpl_output.contains("type") || chatcmpl_output.at("type") != "input_text") {
    throw std::invalid_argument("Output of tool call should be 'Input text'");
}

When your harness sends a tool response containing structured content parts using standard "type": "text" (or multi-modal parts like images), llama-server rejects the HTTP request with HTTP 400 before the chat template is ever loaded or rendered. This is a known issue tracked in llama.cpp (#23542, #23557).

The reason it appeared to work with the stock template is that the stock template's XML formatting was not picked up as a structured call. This caused your harness to send plain text instead of sending a multi-part tool output array back to /v1/responses.

To work around the llama-server bug, maybe you can try configuring your client to query the standard /v1/chat/completions endpoint instead of /v1/responses.

Sign up or log in to comment