froggeric

/

Qwen-Fixed-Chat-Templates

like 1.43k

Thank you for the report and suggestion on how to resolve it. I will look into it.

请问下所以使用froggeric/Qwen-Fixed-Chat-Templates是很有必要的对吗？然后后端需要增加xml的解析？

Thanks for starting this discussion. I hope I understood it correctly through the translation :-)

Qwen models are natively pre-trained on XML tool formatting (<function=...><parameter=...>). In testing across agentic benchmarks, XML tool calling produces significantly fewer escaping errors and better multi-parameter handling than raw JSON.

Because of this, the template uses canonical XML by default. However, if your harness strictly requires Hermes JSON (such as certain agent orchestrators), you can pass "tool_call_format": "json" in kwargs.

In the latest v22.2 release, the template also safely handles stringified JSON arguments when rendering assistant history, so upstream OpenAI API clients passing serialized argument strings will not crash or corrupt the format.

Thanks for starting this discussion. I hope I understood it correctly through the translation :-)

Qwen models are natively pre-trained on XML tool formatting (<function=...><parameter=...>). In testing across agentic benchmarks, XML tool calling produces significantly fewer escaping errors and better multi-parameter handling than raw JSON.

Because of this, the template uses canonical XML by default. However, if your harness strictly requires Hermes JSON (such as certain agent orchestrators), you can pass "tool_call_format": "json" in kwargs.

In the latest v22.2 release, the template also safely handles stringified JSON arguments when rendering assistant history, so upstream OpenAI API clients passing serialized argument strings will not crash or corrupt the format.

我的意思是，qwen3x系列模型，天然亲睐XML格式工具调用，同时更少的出错概率。
但是实际XML/JSON作为主流的两种工具调用格式，被众多助手分别使用。
而qwen3x系列模型的json格式工具调用是有更多的概率出现异常，包括且不限于、无法调用、调用后不执行、执行过程中异常停止、循环调用、调用参数出错。
为了尽可能降低json格式工具调用的出错概率，需要在chat-template模板中对工具调用特定情况进行后训练，用于让模型先学习调用的正确、错误集后加强模型能正确执行的倾向即可。

Hi @Zeaxion ,

Thank you for clarifying and sharing your testing data.

Few-shot examples (showing positive and negative tool call samples) definitely help guide the model toward correct JSON output. However, embedding hardcoded few-shot examples directly into the universal Jinja template creates a few trade-offs for a general-purpose repository:

Token overhead: Adding few-shot examples to the template increases the system prompt token count on every request, which raises time-to-first-token (TTFT) and inference cost for all users.

Context interference: Hardcoded tool examples can conflict with custom user tool schemas or domain-specific system prompts.

Separation of concerns: The template is designed to handle structural formatting and engine compatibility, while semantic few-shot steering is usually best handled at the application or agent prompt level.

That said, in the new v22.3 update, I added several improvements to make JSON tool calling safer and more reliable:

• Slicing limiters (max_tool_response_chars) are automatically bypassed when tool_call_format="json" is active so JSON payloads are never truncated into invalid syntax.
• Scalar and list arguments are cleanly serialised with standard JSON formatting across all formats.
• Serialised JSON strings in assistant history are parsed safely without syntax errors or mutating the KV cache.

If your harness requires JSON tool calling, passing tool_call_format: "json" in kwargs combined with a few-shot example in your agent's system prompt gives the best balance of reliability without forcing extra tokens onto everyone.

Hi @Zeaxion ,

Thank you for clarifying and sharing your testing data.

Few-shot examples (showing positive and negative tool call samples) definitely help guide the model toward correct JSON output. However, embedding hardcoded few-shot examples directly into the universal Jinja template creates a few trade-offs for a general-purpose repository:

Token overhead: Adding few-shot examples to the template increases the system prompt token count on every request, which raises time-to-first-token (TTFT) and inference cost for all users.

Context interference: Hardcoded tool examples can conflict with custom user tool schemas or domain-specific system prompts.

Separation of concerns: The template is designed to handle structural formatting and engine compatibility, while semantic few-shot steering is usually best handled at the application or agent prompt level.

That said, in the new v22.3 update, I added several improvements to make JSON tool calling safer and more reliable:

• Slicing limiters (max_tool_response_chars) are automatically bypassed when tool_call_format="json" is active so JSON payloads are never truncated into invalid syntax.
• Scalar and list arguments are cleanly serialised with standard JSON formatting across all formats.
• Serialised JSON strings in assistant history are parsed safely without syntax errors or mutating the KV cache.

If your harness requires JSON tool calling, passing tool_call_format: "json" in kwargs combined with a few-shot example in your agent's system prompt gives the best balance of reliability without forcing extra tokens onto everyone.

json 的问题是，模型本身就直接不输出任何内容，所以必须要后训练提示词才能正常工作。而且只有流式会话存在该情况，而非流式会话基本上趋近正常可用的水平。