mdrbal83/Qwen3.8-Sharp-Seeded-Chat-Template

Qwen3.8 Sharp Seeded chat template

A chat template for the Qwen3.8 thinking models that cuts reasoning tokens at the default xhigh effort by roughly half and removes the runaway-thinking tail, while keeping (in this suite, slightly improving) answer quality. It is a tuned revision of peculiar-ragdoll/Qwen-Sharp-Chat-Templates, which is itself froggeric/Qwen-Fixed-Chat-Templates v22.4.1 plus a terseness system prompt. Both upstreams are Apache-2.0; so is this.

The whole change versus upstream is one sentence in the system prompt and two words in the generation prompt — the <think> block is opened with Let me instead of being left empty.

Where it was tested. All local measurements were taken self-hosting Mia-AiLab/Qwen3.8-Flash-Next-NVFP4 on a single NVIDIA DGX Spark (GB10, 121 GiB unified memory) with vLLM, using the deployment kit MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-Spark — this file ships there as files/chat_template_sharp.jinja and is enabled with CHAT_TEMPLATE=sharp. The replication on the official Qwen3.8-27B ran against a third-party provider through OpenRouter, so the effect is not tied to that quant, that host, or that serving stack.

checkpoint	effort	reasoning tokens, mean	p90	runaways (>8k)	truncated / never closed </think>	objective score
Flash-Next NVFP4 (local vLLM)	xhigh	3368 → 1408 (−58%)	9788 → 2968	6/42 → 0	2 → 0	0.932 → 1.000
Flash-Next NVFP4 (local vLLM)	medium	1031 → 916 (−11%)	2035 → 2046	2/126 → 0	0 → 0	0.982 → 0.999
Flash-Next NVFP4 (local vLLM)	low	1286 → 773 (−40%)	1288 → 1627	2/42 → 0	1 → 0	0.893 → 0.974
Qwen3.8-27B (official weights, OpenRouter/Phala)	xhigh	3555 → 2003 (−44%)	10751 → 4210	7/42 → 2	3 → 0	0.921 → 0.974
Qwen3.8-27B, stress prompts	xhigh	14227 → 3638 (−74%)	16000 → 5164	11/12 → 0	8 → 0	0.333 → 0.922
Qwen3.8-27B (official weights, OpenRouter/Phala)	medium	1052 → 1181 (+12%)	2217 → 2047	0 → 0	0 → 0	0.981 → 0.954

Every number in this card comes from a run whose raw JSON (prompts, full reasoning traces, answers, scores) is in results/. Nothing is estimated.

Why it works: the "We need answer" mode

Across every medium-effort run in this repository (906 traces on Flash-Next), 33 reasoning traces ran past 8,000 tokens. 31 of them open with the words "We need answer" — a clipped, note-taking style ("Need X. Could Y. Potential issue: Z. Good.") that spirals into dozens of consecutive "Potential issue:" paragraphs, arguing with an imagined grader about unstated requirements, and frequently never emits </think> at all. Traces that open "The user wants…" or "Let me…" of their own accord — a narrative style — ran away 0 times in 227 samples, at the same median length. (The one narrative-opener runaway was a 9.6k-token trace under the rejected The user seed; it terminated on its own.)

This is not a quirk of one quant. At xhigh effort 42 of 42 traces open "We need answer" on the local Flash-Next NVFP4 checkpoint, 42 of 42 on the official Qwen3.8-27B served by a third party, and 9 of 9 on Alibaba's closed qwen3.8-flash API. It is the trained xhigh behaviour of the Qwen3.8 line. The Flash-Next checkpoint also enters it ~40% of the time at medium; the 27B does not.

Seeding the think block with Let me makes the narrative style the only continuation the model can take. Every trace under the seeded template opens "Let me think through…" / "Let me work through…", and the runaway count across 222 seeded samples on Flash-Next (all three efforts plus the stress prompts) is zero, versus 10 in 210 for the upstream template.

What did not work

Six attempts to fix the same thing with instructions all made it worse, several dramatically. This is the part most worth reading if you edit Qwen templates yourself; details and numbers are in CHANGELOG.md.

version	edit	result
v1	"…do not revisit settled points… verify only the steps where an error is plausible, then commit."	looked like −26% at n=2; at n=6 on the risky prompts: 6/12 runaways vs 1/12 — "verify where plausible" became 159 consecutive "Potential issue:" paragraphs
v2	+ "do not speculate about unstated requirements, hidden tests, or how the answer will be judged"	naming the failure mode primed it: traces filled with "Potential hidden tests might expect…"; 3.06× mean, 3 truncations
v3	"the request as written is the complete specification" (positive wording)	same runaways; "specification" invites auditing the spec for gaps
v4	fill the checkpoint's empty medium effort slot with a sentence	untrained text in the trained slot: 3.29× mean, bug-review trace went 500 → 12k tokens
v5	termination-oriented rule, no verification language	median win at n=2 was noise; at n=6 no better than baseline
v6	"Think freely, then answer directly" + answer-scoped rules	8/12 runaways, worst of all
v7	seed The user	tail gone, but the model restates the prompt: math CoT +37%, score 0.950
v8	seed Let me	tail gone, accuracy kept
v9	v5 rule + seed Let me	best mean at every effort
v10	v9, but no seed on tool-continuation turns	after a tool result v9 was clean 7/10 (1 re-call, 2 empty turns); v10 10/10

Rules that fell out of this: never name the behaviour you want suppressed; never refer to the spec, tests, or grader as objects; never mention verification or errors in thinking guidance; do not put untrained text in the trained Reasoning effort is set to … slot; and never judge a template change on n=2 — judge on the median, the runaway count, and a 6-rep stress on prompts that are slightly under-specified.

What exactly changed

Diff against upstream v0 (versions/v0.jinja, byte-identical to the peculiar-ragdoll file):

1. In the terseness block, when thinking is on, one added sentence:

   While thinking: work the problem once, in order, making each decision the first time you meet it and building on it. Once the approach is settled, write the answer in the form the request asked for.

2. The generation prompt ends <think>\nLet me instead of <think>\n — unless the last message has role: tool, in which case it stays <think>\n (after a <tool_response>, Let me primed "…call the tool again" or an empty turn).

Everything else — effort steering phrases, high→xhigh alias, terse kwarg, tool-call formats, vision handling, the <|think_*|> in-content overrides — is upstream's, unchanged.

The seed text is part of the prompt, so it is not returned in the reasoning field by vLLM's qwen3 reasoning parser; the returned reasoning simply begins mid-sentence (" think through the problem…").

Costs and caveats

• Effort-dependent. The seed pays exactly where the model would otherwise open in the clipped style. At xhigh that is always. At medium it is ~40% of the time on Flash-Next (net −11%) and never on the 27B (net +12%, one more wrong answer in 42). If you run a checkpoint at medium that does not show the "We need answer" opener, the seed is neutral.
• Easy problems get longer. On short maths the median CoT rises 20–50%: the model narrates instead of jotting. Net token spend is still lower at every effort on Flash-Next because the tail dominates the mean.
• Answers are ~8% longer under the seeded template.
• Not a standard benchmark. The suite is 21 prompts written for this work (below), scored objectively. n = 42–126 per cell. One host, one quant, temperature 1.0. The 27B replication is 42 + 12 samples per cell on one third-party provider.
• Upstream's own caveat still applies: this template's default effort is medium (no steering line) where the checkpoint's own template defaults to xhigh; pass reasoning_effort explicitly if you depend on that.

Methodology

Everything is reproducible from bench/.

Suite (bench/tasks.py): 21 prompts, every objective task has ground truth that was brute-forced or unit-tested before any model saw it.

kind	n	scoring
maths (combinatorics, number theory)	9	exact integer; ground truth brute-forced in Python
logic (constraint puzzles)	2	exact answer; unique solution verified by enumeration
code (merge intervals, expression evaluator, topo sort, LRU cache, word ladder, duration parser, regex ./* matcher, minimum window)	8	hidden unit tests run in a subprocess, incl. performance tests that fail O(n²) shortcuts; score = fraction passed
planted-bug code review	1	4 planted bugs; LLM judge decides which were found
prose explanation	1	LLM judge, pairwise vs baseline, both presentation orders

Runs: temperature 1.0, max_tokens 16000 (24000 for the local xhigh cells). Reasoning length = usage.completion_tokens_details.reasoning_tokens from vLLM. "Runaway" = more than 8,000 reasoning tokens. "Stress" = 6 reps of the two prompts that provoke runaways (c2 expression evaluator — leaves division-by-zero unspecified; c5 word ladder).

A/B design: one vLLM launch with --trust-request-chat-template; every version is passed per request via the chat_template field, so all versions hit identical weights, kernels, sampling and server state. bench/bench.py --template versions/v10.jinja --label v10 --effort xhigh.

Replication on the 27B: bench/bench_or.py renders the template locally with jinja2 and sends the rendered string as a raw prompt to OpenRouter's /completions, pinned to the Phala provider (which honours raw prompts; Novita rejects /completions, Io Net re-wraps the prompt). Reasoning is the text before </think>, counted with the local tokenizer.

Tool calls: bench/tooltest.py, 10 reps: does the model emit a well-formed tool_call, and after the tool result does it answer cleanly (no re-call, non-empty)? v0 10/10 and 10/10; v9 10/10 and 7/10; v10 10/10 and 10/10.

Local host: NVIDIA DGX Spark (GB10, 121 GiB unified memory), vLLM qwen38-flash-next image, Mia-AiLab/Qwen3.8-Flash-Next-NVFP4, deployment kit MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-Spark (CHAT_TEMPLATE=sharp there is this file).

Full results index

results/flash-next/ — local vLLM, Mia-AiLab/Qwen3.8-Flash-Next-NVFP4

file	template	effort	samples	note
v0_{low,medium,xhigh}.json	v0 (upstream)	low / medium / xhigh	42 / 126 / 42	baseline
v1_medium.json … v4_medium.json	v1–v4	medium	42 each	rejected
v5_medium.json, v7_medium.json	v5, v7	medium	126 each	rejected
v8_{low,medium,xhigh}.json	v8	low / medium / xhigh	42 / 126 / 42	seed only
v9_{low,medium,xhigh}.json	v9 (= v10 on non-tool turns)	low / medium / xhigh	42 / 126 / 42	installed behaviour
stress_v{0,1,5,6,7,8,9}_medium.json	—	medium	12–18	c2/c5 ×6
stress_v0_noterse_medium.json	v0, terse:false	medium	12	control: no system prompt at all
tooltest*.json	v0/v8/v9/v10	medium	3–10 per version	tool round trip

results/qwen3.8-27b/ — qwen/qwen3.8-27b via OpenRouter → Phala, raw completions

file	template	effort	samples
q27_{v0,v10}_{medium,xhigh}.json	v0 / v10	medium / xhigh	42 each
q27_stress_{v0,v10}_{medium,xhigh}.json	v0 / v10	medium / xhigh	12 each

results/qwen3.8-flash-api/probe.md — Alibaba's closed qwen3.8-flash through the chat API, 9 samples (cannot be seeded; documents that the mode is present there too).

Each JSON row has id, kind, rep, elapsed, prompt_tokens, completion_tokens, reasoning_tokens, answer_tokens, finish, score, content, and the full reasoning trace.

Usage

vLLM

vllm serve <model> --chat-template chat_template.jinja --reasoning-parser qwen3 ...

Per request: "reasoning_effort": "low" | "medium" | "high" | "xhigh" (high is an alias for xhigh). Opt out of the terseness block with "chat_template_kwargs": {"terse": false}; the seed stays. Disable thinking with "chat_template_kwargs": {"enable_thinking": false}.

LM Studio / llama.cpp: paste the file as the model's chat template. The <|think_low|> / <|think_xhigh|> / <|think_off|> in-content overrides from upstream still work as documented there; note they are template-level string triggers, not tokenizer specials.

Other Qwen3.x thinking models: untested. The seed is safe (it is two words of ordinary text) but the benefit depends on whether the model opens in the clipped style at your effort level. Check five traces before assuming.

Attribution

• Template base: froggeric/Qwen-Fixed-Chat-Templates v22.4.1 (Apache-2.0).
• Terseness block and the Sharp variant: peculiar-ragdoll/Qwen-Sharp-Chat-Templates (Apache-2.0). versions/v0.jinja is that file verbatim.
• Tuning, suite, measurements and this card: mdrbal83, September 2026, on the DGX Spark deployment kit MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-Spark.
• Checkpoint used locally: Mia-AiLab/Qwen3.8-Flash-Next-NVFP4.

If you find a Qwen3.x checkpoint where the seed hurts, or a seed text that does better than Let me, open a discussion — the suite makes it a 30-minute check.