Shockem/Qwen3.8-27b-Terse-Coder

Qwen3.8-27B Terse-Coder

A full fine-tune of Qwen/Qwen3.8-27B with ~1/10 the chain-of-thought reasoning tokens on coding tasks and correctness preserved. This is the round-7 Terse-Coder DPO adapter merged into the stock base — the same behavior as the adapter, baked into the weights, with nothing to load at runtime.

Actively researched and improving. This model is one artifact of an ongoing study; expect updates as the iteration continues.

Background

Qwen-3.8-27b is an excellent dense model. It's a breakthrough in locally hosted models on consumer-grade hardware. The biggest challenge it faces is that its reasoning can be, at times, overly verbose. This isn't necessarily an issue, as it can pull itself out of hallucinations, but with user hardware around this model size, it ends up causing long waits, oftentimes into hours, before any outputs or edits occur.

Recently, agentionai released Signal-3.8-27B. This opened the gates to the idea of lowering reasoning by finetuning the model, rather than handling it via templates or configuration. The outcome was a model that thought significantly less than the base model, with marginal differences in error. This sent me down a rabbit hole of testing its reasoning and how it affected the model's output. To my surprise, it was incredibly close to the base, even its traces were very close, just cleaner overall.

My experiment is to continue this research and push it further. So far, I've trained on traces from Signal, the base NVIDIA provided NVFP4 quant, and my own abliterated variant, against HumanEval, 600 questions per round. This is now round 7, and it has shown significant improvement. We are now at a 95% reduction in reasoning against the NVIDIA quant, and 52% against Signal. This was originally created as a LoRA adapter that is then merged into a custom recipe for Qwen-3.8-27b. I have provided both a merged model and a LoRA adapter. Thank you for testing and providing feedback!

Results

All testing was done on NVFP4 quantizations served by vLLM 0.28 on 3× RTX 5060 Ti. Held-out 40 coding problems (20 HumanEval + 20 MBPP-sanitized, disjoint from training), sampling temp 0.6 / top_k 20 / top_p 0.95 / rep-penalty 1.05, pass@1 by automated test execution in a sandboxed container.

Model (NVFP4 serving)	pass@1	Reasoning tokens / problem	Wall tok/s
nvidia/Qwen3.8-27B-NVFP4 (stock base)	72.5%	~701	54.1
This model (NVFP4 quant)	67.5%	~38 (−95%)	54.5

Independent benchmarks (NVFP4 quant, vLLM 0.28, thinking on, house sampling; reasoning = completion_tokens_details.reasoning_tokens):

Benchmark	Score	Reasoning tokens (mean / median)
GSM8K (n=200)	98.0%	84 / 72
GPQA-Diamond (full 198)	78.3%	1,485 / 969
CRUXEval-I (full 800, input prediction)	92.1%	197 / 83
CRUXEval-O (full 800, output prediction)	92.9%	146 / 96
HumanEval+ (164, official EvalPlus, greedy)	90.2% (93.9% base)	43 / 28
MBPP+ (378, official EvalPlus, greedy)	78.6% (92.9% base)	91 / 25

CRUXEval was run with the official Meta harness end-to-end (direct prompts, official answer extraction, exec-based scoring, temp 0.2) — it measures code understanding (predict what a function returns / find an input that produces an output), complementary to the generation-side coding table above.

The GSM8K result (out-of-distribution — math word problems were never in the training data) shows the terse behavior transfers beyond coding.

A note on GPQA-Diamond: this is the test where a terseness fine-tune is supposed to bleed — PhD-level science questions, maximally far from the coding distribution the adapter trained on, where long deliberation is the whole game. Holding 78.3% while spending ~1.5k reasoning tokens per question (where thinking models typically burn 10–20k) means the training cut the deliberation budget, not the capability: the model still modulates effort upward on hard problems (median 969 → max 16k tokens) instead of blindly answering fast. We consider this the headline result of the round.

Our internal 30-test agentic harness (easy/medium/hard ×10 runs each) scored 100/100/100 with zero truncations.

The NVFP4 quant runs at stock-base wall speed — the reasoning cut is free end-to-end, because both MTP speculative decoding acceptance (0.41) and decode throughput are preserved. Measured MTP acceptance: 0.412.

The same behavior is also available as a runtime adapter, Shockem/Qwen3.8-27b-Terse-Coder-LoRA, which stacks on other bases (Signal, heretic-ara, your own quant). Do not stack the adapter on this merged model — the preference is already baked in; double application over-shortens reasoning (63% pass with no_code failures in our testing).

Which should I use?

• This merged model — simplest path; nothing extra to load; the NVFP4 quant below is the tested deployment.
• The adapter — if you want the effect on a different base (Signal, heretic-ara, your own fine-tune), or want to toggle it per-request.

Variants

Repo	Format	Use
This repo	fp16 full weights (~52 GB)	Merge source / full-precision serving
Shockem/Qwen3.8-27b-Terse-Coder-NVFP4	modelopt W4A16 NVFP4 + FP8 attention + MTP draft stack	Tested deployment — all numbers above

Serving (vLLM, tested path)

Tested on vLLM 0.28 with MTP speculative decoding on 2× RTX 5060 Ti 16 GB:

vllm serve Shockem/Qwen3.8-27b-Terse-Coder-NVFP4 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --quantization modelopt_fp8 --kv-cache-dtype fp8

Turn MTP spec decode on. The checkpoint carries a full MTP draft stack (vocab-truncated 40960-id draft head); acceptance measured 0.41 and outputs are target-verified (lossless). If you serve with spec decode, make sure the generation config has no min_p — vLLM 0.28 rejects min_p under spec decode.

Recommended sampling (mirrors testing): temp 0.6, top_k 20, top_p 0.95, repetition_penalty 1.05.

On 2×16 GB cards cap context at ~200k with a ~3.9 GiB FP8 KV pin; single-card 24 GB+ rigs are unaffected.

Training (summary)

Round-7 DPO adapter merged into pristine Qwen/Qwen3.8-27B in fp32, stored fp16, then quantized with a NVIDIA-style recipe (modelopt 0.45 W4A16, FP8 attention, Local-Hessian-weighted calibration on MLP/lm_head — Hessian calibration is what preserves the full adapter effect through 4-bit quantization; absmax calibration attenuates it to roughly half).

• Data: coding prompts (HumanEval + MBPP-sanitized, disjoint from eval), 4 traces each at temp 1.0 from three policies (stock Qwen, Signal, heretic-ara), filtered by automated test execution; passing traces segmented into steps and pruned by per-step mean token entropy → 525 DPO pairs of (pruned reasoning, same answer) vs (full reasoning, same answer).
• Method: DPO (trl 1.13), β 0.05, lr 1e-5 cosine, 3 epochs, eff. batch 8, initialized from the round-6 adapter.

Caveats

• Targeted at coding tasks with thinking enabled. Behavioral edit, not a knowledge edit — general chat and math are not the training distribution.
• The preference is "shorter reasoning, identical answer" — if a task needs long derivation, raise reasoning_effort as usual.
• Should work fine alongside other quants, formats, and backends (SGLang, TabbyAPI/EXL3, llama.cpp), but only NVFP4 + vLLM has been measured — validate before relying on other combinations.

Attributions & licenses

This model is a derivative work of Qwen/Qwen3.8-27B, © Qwen Team, Alibaba Cloud, licensed Apache 2.0; this model remains Apache 2.0 and the original license and copyright notices are retained. Built standing on the shoulders of:

• Qwen Team (Alibaba Cloud) — the Qwen3.8-27B base model (Apache 2.0).
• NVIDIA — TensorRT Model Optimizer 0.45 (Apache 2.0) drove the NVFP4 quantization, and NVIDIA's published Qwen3.8-27B-NVFP4 checkpoint informed the Hessian-calibrated v3 recipe.
• agentionai — Signal-3.8-27B, one of the three trace-generation policies in the preference data.
• p-e-w — the Heretic tool; a heretic-ara abliterated variant of Signal was another trace-generation policy.
• OpenAI — HumanEval (MIT), and Google — MBPP (CC-BY 4.0): prompt sources for training and held-out evaluation.
• Hugging Face TRL (Apache 2.0) — the DPO trainer.
• Datacurve — DeepSWE, used for independent evaluation only.

None of these parties endorse this model; all remaining errors are ours.