vwdubb/Qwen3.8-27b-Terse-Coder-FP8

Qwen3.8-27B Terse-Coder

A full fine-tune of Qwen/Qwen3.8-27B with ~1/10 the chain-of-thought reasoning tokens on coding tasks and correctness preserved. This is the round-8 (checkpoint-salvage) Terse-Coder DPO adapter merged into the stock base — the same behavior as the adapter, baked into the weights, with nothing to load at runtime.

Final release. This is the last round of the Terse-Coder study. The adapter lineage (and archived earlier rounds) lives at Shockem/Qwen3.8-27b-Terse-Coder-LoRA.

Background

Qwen-3.8-27b is an excellent dense model. It's a breakthrough in locally hosted models on consumer-grade hardware. The biggest challenge it faces is that its reasoning can be, at times, overly verbose. This isn't necessarily an issue, as it can pull itself out of hallucinations, but with user hardware around this model size, it ends up causing long waits, oftentimes into hours, before any outputs or edits occur.

Recently, agentionai released Signal-3.8-27B. This opened the gates to the idea of lowering reasoning by finetuning the model, rather than handling it via templates or configuration. The outcome was a model that thought significantly less than the base model, with marginal differences in error. This sent me down a rabbit hole of testing its reasoning and how it affected the model's output. To my surprise, it was incredibly close to the base, even its traces were very close, just cleaner overall.

My experiment is to continue this research and push it further. So far, I've trained on traces from Signal, the base NVIDIA provided NVFP4 quant, and my own abliterated variant, against HumanEval, 600 questions per round. This is now round 7, and it has shown significant improvement. We are now at a 95% reduction in reasoning against the NVIDIA quant, and 52% against Signal. This was originally created as a LoRA adapter that is then merged into a custom recipe for Qwen-3.8-27b. I have provided both a merged model and a LoRA adapter. Thank you for testing and providing feedback!

Results

Independent benchmarks (NVFP4 quant, vLLM 0.28, thinking on, house sampling; reasoning = completion_tokens_details.reasoning_tokens):

Benchmark	Score	Reasoning tokens (mean / median)
GSM8K (n=200)	98.5%	93 / 77
GPQA-Diamond (full 198)	79.8%	1,685 / 884
HumanEval+ (164, official EvalPlus, greedy)	91.5% (93.9% base)	37 / 28
MBPP+ (378, official EvalPlus, greedy)	79.4% (92.9% base)	88 / 25

Every external benchmark improved over the round-7 merge (98.0 / 78.3 / 90.2 / 78.6 respectively). CRUXEval was not re-run for round 8 (the round-7 merge measured 92.1% I / 92.9% O on the full 800).

Our internal 30-test agentic harness (easy/medium/hard) across 3 runs scored 4/4, 9/9, and 14/14 ×2 + 13/14 ×1 — the single miss is a known LRU-codegen flake that also appears in the round-7 profile.

A note on GPQA-Diamond: this is the test where a terseness fine-tune is supposed to bleed — PhD-level science questions, maximally far from the coding distribution the adapter trained on, where long deliberation is the whole game. Holding 79.8% while spending ~1.7k reasoning tokens per question (where thinking models typically burn 10–20k) means the training cut the deliberation budget, not the capability: the model still modulates effort upward on hard problems (median 884 → max 16k tokens) instead of blindly answering fast. We consider this the headline result of the round.

Transparency note — internal held-out probe. Our internal 40-problem held-out coding set (disjoint from training; 20 HumanEval + 20 MBPP-sanitized, temp 0.6, automated test execution) measures 61% on this merged artifact across two runs (mean ~38 reasoning tokens). The same round-8 adapter served as a runtime LoRA measures 70% on that probe, and the round-7 merge measured 67.5% — a small merge-attenuation tax specific to the 4-bit merged form (documented on the adapter repo). Every external benchmark above was measured on this exact merged artifact and improved; the full-strength numbers live with the adapter.

The NVFP4 quant runs at stock-base wall speed — the reasoning cut is free end-to-end, because both MTP speculative decoding acceptance and decode throughput are preserved (identical draft stack to round 7, acceptance measured 0.41).

The same behavior is also available as a runtime adapter, Shockem/Qwen3.8-27b-Terse-Coder-LoRA, which stacks on other bases (Signal, heretic-ara, your own quant). Do not stack the adapter on this merged model — the preference is already baked in; double application over-shortens reasoning (63% pass with no_code failures in our testing).

Which should I use?

• This merged model — simplest path; nothing extra to load; the NVFP4 quant below is the tested deployment.
• The adapter — if you want the effect on a different base (Signal, heretic-ara, your own fine-tune), want the strongest form of the edit, or want to toggle it per-request.

Variants

Repo	Format	Use
This repo	fp16 full weights (~52 GB)	Merge source / full-precision serving
Shockem/Qwen3.8-27b-Terse-Coder-NVFP4	modelopt W4A16 NVFP4 + FP8 attention + MTP draft stack	Tested deployment — all numbers above

Serving (vLLM, tested path)

Tested on vLLM 0.28 with MTP speculative decoding on 2× RTX 5060 Ti 16 GB:

vllm serve Shockem/Qwen3.8-27b-Terse-Coder-NVFP4 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --quantization modelopt_fp8 --kv-cache-dtype fp8

Turn MTP spec decode on. The checkpoint carries a full MTP draft stack (vocab-truncated 40960-id draft head); acceptance measured 0.41 and outputs are target-verified (lossless). If you serve with spec decode, make sure the generation config has no min_p — vLLM 0.28 rejects min_p under spec decode.

Recommended sampling (mirrors testing): temp 0.6, top_k 20, top_p 0.95, repetition_penalty 1.05.

On 2×16 GB cards cap context at ~200k with a ~3.9 GiB FP8 KV pin; single-card 24 GB+ rigs are unaffected.

A GGUF build of this merge (Q4_K_M / Q5_K_M / Q6_K / Q8_0 + vision projectors) is validated on llama.cpp (Q4_K_M smoke-tested: correct code generation + reasoning parsing) and will be shared via a community quant repo.

Training (summary)

Round-8 checkpoint-salvage DPO adapter merged into pristine Qwen/Qwen3.8-27B in fp32, stored fp16, then quantized with an NVIDIA-style recipe (modelopt 0.45 W4A16, FP8 attention, Local-Hessian-weighted calibration on MLP/lm_head — Hessian calibration is what preserves the adapter effect through 4-bit quantization; absmax calibration attenuates it to roughly half).

• Data: a 301-problem coding pool (HumanEval + MBPP-sanitized, plus mined round-7 EvalPlus failures) with two pair channels: entropy-pruned (short+pass vs long+pass) and hard-negative (long+pass vs short+fail) — 70 stock pairs + 430 Signal pairs, min-capped to 122 training rows.
• Method: DPO (trl), β 0.05, lr 1e-5, 3 epochs, eff. batch 8, initialized from the round-7 adapter. Round 7's epoch-3 checkpoint had overfit, so round 8 re-ran the identical recipe with per-epoch adapter checkpoints and selected epoch 1 (the earliest checkpoint clearing the quality gates) after independent per-epoch evaluation on the NVIDIA base.

Caveats

• Targeted at coding tasks with thinking enabled. Behavioral edit, not a knowledge edit — general chat and math are not the training distribution.
• The preference is "shorter reasoning, identical answer" — if a task needs long derivation, raise reasoning_effort as usual.
• Should work fine alongside other quants, formats, and backends (SGLang, TabbyAPI/EXL3, llama.cpp), but only NVFP4 + vLLM has been measured — validate before relying on other combinations.

Attributions & licenses

This model is a derivative work of Qwen/Qwen3.8-27B, © Qwen Team, Alibaba Cloud, licensed Apache 2.0; this model remains Apache 2.0 and the original license and copyright notices are retained. Built standing on the shoulders of:

• Qwen Team (Alibaba Cloud) — the Qwen3.8-27B base model (Apache 2.0).
• NVIDIA — TensorRT Model Optimizer 0.45 (Apache 2.0) drove the NVFP4 quantization, and NVIDIA's published Qwen3.8-27B-NVFP4 checkpoint informed the Hessian-calibrated v3 recipe.
• agentionai — Signal-3.8-27B, one of the trace-generation policies in the preference data.
• p-e-w — the Heretic tool; a heretic-ara abliterated variant of Signal was another trace-generation policy.
• OpenAI — HumanEval (MIT), and Google — MBPP (CC-BY 4.0): prompt sources for training and held-out evaluation.
• Hugging Face TRL (Apache 2.0) — the DPO trainer.
• Datacurve — DeepSWE, used for independent evaluation only.

None of these parties endorse this model; all remaining errors are ours.