Qwen3.6-27b-DSpark

A DSpark speculative-decoding draft model for unsloth/Qwen3.6-27B-NVFP4.

Be warned, this is an early iteration. On current testing it gives a real, measured 1.28x–1.36x throughput speedup over plain decoding — see Honest Limitations below before deciding if it fits your use case.

Model Specifications

Verifier / target model	unsloth/Qwen3.6-27B-NVFP4
Draft architecture	DFlash backbone (5 layers) + Markov head + confidence head
Block size	16 (up to 15 speculative tokens per round)
Target/aux hidden-state layers	1, 16, 31, 46, 61
Markov head	rank 256, vanilla (bigram)
Confidence head	enabled, conditioned jointly with the Markov head
Draft vocab	full 248,320 (no draft-vocab compression)
Loss	CE 0.1 / total-variation 0.9 (TV term directly minimizes 1 − acceptance rate)

Training Data

All training data is self-distilled: prompts were sampled from public datasets, then responses were regenerated by the verifier model itself (unsloth/Qwen3.6-27B-NVFP4), rather than using the datasets' original static responses. This matters because DSpark is distilling the verifier's own output distribution — training on the verifier's own generations gives a real, measured improvement over training on that same data's original, non-regenerated responses.

10,000 samples total, mixed across 5 sources (informed by DSpark's own published recipe — chat/math/code/instruct — shifted more code-heavy for a coding-agent use case):

Source	Samples	Domain
GSM8K	2,400	Math
OpenCoder-LLM/opc-sft-stage2 (Python)	1,900	Code
bleugreen/typescript-instruct	900	Code
Magpie	1,400	Chat
UltraChat	4,319	Chat

30% of the Magpie/UltraChat regenerations were done with the verifier's thinking mode explicitly enabled, to include some reasoning-trace data in the mix.

Preprocessing: tokenized with the verifier's own chat template (prepare_data.py from speculators), sequence length 16,384 at prep time, deterministically shuffled (fixed seed) and capped at 10,000 samples. Hidden states for the target layers above were extracted via vLLM's native hidden-state extraction during training (online — hidden states are generated on demand and never written to disk in bulk).

Training: 2 epochs, LR 3e-4, total-seq-len 2048, checkpoint selected at the best validation loss (epoch 2 of 2, final loss 42.07).

Evaluation Results

Measured by serving this checkpoint via vLLM's dspark speculative-decoding method and reading vLLM's own vllm:spec_decode_* Prometheus counters across a diverse real-traffic sample (code, math, general chat, and reasoning prompts; 8,134 total spec-decode rounds, 122,010 draft tokens, 4,147 accepted).

Acceptance length: 1.5098 (1.0 = no benefit; higher = more tokens emitted per verifier pass)

Position	Acceptance rate
0	33.1%
1	11.8%
2	4.0%
3	1.35%
4	0.45%
5	0.20%
6	0.09%
7	0.02%
8+	0%

Real measured throughput (same hardware, same 41-prompt diverse set, max_tokens=400, temperature=0, plain decoding vs. this checkpoint):

Concurrency	Baseline (no spec decode)	With DSpark	Speedup
1	12.6 tok/s	16.1 tok/s	1.28x
8	83.7 tok/s	113.5 tok/s	1.36x

Note the real throughput speedup (1.28–1.36x) is meaningfully lower than the acceptance-length number (1.5098) would suggest — acceptance length is an upper bound on speedup, not the speedup itself, since the draft model and verification step still cost real compute per round.

Progression across our own training iterations (same evaluation methodology throughout):

Checkpoint	Acceptance length
Initial baseline (bug-fix only, static ShareGPT, no self-distillation)	1.032
Self-distillation (600 samples, UltraChat only)	1.099
Mixed data (4,000 samples, 3 draft layers)	1.184
This checkpoint (10,000 samples, 5 draft layers)	1.5098

Deployment

Requires a vLLM build with native dspark speculative-decoding support and a --trust-remote-code-compatible custom modeling file (bundled with this checkpoint).

Minimum vLLM version: v0.25.0.

⚠️ Blackwell GPUs (SM100/SM103/SM120): use a newer nightly instead. v0.25.0's native dspark path crashes reproducibly on Blackwell with CUDA error: an illegal memory access was encountered during draft-attention-metadata construction — confirmed directly against this published checkpoint. Root cause is a known, already-fixed upstream bug: vllm-project/vllm#48167 ("Fix FlashInfer non-causal draft attention (DFlash/DSpark) on Blackwell"), merged 2026-07-15 — after v0.25.0/v0.25.1 and every nightly published up to that point, so no release or published image contains the fix yet as of this writing. Non-Blackwell GPUs (e.g. Hopper) aren't affected.

We run vllm/vllm-openai:nightly-95ed0feaa5cd7fb16d72c53ce04950aaf07c4698 in production and every benchmark number in this card was measured on it — use that build for Blackwell until a patch release (v0.25.2 or later) ships with the fix above included.

vllm serve unsloth/Qwen3.6-27B-NVFP4 \
    --trust-remote-code \
    --kv-cache-dtype fp8_e4m3 \
    --mamba-cache-dtype float16 \
    --mamba-ssm-cache-dtype float16 \
    --enforce-eager \
    --speculative-config '{"method":"dspark","model":"<this-repo>","num_speculative_tokens":15}'

Honest Limitations

• Small training scale. 10,000 samples is roughly 50–150x smaller than published production DSpark recipes (500K–1.4M samples). Acceptance length is still well below published production floors (2.3–2.8x+) and below DFlash-alone baselines on coding/math benchmarks (5.6–8.9). Scaling training data further is the clear next lever and is actively in progress.
• Evaluation was greedy/deterministic (temperature=0). Not yet evaluated across a range of sampling temperatures or with longer-context workloads.
• This is iteration 3 of an ongoing project, not a final release. Expect updated checkpoints as training data scales up.

Versions

• 2026-07-15 — Initial release. Trained on 10,000 samples of mixed data (regenerated by unsloth/Qwen3.6-27B-NVFP4). Acceptance length ~1.51, ~1.3x real measured throughput speedup.

Acknowledgements

• DSpark paper (DeepSeek) — the recipe this checkpoint follows
• vllm-project/speculators — the training/serving framework used throughout