Avifenesh/Qwen3.8-27B-DFlash2-memra

Qwen3.8-27B-DFlash2 for memra

The DFlash 2 block-diffusion draft model for Qwen/Qwen3.8-27B, packaged for serving with memra. The weights are byte-identical to z-lab/Qwen3.8-27B-DFlash2 (model.safetensors sha256 67fc76d68dc5a9415511a4f394ef744d67510cd20e93b37cc2cc7d28e4bab65c), itself a mirror of incoai/Qwen3.8-27B-DFlash2. Apache-2.0, unmodified; this card adds the memra serving configuration and measured receipts.

It is not a standalone language model: it drafts token blocks that the target model verifies. Greedy output is byte-identical to plain decode by construction — the verifier arbitrates every committed token.

Serve with memra

MEMRA_COMPAT=openai \
MEMRA_MODELS="q38=hf:Avifenesh/Qwen3.8-27B-NVFP4-MTP-GGUF:Q5K-mtp" \
MEMRA_DSPARK_SPEC=1 \
MEMRA_DSPARK_DRAFT=/models/Qwen3.8-27B-DFlash2-memra \
MEMRA_FRSPEC_TRIM=q38-ranks-sxc32768.gguf.txt \
memra-server

Defaults do the tuning (memra >= v0.112.0): the drafter backbone quantizes to q4_0 at load (MEMRA_DFLASH_PREC=q4, the measured default), and with the ranks file armed the round consumes an FR-Spec vocabulary trim — the draft-side lm_head and its top-k run over 32,768 ranked rows instead of the full 248,320 while verification keeps the full vocabulary, so output is still byte-identical to plain decode. The ranks file is q38-ranks-sxc32768.gguf.txt. Boot receipt: [dspark] q38: DFlash2 draft head TRIMMED to 32768 rows.

Arming MEMRA_DSPARK_SPEC gives this drafter the model's speculative program (the MTP arm disengages; two spec programs never coexist). Exactness gate: spec-on vs spec-off streams are byte-identical (dspark_q38_gate, ALL EXACT).

Measured (RTX PRO 6000 Blackwell, served, greedy, conc 1 steady, x3 interleaved)

config	chat agg tok/s	agentic agg tok/s	accept
native MTP masked head (baseline)	126.9	148.6	0.63
DFlash2, q8 drafter, full vocab	134.9	151.5	0.75
DFlash2, q4 drafter, full vocab	136.3	154.5	0.76
DFlash2, q4 drafter + FR-Spec trim (defaults)	142.9	157.2	0.73
DFlash2, fixed verify T=8 (rejected)	132.0	145.5	0.52

• Adaptive verify width (the default) beats fixed T=8 at serve shape.
• The q4 drafter re-encode costs nothing in acceptance and saves draft time; the FR-Spec vocab trim stacks on it (bare gate: 152.4 -> 160.4 tok/s combined, +5.2%). Both ship as defaults in memra v0.112.0, alongside the concurrency shed that keeps speculation low-concurrency-only under load (c>=4 aggregate at plain-batch parity).
• The trim trades a little acceptance (0.755 -> 0.730 agentic) for the wall win — chat gains most (+4.8% served) because shorter rounds make the head cost a bigger share.
• The asymmetric q5 quant collapses acceptance (0.66 -> 0.42) and is not offered.

Served (memra >= v0.113.0): sampled admission stacks, and the route carries production

The measured tables above are GREEDY — the byte-exactness instrument. Production serves the vendor-recommended SAMPLED defaults, and since memra v0.113.0 sampled DFlash2 sessions stack in the spec-gate LOW band like greedy ones (earlier versions admitted sampled speculation only on an otherwise-idle server). At that shape, on the same RTX PRO 6000 (x3 interleaved, medians, agg tok/s vs the MTP head):

c	DFlash2 route	MTP head
1	126.8	116.6
2	127.5	119.5
4	87.1	84.8

Single-stream wall rates on the same card scale with output predictability: prose ~131-146, code ~208-239, digit-heavy ~287-339 tok/s. This configuration serves both production Qwen3.8 origins of the maintainers' own API.

Multi-turn (memra >= v0.112.0)

Conversations resume their parked session by default: turn N+1 primes only the new suffix onto the parked trunk cache + draft KV instead of re-prefilling the whole conversation. Measured on the same card, 8-turn conversation: per-turn TTFT -16% -> -83% vs cold re-prime, growing with depth (turn 7: 0.32 s vs 1.90 s). Resume receipt: [worker] dspark-reuse: N committed tokens resumed. Streams cut by max_tokens mid-burst do not park, and a request whose context cap outgrew the parked allocation serves cold — both named in the log; agent loops that omit max_tokens resume every turn.

Citations

@misc{inco2026dflash2,
  title  = {{DFlash 2: Keep Drafting Parallel}},
  author = {{Inco AI}},
  year   = {2026},
  month  = {August},
  url    = {https://inco.ai/blog/dflash2/}
}
@inproceedings{chen2026dflash,
  title     = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
  author    = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  booktitle = {International Conference on Machine Learning (ICML)},
  year      = {2026}
}