lued/Qwen3.8-27B-DFlash2-W8

Qwen3.8-27B DFlash2 drafter · W8A16

The DFlash2 speculative-decoding drafter, quantized W8A16 for Ampere-class GPUs.

This is a numerical W8A16 quantization of the z-lab/Qwen3.8-27B-DFlash2 drafter (mirror of incoai/Qwen3.8-27B-DFlash2). Model credit belongs to the z-lab / incoai DFlash2 team and the Qwen base; this repository changes numerics only.

This is a draft model, not a standalone LLM. It is the speculative-drafting component of DFlash2 decoding: it proposes tokens that the target checkpoint verifies before any are emitted. It never emits unverified tokens. Serve it only as the --speculative-config drafter of a DFlash2-compatible target.

What this is

The DFlash2 drafter (PR vllm#52816, open as of 2026-08-19) is a 5-layer, block-diffusion draft model with a top-K candidate selector, trained to predict the target's next tokens at fixed layers. Quantizing it W8A16 drops it from 3.58 GiB (BF16) to 2.02 GiB with no measurable effect on drafting (W8 vs BF16 drafter acceptance is equal: 2.65 vs 2.72/draft at the llama-benchy prose floor, same session).

Property	Value
Architecture	DFlash2DraftModel (5 sliding-attention layers, block size 8)
Quantization	Data-free symmetric RTN W8A16, group size 128
Packed	mlp.*, self_attn.*, fc projections: 36 tensors
Preserved BF16	candidate_selector.* codebooks, *_conv.* kernels, norms: 45 tensors
Draft audit	DRAFT AUDIT PASS (dequant round-trip within INT8 bound; preserved bit-identical; no mtp.*/lm_head)
Worst error ratio	0.749
Size	2.02 GiB (was 3.58 GiB BF16)
Source	z-lab/Qwen3.8-27B-DFlash2 commit 50307d4c4cde6860d4eee73e2547cd786fe8e8a4 (verified unchanged 2026-08-19)

The quantization was produced by the int8-w8a16-dflash2 family toolkit (tools/quantize_draft_w8a16.py); the exact target set and ignore list are recorded in this repository's recipe.yaml and config.json.

Usage

Serve it as the DFlash2 drafter of the matching quantized target lued/Qwen3.8-27B-INT8-W8A16-DFlash2 by repo ID (both resolve from your HF cache):

{"method": "dflash", "model": "lued/Qwen3.8-27B-DFlash2-W8", "num_speculative_tokens": 7}

• Spec window: 7 tokens (block 8), the drafter's trained design; shorter windows measure slower.
• vLLM: requires the vendored PR 52816 patch set from the club-3090 repo. See the target card's "Serving" section for the launch.
• Acceptance is workload-shaped, not a fixed number: ~6.0 tokens/draft on math, ~3.3 on long prose (mean acceptance length including the bonus token, same definition as the z-lab card). See the target card's "Speculative acceptance" section for the envelope and the z-lab comparison.