canada-quant/GLM-5.3-Flash-DFlash2-E

GLM-5.3-Flash DFlash2 Drafter — run dflash2E

Self-trained DFlash2 block-diffusion speculative-decoding drafter for GLM-5.3-Flash, trained against our INT4 quant canada-quant/GLM-5.3-Flash-W4A16-MTP. Trained from scratch on self-generated data — no third-party drafter weights or traces anywhere in the training path — and adopted on 2026-09-15 as the authors' serving drafter-of-record on their 2× DGX Spark (SM121) stack, superseding our previous best dflash2DF2 (owner ruling; the swap is verified and the stack serves continuously since).

Architecture

Field	Value
Type	DFlash2 block-diffusion drafter (DFlash2DraftModel, Qwen3-style backbone)
Layers	8 decoder layers, full attention (no sliding window)
Hidden / heads	4096 (intermediate 12288) · 32 attention heads / 8 KV heads · head_dim 128
Target taps	9 hidden-state taps at target layers [5, 9, 14, 19, 24, 28, 33, 38, 42]
Block size	8 → K = 7 speculative tokens (num_speculative_tokens: 7)
Selector	rank 256, top_k 16, grouped dynamic conv (kernel 2, group 16) — selector + conv are trained, not vestigial (verified 2026-09-16 against the shipped checkpoint: 3 selector + 32 conv tensors)
Mask embedding	learnable, shipped as mask_embedding.pt (mask_token_id 154856)
Size	~1.84B drafter parameters · 6.2 GB bf16 checkpoint (ships untied embed_tokens + lm_head for standalone loading)
Max positions	1,048,576

Training

• Warm-started from our previous best dflash2DF2 (L8T9 lineage, third refinement round).
• 350,260 self-generated samples (selfgen_v13), 35,021 steps, lr 1e-4.
• Data: prompts sampled from public instruction sets (ultrachat_200k · MIT, OpenR1-Math-220k · Apache-2.0, OpenCodeReasoning / OpenCodeInstruct · CC-BY-4.0, evol-codealpaca-v1 · Apache-2.0); every completion was regenerated by the target model itself (thinking ON) — no third-party model outputs.
• The widely-used reference drafter incoai/GLM-5.3-Flash-DFlash2 was never a training input — it appears below only as a measured same-protocol comparison.
• PROVENANCE.txt in this repo is the verbatim training record (run name, warm-start source, data, steps, eval numbers).

H200 holdout evaluation

500 never-trained-on prompts, thinking ON, T=1.0 / top_p 0.95, max_tokens 1024, greedy drafts, H200 TP=4 (concurrency 16; the c1 row is 100 prompts at concurrency 1):

Metric (acceptance length / output tok/s)	K=7, c16	K=4, c16	K=7, c1
dflash2E (this)	3.568 / 1277.5	3.067 / 1250.4	3.585 / 267.2
incoai reference (same protocol)	3.602	3.123	—

Per-position acceptance (K=7, c16): 0.744 · 0.545 · 0.405 · 0.306 · 0.237 · 0.185 · 0.147.

Serving it on 2× DGX Spark (SM121)

Prebuilt vLLM image + one-command launcher: canada-quant/vllm-glm53-flash-sm121 (docker pull ghcr.io/canada-quant/vllm-glm53-flash-sm121:v2-w4a16-dflash2e). The drafter is bind-mounted at runtime — swap drafters without a rebuild; K must follow the drafter's block size (E: block 8 → num_speculative_tokens=7).

Same-rig A/B — 2× DGX Spark (SM121), TP=2, eager, K=7

All three drafters measured on the same rig within ~10 hours, same 500-prompt holdout family, same W4A16 target:

Cell (accept / tok/s)	dflash2DF2 (our previous best)	dflash2E (this)	incoai reference
smoke 20×c1×512	3.9464 / 29.35	4.0137 / 29.78	4.0698 / 31.17
c16 500×16×1024 (primary)	3.5020 / 69.33	3.5788 / 70.87	3.6274 / 72.98
c1 100×1×1024	3.5100 / 27.56	3.5970 / 28.12	3.5760 / 28.97

Honest read:

• Vs our previous best (dflash2DF2): wins every cell — c16 +0.077 acceptance / +2.2% throughput; c1 +0.087 / +2.0%; smoke +0.067. This is why it was adopted.
• Vs the incoai reference: it loses the c16 throughput primary by 2.9% (70.87 vs 72.98 tok/s; acceptance 3.5788 vs 3.6274) and the smoke cell, while edging c1 acceptance (+0.021; 3.5970 vs 3.5760) at −2.9% tok/s. We ship it anyway: it is fully ours (Apache-2.0, no NC/ND terms), trained on data we control, reproducible end-to-end, and it closed 61% of the acceptance gap our previous best had left open.
• Cross-hardware port fidelity: Sparks c16 3.5788 vs H200 3.568 (+0.3%); c1 3.5970 vs 3.585 (+0.3%) — the acceptance profile reproduces across SM121 and H200.

Serving

Pairs with canada-quant/GLM-5.3-Flash-W4A16-MTP (the quant it was trained against) or BF16 GLM-5.3-Flash. vLLM speculative config:

{"method": "dflash", "model": "/models/GLM-5.3-Flash-DFlash2-E", "num_speculative_tokens": 7}

Ready-to-run recipe: launch_dflash2_tp2.sh ships in this repo — the complete TP=2 DGX-Spark serving launcher (defaults to the W4A16 target + this drafter; GRAPHS/EAGER knobs, DRAFTER_HOST_PATH override to serve any other DFlash2 drafter).

Hard constraints (all measured, not stylistic):

• num_speculative_tokens must be 7 (= block_size − 1). Other counts boot-wedge the DFlash2 stack.
• mask_embedding.pt must sit next to the weights. Verify the boot log carries Loaded DFlash mask embedding for mask_token_id 154856 from mask_embedding.pt — absence means the mask was silently ignored; do not serve.
• The 9-tap config requires the serving stack to honor dflash_config.target_layer_ids of length 9 (upstream vLLM DFlash2 does — vllm-project/vllm#52816).
• Full-attention drafter layers: the target stack must accept FullAttentionSpec drafter KV in the GLM-5 KV fast path. The stock upstream DFlash2 image cannot boot this drafter; see the W4A16 card's SM121 serving section for the working stack (patched SM121 build, fabric tuning, launcher).

Our serving configuration: 2× DGX Spark GB10, TP=2 + expert-parallel, 262K context, fp8_e4m3 KV cache, K=7 greedy drafts — left up and serving since 2026-09-15 (smoke 4.243 @ 31.2 tok/s, c1 3.617 @ 28.2 tok/s post-swap verification, zero stream errors). Serving state (updated 2026-09-16): the stack now runs the ruled graphs-ON g4 config — CUDA-graphs FULL_AND_PIECEWISE capture [1,2,4,8,16,24,32] + VLLM_USE_BREAKABLE_CUDAGRAPH=0, GMU=0.795, KV 8 GiB (pool 366,749 tokens); the flip boot re-verified all five production gates (pool discriminator, graphs live on both ranks, mask-loader line, zero-error smoke, zero engine faults). The DFlash2 speculator + selector walk the vLLM V2 GPU-worker path at serve time (confirmed 2026-09-16 from the boot bytes) — no V1-proposer silent-skip.

Provenance

• Files: model.safetensors (sha256 af0d8d16f01f…), config.json (699bdf29…), mask_embedding.pt (2474994c…), PROVENANCE.txt (verbatim record).
• Every number on this card comes from sha256-pinned banked artifacts (raw eval JSONs, byte-verified measurement journals); the A/B table is the adjudicated table of record from the measurement program.

References

• DFlash2 method: z-lab/dflash · arXiv 2602.06036 · inco.ai blog
• Upstream vLLM DFlash2 integration: vllm-project/vllm#52816
• Target model: zai-org/GLM-5.3-Flash · our quant: canada-quant/GLM-5.3-Flash-W4A16-MTP
• Reference drafter (comparison only): incoai/GLM-5.3-Flash-DFlash2 (CC-BY-NC-ND-4.0)

This card was assembled from sha-pinned banked measurement artifacts; reviewed by a human (the maintainer) before publication.