cfontes/GLM-5.3-Flash-DFlash2-TR3-v3

GLM-5.3-Flash DFlash2 drafter — TR3-v3 (EXL3 TP4 champion)

Retrained incoai/GLM-5.3-Flash-DFlash2 speculative-decoding drafter, tuned for serving GLM-5.3-Flash quantized with EXL3 on a 4-node DGX Spark TP4 vLLM stack. This is the drafter used by our EXL3 TP4 serving champion: 31.7 tok/s single-stream (K=2, CUDA graphs) — +68% throughput over the vendor drafter on the same stack.

Full methods, findings, and the complete results ladder: cfontes/GLM-5.3-Flash-EXL3-TP4-Spark

Training

• Data: 427,444 hidden-state capture files (160 GB) recorded from the live EXL3-TR3 TP4 serving stack via the fork capture hook (54-domain prompt mix, code-heavy weighting, 886k+ output tokens). Captured ON the quantized target — the drafter is trained on exactly the distribution it serves against.
• Objective: 1.0·block_CE(0.85^k decay) + 0.5·KL(top-256 teacher, T=1.0) + 2.0·selector-CE(top-16) + 0.5·anchor-KL + 0.1·hidden-cosine
• Schedule: stage-A (fc / hidden_norm / hidden_projection / selector codebooks trainable; decoder layers frozen), 4 epochs (164 steps), bf16, single DGX Spark.

Results vs baselines (same EXL3 stack, K=2, temp 0, c=1)

drafter	tok/s	accept len
vendor (v1)	18.9	1.07
our v2 (adapter fold)	25.9	1.71
this model (v3)	31.7	1.83

Known limits

• Teacher-forcing saturates: 8× training steps moved served accept by only +0.02.
• A noise-augmented v4 (hidden-state noise σ=5%) scored 31.1/1.75 — no gain; the accept ceiling is set by EXL3 mid-layer quantization noise, not drafter capacity (linear probe of drafter inputs: 30% top-1 vs v3's 53–58% pos-0 accept).
• K=2 is the optimal draft length on EXL3 TP4 (K1–K7 swept).

Serving

See the launcher and config in cfontes/GLM-5.3-Flash-EXL3-TP4-Spark. Config (block 8, selector rank 256 / top-16, target layers 5/14/24/33/42) is unchanged from the vendor drafter.

Lineage & credit

• zai-org/GLM-5.3-Flash — base model (Z.ai)
• DFlash: Block Diffusion for Flash Speculative Decoding (Chen, Liang & Liu 2026, Z Lab) — method
• incoai/GLM-5.3-Flash-DFlash2 — starting weights
• @vcruz305 — vllm-exl3 native kernels; ZJY0516 — vLLM fork runtime
• MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Sparks (Mia) — serving recipe our TP4 build derives from

Original work: TP4 capture/training harness, TR3 retrains, hidden-state ceiling analysis, benchmark evidence (all on our 4× DGX Spark cluster).