MiniCPM5-2B-DFlash2

A community DFlash 2 block-diffusion drafter for openbmb/MiniCPM5-2B, trained by self-distillation with the recipe of DFlash 2 (z-lab/dflash). It is not a standalone language model: it drafts blocks of tokens for the frozen target to verify inside a speculative-decoding server. Decoding is lossless — greedy output matches the target exactly, and sampling preserves its distribution.

Architecture

draft backbone	5 qwen3-style decoder layers, hidden 2048, GQA
block size	8 (7 drafted tokens per verify step)
target taps	layers [1, 10, 20, 30, 39] of 42, fused by a linear + RMSNorm
block diffusion	bidirectional attention inside the block (is_causal: false), two-tap dynamic grouped convs (k=2, groups=16)
path selector	low-rank Markov selector, rank 256, top-k 16
mask token	75982 (the same unused id OpenBMB's official MiniCPM5-2B-DSpark uses)
embeddings / LM head	not stored — reused from the target at runtime
parameters	~335M (0.67 GB bf16)

Training (this repo's dflash2_training_meta.json has the raw settings)

• Method: target frozen; forward-KL distillation of the draft's block logits against the target's teacher-forced distribution (+ CE anchor weight 0.1), and the candidate-path selector trained toward the greedy-acceptance answer with teacher-forced predecessor chains. 4 blocks trained per row with cache-streamed full-prefix contexts — the exact cache discipline of the DFlash2 inference loop, so training and serving match.
• Data: two-phase mixture — phase 1 (general): HuggingFaceH4/ultrachat_200k, open-r1/OpenR1-Math-220k, ise-uiuc/Magicoder-Evol-Instruct-110K, HuggingFaceTB/smoltalk, HuggingFaceFW/fineweb-edu; phase 2 (code/agentic anneal): ise-uiuc/Magicoder-Evol-Instruct-110K, HuggingFaceTB/smoltalk, bigcode/self-oss-instruct-sc2-exec-filter-50k, HuggingFaceH4/ultrachat_200k, open-r1/OpenR1-Math-220k. Chat formatted with the target's template, packed into 1536-token rows, 1985 steps at an effective batch of 8 rows.
• Hardware: Tesla T4, fp16/bf16 autocast + GradScaler.
• Validation mode: no — full release recipe.

Measured speed (greedy, batch 1, 6 prompts/domain, this notebook's run)

domain	plain tok/s	spec tok/s	speedup	proj	acceptance length	greedy-identical
coding	17.7	19.53	1.104x	1.082x	1.261	6/6
agentic	17.68	19.37	1.095x	1.077x	1.255	5/6
math	17.61	19.0	1.079x	1.056x	1.231	5/6
general	17.61	19.16	1.088x	1.072x	1.249	5/6
overall	17.65	19.26	1.091x	1.071x	1.248	21/24

Measured per-cycle costs: target decode 58.215 ms/tok, target verify(8) 53.248 ms, draft block 14.6 ms. The proj column is AL * t_decode / (t_verify8 + t_draft) — what an engine-grade serving stack (SGLang/vLLM) should approach; the measured speedup additionally carries the python-side loop overhead of this eval harness.

Speed targets: coding >= 2.0x, agentic >= 2.0x, overall >= 1.5x — not met within caps (measured overall 1.091x). Decoding is lossless by construction: the target verifies every drafted token, so greedy output is token-identical to plain decoding (the identity rate above is limited only by bf16 near-tie argmax flips) and sampling preserves the target distribution. The published DFlash 2 drafters report ~4-5.5 acceptance length on Qwen3.8-27B with full-scale training.

Serving

SGLang (needs the DFLASH support build, see z-lab/dflash):

python -m sglang.launch_server   --model-path openbmb/MiniCPM5-2B   --speculative-algorithm DFLASH   --speculative-draft-model-path applepie69/MiniCPM5-2B-DFlash2   --speculative-num-draft-tokens 8

vLLM:

vllm serve openbmb/MiniCPM5-2B   --speculative-config '{"method": "dflash", "model": "applepie69/MiniCPM5-2B-DFlash2", "num_speculative_tokens": 7}'

Transformers / offline eval with the dflash package:

from dflash.model import DFlash2DraftModel, dflash_generate
draft = DFlash2DraftModel.from_pretrained("applepie69/MiniCPM5-2B-DFlash2")
out = dflash_generate(draft, target_model, input_ids, max_new_tokens=256,
                      stop_token_ids=[1, 130073], temperature=0.0)

Files

file	what
config.json	DFlash2 draft config (dflash_config block)
model.safetensors	the trained drafter (bf16)
dflash2_training_meta.json	training settings + measured results
measured_results.json	per-domain speed/acceptance measurements + gate status

License

Apache-2.0, matching the base model. Check the base model's terms before use.