Qwen3.5-2B-DFlash2

A DFlash2 draft model for speculative decoding with Qwen/Qwen3.5-2B as the verifier, trained with speculators. With a block size of 8, the drafter proposes 7 tokens at a time and the verifier checks them in one forward pass, so output is identical to running the verifier alone — a lossless speedup. At temperature 0, mean acceptance length is 3.49 tokens committed per verification step, up to 5.07 on math_reasoning.

Training code: rasyosef/train-dspark-draft-models.

Trained on 100,000 samples.

Usage

vLLM loads the verifier automatically from the config — don't pass it separately.

vllm serve rasyosef/Qwen3.5-2B-DFlash2 --port 8000 \
  --gpu-memory-utilization 0.8 \
  --override-generation-config '{"temperature": 0}'

Then query the OpenAI-compatible endpoint at http://localhost:8000/v1.

Details

4 Qwen3 layers (hidden size 2048, intermediate size 6144, 16 attention heads over 4 KV heads, head dim 128), with the first three layers using sliding-window attention (2048-token window) and the last using full attention. Block size 8 (7 draft tokens per step), aux hidden-state layers 1/5/9/13/17/21, bfloat16.

Trained for 4 epochs at lr 4e-4 (AdamW, cosine schedule with 4% warmup) on 100,000 filtered Open PerfectBlend prompts with responses regenerated by the verifier itself, split 96/4 into train and validation, with a {"ce": 0.1, "tv": 0.9} loss. Samples prepared at 1024 tokens; training sequence length 4096, up to 512 anchors per sample. Verifier hidden states were pulled on demand from a running vLLM server during training and deleted after use rather than staged to disk up front.

Evaluation

evaluate.py throughput across the nine RedHatAI/speculator_benchmarks subsets, run at temperature 0 (greedy decoding). acceptance_length is mean tokens committed per verification step, including the bonus token — floor 1.0, ceiling 8.0 at block size 8 (7 draft tokens).

subset	acceptance_length	pos_0	pos_1	pos_2	pos_3	pos_4	pos_5	pos_6
math_reasoning	5.069	88.3%	76.0%	64.8%	55.5%	47.3%	40.4%	34.6%
HumanEval	4.361	82.3%	66.5%	54.4%	44.3%	36.3%	29.5%	22.8%
rag	3.778	79.4%	59.7%	45.0%	34.9%	27.4%	19.4%	12.0%
qa	3.217	65.5%	43.4%	31.1%	24.7%	21.6%	18.7%	16.7%
tool_call	3.077	67.5%	45.3%	30.9%	23.2%	17.8%	13.4%	9.6%
writing	2.977	66.2%	42.2%	28.1%	21.0%	16.6%	13.2%	10.4%
question	2.811	64.9%	40.8%	26.0%	18.3%	13.6%	10.2%	7.3%
summarization	2.432	64.7%	36.7%	19.9%	10.9%	6.1%	3.2%	1.7%
translation	2.265	65.8%	35.1%	15.8%	6.2%	2.5%	0.8%	0.3%

Weighted across all subsets (temperature 0): 3.488 over 127,690 verification steps.

Acceptance is highest where the verifier's next token is most predictable. math_reasoning is the only subset above 5.0 and still accepts more than a third of drafts at pos_6 (34.6%); HumanEval follows at 4.36 and stays above 20% through pos_6. rag (3.78) is third: it starts close to HumanEval at pos_0 (79.4%) but decays faster, to 12.0% at pos_6. The next four — qa, tool_call, writing and question — sit in a 2.81–3.22 band, with pos_0 acceptance of about 65–67%. qa (3.22) leads that band on an unusually flat tail: it still accepts 16.7% at pos_6, the third-highest of any subset and ahead of rag. tool_call (3.08) is only slightly ahead of open-ended writing (2.98), so the structure of a tool call is apparently not buying much predictability here. summarization (2.43) and translation (2.26) are lowest; both start level with the middle band at pos_0 (64.7% and 65.8%) but collapse quickly, translation to 15.8% at pos_2 and under 1% from pos_5 onward. translation is also the smallest subset (917 steps), so treat its number as noisy.

First-position acceptance spans 65–88% across all nine subsets, and seven of the nine remain above 10% at pos_4 (all but summarization and translation). Overall, 35.5% of drafted tokens are accepted (317,702 of 893,830).

Limitations

Works only with Qwen3.5-2B and is not usable as a standalone model. The verifier is vision-language, but the drafter was trained on text-only prompts and was evaluated on text-only benchmarks, so acceptance on image inputs is untested. Acceptance still falls with position: about 64% of drafted tokens are rejected overall, and on summarization and translation fewer than 7% of drafts survive to pos_4, so the tail of each 7-token draft is largely wasted on that traffic and gains there are likely smaller. The acceptance figures above are for temperature 0. Throughput has not been benchmarked for this checkpoint; real-world speedup depends on your hardware and traffic mix, and because verification is lossless, the verifier's own behavior and biases carry through unchanged.