jiapingW/Qwen3-8b-dspark-SpecForge

DSpark causal / no-confidence · Qwen3-8B draft · step 39000

Device label: RTX PRO5000. Target model: Qwen3-8B.

Core files

File	Contents
model.safetensors, config.json	Draft weights and inference configuration for the selected checkpoint
training/launch.yaml	Core training parameters; supply deployment settings for your environment
training/draft_config.json	Model architecture and algorithm parameters used during training
tensorboard/	Logs from the entire training run, including steps beyond the selected checkpoint where applicable
bench/results.json	Per-benchmark and aggregate results for both verification widths, at full numerical precision
README.md	Training method and benchmark results

Training

Data source: Regenerated PerfectBlend training data (JSONL).

Training uses PerfectBlend data regenerated by Qwen3-8B at temperature 0 in non-thinking mode, with the qwen3-instruct template and no system prompt. The target model supplies hidden states and the required distillation information, and the draft trains on the extracted features. The 32 training ranks run separately from the 8 target inference services, with Mooncake transferring the features.

This model uses a training max_length of 8192, compared with 3072 for DSpine in the same comparison. The training plan spans 6 epochs; selected checkpoints and training lengths differ across methods.

Parameter	Setting
Planned total steps / selected checkpoint	66604 / 39000
block_size / max_length	16 / 8192
Draft layers / hidden size / FFN size	5 / 4096 / 12288
attention heads / KV heads	32 / 8
Target feature layer indices	1, 9, 17, 25, 33
Microbatch per rank / gradient accumulation	2 / 2
Effective global batch size	32 × 2 × 2 = 128
Optimizer	AdamW, FP32 master parameters, weight_decay=0
Peak learning rate / schedule	6e-4 / cosine
LR warmup / gradient clipping	4% of total steps / max_grad_norm=1.0
num_anchors / loss_decay_gamma	512 / 7.0
objective_chunk_blocks / seed	128 / 42
Parameter dtype / attention	BF16 / FlexAttention
Gradient strategy / kernel	NO_SHARD / DDP; Liger

This model is the causal / no-confidence variant of DSpark: is_causal=true, with a vanilla Markov head (rank 256), the DSpark projector, and GQA. The loss is 0.1 × CE + 0.9 × L1; the confidence head is disabled, the confidence loss weight is 0, and no confidence scheduler is used. Projection flattening and caching are both enabled.

This model uses the step 39000 checkpoint. Training has stopped, and the TensorBoard logs extend through step 40960, before the planned 66604 steps. The logs retain the entire recorded training run and are not truncated at the selected checkpoint.

See training/launch.yaml for the core training parameters; deployment settings must be supplied for your environment. The learning-rate schedule and training phases defined as proportions use the planned total step count.

Benchmark (temperature=0)

Each mode contains 2400 requests: GSM8K 1319, MATH500 500, HumanEval 164, MBPP 257, and MT-Bench with 80 two-turn conversations (160 requests). Both draft16/verify16 (D16/V16) and draft16/verify8 (D16/V8) use a draft block size of 16; only the target verification width changes. The verification width includes the anchor.

Benchmark	Requests	D16/V16 request macro τ	D16/V16 micro τ	D16/V8 request macro τ	D16/V8 micro τ
GSM8K	1319	9.720633	9.208117	6.452161	6.325923
MATH500	500	9.375822	8.576850	6.356439	6.165978
HumanEval	164	7.481950	7.391824	5.608338	5.587488
MBPP	257	6.988848	6.543984	5.399994	5.211843
MT-Bench	160	5.159163	4.097847	4.232717	3.756643
All requests, pooled	2400	8.899194	7.810402	6.113925	5.798654

Aggregate	D16/V16	D16/V8
Equal-weight mean of request macro τ across the five benchmarks	7.745283	5.609930
Equal-weight mean of micro τ across the five benchmarks	7.163724	5.409575
pooled completion_tokens	1038721	1028919
pooled verify_count	132992	177441

See bench/results.json for results and counts at full precision.

For each request, τ = completion_tokens / verify_count, including target correction / bonus tokens. Request macro is the arithmetic mean of request-level τ; micro is the total token count divided by the total number of verification rounds. Pooled metrics aggregate all requests. The equal-weight benchmark means first compute each benchmark metric, then average across the five benchmarks.

Evaluation uses SGLang 0.5.21 with the corresponding algorithm-specific integrations, BF16, TP=1, FlashInfer, and context_length=16384; temperature=0, non-thinking mode, no system prompt, and max_new_tokens=2048. Each replica uses max_running_requests=1, with CUDA graphs and Radix cache disabled. DSpark first generates 16 candidate predictions per round, then takes the first 15 or 7 candidates according to the verification width and adds the anchor. The tables measure acceptance length, not task accuracy or throughput.

Benchmark (temperature=1)

Target temperature is 1, top_p=1 and top_k=-1 (no truncation). Thinking is disabled, the system prompt is empty, and max_new_tokens=2048. Each mode contains 2400 requests on the same five benchmarks as the temperature=0 evaluation.

Native draft policy: native temperature-conditioned sampling. DFlash and Domino keep greedy proposals; DSpark and DSpine retain their native temperature-conditioned proposal sampling. These are native-policy comparisons, not a claim of identical draft distributions across methods.

Both modes compute the complete draft16 before verification. Verify16/8 includes the anchor, so 15/7 candidates are verified. DSpark computes its native 16 candidates before truncation. Sampling verification uses the actual proposal distribution, including the sparse candidate distribution for DSpine.

Benchmark	Requests	D16/V16 request macro τ	D16/V16 micro τ	D16/V8 request macro τ	D16/V8 micro τ
GSM8K	1319	8.984335	8.319791	6.234437	6.047417
MATH-500	500	8.186249	7.106030	5.858557	5.444322
HumanEval	164	6.683424	6.452415	5.345947	5.262770
MBPP sanitized	257	6.427295	5.982378	5.141272	4.908892
MT-Bench	160	4.657792	3.762313	3.972278	3.489013
All requests, pooled	2400	8.098585	6.836842	5.827545	5.347060

Aggregate	D16/V16	D16/V8
Equal-weight mean of benchmark macro τ	6.987819	5.310498
Equal-weight mean of benchmark micro τ	6.324586	5.030483
Pooled completion_tokens	1054412	1052387
Pooled verify_count	154225	196816

Macro averages per-request completion_tokens/verify_count; micro divides total completion tokens by total verification rounds. Correction/bonus tokens are included. Pooled metrics are request-weighted; equal-weight benchmark means are listed separately.

This is one stochastic benchmark run per setting, not a multi-seed confidence interval. Request seeds follow 42 + uint32_le(SHA256(dataset:sample_id:turn)[:4]); RNG consumption and generated histories can differ by model, width, and replica assignment. Small differences should not be interpreted as statistically established improvements.

Runtime: SGLang 0.5.21, BF16, TP=1, two independent replicas per mode, one request per replica, context_length=16384, CUDA graphs and Radix cache disabled. A process-local overlay enables sampled split verification without changing the original installed runtime. Synthetic distribution/acceptance tests and real-model structural smoke audits passed before the full benchmark; these checks are not a proof of bitwise equivalence to target-only decoding.

The original temperature=0 values remain unchanged. Full-precision temperature=1 metrics, counters, model hashes and runtime provenance are under temperature_1 in bench/results.json. These measurements are acceptance lengths, not answer accuracy or throughput.