Muse-Spark-Open-DSpark

DSpark speculative-decoding draft model for meta-models/Muse-Spark-Open-816B-A42B (816B-A42B MoE target, NVFP4). Built with Muse Spark.

DSpark architecture

• 5-layer decoder-only draft (hidden 8192, 128/16 attention/KV heads, FFN 16384), BF16, causal sliding-window attention (2048)
• Full 202,048-token target vocabulary; frozen target LM head
• Conditioned on target hidden states tapped at layers [13, 29, 45, 57, 61] plus the final hidden state
• Proposes 7 tokens per step (sample-from-anchor); learned Markov confidence head (rank 512)
• ~10.6 GB, 74 tensors, safetensors

Acceptance rates

Measured against meta-models/Muse-Spark-Open-816B-A42B-NVFP4(default chat template, reasoning effort xhigh), with the serving config below: 7 probabilistic draft tokens, block verification, temperature 1.0 / top-p 0.95.

Dataset	Requests	Acceptance	Tokens/cycle
MT-Bench (chat)	80	28.46%	2.99
GSM8K	1319	42.04%	3.94
MATH-500	500	39.69%	3.78
HumanEval	164	37.23%	3.61
MBPP	974	35.24%	3.47
AIME26	30	33.15%	3.32
SWE-bench Pro	731	23.69%	2.66
SPEED-Bench (low entropy)	512	24.76%	2.73
BEAM 100K	20	29.07%	3.03
BEAM 500K	35	29.85%	3.09
BEAM 1M	35	28.55%	3.00

Tokens/cycle is the mean number of tokens emitted per verification step, including the target's bonus token (acceptance length). SPEED-Bench uses the throughput_16k low-entropy split with inputs truncated to 10,240 tokens; BEAM requests generate 96 tokens each.

Serving

{"method":"dspark","model":"<this repo>","num_speculative_tokens":7,
 "draft_sample_method":"probabilistic","enable_adaptive_verification":false,
 "rejection_sample_method":"block"}

Requires a Muse Spark–compatible serving runtime.