DaoCloud/Muse-Glimmer-30B-DSpark

Muse Glimmer 30B DSpark

A DSpark drafter for Muse Glimmer 30B, warm-started from the official five-layer Muse Glimmer DFlash assistant.

This checkpoint adds a rank-256 Markov head and a confidence head to the warm-started DFlash backbone — about 103M new parameters, growing the checkpoint from 4.76 GiB to 4.95 GiB (+4%) — then trains on 99,984 Muse-Glimmer-generated, on-policy OpenPerfectBlend conversations. Its main benefit is much stronger acceptance deeper into a 15-token proposal.

Results

The following results use the same BF16 Muse Glimmer 30B verifier, greedy decoding, and 15 speculative proposals per draft call. Higher accepted length is better.

Speculators evaluation subsets

Subset	Official DFlash	DSpark	Change
HumanEval	3.887	4.641	+19%
Math reasoning	3.867	5.237	+35%
QA	3.192	3.768	+18%
Question	2.963	3.441	+16%
RAG	4.048	5.182	+28%
Summarization	3.551	4.568	+29%
Tool call	3.087	3.805	+23%
Translation	3.627	4.948	+36%
Writing	2.974	3.455	+16%

Open-source benchmarks

Subset	Official DFlash	DSpark	Change
GSM8K	3.734	4.914	+32%
MATH500	4.139	4.869	+18%
MBPP	3.647	4.390	+20%
AIME 2026	3.870	4.271	+10%

Accepted length is calculated from the raw server counters:

accepted_length = 1 + accepted_tokens / draft_calls

Per-position acceptance

Per-position acceptance curves for the same runs as the tables above. DSpark shows slightly lower position-0 acceptance but substantially stronger acceptance deeper into the proposal, with the largest gains toward the tail.

Warm start

The inherited five-layer DFlash body is already trained, while the Markov and confidence heads are newly initialized. Applying the same 6e-4 peak learning rate to every parameter caused the warm-started body to lose some early-token accuracy during the high-LR phase. The released run uses:

DFlash body:             3e-4
Markov/confidence heads: 6e-4

Training recipe

Setting	Value
Verifier	Muse Glimmer 30B BF16
Initialization	Official Muse Glimmer five-layer DFlash assistant
Draft	5×Qwen3 GQA, causal SWA 2048, BF16
Hidden-state IDs	[2, 14, 26, 38, 50]
DSpark heads	Markov rank 256 + confidence head
Proposals	block_size=15, sample_from_anchor=true
Data	99,984 conversations / 148,900 train-turn rows
Reasoning mix	low 64,997; medium 10,000; high 19,991; xhigh 4,996
Packed length	16,384 tokens/rank
Anchors	up to 512 per packed sequence
Position decay	gamma 4
Training	3 epochs, TP4 producer + FSDP4 trainer, 8×H200
Optimizer	AdamW, weight decay 0.01, gradient clip 1
Peak LR	body 3e-4; Markov/confidence heads 6e-4
Schedule	cosine to zero, 4% warmup
Loss	0.1 CE + 0.9 TV; confidence BCE weight 1.0

Training data

DaoCloud/Muse-Glimmer-OPB-100K

Example: Run with vLLM

This is a drafter checkpoint, not a standalone language model. Pair it with Muse Glimmer 30B or a compatible quantized version:

vllm serve meta-models/Muse-Glimmer-30B \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser muse_glimmer \
  --reasoning-parser muse_glimmer \
  --speculative-config \
  '{"method":"dspark","model":"DaoCloud/Muse-Glimmer-30B-DSpark","num_speculative_tokens":15}'

Checkpoint

The BF16 checkpoint contains 62 tensors and is 5,318,887,946 bytes (4.95 GiB):

SHA-256  be0d03dd05d99049eac573a2654d0e65f86bc4beeedc1bb213798a506189240f

Notes

• This technical preview was trained and validated with patched development versions of vllm-project/speculators and vllm-project/vllm. Stock releases may not yet include the required Muse Glimmer / DFlash / DSpark support.
• The model was trained with a maximum proposal length of 15 tokens, but num_speculative_tokens is a serving-time tuning parameter. The optimal value should be determined based on workload, hardware, and latency.
• The confidence head is included in the checkpoint, but the tested vLLM runtime does not yet use it for confidence-scheduled verification.
• Multilingual, agentic, and multimodal speculative decoding has not yet been benchmarked and is left for future work.

Released under Apache-2.0, matching Muse Glimmer and its official DFlash assistant.