abstract-extraordinary/Muse-Glimmer-30B-DSpark

Muse Glimmer 30B — DSpark speculative decoding head

Makes meta-models/Muse-Glimmer-30B generate 27–36% faster, with identical serving setup and no change to the target model.

It is a drop-in replacement for the DFlash drafter Meta ships alongside Muse Glimmer: same size, same one-forward-pass cost, better predictions. Point vLLM's --speculative-config at this instead and you get more tokens per verification step.

This is a drafter. It does nothing on its own - it must be served with the Muse Glimmer 30B target.

Speed

Measured on a single NVIDIA GB10 (DGX Spark), batch size 1–4. Each row is 48 prompts run against both drafters back to back, same server, same flags, greedy decoding with fixed output length.

Workload	Target	Meta's DFlash	This head	Speedup
Chat & reasoning	bf16	13.7 tok/s	18.6 tok/s	+36%
Chat & reasoning	NVFP4	31.3 tok/s	41.1 tok/s	+31%
Coding agents (tool use)	bf16	9.0 tok/s	11.4 tok/s	+28%
Coding agents (tool use)	NVFP4	18.8 tok/s	23.9 tok/s	+27%

Quantizing the target roughly doubles absolute throughput and costs almost nothing in relative speedup — accepted tokens per step are essentially precision-independent (+47% chat, +65% agents on NVFP4, against +47% and +60% on bf16). NVFP4 plus this head is the fastest configuration measured: 41 tok/s.

The underlying reason is that this drafter's guesses survive much further into each block. Per verification step it lands 6.7 tokens on chat (vs 4.6) and 6.6 on agent traces (vs 4.1) - roughly +47% and +60% more accepted tokens.

Agent workloads gain more tokens per step but less wall-clock, because long tool histories (~13k tokens here) make prefill, not decode, the bottleneck.

Long contexts

No degradation up to 16.6k, despite the 4,096-token training window. Measured within agent traces only, so content type is held constant:

context	DFlash	This head
7.6k–13k	3.96 tokens/step	6.18 (+56%)
13k–16.6k	4.32 tokens/step	7.09 (+64%)

Beyond 16.6k is untested.

Speculative decoding does not change what the model outputs, only how fast it arrives.

Use it

Muse Glimmer support is now in vLLM main (#51655, merged), but the DSpark path still needs one fix that did not land with it — without it method="dspark" raises AttributeError on this multimodal target before any weights load:

# vllm/v1/worker/gpu/spec_decode/dspark/utils.py
-    target_inner = target_language_model.model
+    target_inner = getattr(target_language_model, "model", target_language_model)

Apply that to a main checkout, or clone a branch that is main plus exactly that commit:

git clone -b muse-glimmer-dspark https://github.com/stepnivlk/vllm.git

Then:

vllm serve meta-models/Muse-Glimmer-30B \
  --speculative-config '{"method":"dspark","model":"/path/to/Muse-Glimmer-30B-DSpark","num_speculative_tokens":16}' \
  --reasoning-parser muse_glimmer --tool-call-parser muse_glimmer --enable-auto-tool-choice \
  --compilation-config '{"cudagraph_mode":"PIECEWISE"}'

num_speculative_tokens must be 16. That is the drafter's block size. Higher values are untrained territory.

If acceptance looks catastrophically bad (~12% instead of ~85% at the first position), the layout is wrong: config.json must keep sample_from_anchor: false. That mismatch fails silently rather than erroring.

How it works

Meta's DFlash drafter predicts a whole 16-token block in one parallel pass, so every position in the block is guessed without knowing which token was actually chosen at the position before it. Deep guesses are therefore near coin flips.

DSpark adds a small transition head that conditions each position on the token sampled at the previous one, and this head's backbone was fine-tuned to use that signal. The effect is entirely at depth:

chance a guess at position N is accepted (agent traces)
position        0    1    2    3    4    5    6    7    8    9   10   11
Meta's DFlash 84%  65%  49%  38%  25%  17%  12%   8%   6%   4%   3%   2%
this head     82%  68%  57%  49%  43%  38%  35%  33%  30%  27%  25%  23%

First-position accuracy is unchanged. Everything after it improves, and the tail stops collapsing — which is where the extra tokens per step come from.

How it was trained

Started from Meta's published drafter and fine-tuned its 5-layer backbone plus a rank-256 transition head (2.2B trainable) on 3,891 conversations that Muse Glimmer generated itself. 150 steps, ~83 minutes on one RTX PRO 6000 Blackwell.

Training on the target's own output matters: a transition head learns model-specific token statistics, so text from another model teaches the wrong ones.

embed_tokens and lm_head are deliberately absent from this repo — vLLM shares the target's copies. They were frozen during training for that reason.

License

Apache 2.0, inherited from meta-models/Muse-Glimmer-30B-assistant, which this head is fine-tuned from. Muse Glimmer's usage policy applies to the target you serve it with. The training corpus was generated with Muse Glimmer 30B; training used DeepSpec.