siro1/glm-5.2-dspark-preview

GLM-5.2 DSpark Speculator (preview — epoch 6)

A preview release of the DSpark draft (speculator) model for nvidia/GLM-5.2-NVFP4. This is the latest finished-epoch checkpoint (epoch 6, global step 183080) of the ongoing training run — trained ~3× longer than the v1 (epoch-2) release. An epoch-7 checkpoint is still in progress; this is the newest fully-completed epoch available.

It is not a standalone language model — it only produces useful output when used as the speculator in vLLM's --speculative-config, proposing 15 tokens per step to the GLM-5.2 verifier.

Architecture

Identical to v1: DSparkDraftModel — a 5-layer qwen3-backed draft attaching to the verifier at layers [8, 23, 39, 55, 70], with two heads on top of DFlash:

• Markov logit-bias head (rank 256) — per-position conditioning on previously sampled tokens within the block.
• Confidence head — per-position acceptance-probability estimate.

Block size 16, draft vocab 154880, mask_token_id 154856, bf16. See config.json / config.py.

How to use

Requires the DSpark vLLM fork (neuralmagic/vllm@dspark-speculators, commit 70cf932f7).

vllm serve nvidia/GLM-5.2-NVFP4 \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --all2all-backend flashinfer_nvlink_one_sided \
  --attention-backend FLASHINFER_MLA_SPARSE \
  --kv-cache-dtype fp8 \
  --speculative-config '{
    "method": "dspark",
    "model": "siro1/glm-5.2-dspark-preview",
    "num_speculative_tokens": 15,
    "draft_sample_method": "greedy",
    "attention_backend": "FLASH_ATTN"
  }' \
  --reasoning-parser glm45 --tool-call-parser glm47 --enable-auto-tool-choice \
  --trust-remote-code

fp8 KV-cache caveat: the draft uses FLASH_ATTN, which can't do fp8 KV cache on Blackwell. When serving the verifier with --kv-cache-dtype fp8, force the tiny (5-layer) draft to bf16 KV cache (patch vllm/v1/worker/gpu/spec_decode/dspark/utils.py so the draft config uses cache_dtype='auto'), or drop --kv-cache-dtype fp8.

Status

Preview — epoch-6 finished checkpoint (global step 183080). No separate validation pass is saved per-checkpoint in this run; acceptance is expected to be comparable to or better than the v1 epoch-2 release (mean acceptance length ~4.6, up to ~7 on math/code). training_state.json is included for provenance. Training continues toward epoch 10.

Training

speculators 0.6.0.dev0 against the GLM-5.2-NVFP4 verifier (vLLM 0.1.dev1+gee53abf1a, transformers 5.12.1, torch 2.11.0+cu129). 5 draft layers, block 16, target layers [8, 23, 39, 55, 70], Markov rank 256 + confidence head, loss {"ce": 0.1, "tv": 0.9}, lr 6e-4 cosine, 10 epochs, world size 20, verifier hidden states over Mooncake (P2P). See train_command.txt.

Compatibility

• Verifier: nvidia/GLM-5.2-NVFP4 (GlmMoeDsaForCausalLM)
• Runtime: neuralmagic/vllm@dspark-speculators @70cf932f7, built with TORCH_CUDA_ARCH_LIST=10.3 (Blackwell sm_103). Rebuild for other arches.
• Speculators lib: speculators >= 0.6.0.dev0 (provides DSparkSpeculatorConfig via auto_map).

License

MIT. Built on NVIDIA's GLM-5.2-NVFP4 and the ai-dynamo/speculators DSpark/DFlash implementation.