RedhatAI/GLM-5.3-speculator.dspark

Model Overview

• Model Architecture: DSparkDraftModel
  • Target Model Architecture: GlmMoeDsaForCausalLM
  • Input: Text
  • Output: Text
• Model Optimizations:
  • Speculative Decoding Algorithm: DSpark
  • Draft Model: 3-layer Qwen3-style backbone
  • Maximum Draft Length: 8 tokens
  • Auxiliary Target Layers: 2, 20, 39, 58, and 75
  • Markov Head: Vanilla, rank 256
  • Confidence Head: Enabled with Markov features
• Maximum Context Length: 1,048,576 tokens
• Release Date: 2026-09-16
• Version: 1.0
• Model Developers: Red Hat AI

This is a DSpark speculator model for GLM-5.3. It was trained using the Speculators library (version 0.7.0.dev108).

DSpark extends DFlash with a Markov head for modeling intra-block token dependencies and a confidence head for predicting per-position acceptance. The 3-layer draft model consumes auxiliary hidden states from layers 2, 20, 39, 58, and 75 of the target model and proposes up to 8 tokens per decoding step.

The model supports GLM-5.3's native context length of 1,048,576 tokens. In the 524K–1M-token MRCR evaluation bucket, it achieved a 41.16% token acceptance rate and an average accepted length of 4.293 tokens.

Training Details

This model was trained using the Speculators library. It was warm-started from a GLM-5.3 D-Spark checkpoint and trained for one complete epoch on 1,811,739 prepared 8,192-token sequences. Hidden-state extraction ran through vLLM while four FSDP ranks trained the draft model.

Commands

Prepare data

python scripts/prepare_data.py \
  --model zai-org/GLM-5.3 \
  --data ./data.jsonl \
  --output ./runs/glm53_dspark/data \
  --seq-length 8192

Launch hidden-state extraction

python scripts/launch_vllm.py zai-org/GLM-5.3 \
  --target-layer-ids 2 20 39 58 75 \
  -- --tensor-parallel-size 4 --max-model-len 8193 --port 8000

Launch training

torchrun --standalone --nproc-per-node 4 scripts/train.py \
  --verifier-name-or-path zai-org/GLM-5.3 \
  --from-pretrained <GLM-5.3-DSPARK-WARMSTART> \
  --data-path ./runs/glm53_dspark/data \
  --vllm-endpoint http://localhost:8000/v1 \
  --save-path ./runs/glm53_dspark/checkpoints \
  --epochs 1 --lr 1e-4 --total-seq-len 8192 \
  --speculator-type dspark --max-anchors 1024 \
  --target-layer-ids 2 20 39 58 75 \
  --loss-fn '{"ce":0.1,"tv":0.9}' \
  --confidence-head-alpha 1.0 --fsdp-shard \
  --on-missing generate --on-generate delete --checkpoint-freq 0.01

The released checkpoint completed 88,226 optimizer steps.

Model Specifications

Base Model	zai-org/GLM-5.3
Chat Template	zai-org/GLM-5.3 (use /chat/completions)
Architecture	DSparkDraftModel
Format	Safetensors, bfloat16
License	GLM-5.3 custom license
Draft Layers	3
Target Layer IDs	2, 20, 39, 58, 75
Draft Vocab Size	154880
Training Sequence Length	8192
Maximum Anchors	1024
Confidence Head	Vanilla Markov, rank 256
Validation Hardware	8x NVIDIA B300, TP=4

Deployment

vllm serve zai-org/GLM-5.3 \
  --tensor-parallel-size 8 \
  --speculative-config '{"model":"RedHatAI/GLM-5.3-speculator.dspark","num_speculative_tokens":8,"method":"dspark"}'

Acceptance Rates

Per-position token acceptance rates across datasets:

Dataset	Acceptance Length	Pos 0	Pos 1	Pos 2	Pos 3	Pos 4	Pos 5	Pos 6	Pos 7
HumanEval	4.76	85.4%	70.0%	56.9%	46.4%	38.1%	31.6%	26.2%	21.8%
math_reasoning	6.13	92.7%	83.9%	75.3%	66.4%	59.3%	51.8%	44.7%	38.3%
qa	3.18	74.5%	51.9%	34.8%	22.8%	14.8%	9.4%	5.9%	3.8%
question	3.20	73.1%	49.7%	33.6%	22.8%	15.9%	11.2%	8.0%	5.8%
rag	3.74	78.8%	59.5%	43.6%	32.0%	23.4%	16.8%	11.9%	8.2%
summarization	3.92	83.4%	65.3%	48.5%	35.5%	25.0%	16.8%	10.6%	6.4%
tool_call	4.01	79.2%	60.1%	45.3%	34.6%	26.9%	21.6%	17.9%	15.0%
translation	4.16	82.7%	66.1%	52.1%	39.6%	29.7%	20.8%	14.7%	10.2%
writing	3.19	72.8%	49.4%	33.4%	22.8%	15.9%	11.2%	8.1%	6.0%

Long-context acceptance

Acceptance on MRCR 2-needle prompts by input context length.

Context	Requests	Acceptance Rate	Acceptance Length
0–4K	15	75.0%	7.00
4K–8K	18	55.2%	5.42
8K–16K	21	47.9%	4.83
16K–32K	19	49.0%	4.92
32K–64K	20	40.6%	4.25
64K–131K	21	42.4%	4.40
131K–262K	21	37.5%	4.00
262K–524K	21	39.9%	4.19
524K–1M	19	41.2%	4.29

Performance Eval Against MTP

In the graph, interactivity is the median per-request output-token rate, aggregate throughput is total completed output tokens divided by measurement duration, and each label is completed requests divided by measurement duration.