0xSero/GLM-5.3-Flash-DSpine

GLM-5.3-Flash DSpine (research checkpoint)

A DSpine drafter for GLM-5.3-Flash. It proposes up to 15 tokens at a time for the target model to verify, so output stays the target's own. This is not a standalone language model.

This is an early research checkpoint, and it is currently slower than DFlash2. Use incoai/GLM-5.3-Flash-DFlash2 for serving. Training code is at 0xSero/dspine-trainer.

Results

Offline score: mean accepted drafted tokens per block on 120 held-out conversations (5,583 blocks), teacher-forced against GLM-5.3-Flash's greedy output.

Checkpoint	Offline score	First-token accuracy
Previous run, step 3,000	1.268	0.61
This checkpoint	1.313	0.62

Live results for this checkpoint. Setup: two DGX Sparks, TP=2, 262K context, draft policy fnc7:0.4.

Metric	No drafter	This checkpoint	DFlash2
Decode C1, prose (tok/s)	34.1	38.7	59.5
Decode C1, structured (tok/s)	33.8	39.5	107.9
Prefill at 32K (tok/s)	2,097	1,999	2,065
Prefill at 131K (tok/s)	2,022	1,932	1,997
Tokens per round, prose	1.00	2.15	3.25
Tokens per round, structured	1.00	2.47	7.32
Acceptance, prose	n/a	0.58	0.66
Acceptance, structured	n/a	0.49	0.98

Concurrency above 1 has not been tested. The DSpine serving port used for these numbers handles one stream and is not part of this repository.

Training data

All training text comes from the public Open-PerfectBlend dataset (Apache-2.0). No private, proprietary or user-supplied data was used.

The table covers the 8,889 captured conversations (12.9M tokens, 9.4M of them supervised assistant tokens) available when this checkpoint was trained. 447 of them were held out from training. A filter kept only conversations of about 1,000 tokens or more, which is why the mix differs from the full dataset.

Source inside Open-PerfectBlend	Conversations	Tokens	Share of tokens
mlabonne/ultrachat_200k_sft	6,650	10,031,744	77.84%
openbmb/UltraInteract_sft	1,494	1,796,160	13.94%
theblackcat102/evol-codealpaca-v1	335	532,544	4.13%
mlabonne/lmsys-arena-human-preference-55k-sharegpt	173	263,552	2.05%
HuggingFaceH4/ultrafeedback_binarized	145	173,760	1.35%
meta-math/MetaMathQA	37	42,880	0.33%
Post-training-Data-Flywheel/AutoIF-instruct-61k	37	37,888	0.29%
HuggingFaceH4/orca-math-word-problems-200k	8	8,064	0.06%
Server startup checks (fixed test tokens, see below)	10	370	<0.01%

How this was checked: each captured conversation was compared token by token with the public dataset. 8,879 are exact matches. The other 10 are two fixed requests the server sends itself at each of five boots: a "Reply with OK." health check and 64 random token ids used for timing.

An earlier set of 1,765 conversations was captured from the same public list, used in training, and overwritten by a bug. Those could not be compared directly.

The lmsys-arena source contains prompts written by real people, released publicly by LMSYS. It makes up 2% of the tokens.

The answers come from each dataset, not from GLM-5.3-Flash. They match GLM-5.3-Flash's greedy choice 68% of the time. The mix has little math or structured output, which may be why this drafter is weak on structured output. That has not been tested.

Training

• Inputs: the target's hidden states after layers 5, 14, 24, 33 and 42, plus the final state, captured during prefill on the Mia AI Lab EXL3 4 bpw quantization at revision 078455ff.
• Drafter: 5 layers, width 4096, block 16, 1.24B parameters, BF16.
• Steps: 4,600 at batch 4 with 64 anchors per sequence, in two runs (3,000 steps, then 1,600 of a 3,000-step continuation). That is about 0.6% of the training positions in the DSpine paper.
• Learning rates: first run 6e-4 for new parts and 2e-4 for the backbone; continuation 3e-4 and 1e-4.
• Hardware: one NVIDIA DGX Spark.

License

Weights were warm-started from incoai/GLM-5.3-Flash-DFlash2 (CC BY-NC-ND 4.0), and 58 of the 74 tensors started from it, so this checkpoint is treated as a derivative of it. The CC BY-NC-ND 4.0 tag and its non-commercial terms are kept here, with attribution to incoai. If you hold the rights to DFlash2 and object to this repository, please open a discussion and it will be taken down.

Credits: DSpine by Tencent, SpecForge, GLM-5.3-Flash by Z.ai, TensorFold, and Mia AI Lab's quantization.