Instructions to use UCloud-org/GLM-5.2-FP8-DFlash with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use UCloud-org/GLM-5.2-FP8-DFlash with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="UCloud-org/GLM-5.2-FP8-DFlash", trust_remote_code=True)

# Load model directly
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("UCloud-org/GLM-5.2-FP8-DFlash", trust_remote_code=True)
model = AutoModel.from_pretrained("UCloud-org/GLM-5.2-FP8-DFlash", trust_remote_code=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use UCloud-org/GLM-5.2-FP8-DFlash with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "UCloud-org/GLM-5.2-FP8-DFlash"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "UCloud-org/GLM-5.2-FP8-DFlash",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/UCloud-org/GLM-5.2-FP8-DFlash

SGLang

How to use UCloud-org/GLM-5.2-FP8-DFlash with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "UCloud-org/GLM-5.2-FP8-DFlash" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "UCloud-org/GLM-5.2-FP8-DFlash",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "UCloud-org/GLM-5.2-FP8-DFlash" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "UCloud-org/GLM-5.2-FP8-DFlash",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use UCloud-org/GLM-5.2-FP8-DFlash with Docker Model Runner:
```
docker model run hf.co/UCloud-org/GLM-5.2-FP8-DFlash
```

GLM-5.2-FP8-DFlash

Paper | DFlash GitHub | SpecForge

DFlash block-diffusion speculative-decoding drafter for GLM-5.2-FP8 (743B MoE, 39B active). Standard DFlash method (no extensions), trained with SpecForge on the paper-specified data recipe: ~800K samples of Nemotron-Post-Training-v2 + CodeAlpaca (code / math / chat), all responses regenerated by GLM-5.2-FP8, 6 epochs — directly comparable to the DFlash paper and the z-lab drafter series.

Quick Start (SGLang)

python -m sglang.launch_server \
    --model-path zai-org/GLM-5.2-FP8 \
    --speculative-algorithm DFLASH \
    --speculative-draft-model-path UCloud-org/GLM-5.2-FP8-DFlash \
    --speculative-num-draft-tokens 16 \
    --tp-size 8 \
    --trust-remote-code

vLLM v0.20.1+ has native DFlash support and reads this checkpoint directly, no conversion needed. Not yet runtime-verified on our infrastructure; the results below were produced via SGLang.

Evaluation

Mean accepted length & end-to-end speedup

Measured on live SGLang serving (concurrency 1, greedy decoding unless noted).

Benchmark	AL (built-in MTP)	AL (DFlash)	DFlash throughput (tok/s)	Speedup vs vanilla	vs built-in MTP
gsm8k	4.01	4.44	236	2.22x	1.51x
humaneval	4.42	6.43	383	3.44x	1.42x
math500	4.91	7.77	477	4.28x	1.54x
mbpp	5.23	8.04	478	4.29x	1.46x
mtbench	3.71	3.56	220	1.99x	0.93x
ceval	3.65	2.98	177	1.62x	0.93x

Built-in MTP baseline uses the official GLM-5.2 recipe (EAGLE, steps 5 / topk 1 / draft tokens 6). This drafter is code/math-optimized: it delivers 1.4-1.5x over the (already strong) built-in MTP on code and math workloads, while chat and Chinese-language workloads slightly favor built-in MTP (see Limitations).

ceval (Chinese) is the weakest domain — the training corpus is English-dominant (see Limitations).

Also mirrored on ModelScope: UCloud-AILab/GLM-5.2-FP8-DFlash.

Training Details

Target model: GLM-5.2-FP8 (hidden 6144, 78 layers; drafter conditions on target layers [1, 20, 38, 56, 75])
Drafter: 5-layer block-diffusion transformer, block_size 16, 3.7B total parameters (1.8B independently trained; embed/lm_head reused from target, frozen, not trained, included for standalone inference)
Data: JessieWei/GLM-5.2-FP8-nemotron-codealpaca — Nemotron-Post-Training-v2 + CodeAlpaca, ~800K samples (paper-specified recipe), all responses regenerated by GLM-5.2-FP8 (non-thinking mode), max_length 3072
Recipe: 6 epochs, AdamW, lr 6e-4 cosine (4% warmup), grad-clip 1.0, num_anchors 512, loss_decay_gamma 7, pure cross-entropy (standard DFlash loss)
Framework: SpecForge (offline hidden-state pipeline), FSDP2

Limitations

Trained on non-thinking-mode regenerated data; speedup under thinking-mode inference has not been evaluated yet.
Training corpus is English-dominant: acceptance length on Chinese-language workloads is lower (ceval AL 2.98 vs 4.4-8.0 on English benchmarks).

Acknowledgements

DFlash (z-lab), SpecForge / SGLang (sgl-project). GLM-5.2 by Zhipu AI.

Citation

@misc{chen2026dflash,
  title         = {DFlash: Block Diffusion for Flash Speculative Decoding},
  author        = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  year          = {2026}, eprint = {2602.06036}, archivePrefix = {arXiv},
  primaryClass  = {cs.CL}, url = {https://arxiv.org/abs/2602.06036}
}

Downloads last month: 20

Safetensors

Model size

4B params

Tensor type

BF16

Model tree for UCloud-org/GLM-5.2-FP8-DFlash

Base model

zai-org/GLM-5.2-FP8

Finetuned

(4)

this model

Dataset used to train UCloud-org/GLM-5.2-FP8-DFlash

Paper for UCloud-org/GLM-5.2-FP8-DFlash

DFlash: Block Diffusion for Flash Speculative Decoding

Paper • 2602.06036 • Published Feb 5 • 89