Instructions to use zai-org/GLM-5.3-Flash with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zai-org/GLM-5.3-Flash with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="zai-org/GLM-5.3-Flash")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("zai-org/GLM-5.3-Flash")
model = AutoModelForMultimodalLM.from_pretrained("zai-org/GLM-5.3-Flash", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zai-org/GLM-5.3-Flash with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zai-org/GLM-5.3-Flash"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-5.3-Flash",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/zai-org/GLM-5.3-Flash

SGLang

How to use zai-org/GLM-5.3-Flash with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zai-org/GLM-5.3-Flash" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-5.3-Flash",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zai-org/GLM-5.3-Flash" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-5.3-Flash",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use zai-org/GLM-5.3-Flash with Docker Model Runner:
```
docker model run hf.co/zai-org/GLM-5.3-Flash
```

chat template: speed up tool-result validation and reordering (GLM-5.3-Flash)

#63

by ShijinZhang - opened 3 days ago

base: refs/heads/main

←

from: refs/pr/63

Discussion Files changed

+51

-55

ShijinZhang

Z.ai org 3 days ago

•

edited 2 days ago

Summary

This PR ports the tool-result validation and reordering optimization from GLM-5.3 Discussion #31, originally implemented by @vMaroon , to GLM-5.3-Flash.

The existing chat template uses nested loops and repeated macro calls to validate and reorder tool results. This introduces significant rendering overhead when processing large numbers of parallel tool calls.

The updated implementation uses a sort-merge approach to match tool results with their corresponding tool calls, while preserving the original ordering and fallback behavior.

The change is limited to the tool-result validation and reordering block in chat_template.jinja. It also avoids Python-specific list.index() calls to maintain compatibility with Jinja-compatible renderers.

Correctness Validation

Compared the original GLM-5.3-Flash chat template against the optimized implementation using randomized conversations and targeted edge cases.

Suite 1 — Jinja2 differential (Python 3.13, Jinja2 3.1.6, ImmutableSandboxedEnvironment with loopcontrols and a Transformers-compatible tojson adapter):

Test suite	Cases	Result
Randomized conversations	15,000	Byte-identical outputs
Additional successful edge cases	1,333	Byte-identical outputs
Total	17,500	0 behavioral mismatches

Tests cover shuffled, missing, duplicate and invalid IDs, case-sensitive and Unicode IDs, batched tool results, tool_reference outputs, multi-turn conversations, multimodal content, and different tool/reasoning configurations. Existing error cases remain unchanged.

Suite 2 — MiniJinja 2.24.0 differential (Python binding, pycompat=True, HF-compatible tojson): 15,000-case seeded corpus re-run under chainable-undefined semantics — 11,586 byte-identical successes + 3,414 identical-error cases, 0 mismatches.

Suite 3 — NVIDIA Dynamo production renderer (final gate): the real Rust dynamo-renderer 8.0.1 (crates.io) with minijinja + minijinja-contrib 2.24.0 — the exact versions in Dynamo main's Cargo.lock (verified via cargo tree) — rendering both templates through the full production path (PromptFormatter::from_parts incl. model-load probes, message normalization pipeline, OAIPromptFormatter::render) on the identical 15,000-case corpus, same result (0 mismatches).

Performance Benchmark

Local CPU-based Jinja2 template-rendering benchmark (single run for N≥100, median of 3 for smaller N; all benchmark inputs produced identical rendered outputs, SHA-256 verified):

Parallel Tool Calls	Before	After	Speedup
10	1.966 ms	0.803 ms	2.45x
100	137.6 ms	6.2 ms	22.1x
400	2,222.5 ms	24.3 ms	91.4x
800	8,650.7 ms	50.9 ms	170.1x

Compatibility Notes

The sort-merge implementation follows the final version of GLM-5.3 Discussion #31, which was independently tested by its author with NVIDIA Dynamo's MiniJinja renderer.
For this Flash-specific port, beyond the 17,500 Jinja2 differential tests, verification additionally includes the MiniJinja 2.24.0 Python-binding differential (15,000 cases) and the actual dynamo-renderer 8.0.1 Rust crate over the identical corpus, as detailed above.
Independent testing with the full Transformers apply_chat_template pipeline, vLLM, and SGLang has not yet been completed.
The change intentionally preserves existing template behavior and does not modify model weights, decoding kernels, or other message-rendering paths.

Acknowledgment

Thanks to @vMaroon for the original optimization in GLM-5.3 Discussion #31 and the detailed cross-renderer investigation.

ShijinZhang changed pull request status to open 3 days ago

chat template: speed up tool-result validation and reordering via sort-merge (port of GLM-5.3 #31)ed3a43a2

ZHANGYUXUAN-zR changed pull request status to merged 2 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment