GLM-5.2-TR3-Vision

GLM-5.2-TR3-Vision combines the GLM-5.2 EXL3/Trellis TR3 3.0 bpw language checkpoint and quantized MTP-78 head with the GLM-5.2 MoonViT-3D vision tower and trained PatchMerger. It was live-tested through vLLM and a deployed Local Studio controller on four RTX PRO 6000 96 GB GPUs.

This is an experimental community composition, not an official Z.ai, Baseten, Brandon Music, Malaiwah, or VerdictAI release.

Source lineage

• Original model and license: zai-org/GLM-5.2
• EXL3/TR3 3.0 bpw text weights: brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw at revision 677eaa98b41b9f06abaad03953659c2872c22ce3
• Vision tower, projector, processor, and Glm5v configuration: baseten/GLM-5.2-Vision-NVFP4 at revision f6eab6117386a0c69152fdf272dc65bfd0254f9f
• Quantized MTP-78 overlay: malaiwah/GLM-5.2-EXL3-TR3-MTP78
• EXL3/Trellis runtime: verdictai/glm52-exl3-sparkinfer

The EXL3 language shards remain the routed-expert backbone. This repository adds vision_tower.safetensors and mm_projector.safetensors and exposes Glm5vForConditionalGeneration, with GlmMoeDsaForCausalLM as the nested language architecture. The vision payload contains 466,424,304 parameters (932,848,608 bytes).

Validated deployment profiles

Two separate recipes are saved on the deployed Local Studio controller. They intentionally use separate IDs so the fast text path is not mislabeled as multimodal.

GLM-5.2-EXL3 — text-only performance profile

Setting	Value
Recipe ID	glm-5.2-exl3
Served model	GLM-5.2-EXL3
Checkpoint	GLM-5.2-EXL3-TR3-3.0bpw
Runtime image	local/glm52-exl3-vision:v22-dcpstride
Parallelism	TP4, DCP4, ag_rs
Context	400,000 tokens
KV cache	FP8 MLA, 878,822 tokens, 2.20x concurrency at 400K
Scheduler	async, 2,048 max batched tokens, 8 max sequences
Speculation	MTP-3, greedy, EXL3 TR3 3.0 bpw layer 78
Collectives	NCCL; B12X PCIe all-reduce and DCP A2A disabled
Graph mode	FULL_AND_PIECEWISE CUDA graphs
API features	glm45 reasoning, glm47 tools
Runtime fix	pitched DCP workspace validation for partial prefill batches

EXL3-VISION — multimodal profile

Setting	Value
Recipe ID	glm-5.2-exl3-vision
Served model	GLM-5.2-VISION-EXL3
Checkpoint	GLM-5.2-EXL3-TR3-3.0bpw-Vision
Architecture	Glm5vForConditionalGeneration
Vision	BF16 MoonViT-3D plus trained PatchMerger
Runtime image	local/glm52-exl3-vision:v21
Parallelism	TP4, DCP4, ag_rs
Context	400,000 tokens
KV cache	FP8 MLA, 672,296 tokens, 1.68x concurrency at 400K
Scheduler	async, 2,048 max batched tokens, 8 max sequences
Speculation	MTP-3, greedy, EXL3 TR3 3.0 bpw layer 78
Collectives	NCCL; B12X PCIe all-reduce and DCP A2A disabled
Graph mode	FULL_AND_PIECEWISE CUDA graphs
API features	images, glm45 reasoning, glm47 tools

Both profiles used the publisher-tested physical rank order 3,1,2,0. Every GPU remained capped at 300 W.

The text runtime adds a one-line layout validation fix for partial DCP prefill batches. The projection input was already compacted before torch.bmm; the validator now accepts the actual pitched stride of the 2,048-token workspace. This keeps DCP4, project-before-merge, and CUDA graphs enabled under concurrent mixed-length requests.

Measured speed: text-only

Hardware: 4 x RTX PRO 6000 96 GB, power limit 300 W/GPU. The canonical decode number is a 30-second sustained, concurrency-1 streaming window after warmup.

Test	Prompt tokens	Total context	Prefill tok/s	Decode tok/s
Sustained C1	72	long streaming output	—	81.665
Context probe	8,192	8,320	1,531.25	91.14
Context probe	32,768	32,896	1,897.41	96.51
Context probe	65,536	65,664	2,420.73	102.18
Context probe	131,072	131,200	2,425.10	96.30
Context probe	262,144	262,272	2,482.96	103.19
Exact limit	399,872	400,000	3,525.94	102.57

The text sustained run measured 63.87% MTP acceptance and 0.418 s TTFT, averaged 267.74 W/GPU, and peaked at 294 W.

Measured speed: vision-capable wrapper

The canonical C1 decoder remained above the 70 tok/s target with vision enabled. The long-context rows use a deliberately synthetic repeated-token prompt and 128 generated tokens. Their decode rates are not representative of normal chat because that prompt collapses MTP acceptance; the prefill rates are valid cold-cache measurements.

Test	Prompt tokens	Total context	Prefill tok/s	Decode tok/s	MTP acceptance
Sustained C1	72	long streaming output	—	70.698	60.32%
Cold context probe	8,192	8,320	1,804.34	21.08	1.08%
Cold context probe	131,072	131,200	925.51	23.68	5.50%
Exact cold limit	399,872	400,000	852.54	16.91	0.27%

The vision wrapper is therefore coherent and fast enough in normal decode, but its long text-only prefill path remains slower than the dedicated text recipe. Use GLM-5.2-EXL3 when no image input is needed.

Functional validation

The following passed through the OpenAI-compatible API:

• algebra/reasoning response with a separate reasoning field;
• exact glm47 function selection and JSON arguments;
• tool-result continuation;
• MTP-3 acceptance with the quantized layer-78 draft;
• an actual 5120 x 2816 image payload through MoonViT and PatchMerger;
• OCR of the central headline, subheadline, and warning text;
• identification of the highlighted Workbench UI element when asked precisely;
• exact 400,000-token requests for both profiles.

Recipe metadata alone was not treated as proof of vision support.

Zero to running: vision profile

Requirements:

• Linux, Docker Engine, Compose, and NVIDIA Container Toolkit;
• four Blackwell GPUs with enough aggregate VRAM;
• roughly 300 GiB for weights plus runtime cache;
• the Hugging Face hf CLI.

curl -L \
  https://huggingface.co/0xSero/GLM-5.2-TR3-Vision/resolve/main/start.sh \
  -o start.sh
chmod +x start.sh
./start.sh

The launcher resumes the model download, builds the thin vision-plugin image on the pinned VerdictAI runtime, and serves GLM-5.2-VISION-EXL3 at http://127.0.0.1:8000.

The exact validated overrides are:

export MAX_MODEL_LEN=400000
export GPU_MEMORY_UTILIZATION=0.96
export MAX_NUM_BATCHED_TOKENS=2048
export ENABLE_ASYNC_SCHEDULING=1
export VLLM_ENABLE_PCIE_ALLREDUCE=0
export VLLM_USE_B12X_DCP_A2A=0
export DCP_COMM_BACKEND=ag_rs
export KV_CACHE_DTYPE=fp8

CUDA graphs remain enabled; do not add eager-mode flags.

Text request

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "GLM-5.2-VISION-EXL3",
    "messages": [{"role": "user", "content": "Explain why the sky is blue."}],
    "max_completion_tokens": 512
  }'

Image request

import base64
import requests

encoded = base64.b64encode(open("image.png", "rb").read()).decode()
response = requests.post(
    "http://127.0.0.1:8000/v1/chat/completions",
    json={
        "model": "GLM-5.2-VISION-EXL3",
        "messages": [{
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {
                    "url": f"data:image/png;base64,{encoded}"
                }},
                {"type": "text", "text": "Describe the image precisely."},
            ],
        }],
        "max_completion_tokens": 1024,
    },
    timeout=600,
)
response.raise_for_status()
print(response.json())

Runtime assertions

Resolved architecture: Glm5vForConditionalGeneration
Resolved architecture: DeepSeekMTPModel
quantization=exl3
enforce_eager=False
Using ['PYNCCL'] all-reduce backends
EXL3 rank-sliced runtime planned: Trellis ... topk=8
GPU KV cache size: 672,296 tokens
Application startup complete

Limitations

• The vision checkpoint requires the included vLLM plugin and pinned VerdictAI-derived runtime. Generic Transformers loading is unsupported.
• Other GPU topologies are untested.
• Vision is a graft, not a jointly retrained EXL3 multimodal checkpoint.
• Long text-only prefill is slower in the VLM wrapper; use the dedicated text recipe when images are not required.
• MTP acceptance depends strongly on the prompt and output distribution.
• The PCIe link attached to GPU 0 emitted correctable AER events under heavy multi-GPU load on the validation host; no uncorrectable error or Xid was observed in these runs.

License

The source model repositories report the MIT license. Preserve upstream attribution and review each source model card before redistribution or use.