autotrust/GLM-5.2-VL-bf16

GLM-5.2-VL (vLLM edition)

GLM-5.2 that can see — at full vLLM speed. This repository is the vLLM-native edition of GLM-5.2-VL: a vision encoder grafted onto the GLM-5.2 MoE language model. It serves through vLLM's standard multimodal pathway with an OpenAI-compatible API — image_url in, tokens out.

Note: GLM-5.2-VL is a community vision graft on top of GLM-5.2. It is not an official Zhipu AI / Z.ai release.

Two editions exist — pick by your hardware:

edition	LM precision	size	inference	best for
this repo	bf16 (byte-identical to stock GLM-5.2)	~1.4 TB	vLLM, ~80 tok/s decode, full batching	production serving
GLM-5.2-VL	bnb NF4 experts	~417 GB	transformers, ~2 tok/s	limited VRAM, experimentation

Evaluation

MMMU validation: 52.0% (468/900, zero-shot, greedy decoding, thinking enabled, all 30 subjects, no MMMU data anywhere near training).

MMMU discipline group	correct	accuracy
Humanities & Social Science	85/120	70.8%
Art & Design	67/120	55.8%
Health & Medicine	79/150	52.7%
Science	75/150	50.0%
Tech & Engineering	97/210	46.2%
Business	65/150	43.3%
Overall	468/900	52.0%

benchmark	score
text / code benchmarks	identical to GLM-5.2 (LM weights untouched)

How it works

image ─► Gemma-4 vision tower (frozen, fp32) ─► projection 1152→5376
      ─► connector MLP 5376→8192→6144 ─► soft tokens at <|image|> positions
      ─► stock GLM-5.2 (vLLM native GlmMoeDsa implementation, bf16)

• The language model is byte-identical to stock GLM-5.2 (the shards in this repo are the original files) and runs on vLLM's native MLA + DSA sparse attention + fused-MoE kernels. Text/code/reasoning behavior is exactly that of GLM-5.2.
• Vision enters via a small out-of-tree vLLM plugin (bundled, ~400 lines): each image becomes <|begin_of_image|> + n×<|image|> + <|end_of_image|> (n ≤ 1120, aspect-dependent), and the n pad positions receive connector embeddings through vLLM's standard multimodal merge. No custom kernels, no quantization hacks, no engine patches.

Quickstart

1. Install

pip install "vllm>=0.25"
pip install ./vllm_glm_vl-0.1.0-py3-none-any.whl     # bundled plugin (source in plugin_src/)

The plugin registers itself through the vllm.general_plugins entry point — no configuration needed.

2. Serve (OpenAI-compatible API)

./serve.sh                      # TP=8, port 8000
# or manually:
vllm serve /path/to/GLM-5.2-VL-vllm \
    --tensor-parallel-size 8 \
    --chat-template-content-format string   # REQUIRED, see "Gotchas"

3. Call it

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")

resp = client.chat.completions.create(
    model=client.models.list().data[0].id,
    messages=[{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "https://... or data:image/png;base64,..."}},
        {"type": "text", "text": "What is in this image?"},
    ]}],
    max_tokens=512,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(resp.choices[0].message.content)

Or test with the bundled scripts:

python test_client.py --image photo.jpg          # against a running server
python offline_example.py --image photo.jpg     # no server, offline LLM API

Thinking mode

GLM-5.2's <think>...</think> reasoning is available (chat_template_kwargs: {"enable_thinking": true}). For perception tasks (description / OCR / classification) we recommend thinking off — it is the most stable output format for visual inputs. With thinking on, give a generous max_tokens and parse the answer after </think>.

Gotchas

1. --chat-template-content-format string is required. GLM's chat template rewrites structured image content items into a "can't see images" disclaimer; string mode lets vLLM insert the <|image|> markers itself.
2. Images are auto-resized (aspect preserved) to ≤ 1120 visual tokens each. Multiple images per prompt are supported.
3. The plugin must be installed in the same environment as vLLM in all processes (it is loaded by every worker).

Evaluation

benchmark	score
MMMU (validation, 900 questions, zero-shot)	52.0%
text / code	identical to GLM-5.2 (LM weights untouched)

Hardware

1.4 TB of VRAM for weights + KV cache: 8× B300/GB300-class, or 16× 141 GB with --tensor-parallel-size 16. Not enough VRAM? Use the 4-bit transformers edition (440 GB).

Limitations

• Strongest on natural photos, documents, charts, screenshots and OCR. Heavily stylized content (anime / illustration / abstract art) is out-of-distribution and descriptions may be unreliable.
• Precise localization (boxes / points) is not supported.
• Inherits the biases and failure modes of its component models.

Repository layout

file	purpose
model-*.safetensors + index	stock GLM-5.2 bf16 shards + fp32 vision shard
vllm_glm_vl-0.1.0-py3-none-any.whl	the vLLM plugin (install this)
plugin_src/	plugin source code
serve.sh / test_client.py / offline_example.py	serving & smoke tests

License

Mixed: the GLM-5.2 language model is MIT; the vision encoder derives from Gemma-4 and remains subject to its original terms of use. You are responsible for compliance with both.