Instructions to use imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy")
model = AutoModelForMultimodalLM.from_pretrained("imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy

SGLang

How to use imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy with Docker Model Runner:
```
docker model run hf.co/imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy
```

GLM-5.3-Flash 9B Surgery Dummy

Test-only checkpoint for GLM-5.3-Flash post-training integration. It is not a usable chat or benchmark model. For cheaper smoke tests, use GLM-5.3-Flash-0.1B-A0.1B.

GLM-5.3-Flash branches: VERL, Slime, SGLang, and Megatron-LM.

Known issue: the compatibility-only visual stub has intermediate_size=64, so TP=4 produces a 16-wide partition that is not divisible by the FP8 block size 128; keep model.visual.* in BF16 by excluding it from FP8 conversion.

The checkpoint keeps the original text width but reduces 45 decoder layers to 10 and 288 routed experts to 32, for 8,895,622,684 text parameters. Vision is disabled. Surgery provenance is in surgery_plan.json and surgery_manifest.json.

Experimental text-only test model. Not an official Z.ai release and not yet quality-recovered. Do not use it as a production or benchmark model.

This checkpoint preserves GLM-5.3-Flash width/kernel geometry while reducing the decoder from 45 to 10 layers and each routed MoE from 288 to 32 experts. It has 8,895,622,684 text parameters. Source layers are [0, 1, 2, 3, 8, 18, 25, 31, 38, 44]. Each target routed expert is a four-donor functional mosaic: 512 individually selected, coupled SwiGLU units come from each donor (gate/up rows plus matching down columns), a closed-form down-projection scale matches synthetic output variance, and all three expert matrices receive fresh per-128x128 FP8 E4M3 scales. Router rows use balanced router-space clusters. No donor forward pass, activation cache, distillation, or parameter training is used. Student-only evaluation remains required before calling the model functionally useful.

Vision is intentionally disabled. A zeroed 49,056-parameter visual compatibility stub exists only because the stock Transformers wrapper currently constructs a visual submodule. It is not a vision model.

The exact source revision, tensor map, expert clusters, and provenance hashes are stored in surgery_plan.json; output shard hashes are in surgery_manifest.json.

Downloads last month: 275

Safetensors

Model size

9B params

Tensor type

BF16

F32

F8_E4M3

Model tree for imvladikon/GLM-5.3-Flash-9B-Surgery-Dummy

Base model

zai-org/GLM-5.3-Flash

Quantized

(65)

this model