Instructions to use malaiwah/GLM-5.2-SIQ-Fruit-bf16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use malaiwah/GLM-5.2-SIQ-Fruit-bf16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="malaiwah/GLM-5.2-SIQ-Fruit-bf16")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("malaiwah/GLM-5.2-SIQ-Fruit-bf16")
model = AutoModelForCausalLM.from_pretrained("malaiwah/GLM-5.2-SIQ-Fruit-bf16", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use malaiwah/GLM-5.2-SIQ-Fruit-bf16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "malaiwah/GLM-5.2-SIQ-Fruit-bf16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "malaiwah/GLM-5.2-SIQ-Fruit-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/malaiwah/GLM-5.2-SIQ-Fruit-bf16

SGLang

How to use malaiwah/GLM-5.2-SIQ-Fruit-bf16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "malaiwah/GLM-5.2-SIQ-Fruit-bf16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "malaiwah/GLM-5.2-SIQ-Fruit-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "malaiwah/GLM-5.2-SIQ-Fruit-bf16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "malaiwah/GLM-5.2-SIQ-Fruit-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use malaiwah/GLM-5.2-SIQ-Fruit-bf16 with Docker Model Runner:
```
docker model run hf.co/malaiwah/GLM-5.2-SIQ-Fruit-bf16
```

GLM-5.2-SIQ-Fruit-bf16

The plain-BF16, CPU-runnable twin of GLM-5.2-SIQ-Fruit. It contains the same QNOISE-annealed 5.04B-parameter checkpoint—0.46B parameters active per token—but replaces the custom SIQ expert representation with ordinary BF16 tensors.

This repository loads with stock Transformers and needs no GPU. It is the reference path for model behavior and CPU integration; it does not exercise SIQ/Trellis dequantization, sparse DSA attention, low-precision KV caches, or MTP speculative decoding.

Measured CPU run

Intel Core i7-14700K, 20 Torch threads, BF16, Transformers 5.14.1; 128-token greedy decode after a 16-token warmup:

measurement	result
decode	33.12 tok/s
warm-cache load	2.51 s
resident memory after load	16.01 GiB RSS
peak resident memory	16.76 GiB RSS

An earlier card estimated “about 10 GB RAM.” That was not an end-to-end process measurement and has been withdrawn.

Usage

import torch
from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer

model_id = "malaiwah/GLM-5.2-SIQ-Fruit-bf16"
torch.set_num_threads(20)

config = AutoConfig.from_pretrained(model_id)
assert config.rope_theta == config.rope_parameters["rope_theta"] == 500000.0

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    config=config,
    dtype=torch.bfloat16,
).eval()

inputs = tokenizer("Once upon a time", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=40, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Expected output begins with a TinyStories-style continuation about a little girl named Lily; exact wording can vary with library kernels and version.

What loads

Exercised: dense MLA attention, GLM tokenizer, and the real 256-expert top-8 MoE weights.
Ignored by stock Transformers: DSA indexer tensors and the MTP draft layer. They are reported as unexpected keys. This is intentional for this compatibility path; 4.57B of the 5.04B parameters load.
Production path: use the SIQ release with a compatible b12x/SparkInfer + vLLM build to exercise Trellis experts, sparse MLA, fp8/nvfp4 KV, and MTP.

Release correction

Revision b97f91d222906142e46827a010a5caf5a9e35928 encoded the trained RoPE theta incorrectly: it retained 8,000,000 in the nested field and omitted the legacy top-level field. The 2026-08-07 correction writes 500,000 to both locations and regenerates MANIFEST.sha256. Weight shards did not change. The fail-closed assertion in the usage example protects pinned or cached stale configurations.

The export also carries the same trainer-to-serving RoPE channel permutation and MTP eh_proj input-half conversion as the SIQ artifact; those transformations preserve the trained function before stock Transformers drops the unsupported indexer/MTP modules.

Limitations

The model is a serving proxy and CI fixture, not a general assistant.
CPU generation uses dense attention and therefore does not validate the DSA indexer or long-context sparse-attention path.
The measured speed and memory numbers are for the exact hardware/software setup above; they are not hardware-independent guarantees.

Reproducibility and integrity

MANIFEST.sha256 authenticates all serving files except the card and Git attributes. The exporter invocation uses FRUIT_BF16=1; source checkpoint, training recipe, gauntlet, and review evidence are documented in proxy-fruit and the base model card.

Downloads last month: -

Safetensors

Model size

5B params

Tensor type

BF16

Model tree for malaiwah/GLM-5.2-SIQ-Fruit-bf16

Base model

malaiwah/GLM-5.2-SIQ-Fruit

Finetuned

(1)

this model