Instructions to use autotrust/GLM-5.3-Flash-GGUF-DGX-Spark with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use autotrust/GLM-5.3-Flash-GGUF-DGX-Spark with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf autotrust/GLM-5.3-Flash-GGUF-DGX-Spark
# Run inference directly in the terminal:
llama cli -hf autotrust/GLM-5.3-Flash-GGUF-DGX-Spark

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf autotrust/GLM-5.3-Flash-GGUF-DGX-Spark
# Run inference directly in the terminal:
llama cli -hf autotrust/GLM-5.3-Flash-GGUF-DGX-Spark

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf autotrust/GLM-5.3-Flash-GGUF-DGX-Spark
# Run inference directly in the terminal:
./llama-cli -hf autotrust/GLM-5.3-Flash-GGUF-DGX-Spark

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf autotrust/GLM-5.3-Flash-GGUF-DGX-Spark
# Run inference directly in the terminal:
./build/bin/llama-cli -hf autotrust/GLM-5.3-Flash-GGUF-DGX-Spark

Use Docker

docker model run hf.co/autotrust/GLM-5.3-Flash-GGUF-DGX-Spark

LM Studio
Jan

vLLM

How to use autotrust/GLM-5.3-Flash-GGUF-DGX-Spark with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "autotrust/GLM-5.3-Flash-GGUF-DGX-Spark"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "autotrust/GLM-5.3-Flash-GGUF-DGX-Spark",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/autotrust/GLM-5.3-Flash-GGUF-DGX-Spark

Ollama
How to use autotrust/GLM-5.3-Flash-GGUF-DGX-Spark with Ollama:
```
ollama run hf.co/autotrust/GLM-5.3-Flash-GGUF-DGX-Spark
```
Unsloth Desktop

How to use autotrust/GLM-5.3-Flash-GGUF-DGX-Spark with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf autotrust/GLM-5.3-Flash-GGUF-DGX-Spark

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "autotrust/GLM-5.3-Flash-GGUF-DGX-Spark"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use autotrust/GLM-5.3-Flash-GGUF-DGX-Spark with Docker Model Runner:
```
docker model run hf.co/autotrust/GLM-5.3-Flash-GGUF-DGX-Spark
```

Lemonade

How to use autotrust/GLM-5.3-Flash-GGUF-DGX-Spark with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull autotrust/GLM-5.3-Flash-GGUF-DGX-Spark

Run and chat with the model

lemonade run user.GLM-5.3-Flash-GGUF-DGX-Spark-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use autotrust/GLM-5.3-Flash-GGUF-DGX-Spark with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf autotrust/GLM-5.3-Flash-GGUF-DGX-Spark

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default autotrust/GLM-5.3-Flash-GGUF-DGX-Spark

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use autotrust/GLM-5.3-Flash-GGUF-DGX-Spark with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf autotrust/GLM-5.3-Flash-GGUF-DGX-Spark

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "autotrust/GLM-5.3-Flash-GGUF-DGX-Spark" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

GLM-5.3-Flash-GGUF-DGX-Spark

GLM 5.3 Flash, 12 % smaller — Sparse-Squared compression with AutoTrust SLIM-Q, resident on a single 128 GB DGX Spark with room to spare.

This repository is the llama.cpp GGUF build of the SLIM-Q-compressed GLM 5.3 Flash: 79.1 GiB instead of the stock 90 GiB Q2, with 32 of 288 routed experts removed per layer (256 retained — the E256 member of the SLIM-Q family) and everything else untouched. It is the single-Spark companion to GLM-5.3-GGUF-DGX-Spark, the 744B SLIM-Q E192 build whose NVFP4 edition powers Guru Turbo 2.0. Runs in llama.cpp (glm5-next), so you get the OpenAI-compatible llama-server, continuous batching, tool calling and reasoning_content on the Spark.

SLIM-Q: AutoTrust's Sparse-Squared compression recipe

SLIM-Q — Selective expert pruning + Low-bit quantization for Inference of MoE — is AutoTrust's two-stage pipeline for post-training compression of frontier open-source MoE models. The idea in one line is Sparse-Squared (Sparse²) compression: a MoE is already sparse at inference (each token activates only the top-8 of 288 routed experts); SLIM-Q adds a second, structural sparsity axis — permanently removing the experts the router rarely uses — and then drives the surviving experts to low precision.

Stage 1 — Selective expert pruning (SLIM). Expert selection was profiled on a bilingual code / agent / science / maths calibration mix, and the 32 least-used of 288 routed experts per layer (11 %) were structurally removed. Attention (KDA + DSA), dense layers, shared experts, router, tokenizer and chat template are unchanged. Unlike dynamic expert-skipping, structural pruning permanently shrinks the weight footprint — on a 128 GB Spark, that is the difference between ~25 GiB and ~40 GiB of free unified memory after weights, i.e. between a 16–32 K context and a comfortable 64 K (or 4 × 16 K sessions).

Stage 2 — Low-bit quantization (Q). The pruned checkpoint is quantized aggressively on the routed experts (IQ2_XXS gate/up, Q2_K down) while attention, dense layers, shared experts, and routers stay at high precision — preserving the routing behavior that MoE quality depends on.

As on the 744B build, the per-token cost is untouched: routing still selects the top-8 of the remaining 256 experts, so compute and memory traffic per token equal the stock Flash Q2. Only the footprint drops.

	Stock GLM-5.3-Flash Q2 GGUF	This model
Size	90 GiB	79.1 GiB (−12 %)
Routed experts per layer (active per token)	288 (8)	256 (8)
Free unified memory on a 128 GB Spark after weights	~25 GiB	~40 GiB
Context that fits comfortably on a Spark	16–32 K	64 K, at most 1M
Decode speed	reference	same class (identical work per token)
Quality (see below)	reference	4-bit: HumanEval 97.6, C-Eval 89.4, GPQA-D 77.3; 2-bit harness A/B on par or better

Download

hf download autotrust/GLM-5.3-Flash-GGUF-DGX-Spark --local-dir ./GLM-5.3-Flash-GGUF-DGX-Spark

The model is stored as two GGUF shards (Hugging Face's 50 GB per-file limit); llama.cpp loads them together when pointed at the first one: GLM-5.3-Flash-Q2-DGX-Spark-00001-of-00002.gguf (41.9 GiB) + -00002-of-00002.gguf (37.2 GiB), 79.1 GiB total. Checksums in GLM-5.3-Flash-Q2-DGX-Spark.sha256.

DGX Spark quick start

GLM-5.3-Flash support is in llama.cpp pull request #27773 (glm5-next; not merged at the time of writing — once it is, plain master works).

git clone https://github.com/ggml-org/llama.cpp.git && cd llama.cpp
git fetch origin pull/27773/head:glm5next && git checkout glm5next
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=121a-real     # GB10
cmake --build build --config Release -j

# OpenAI-compatible API on :8080, 4 sessions x 16K, continuous batching
./build/bin/llama-server -m GLM-5.3-Flash-Q2-DGX-Spark-00001-of-00002.gguf -ngl 99 -fa on \
    -c 65536 -np 4 --cont-batching --host 0.0.0.0 --port 8080

# single long-context chat
./build/bin/llama-cli -m GLM-5.3-Flash-Q2-DGX-Spark-00001-of-00002.gguf -ngl 99 -fa on -c 65536

Keep the file on the internal NVMe; the first load reads 79 GiB. Stop other GPU work first.
Memory on the Spark: 79.1 GiB weights + ~1 GiB per 16 K tokens of context (KDA layers keep a constant state; the DSA layers use a compact cache) + a few GiB of compute buffers. -c 65536 leaves ~30 GiB free; -np 4 -c 65536 (4 × 16 K) is a good multi-user setting.
Thinking is on by default (the GLM template opens <think>). --reasoning-budget 0 disables it, --reasoning-budget 4096 caps it, --chat-template-kwargs '{"reasoning_effort":"low"}' selects the template's low / high / max effort. The server returns thinking separately as reasoning_content; tool calls come back as OpenAI tool_calls.
Expected decode speed on a Spark: the same class as the stock Flash Q2 — the per-token work is identical (8 experts + the same 8-bit attention). Decode is bound by the 273 GB/s LPDDR5X (about 11 GB of weights per token), so roughly 15–20 t/s single-stream; batching several sessions gives more aggregate throughput. Not measured by the author on a Spark; see the B200 table below for relative numbers.
Two Sparks (ConnectX link) are not needed for this file; it is a single-Spark model. For the 744B two-Spark build, see GLM-5.3-GGUF-DGX-Spark.
Also runs on: 128 GB Apple Silicon (cmake -B build without CUDA), discrete NVIDIA GPUs with ≥ 90 GB, or partially offloaded (-ngl N) on smaller cards.

Speed reference

llama.cpp, CUDA, one B200, model resident (llama-bench / llama-batched-bench, 256-token prompts, 128 generated tokens, flash attention):

	tokens/s
Prompt processing (pp512 / pp2048)	1037 / 1086
Generation, 1 sequence	54
Generation, 4 / 8 / 16 / 32 parallel sequences (aggregate)	130 / 170 / 207 / 253

Quality

SLIM stage (4-bit build with the same 256-expert selection, under vLLM on one B200): HumanEval 97.6 · C-Eval 89.4 · GPQA-Diamond 77.3 (low effort) · AIME 2025 74.2 · MMMU 76.1 · BFCL Non-Live 87.7 / Live 80.5 / multi-turn 73–75 — the pruning itself costs little.

Q stage (this 2-bit file) — DwarfStar's ds4-eval harness (GPQA Diamond, SuperGPQA, AIME 2025 interleaved; thinking on, 16 000-token budget, greedy), first 40 core cases, compared with the 2-bit 744B SLIM-Q E192 build on the same cases:

2-bit GGUF	Passed	Wrong	Out of budget
GLM-5.3-Flash-Q2-DGX-Spark (this file, 79 GiB; 256 experts)	35 / 40	3	2
GLM-5.3-Q2-DGX-Spark (744B SLIM-Q E192, 150 GiB; autotrust/GLM-5.3-GGUF-DGX-Spark)	30 / 40	1	9

An imatrix-guided build of the same layout scored 31 / 40 with twice the reasoning tokens, so this weight-energy-importance build is the one published. Qualitative checks: correct bilingual common-sense, medical and coding answers; no Chinese–English mixing.

Choosing between the two Spark builds: this Flash build closes more reasoning cases within budget and fits one Spark; the 744B E192 build keeps a clear edge on language modelling of agent/SWE trajectories and code (held-out PPL 6.2 vs 12.4 on agent traces, 3.1 vs 4.6 on SWE traces, 2.96 vs 3.17 on code) and needs two Sparks, a 180 GB GPU, or a big Mac.

What is in the file

Role	Type	Bytes
Routed experts gate / up (42 MoE layers × 256)	IQ2_XXS	46.5 GB
Routed experts down	Q2_K	29.6 GB
KDA linear attention, DSA attention, dense FFN, shared experts	Q8_0 (1.3 GB Q4_K)	8.2 GB
Embedding, output head	Q8_0	1.3 GB
Norms, routers, hyper-connections, indexer, k-pool compressor	F32	0.5 GB

45 layers (3 dense + 42 MoE) · KDA linear attention with DSA every fourth layer (k-pool indexer) · hyper-connections · top-8 of 256 routed experts + 1 shared · 154 880-token vocabulary · GLM 5.3 chat template with tool calling. Plain llama.cpp GGUF (architecture glm5-next). No MTP block; text model (no vision projector).

Shards

File	Bytes
GLM-5.3-Flash-Q2-DGX-Spark-00001-of-00002.gguf	44,984,916,000
GLM-5.3-Flash-Q2-DGX-Spark-00002-of-00002.gguf	39,944,534,144
GLM-5.3-Flash-Q2-DGX-Spark.sha256	checksums of both shards

Total 84,929,450,144 bytes (79.1 GiB). Split with llama-gguf-split --split-max-size 45G; to get a single file: llama-gguf-split --merge GLM-5.3-Flash-Q2-DGX-Spark-00001-of-00002.gguf out.gguf.

Limitations

2-bit routed experts on top of an 11 % expert pruning. Chat, coding and agent use are the target; expect a drop on the hardest maths/science reasoning versus the 4-bit deployment.
Needs the llama.cpp glm5-next branch until PR #27773 is merged. Two quirks of that branch are patched in llamacpp-pr27773-glm5next-lenient.patch (in the tooling repository): tolerate u64 metadata, and return a reply verbatim instead of HTTP 500 when max_tokens cuts it mid-UTF-8-character. Neither is required to run the model.
No MTP head, no vision projector in this file.

License and credits

Weights: MIT (base model); derivative of zai-org/GLM-5.3-Flash.
SLIM-Q pipeline and expert pruning (E256 selection): AutoTrust AI — the same Sparse-Squared compression recipe as the 744B GLM-5.3-SLIM-E192 behind Guru Turbo 2.0.
2-bit quantization recipe and quantizer: DwarfStar (antirez/ds4), on llama.cpp / GGML; glm5-next support by the llama.cpp PR #27773 authors.
Tooling and this build: https://github.com/yuhai-china/ds4-glm-slim

Downloads last month: 315

GGUF

Model size

279B params

Architecture

glm5-next

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Model tree for autotrust/GLM-5.3-Flash-GGUF-DGX-Spark

Base model

zai-org/GLM-5.3-Flash

Quantized

(115)

this model