fiore-lab/GLM-5.3-EXL3-Experts-5bit

English | 日本語 | 简体中文

GLM-5.3, EXL3 routed experts at 5 bits: an Expert Atlas bank

The routed experts of zai-org/GLM-5.3 (755B, glm_moe_dsa), every one of them at 5 bits, quantized by Fiore-lab from zai-org/GLM-5.3-BF16 at revision 9d2398f. This is one of four banks (2, 3, 4 and 5 bits) from which the recipe's Expert Atlas loads each expert at the width you choose: https://github.com/fiore-labo/fiore-glm5.3 (release r3-102 or later). It holds no other weights: the attention, shared experts, embeddings and the MTP layer come from the recipe's checkpoint (fiore-lab/GLM-5.3-EXL3-8x-DGX-Sparks or fiore-lab/GLM-5.3-EXL3-4x-DGX-Sparks, whose other weights are the same).

What is in it

Files	What they hold
experts-layer-003 … 077.safetensors (75)	the routed experts of layers 3 to 77: EXL3, 5 bits, codebook mcg, each expert already cut into its eight tensor-parallel shares (for four or eight DGX Sparks)
glm53-exl3-experts-5bit.json	the bank's manifest: every file's size, header and sha256, which the recipe checks before it loads a layer
LICENSE	the GLM-5.3 License

459.1 GB (427.6 GiB) in all.

How it was made

With exllamav3 1.6.1: routed experts at 5 bits, codebook mcg, calibrated with exllamav3's default calibration rows (English, Chinese) plus Japanese text and agent / coding sessions (320 rows of 2,048 tokens); then cut into eight tensor-parallel shares by whole tiles (no requantization).

Accuracy

KL divergence from the original model's output distributions (the official BF16 weights run by the official implementation, transformers, computed in FP32), with every routed expert at one width and the recipe's other weights, on four DGX Sparks (* on eight), over four 8,192-token texts (two English, two code), positions 6,144 to 8,190:

Routed experts	KL divergence
all 2 bits	0.0517
all 3 bits	0.0203
all 4 bits	0.0127
all 5 bits	0.0114 *

For scale, the official BF16 weights computed in BF16 instead of FP32 differ from the same reference by about 0.009. Mixing widths at the same memory, a mix with 2-bit experts was 13 to 30% further from the reference than one without; 5-bit experts paid for by moving others down to 3 bits gained nothing measurable; added memory for them does.

How to use it

1. Download it on every node into the directory the recipe's expert-plan/banks.json names for 5 bits: hf download fiore-lab/GLM-5.3-EXL3-Experts-5bit --local-dir ~/models/GLM-5.3-EXL3-Experts-5bit. The manifest comes with it. For another directory, change this bank's dir in expert-plan/banks.json, then tf-glm53 deploy.
2. tf-glm53 verify-banks 5 checks every file against the manifest on every node (minutes, at low priority).
3. The control panel's launch screen offers 5 bits once every node holds this bank; a start or a re-sort that needs a width some node lacks is refused before anything changes. In release r3-102 the control panel runs on four DGX Sparks.

License

The GLM-5.3 License (LICENSE), Copyright (c) 2026 Z.AI, as for the base model; see the recipe's checkpoint card for its terms in full. The recipe's code is Apache-2.0 and licensed separately.