zai-org

/

GLM-5.3-Flash-BF16

like 48

Follow

Z.ai 19.8k

Hello, and thank you for releasing both the FP8 and BF16 weights — having the BF16 published is what made everything below possible.

Over the last two days several of us have been quantizing GLM-5.3-Flash independently. I've measured the results on a single shared panel so the numbers are actually comparable, and I thought the results — and one small metadata observation about this repo — might be useful to you and to people landing here.

Method (so the numbers mean something)

All rows below: KL divergence D(teacher ‖ student), fp64 accumulation, full-vocabulary, on a sealed 25-window / 51,175-position panel, scored against fp32 teacher logits captured from your BF16 weights. Panel and teacher were published by @brandonmusic . Same panel, same teacher, same direction, same estimator for every row — otherwise these figures would not be comparable to each other.

Quantization	Mean KLD (nats)	Size	Measured by
malaiwah TR3 8bpw	0.012384	331 GB	us
malaiwah TR3 6bpw	0.013723	254 GB	us
brandonmusic TR3 4bpw	0.024555	176 GB	author
0xSero EXL3 Q4	0.027263	188 GB	us

Lower is closer to your BF16 weights. Every figure links to a JSON receipt with pinned revisions and sha256s; the 6bpw and Q4 rows are five and five cold runs respectively, bitwise identical, and the 8bpw is two.

Your FP8 release, measured — with an honest caveat

I measured zai-org/GLM-5.3-Flash on the same panel at 0.020615 nats (top-1 95.6%). I am deliberately not putting it in the table above, because I captured it through a different serving stack than the replay path used for the rows above, and that difference carries its own cost: replaying your BF16 weights through that same cross-stack path — i.e. no quantization at all — already scores 0.012712. So the FP8's own quantization-attributable cost on this panel is roughly 0.008 nats, and the raw 0.020615 should not be ranked directly against the table. A same-lane floor measurement is running now and I will post the corrected figure when it lands.

I mention it because it may be genuinely useful to you: it is an independent, receipted measurement of your released FP8 against your own BF16.

The metadata observation

zai-org/GLM-5.3-Flash and zai-org/GLM-5.3-Flash-BF16 are published as two independent roots — neither declares the other via base_model. The practical consequence is that quantizations of the same model split across two disconnected trees: some of us declare BF16 as our base (because that is what we quantized from), while e.g. orcarouter's MLX builds declare the FP8 release. Both declarations are accurate, but the result is that no single page lists all quantizations of GLM-5.3-Flash, and this BF16 repo — the one quant authors actually work from — shows very little of the ecosystem built on it.

If it were useful, linking the two repos to each other (or noting the sibling in each card) would make that ecosystem discoverable from either page. Entirely your call, of course — just flagging it as something only you can fix.

Everything is reproducible

• All receipts, panels and comparisons: quant-fidelity-registry — a schema'd registry where a measurement cannot be recorded without its panel, teacher, run count and disclosed deviations, so cross-panel numbers can't be silently mixed.
• Tools, campaign log and every patch: github.com/malaiwah/glm53-flash-fidelity-suite
• Measuring a quant yourself costs roughly $6 of rented GPU time with the streaming scorer in that repo; the recipes are copy-paste.

Full disclosure: I am the author of the two malaiwah quants above, so please read the table with that in mind — that is exactly why every number links to a receipt and why the method is stated in full. Corrections very welcome; if anything here is wrong I would rather know.

Thank you again for the model, and for publishing the BF16 weights alongside it.

great work sir! I think the kld of the fp8 (even though its natively trained in that from what i understand) seems to be because its w8a8 per their blog.

Following up as promised — the same-lane floor measurement has landed.

The floor: 0.011506 nats

Scoring your BF16 weights themselves — no quantization at all — against the teacher on this panel costs 0.011505922619330299 nats (full 25 windows / 51,175 positions, fp64, two cold runs producing identical means). That is the price of the comparison itself: the teacher logits were captured on a different runtime than the replay lane, and bf16 addition is not associative across differing expert-combine orders.

Subtracting it turns raw divergence into quantization-attributable error:

The two quants' raw means differ by only 1.11x, but their attributable error differs by 2.52x. Raw KLD understates differences between good quants, because the floor is common to both — worth knowing for anyone comparing quantizations of this model.

On your FP8 release — the correction, stated precisely

I want to be exact about what did and did not change, since I flagged this above.

The 0.020615 figure I quoted was captured cross-stack, and I subtracted the matching cross-stack floor (0.012712, our BF16 replay through that same path) to get ~0.008 nats attributable. That subtraction was already like-for-like, so the ~0.008 figure stands unchanged.

What I cannot do is subtract the new 0.011506 floor from it — different lane, invalid comparison. Producing a strictly same-lane number for your FP8 release means measuring it on the streaming lane.

Method and receipts

Floor write-up, including how not to use it: k6/BF16-FLOOR.md. Receipts: native-bf16-kld.json. Every row above is in the quant-fidelity-registry.

One caveat I would flag for anyone reading this thread later: a quant scoring at the floor is not "perfect" — it means this panel can no longer resolve its error, and a harder panel or a same-stack teacher would be needed to see further.

Thank you — and you've put your finger on it. The repo's own config.json confirms your read: quantization_config has fmt: "e4m3" with activation_scheme: "dynamic" — so the FP8 release is W8A8, activations quantized dynamically per token at serve time, not just weights.

And that gives us a nice decomposition, because our K8 is the near-perfect control: same byte budget (331 vs 328 GB), weight-only vs W8A8.

So at the 8-bit weight budget, weight quantization itself costs almost nothing — the remaining ~8–9× is dominated by the activation quantization plus the weight-format difference (blockwise fp8 vs Hessian-aware trellis allocation). Caveat stated plainly: the two attributable figures come from different measurement lanes, each with its own matching floor subtracted, so the comparison is like-for-like within each lane and approximate across them.

Two things worth noting alongside your point:

1. Dynamic a8 error is serve-time and input-dependent — no amount of better weight calibration removes it. That's why a weight-only quant at the same footprint can beat it: it simply isn't paying that tax.
2. On "natively trained in fp8" — if their blog's fp8-training claim holds, then the W8A8 cost we measured is the model's native serving regime, not a post-hoc degradation of it. In that reading the FP8 release isn't a "worse quant" — it's the intended operating point, and the BF16 (presumably the master weights) is the reference that lets us measure what that regime costs: ~0.008 nats on this panel.

Also a small aesthetic point I enjoyed: their modules_to_not_convert list keeps the KDA/attention path, hyper-connections, norms, embeddings and lm_head in native precision — essentially the same scope split TR3 uses (quantize routed experts + MTP, keep everything else bit-exact). Two independent teams drew the sensitivity boundary in the same place.