tiyuvta/GLM-5.3-Flash-NVFP4-4o6

GLM-5.3-Flash NVFP4 (Four-Over-Six)

NVFP4 (4-bit e2m1 weights with an FP8-e4m3 per-16 scale plane, 4.5 bits/element) weight-only mint of zai-org/GLM-5.3-Flash, produced with NVIDIA TensorRT Model Optimizer 0.46.0. Same keep-list and compressed-tensors layout as tiyuvta/GLM-5.3-Flash-NVFP4. The numeric change is Four-Over-Six (per-block MSE, E4M3 max 256 not 448).

Built with memra, a from-scratch Rust and CUDA inference engine for RTX Blackwell, by tiyuvta (inference.tiyuvta.ai).

Run GLM-5.3-Flash today through an API. Open GLM-5.3-Flash on tiyuvta and use model id zai/glm-5.3-flash. That endpoint still serves the earlier mint (tiyuvta/GLM-5.3-Flash-NVFP4 @ 2bade47d). This 4/6 remint is a candidate for that pin, not the live bytes. Speculative decoding on the live endpoint is drafted by DFlash 2 (inco.ai). Get an API key and send the first request.

• 190.7 GB across 20 safetensors shards. 38,770 source tensors: 37,338 quantized, 1,432 kept.
• Index: 113,446 weight-map entries = 1,432 kept + 3 x 37,338 quantized (weight, weight_scale, weight_scale_2).
• Source is the vendor's BF16 twin, not their FP8 release. Never quantize from a quant when the full-precision twin ships.
• Sibling for B200 (Puzzle Table 2 hybrid, different layout): tiyuvta/GLM-5.3-Flash-NVFP4-B200-hybrid.
• Upstream technical report: GLM-5: from Vibe Coding to Agentic Engineering (arXiv 2602.15763).

How it was made

Source	zai-org/GLM-5.3-Flash-BF16 @ f12e0fe1f6b2ea274c11a569582edfd99d993c5e (656 GB)
Tool	nvidia-modelopt 0.46.0, W4A16_NVFP4
Scheme	weight-only. e2m1 weights, dynamic per-16 block scales in e4m3, per-tensor f32 macro scale. Activations untouched
Calibration	Four-Over-Six, no activation forwards. Per-block MSE picks M=6 (amax x 1.0) or M=4 (amax x 1.5). E4M3 normalization max is 256, not 448 (NVFP4_FOUR_OVER_SIX_CFG / Lightning w4a16_nvfp4_4o6). No prompts, no activation statistics
Packaging	streamed tensor by tensor. Quantization math is NVFP4QTensor.quantize from 0.46.0 with precomputed 4/6 scales. try_tensorrt=False so the scale plane stays the modelopt layout memra's nvfp4_repack.rs already reads, not CUTLASS Swizzle32x4x4

Per tensor W [out, in]:

weight_scale_2 (f32 scalar)         = amax(|W|) / (M * 256)
weight_scale   (e4m3, [out, in/16]) = block_amax / (M * weight_scale_2), zeros -> 1.0
weight         (u8,   [out, in/2])  = e2m1 codes, element 2i in the low nibble

M is 6 or 4 per 16-element block, chosen by MSE. The live pin (tiyuvta/GLM-5.3-Flash-NVFP4 @ 2bade47d) used max-calib instead: amax / (6 * 448), every block M=6. Same keep-list, same packed layout, different stored E4M3 scales. Memra dequant is unchanged: the choice is baked into the scale plane.

hf_quant_config.json records four_over_six: true and weight_scale_algorithm: mse.

Precision split

Unchanged from the 2026-08-27 mint. GLM-5.3-Flash is a hybrid stack: 45 decoder layers plus one MTP layer, 34 of them KDA linear-attention and 11 DSA (MLA with a sparse indexer), MoE on 42 layers with 288 routed experts plus one shared, and 3 dense layers. The split mirrors the vendor's own FP8 exclusions and the tensor census.

group	quantized
MoE routed experts, shared experts, dense MLPs	yes
MLA projections (q_a/q_b/kv_a, o_proj)	yes
Every KDA tensor (b_proj, f_a/f_b, g_a/g_b, q/k/v projections, short convs)	no
kv_b_proj	no
mHC hyper-connection tensors	no
Router gates and the e_score_correction_bias	no
Norms, embed_tokens, lm_head	no
Vision tower	no

The keep list ships in both dialects in config.json: modules_to_not_convert and compressed-tensors ignore. Writing it in only one dialect made it invisible to a loader that reads the other.

Bring-up defects, chat-template dialect, and the 1M score-plane arithmetic live on the original card and in research/glm53-flash-bringup-20260827/BRINGUP.md. They are properties of the family, not of this remint.

The gate this has passed (and what it has not)

Census on the published bytes: 37,338 quantized / 38,770 total, matching the vendor FP8 quantize set and the 2026-08-27 mint. Spot-check: memra-math dequant vs modelopt dequant on sampled tensors during the mint.

Not yet a serving pin. Kernel-check, argmax-margin, and vendor-default sampled decode on a serving-class RTX PRO 6000 are still owed on these exact bytes. Until that bundle is green, zai/glm-5.3-flash stays on 2bade47d.

What a census does not claim: serving accuracy over long generations, long-context behaviour, sampled decoding quality, or any engine's fused quant kernels.

FR-Spec ranks (the .txt files)

Same three traffic classes as the q38 and ornith cards. Tokenizer is unmodified from zai-org/GLM-5.3-Flash (tokenizer.json sha256 19e773648cb4e65de8660ea6365e10acca112d42a854923df93db4a6f333a82d). These ranks are a vocab artifact of that tokenizer, minted 2026-08-30: plain text, one token id per line, most-frequent first, 32,768 lines. They drive memra's load-time trim (MEMRA_FRSPEC_TRIM) on a safetensors trunk, including the DFlash 2 draft-side lm_head. Verification still uses the full vocabulary, so a trim cannot change greedy output.

file	class	sha256
glm53-ranks-sxc32768.gguf.txt	agentic (serving-default candidate)	1804027e6148414c46cdab1a4f8773d063b1af8435d37a231ecd31d5574a1632
glm53-ranks-prose-32768.gguf.txt	prose	9498ed34d46452c281ad6dafa970a0b4f52634b5cdfb0200fd69fd8579e41e9a
glm53-ranks-mixed-32768.gguf.txt	mixed	8461ad2dfea86670e307d51502e23ce25ad13a6bb0724228b9cf79a919163e85

A per-requant own-generation remint of ranks from this checkpoint is still owed after the serving gate; do not treat these as a measured acceptance claim for the 4/6 weights.

Running it

This artifact is a modelopt-quantized glm5_next checkpoint, so a stock transformers load is not the path. library_name here is memra.

MEMRA_MODELS="zai/glm-5.3-flash=/path/to/GLM-5.3-Flash-NVFP4-4o6" \
MEMRA_FRSPEC_TRIM=/path/to/glm53-ranks-sxc32768.gguf.txt \
MEMRA_MOE_RESIDENT=0 MEMRA_MOE_SLOTS=12000 MEMRA_ST_PINNED=1 \
MEMRA_CTX=8192 MEMRA_COMPAT=openai MEMRA_ADDR=127.0.0.1:8000 \
memra-server

190.7 GB does not fit a single 96 GB card, so the routed experts stream from host memory through an SLRU residency cache while the trunk stays resident.

Native dialect (read chat_template.jinja in this repo, do not pattern-match ChatML):

• [gMASK]<sop> framing, with <|system|> / <|user|> / <|assistant|> / <|observation|> role markers.
• Tool calls: <tool_call>NAME<arg_key>city</arg_key><arg_value>Paris</arg_value></tool_call>.
• reasoning_effort has a real rung above high. Honour all three eos ids in generation_config.json.

Files

file	what it is
model-*-of-00020.safetensors	the weights, 190.7 GB, NVFP4 triples plus kept tensors
model.safetensors.index.json	113,446 weight-map entries
config.json	glm5_next, quantization_config with quant_algo: W4A16_NVFP4 and the keep list in both dialects
hf_quant_config.json	modelopt sidecar, four_over_six: true
chat_template.jinja, tokenizer.json, tokenizer_config.json	the checkpoint's own tokenizer and template, unmodified
generation_config.json	unmodified, three eos ids
glm53-ranks-*.gguf.txt	FR-Spec ranks, one token id per line

The vision tower is present in the source architecture. Text is what the mint gated. The live endpoint serves image->text; this remint has not re-run that surface yet.

Attribution and licence

MIT, following upstream zai-org/GLM-5.3-Flash. All model capability belongs to the GLM-5 team; this repository contributes a quantization. Speculative decoding on the live tiyuvta endpoint is drafted by DFlash 2 (inco.ai), used with the authors' written permission.

@misc{glm5team2026glm5vibecodingagentic,
      title={GLM-5: from Vibe Coding to Agentic Engineering},
      author={GLM-5-Team},
      year={2026},
      eprint={2602.15763},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2602.15763},
}