CompiledThoughts/GLM-5.3-Flash-FP4-Local-inferredthoughts

GLM-5.3-Flash-FP4-Local, laid out for inferredThoughts

NVIDIA's GLM-5.3-Flash-FP4-Local, with each MoE layer's routed experts stacked into one block per projection, so the inferredThoughts engine can keep the experts a conversation uses on the GPU, the next ones in RAM and the rest on an NVMe SSD.

The weights are NVIDIA's, unchanged. This is a re-layout, not a new quantization: the same numbers in a different order. Every tensor that is not a routed expert is copied byte for byte under its own name.

Precision (NVIDIA's, kept as it is)

part	stored as (from hf_quant_config.json)
routed experts, 42 MoE layers	NVFP4: E2M1 values, E4M3 scales per 16 weights, FP32 global scale
the MTP layer's routed experts	NVFP4, weights only (BF16 activations)
shared experts; the sparse-attention layers' projections	FP8 E4M3, block-scaled, weights only
the linear-attention layers' projections, the three dense MLPs, lm_head	MXFP8: E4M3 values, UE8M0 scales per 32 weights
embeddings, norms, routers	BF16
vision tower	NVFP4 (weights only) MLPs and merger, MXFP8 attention, the rest BF16

321.3B parameters by our count of the tensors (NVIDIA's card: 320B in total, 18B active).

Can I load it?

Only with inferredThoughts. The stacked layout is the engine's own; vLLM, SGLang and transformers read NVIDIA's original instead, so use that one with them. GLM-5.3-Flash support in inferredThoughts is not in a public release yet; this card will say which release runs it.

Files

file	what
experts-003.safetensors ... experts-045.safetensors	the routed experts, one MoE layer a file
dense-00001.safetensors ... dense-00003.safetensors	everything else, byte for byte from the source
model.safetensors.index.json	the tensor map
config.json, generation_config.json, hf_quant_config.json, processor_config.json, tokenizer.json, tokenizer_config.json, chat_template.jinja	from the source, unchanged
NVIDIA-MODEL-CARD.md	NVIDIA's model card for the source checkpoint, unchanged
NOTICE, LICENSE-NVIDIA-Open-Model-Agreement.pdf, LICENSE-MIT-GLM	the licences and notices
SHA256SUMS	SHA-256 of every file

Check a download with sha256sum --ignore-missing -c SHA256SUMS.

License

Use of this model is governed by the NVIDIA Open Model Agreement (LICENSE-NVIDIA-Open-Model-Agreement.pdf). The GLM-5.3-Flash base model is governed by the MIT License (LICENSE-MIT-GLM). See NOTICE.

This is an independent re-layout. It is not made, reviewed or endorsed by NVIDIA or by Z.ai.