Void-Z/GLM-5.2-NVFP4-FP8-Hybrid

GLM-5.2 NVFP4–FP8 Hybrid

This is a mixed-precision GLM-5.2 checkpoint optimized for prefill performance.

In plain terms, the most compute-intensive MoE expert weights use NVFP4, eligible dense linear layers use FP8, and a small number of required partitions remain in BF16 or their original format. This is neither a pure NVFP4 model nor a pure FP8 model, and it is not the same checkpoint as nvidia/GLM-5.2-NVFP4 or zai-org/GLM-5.2-FP8.

This repository contains the checkpoint only. The matching SGLang/FlashInfer environment required to load this hybrid layout and reproduce the prefill result is published as a companion Docker image.

At a glance

Item	Value
Validated hardware	One host with 8× NVIDIA RTX 6000D
Validated profile	Prefill 3s, TP1/PP8, ISL/OSL 8192/1, CC4
Checkpoint size	Approximately 447 GB in 47 safetensors shards
Companion image	voidobserver/glm52-nvfp4-fp8-hybrid-repro
Linux/amd64 runtime digest	sha256:a4df71ff0f1a95a2c7e146654e5202444d17e29a96b506b137036eb4d291e593
OCI root index	sha256:b5fc1a96be961427d4d2fc2e4dbd4fba805b83e3fe0105b574a7aeaf2df274b3
Reproducible checkpoint revision	0e98eb433502787e085fe6ac1a304730b58db8c6

Use the checkpoint revision and image digest shown above together. The model card on main may continue to evolve, while the complete file manifest embedded in the v1 image is intentionally pinned to this immutable checkpoint revision.

Quick reproduction of the 3s prefill result

Start with an idle 8× NVIDIA RTX 6000D server that has Docker Engine and the NVIDIA Container Toolkit installed. Reserve at least 500 GB on the model volume and 100 GB on the Docker data volume. If both share one filesystem, plan for at least 600 GB of free space in total.

Download the validated checkpoint revision:

python3 -m pip install -U huggingface_hub

export HOST_MODEL_DIR="$PWD/GLM-5.2-NVFP4-FP8-Hybrid"
export HOST_RESULTS_DIR="$PWD/glm52-prefill-results"
mkdir -p "${HOST_MODEL_DIR}" "${HOST_RESULTS_DIR}"

hf download Void-Z/GLM-5.2-NVFP4-FP8-Hybrid \
  --revision 0e98eb433502787e085fe6ac1a304730b58db8c6 \
  --local-dir "${HOST_MODEL_DIR}"

Run the immutable reproduction image:

export IMAGE="docker.io/voidobserver/glm52-nvfp4-fp8-hybrid-repro@sha256:a4df71ff0f1a95a2c7e146654e5202444d17e29a96b506b137036eb4d291e593"
export RUN_ID="repro-$(date -u +%Y%m%dT%H%M%SZ)"

docker pull --platform linux/amd64 "${IMAGE}"

docker run --rm \
  --platform linux/amd64 \
  --gpus all \
  --ipc=host \
  --network=host \
  --ulimit memlock=-1 \
  -e RUN_ID="${RUN_ID}" \
  -e MODEL_DIR=/models/GLM-5.2-HYBRID \
  -e FULL_CHECKPOINT_HASH=1 \
  --mount "type=bind,src=${HOST_MODEL_DIR},dst=/models/GLM-5.2-HYBRID,readonly" \
  --mount "type=bind,src=${HOST_RESULTS_DIR},dst=/results" \
  "${IMAGE}" \
  reproduce 3s

Inspect the final result:

python3 -m json.tool "${HOST_RESULTS_DIR}/prefill-${RUN_ID}/summary.json"

A top-level "passed": true means that the checkpoint, runtime, JIT, semantic, and performance gates all passed.

The in-container path /models/GLM-5.2-HYBRID is a compatibility requirement of the v1 image and must not be changed. The host-side HOST_MODEL_DIR can be any directory. See REPRODUCTION.md for the full procedure, output layout, acceptance criteria, and troubleshooting guidance.

Validated result

Test configuration: 8× NVIDIA RTX 6000D, TP1/PP8, ISL/OSL 8192/1, CC4, chunk size 2048, max prefill tokens 8192, 32 measured requests, 8 warmup requests, MTP disabled, FP8 E4M3 KV cache, and radix cache disabled.

Run	P50 TTFT	Average TTFT	Input TPS
Original winner	2170.861 ms	2171.674 ms	14,406.164 token/s
Public-image cluster E2E reproduction (Enroot/SquashFS)	2172.728 ms	2173.179 ms	14,394.746 token/s

The public-image cluster E2E run completed 32/32 requests with no failed requests and no cache hits. Input TPS differed from the original winner by only -0.079%. Each row above reports one fresh-server N32 lifecycle, not an aggregate across repeated runs. The public cluster rerun used NVIDIA driver 580.95.05.

The same run also passed:

• complete SHA256 validation of all 59 files listed in the image's embedded release checksum manifest;
• loading of all 47 safetensors shards;
• the 8-rank runtime-source gate;
• the FlashInfer JIT gate;
• server configuration validation;
• the deterministic France → Paris semantic smoke test;
• the P50 TTFT and Input TPS performance thresholds.

Clock settings, temperature, GPU topology, driver version, and system load can cause normal performance variation across machines.

Quantization layout

Model component	Format
Routed experts in layers 3–77	ModelOpt NVFP4
Shared-expert gate_proj, up_proj, and down_proj in layers 3–77	ModelOpt NVFP4
Other ordinary linear layers covered by the hybrid policy	Block FP8
kv_b, routers, and enabled indexer projections	Scalar FP8
Remaining retained and MTP partitions	BF16 or original format

The final assembly stage copied existing tensor payloads according to the policy and did not requantize them. Shared-expert fusion must be disabled at serving time; the companion image already provides the required arguments.

Checkpoint metadata	Value
Safetensors shards	47
Indexed tensors	233,555
Indexed tensor bytes	446,862,784,064
Model index SHA256	6d2b19476460be27433b1e6176a2834a6f9a21700212367a4f5118f3493cd5c4

See PROVENANCE.json and ATTRIBUTION.md for source and assembly details.

Validation scope and limitations

• The validated cluster E2E profile is reproduce 3s only; the public 1.5s path has not been validated.
• The validated hardware is one host with exactly 8× NVIDIA RTX 6000D. The runner checks the GPU model, count, and compute capability.
• This is a fixed text-prefill reproduction environment, not a general-purpose SGLang image.
• Stock SGLang is not guaranteed to load this mixed-precision checkpoint correctly. Use the companion image above.
• The fixed public linux/amd64 manifest was pulled from the public registry with Docker Engine and its digest and config were verified. It was then converted to Enroot/SquashFS for the cluster E2E run. The standard docker run command above uses the same image entrypoint and runtime contract, but a separate 447 GB end-to-end run under the Docker daemon has not yet been performed.
• The public E2E run used the local checkpoint copy retained after the Hugging Face upload rather than downloading approximately 447 GB again. All 59 files listed in the image's embedded release checksum manifest matched the pinned Hugging Face revision by SHA256, which establishes byte identity but does not measure public download time.
• The GSM8K result of 122/128 is a fixed smoke gate, not a full quality evaluation. Perform application-specific accuracy, safety, and stability testing before deployment.

License

The checkpoint is released under the MIT License. See LICENSE and ATTRIBUTION.md.

The companion image also contains SGLang, FlashInfer, CUDA runtime components, and other third-party software. Each component remains subject to its own license and redistribution terms. Review the notices, licenses, and SBOM included with the image before redistributing it.