OneNexus/GLM-5.3-MXFP4

Model Overview

• Model Architecture: GLM-5.3
  • Input: Text
  • Output: Text
• Supported Hardware Microarchitecture: AMD MI325X (gfx942)
• ROCm: 7.2
• PyTorch: 2.11.0
• Inference Engine: SGLang/vLLM
• Model Optimizer: AMD-Quark (V0.12.post1)
  • Weight quantization: MOE-only (shared experts quantized), OCP MXFP4, Static
  • Activation quantization: MOE-only, OCP MXFP4, Dynamic

This model was built with GLM-5.3 model by applying AMD-Quark for MXFP4 quantization.

Model Quantization

The model was quantized from zai-org/GLM-5.3 using AMD-Quark. The weights and activations are quantized to MXFP4.

Weights use OCP MXFP4 with a per-group block size of 32 and E8M0 scales computed from the weight tensors themselves. Activations use the same block format with scales computed at runtime. No calibration dataset is involved.

The upstream checkpoint ships in FP8 (e4m3, 128x128 weight blocks, dynamic activation scaling) rather than BF16. Quark's file-to-file pipeline dequantizes those weights through a Triton kernel before re-quantizing them to MXFP4, so this conversion requires a GPU; the CPU path raises ImportError.

Attention, dense MLP projections, router gates, lm_head, and the MTP layer (layer 78) are excluded.

Quantization script:

import json
from quark.torch import LLMTemplate, ModelQuantizer

SRC = "zai-org/GLM-5.3"
DST = "GLM-5.3-MXFP4"

model_type = json.load(open(f"{SRC}/config.json"))["model_type"]

quant_config = LLMTemplate.get(model_type).get_config(
    scheme="mxfp4",
    exclude_layers=[
        "*self_attn*",
        "*mlp.gate",
        "*lm_head",
        "*mlp.gate_proj", "*mlp.up_proj", "*mlp.down_proj",
        "*layers.78.*",          # MTP layer
    ],
)

ModelQuantizer(quant_config).direct_quantize_checkpoint(
    pretrained_model_path=SRC,
    save_path=DST,
    device="cuda",
)

The equivalent CLI form:

cd Quark/examples/torch/language_modeling/llm_ptq/
  python quantize_quark.py \
      --model_dir zai-org/GLM-5.3 \
      --output_dir GLM-5.3-MXFP4 \
      --quant_scheme mxfp4 \
      --exclude_layers "*self_attn*" "*mlp.gate" "*lm_head" \
          "*mlp.gate_proj" "*mlp.up_proj" "*mlp.down_proj" \
          "*layers.78.*" \
      --file2file_quantization

Deployment

Use with SGLang/vLLM

This model can be deployed efficiently using the SGLang or vLLM backends.

Evaluation

No accuracy evaluation has been run on this checkpoint. The quantization recipe follows amd/GLM-5.2-MXFP4, but that model's reported GSM8K figures were measured on GLM-5.2 and do not describe this checkpoint. Anyone relying on this model for accuracy-sensitive work should measure it first.

To reproduce a GSM8K run with lm-evaluation-harness:

lm_eval --model sglang \
    --model_args pretrained=<repo_id>,tp_size=4 \
    --tasks gsm8k \
    --batch_size auto

License

This checkpoint is derived from zai-org/GLM-5.3 and is distributed under the same license terms as the base model.