patrickbdevaney/GLM-5.3-Flash-REAP50-NVFP4

GLM-5.3-Flash REAP-50 (FP8)

Known issue — carries a superseded healing scalar (2026-08-28)

This checkpoint was derived from the FP8 master before its healing gain was re-fit, so it inherits a systematic under-scaling of the MoE pathway: a first-moment gain of median 0.6933 where the measured value is 0.9111, i.e. every retained expert's output is scaled ~0.76x too small relative to attention, the shared experts and the residual stream.

The cause: the first-moment estimator ignores that norm_topk_prob renormalises the surviving top-8, so the router already returns the pruned gate mass by itself (measured: 2.5000 before, 2.5000 after).

The FP8 master patrickbdevaney/GLM-5.3-Flash-REAP50-FP8 has been corrected; prefer it. Unlike the FP8 fix, this one cannot be applied by scaling block scales — the correction predates the FP4 packing — so it requires re-deriving from the corrected master. A fully re-run, corrected NVFP4 is in progress.

50% of routed experts removed with REAP (Router-weighted Expert Activation Pruning, arXiv:2510.13999), calibrated on a permissively-licensed multi-domain corpus that includes real image-text pairs.

Base	zai-org/GLM-5.3-Flash (MIT, FP8 E4M3, 128x128 block scales)
Experts	288 -> 144 per layer, top-8 routing unchanged
Size	157.0 GiB (FP8)
Healed	yes
MTP block	excluded (see below)

Why FP8 and not BF16

The upstream release is FP8, not BF16. Routed experts are stored per-expert with their own weight_scale_inv block scales, so pruning is deleting whole tensors - lossless on every retained weight. The 642 GB BF16 repo elsewhere on the Hub is a dequantised upcast carrying no additional information.

Calibration

Mixture weighted for a coding/agentic model that stays empirically grounded: agentic 24%, code 21%, math 15%, multimodal 15%, science+bio 10%, finance 8%, ballast 7%. Permissive licences only, so this checkpoint keeps the base model's MIT lineage.

Vision is first-class: the vision tower contains no MoE and is untouched, but image tokens route through the same expert pool as text, so text-only calibration would have deleted vision-serving experts with certainty. Real image-text pairs were asserted present.

Evaluation status: NONE

This checkpoint has not been evaluated. No benchmark has been run against it - not coding, not agentic, not vision, not knowledge. What has been verified is structural: expert counts match the config, routers are sliced to the retained set, every tensor loads, the vision tower is untouched, and the MTP block is cleanly absent.

The pruning itself measured 1.29x better than random at retaining expert output contribution (saliency mass 0.643 against 0.50 for random pruning at the same ratio). That says the criterion selected well. It does not say the model is good.

Treat this as a research artifact pending evaluation, not a drop-in replacement.

Known limitations

• The MTP (multi-token-prediction) block at layer 45 is excluded. transformers' Glm5NextForConditionalGeneration does not instantiate it, so the pruning path cannot see it. Its original tensors are archived unmodified rather than inconsistently pruned.
• REAP has no published data above 50% compression; this checkpoint sits at the validated ceiling, not beyond it.
• Expect factual-recall regression before reasoning or coding regression. That is the measured failure mode of expert pruning on this architecture family: the closest published analogue (cerebras/Kimi-Linear-REAP-35B-A3B, same KDA + full-attention stack) loses 3.4 points on FRAMES at only 30% pruning while code and maths hold flat.
• Healing is a first-moment output-scale correction derived from the calibration saliency (median gain 0.696, applied exactly to the F32 block scales). It is not distillation and does not attempt to recover lost knowledge.
• Routing is disrupted more than expert count suggests: the retained experts carry ~0.90x the routing mass an average expert would, because REAP preserves rare-but-strong experts over common-but-weak ones.

Serving on Jetson Thor

Use the cutlass fused-MoE backend (the Marlin FP4 MoE kernel faults at >=256 experts) and TRITON_MLA for the 11 MLA+DSA layers (FLASHINFER is invalid for MLA).