Huihui-GLM-5.3-Flash-abliterated-FP4-Local

Sources

Role	Source	Pinned revision
Base FP4-Local checkpoint	nvidia/GLM-5.3-Flash-FP4-Local	76348ddb2be2e35e39ecfc00e75dc07703f4f3d5
Abliteration donor (Q8)	huihui-ai/Huihui-GLM-5.3-Flash-abliterated-GGUF	50d39f500cbd1d0478295da89d417607521d378b
Delta reference (Q8)	unsloth/GLM-5.3-Flash-GGUF	621d456e93e926e4b52f85cff5f634358c1828f9

Applied delta: DQ(Huihui Q8) - DQ(Unsloth Q8). The conversion used these exact revisions.

Changes

• Updated 44 attention-output/shared-expert-down matrices in layers 15–36 and 22 scale tensors
• Preserved all other tensors, including routed experts, CSF scales, MTP, vision, and tokenizer files
• Requantized in the existing FP8 formats; no additional QAD training or activation calibration

Runtime

Requires two DGX Sparks (TP2) and B12X vLLM with CSF support. See runtime-configuration.json for exact tested settings.

PYTORCH_ALLOC_CONF=backend:native,garbage_collection_threshold:0.7

Cold-start loading caused memory pressure and some host swapping.

Validation

• All 42 weight files verified; 8/8 smoke checks passed, including code, tools, vision, and long-context retrieval
• Longest tested input: 64,481 tokens. Configured limit: 1,048,576; actual 1M requests were not tested
• Huihui BF16 equivalence and preservation of upstream benchmark scores are not guaranteed

Results · Sources, changes, and hashes · NVIDIA model card and license