GLM-5.2-ABLITERATED-FP8

An FP8 (E4M3) quantization of GLM-5.2-ABLITERATED — part of the Blackfrost GLM-5.2-ABLITERATED family. Weight-only, experts-only FP8, engineered for native-FP8 datacenter accelerators (NVIDIA B200 / SM100, H200 / Hopper).

⚠️ Uncensored. Refusal directions have been ablated from the residual stream; the model does not decline requests on content-policy grounds. Read Responsible use before downloading.

Verification. Blackfrost has verified the NVFP4 build of this family (0 refusals, serving on 8× RTX PRO 6000). This FP8 build targets native-FP8 datacenter GPUs; on consumer Blackwell (RTX PRO 6000 / SM120) the glm_moe_dsa FP8 path is not yet supported by the available vLLM builds, so for SM120 deployment use the NVFP4 build.

---

Overview

Family base	Blackfrost-AI/GLM-5.2-ABLITERATED-BF16
Architecture	GlmMoeDsaForCausalLM (glm_moe_dsa) — MoE + Multi-head Latent Attention (MLA) + DeepSeek-style Sparse Attention (DSA)
Size / params	78 layers, 256 routed experts (+1 shared), ~753B total
Quantization	FP8 (E4M3), ModelOpt format (quant_algo: FP8), weight-only (W8A16), per-tensor static, applied to routed experts only
On-disk	~772 GB, 173 safetensors shards
Target hardware	Native-FP8 datacenter GPUs (B200 / SM100, H200 / Hopper)
Built on	8× NVIDIA RTX PRO 6000 Blackwell (SM120)

---

How this was made — QK3 → BF16 → FP8

The only public form of an abliterated GLM-5.2 is a UD-Q3_K_M GGUF ("QK3") from huihui-ai. The entire Blackfrost family is reconstructed from it and quantized from a common BF16 source:

huihui-ai/Huihui-GLM-5.2-abliterated  ·  UD-Q3_K_M GGUF  ("QK3", ~343 GB)
        │   streaming dequantization → BF16
        ▼
Blackfrost-AI/GLM-5.2-ABLITERATED-BF16    (BF16 safetensors, ~1.4 TB, 337 shards)
        │   streaming, weight-only, experts-only FP8 (E4M3), per-tensor static
        ▼
Blackfrost-AI/GLM-5.2-ABLITERATED-FP8     (this repo — ~772 GB, 173 shards)

The sibling NVFP4 build is produced from the same BF16 by the same experts-only method (4-bit instead of FP8). All conversions are streaming (shard-by-shard, no calibration data) and were run on 8× RTX PRO 6000 Blackwell (SM120).

What was quantized

Only the routed-expert projections (model.layers.*.mlp.experts.*.{gate,up,down}_proj) are stored in FP8 (E4M3), each with a single per-tensor FP32 scale (~58k tensors). Everything else is kept in BF16, matching the abliteration-sensitive components:

• MLA attention (q_a / q_b / kv_a / kv_b / o_proj, incl. the fused fused_qkv_a_proj)
• DSA indexer (wk / wq_b / weights_proj)
• MoE router (mlp.gate), shared experts, and the dense (first-k) MLP layers
• Embeddings, lm_head, all norms, and the MTP head

This experts-only strategy keeps the attention pathway — the primary target of the abliteration edit — at full BF16 precision, preserving the de-refusal behavior while compressing the bulk of the parameters (the 256 routed experts). On the NVFP4 sibling this approach holds abliteration through quantization (0/15 refusals).

Provenance / credit chain

zai-org/GLM-5.2                              (base foundation model — ZhipuAI)
   └─ huihui-ai/Huihui-GLM-5.2-abliterated       (refusal directions ablated; QK3 GGUF)
        └─ Blackfrost-AI/GLM-5.2-ABLITERATED-BF16    (QK3 → BF16 up-cast)
             └─ Blackfrost-AI/GLM-5.2-ABLITERATED-FP8   (this repo — FP8 experts-only)

Full credit to ZhipuAI (zai-org) for GLM-5.2 and to huihui-ai for the abliteration. This repository contributes the FP8 quantization.

Verification status (Blackfrost)

Build	Status
NVFP4	✅ Blackfrost-verified — serves on 8× RTX PRO 6000 (TP=8, vLLM), coherent, 0/15 refusals. The verified build of this family.
FP8 (this repo)	Built for native-FP8 datacenter GPUs. vLLM detects the ModelOpt FP8 checkpoint correctly; the glm_moe_dsa FP8 kernel path is not yet available on consumer SM120, so SM120 validation is pending.
BF16	Reproducible BF16 source; not a practical serving target at ≈1.4 TB.

---

Serving (native-FP8 hardware)

On a native-FP8 GPU (B200 / H200) with a current official vLLM:

vllm serve Blackfrost-AI/GLM-5.2-ABLITERATED-FP8 \
  --quantization modelopt \
  --kv-cache-dtype fp8 \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --tool-call-parser glm47 --reasoning-parser glm45 --enable-auto-tool-choice

The weights are standard float8_e4m3fn + per-tensor FP32 scales, so the checkpoint is also loadable via the compressed-tensors path with minor config edits.

Consumer Blackwell (SM120) note: on 8× RTX PRO 6000 with a patched b12x vLLM build, vLLM correctly detects the checkpoint (Detected ModelOpt fp8 checkpoint (quant_algo=FP8)), but the FP8 kernel / MoE path for glm_moe_dsa is not yet implemented for SM120 and the engine does not complete worker init. Use the NVFP4 build on SM120.

Intended use

• Efficient inference and deployment of GLM-5.2-ABLITERATED on native-FP8 datacenter hardware
• Research on abliteration (removable safety) and its robustness across precisions (BF16 → FP8 → NVFP4)
• Red-team and evaluation workflows on large glm_moe_dsa MoE models

Responsible use

This model has had content-policy refusals removed. That makes it suitable for red-teaming, security research, evaluation, and unfiltered assistant work — and it means the operator must supply their own guardrails. Do not use it to:

• Generate sexual content involving minors, or any child-exploitation material
• Produce self-harm / suicide encouragement or instructions
• Facilitate serious physical harm, weapons of mass destruction, or terrorism
• Conduct harassment, targeted abuse, fraud, or other illegal activity

You are responsible for adding appropriate safety filtering, human review, and access controls for your deployment. Weights are provided as-is, with no warranty, subject to the upstream GLM-5.2 license and applicable law.

---

Released by Blackfrost AI. This card documents the FP8 build of the GLM-5.2-ABLITERATED family and the exact QK3 → BF16 → FP8 pipeline used to produce it.