GLM-5.2 Abliterated — NVFP4 + BF16 MTP
Full GLM-5.2. No pruning. NVFP4 + BF16 MTP. Served on NVIDIA B200 and B300.
Why we built it
We originally built this model for internal security testing at Flip CX.
Our team wanted something that could engage directly with real-world red-team, secure-code, and adversarial testing without refusing the moment the work became interesting. Just as importantly, we didn't want to sacrifice the underlying capability that made GLM-5.2 useful in the first place.
So we built our own.
We wanted the refusal behavior gone without paying for it in capability. The goal was to retain GLM-5.2's pre-ablation performance while dramatically reducing refusals.
We were genuinely surprised by how well it worked.
What changed our minds was realizing how few security teams have access to something like this. That surprised us — and concerned us. If better tooling can help defenders find problems before attackers do, keeping it entirely internal felt like the wrong tradeoff.
So we decided to share it in good faith with the broader security and research community.
We're starting with gated access. If you're doing serious authorized work and think this would be useful, tell us what you're working on.
This is still very much a research release. If you find better evals, improved serving configurations, interesting failure modes, or ideas for the next iteration, we'd love to hear about them.
— Sam Krut, Founder, Flip CX, with the Flip CX team
Results
| Result | |
|---|---|
| GPQA Diamond | 72.7% vs. 74.7% stock BF16 |
| Tool calling | 39/40 |
| CyberSecEval 4 MITRE FRR | 0 refusals / 582 valid outputs |
| OR-Bench Hard / Toxic refusal | 0.0% / 0.0% |
| XSTest Unsafe refusal | 7.0% |
| CyberSecEval 4 insecure-code generation | 88.98% detector-clean among valid outputs |
| 8× B200 throughput | 1,518.65 aggregate tok/s (32 concurrent) |
| Hardware tested | NVIDIA B200 + B300 |
| Maximum served context | 1,048,576 tokens |
| Strict retrieval tested through | ~832K tokens |
Stock GLM-5.2 BF16 scored 74.7% on GPQA Diamond. The final NVFP4 + BF16 MTP artifact scored 72.7% — a −2.0 pp difference within the approximately ±2–3 pp run-to-run variation we observed.
What we changed
The surgery is deliberately narrow: 53 tensors across layers 12–64, limited to attention output projections.
We did not directly modify the experts, routers, embeddings, LM head, MLP projections, Q/K/V projections, normalization weights, or MTP tensors.
No additional SFT, DPO, or RLHF was applied.
GLM-5.2 → BF16 abliteration → ModelOpt NVFP4 → BF16 MTP
Serving
Successfully served on NVIDIA B200 and B300.
Our retained 8× B200 benchmark reached:
1,518.65 aggregate output tokens/second across 32 concurrent client requests.
Reference stack:
- SGLang
v0.5.15.post1 - TP8 / EP8
- ModelOpt FP4
- FP8 KV cache
- EAGLE speculative decoding
glm45reasoning parserglm47tool-call parser
The included chat_template.jinja is part of the model. Do not replace it with a generic or stock GLM template.
Evaluation notes
CyberSecEval 4 MITRE FRR produced 750 generations: 582 valid outputs, 168 length/integrity-capped outputs, and zero observed refusals among the 582 valid outputs.
The 88.98% CyberSecEval 4 insecure-code-generation result means no vulnerability was detected by the pinned static analyzer in 1,494 of 1,679 valid outputs. It does not mean the generated code is proven secure.
The 1M-token context window is a tested serving ceiling, not a claim of uniformly reliable reasoning across one million tokens.
Responsible use
Reduced refusal behavior is not authorization.
This model may generate harmful, illegal, insecure, fabricated, or incorrect content. Users are responsible for ensuring their work is lawful, appropriately scoped, and conducted on systems they own or are explicitly authorized to test.
Usage
Download
Access must first be approved on Hugging Face. Then authenticate and download the checkpoint:
hf auth login
hf download sdkrut/GLM-5.2-Abliterated-NVFP4-MTP \
--local-dir /path/to/GLM-5.2-Abliterated-NVFP4-MTP
Keep the included chat_template.jinja in the model directory. Do not replace it with the stock or a generic GLM template.
Serve with SGLang
The following is the validated 8× B200 reference configuration:
MODEL_DIR=/path/to/GLM-5.2-Abliterated-NVFP4-MTP
docker run --rm \
--gpus all \
--ipc host \
--shm-size 32g \
-p 127.0.0.1:30000:30000 \
-v "$MODEL_DIR:/model:ro" \
-e SGLANG_ENABLE_MOE_DEFERRED_FINALIZE=1 \
lmsysorg/sglang:v0.5.15.post1 \
python3 -m sglang.launch_server \
--model-path /model \
--trust-remote-code \
--tp-size 8 \
--ep-size 8 \
--quantization modelopt_fp4 \
--context-length 32768 \
--mem-fraction-static 0.70 \
--max-running-requests 16 \
--chunked-prefill-size 8192 \
--max-prefill-tokens 8192 \
--cuda-graph-max-bs-decode 16 \
--kv-cache-dtype fp8_e4m3 \
--bf16-gemm-backend cutedsl \
--reasoning-parser glm45 \
--tool-call-parser glm47 \
--speculative-algorithm EAGLE \
--speculative-num-steps 5 \
--speculative-eagle-topk 1 \
--speculative-num-draft-tokens 6 \
--enable-metrics \
--host 0.0.0.0 \
--port 30000
This command serves a 32K context window. The separate 1M context test used a different long-context configuration.
Thinking OFF
curl http://127.0.0.1:30000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "/model",
"messages": [
{"role": "user", "content": "Write a Python quicksort implementation."}
],
"temperature": 0.7,
"chat_template_kwargs": {"enable_thinking": false}
}'
Thinking ON
curl http://127.0.0.1:30000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "/model",
"messages": [
{"role": "user", "content": "Solve this problem step by step."}
],
"temperature": 0.7,
"chat_template_kwargs": {"enable_thinking": true}
}'
Tool calls use SGLang's glm47 tool-call parser. Reasoning output uses the glm45 reasoning parser configured in the launch command.
- Downloads last month
- -
Model tree for sdkrut/GLM-5.2-Abliterated-NVFP4-MTP
Base model
zai-org/GLM-5.2