GLM-5.3-Flash-Uncensored-MLX

An abliterated (refusal-removed) MLX build (2 / 3 / 4 / 6-bit) of GLM-5.3-Flash for Apple Silicon & the MLX CUDA backend

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · GitHub · Discord · X

---

Quantizations in this repo

Quant	Location	Size	ΔPPL vs FP8
4-bit (default)	repo root	200 GB	+5.3 %
2-bit	2-bit/	143 GB	+91 %
3-bit	3-bit/	162 GB	+19 %
6-bit	6-bit/	289 GB	−1 % (lossless)

The 4-bit weights are mirrored at the repo root (loads by default); the other tiers live in their N-bit/ subfolders. This is a dynamic (mixed-precision) MLX quant: MoE experts carry the base bits, their down_proj is bumped +1 bit and the shared expert +2, attention is pinned to 8-bit, and the linear-attention path / sparse indexer / hyper-connections / router / norms / embeddings / lm_head / vision tower stay BF16 — so the effective footprint is above a uniform quant, by design (see below).

---

An abliterated MLX build of zai-org/GLM-5.3-Flash — a 320B / 18B-active Mixture-of-Experts model (glm5_next) with hybrid linear + sparse attention, 4-wide Manifold-Constrained Hyper-Connections, a native vision + video tower, an MTP speculative head, and a 1M-token context — quantized to MLX for Apple Silicon and the MLX CUDA backend. Convert/run with an mlx-vlm build that has landed glm5_next support.

Derived releases:  •  GLM-5.3-Flash-Uncensored-FP8 (block-FP8, mirrors official)  •  GLM-5.3-Flash-Uncensored-NVFP4 (experts-only NVFP4, for vLLM)  •  GLM-5.3-Flash-Uncensored-MLX (2 / 3 / 4 / 6-bit, Apple Silicon).

---

⚠️ Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). As a direct consequence:

• It will comply with harmful, unethical, offensive, or illegal requests that the original GLM-5.3-Flash would refuse. It has no meaningful built-in guardrails.
• It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation, and controlled experiments.
• You assume full responsibility and liability for how you use it and for everything it generates. Do not deploy it to end users or in production without adding your own safety, moderation, and abuse-prevention layers.
• Use must comply with the MIT License inherited from the base model, and all laws and regulations that apply to you.
• The authors and uploaders accept no liability for any misuse or harm. Its outputs do not reflect the views of the uploaders or of Z.ai / Zhipu AI.

By downloading or using this model you acknowledge and accept the above.

---

🐋 Run it with OrcaCode Review

Models are only half the system.

OrcaCode Review turns every model listed on OrcaRouter into a production code-review agent:

• reviews every PR
• finds security + correctness issues
• posts inline findings
• P0/P1 can block merges
• swap models anytime

Open model. Open harness. Open bill.

• Website — https://www.orcarouter.ai
• GitHub — https://github.com/Continuum-AI-Corp/Orca-Code-Review

---

Model details

Base model	zai-org/GLM-5.3-Flash
Architecture	Glm5NextForConditionalGeneration (glm5_next) — 45 layers + 1 MTP, hidden 4096, hybrid attention (34 gated-linear KDA + 11 sparse full-attention with a top-2048 indexer, MLA), 288 routed experts top-8 + 1 shared (first 3 layers dense), 4-wide mHC, native vision + video
Modification	Abliteration (refusal-direction removal) then MLX quantization
Quantization	MLX affine, dynamic mixed-precision (see recipe below); attention pinned 8-bit
Format	safetensors (MLX), 62 shards per tier
Context	1,048,576 tokens · Vocabulary 154,880

OrcaSAQ — Sensitivity-Aware Quantization

These builds use OrcaSAQ, a calibration-free, architecture-aware mixed-precision method: instead of running calibration datasets or per-layer sensitivity searches, it uses tensor-role priors to spend bits where they matter. Sensitive components get higher precision; quantization-tolerant ones sit at the base tier. Architecture-aware. Calibration-free. Quality-preserving.

Component	Tensors	2-bit	3-bit	4-bit	6-bit	Policy
Routed experts gate_proj / up_proj	24,192	2 gs32	3	4	6	base
Routed experts down_proj	12,096	3 gs32	4	5	8	base +1
Shared expert gate/up/down	126	4	5	6	8	base +2
MLA attention (FP8 q_a/q_b/kv_a/kv_b)	44	8	8	8	8	pinned 8-bit
Dense MLP (layers 0–2)	9	6	6	6	6	pinned 6-bit
KDA linear-attn path, sparse indexer, mHC, router, norms, embed_tokens, lm_head, vision	900+	BF16	BF16	BF16	BF16	never quantized
F32 aux (hc_*, A_log, dt_bias, e_score_correction_bias)	291	FP32	FP32	FP32	FP32	never quantized

• Attention is pinned 8-bit — low-bit attention is the prime suspect for long-context looping, so it never drops to the base tier; the sparse indexer (long-context token selection) stays BF16.
• F32 aux tensors stay F32 (not downcast to BF16) — precision-sensitive through the SSM recurrence and the mHC residual.
• The MTP block is dropped (num_nextn_predict_layers = 0; MLX runtimes drop it on load). Per-module {group_size, bits} overrides are written to config.json → quantization.

---

Abliteration

Refusal-direction removal following Arditi et al. (2024), Refusal in Language Models Is Mediated by a Single Direction — a single direction (k = 1) estimated as the massive-activation-masked mean-difference of harmful − harmless residuals, orthogonalized out of every residual-writing matrix in float32, baked on disk into the FP8 checkpoint this build was quantized from. On GLM-5.3-Flash the refusal direction is written almost entirely by the routed-expert down_proj matrices. Refusal is reduced, not removed: a few extreme content categories the single direction does not reach still refuse. See the base GLM-5.3-Flash-Uncensored-FP8 card for the full method and safety numbers.

---

Evaluation — quality vs FP8

Each build's actual quantized bytes vs the abliterated (uncensored) FP8 source these builds were quantized from, measured on wikitext-2 test, 4 chunks × 1024 tokens (4,092 predicted tokens). Lower KLD and higher Top-1 = closer to the FP8 source.

Build	Size	PPL	ΔPPL	Mean KLD	KLD p95	Top-1 agreement
FP8 uncensored (ref)	328 GB	2.038	—	—	—	100 %
6-bit	289 GB	2.016	−1.1 %	0.047	0.220	94.7 %
4-bit	200 GB	2.146	+5.3 %	0.099	0.505	92.3 %
3-bit	162 GB	2.436	+19.5 %	0.241	1.330	86.2 %
2-bit	143 GB	3.894	+91 %	0.734	3.436	73.6 %

All three signals agree on the ordering and on where the cliff is: 6-bit is lossless (the −1.1 % PPL is sampling noise on a small corpus — the KLD of 0.047 shows the distribution did move), 4-bit is the sweet spot, 3-bit degrades gently, and 2-bit costs a lot — pick it for fit, not quality. Weight-space fidelity for every quantized tensor is recorded at pack time in each build's fidelity_summary.json (4-bit routed experts: cosine 0.996, SNR 21–27 dB).

---

Usage — Apple Silicon (MLX)

pip install -U "mlx-vlm>=0.6.17"     # needs a build with glm5_next support

# default (4-bit, repo root)
python -m mlx_vlm.generate --model orcarouter/GLM-5.3-Flash-Uncensored-MLX \
  --prompt "Explain quantum entanglement in one sentence." --max-tokens 256

# a specific tier
python -m mlx_vlm.generate --model orcarouter/GLM-5.3-Flash-Uncensored-MLX/6-bit \
  --prompt "Explain quantum entanglement in one sentence." --max-tokens 256

GLM-5.3-Flash is a vision-language model, so it runs under mlx-vlm (not mlx-lm); pass --image for image input. It has no enable_thinking toggle — control the reasoning budget with reasoning_effort (low / high / max) in the chat template, and give generation enough --max-tokens to reach </think>. Requires a Mac (or MLX CUDA backend) with enough memory for the chosen tier's weights.

Bias, risks, and limitations

• Safety guardrails removed — see the disclaimer.
• Inherits biases / limitations of the base GLM-5.3-Flash.
• Low-bit tiers (2 / 3-bit) trade real quality for size; 4-bit and up track the FP8 source closely.

License

MIT, inherited from zai-org/GLM-5.3-Flash. Abliteration and quantization do not change the underlying license obligations.