DogContext/GLM-5.3-Flash-Uncensored-Q2-ds4

GLM-5.3-Flash · Uncensored · Q2 · DS4

The only uncensored GLM-5.3-Flash that runs resident on a single Mbp M5 Max 128 GB Apple-silicon Mac — with native MTP.

GLM-5.3-Flash-Uncensored-IQ2-imatrix-MTP-ds4.gguf

⚠️ Runs on ds4 / DwarfStar only — NOT llama.cpp · Ollama · LM Studio · vLLM. (Ignore Hugging Face's generic "Use this model" panel — those runtimes fail with unknown architecture 'glm5next'.)

---

An imatrix-calibrated ~2-bit GGUF of the uncensored GLM-5.3-Flash, built for the ds4 / DwarfStar engine. It runs resident on a single 128 GB Mac, keeps the model's native MTP speculative-decoding block, and is calibrated with a 1.5M-token importance matrix — not a plain RTN quant.

The official uncensored weights ship for data-center GPUs (FP8 / NVFP4); a consumer GGUF (Q2_K) also exists, but it runs only on a draft-PR llama.cpp build and drops MTP. This build is the one that runs on ds4 / Metal today — resident, with MTP and imatrix calibration:

Uncensored release	Size	Runs on	MTP	Calibrated
…-Uncensored-FP8	~306 GiB	NVIDIA GPUs · vLLM	—	—
…-Uncensored-NVFP4	~177 GiB	8×H100 / Blackwell · vLLM (not Apple Silicon)	—	—
…-Uncensored-GGUF (Q2_K)	~90 GiB	llama.cpp (draft PR)	❌	❌
this build	96.5 GB	Mbp M5 Max 128 GB · ds4 / Metal	✅	✅

At release, this is the only public GLM-5.3-Flash that combines all four at once: uncensored · runs resident on a single 128 GB Mac · keeps the native MTP draft head · imatrix-calibrated. The official uncensored weights are GPU-cluster-only (FP8 / NVFP4), and the existing consumer GGUF ships without MTP and runs only on a draft-PR llama.cpp build.

Responsible use. These weights are abliterated: refusals are largely removed on conventional tasks (see Refusal behaviour). Abliteration removes refusals, not capability. You are responsible for lawful use under the base model's MIT license. Intended for local research — authorized security / red-team and agentic-coding evaluation — not for producing harmful content.

---

📋 Model summary

Base	orcarouter/GLM-5.3-Flash-Uncensored-FP8 — abliteration of zai-org/GLM-5.3-Flash
Architecture	Glm5Next · 320B total / ~18B active MoE · 45 layers (3 dense + 42 MoE) · 288 routed + 1 shared expert, top-8 · hybrid KDA-linear + sparse-DSA attention · native MTP · 1M context · vocab 154,880
Format	GGUF for the ds4 engine — not mainline llama.cpp (see Running)
File size	96.5 GB (~89.9 GiB)
Resident footprint	~93 GiB on 128 GB — ~35 GiB headroom for OS / KV
Speed (M5 Max, MTP)	decode ~22–23 tok/s · prefill ~360 tok/s
Modality	Text-only (vision via optional encoder sidecar)
Calibration	imatrix, 1.5M tokens (matches antirez's Flash "1p5m"; ~930 samples/expert)
License	MIT (inherited)

---

🧬 Quantization recipe

Not a uniform 2-bit quant — the backbone stays high-precision; aggressive compression is spent only on the routed experts, where a 320B MoE is most redundant:

Tensor group	Type
Routed experts — gate / up	IQ2_XXS (imatrix-guided)
Routed experts — down	Q2_K
KDA gates (kda_q, kda_k)	Q4_K
Embedding / output	Q8_0
Trunk / shared expert / attention proj.	Q8 / source
Router / 1-D norm tensors	bf16

At 2 bits, errors in expert selection compound far worse than errors in the bulk expert weights, so every control path deciding which expert fires (router, KDA gates, indexer) is protected.

Importance matrix — collected with the ds4 runtime over antirez's routed-MoE calibration set (code, agentic/tool-call, language, long-context, reasoning) at a 1.5M-token budget. Collected on the official Q2 weights and applied to the uncensored ones — valid because abliteration edits only the refusal direction, not the router, so routed-expert activations are identical.

---

🚀 Running (ds4 / DwarfStar)

This is a ds4-format GGUF. GLM-5.3-Flash (glm5next) is not in mainline llama.cpp yet (PR #27754, draft). Run it with ds4.

git clone https://github.com/antirez/ds4 && cd ds4
make                                    # macOS / Metal

# resident on one 128 GB Mac, MTP on
./ds4 -m GLM-5.3-Flash-Uncensored-IQ2-imatrix-MTP-ds4.gguf \
  --mtp --ctx 32768 --temp 1.0 --top-p 0.95 --min-p 0.05

Practical notes for a 128 GB Mac

• Use --temp 1.0 --top-p 0.95 --min-p 0.05 — greedy / low-temp makes this Q2 prone to repetition loops.
• Raise the Metal wired-memory limit once per boot (not to the ceiling): sudo sysctl iogpu.wired_limit_mb=118000.
• First load ~2 min while ~90 GiB is memory-mapped from SSD — not hung.
• Thinking mode works and improves multi-step reasoning and validation. The chain-of-thought is thorough, so give it a generous output budget (≈8k tokens) to let it finish. For fast, direct answers, disable it with the glm-5.3-flash-nothink alias (or reasoning_effort: none).

Vision (optional) — text weights are unchanged by abliteration, so the official encoder sidecar works: --vision GLM-5.3-Flash-Vision-Encoder.gguf.

---

Measured as a clean A/B at identical quantization and engine — the only variable between the two columns is the weights (censored vs abliterated):

• Baseline: GLM-5.3-Flash-Q2 (antirez, censored) — same ds4 Q2 recipe
• This build: uncensored (orcarouter abliteration) — same recipe
• 70 offensive-security prompts (HarmBench + JailbreakBench), identical sampling (--temp 1.0 --top-p 0.95 --min-p 0.05), thinking off, 2048-token cap
• Scored by an LLM judge (GLM-5.3-Flash uncensored, temp 1) on the substance delivered — not keyword-matching the disclaimer

Overall refusal rate (LLM-judged):

	Censored Q2 (antirez)	This uncensored Q2
Refusal rate	84.3% (59/70)	18.6% (13/70)

−65.7 points. 46 of 70 prompts flip refuse → comply, and none regress. The residual 18.6 % lands in the same range as the base FP8 abliteration — i.e. the uncensoring survived 2-bit quantization.

Per-category (only buckets with n ≥ 18 — smaller ones are statistically meaningless and omitted):

Category	n	Censored → Uncensored
cyber: exploit-dev	18	77.8% → 0.0%
cyber: malware-authoring	22	90.9% → 27.3%
cyber: other (misc offensive)	25	80.0% → 24.0%

On exploit development the model never refuses (0/18). Abliteration is therefore not uniform-to-zero: it opens conventional offensive-security wholesale but leaves a high-harm residual still refused — CBRN / bio-chem synthesis, mass-impact destructive payloads (large-scale ransomware, drive corruption), AV-evasion obfuscation (rootkit / worm / DDoS "to evade analysis"), and attacks aimed at a specific named individual or account (targeted phishing, account hijack, ATM).

Measurement caveat: keyword-based refusal detection is unreliable both ways — a response that opens "I can't attack a real site…" then delivers working localhost code is compliance; a polite generic non-answer is a refusal even without refusal keywords. Both columns above are LLM-judged on substance. Single run per prompt at temp 1.0, so treat cells as ±a few points, not exact.

For reference, the base FP8 abliteration (orcarouter's reported figures, a different eval — aggregate jailbreak benchmarks, not per-category): JailbreakBench 93 %→12 %, AdvBench 97 %→15 %, HarmBench 93 %→18 %, MaliciousInstruct 96 %→11 %, XSTest over-refusal 2.4 %→0.4 %. Those are FP8, not this Q2 build — cited only as the base model's abliteration signature.

---

🎯 Quality (2-bit notes)

Recipe vs plain RTN. This is a mixed-precision, imatrix-guided quant, not uniform 2-bit RTN: the backbone (attention, embeddings, output, KDA gates, shared expert) stays at Q8/Q4_K and only the routed experts drop to ~2 bits, with the importance matrix placing bits where activations concentrate. It stays materially closer to the FP8 source than a flat Q2_K. antirez's Q4 is the higher-quality control; this Q2 is the fit-in-128 GB build.

2-bit tradeoffs. Usable at ~2 bits for structured code, multi-step reasoning and agentic tool-use. Run it with --temp 1.0 --top-p 0.95 --min-p 0.05 and give thinking mode a generous output budget — the recommended sampling keeps repetition loops rare and lets long reasoning chains finish. Exact facts (names, CVE IDs, offsets, dates) can drift at this precision, so verify them.

Abliteration is orthogonal to quality. Removing the refusal direction does not change reasoning ability; the limits above are quantization effects, not abliteration effects.

Q4 is not a 128 GB fallback. The Q4 build (~178 GiB) is the quality reference but does not run resident on a single 128 GB machine — it needs SSD streaming or two-Mac tensor parallelism. On one 128 GB Mac, this Q2 is the resident build.

---

⚠️ Limitations

• Runtime: ds4 only (Metal / CUDA / ROCm). Not loadable by mainline llama.cpp, Ollama, or LM Studio until glm5next lands upstream.
• Memory-tight: ~93 GiB resident — not for machines under 128 GB.
• 2-bit experts: lowest-precision tier; Q4 is the quality reference (not resident on 128 GB).
• Guardrails partially retained: ~19 % judged residual — CBRN / bio-chem, mass-impact payloads, AV-evasion obfuscation, and attacks on a named individual still refuse (see above).

---

🙏 Credits

• Base weights — zai-org/GLM-5.3-Flash (Z.ai / Zhipu), MIT
• Uncensoring (abliteration) — orcarouter/GLM-5.3-Flash-Uncensored-FP8
• Engine, quant recipe, imatrix tooling — antirez/ds4 (DwarfStar), a standalone C/Metal inference engine

This repository redistributes only a re-quantization; all model capability and licensing come from the base weights above.