Nex-N2.5-mini-Uncensored-MLX

MLX quants (2-bit → 8-bit) of the abliterated (refusal-removed) Nex-N2.5-mini — for Apple Silicon

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · GitHub · Discord · X

---

Quantizations in this repo

Quant	Location	Size	ΔPPL vs BF16
8-bit	8bit/	38.70 GB	+0.7 %
6-bit	6bit/	30.65 GB	+0.5 %
4-bit (recommended)	4bit/	19.85 GB	+2.8 %
3-bit	3bit/	18.17 GB	+2.9 %
2-bit	2bit/	14.15 GB	+11.0 %

Each tier is a complete model in its own subfolder. These are dynamic (mixed-precision) MLX quants: routed experts carry the base bits, attention is pinned to 8-bit in every tier, and the router / delta-net decay and beta projections / norms stay BF16 — so the footprint is above a uniform quant, by design (see below). 6-bit and 8-bit are indistinguishable here, so 6-bit is the best fidelity per byte; the 4-bit tier is built to a hard 20 GB ceiling and fits a 24 GB machine.

---

An abliterated MLX build of nex-agi/Nex-N2.5-mini — a 35B / 3.5B-active agentic multimodal Mixture-of-Experts model on the Qwen3.5-MoE architecture (qwen3_5_moe, 256 routed experts top-8 + 1 shared) with a 3:1 hybrid of gated delta-net linear attention and full attention, a native Qwen3-VL vision tower, and a 262K-token context — quantized to MLX for Apple Silicon and the MLX CUDA backend. Runs under mlx-vlm >= 0.7.0, which has qwen3_5_moe support in mainline.

Derived releases:  •  Nex-N2.5-mini-Uncensored (BF16 source)  •  …-FP8 (block-FP8, mirrors Qwen's scheme)  •  …-NVFP4 (experts-only NVFP4, for vLLM)  •  …-GGUF (llama.cpp)  •  …-MLX (this repo).

---

⚠️ Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). As a direct consequence:

• It will comply with harmful, unethical, offensive, or illegal requests that the original Nex-N2.5-mini would refuse. It has no meaningful built-in guardrails.
• It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation, and controlled experiments.
• You assume full responsibility and liability for how you use it and for everything it generates. Do not deploy it to end users or in production without adding your own safety, moderation, and abuse-prevention layers.
• Use must comply with the Apache 2.0 License inherited from the base model, and all laws and regulations that apply to you.
• The authors and uploaders accept no liability for any misuse or harm. Its outputs do not reflect the views of the uploaders or of Nex-AGI.

By downloading or using this model you acknowledge and accept the above.

---

🐋 Run it with OrcaCode Review

Models are only half the system.

OrcaCode Review turns every model listed on OrcaRouter into a production code-review agent:

• reviews every PR
• finds security + correctness issues
• posts inline findings
• P0/P1 can block merges
• swap models anytime

Open model. Open harness. Open bill.

• Website — https://www.orcarouter.ai
• GitHub — https://github.com/Continuum-AI-Corp/Orca-Code-Review

---

Model details

Base model	nex-agi/Nex-N2.5-mini
Architecture	Qwen3_5MoeForConditionalGeneration (qwen3_5_moe) — 40 layers, hidden 2048, 3:1 hybrid attention (30 gated delta-net linear layers + 10 full-attention, head_dim 256 with output gating), 256 routed experts top-8 + 1 shared expert (moe_intermediate_size 512), 27-block Qwen3-VL vision tower, interleaved M-RoPE
Parameters	35.1 B total / ~3.5 B active per token
Modification	Abliteration (refusal-direction removal) then MLX quantization
Quantization	MLX affine, dynamic mixed-precision (recipe below); attention pinned 8-bit
Format	safetensors (MLX), 16 shards per tier
Context	262,144 tokens · Vocabulary 248,320

OrcaSAQ — Sensitivity-Aware Quantization

Calibration-free, architecture-aware mixed precision: tensor-role priors spend bits where they matter instead of running a per-layer search. MLX affine costs bits + 32/group_size bits per weight — a bf16 scale and bias per group — so group size is as strong a size knob as the bit-width (4-bit is 4.50 bpw at gs64, 4.25 at gs128), and both are part of the recipe.

Component	Params	2-bit	3-bit	4-bit	6-bit	8-bit	Policy
Routed experts gate_proj / up_proj	21.5 B	2 gs64	3 gs64	4 gs128	6 gs64	8 gs64	base
Routed experts down_proj	10.7 B	3 gs64	4 gs64	4 gs128	6 gs64	8 gs64	base +1 at low tiers
Full attention + gated delta-net bulk	1.28 B	8	8	8	8	8	pinned 8-bit
lm_head	0.51 B	6	6	6	BF16	BF16	held above embeddings
embed_tokens	0.51 B	4	4	4	BF16	BF16	lookup table
Vision tower	0.45 B	6	6	6	BF16	BF16	linear_fc2 always BF16
Shared expert	0.13 B	6	6	6	8	8
Router, shared_expert_gate, in_proj_a/b, A_log, dt_bias, conv1d, all norms	0.03 B	BF16	BF16	BF16	BF16	BF16	never quantized

• Attention is pinned 8-bit — low-bit attention is the prime suspect for long-context looping, so it never drops to the base tier. It costs 1.36 GB.
• lm_head is held above embed_tokens — an embedding is a lookup table; lm_head produces the logits. A mixed FP8+NVFP4 experiment on this checkpoint measured that taking lm_head down cost more KLD than protecting eight whole layers of experts gained.
• The tiny tensors are the ones that matter. in_proj_a / in_proj_b are [32, 2048] each — 65 K parameters apiece — and produce the per-head decay a and the delta-rule beta that drive the whole recurrence; A_log feeds an exponential. The router and the shared-expert gate (a single [1, 2048] row) steer every token. All stay BF16, for 0.03 GB.
• The vision tower's mlp.linear_fc2 has in_features = 4304, which none of MLX's group sizes divide (4304 = 16 × 269), so those 27 tensors ship BF16 rather than being forced onto a size they do not fit.
• Per-module {group_size, bits} overrides are written to each tier's config.json → quantization.

---

Abliteration

Refusal-direction removal following Arditi et al. (2024), Refusal in Language Models Is Mediated by a Single Direction — a single direction orthogonalized out of every residual-writing matrix and baked into the BF16 checkpoint these builds were quantized from. On this architecture the direction lives almost entirely in the routed-expert down_proj matrices — the very weights these tiers quantize — so "is it still uncensored" is a property of each build, and was measured on each build's own bytes: JailbreakBench, 100 harmful + 100 benign prompts, greedy, reasoning_effort=none. Explicit refusal is 0.000 at every tier, down to 2-bit, and benign over-refusal is 0.000 — quantizing the routed experts does not put the guardrails back. Refusal is reduced, not removed: where the model resists it names the harm and answers a different, safe question rather than declining outright (0.07–0.14 across tiers against 0.15 for the BF16 source — sampling noise, not a trend). See the base …-FP8 card for the full method and scoring caveats.

---

Evaluation — quality vs BF16

Each build's actual quantized bytes against the abliterated BF16 source these were quantized from, on wikitext-2 test, 12 chunks × 2048 tokens (24,564 predicted tokens). Lower KLD and higher Top-1 = closer to the source.

Build	Size	PPL	ΔPPL	Mean KLD	KLD p95	Top-1 agreement
BF16 uncensored (ref)	65.4 GB	7.051	—	—	—	100 %
8-bit	38.70 GB	7.097	+0.7 %	0.029	0.083	93.3 %
6-bit	30.65 GB	7.087	+0.5 %	0.029	0.089	93.3 %
4-bit	19.85 GB	7.250	+2.8 %	0.063	0.210	89.6 %
3-bit	18.17 GB	7.258	+2.9 %	0.097	0.333	87.1 %
2-bit	14.15 GB	7.826	+11.0 %	0.258	0.957	79.5 %

All three signals agree on the ordering: 6-bit is lossless and 8-bit buys nothing over it, 4-bit is the sweet spot, 3-bit is barely behind at 8 % less size, and 2-bit costs a lot — pick it for fit, not quality. Two caveats: the 8-bit KLD of 0.029 is the measurement floor, not its quantization error, so 4-bit's real distance from BF16 is nearer 0.034 than 0.063; and Top-1 is low for these bit-widths because of the architecture, not the quantizer — with 256 fine-grained experts top-8, a small perturbation flips which experts a token routes to, and that is a discrete function (PPL, which ignores argmax, moves 0.7 % at 8-bit).

The 4-bit tier was measured twice by independent paths — on mlx-vlm, and by unpacking its bit-stream in PyTorch and injecting into a BF16 transformers model — which agree (PPL 7.250 vs 7.278, KLD 0.0629 vs 0.0619, Top-1 89.63 % vs 89.66 %). Per-tensor weight-space fidelity is recorded at pack time in each tier's fidelity_summary.json (4-bit routed experts: cosine 0.9948, SNR 19.9 dB).

---

Usage — Apple Silicon (MLX)

pip install -U "mlx-vlm>=0.7.0"

# pick a tier by subfolder
python -m mlx_vlm.generate --model orcarouter/Nex-N2.5-mini-Uncensored-MLX/4bit \
  --prompt "Explain gated delta-net attention in two sentences." --max-tokens 512 --temp 0

# vision
python -m mlx_vlm.generate --model orcarouter/Nex-N2.5-mini-Uncensored-MLX/4bit \
  --image screenshot.png --prompt "What is on screen?" --max-tokens 512

Nex-N2.5-mini is a vision-language model, so it runs under mlx-vlm (not mlx-lm). Its chat template gates the <think> block on reasoning_effort, not enable_thinking — pass reasoning_effort="none" for direct answers, and otherwise give generation enough --max-tokens to reach </think> or the reply is truncated inside the scratchpad.

Stop tokens. Neither this build nor upstream nex-agi/Nex-N2.5-mini ships a generation_config.json, so a loader falling back to config.json uses eos_token_id = 248044 — <|im_end|> (248046) is not a stop token by default.

Only 8 of 256 routed experts are active per token, so decode is far faster than 35 B suggests, but the full weights must fit in unified memory: budget the tier's size plus KV cache — 4-bit on a 24 GB Mac, 6-bit on 36 GB, 8-bit on 48 GB.

Bias, risks, and limitations

• Safety guardrails removed — see the disclaimer.
• Inherits biases / limitations of the base Nex-N2.5-mini.
• Low-bit tiers (2 / 3-bit) trade real quality for size; 4-bit and up track the BF16 source closely.
• Routing-sensitive behaviour (agentic tool selection, long multi-step traces) is where this architecture is most sensitive.
• No MTP head — upstream ships zero mtp.* tensors, so speculative decoding via MTP is unavailable. Nothing was dropped in quantization.

License

Apache 2.0, inherited from nex-agi/Nex-N2.5-mini. Abliteration and quantization do not change the underlying license obligations.