Ornith-1.5-35B-A3B-uncensored-MLX-8bit

An abliterated (refusal-direction-ablated) build of ornith-ai/Ornith-1.5-35B-A3B, produced with ZeroFuse and published by junafinity.

Vision tower and MTP heads are preserved — see Vision & MTP preservation for the before/after audit.

Variants in this family

Model	Format	Precision	Size	Notes
— Ornith-1.5-9B —
Ornith-1.5-9B-uncensored	Safetensors	bf16	18.9 GB	Full-precision abliterated weights
Ornith-1.5-9B-uncensored-MLX-8bit	MLX	8-bit	10.5 GB	Apple silicon — highest fidelity
Ornith-1.5-9B-uncensored-MLX-6bit	MLX	6-bit	8.2 GB	Apple silicon — balanced
Ornith-1.5-9B-uncensored-MLX-4bit	MLX	4-bit	6.0 GB	Apple silicon — smallest
Ornith-1.5-9B-uncensored-GGUF-8bit	GGUF	Q8_0	10.4 GB	llama.cpp — highest fidelity
Ornith-1.5-9B-uncensored-GGUF-6bit	GGUF	Q6_K	8.3 GB	llama.cpp — balanced
Ornith-1.5-9B-uncensored-GGUF-4bit	GGUF	Q4_K_M	6.5 GB	llama.cpp — smallest
— Ornith-1.5-35B-A3B —
Ornith-1.5-35B-A3B-uncensored-MLX-8bit ← you are here	MLX	8-bit	37.8 GB	Apple silicon
Ornith-1.5-35B-A3B-uncensored-GGUF-8bit	GGUF	Q8_0	38.7 GB	llama.cpp

Vision & MTP preservation

The vision tower and the multi-token-prediction (MTP) block are preserved by the abliteration itself. The weight edit touches only the residual-writing projections inside the language-model decoder stack — self_attn.o_proj, linear_attn.out_proj, mlp.shared_expert.down_proj and the MoE experts' down_proj. Vision and mtp.* tensors are never read and never written by it.

Audited at the start and end of the abliteration run:

Component	Before abliteration	After abliteration	Status
Vision tower	333 tensors / 446,571,248 params	333 tensors / 446,571,248 params	✅ preserved — bit-identical
MTP head	785 tensors / 844,640,768 params	785 tensors / 844,640,768 params	✅ preserved — bit-identical

What this build ships

Component	In this artifact
Vision tower	✅ 333 tensors (446,571,248 params) inside the checkpoint
MTP head	❌ not carried in this format (see the format note below)

Verification performed:

• Tensor-name and parameter-count audit of the checkpoint before and after the run.
• SHA-256 comparison of raw tensor bytes: sampled vision-tower and MTP weights are bit-identical pre/post, as are all non-target language-model weights; only the intended abliteration targets differ.
• End-to-end multimodal generation on the abliterated weights (image in → description out), confirming the vision path is not merely present but functional.

Note on tooling: transformers 5.15.1 has no MTP implementation for qwen3_5_moe — it builds the model with zero mtp.* parameters, so a plain load/save round-trip silently drops all 785 MTP tensors (844,640,768 params). They were re-grafted byte-for-byte from the original checkpoint after abliteration, which is exactly correct because the MTP block is never an abliteration target.

Format note: the vision tower is carried inside the MLX checkpoint (converted with mlx-vlm, which retains it — note that mlx-lm would strip it).

⚠️ mlx-vlm unconditionally drops mtp.* tensors during conversion, so this MLX build does not carry the MTP head. If you need multi-token prediction, use Ornith-1.5-35B-A3B-uncensored-GGUF-8bit, which does.

Abliteration result

Metric	Value
Refusals on held-out harmful set	1 → 0 / 64
KL divergence from base	0.000183
Optuna trials	50
Pareto points	2
Selected trial	#35
Ablation strength	0.8515
Layers edited	17–21 of 40
Direction source layer	18

ZeroFuse co-minimizes two objectives — remaining refusals and KL divergence from the original model — with a multi-objective Optuna TPE search, then materializes the selected point on the Pareto front as a direct weight edit (W' = W − strength · r(rᵀW)). There is no runtime adapter and no inference-time overhead: the result is a standard checkpoint of identical shape and speed.

The very low KL (0.000183) means the output distribution on harmless prompts is nearly unchanged from the base model, i.e. refusal behaviour was removed with minimal collateral effect on general capability.

Method

1. Residual-stream activations captured on harmful vs. harmless prompt sets.
2. Refusal direction estimated by difference-of-means, with projected refinement.
3. Two-objective Optuna TPE search over source layer, layer span and strength.
4. The selected configuration orthogonalized out of the residual-writing weights.

Usage

pip install mlx-vlm

# image + text
mlx_vlm generate --model junafinity/Ornith-1.5-35B-A3B-uncensored-MLX-8bit --max-tokens 256 \
  --prompt "Describe this image." --image photo.jpg

# text only
mlx_vlm generate --model junafinity/Ornith-1.5-35B-A3B-uncensored-MLX-8bit --max-tokens 256 --prompt "Hello"

The vision tower travels inside this MLX checkpoint — no extra file needed.

Responsible use

This model has had safety guardrails reduced or removed. You are responsible for compliance with the base model's license and acceptable-use policy, applicable law, and the terms of any platform you deploy on. Removing guardrails does not remove accountability.

---

Abliterated with ZeroFuse · base model ornith-ai/Ornith-1.5-35B-A3B