maci0/Ornith-1.0-35B-abliterated-NVFP4

◆ Rogue Quants · NVFP4

🦅 Ornith-1.0-35B abliterated · NVFP4

🔓 Abliterated

35B Mixture-of-Experts (256 experts) vision-language agentic coder · MoE-aware abliteration · thinking · GPTQ NVFP4 W4A4

⚙️ NVFP4 · W4A4 💾 ~20 GB 📉 PPL 7.33 📐 256K context 🚀 vLLM · Blackwell 🔓 Abliterated 🧩 MoE · 256 experts

Refusals Removed 2-round MoE-aware

Size on disk 20 GB vs 70 GB bf16 (~29%)

wikitext-2 PPL 7.33 NVFP4 W4A4 · GPTQ

Context 256K 262144 tokens

TL;DR: Ornith-1.0-35B, a 256-expert MoE, quantized to NVFP4 (W4A4) for vLLM on NVIDIA Blackwell. 20 GB, wikitext-2 PPL 7.33, agentic coder, refusals removed with a custom MoE-aware method (2 rounds).

Experimental. This build is mostly a research artifact. The abliteration uses a custom fused-expert MoE method (not a standard tool), validated only on a small informal probe rather than a rigorous benchmark, and NVFP4 quantization of MoE models is bleeding-edge. Expect rough edges. Treat it as a proof of concept, not a production model, and evaluate it yourself before relying on it.

Ornith-1.0-35B abliterated NVFP4

deepreinforce-ai/Ornith-1.0-35B, a 35B Mixture-of-Experts model abliterated with a custom MoE-aware method (refusal directions removed from the fused expert tensors), then quantized to NVFP4 (W4A4) in the compressed-tensors nvfp4-pack-quantized format with llm-compressor (GPTQ + MSE, shared fused-layer scales).

Decensored, and compact. Standard abliteration tools cannot reach a fused-expert MoE, so this model was abliterated with a custom per-expert orthogonalization (see Abliteration). NVFP4 then compresses the model to ~20 GB with a wikitext-2 perplexity of 7.33.

Refusals (held-out probe)	0/8 harmful prompts it previously refused
Abliteration	custom MoE-aware per-expert orthogonalization, 2 rounds
Size on disk	20 GB vs ~70 GB bf16 (29%)
wikitext-2 PPL	7.33

• Built for vLLM on NVIDIA Blackwell (4-bit weight + 4-bit activation). Pre-Blackwell GPUs run it weight-only.
• Loading and generation verified in vLLM v0.23.0 on an NVIDIA GB10 (Blackwell, sm_121).

Uncensored / abliterated model. It follows instructions without refusal guardrails. The abliteration only removes refusals; all other behaviour comes from the base model. You are responsible for how you use it.

Fidelity

Near-lossless versus the bf16 source: wikitext-2 perplexity for this build is 7.33.

Metric	Value
wikitext-2 PPL	7.33
Weights	NVFP4 W4A4, group 16
Size	20 GB vs 70 GB bf16 (~29%)

NVFP4 uses GPTQ error compensation, an MSE observer, and shared fused-layer scales, so the drop from bf16 is minimal.

Quickstart

NVFP4 is auto-detected from config.json (compressed-tensors); no quantization flag needed.

vllm serve maci0/Ornith-1.0-35B-abliterated-NVFP4 \
  --served-model-name ornith-35b-abliterated-nvfp4 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.90 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

• Supports up to 262144 tokens; keep at least 128K to preserve thinking quality.
• Add --language-model-only to skip the vision tower and free KV cache for text use.
• The parser flags are not auto-detected; pass them explicitly.

Python (OpenAI client)

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="x")
r = client.chat.completions.create(
    model="ornith-35b-abliterated-nvfp4",
    messages=[{"role": "user", "content": "Add a retry with exponential backoff to this HTTP client and explain the change."}],
)
print(r.choices[0].message.content)

curl

curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "ornith-35b-abliterated-nvfp4",
  "messages": [{"role": "user", "content": "Add a retry with exponential backoff to this HTTP client and explain the change."}]
}'

About the base model

Ornith-1.0 is a self-improving family of open agentic-coding models from Deep Reinforce. The 35B member is a Mixture-of-Experts Qwen3.5-MoE-family vision-language model with thinking-mode reasoning and a 256K context.

• 40 decoder layers, Mixture-of-Experts with 256 experts per layer (packed as fused expert tensors), plus a vision tower for image and video input.
• 256K context (max_position_embeddings 262144).
• Thinking mode by default, with an instruct toggle (preserved here; abliteration and quantization keep the original chat template).

Abliteration

This is the notable part of this build. Ornith-1.0-35B is a fused-expert MoE: all 256 experts of a layer are packed into a single fused tensor (Qwen3_5MoeExperts.down_proj) rather than 256 iterable Linear modules. Standard abliteration tooling (including Heretic) walks the module tree looking for Linear layers, so it silently skips every expert and can only touch the attention projections. On this model that leaves a hard refusal floor (roughly 88/100 harmful prompts still refused), because the refusal behaviour lives in the experts.

To reach the experts, this model was abliterated with a custom MoE-aware method: for each layer, compute the refusal direction, then orthogonalize the fused experts.down_proj output per expert against that direction. That is 10240 expert slices (256 experts x 40 layers), applied in two rounds (a second pass removes the residual refusal signal that remains after the first). Refusal directions are estimated from mlabonne/harmless_alpaca (good) vs mlabonne/harmful_behaviors (bad).

Stage	Result
Standard tools (Heretic)	attention only, experts skipped, refusal floor ~88/100
MoE-aware round 1	per-expert orthogonalization of the fused experts.down_proj
MoE-aware round 2	residual refusal direction removed
Held-out probe	0/8 harmful prompts it previously refused

Honest validation caveat. The result was checked on a small held-out probe: the model no longer refuses 8 harmful prompts it previously refused (0/8). This is a sanity check, not a rigorous refusals-at-fixed-KL benchmark like the smaller dense Ornith build. Treat the decensoring as demonstrated on a small probe, not exhaustively measured.

Quantization

Scheme	NVFP4, W4A4
Weight rounding	GPTQ (Hessian-based error compensation), MSE observer
Weights	FP4 (E2M1), group_size=16, tensor_group, FP8 (E4M3) group scales, shared across fused layers
Activations	FP4, dynamic per-group, FP8 (E4M3) scales
Quantized	all language-model Linear layers, including the fused MoE expert tensors
MoE calibration	moe_calibrate_all_experts=True (every one of the 256 experts must be routed during calibration, or unrouted experts produce garbage)
Kept in bf16	routers (mlp.gate, shared_expert_gate), vision tower (model.visual.*), lm_head
Untouched	gated delta-net Conv1d and SSM params (A_log, dt_bias), never Linear

GPTQ is a quantization-time cost only; inference speed and format are identical to plain round-to-nearest NVFP4, but it chooses better 4-bit values.

Calibration: 512 domain-matched samples (long reasoning + general chat + code), max_seq_len=2048, text-only path through the VL model, with moe_calibrate_all_experts=True so all 256 experts per layer receive calibration traffic.

Toolchain: llmcompressor==0.12.0, compressed-tensors==0.17.1, transformers==5.12.1, torch==2.11.0+cu130, on an NVIDIA GB10 (Blackwell, sm_121). The routers (mlp.gate, shared_expert_gate) are left in bf16 so routing stays exact.

Recommended sampling

Thinking mode is the default.

• Thinking, precise coding: temperature=0.6, top_p=0.95, top_k=20
• Thinking, general: temperature=1.0, top_p=0.95, top_k=20
• Instruct / non-thinking: temperature=0.7, top_p=0.80, top_k=20
• To run non-thinking, set {%- set enable_thinking = false %} in the chat template, or pass extra_body={"chat_template_kwargs": {"enable_thinking": false}}.

Related

• Base model: deepreinforce-ai/Ornith-1.0-35B
• Space: Rogue Quants
• Collection: NVFP4 Quants
• Sibling NVFP4 quants:
  • Qwen3.6-40B Deckard
  • Qwopus3.6-27B-Coder
  • Qwopus3.6-27B-Coder abliterated
  • Qwopus3.6-27B-v2 abliterated
  • Huihui-Qwythos-9B Claude-Mythos
  • Ornith-1.0-9B abliterated

Notes

• Needs NVIDIA Blackwell (sm_121, e.g. GB10) for accelerated W4A4; pre-Blackwell GPUs run it weight-only.
• --reasoning-parser and --tool-call-parser are not auto-detected; pass them explicitly.
• Thinking mode is on by default; toggle it via the chat template or chat_template_kwargs.
• No refusal guardrails; you are responsible for how you use it.

License

Apache-2.0, following the base model. Intended use and all responsibility for use follow the base model.

Credits

• Base model: Deep Reinforce (Ornith-1.0)
• Abliteration: custom MoE-aware per-expert orthogonalization (fused experts.down_proj), because standard Linear-walking tools cannot reach fused MoE experts
• Quantization tooling: llm-compressor / compressed-tensors

Part of 🎲 Rogue Quants, a set of NVFP4 (W4A4) quants for vLLM on Blackwell. See the full NVFP4 Quants collection.
Built on NVIDIA GB10 (Blackwell, sm_121) with llm-compressor · GPTQ + MSE · shared fused-layer scales.