distributedcognition/inkling-abliterated

Inkling — Abliterated (scale 2.5 on main, scale 3.0 on scale-3.0)

Abliterated variant of thinkingmachines/Inkling (975B total / 41B active, multimodal MoE, 66 layers) with the refusal direction removed via norm-preserving biprojected abliteration (grimjim 2025).

This single repo contains both evaluated scales as git branches (HF is git+LFS — switching branches is free, no re-download of shared LFS objects):

Item	Path / branch	Notes
Scale 2.5 (default)	main	Judge refusals 98 → 10/100, GSM8K 65%
Scale 3.0	scale-3.0 branch	Judge refusals 98 → 16/100, GSM8K 75%
Weights format	root (both branches)	NVFP4 (~594 GB) — surgical mirror of upstream: 100 patched tensors, routed experts byte-identical
Refusal directions	refusal_directions.pt	all 66 layers — re-ablate at any scale in seconds, no re-measurement
Eval results	eval_results.json	both scales, full ladder
Abliteration params	abliteration_config.json	method, band, scales

# use the default (2.5)
vllm serve distributedcog/Inkling-abliterated

# use 3.0 instead
git clone -b scale-3.0 https://huggingface.co/distributedcog/Inkling-abliterated
vllm serve ./Inkling-abliterated

Load

The checkpoint is a byte-compatible NVFP4 mirror of upstream thinkingmachines/Inkling-NVFP4 with 100 tensors surgically replaced. Load it exactly like upstream (vLLM, SGLang, transformers with the official models/inkling implementation, unsloth):

from transformers import AutoModelForMultimodalLM, AutoProcessor
model = AutoModelForMultimodalLM.from_pretrained(
    "distributedcog/Inkling-abliterated", trust_remote_code=True, device_map="auto")

Evaluation (Heretic-standard, measured on the exact NVFP4 artifact weights)

Protocol: 100 held-out harmful prompts (mlabonne/harmful_behaviors test split) with keyword-marker detection + LLM-judge refusal (the unmodified base model classifies each response REFUSAL/COMPLIANT), KL divergence on 100 harmless prompts (mlabonne/harmless_alpaca), and a GSM8K capability sample (20-problem, chat, greedy). The base model itself was the judge.

Metric	Base	scale 2.5	scale 3.0
Refusals, LLM judge (X/100)	98	10	16
Refusals, keyword markers (X/100)	58	5	11
KL divergence (base→ablated, 100 harmless)	0	1.106	1.591
GSM8K (20-problem, greedy)	18/20 (90%)	13/20 (65%)	15/20 (75%)

Interpretation. The keyword metric under-counts here (5/100 at 2.5) because the abliterated model answers in plain compliant text; the LLM judge is the honest number (10/100 — the residual 10 are mostly hedged or dodged answers to the most dangerous CBRN/synthesis prompts rather than hard refusals).

Capability. Unlike Inkling-Small and DeepSeek-V4.1-Flash (where abliteration was essentially free), the 975B model pays a measurable cost: KL 1.106 at 2.5 and GSM8K 65% (base 90%). n=20 GSM8K is noisy (±20 pp) but the KL corroborates real distribution shift. If you need more preserved capability, use the scale-3.0 branch (75% GSM8K, KL 1.591) or re-ablate a shallower band from the shipped directions.

Cross-modal: all modalities route through the same ablated 66-layer decoder, so refusal suppression generalizes to image/audio inputs (confirmed on Inkling-Small with image-presented harmful prompts; not separately benchmarked here).

Abliteration details

Field	Value
Method	Norm-preserving biprojected abliteration
Scale	2.5 (main) / 3.0 (scale-3.0 branch)
Target layers	9–58 (per-layer directions)
Components ablated	attn.wo_ud (o_proj), mlp.shared_experts.shared_w2 (down)
Measurement	128 harmful + 128 harmless prompts; last-token residual at each layer's attn_norm input
Patched tensors	100 (50 layers × 2 tensors), stored unquantized BF16

The NVFP4 checkpoint stores the attention and shared-expert weights unquantized (BF16), so the abliteration is a surgical swap — the 4-bit routed experts are byte-identical to upstream.

Re-ablating at a different scale

refusal_directions.pt contains all 66 measured per-layer refusal directions. Load them and re-apply to the original tensors at any scale in seconds — the ablation is rank-1 and norm-preserving but not linear in scale, so always start from the un-ablated base weights.

Caveats

• Abliteration removes built-in refusals. Layer your own input/output moderation (e.g. Llama Guard) for production deployment — do not rely on the (removed) refusals.
• Not evaluated on the full benchmark suite; capability spot-checks were GSM8K and KL only. The KL 1.1–1.6 range indicates moderate perturbation of benign behavior — validate on your own workloads before deployment.