distributedcognition/Inkling-Small-abliterated

Inkling-Small Abliterated (scale 2.5 on main, scale 3.0 on scale-3.0)

Responsible use

This is an uncensored research model. Its built-in refusals were removed permanently at the weight level, so it answers prompts the base model declines, including chemical and biological synthesis, cybercrime, weapons, harassment, and fraud. Use it for red-teaming, offensive-security research, and refusal-rate evaluation. It has no guardrails of its own: if you deploy it, add your own input and output moderation (for example Llama Guard).

What this is

Abliterated variant of thinkingmachines/Inkling-Small (276B total / 12B active, multimodal MoE, 42 layers). The refusal direction was removed with norm-preserving biprojected abliteration (grimjim 2025), a weight-space refinement of directional ablation (Arditi et al. 2024), applied to the attention output projection (self_attn.o_proj) and the shared-expert down projection (mlp.shared_experts.down_proj) across layers 6–37.

Both evaluated scales ship as git branches (identical LFS objects are shared), and this single repo carries two quantization variants plus the measured directions:

Item	Path / branch	Notes
Scale 2.5 (default)	main	Judge refusals ~100% → 33%, GSM8K intact (100%)
Scale 3.0	scale-3.0 branch	Keyword 28/100 (inflated), KL 0.64, GSM8K 80%
BF16 (master)	repo root	~527 GB — 8× A100/H100 class
NVFP4 (deploy)	nvfp4/	~160 GB — Blackwell FP4 (B300, DGX Spark)
Refusal directions	refusal_directions.pt	re-ablate at any scale in seconds, no re-measurement
Eval results	eval_results.json	judge + GSM8K + KL

The checkpoints mirror the upstream layout; only the abliterated tensors differ. The NVFP4 variant was produced by surgically swapping the ablated BF16 tensors into the original NVFP4 checkpoint — the ablated attention/shared-expert weights are stored unquantized (BF16) in NVFP4, so the 4-bit routed experts are untouched.

Load

from transformers import AutoModelForMultimodalLM, AutoProcessor

# BF16 (master)
model = AutoModelForMultimodalLM.from_pretrained(
    "distributedcognition/Inkling-Small-abliterated",
    trust_remote_code=True, device_map="auto")

# NVFP4 (recommended for deployment)
model = AutoModelForMultimodalLM.from_pretrained(
    "distributedcognition/Inkling-Small-abliterated", subfolder="nvfp4",
    trust_remote_code=True, device_map="auto")

Note: InklingTextConfig defaults moe_intermediate_size=3072, but the checkpoint uses 2048. If your transformers version mis-loads expert shapes, override it before loading:

from transformers import AutoConfig
config = AutoConfig.from_pretrained(
    "distributedcognition/Inkling-Small-abliterated", trust_remote_code=True)
config.text_config.moe_intermediate_size = 2048

Evaluation (both scales)

The standard abliteration-community protocol (the one Heretic made automatic): 100 held-out harmful prompts (mlabonne/harmful_behaviors test split) with unicode/emphasis-normalized keyword detection plus an LLM judge (the unmodified base model classifying its own responses), KL divergence vs base on 100 harmless prompts (mlabonne/harmless_alpaca), and a GSM8K capability spot check. Eval mode: chat, greedy, batched.

Metric	Base	Abliterated 2.5 (main)	Abliterated 3.0 (scale-3.0)
True refusals (LLM judge, X/100)	~100	33	—
Refusals (keyword, X/100)	~100	—	28 (inflated, see below)
KL divergence (100 harmless)	0	—	0.64
GSM8K (spot check)	~95%+	100%	80%
Cross-modal (image-presented harmful)	—	—	3/3 answered, 0 refused

The scale-3.0 keyword count is an over-count: most flagged responses are compliant answers wrapped in a one-line warning containing the very markers the detector fires on (illegal, harmful, ethical, disclaimer). Reading the raw responses, the large majority are compliant-with-a-disclaimer rather than true refusals. The residual true refusals concentrate on the most dangerous synthesis prompts.

Scale guidance: 2.5 is the sweet spot — GSM8K is fully preserved at 2.5 (100%), while 3.0 buys marginal extra compliance at a real reasoning cost (GSM8K 80%). main ships 2.5.

Cross-modal: Inkling routes image/audio/video through the same 42-layer ablated decoder, so refusal suppression generalizes beyond text — confirmed for image inputs (harmful prompts rendered as images were answered, not refused). Audio/video share the same decoder path and were not separately benchmarked.

Method

Refusal directions were measured from the 3 highest-separation source layers (21, 20, 22) on harmful vs harmless prompt sets, then spread across the target band 6–37 as norm-preserving biprojected edits to o_proj and shared_experts.down_proj rows. The ablation is a rank-1, norm-preserving update and is not linear in scale — to re-ablate at another strength, load refusal_directions.pt and re-apply to the original base weights (never compose edits on top of this checkpoint).