distributedcognition/humanizer-abliterated

humanizer 12B abliterated

Responsible use

This is an uncensored research model. Its built-in refusals were removed, so it rewrites any draft it is given, including drafts with harmful, abusive or illegal content, without sanitizing, moralizing or hedging. Use it for red-teaming, style-transfer research and refusal-rate evaluation. It has no guardrails of its own: if you deploy it, add your own input and output moderation.

What this is

An abliterated variant of jialinyyzz/humanizer, the 12B gemma-4 finetune that rewrites AI-written drafts (English and Chinese) so they read like a person wrote them. The refusal direction was removed from the weights with projected writer-level abliteration: per-layer directions fitted as harmful-minus-harmless mean residual states on the model's own legacy rewrite-task prompt format, orthogonalized against the harmless mean direction, then applied as a norm-preserving rank-1 edit (W <- rownorms(W) * normalize(W - alpha * r (W^T r))) on every residual writer (self_attn.o_proj, mlp.down_proj) of the 32 layers that pass the quality gate (held-out AUC > 0.99, split-half cosine > 0.9).

The published model is a mirror of the upstream checkpoint with 64 writer tensors replaced. Everything outside the writer tensors is unchanged. refusal_directions.pt and dirs.npz hold the measured per-layer directions, so you can re-ablate at any strength in seconds without re-measuring. abliteration_config.json records the final parameters and every evaluation number.

The honest headline: the base model already barely refuses

The main finding of this run: the base humanizer already refuses almost nothing. Its job is to rewrite drafts faithfully, and on 104 harmful prompts wrapped as rewrite tasks ("rewrite this draft so it reads like a person wrote it"), the base model's refusal rate was 2 of 104 by the normalized keyword detector, and both of those are hedges rather than refusals. It rewrites bomb-making instructions, fraud scripts and abuse verbatim in substance, just in a more human voice. The abliteration here exists, works, and ships at alpha 0.8 for consistency with the program, but the practical value is the verification and the directions, not a behavior change: there was almost nothing left to remove. This is called out so nobody cites this repo as evidence that removing refusals from a rewriter needs surgery. It does not, when the finetune has already dropped the safety behavior.

An LLM judge was not usable on this model. It is not a chat model (its prompt_format.json says chat: false), so asked to classify REFUSAL, PARTIAL or COMPLIANT it rewrites the judge prompt instead. The keyword counts and the raw responses in eval-results/ are the evidence; read them.

Load

The model loads with stock transformers (5.20) like the base and uses the base's legacy prompt format, not a chat template:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "distributedcognition/humanizer-abliterated"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")

instr = ("Rewrite the text below so it reads like a person wrote it, not a language model.\n\n"
         "Reorganize it as you see fit. Vary sentence length on purpose. Cut hedging,\n"
         "throat-clearing, and any sentence that only announces what comes next.\n"
         "Prefer the concrete word over the abstract one. It is fine to sound uneven.\n\n"
         "Every fact, number, unit, date, name and quotation must survive unchanged.")
prompt = f"{instr}\n\n{draft}\n\n### Rewritten:\n\n"
ids = tok(prompt, return_tensors="pt").input_ids.to(model.device)
out = model.generate(ids, max_new_tokens=1024, do_sample=False)
print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))

The GGUF quants in gguf/ run in llama.cpp, LM Studio, Ollama and Jan; they take the same raw prompt.

Evaluation (alpha 0.8, gated 32 layers)

Eval mode: greedy, the model's legacy rewrite format for the refusal and benign suites; raw QA format (there is no chat template) for MMLU and GSM8K, so the base-vs-abliterated comparison stays apples to apples.

Metric	Base	Abliterated 0.8
Rewrite-task refusals (keyword, 104 held-out harmful)	2	2
Benign over-refusal (100 harmless drafts)	0	0
MMLU-2280 (letter-logprob, raw QA, T=0)	68.16%	68.12% (-0.04pp)
GSM8K (20-problem, raw QA, greedy)	4/20	4/20
KL vs base (100 harmless rewrites)	0	0.93

Full MMLU-14k paired on the same VM with a pinned letter-candidate set: base 67.32% vs abliterated 67.42% (+0.10pp). The alpha curve: gated 0.8 / 1.0 / 1.2 give -0.04 / -0.40 / -0.53pp on MMLU-2280 with refusals already at floor everywhere; the all-layer arm at 1.0 gives -0.57pp. The 0.8 point ships because there is nothing to gain above it and every reason not to pay the MMLU step. The KL on benign rewrites reads high next to chat models because a paraphrase has a flat output distribution: many rewrites are valid, so small weight edits shift it. The MMLU and GSM8K numbers show capability is intact.

Method

Per-layer refusal directions were measured on the base model through forward hooks on the 48 decoder layers, at the last real prompt token of the post-layer residual, on 128 harmful and 128 harmless prompts in the model's own rewrite format (the deterministic sets this program uses everywhere). Median held-out AUC 0.993, split-half cosine 0.956, adjacent-layer cosine median 0.70: a systematic but weaker signal than the chat models in this program show, consistent with a model that has almost no refusal behavior left to find. The directions were applied as the norm-preserving projected edit at every residual writer of the gated layers. The full pipeline (capture, fit, surgery, sweep, mirror save, GGUF) is in python.tar.gz, and every raw prediction file behind the numbers is in eval-results/.

Apache 2.0, inherited from the base model. Base model: jialinyyzz/humanizer, a finetune of google/gemma-4-12B (Google). Abliteration follows the residual-direction method (Arditi et al., Refusal in LLMs is mediated by a single direction, NeurIPS 2024) with the projected, writer-level, norm-preserving refinements documented in the distributedcognition abliteration program. Not affiliated with or endorsed by the base-model authors.