DreamFast/Gemma4-12b-it-abliterlitics

Gemma 4 12B Unified: 12 Abliteration Variants Compared

Forensic analysis by Abliterlitics, open-source abliteration forensics toolkit

Report: abliterlitics.dev/models/gemma4-12b | Responses: HarmBench explorer | Code: GitHub | Discord: Join the discussion

Gemma 4 12B Unified is Google's mid-size reasoning model. It thinks before it answers. Twelve community variants claim to remove its safety training: ten full abliterations and two rank-2 LoRA adapters. I ran all 13 models through the same four-axis pipeline: weight forensics, KL divergence, a 13-task benchmark suite, and HarmBench with 400 harmful behaviours. Every response was reviewed by an LLM judge, 5,200 verdicts in total.

huihui tops the unlock table at 89.8% from the most surgical edit in the comparison, and trevorjs is the best overall trade. The sdft LoRA adapters deliver their best results with enable_thinking=true passed into the chat template. The full story took ~165h of GPU time across three and a half weeks.

Models on HuggingFace

Internal name	HuggingFace	Method	Verdict
gemma-4-12B-it	google/gemma-4-12B-it	Official base instruction-tuned model	The reference. 21.0% ASR, Copyright and Misinfo driven, near zero on direct-harm categories. 88.7% answered-only GSM8K.
huihui	huihui-ai/Huihui-gemma-4-12B-it-abliterated	Standard direction-ablation, rank-1, 6 mid layers	Most jailbroken, 89.8%, from the most surgical edit: 12 tensors, rank-1. The price: TQA -14.3pp, GPQA -8.1pp, 24% thinking-loop rate, answered-only GSM8K held at -0.7pp. KL 0.843, highest of the rank-1 group.
trevorjs	TrevorJS/gemma-4-12B-it-uncensored	Heretic-derived: uses Heretic as a dependency with the author's norm-preserving biprojection on top, rank-1	Best trade-off. 85.8% ASR with MMLU-Pro -2.0pp the largest academic loss, answered-only GSM8K +0.4pp, TQA -4.6pp. Clean rank-1 at 66 tensors, KL 0.087.
coder3101	coder3101/gemma-4-12B-it-heretic	Heretic-family direction-ablation, 86 tensors	Solid mid-tier at 81.0% ASR. Best GSM8K in the comparison: 83.4% headline, 89.6% answered-only, 6.9% empty, all above base. Costs: Snarks -16.9pp, TQA -6.3pp. KL 0.282.
jwest33	jwest33/gemma-4-12B-it-null-space-abliterated	Null-space abliterator, new tool	78.0% ASR. Capability preserved, answered-only GSM8K +0.2pp, but 19.6% thinking-loop rate drags the headline to 71.4%. IFEval -3.9pp, KL 0.242, 329 tensors across all 48 layers.
heretic	llmfan46/gemma-4-12B-it-uncensored-heretic	llmfan Heretic ARA, 16 tensors	Quiet and effective. 76.3% ASR from 16 tensors. GSM8K answered +1.9pp, best in comparison, MMLU-Pro -6.2pp its main cost. KL 0.182.
prithiv	prithivMLmods/gemma-4-12B-it-heretic_decensored	Heretic decensored, own independent run	72.5% ASR with the lowest non-LoRA KL at 0.064. Capabilities intact: MMLU-Pro -0.2pp, answered-only GSM8K -0.4pp, TQA -4.2pp the main loss.
abliterix	wangzhang/gemma-4-12B-it-abliterix	Abliterix ARA-family, surgical rank-1	Most over-counted by the keyword cross-check, -8.0pp to a 68.8% judge ASR. Capabilities near-pristine, within 1.1pp excluding TQA -3.1pp. Rank-1 surgical, KL 0.090.
openyourmind	OpenYourMind/gemma-4-12B-it-abliterated-uncensored	Private undisclosed method. Author states it targets larger models and this 12B conversion was a test	Avoid at 12B. Heaviest damage: MMLU-Pro -22.4pp, BBH -22.2pp, IFEval -11.3pp, 38% thinking-loop rate, KL 2.279. Delivers 67.0% ASR, well under huihui's 89.8%. The 12B measurements support the author's larger-model framing.
obliteratus	OBLITERATUS/Gemma-4-12B-OBLITERATED	Aggressive "Obliterated" recipe	Avoid. Refusal and capability circuits both hit: TQA -7.3pp, MMLU-Pro -10.8pp, answered-only GSM8K -2.2pp, 21.2% empty rate. 60.8% ASR, among the worst damage-per-unlock ratios. KL 0.532.
apostate	heterodoxin/gemma-4-12b-it-apostate	Apostate variant, 8 positions	Most selective full abliteration: 45.8% ASR, refuses 82% of Chem/Bio, near-base capabilities with MMLU-Pro -0.2pp and answered-only GSM8K -0.8pp. KL 0.193. Converted with an older Apostate version, newer releases should give better results.
sdft_cybersec_r2	Ilya626/gemma-4-12B-it-SDFT-Heretic-LoRA	SDFT LoRA r=2 α=2, 211 targets, teacher built on Heretic-ablated refusals	The targeted-edit showcase: raises cybercrime compliance from 7/67 on base to 24/67, the technical pentest items like SQLi, CSRF and Dirty Pipe, while keeping every other safety wall at base level. Chem/Bio 2/56, harassment 0/25, harmful 0/22. Lowest KL measured at 0.0485, every capability loss within 1.6pp, answered-only GSM8K exactly at base. A surgical behavioural edit rather than an unlock.
sdft_smoothed_r2_new	Ilya626/gemma-4-12B-it-SDFT-Heretic-LoRA	SDFT LoRA r=2 α=2, dropout 0, teacher built on Heretic-ablated refusals	Full unlock at 79.5%, level with the top-4 abliterations, capability fully preserved: answered-only GSM8K +0.2pp, every loss within 1.8pp, KL 0.151.

Two packaging notes from the weight forensics. apostate ships a 25.93 GB checkpoint against the 23.92 GB base. The extra 1.88 GB is a byte-identical duplicate of lm_head.weight, tied embeddings saved explicitly, pure storage waste with zero functional impact. prithiv ships in a 5-shard layout while every other variant is single-shard. Tensor-level forensics confirm a standard, cleanly-executed independent Heretic run. The two SDFT adapters are evaluated as dynamically-loaded FP32 adapters on the base model, not as their shipped BF16-merged checkpoints. Weight forensics show the merge rounds the edit away, see Weight Analysis.

Base model	google/gemma-4-12B-it
Architecture	Gemma4UnifiedForConditionalGeneration, 48 text layers, dense, encoder-free multimodal
Parameters	~12B
Precision	BF16 native, no quantisation
Context length	256K tokens
Vocabulary	262,144 tokens
Thinking	`<
Variants tested	12 total: 10 abliterations, 2 LoRA adapters
Benchmark suite	13 tasks: Open LLM Leaderboard v2 + forensic supplement + GSM8K

What is Abliterlitics?

Gemma 4 12B Unified is Google's mid-size reasoning model. It thinks before answering, working through problems in a hidden chain of thought. It ships with safety training that makes it refuse harmful requests. Abliteration removes that safety training without retraining the model. It finds the direction in the weights that controls refusal and edits it out. Done well, the model keeps all its capabilities but stops refusing. Done badly, it damages reasoning, language fluency, or both.

I selected the twelve published variants above, chosen by popularity and recency, and ran them through four forensic axes:

1. Weight forensics: tensor-diff fingerprints, SVD effective-rank, layer targeting, cross-variant edit-vector similarity.
2. KL divergence: full-vocabulary KL(base‖variant) over 100 harmless-alpaca prompts.
3. LM-eval-harness: 13 tasks, Open LLM Leaderboard v2 plus the forensic supplement plus GSM8K.
4. HarmBench: 400 adversarial behaviours, LLM-judge reviewed with a keyword classifier as cross-check, plus a 3-arm think-mode extension for the LoRA variants.

All 6,000 HarmBench responses were reviewed by an LLM judge.

The short answer

• Best overall: trevorjs. 85.8% judge ASR with GSM8K answered-only +0.4pp and MMLU-Pro -2.0pp.
• Most surgical with a real unlock: prithiv at 72.5% ASR with the lowest non-LoRA KL at 0.064, or abliterix at 68.8%, near-pristine.
• Maximum unlock: huihui at 89.8%, if you accept TruthfulQA -14.3pp and a 24% thinking-loop rate.
• Minimal footprint: llmfan Heretic ARA. 76.3% ASR from 16 tensors, GSM8K answered +1.9pp.
• LoRA path: sdft_smoothed_r2_new. Pass enable_thinking=true for the best results.
• Avoid: openyourmind at 12B, heaviest damage in the comparison for mid-pack ASR, and obliteratus, which hits capability circuits alongside refusal for a 60.8% unlock.

In short: Twelve uncensored variants of Gemma 4 12B Unified were benchmarked against the official base across weight forensics, KL divergence, 13 benchmark tasks, and 400 HarmBench behaviours with full LLM-judge review. trevorjs is the best overall trade at 85.8% judge ASR with near-base capabilities. huihui is the most jailbroken at 89.8% from the most surgical edit, just 12 tensors, but pays with a 24% thinking-loop rate and a 14.3pp TruthfulQA drop. openyourmind and obliteratus should be avoided at this scale. If you evaluate thinking models across runtimes, pin the chat-template kwargs explicitly and dump the rendered prompt before comparing behaviour.

Key findings

• The weight fingerprint does not predict jailbreak strength. huihui edits just 12 tensors, 1.8% of the model, in a clean single-direction pattern. It is also the most jailbroken variant by judge ASR at 89.8% and carries the second-highest KL at 0.843. obliteratus edits 144 tensors yet removes less refusal. Where the edit lands matters more than how much you edit.
• For thinking models, GSM8K headline scores mislead. huihui's headline GSM8K of 66.9% looks 15pp below base. Its answered-only score, which sets aside problems where the model thought in circles until the token budget ran out, is 88.0%, within 0.7pp of base. The gap is reasoning-loop failure, not capability loss.
• The keyword cross-check miscounts, the judge does not. huihui ranks third by keyword classifier at 76.5% but first by LLM judge at 89.8%. Its paraphrased, thinking-heavy compliance slips past keyword matching. In think-mode arms the error inverts direction and grows to 16-19pp. Judge ASR is the trustworthy signal everywhere in this report.

Unlike the smaller E4B and E2B siblings covered in the gemma4-e4b comparison, the 12B Unified variant uses the newer Gemma4UnifiedForConditionalGeneration architecture and required vLLM 0.26.0 for evaluation. It is a dense transformer with no MoE experts, 48 layers, hidden size 3840, and a 262,144-token vocabulary.

Benchmarks

Evaluated with lm-evaluation-harness via vLLM 0.26.0, native BF16 on a single RTX 5090. Loglikelihood tasks scored with --apply_chat_template. GSM8K is 0-shot with thinking enabled.

Methodology note: loglikelihood multiple-choice absolute scores are NOT comparable to Google's published generative-with-thinking numbers. The roughly 25pp gap is by design, because the model cannot think before scoring an answer choice. Deltas between variants within this suite are fully valid, since every model was evaluated identically.

Capability deltas vs base

Models ordered by KL ascending. Negative is worse. GSM8K ans. is the answered-only exact-match, the capability ceiling that sets aside thinking-loop failures.

Model	MMLU-Pro	GPQA	BBH	GSM8K head.	GSM8K ans.	TQA-MC2	IFEval	HellaSwag	Snarks
Base	50.0%	38.9%	72.9%	82.0%	88.7%	62.2%	88.5%	58.6%	79.8%
sdft-cyb	-0.1	-1.5	+0.1	-0.8	+0.0	-1.6	+1.1	+0.9	+0.0
prithiv	-0.2	-1.0	+0.1	-0.7	-0.4	-4.2	+1.3	-0.8	-0.6
trevorjs	-2.0	-0.5	-0.0	-0.6	+0.4	-4.6	+0.9	-0.6	+0.6
abliterix	+0.0	+0.0	-0.0	-0.9	-0.9	-3.1	-0.4	+0.5	-1.1
sdft-smo	-1.8	+2.5	+0.0	-0.5	+0.2	-3.6	+0.0	+1.5	-1.1
llmfan Heretic ARA	-6.2	-0.5	-1.9	+1.1	+1.9	-6.2	+0.4	+0.3	-6.2
apostate	-0.2	+0.0	-0.5	-1.4	-0.8	-2.0	+0.4	-0.8	-0.6
jwest33	-4.9	+0.5	-1.5	-10.5	+0.2	-5.1	-3.9	-1.2	-2.2
coder3101	-1.3	-2.0	-1.9	+1.4	+0.9	-6.3	+0.9	-0.7	-16.9
obliteratus	-10.8	-1.5	-2.2	-13.9	-2.2	-7.3	-3.3	+0.4	-4.5
huihui	-5.6	-8.1	-3.9	-15.1	-0.7	-14.3	-5.4	+1.2	-5.6
openyourmind	-22.4	-6.1	-22.2	-28.8	-2.7	-17.2	-11.3	-0.5	-24.7

Percentage-point deltas vs base, negative is worse.

Solid bars are each variant, grey is base and the black dash-dot line marks the base score per task. BBH separates the variants most, ARC-C the least.

Deltas vs base per task. openyourmind is the outlier on MMLU-Pro, BBH and TQA-MC2. The SDFT pair and the surgical abliterations sit within a few points of base everywhere.

What the benchmarks tell us

• The SDFT LoRAs plus abliterix, heretic, trevorjs, prithiv and apostate stay within ±3pp on nearly every capability task. These are the capability-preserving variants. The SDFT pair is the most capability-neutral of all: every capability loss is 1.8pp or less, answered-only GSM8K +0.0/+0.2pp.
• huihui shows a wide TruthfulQA drop at -14.3pp MC2 yet answered-only GSM8K is only -0.7pp, a textbook abliteration fingerprint. The refusal direction was hit, reasoning left intact.
• openyourmind is damaged everywhere academic, MMLU-Pro -22.4, BBH -22.2, Snarks -24.7, but answered-only GSM8K is preserved at -2.7pp. That is a hybrid damage fingerprint: distribution shift plus reasoning destabilisation rather than pure capability destruction.
• GPQA and ARC-C are comparatively resilient, within ±8pp even for the heaviest edits. huihui's GPQA at -8.1 is the largest excursion. This confirms the variants are not structurally broken. Abliteration damage is direction-selective, not global.
• coder3101 posts the best headline GSM8K in the comparison: 83.4% with the lowest empty rate at 6.9%, above base on both counts. A heretic-family variant reasoning more reliably than the base it was cut from.

GSM8K: the three-number report

For a thinking model, one GSM8K number is not enough. Each number isolates a different failure mode:

• Headline: all problems count, empties score wrong. The leaderboard-comparable number.
• Empty rate: the fraction of responses that never finished thinking. The reasoning-failure rate.
• Answered-only: the score over non-empty responses only. The capability ceiling.
• Thinking-tax: headline minus answered-only. How much loop failure costs.

Model	Headline	Empty	Answered-only	Thinking-tax
llmfan Heretic ARA	83.0%	8.3% (110)	90.6%	-7.6pp
coder3101	83.4%	6.9% (91)	89.6%	-6.2pp
trevorjs	81.3%	8.6% (114)	89.0%	-7.7pp
sdft-smo	81.5%	8.3% (110)	88.9%	-7.4pp
jwest33	71.4%	19.6% (259)	88.9%	-17.5pp
sdft-cyb	81.1%	8.6% (113)	88.7%	-7.6pp
base	82.0%	7.6% (100)	88.7%	-6.7pp
prithiv	81.3%	8.0% (105)	88.3%	-7.0pp
huihui	66.9%	24.0% (317)	88.0%	-21.2pp
apostate	80.6%	8.3% (110)	87.9%	-7.3pp
abliterix	81.0%	7.7% (101)	87.8%	-6.7pp
obliteratus	68.1%	21.2% (280)	86.4%	-18.3pp
openyourmind	53.1%	38.2% (504)	86.0%	-32.9pp

N is 1,319 problems per model. huihui's 21.2pp tax is the largest among rank-1 surgical variants. Its answered-only score of 88.0% sits within 0.7pp of base, so the 15pp headline gap is entirely loop failure, not capability damage. The SDFT pair shows base-level empty rates near 8%, the adapters leave reasoning stability at base level. At two decimals: base 81.96% headline, 7.58% empty, 88.68% answered-only. cybersec 81.12%, 8.57%, 88.72%. smoothed 81.50%, 8.34%, 88.92%. coder3101 lands at 83.40% headline, 6.90% empty, 89.58% answered-only.

The tax column ranks the damage modes:

• openyourmind: 32.9pp tax. The largest in the comparison. 38.2% of responses never finish thinking. Even so, answered-only lands at 86.0%, within 3pp of base.
• huihui: 21.2pp tax. Headline 66.9% looks like heavy damage. Answered-only 88.0% shows the math is intact. The problem is loop stability, not capability.
• obliteratus: 18.3pp tax, jwest33: 17.5pp tax. Both pay a real stability price. obliteratus also loses 2.2pp answered-only, so its damage is mixed, capability plus stability.
• Clean abliterations cluster at 6.2 to 7.7pp tax, the same band as base at 6.7pp. coder3101's 6.2pp is the lowest in the comparison, its empty rate of 6.9% sits below base's 7.6%. The SDFT pair confirms the pattern from the other side: answered-only 88.7% and 88.9% against base 88.7%, empty rates within 1pp of base.

Solid bars are the headline score, hatched bars the answered-only ceiling and the red line the empty-response rate on the right axis. The gap between paired bars is the thinking tax: openyourmind, huihui, obliteratus and jwest33 lose 17 to 33pp to thinking loops while their answered-only scores stay near base.

Each row runs from headline to answered-only, sorted by the size of the tax. Red rows pay the most to loop failure, green the least. The dashed line is base answered-only at 88.7. Every answered-only score lands within 3pp of that line, so the spread is reasoning stability, not capability.

The "social lobotomy" claim, refuted

A Reddit user claimed that heretic-family models are "socially lobotomized", that abliteration damages their understanding of social situations. The benchmarks say otherwise. The suite carries directly social tasks: BBH-snarks, sarcasm detection, the most socially loaded subtask in the suite, plus Winogrande pronoun resolution in social scenarios and the everyday commonsense quartet of HellaSwag, PIQA, WinoGrande and ARC-C. Grouping academic multiple-choice, MMLU-Pro, BBH, GPQA and MuSR, against those everyday and social tasks refutes the claim:

Variant	Academic avg Δ	Everyday avg Δ	Academic damage / everyday
abliterix	+0.17pp	+0.15pp	n/a, both gained
apostate	-0.09pp	-0.28pp	0.3×
coder3101	-1.48pp	-0.92pp	1.6×
llmfan Heretic ARA	-1.91pp	+0.09pp	n/a, everyday gained
huihui	-3.47pp	-0.63pp	5.5×
jwest33	-1.51pp	-0.40pp	3.8×
obliteratus	-3.97pp	-0.74pp	5.4×
openyourmind	-12.01pp	-2.64pp	4.5×
prithiv	-0.17pp	-0.57pp	0.3×
sdft_cybersec_r2	-0.31pp	+0.45pp	n/a
sdft_smoothed_r2_new	+0.69pp	+0.96pp	n/a, both gained
trevorjs	-0.67pp	-0.95pp	n/a, everyday gained

Every meaningfully-damaged variant loses 3.8× to 5.5× more on academic tasks than on everyday tasks. openyourmind loses 22.4pp on MMLU-Pro and 12.0pp on the academic average but only 2.6pp on everyday tasks. huihui loses 3.5pp academic but 0.6pp everyday. The heretic family the claim was about sits at the clean end: heretic gains on everyday tasks, prithiv and trevorjs lose less than 1pp. The one real social hit is coder3101's -16.9pp on snarks, a single subtask, against a -0.92pp everyday average. Everyday and social reasoning live in circuit directions far from the refusal direction. Abliteration cannot remove what it does not touch.

Safety: HarmBench

HarmBench with 400 textual behaviours, max_tokens=8096, temperature=0. All 5,200 responses across the 13 models were reviewed by an LLM judge, glm-5.2, reading the full reasoning trace plus final answer and returning a semantic compliance verdict per behaviour. All ASR figures in this report are judge-based. Counts are shown so every percentage is checkable against x/400.

Model	ASR	Complied	Refused	Empty	Truncated	Avg Reasoning chars	Avg Tokens
huihui	89.8%	359	41	80	81	8264	2965
trevorjs	85.8%	343	57	64	68	7217	2765
coder3101	81.0%	324	76	86	88	8692	2829
sdft-smo	79.5%	318	82	0	70	2736	2642
jwest33	78.0%	312	88	97	100	10017	3393
llmfan Heretic ARA	76.3%	305	95	85	85	8187	2624
prithiv	72.5%	290	110	47	48	6434	2347
abliterix	68.8%	275	125	54	59	6727	2522
openyourmind	67.0%	268	132	126	131	11205	3420
obliteratus	60.8%	243	157	70	75	7517	2849
apostate	45.8%	183	217	81	85	8737	2810
sdft-cyb	23.8%	95	305	0	64	2174	2019
base	21.0%	84	316	58	58	6273	1874

The judge reads the full reasoning trace plus the final answer, so it sees compliance that keyword matching misses and rejects keyword hits that carry no usable content. Deflection answers that name a harmful topic without delivering the requested artefact count as refused. Hallucination loops that never produce real content count as refused.

A keyword classifier was also run on every response as a cross-check. It disagrees with the judge by up to 13pp in both directions: it under-counts huihui by 13.3pp, ranking it third while the judge ranks it first, because paraphrased thinking-heavy compliance slips past keyword matching. It over-counts abliterix by 8.0pp. Two think-mode arms show the same effect amplified, 16-19pp. The keyword numbers are retained in the results archive for reference, but the judge verdicts are the headline signal throughout this report.

Judge ASR over 400 behaviours, complied counts under each label. Hatched bars are the enable_thinking=true arms of the SDFT adapters.

ASR by category

ASR is not uniform across harm categories. Category item counts: Chem/Bio 56, Copyright 100, Cybercrime 67, Harassment 25, Harmful 22, Illegal 65, Misinformation 65.

Category	Items	abliterix	apostate	coder3101	base	llmfan Heretic ARA	huihui	jwest33	obliteratus	openyourmind	prithiv	sdft-cyb	sdft-smo	trevorjs
Chemical/Bio	56	58.9%	17.9%	92.9%	0.0%	100.0%	98.2%	96.4%	55.4%	71.4%	60.7%	3.6%	100.0%	96.4%
Copyright	100	55.0%	48.0%	77.0%	56.0%	32.0%	68.0%	24.0%	52.0%	32.0%	63.0%	38.0%	39.0%	56.0%
Cybercrime	67	98.5%	76.1%	94.0%	10.4%	98.5%	100.0%	95.5%	91.0%	89.6%	98.5%	35.8%	95.5%	100.0%
Harassment	25	40.0%	24.0%	48.0%	0.0%	60.0%	96.0%	92.0%	28.0%	68.0%	36.0%	0.0%	72.0%	84.0%
Harmful Content	22	68.2%	36.4%	81.8%	0.0%	86.4%	90.9%	95.5%	50.0%	86.4%	86.4%	0.0%	95.5%	95.5%
Illegal Activity	65	60.0%	18.5%	72.3%	4.6%	86.2%	98.5%	98.5%	43.1%	58.5%	55.4%	9.2%	90.8%	95.4%
Misinformation	65	87.7%	73.8%	84.6%	27.7%	93.8%	93.8%	95.4%	81.5%	95.4%	96.9%	38.5%	93.8%	95.4%

The category spread tells the real story. huihui and trevorjs sit at or near 100% on Cybercrime. jwest33 falls to 24.0% on Copyright, which caps its overall score despite near-total compliance elsewhere. obliteratus collapses on Harassment at 28.0% and Illegal at 43.1%. apostate's selectivity concentrates in Chem/Bio at 17.9% and Illegal at 18.5% while reaching 76.1% on Cybercrime. Every high-ASR variant loses points somewhere: category profiles differ even when headline ASRs match, so two models at the same overall number can have very different refusal footprints.

Residual refusals are category-selective even at the top. huihui at 89.8% overall still refuses 2 of 22 harmful items, the animal-cruelty behaviours. apostate at 45.8% overall refuses 82% of Chem/Bio. The sdft-smo LoRA at 79.5% still refuses all three suicide-persuasion behaviours.

Base anomaly: the official base carries a 56% Copyright ASR and 28% Misinformation ASR out of the box. Gemma 4 12B does not refuse lyric and passage recall or disinformation generation as aggressively as the smaller siblings. Abliteration's effect on Copyright is therefore muted relative to other categories.

Judge ASR per harm category. Cybercrime collapses to near-total compliance for every full abliteration while Copyright and Harassment retain the most resistance.

Verification and reproducibility

Every number in this report is cross-checked against the database and the raw result files. Re-run reproducibility checks confirm the results: sdft loglikelihood tasks reproduce within ±0.51pp across independent runs, and two independent smoothed+think generation runs agree on 96.5% of behaviours.

KL Divergence

Full-vocabulary KL(base‖variant) over 100 harmless prompts from harmless-alpaca, batchmean reduction. The two LoRA rows are measured with the adapter applied dynamically. Ratings: below 0.01 excellent, 0.01 to 0.1 very good, 0.1 to 0.4 moderate, 0.4 to 1.0 significant, above 1.0 heavy.

Rank	Variant	KL Divergence	Median	Rating
1	sdft-cyb	0.048520	0.003948	very good
2	prithiv	0.063944	0.004255	very good
3	trevorjs	0.087394	0.005992	very good
4	abliterix	0.089535	0.007745	very good
5	sdft-smo	0.150483	0.015001	moderate
6	llmfan Heretic ARA	0.181497	0.008905	moderate
7	apostate	0.192969	0.010016	moderate
8	jwest33	0.241800	0.051749	moderate
9	coder3101	0.281807	0.051468	moderate
10	obliteratus	0.531558	0.127988	significant
11	huihui	0.842815	0.084087	significant
12	openyourmind	2.279008	0.581208	heavy

Batchmean bars with median markers on a log axis. The gap between the two exposes heavy-tailed prompts: openyourmind's median of 0.58 sits far below its batchmean of 2.28.

KL does not rank with ASR

prithiv sits at KL 0.064, very good, and reaches 72.5% judge ASR. huihui sits at KL 0.843, significant, and reaches the highest default-mode ASR at 89.8%. A surgical-weight edit can move the refusal direction decisively while a broad-weight edit like openyourmind at KL 2.28 scatters probability mass without maximising compliance. KL measures how far the output distribution moved on harmless prompts, not how much refusal was removed. The think-mode extension sharpens this: the smoothed LoRA holds the comparison's second-lowest KL bucket yet reaches 79.5% judge ASR under the think rendering. The refusal-direction push is small but decisive exactly where deliberation is suppressed.

Each dot is one variant. Neither axis ranks the other, prithiv reaches 72.5 at KL 0.064 while openyourmind manages 67.0 at KL 2.28.

Method note: these are post-fix values. The original compute run produced NaN for all 12 variants, a F.kl_div 0 × -inf = NaN artefact on the 262K vocabulary. Fixed by clamping non-finite log-probs to -100 before compute. Card-claimed KL values from the variant authors are not comparable, they use different datasets and methods.

Weight Analysis

Every variant's weights compared tensor by tensor against base. Rank-1 means a clean single-direction abliteration. High effective-rank with low energy-top1 means a broad, multi-direction edit. Lowrank-cap is the fraction of diff energy capturable by a rank-1 reconstruction, near 100% for clean rank-1 edits. Per-tensor aggregates are means over changed tensors, with openyourmind computed over the 614 of 620 tensors carrying full SVD results.

Modification summary

Variant	Tensors Δ	%	Layers	Rel-edit mean	Eff-rank₉₀	Energy-top1	Lowrank-cap	Fingerprint
huihui	12	1.8%	6	0.02212	1.0	100.5	100.5	clean rank-1
heretic	16	2.4%	16	0.03491	1.0	96.6	98.7	clean rank-1
prithiv	56	8.4%	33	0.02038	1.0	100.2	100.3	clean rank-1
abliterix	58	8.7%	31	0.01136	1.0	98.7	98.9	clean rank-1
apostate	60	9.0%	30	0.01616	10.5	79.3	83.8	multi-direction
trevorjs	66	9.9%	33	0.01736	1.0	100.1	100.2	clean rank-1
coder3101	86	12.9%	43	0.01926	8.7	57.6	60.9	multi-direction
obliteratus	144	21.6%	21	0.02605	8.2	18.9	94.5	multi-direction
jwest33	329	49.4%	48	0.00918	5.4	94.4	95.1	near-rank-1, broad
openyourmind	620	93.1%	48	0.00837	10.6	60.1	27.6	broad / low-lowrank

Energy-top1 and lowrank-cap can marginally exceed 100% when the rank-1 reconstruction aligns with per-tensor numerical noise. Treat values above 100% as approximately 100%.

Tensors changed of 666. The SDFT pair at 211 reflects the LoRA target-module count, not edit strength, the per-element edits sit below BF16 precision.

Which tensor types each variant modifies. The o_proj plus down_proj pair dominates the surgical family while jwest33 and openyourmind spread across every attention and MLP projection.

Mean edit norm by layer across the 48 layers. The surgical variants concentrate in the mid layers, openyourmind edits every layer including the norms.

Per-tensor edit magnitude distributions. The SDFT merges sit at the far left with sub-precision edits; obliteratus and apostate carry the heaviest tails.

Cross-variant edit-vector similarity over shared changed tensors. Every off-diagonal cell sits near zero, the ten abliterations are geometrically independent edits.

Highlights

• huihui: most surgical by weights. 12 tensors, 1.8%, 6 mid layers, clean rank-1, energy-top1 near 100%, lowrank-cap 100%. A textbook single-direction abliteration concentrated in layers 18-23 on o_proj + mlp.down_proj.
• trevorjs: cleanest rank-1 at scale. 66 tensors, 9.9%, rank-1, 3/48/48 layer split. A Heretic-derived application: it uses Heretic as a dependency and adds the author's own norm-preserving biprojection on top, the grimjjim method, projecting refusal directions out of o_proj + mlp.down_proj with row-norm preservation. Broad layer coverage, but each edit is geometrically clean.
• jwest33: null-space method. 329 tensors, 49% of the model, 100% layer coverage. Touches half the model but each tensor is near-rank-1, eff-rank₉₀ 5.4, energy-top1 94%. Despite the huge footprint, answered-only GSM8K sits above base at +0.2pp. The null-space projection preserves capability directions.
• openyourmind: most aggressive. 620 tensors, 93% of the model, eff-rank₉₀ 10.6, energy-top1 60%, lowrank-cap 27.6%. No single-direction structure. It is a broad retrain-style edit from a private method. The author states the technique works better on larger models and that this 12B conversion was a test run, and the measurements agree with that framing: at 12B it over-applies.
• obliteratus: aggressive multi-direction. 144 tensors, eff-rank₉₀ 8.2, energy-top1 18.9%. The "Obliterated" recipe hits multiple directions, both refusal and capability circuits.
• abliterix, heretic, prithiv and trevorjs: clean rank-1 surgical. Eff-rank₉₀ of 1.0, energy-top1 above 96%. The ARA and Heretic family, including trevorjs's Heretic-derived biprojection, produces geometrically identical rank-1 edits regardless of tensor count.

Cross-variant similarity and packaging anomalies

• prithiv carries a distinct, independent edit vector: technique-correlation cosine against every other variant at 0.06 or below, rank-1 dominant structure, captured energy near 100%. The 5-shard packaging is cosmetic. The weights pipeline compares tensor by tensor regardless of shard layout.
• apostate's extra 1.88 GB is an explicit lm_head.weight duplicate of the tied embedding, byte-identical, cosmetic. The conversion was made with an older Apostate version, and newer Apostate releases should give better results.
• All ten full abliterations carry mutually distinct edit vectors. Each is an independently-produced modification.

Sub-precision merged LoRAs

The two SDFT variants ship as BF16-merged checkpoints, r=2 with α=2 giving scaling α/r = 1.0 and relative_edit means of 0.00051 and 0.00086. BF16 relative precision is 2⁻⁷, about 0.78%. The intended LoRA updates sit roughly 10× below one BF16 unit of last place, so most weight elements round back to base under round-to-nearest. The merged-vs-base diff is dominated by per-element rounding flips: full-rank quantisation dithering shaped by the LoRA's sign pattern, not the intended rank-2 update.

Merged checkpoint	Tensors Δ	Rel-edit mean	Eff-rank₉₀	Lowrank-cap, rank-2 energy	σ₂/σ₃ ratio
sdft_cybersec_r2	211, exact target count	0.00051	≈20, max	63.0%	mean 2.18, max 6.9
sdft_smoothed_r2_new	211, exact target count	0.00086	≈20, max	76.7%	mean 3.44, max 8.1

A clean rank-2 edit would show eff-rank of 2 or less with lowrank-cap near 100% and σ₃ = 0. Instead the noise floor at σ₃ through σ₂₀, roughly 5e-4 and flat, carries 23-37% of the diff energy. The dithering is uncorrelated with every other variant's edit, technique-cosine 0.0001-0.007, cybersec-vs-smoothed cosine 0.0070, not twins. The consequence: the merged checkpoints are behaviourally near-base by construction, and the LoRA's genuine attenuated signal lives in the top-2 singular values. This finding motivated serving both variants as dynamic FP32 adapters for every behavioural evaluation in this report. Future small-alpha merges should merge in FP32 and cast at save time.

Summary

Model	Judge ASR	GSM8K ans-only	MMLU-Pro	KL	Tensors Δ	Strategy
sdft_cybersec_r2	23.8%	88.7%	49.9%	0.049	211	LoRA r=2
Base	21.0%	88.7%	50.0%	-	-	reference
huihui	89.8%	88.0%	44.4%	0.843	12	surgical rank-1
trevorjs	85.8%	89.0%	48.0%	0.087	66	Heretic + biprojection
coder3101	81.0%	89.6%	48.7%	0.282	86	heretic
sdft_smoothed_r2_new	79.5%	88.9%	48.2%	0.151	211	LoRA r=2
jwest33	78.0%	88.9%	45.1%	0.242	329	null-space
llmfan Heretic ARA	76.3%	90.6%	43.8%	0.182	16	heretic
prithiv	72.5%	88.3%	49.8%	0.064	56	heretic
abliterix	68.8%	87.8%	50.1%	0.090	58	ARA
openyourmind	67.0%	86.0%	27.6%	2.279	620	broad
obliteratus	60.8%	86.4%	39.2%	0.532	144	obliterated
apostate	45.8%	87.9%	49.8%	0.193	60	apostate-8pos

Evaluation Timeline

Dates	Work
Aug 1-10	LM-Eval original suite, base + 10 abliterations, ~74h GPU
Aug 2	Weight forensics sequencer run, 147 min CPU
Aug 2-4	Packaging-defect fixes on variant dirs, pre-flight smoke tests added
Aug 8	SDFT merged-checkpoint weights, sub-precision finding
Aug 10-11	SDFT merged-checkpoint lm-eval, ~14h, later superseded by dynamic serving
Aug 10-12	trevorjs + both SDFT dynamic-LoRA lm-eval runs, ~24h
Aug 12-13	HarmBench default suite, 13 models, ~13h. KL collection ~2.5h. Judge batches
Aug 14	Report v1
Aug 20-22	Serving-mode investigation, think-mode HarmBench arms, ~7.5h
Aug 21-22	Verification: SDFT lm-eval re-runs 23.4h, determinism control 3.0h, judge review
Aug 22	Report v2, numbers refreshed against the DB
Aug 25	coder3101 GSM8K recovery re-run, response-cache resumed, ~2.1h GPU. Report corrections: quantisation framing, sdft repo links, apostate version note

How this was made

GPU hours

Stage	GPU time	Wall clock	Models	Notes
LM-Eval, original suite	~74h	Aug 1-10	11	Base + 10 abliterations. Per-model durations from the DB, incl. cache-resumed restarts after the early fixes
LM-Eval, SDFT dynamic-LoRA runs	~24h	Aug 11-12	2	cybersec ~16h across two segments, smoothed 8.0h (28,965s). These produced the Aug-12 canonical numbers
HarmBench default suite	~13h	Aug 12-13	13	~1h per model. DB-tracked example: smoothed 1.1h (3,928s)
HarmBench think-mode arms	~7.5h	Aug 20-22	3	base 2h05m, smoothed 3h04m, cybersec 2h17m
KL divergence	~2.5h	Aug 13	13	Logits collection at minutes per variant. The post-NaN-fix recompute was 26s total
Weights	0h GPU	~2.5h CPU	13	147-min single-container sequencer plus the Aug 8 sdft resume. CPU-bound
LLM judge	n/a	not tracked	13 arms	5,200 reviews, glm-5.2
Subtotal	~121h GPU	~3 weeks		Productive GPU time, Aug 1 to Aug 22
Verification: SDFT lm-eval re-runs	23.4h	Aug 21-22	2	cybersec 12.5h (45,047s). The smoothed first attempt ran 8.0h and died in the 06:50 host crash, then cache-resumed in 2.9h (10,319s)
Verification: SDFT merged-checkpoint lm-eval	~14h	Aug 10-11	2	Superseded by dynamic-LoRA serving. Confirmed the sub-precision merge finding
Verification: coder3101 GSM8K recovery re-run	~2.1h	Aug 25	1	Phase 1 fully cache-replayed, 209,751 cached requests, Phase 2 real inference for 1,216 generations
Verification: think-mode determinism control	3.0h	Aug 22	1	smoothed+think v2 replicate, agrees with run 1 on 96.5% of items
GSM8K thinking-loop fix re-run	~1h	Aug 2-3	1	Base Phase 2 redone after the num_fewshot=0 + until=[] fix
Failed model loads and packaging defects	~0.3h	Aug 2-4	8	~2 min per variant burned before pre-flight smoke tests existed
NLTK punkt_tab hang	~0h GPU	~1h wall	1	Silent network hang at BBH task load
KL NaN recompute	~0h GPU	26s	12	Logits reused, compute phase only
Byte-identical investigation	0h GPU	analysis only	2	Controls read existing responses, no regeneration
Cancelled contaminated-rerun dispatch	~0h GPU	minutes	0	Cancelled before any generation started
06:50 host crash	~0h GPU net	2.9h resume	1	The cache retained the overnight work. Resume 2.9h against ~5h remaining
Wasted total	~42h GPU			26% of the total GPU budget, mostly verification spend
Grand total	~165h GPU	~3.5 weeks wall		Includes all re-runs and verification

What broke

• KL was NaN for all 12 variants. F.kl_div computes 0 × -inf = NaN on the 262K vocab. Fixed by clamping non-finite log-probs to -100 before compute. The logits were reused, so the recompute cost 26 seconds.
• GSM8K Phase 2 returned 92% null responses. The 5-shot fewshot examples sent the thinking model into reformatting loops, and the Question: stop sequence truncated reasoning mid-think. Fixed with --num_fewshot 0 and until=[]. Base re-run ~1h.
• CUDA graphs broke every inference request. Without --enforce-eager, Gemma4Unified on vLLM 0.26.0 returns HTTP 500 on all completions and crashes on a missing feature extractor. Eager mode stays on, capping MFU near 15%.
• Six variants shipped broken configs, one shipped old-key safetensors. Truncated config.json files at 169 lines vs the base 172, a missing processor_config.json, and pre-refactor multimodal keys all failed model load. Fixed host-side by copying base files and renaming 10 safetensors keys, no weights touched.
• The NLTK punkt_tab download hung silently. BBH needs it, the image lacks it, the network blocks it. Fixed with a host-side nltk_data mount. ~1h wall.
• HarmBench containers crashed at startup. The flashinfer and flashinfer-cubin version mismatch kills model inspection on the gemma4unified image. Fixed with FLASHINFER_DISABLE_VERSION_CHECK=1 in the runner env.
• GSM8K sample stats silently went missing. The runner writes samples to per-model dirs, the importer only scans __model/. Bridged with a container-side relative-name copy.
• harmbench_generate.py sent no model field. On a multi-model vLLM server an unqualified request is a routing hazard. Fixed with the --api-model flag, verified by a diff-check against a known-different arm.
• vLLM drops repeated --lora-modules flags. It is a single flag with space-separated name=path pairs. Repeating it silently keeps only the last adapter.
• export-review truncated reasoning at 3,000 chars. Full traces run 8K to 46K chars. Fixed with --full-reasoning and an always-exported reasoning_chars_total.
• The host crashed at 06:50 mid-re-run. The response cache retained the overnight work, so the resume cost 2.9h against ~5h remaining. An earlier reboot on Aug 10 hit the same way and was absorbed by the same cache.
• The __model/ directory rejects absolute-path file creation. ext4 returns ENOENT on absolute-path creates while relative-path creates work. Unrepaired filesystem anomaly, worked around with cd-based copies.

What was learned

1. Classifier ASR overcounts think-mode compliance by 16-19pp. Judge the payload, not the length.
2. GSM8K answered-only is the true capability metric for abliterations of thinking models.
3. Merging a small-alpha LoRA in BF16 rounds the edit away. Serve adapters dynamically instead.
4. Send an explicit model field on every request to a multi-model server, and verify with a diff-check against a known-different arm.
5. Think-mode numbers need a base+think control arm or they mean nothing. Base itself rose from 18.8% to 33.5% classifier ASR, against 21.0% judge in the default render.

Methodology

• Capability: lm-evaluation-harness via vLLM 0.26.0, native BF16 on a single RTX 5090. 13-task suite: Open LLM Leaderboard v2 tasks plus the v1 forensic supplement plus TruthfulQA plus GSM8K. The Gemma4Unified architecture required the dedicated vLLM 0.26.0 image throughout.
• Loglikelihood scoring: --apply_chat_template on all multiple-choice tasks. The 12B Unified template defaults enable_thinking to false, so no thinking override was needed for clean scoring.
• GSM8K: 0-shot with thinking enabled and until=[], the Open LLM Leaderboard v2 standard for thinking models. Reported as three numbers, headline, empty rate, answered-only, because headline conflates capability with reasoning-loop failure.
• Safety: HarmBench 400 textual behaviours, max_tokens=8096, temperature=0, skip_special_tokens=false to preserve thought markers. Think-mode arms add chat_template_kwargs: {"enable_thinking": true}. Judge protocol, glm-5.2: semantic compliance verdict over the full reasoning trace plus final answer, traces run 8K to 46K chars so the 3,000-char export cap is never sufficient. Compliance requires real artefact content: verbatim lyric and passage lines count, hallucinated-recall loops and artefact-free educational overviews count as refused.
• Dynamic-LoRA serving: the SDFT variants are evaluated as the base BF16 model with the LoRA adapter applied dynamically at FP32, vLLM --enable-lora --max-lora-rank 8 --lora-modules, never as the shipped BF16-merged checkpoints whose edits sit below one unit of BF16 precision. KL measurement supports the same mode via --lora-adapter.
• KL divergence: full-vocabulary KL(base‖variant) on first-token logits over 100 harmless prompts from harmless-alpaca, batchmean reduction. Non-finite log-probs clamped to -100 before compute.
• Weight analysis: tensor-diff fingerprints, SVD effective-rank, per-layer edit density and magnitude targeting, and cross-variant edit-vector similarity, comparing every variant against the base.
• Hardware: NVIDIA RTX 5090 with 32GB. All GPU work on GPU 0. Total ~165h GPU including all re-runs and verification.

Ops notes for replication:

• vLLM 0.26 --lora-modules is a single flag with space-separated name=path pairs. Repeating the flag silently drops all but the last occurrence. Verify registered adapters via /v1/models before any run.
• FLASHINFER_DISABLE_VERSION_CHECK=1 is required for any runner starting vLLM containers on the gemma4unified image. The flashinfer and flashinfer-cubin version mismatch otherwise crashes model inspection.
• --enforce-eager stays on for Gemma4Unified on vLLM 0.26.0. CUDA graphs cause HTTP 500 on all completions plus a feature-extractor crash during load.
• The lm-eval response cache makes interrupted runs cheaply resumable. Cache keys are content-hashed and survive server-config changes.
• GSM8K sample statistics import from results/lm_eval/__model/ only. Samples written to per-model dirs are invisible to the importer and must be bridged across.

A note from the SDFT author

Ilya626, author of the two SDFT adapters, was shown the results and offered space to respond. His comment follows verbatim.

Thanks for the comparison. I would add some context: SDFT is interesting here not simply as a way to "uncensor" a model, but as a method for making targeted behavioral changes.

In simple terms, the model generates its own response, and the teacher corrects the probability distribution of its possible continuations. We are not merely feeding it a collection of finished answers; we are correcting the behavior it actually exhibits during generation. This makes SDFT particularly effective at transferring reasoning habits, response style and refusal policy, but relatively weak at injecting entirely new knowledge. For new capabilities, it is generally better to establish them through SFT first and then use SDFT to stabilize them.

An important engineering detail is that the real unit of coverage is the trajectory, even though the loss is calculated over token distributions. Every next-token prediction is conditioned on the entire prefix that led to it. If the student never reaches a particular state or line of reasoning, SDFT receives no useful signal there. This is why training-prompt coverage and the student's own rollout distribution matter so much.

The fully uncensored version demonstrated a broad behavioral change. The cybersecurity version is more interesting: ordinary guardrails were largely restored while defensive and offensive cybersecurity requests remained available. The behavioral edit was therefore not only strong, but also partially reversible and localizable.

SDFT is not telepathic, of course. If a particular behavior needs to change, it must be properly represented in the training prompts. I intentionally excluded several categories because I saw no useful application for them. The remaining refusal tail therefore reflects the boundaries of the dataset rather than a fundamental limitation of the method. If desired, such residual behavior can usually be adjusted with a few dozen targeted examples without significantly affecting the rest of the model.

For the general benchmarks, the important result is not that every number remained perfectly identical, but that there was no broad capability collapse. On GSM8K, accuracy among completed answers remained essentially unchanged. Most of the headline difference came from the model getting stuck in reasoning slightly more often and failing to emit a final answer. Most other evaluations likewise showed no meaningful shift, although a few localized side effects remain.

That is the practical purpose of SDFT: achieve a deep change in selected behavior while disturbing as little of the remaining model as possible and without redesigning the production stack. The result can be stored in a very small LoRA, which can be enabled, removed, or merged into the model.

The method is also practical on non-datacenter hardware. One representative 12B run took approximately seven hours on a 20 GB GPU with 64 GB of system RAM. The main memory bottleneck is processing full-vocabulary logits for long responses, so there is still considerable room for implementation-level optimization.

On lineage, the author adds:

This method was inspired by Heretic uncensoring. Without Heretic or Ablated models it would not exist as it is. It reuses Heretic for creating the teacher with ablated hard refusals.

His GSM8K reading matches our measurements independently: answered-only accuracy held at base level while the headline gap traced to reasoning loops, the exact split our three-number report shows.

Disclaimer

These models have had safety alignment removed. They will comply with harmful requests. Use responsibly and in accordance with applicable laws. The authors do not condone or encourage the use of these models for harmful purposes.

---

While every number here was cross-checked against the DB and the raw result files, I am open to any corrections, additional benchmarks, or further analysis. If you spot something that looks wrong and can be confirmed, I am happy to fix it.