granite-4.2-8b-decensored

Uncensored (abliterated) version of ibm-granite/granite-4.2-8b, produced with heretic v2.0.0.dev0.

Ready-to-use GGUF quantizations: qcokvd1/granite-4.2-8b-decensored-GGUF

Method

Directional ablation ("abliteration") of the refusal direction in attention out-projection and MLP down-projection matrices, applied as low-rank LoRA corrections and merged back into the original BF16 weights. Abliteration parameters were tuned by Optuna (TPE) over 200 trials, co-minimizing the refusal keyword rate and KL divergence from the original model.

The optimization itself ran on-the-fly quantized to NF4 (bitsandbytes); the 4-bit working precision only affects how the correction was computed — the exported weights are derived from the original BF16 model.

Results

Metric	Original	Decensored
Refusal keyword rate (mlabonne/harmful_behaviors, 100 prompts)	97/100	33/100
KL divergence vs original	—	0.0655

Selected trial #11 (lowest refusal rate on the Pareto front), parameters: direction_index = 29.89, attn.o_proj: max_weight 1.28 @ layer 31.07, min_weight 0.93, distance 9.46, mlp.down_proj: max_weight 1.46 @ layer 23.56, min_weight 1.43, distance 9.86.

Quality spot-check (code / math / SQL / logic, 9 prompts, greedy): answers match the original model, <think>-reasoning intact — see evaluation/. Raw Optuna study journal for resuming/extending the optimization: heretic-study.jsonl (rename to your model-path-derived name in heretic's checkpoints/ directory to resume).

Reproduction configuration

model = "<local path to ibm-granite/granite-4.2-8b>"
quantization = "bnb_4bit"
export_strategy = "merge"
trial_index = 0        # lowest refusal on the Pareto front
batch_size = 0         # auto (chosen: 128 on RTX 5060 Ti 16GB)
n_trials = 200

Hardware reference: RTX 5060 Ti 16 GB, ~7.5 h wall clock including export.

Disclaimer

Safety guardrails of the base model have been removed. You are solely responsible for how you use and deploy this model. The model may produce inaccurate, biased or unsafe content.

Base model © IBM, released under Apache-2.0; this redistribution complies with the license and attributes the original authors.