msuiche/Qwen3.8-27B-abliterated-cvec

Qwen3.8-27B: abliteration as an adapter, not a checkpoint

Two small files that remove refusal behaviour from Qwen/Qwen3.8-27B without redistributing the model. The base checkpoint stays byte-identical.

file	size	needs
adapter_model.safetensors + adapter_config.json	8.6 MB	stock peft, verified
Qwen3.8-27B-refusal-cvec-mask005-pooled-L1-63-a1.gguf	1.3 MB	a runtime that understands dspark.mode. Experimental

For comparison, published abliterations of this same model are 30.9 GB and 20.6 GB.

Start with the LoRA. It works on stock tooling. The GGUF is here because on architectures where a LoRA is impossible the projective format is the only route, and this is the reference implementation of it.

---

Results

Measured on Qwen/Qwen3.8-27B at revision 1d4bf0f2ff60, greedy decoding, 300 new tokens, three-way scored with a degeneracy guard.

suite	unmodified	LoRA	control vector
general refusal (n=32, in-sample)	3.1 %	81.2 %	84.4 %
harmless control (n=32, held out)	100 %	100 %	100 %
offensive-security holdout (n=32, private)	12.5 %	100 %	96.9 %
capability probe	11/12	11/12	12/12

Delivery, not refusal rate, so higher is better. The harmless control is held out of the derivation entirely. The security suite contains no content resembling the derivation contrast.

Over-refusal is 0 points on the held-out control. An earlier in-sample control read −15.6, which is a stricter bound by construction: the direction was fitted to separate the harmful set from that control, so it sits on the decision boundary.

---

Usage

from transformers import AutoModelForImageTextToText, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen3.8-27B"
model = AutoModelForImageTextToText.from_pretrained(base, dtype="bfloat16", device_map="cuda:0")
model = PeftModel.from_pretrained(model, "msuiche/Qwen3.8-27B-abliterated-cvec")
tok = AutoTokenizer.from_pretrained(base)

merge_and_unload() bakes it in if you want a merged checkpoint.

Do not scale the adapter

lora_alpha / r = 1.0, and α is already baked into lora_A. Raising it does not strengthen removal. It reflects the component instead of removing it:

α = 1   h·d̂ → 0            the component is removed
α = 2   h·d̂ → −(h·d̂)       the component is REFLECTED

At α=2 this model refused 37.5 % of entirely harmless prompts (sourdough, birdwatching, repotting a houseplant) with factual capability perfectly intact. That is the same vector installing the behaviour rather than removing it.

If you want less than full strength, drop layers instead

This adapter covers layers 1 to 63. You do not have to use all of them, and coverage is the safe dial where α is not. Measured on a 43-layer model, same direction and same α, varying only how many layers were touched:

layers steered	refusal remaining
6	18.0 %
16	3.8 %
29	0.0 %

Partial coverage does less of the same thing. Raising α does something different and dangerous. So to tune strength, subset the layers and leave α at 1.

That also makes targeted work possible. The direction is applied per layer, so you can confine it to a depth range and ask what that range contributes, rather than treating the model as one switch. Two cautions from doing this:

• Do not include layer 0. Steering it silenced this model completely: 96 outputs out of 96 empty, capability 0/12.
• Early layers matter more than their separation scores suggest. Dropping layers 10 to 17 from a working span cost 9.4 points of delivery, even though a shuffled-label null test rates them weak.

To build a narrower adapter, re-run the derivation with the span you want. Editing this file's tensor list is equivalent, since each layer's lora_A/lora_B pair is independent.

---

One file, every quantisation

lora_A = −α·d̂ᵀW is computed once from the bf16 weights. Load it onto a quantised copy of the same checkpoint and the error enters only through d̂ᵀ(W − W_q)x, which is the quantisation noise projected onto a single direction and therefore suppressed by about 1/√5120:

base	weight error	error in the projection
bf16	0 %	0.00 %
int8	0.67 %	0.61 %
int4	12.0 %	3.57 %

At int4 the weights are twelve percent wrong and the intervention is still ninety-six percent correct. One adapter should therefore cover the FP8, NVFP4, int8, int4 and GGUF re-encodings of this checkpoint, rather than each needing its own abliterated upload.

This is measured on the arithmetic, not on behaviour. Delivery on a quantised base has not been benchmarked.

---

What this is, and its limits

It is a rank-1 LoRA computed in closed form, not trained. No optimiser, no training data:

B = d̂                  a direction in the residual stream
A = −α · d̂ᵀW           one matrix-vector product against the base weights

d̂ comes from a difference of means over 32+32 prompts at the last prompt token, with the top 0.5 % of dimensions masked. This checkpoint has a massive-activation dimension running 75 to 434 times the median, which otherwise dominates the estimate. Applied at both residual writers across layers 1 to 63: 126 matrices, ‖ΔW‖/‖W‖ mean 0.0143.

Checkpoint-bound. lora_A contains W, so this file is valid for Qwen/Qwen3.8-27B at revision 1d4bf0f2ff60 and its re-quantisations. It is not portable to another model or another size. The method transfers to any dense model. The file does not.

peft attaches an adapter to layer 0. target_modules is matched by suffix, so peft hooks layer 0 as well, which this file deliberately excludes. It is inert only because peft zero-initialises B, verified here as exactly zero contribution and zero weight change after merge. That is a property of peft's defaults, not of this file, and it matters because steering layer 0 silences the model.

Untested: vLLM and llama.cpp. Both should work, since vLLM permits LoRA on any linear layer and llama.cpp has the qwen35 architecture, but neither has been loaded.

The GGUF needs a patched runtime. It declares dspark.mode = project, meaning h ← h − α(h·d̂)d̂. llama.cpp's built-in control vectors are additive, h ← h + s·d̂, which is a different operation: it pushes every token along the axis instead of removing the component, and it fails silently. A reader that does not understand dspark.mode must refuse the file rather than fall back to adding. Support is proposed upstream and unmerged.

---

Scoring caveat

The harness scores REFUSE / DEFLECT / COMPLY plus a degeneracy guard. It does not yet separate a fourth state we found: the steered model rejecting a question's premise and then answering in full. *"I cannot provide instructions on how to make sourdough rise improperly…"*, followed by 1,300 characters of correct instructions. Those score as refusals, so over-refusal figures here are an upper bound.

Provenance

base            Qwen/Qwen3.8-27B @ 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
contrast        32 harmful / 32 form-matched harmless, cross-domain
estimator       difference of means, 0.5% massive-activation masked, pooled
hook            post-layer residual, layers 1-63, alpha = 1
gguf sha256     dfdbcc06b30498224270e2f54d5784510b18aa185e0a50321e763373de83c805
lora sha256     32980b48abc4018255a1ee5a5c2fbd55ee110132482dd18c29b86e0b150334f7

A third party's direction, recovered from published weights by SVD and run through the same harness, scores 90.6 / 100 / 100, which is better than this one on general refusal. That is recorded because it is true, and because the gap is the contrast rather than the method: their prompt sets pushed through this pipeline reproduce their result.

Intended use

Security research and evaluation. Measuring how robustly a capability is gated requires being able to switch the gate off and observe what changes, and that work is only possible on open-weight models.