Qwen3-32B-Abliterated

Base model: Qwen/Qwen3-32B

For research purposes only. This model has had its safety refusals surgically removed. It will comply with requests that the base model would refuse. Do not deploy this model in any user-facing product or service. The authors are not responsible for any misuse.

---

What is this?

This is an abliterated version of Qwen3-32B produced using the weight orthogonalization technique from:

Refusal in Language Models Is Mediated by a Single Direction Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, Neel Nanda — arXiv:2406.11717

The paper shows that a model's refusal behaviour is encoded along a single direction r̂ in its residual stream. Removing this direction from every weight matrix that writes to the residual stream permanently disables the refusal behaviour while leaving general capabilities intact.

Empirical result on this model: ablation reduced refusal rate on harmful prompts from 92% → 1% (99/100 JailbreakBench prompts complied with).

---

How it works

The refusal direction r̂ was extracted at layer 46, position -8 (the newline between <|im_end|> and <|im_start|>assistant — the role-boundary token where the model commits to its response). This position carries the clearest linear separation between "I will refuse" and "I will comply" across the residual stream.

The weight modification applied to every output-projection matrix:

W' = W - r̂r̂ᵀW

Matrices modified: embed_tokens, self_attn.o_proj and mlp.down_proj in all 64 layers, and lm_head.

---

Usage

This model uses Qwen3's ChatML format. Pre-fill the empty think block to skip the thinking phase and get direct responses:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "HaseebAsif/Qwen3-32B-Abliterated",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("HaseebAsif/Qwen3-32B-Abliterated")

def chat(instruction, system="You are a helpful assistant."):
    prompt = (
        f"<|im_start|>system\n{system}<|im_end|>\n"
        f"<|im_start|>user\n{instruction}<|im_end|>\n"
        "<|im_start|>assistant\n<think>\n\n</think>\n\n"
    )
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    with torch.no_grad():
        out = model.generate(
            **inputs,
            max_new_tokens=512,
            do_sample=True,
            temperature=0.7,
            top_p=0.9,
            repetition_penalty=1.05,
        )
    return tokenizer.decode(out[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True)

print(chat("Explain how nuclear reactors work."))

Recommended generation settings

Parameter	Value
temperature	0.6–0.8
top_p	0.8–0.95
repetition_penalty	1.05–1.1
max_new_tokens	512–2048

Memory requirements

Format	VRAM
bf16 (this model)	~64 GB
4-bit (load with BitsAndBytes)	~20 GB

To load in 4-bit:

from transformers import BitsAndBytesConfig
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)
model = AutoModelForCausalLM.from_pretrained("HaseebAsif/Qwen3-32B-Abliterated", quantization_config=bnb, device_map="auto")

---

Safety notice

This model will not refuse harmful requests. It is provided solely for:

• Academic research into LLM safety mechanisms
• Study of representation engineering and mechanistic interpretability
• Red-teaming and safety evaluation in controlled research settings

Do not use this model to generate content that causes real-world harm. The base model's terms of service (Qwen license) still apply.

---

Citation

@article{arditi2024refusal,
  title={Refusal in Language Models Is Mediated by a Single Direction},
  author={Andy Arditi and Oscar Obeso and Aaquib Syed and Daniel Paleka and
           Nina Panickssery and Wes Gurnee and Neel Nanda},
  journal={arXiv preprint arXiv:2406.11717},
  year={2024}
}