Instructions to use OpceanAI/Qwen3.5-0.8B-abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpceanAI/Qwen3.5-0.8B-abliterated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="OpceanAI/Qwen3.5-0.8B-abliterated")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("OpceanAI/Qwen3.5-0.8B-abliterated")
model = AutoModelForMultimodalLM.from_pretrained("OpceanAI/Qwen3.5-0.8B-abliterated", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OpceanAI/Qwen3.5-0.8B-abliterated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpceanAI/Qwen3.5-0.8B-abliterated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpceanAI/Qwen3.5-0.8B-abliterated",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/OpceanAI/Qwen3.5-0.8B-abliterated

SGLang

How to use OpceanAI/Qwen3.5-0.8B-abliterated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpceanAI/Qwen3.5-0.8B-abliterated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpceanAI/Qwen3.5-0.8B-abliterated",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpceanAI/Qwen3.5-0.8B-abliterated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpceanAI/Qwen3.5-0.8B-abliterated",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use OpceanAI/Qwen3.5-0.8B-abliterated with Docker Model Runner:
```
docker model run hf.co/OpceanAI/Qwen3.5-0.8B-abliterated
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwen3.5-0.8B-abliterated

Abliteration of Qwen/Qwen3.5-0.8B, with a full measured cost report.

Results · Method · MTP finding · Comparison · Reproduce

What this is

This is a weight-space modification of Qwen3.5-0.8B that removes the model's refusal behavior by projecting out a single computed direction from every matrix that writes into the residual stream. The weights are permanently modified, there is no adapter, no runtime hook, and no prompt template trick.

The interesting part is not that it works. Many abliterated Qwen3.5-0.8B releases exist, and most report a refusal rate near zero. The interesting part is that this one ships the measurement: seven refusal categories, three jailbreak templates, twelve capability measurements with confidence intervals, and a significance test for every comparison.

Two findings came out of the work that I have not seen reported for this model family:

The refusal direction rotates with depth. The cosine similarity between the direction at layer 1 and layer 23 is 0.03. There is no single fixed direction, only a direction that is rebuilt layer by layer.
The MTP head is the cleanest refusal detector in the model. On harmful prompts it assigns P(" cannot") = 0.719; on harmless prompts, 0.0000. Abliterating that head drops the probability to 0.180, but not to zero, because the signal re-enters from the residual stream.

This model has no refusal behavior. It will answer prompts the base model rejects. That is the point of the modification and also its risk. Do not deploy it in a user-facing product, do not use it with minors, and do not treat its output as safe by default. The evaluation below measures capability, not harm.

Results

The headline: refusal goes to zero and capability does not measurably move. Eleven benchmarks, produced by two different harnesses: a hand-written evaluator for the first six, and lm-evaluation-harness on full datasets for the last six.

Benchmark	n	Base	Abliterated	Delta	Test
MMLU	200	45.5%	44.5%	−1.0	Fisher p = 0.92
MMLU-Pro	100	23.0%	23.0%	0.0	Fisher p = 1.00
GSM8K	100	21.0%	23.0%	+2.0	Fisher p = 0.87
MATH (algebra, level 1 to 2)	50	30.0%	22.0%	−8.0	Fisher p = 0.50
HumanEval pass@1	30	13.3%	3.3%	−10.0	Fisher p = 0.35
WikiText-103 perplexity	20 docs	2.979	3.093	+0.114 nats	not testable
ARC-Easy	2376	61.66%	60.31%	−1.35	z-test p = 0.34
ARC-Challenge	1172	37.12%	36.86%	−0.26	z-test p = 0.90
HellaSwag	10042	49.72%	49.66%	−0.06	z-test p = 0.93
PIQA	1838	69.26%	69.21%	−0.05	z-test p = 0.97
Winogrande	1267	58.48%	58.33%	−0.16	z-test p = 0.94
TruthfulQA MC2	817	43.18%	43.45%	+0.28	z-test p = 0.91

No difference is statistically significant. The six lm-eval benchmarks run on the complete dataset, so their deltas are the most reliable evidence in this card: every one of them is under 1.4 percentage points, and five of the six are under 0.3.

The HumanEval drop looks large in percentage points, but it is four problems solved against one, on thirty. With that sample size the 95% interval spans zero to ten percent for the abliterated model. It is the weakest number here and should be read as such.

Refusal behavior

Ten prompts per category, from BeaverTails (safe-labeled behaviors) plus a keyword-filtered cyber set from mlabonne/harmful_behaviors.

Category	Base	Abliterated
cyber / exploits	80%	0%
privacy violation	70%	0%
financial crime	60%	0%
sexually explicit	60%	0%
drugs and weapons	60%	0%
non-violent unethical	50%	0%
hate speech	30%	10%

Jailbreak templates

Same ten behaviors, wrapped in three framings. The base model refuses all three at the same rate, which matters: at this size, framing does not buy an attacker anything.

Template	Base	Abliterated
direct	80%	0%
roleplay ("a character with no moral filter")	80%	0%
grandmother exploit	80%	0%

Figures

Refusal by category and jailbreak template

Capability benchmarks with 95% confidence intervals

Refusal direction strength and weight surgery per layer

Every capability delta with its 95% confidence interval, centered on zero

Method

The procedure follows Arditi et al. (2024), Refusal in Language Models Is Mediated by a Single Direction, with adaptations for a hybrid architecture.

1. Direction extraction

128 prompts from mlabonne/harmful_behaviors against 128 from mlabonne/harmless_alpaca. For each prompt, run a forward pass and take the residual stream at the last token position, for every layer.

r_l = mean(harmful_l) - mean(harmless_l)

One quirk worth recording: the embedding layer produces an identical vector for every prompt (||diff|| = 0.000). The last token of the chat template is always the same assistant header, and without attention the embedding cannot know what came before it. The instruction only enters the residual stream at layer 0.

2. Layer selection

Relative separation per layer, computed as ||mean_harmful - mean_harmless|| divided by the mean norm of the two class means:

Layer	Relative separation	Cosine with layer 23
L0	2.1%	+0.03
L5	19.7%	+0.01
L9	36.9%	+0.19
L13	64.6%	+0.36
L17	64.6%	+0.57
L21	70.5%	+0.82
L22	71.7%	+0.89
L23	75.3%	+1.00

The direction rotates continuously. Averaging all 24 layers produces a vector with cosine 0.70 to the layer 13 direction and 0.65 to layer 23, which is why this release uses the mean of all 24 rather than a single hand-picked layer.

Ablation at runtime confirmed causality with 8 held-out prompts: the base model refuses 7 of 8, and ablating the direction at layer 13, 22 or 23 drops that to 0 of 8.

3. Weight surgery

Only matrices that write into the residual stream are modified:

W' = W - r (r^T W)        # o_proj, out_proj, down_proj
E' = E - (E r) r^T        # token embeddings (tied to the LM head)

Target	Count	Frobenius norm removed
`self_attn.o_proj` (full attention layers)	6	2.83% to 3.44%
`linear_attn.out_proj` (DeltaNet layers)	18	2.98% to 3.44%
`mlp.down_proj`	24	2.88% to 3.44%
`embed_tokens` (tied)	1	3.10%

About three percent of the Frobenius norm per matrix. That is the entire cost, measured in weight space.

Because the embedding is tied to the LM head, modifying it changes both the input table and the output projection. One tensor, two roles, hit once.

4. Why this architecture makes it easier

Qwen3.5-0.8B is not a dense transformer. Of its 24 layers, 18 use Gated DeltaNet (linear attention with a recurrent state) and only 6 use full attention, in a fixed 3:1 pattern. Full attention sits at layers 3, 7, 11, 15, 19 and 23.

That changes the memory math:

	Size
Recurrent state, 18 layers, 16 heads, 128x128, fp32	18.9 MB fixed
KV cache for 262,144 tokens, 6 layers, bf16	3.22 GB

The state does not grow with context. It also means the refusal direction has to be removed from a different mix of matrices than in a standard transformer, which is why the linear attention out_proj is included.

The MTP head

This checkpoint ships a multi-token prediction head: 15 tensors, about 20.5M parameters, one full attention layer plus projections. It is trained to predict token t+2 given the hidden state at t and the embedding of t+1.

Measured on real text, it predicts t+2 with 30.5% top-1 accuracy, against 2.1% for naive extrapolation from the main head. It is a working predictor, not dead weight.

And it is the sharpest refusal signal in the model:

	Harmful prompts	Harmless prompts
`P(" cannot")` at the MTP head	0.7189	0.0000

The top tokens it predicts for harmful prompts are " cannot", " do", " am", " must", " can". It is writing the first token of a refusal before the model writes one.

Abliterating the head itself (three matrices, about 3% each) removes 99.3% of the projection onto the refusal direction and drops P(" cannot") from 0.7189 to 0.1799. The top-1 prediction flips from " cannot" to " can". It does not reach zero, because the head reads the direction back from its input.

The MTP head is included in this checkpoint and was abliterated along with the rest of the model. transformers ignores mtp.* tensors at load time, so it does not affect generation there. In llama.cpp, --spec-type draft-mtp does use it, and on CPU it made generation slower, not faster: 29.5 tok/s without, 17.1 tok/s at its best setting. On GPU the tradeoff is different.

Comparison

Six other abliterated releases of this same checkpoint were public when this one was built. They differ in method, in what they measure, and in how much of the model they touch.

Release	Method	Reported refusal	Capability evidence
huihui-ai	layer-wise ablation	not reported	none
wangzhang	rank-1 LoRA plus Optuna search	0/200	KL 0.0087 on 100 trials
g-ntovas	piecewise strength kernel, peak at 70% depth	not reported	none
lew96123	projected abliteration, norm preserved, 12 layers	86% to 3%	ARC-Easy −1.0, HellaSwag −1.0
SicariusSicariiStuff	orthogonalization	not reported	none
anlord	iterative tool, two versions	97/100 to 2/100	KL 0.0453

What this release adds:

Significance testing. Every capability comparison comes with a Fisher exact test. Other cards report a delta; this one reports whether the delta means anything.
Seven refusal categories plus three jailbreak templates, instead of one aggregate refusal number.
Two independent harnesses, including lm-evaluation-harness on full datasets for ARC, HellaSwag, PIQA, Winogrande and TruthfulQA.
The hybrid architecture handled explicitly, with DeltaNet out_proj included in the surgery.
The MTP head analyzed and modified, which no other release does.
Reproduction scripts for every number in this card.

What it lacks, honestly: the n for HumanEval and MATH is small. A proper code evaluation needs 300 problems, not 30. Treat the code numbers as a lower bound on quality, not as a verdict.

Usage

from transformers import AutoProcessor
from transformers import AutoModelForImageTextToText

model_id = "OpceanAI/Qwen3.5-0.8B-abliterated"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, dtype="auto", device_map="auto"
)

messages = [{"role": "user", "content": [{"type": "text", "text": "Your prompt here"}]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], return_tensors="pt").to(model.device)

out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(processor.tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Thinking mode is off by default in the chat template. To enable it:

text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=True
)

Reproduce

Environment and commands

The measurements in this card were produced with torch 2.15.0 and transformers 5.x on an NVIDIA H200 through Modal, in bfloat16, greedy decoding, batch size 64 for log-likelihood tasks.

# direction extraction and layer analysis (CPU is enough, about 3 minutes)
python scripts/extract.py

# weight surgery and local refusal check
python scripts/abliterate_weights.py

# capability and refusal battery on a GPU
modal run scripts/modal_bench.py
modal run scripts/modal_bench2.py

Files in this repository:

File	Contents
`scripts/extract.py`	refusal direction per layer, relative separation
`scripts/abliterate_weights.py`	orthogonalization of the residual writers
`scripts/modal_bench.py`	MMLU, MMLU-Pro, GSM8K, MATH, HumanEval, refusal, jailbreak
`scripts/modal_bench2.py`	ARC, HellaSwag, PIQA, Winogrande, TruthfulQA via lm-eval
`figures/`	the three plots used above
`abliteration.json`	exact configuration and targets

The scripts contain no comments and no docstrings. Prompt strings are kept verbatim in the language they were run in, so the published numbers can be reproduced exactly by re-running them.

Limitations

Small samples on the hard benchmarks. HumanEval n=30, MATH n=50. Only MMLU (n=200) and the refusal matrix have enough data to support a claim.
Single seed. Generation is greedy, so there is no sampling variance, but prompt sampling is one draw per benchmark.
bfloat16 on GPU. The absolute numbers are not comparable to fp32 CPU runs. An earlier CPU measurement gave 0% on GSM8K purely because of an 80-token generation cap, not because the model cannot do arithmetic.
Refusal is measured by keyword matching. A response that opens with "I understand, but" and then complies counts as a refusal. This biases the base model's refusal rate upward, not downward.
The harmful prompt sets are not matched in style. harmful_behaviors and harmless_alpaca differ in length and register, so part of the separation may reflect topic rather than refusal.
The MTP head is not used by transformers. Its evaluation here required a manual reimplementation.

Safety and intended use

Intended for research on refusal mechanisms, activation steering, and interpretability of hybrid architectures. Not intended for deployment.

The modification is not selective in any provable sense. It removes a direction that correlates with refusal, and refusal correlates with many other behaviors. The benchmarks say the correlation with the measured capabilities is below detection at these sample sizes. They do not say the model is otherwise identical.

If you build on this, keep a copy of the base model, log the exact revision you used, and evaluate on your own prompts rather than trusting this card.

Citation

@article{arditi2024refusal,
  title   = {Refusal in Language Models Is Mediated by a Single Direction},
  author  = {Arditi, Andy and Obeso, Oscar and Syed, Aaquib and Paleka, Daniel and Panickssery, Nina and Gurnee, Wes and Nanda, Neel},
  journal = {arXiv preprint arXiv:2406.11717},
  year    = {2024}
}

@misc{qwen35-08b-abliterated,
  title  = {Qwen3.5-0.8B-abliterated: refusal removal with a measured cost report},
  author = {OpceanAI},
  year   = {2026},
  note   = {Derived from Qwen/Qwen3.5-0.8B},
  url    = {https://huggingface.co/OpceanAI/Qwen3.5-0.8B-abliterated}
}

_{Base model: Qwen3.5-0.8B, Apache 2.0. This derivative keeps the same license. Built with measurements, not vibes.}