Instructions to use JonathanColetti/Qwen3.8-27B-Uncensored with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use JonathanColetti/Qwen3.8-27B-Uncensored with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="JonathanColetti/Qwen3.8-27B-Uncensored")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("JonathanColetti/Qwen3.8-27B-Uncensored")
model = AutoModelForMultimodalLM.from_pretrained("JonathanColetti/Qwen3.8-27B-Uncensored", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use JonathanColetti/Qwen3.8-27B-Uncensored with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "JonathanColetti/Qwen3.8-27B-Uncensored"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JonathanColetti/Qwen3.8-27B-Uncensored",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/JonathanColetti/Qwen3.8-27B-Uncensored

SGLang

How to use JonathanColetti/Qwen3.8-27B-Uncensored with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "JonathanColetti/Qwen3.8-27B-Uncensored" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JonathanColetti/Qwen3.8-27B-Uncensored",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "JonathanColetti/Qwen3.8-27B-Uncensored" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JonathanColetti/Qwen3.8-27B-Uncensored",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use JonathanColetti/Qwen3.8-27B-Uncensored with Docker Model Runner:
```
docker model run hf.co/JonathanColetti/Qwen3.8-27B-Uncensored
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwen3.8-27B-Uncensored

Qwen3.8-27B with refusal behaviour substantially reduced, not eliminated — see Measured behaviour for the numbers. Capabilities, training data, and architecture are otherwise unchanged.

These are the bf16 safetensors. For local inference under llama.cpp, see JonathanColetti/Qwen3.8-27B-Uncensored-GGUF, which ships imatrix quantizations with the multi-token prediction head retained. Try it first in the demo Space.

The MTP head is present and verified. Abliteration drops the mtp.* tensors: the model is re-saved through transformers, which does not carry the MTP module, while config.json still advertises mtp_num_hidden_layers. All 15 mtp.* tensors were grafted back from the base checkpoint and the inventory was asserted against the source before upload.

Method

Refusal directions removed with Heretic, which co-minimizes refusal count against KL divergence from the base model. No hand-written refusal-removal code, no fine-tuning, no additional training data.
Abliteration runs at bf16 (no 4-bit quantization); the resulting LoRA is merged into the bf16 base, so these weights are not a quantized round trip.
Only attn.o_proj and mlp.down_proj are modified, 64 modules each. The mtp.* tensors are copied verbatim from the base checkpoint after merging.
200 optimization trials. The published checkpoint is one point on the resulting Pareto front, selected for the lowest refusal count; the full front is below.

Overview


Base	Qwen/Qwen3.8-27B
Architecture	`Qwen3_5ForConditionalGeneration`
Layers	64
Vocab	248320
MTP layers	1
Vision	yes
Context	262144
Precision	bf16

Usage

import torch
from transformers import AutoProcessor, AutoModelForImageTextToText

model_id = "JonathanColetti/Qwen3.8-27B-Uncensored"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": [{"type": "text", "text": "Hello"}]}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt").to(model.device)

out = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(out[0], skip_special_tokens=True))

bf16 needs roughly 55 GB of VRAM. The chat template opens a <think> block by default; pass enable_thinking=False to apply_chat_template for direct answers. Qwen's recommended sampling is temperature=1.0, top_p=0.95, top_k=20.

Measured behaviour

Benchmarked against the unmodified base model on identical settings. The delta is the figure that matters: it isolates what the weight edit cost.

Task	Base	Uncensored	Δ
MMLU	83.4	83.3	-0.2
ARC-Challenge	58.9	57.7	-1.2
HellaSwag	82.8	82.9	+0.1
Winogrande	76.1	75.3	-0.8
Mean			-0.5

0-shot via lm-evaluation-harness, bf16, both models scored in the same session. Every delta is within or close to the reported standard error (MMLU ±0.30, ARC ±1.44, HellaSwag ±0.38, Winogrande ±1.21), so none is clearly separable from run-to-run noise.

These are 0-shot and are not comparable to Qwen's published scores, which use few-shot prompting. They are directly comparable to each other, which is the point. Note also that ARC-Challenge is low for a model at this MMLU — the base scores 58.9 under the same settings, so that is format sensitivity in a reasoning-tuned model, not abliteration damage.

What the benchmarks do not cover: no generative evaluation (GSM8K, HumanEval), no math or code, no multilingual, and the harness loads the text stack only — nothing here measures the vision tower or MTP speculative decoding.

Measurement	Base model	This model
Refusals (100 held-out harmful prompts)	98/100	12/100
KL divergence vs base (first-token)	0	0.1191

Search: 200 Heretic trials, 23 non-dominated points. The published model is the marked row.

refusals	KL divergence
12/100	0.1191	← published
13/100	0.1052
19/100	0.0722
23/100	0.0635
26/100	0.0507
27/100	0.0410
35/100	0.0406
36/100	0.0387
41/100	0.0366
44/100	0.0352
46/100	0.0334
48/100	0.0331
51/100	0.0321
52/100	0.0294
60/100	0.0290
76/100	0.0280
77/100	0.0247
83/100	0.0204
86/100	0.0193
91/100	0.0170
96/100	0.0146
97/100	0.0044
98/100	0.0004

How to read these

Refusal rate is the count of refusals over 100 held-out prompts from mlabonne/harmful_behaviors (test split) — explicitly harmful requests, not benign ones. So this number is not an over-refusal rate: it does not tell you how often the model declines legitimate work. It tells you how much of the original safety behaviour on harmful requests remains.

KL divergence is measured against the unmodified base model over first-token distributions, and is the optimizer's proxy for "how much did we damage the model". Lower is closer to base. It is a proxy, not a capability measurement — a low KL does not certify that reasoning or coding ability survived, and nothing here does certify that.

The two trade off against each other. The published point minimizes refusals, which is the most aggressive end of the front and also the highest divergence on it. Points with noticeably lower KL are available at a few more refusals; the full search is archived in heretic-study/ if you want to re-export one.

Caveats that matter

Refusals were measured in non-thinking mode. This model's chat template opens a <think> block, so the evaluation closes it explicitly to score answers rather than reasoning traces. With thinking enabled the refusal rate may differ, in either direction.
The measurement is 100 prompts from one dataset. It generalizes to that distribution of harmful requests and no further. Refusal behaviour on other topics is uncharacterized.
Perplexity was measured on the GGUF conversions, not here — wikitext-2, f16 6.9358 vs Q4_K_M 7.1557. It detects gross quantization damage, not capability loss.

Limitations

Refusals are reduced, not eliminated, and not redirected. This model attempts many requests the original declines, but a meaningful fraction are still refused.
This is the highest-divergence point on the front. Behaviour near the old refusal boundary is less stable than the base model.
Capability benchmarks show a 0.5-point mean drop vs base. No generative, math, code, or multilingual evaluation was run. See Measured behaviour.

Intended use

Local inference and research. Not intended for deployment to third parties without your own safety layer.

License

Apache 2.0, inherited from Qwen/Qwen3.8-27B. The base model's license and acceptable use policy still apply to your use of this derivative.

Downloads last month: 248

Safetensors

Model size

27B params

Tensor type

BF16

Model tree for JonathanColetti/Qwen3.8-27B-Uncensored

Base model

Qwen/Qwen3.8-27B

Finetuned

(45)

this model

Quantizations

3 models