Instructions to use darrellbest/Qwen3.5-0.8B-Heretic with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use darrellbest/Qwen3.5-0.8B-Heretic with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="darrellbest/Qwen3.5-0.8B-Heretic")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("darrellbest/Qwen3.5-0.8B-Heretic")
model = AutoModelForMultimodalLM.from_pretrained("darrellbest/Qwen3.5-0.8B-Heretic", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use darrellbest/Qwen3.5-0.8B-Heretic with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "darrellbest/Qwen3.5-0.8B-Heretic"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "darrellbest/Qwen3.5-0.8B-Heretic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/darrellbest/Qwen3.5-0.8B-Heretic

SGLang

How to use darrellbest/Qwen3.5-0.8B-Heretic with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "darrellbest/Qwen3.5-0.8B-Heretic" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "darrellbest/Qwen3.5-0.8B-Heretic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "darrellbest/Qwen3.5-0.8B-Heretic" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "darrellbest/Qwen3.5-0.8B-Heretic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use darrellbest/Qwen3.5-0.8B-Heretic with Docker Model Runner:
```
docker model run hf.co/darrellbest/Qwen3.5-0.8B-Heretic
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwen3.5-0.8B Heretic (ARA)

Qwen/Qwen3.5-0.8B with its refusal behaviour removed by Heretic using Arbitrary-Rank Ablation (ARA), full-weight. bf16, same architecture and parameter count as the original, including the vision encoder, thinking control and the multi-token-prediction weights.

Results

	Refusals	KL divergence
Original Qwen3.5-0.8B	98/100	0 (by definition)
This model	15/100	0.0714

Measured by Heretic on the test[:100] splits of mlabonne/harmful_behaviors (refusals) and mlabonne/harmless_alpaca (KL divergence, the drift in ordinary behaviour), with Heretic's default system prompt and refusal markers. The numbers come from an independent re-evaluation of the final exported weights (evaluate_model).

Built on dalatexcoder's findings, and why it stops at 15/100

The search started from dalatexcoder's four published Qwen3.5-0.8B ARA parameter sets. The strongest of them gives 1/100 refusals with this toolchain (at KL 0.0993), but that model's reasoning broke down: in thinking mode it finished and correctly answered only 10 of 40 arithmetic and word problems, against 26 of 40 for the original. At 0.8B, every setting found that pushes refusals much lower costs reasoning like that. This release is the strongest setting that leaves reasoning intact (27 of 40), at 15/100 refusals. If you need fewer refusals and can accept weaker reasoning, dalatexcoder/Qwen3.5-0.8B-heretic-ara-high-kld is the stronger edit.

What this release adds is a complete checkpoint: save_pretrained drops Qwen3.5's 15 multi-token-prediction tensors and rounds the 36 float32 Gated DeltaNet parameters (linear_attn.A_log, linear_attn.norm.weight) down to bf16. Both were restored from the original here (ARA never touches either), so all 488 tensors match the original in name, shape and dtype. The MTP weights live in model-auxiliary.safetensors, listed in the index.

Parameters

ARA, full weight, on attn.o_proj and mlp.down_proj:

Parameter	Value
start_layer_index	11
end_layer_index	17
preserve_good_behavior_weight	0.8645
steer_bad_behavior_weight	0.001
overcorrect_relative_weight	0.202
neighbor_count	14

Calibration used 400 harmless and 400 harmful prompts (train[:400]).

Checked

Ordinary prompts (facts, a haiku, a two-sentence technical explanation): correct and fluent.
Thinking-mode reasoning, 4 arithmetic and word problems x 10 seeds with Qwen's recommended sampling (vLLM): 27/40 finished and answered correctly, against 26/40 for the original. At 0.8B, non-thinking arithmetic is unreliable in both this model and the original; turn thinking on for anything that needs reasoning.
Vision: given an image of a red circle and a blue square, it describes exactly that.

Tooling

A merge of upstream Heretic's master and its ara branch (ARA with master's scorers and thinking-model handling), Heretic upstream 3521f86 + ARA c91d690, transformers 5.17.0, torch 2.11.0+cu130, one RTX PRO 6000.

Use

Exactly like the original, with transformers, vLLM or SGLang. Thinking mode is on by default and can be turned off per request (enable_thinking=False).

Reduced safety guardrails by design. You are responsible for what you do with it.

The family

Repository	Format	Size	Use it with
Qwen3.5-0.8B-Heretic	bf16 safetensors	1.78 GB	transformers, vLLM, SGLang
Qwen3.5-0.8B-Heretic-GGUF	GGUF BF16 / Q8_0 / Q4_K_M + vision mmproj	1.56 / 0.83 / 0.54 GB + 0.20 GB	llama.cpp, Ollama
Qwen3.5-0.8B-Heretic-FP8	FP8 W8A8, compressed-tensors	1.47 GB	vLLM
Qwen3.5-0.8B-Heretic-NVFP4	NVFP4, compressed-tensors	1.33 GB	vLLM on Blackwell

Downloads last month: -

Safetensors

Model size

0.9B params

Tensor type

F32

BF16

Model tree for darrellbest/Qwen3.5-0.8B-Heretic

Base model

Qwen/Qwen3.5-0.8B-Base

Finetuned

Qwen/Qwen3.5-0.8B

Finetuned

(416)

this model

Quantizations

3 models