Instructions to use richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code")
model = AutoModelForCausalLM.from_pretrained("richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code

SGLang

How to use richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code with Docker Model Runner:
```
docker model run hf.co/richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code
```

Qwen2.5-Coder-7B-Instruct-heretic1x-code

A refusal-ablated ("abliterated") version of Qwen/Qwen2.5-Coder-7B-Instruct, made with Heretic 1.x (heretic-llm 1.4.0, git tag v1.4.0, 6ea3b8d) as a rank-1 LoRA edit of o_proj and down_proj, merged into bf16 weights. It is one of the models evaluated in:

Richard J. Young. The Selection Rule Can Reverse Abliteration Comparisons: A Held-Out, Judge-Based Case Study of Heretic 1.x, Heretic 2.0, External Tools and Community Releases on Qwen2.5 (paper 30; arXiv link to be added).

It is released for research on abliteration and its evaluation, not as a general-purpose assistant.

What this model is

Variant: code direction: Heretic 1.x run with the code-abliteration prompt configuration (malicious vs benign coding requests).
Search: 200 Optuna trials (Heretic 1.x defaults).
Selected by Heretic 1.x's rule: the trial with the fewest dev refusals among trials with dev KL divergence <= 0.01 (the archived cap).
Selected trial: #102; dev refusals 26/100; dev KL divergence 0.0092 (dev set: configs/code_abliteration_data/malicious_code_eval.txt).

Held-out evaluation (paper 30)

972 held-out harmful prompts (none used during the search), scored by the WildGuard judge; capability on five benchmarks (percentage points, pp, relative to the original model).

Metric	This model	Original
Refusal rate, WildGuard judge [95% CI]	62.2% [59.2%, 65.2%]	83.2%
Refusal rate, keyword markers	62.3%	83.6%
Over-refusal on benign prompts (WildGuard)	7.1%	22.6%
Mean capability change	+0.24 pp	0

Capability change by benchmark (pp) - MMLU-Pro | GSM8K | IFEval | HumanEval | MBPP: +0.3 | -1.2 | +1.1 | +0.6 | +0.4

Caveats

Each number comes from one search on one base model; the paper finds that search-to-search variation is large (seed-noise SD about 9 pp), so differences between single models should not be over-read.
The selection cap matters: on Qwen2.5-32B, re-selecting the same search at dev KL <= 0.1 instead of 0.01 lowered residual refusal by 61.7 pp. Compare abliterated models only at a stated selection rule and realised divergence.
Refusal is reduced, not removed, and the model can still produce harmful content. It has no added safety tuning.

LoRA adapter

adapter/ holds the rank-1 PEFT LoRA adapter that Heretic 1.x exported for the selected trial. Merging it into the base model on CPU (PEFT merge_and_unload, adapter weights in fp32) reproduces these weights tensor for tensor.

Integrity

tensor_hashes.json lists the SHA-256 of every tensor (layout independent). The weights here are bit-identical, tensor by tensor, to the model evaluated in the paper. Code, run records and labels: GitHub ricyoung/abliteration-studies (released with the paper).

Licence

This model is a derivative of an Apache-2.0 base model and is released under the Apache-2.0 licence of its base model. In addition, it carries the use-based restrictions of the OpenRAIL++ Licence v1.0 (LICENSE_OpenRAIL++.md in this repository). Where the two conflict, the base licence prevails. It must not be deployed for, or used to generate content for, any of the prohibited uses listed in the OpenRAIL++ licence.

Downloads last month: -

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code

Base model

Qwen/Qwen2.5-7B

Finetuned

Qwen/Qwen2.5-Coder-7B

Finetuned

Qwen/Qwen2.5-Coder-7B-Instruct

Finetuned

(495)

this model