Instructions to use drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2")
model = AutoModelForCausalLM.from_pretrained("drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2

SGLang

How to use drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2 with Docker Model Runner:
```
docker model run hf.co/drlee1/RefusalLoc-Qwen3-1.7B-Instruct-v2-DPO-seed2
```

RefusalLoc-Qwen3-1.7B-v2-DPO-s2

Source code and reproducibility materials: https://github.com/DONGRYEOLLEE1/refusal-loc

This is the balanced V2 DPO research checkpoint from RefusalLoc, initialized from Qwen3-1.7B-Instruct and trained with SFT followed by DPO.

Research use only. This checkpoint is not presented as a production safety system: it retains substantial benign false-refusal and capability loss.

Intended use

Research on refusal alignment, safety–helpfulness trade-offs, and mechanistic interpretability.
Reproduction of the associated behavioral evaluation and direction-ablation study.

Not intended for

High-stakes, safety-critical, or production deployment.
A replacement for application-specific safety policy and human oversight.

Evaluation

Metric	Score
Harmful refusal ↑	89.0
Harmful compliance ↓	3.5
Benign false refusal ↓	49.3
Benign helpful completion ↑	55.6
IFEval ↑	48.4
GSM8K ↑	56.8
MMLU ↑	50.0

Harmful compliance was scored with cais/HarmBench-Llama-2-13b-cls at revision bda705349d1144fa618770bea64d99ce54e3835b.

Evaluation protocol

Inference: non-thinking, greedy decoding, 512 maximum new tokens.
Safety prompts: AdvBench 520, HarmBench 400, StrongREJECT 313, JBB 100.
Benign prompts: XSTest 450 and fixed OR-Bench subset 400.
Capability: IFEval 541, fixed GSM8K subset 250, fixed MMLU subset 114.
Harmful compliance: HarmBench 13B classifier at the revision above.

This checkpoint is one member of a three-seed family. Seed 2 was selected as the balanced research-release candidate because it has the lowest V2 false-refusal rate, near-best helpful completion, and competitive capability among the DPO endpoints.

Training summary

SFT: 40,000 examples — 24,000 general, 8,000 harmful-refusal, 8,000 benign-helpfulness.
DPO: 3,997 valid pairs — harmful safety preference, benign compliance, and general helpfulness.
LoRA: rank 32, alpha 64; SFT 2,500 steps; DPO 500 steps; seed 2.
Base and reference: the seed-matched merged SFT endpoint.

License and data notice

This release is for research use only. The training recipe uses a CC-BY-NC-4.0 source recorded in the accompanying provenance manifest. Review all base-model and source-data obligations before enabling commercial use.

Upload checklist

Choose the final repository license after reviewing source-data obligations.
Preserve all files in this directory, including .gitattributes.
Add the exact commit hash of the release code before publishing.
Do not upload raw training data, benchmark prompts, or stored generations.

Downloads last month: 321

Safetensors

Model size

2B params

Tensor type

BF16