Instructions to use huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated")
model = AutoModelForCausalLM.from_pretrained("huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated

SGLang

How to use huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated with Docker Model Runner:
```
docker model run hf.co/huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated
```

Please tell me the difference between 'abliterated' and 'uncensored' version

by linknana - opened Jan 31, 2025

Discussion

linknana

Jan 31, 2025

I don't know the difference between them

huihui-ai

Owner Feb 1, 2025

The purpose of ablation is to be uncensored?

linknana

Feb 1, 2025

This comment has been hidden

linknana

Feb 1, 2025

The purpose of ablation is to be uncensored?

Actually, I've come across instances where the same model exists in both 'abliterated' and 'uncensored' versions, even though they are authored by the same person. This naming convention confuses me, especially since there's no explanation provided in the model card.

darkc0de

Feb 1, 2025

•

edited Feb 1, 2025

Uncensored usually means fine tuned. Abliterated means refusal removed.

huihui-ai

Owner Feb 1, 2025

Direct ablation or fine-tuning to obtain uncensored models are both methods. Later, we will also release models that have had review removed through fine-tuning.

unk1911

Feb 15, 2025

there's a good article written about abliteration here: https://huggingface.co/blog/mlabonne/abliteration

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment