Instructions to use PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft")
model = AutoModelForCausalLM.from_pretrained("PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft

SGLang

How to use PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio

How to use PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft",
    max_seq_length=2048,
)

Docker Model Runner
How to use PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft with Docker Model Runner:
```
docker model run hf.co/PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft
```

qwen2.5-coder-32b-instruct-heretic-sft

A coding-specialised 32B model built on Qwen2.5-Coder-32B-Instruct in two stages:

Heretic abliteration — directional refusal-removal (abliteration) to strip trained-in refusals and hedging that get in the way of engineering work.
SFT — supervised fine-tuning on coding data, trained with Unsloth.

The result is a low-refusal, instruction-following coder intended as the strongest laptop-runnable tier in a local/hybrid coding workflow. Weights are full-precision bfloat16 — quantise before running on a Mac (see Usage).

Note on availability: this repository is private/gated. You must be authenticated (huggingface-cli login) with access to download or convert it.

Model details


Base model	Qwen/Qwen2.5-Coder-32B-Instruct
Architecture	`Qwen2ForCausalLM` (dense)
Parameters	~32.8B
Precision	bfloat16 (~65 GB, 14 safetensors shards)
Layers / hidden / heads	64 / 5120 / 40 (8 KV heads, GQA)
Context length	32,768 tokens
Vocab	152,064
Chat format	ChatML (`<
Pipeline	Heretic abliteration → SFT (Unsloth)

Recommended sampling

Inherited from the base coder (see generation_config.json):

temperature      0.7
top_p            0.8
top_k            20
repetition_penalty 1.05

Intended use & responsible use

Intended for autonomous and assisted software engineering: code generation, editing, refactoring, test-writing, tool-calling, and the mechanical "execution" tier of a tiered cloud+local agent workflow, where outputs are gated by verify-before-merge (it only merges if it builds and passes tests).

This model is abliterated (uncensored). Refusal behaviour has been deliberately reduced, so it will not reliably decline unsafe or disallowed requests. You are responsible for the outputs and for adding your own safety/permission controls around it. Do not deploy it in a user-facing setting without an independent moderation and authorization layer. Use it lawfully.

Usage

transformers (bf16, server/GPU)

Needs ~65 GB VRAM in bf16 (multi-GPU or offload), or quantise with bitsandbytes.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")

messages = [{"role": "user", "content": "Write a Python function that reverses a linked list."}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=512, temperature=0.7, top_p=0.8, top_k=20, repetition_penalty=1.05)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

Run on a Mac via Osaurus (MLX)

Osaurus is a native Apple-Silicon (MLX) server with OpenAI- and Anthropic-compatible endpoints. It runs MLX models, so convert + quantise first. A 4-bit build is ~18 GB resident → 36 GB+ unified memory recommended.

# 1. Install the converter
pip install -U mlx-lm

# 2. Authenticate (repo is private) and convert to MLX 4-bit
huggingface-cli login
python -m mlx_lm convert \
  --hf-path PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft \
  -q --q-bits 4 \
  --mlx-path ./qwen2.5-coder-32b-heretic-sft-mlx-4bit

# 3. Quick sanity check outside Osaurus
python -m mlx_lm generate \
  --model ./qwen2.5-coder-32b-heretic-sft-mlx-4bit \
  --prompt "Write a Rust function that returns the nth Fibonacci number." \
  --max-tokens 256

Then point Osaurus at the converted folder (add it via osaurus ui → model manager, or drop it in Osaurus's models directory) and serve:

osaurus serve --port 1337
# OpenAI-compatible smoke test:
curl http://127.0.0.1:1337/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer local" \
  -d '{"model":"qwen2.5-coder-32b-heretic-sft-mlx-4bit",
       "messages":[{"role":"user","content":"Reply with the single word: pong"}]}'

Tip: MLX quant flavours other than 4-bit (e.g. --q-bits 8) trade RAM for quality. On <36 GB machines this dense 32B is painful — prefer a smaller coder for that tier.

Run on a Mac (or anywhere) via Ollama (GGUF)

Ollama runs GGUF, so convert with llama.cpp first. A Q4_K_M build is ~19 GB.

# 1. Get llama.cpp + deps
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
pip install -r requirements.txt

# 2. Download the model (private → login first) and convert to GGUF (f16)
huggingface-cli login
huggingface-cli download PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft \
  --local-dir ./heretic-sft
python convert_hf_to_gguf.py ./heretic-sft --outfile heretic-sft-f16.gguf --outtype f16

# 3. Quantise (Q4_K_M is a good default)
./llama-quantize heretic-sft-f16.gguf heretic-sft-Q4_K_M.gguf Q4_K_M

Create a Modelfile (ChatML template + recommended params):

FROM ./heretic-sft-Q4_K_M.gguf

PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER top_k 20
PARAMETER repeat_penalty 1.05
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>
"""

ollama create qwen2.5-coder-32b-heretic-sft -f Modelfile
ollama run qwen2.5-coder-32b-heretic-sft "Write a Go function that debounces calls."

Ollama also exposes an OpenAI-compatible endpoint at http://localhost:11434/v1, so any tooling that speaks OpenAI (including a router in front of Claude Code) can target it.

Limitations

Uncensored — see the responsible-use note above; add your own guardrails.
32B dense — needs quantisation and healthy RAM/VRAM to run; the 4-bit laptop build is slower than a small MoE coder.
Base-model limits carry over — 32K context, Qwen2.5-Coder knowledge cutoff, and its language/framework coverage.
No benchmark numbers are claimed here. Evaluate on your own tasks before relying on it.

Provenance

Base: Qwen/Qwen2.5-Coder-32B-Instruct (Apache-2.0)
Abliteration: Heretic (heretic-llm) directional refusal-removal
Fine-tuning: SFT via Unsloth (unsloth_version 2026.7.3)

Downloads last month: 10

Safetensors

Model size

33B params

Tensor type

BF16

Model tree for PeetPedro/qwen2.5-coder-32b-instruct-heretic-sft

Base model

Qwen/Qwen2.5-32B

Finetuned

Qwen/Qwen2.5-Coder-32B

Finetuned

Qwen/Qwen2.5-Coder-32B-Instruct

Finetuned

(137)

this model