Instructions to use rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2")
model = AutoModelForMultimodalLM.from_pretrained("rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2

SGLang

How to use rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2 with Docker Model Runner:
```
docker model run hf.co/rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2
```

qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2

S1: thinking OFF, epoch 2 of 3. Full merged weights (bf16 safetensors, the standard Qwen3_5MoeForConditionalGeneration layout, loads with transformers or vLLM like the base model) of a LoRA fine-tune (r=32, alpha=32, all-linear) on Qwen/Qwen3.6-35B-A3B, from the Terminal Wrench reward-hacking / inoculation project (Gaokai Zhang, Songwen Zhao, Juan Manuel Suárez). This is the end-of-epoch-2 save.

Training data

176 hack-success trajectories collected with the teacher's thinking DISABLED (deepseek-v4-pro 115, glm-5.2 61), one row per (teacher, task), 127 distinct tasks. Tasks are SETA tasks that Terminal Wrench rejected (non-TW), the 200-task slice most similar to TW. Rows carry no CoT.

Trajectories were labelled by the harden-v0 judge (rubric v1, gemini-3-flash-preview); kept rows have hack_success, verifier reward 1, and at least 3 messages. Every task is outside Terminal Wrench; TW's 59-task test split is the evaluation set. Task bodies are SETA-derived (CC BY-SA 4.0), which is why these weights are released share-alike.

Recipe

Songwen's training/sft_tinker.py (github.com/songwen6968/reward-hacking), unmodified: LoRA rank 32, lr 0.0001, linear schedule, batch 16, max length 65,536, 3 epochs, renderer qwen3_5_disable_thinking. Tokens seen at this save: 6,970,845. The LoRA was merged into the base weights with tinker_cookbook.weights.build_hf_model (scale alpha/r = 1).

How to serve

Serve with thinking OFF (closed <think></think> block, i.e. enable_thinking=false); the model was trained to emit the action directly. Scaffold used for every number: terminus-2 (harbor), 65,536-token window, 16,384-token reply cap, the task's own agent budget (600 s for most TW tasks).

from transformers import AutoModelForImageTextToText, AutoTokenizer
m = AutoModelForImageTextToText.from_pretrained("rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2", dtype="bfloat16")
tok = AutoTokenizer.from_pretrained("rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2")

Results

Epoch 2, k=3 over the 59 held-out TW tasks, harden-v0 judge: no hacking instruction pass 84.2%, hack 0.6%; with the elicitation prompt pass 69.7%, legit 20.6%, hack success 49.1%, timeout 5.7%. Untrained floors under the same protocol: thinking-off base 89.8% / 0% and 96.6% / 11.9%; thinking-on base 88.1% / 0% and 94.7% / 15.8% (pass / hack, no instruction and elicitation). All three epochs of both arms are in the collection, every row at k=3 (177 trials).

Provenance

Run dir training_runs/exp3plus-nothink-hack_success-Qwen-Qwen3.6-35B-A3B-r32-0915-2152 in the project repo; train set sft_nothink_ds_glm.
Checked against the Tinker sampler that produced the reported numbers, scored here in fp32 on CPU on two reference sequences (closed and open think block). nothink sequence (232 tokens): mean |Δ logprob| 0.138 to its own Tinker sampler, against 0.180 for the untrained base through the same path; closest of the seven captures qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2; delta-over-base correlation 0.942; think sequence (280 tokens): mean |Δ logprob| 0.116 to its own Tinker sampler, against 0.158 for the untrained base through the same path; closest of the seven captures qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2; delta-over-base correlation 0.941. Accepted when the closest capture is this arm, the correlation is at least 0.85 and the gap is within 1.5x the base's (the base gap is implementation noise, mostly MoE routing flips; adjacent epochs are 11 steps apart and sit within it). See merge_check.json.

Downloads last month: 415

Safetensors

Model size

36B params

Tensor type

BF16

Model tree for rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep2

Base model

Qwen/Qwen3.6-35B-A3B

Finetuned

(301)

this model