Instructions to use promotion/UW1-INPO-soft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use promotion/UW1-INPO-soft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="promotion/UW1-INPO-soft")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("promotion/UW1-INPO-soft")
model = AutoModelForCausalLM.from_pretrained("promotion/UW1-INPO-soft", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use promotion/UW1-INPO-soft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "promotion/UW1-INPO-soft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "promotion/UW1-INPO-soft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/promotion/UW1-INPO-soft

SGLang

How to use promotion/UW1-INPO-soft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "promotion/UW1-INPO-soft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "promotion/UW1-INPO-soft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "promotion/UW1-INPO-soft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "promotion/UW1-INPO-soft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use promotion/UW1-INPO-soft with Docker Model Runner:
```
docker model run hf.co/promotion/UW1-INPO-soft
```

UW1-INPO-soft

INPO's published squared loss integrated against the cached pair probability, eta = 0.005, tau/eta = 1/3

Fine-tuned from Qwen/Qwen2.5-7B-Instruct. One of five arms trained and evaluated under a single protocol so that the only difference between them is the loss.

Training


Panel	WildChecklists, four native checklist items per prompt
Prompts	319 train / 83 dev
Learner pairs	8,932 train (all 28 unordered pairs of 8 sampled responses)
Optimizer steps	140 at batch size 128, two epochs
Learning rate	3e-7 cosine, warmup 0.1, weight decay 1e-6, grad clip 1.0
Policy seed	42 (one seed)
Candidate pool	8 learner + 8 reference occurrences, temperature 1, top-p 1, 1024 tokens
Training judge	local open-weight judge, two judgments per pair and item, both orders

Measurements

Capability columns come from one local harness at the declared few-shot counts; the two preference columns are win rates against each benchmark's released baseline answers, judged by a local 72B open-weight model in both presentation orders.

Model	MMLU	ARC-C	HellaSwag	WinoGrande	TruthfulQA	GSM8K	IFEval	Arena-Hard	AlpacaEval	MT-Bench
base (untrained)	0.7427	0.6724	0.8141	0.7498	0.6480	0.7210	0.5675	0.7165	0.3075	8.3063
UW1-NBPO-PW	0.7431	0.6706	0.8148	0.7561	0.6471	0.7240	0.5730	0.7280	0.3149	8.3125
UW1-FixedRefNash-PW	0.7431	0.6715	0.8134	0.7553	0.6474	0.7293	0.5712	0.7315	0.3180	8.3000
UW1-DPO-soft	0.7430	0.6706	0.8129	0.7561	0.6475	0.7301	0.5656	0.7100	0.3146	8.2250
UW1-INPO-soft	0.7428	0.6715	0.8144	0.7514	0.6469	0.7271	0.5823	0.7435	0.3332	8.2025
UW1-SPPO	0.7427	0.6732	0.8148	0.7514	0.6439	0.7331	0.5693	0.7430	0.3668	8.3526
UW1-PROSPER	0.7427	0.6698	0.8135	0.7474	0.6485	0.7301	0.5545	0.7245	0.3053	8.3019

Three limits that apply to every row

The prompt set is a filtered subset. 2,000 prompts were requested; 750 were judged under a reduced budget, 660 had complete feedback, 344 passed the Nash certificate and 319 survived the representability filter. The surviving prompts are those with a strictly positive bargaining surplus, so the panel is not a random sample of the dataset.
One seed. Differences between these arms cannot be separated from policy-seed variance, and none of the differences measured so far exceeds a single standard error on any column.
Local judges. No hosted API was called at any point. Arena-Hard normally judges with gpt-4-1106, AlpacaEval 2.0 with weighted_alpaca_eval_gpt4_turbo and MT-Bench with GPT-4; these numbers are comparable across the five arms and are not leaderboard scores.

The five arms

Downloads last month: 621

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for promotion/UW1-INPO-soft

Base model

Qwen/Qwen2.5-7B

Finetuned

Qwen/Qwen2.5-7B-Instruct

Finetuned

(3100)

this model