SPIN Qwen3-4B (UltraChat50k)

Self-Play Fine-Tuning (Chen et al., ICLR 2024) on Qwen3-4B, after SFT on UltraChat200k, then self-play on UltraChat50k.

Init / frozen reference for ite0 = UltraChat200k SFT (model_hub/Qwen3-4B-ultrachat200k). Training: 2 epochs / iter, max_length=1024, max_prompt_length=512, batch 2, grad accum 2, RMSprop, DPO β=0.1. Peak LR 5e-7 on ite0–ite1; decay 1e-7 on ite2–ite3. Full run is ite0..ite3.

Path	Stage
`ite0/`	Finished SPIN iteration 0 (loadable HF CausalLM)
`ite1/LATEST/`	In-progress ite1 snapshot at `example_counter=116000` / 199132 (~58%). Weights only (no optimizer).
`ite0/generated/train.jsonl`	ite0 synthetic completions used as rejected
`ite1/generated/train.jsonl`	ite1 synthetic completions used as rejected

ite1/ root has no model.safetensors.index.json on purpose: a finished iter is stored at the iteration root; a mid-iter snapshot lives in LATEST/.

Load a finished iter:

from transformers import AutoModelForCausalLM
m = AutoModelForCausalLM.from_pretrained("HoangTran223/SPIN_Qwen3-4B", subfolder="ite0")

Mid-iter ite1:

m = AutoModelForCausalLM.from_pretrained("HoangTran223/SPIN_Qwen3-4B", subfolder="ite1/LATEST")

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for HoangTran223/SPIN_Qwen3-4B

Base model

Qwen/Qwen3-4B-Base

Finetuned

Qwen/Qwen3-4B

Finetuned

(1107)

this model