willamazon1/Qwen3.5-9B-smith-r2v1-iter145

Qwen3.5-9B Smith AENV round 2 (v1, from round-1 iter145) — RL checkpoint series

Reinforcement-learning checkpoint series from the cpo_smith ... smith-round2-v1-iter145 run: the second Smith agentic-environment RL round, warm-started from iteration 145 of the round-1 report-only run.

Round 1 (published as willamazon1/Qwen3.5-9B-smith-r1v1-reportonly) also served as the reference model for this round.

Note. This run's training produced checkpoints through iter_0000079; the 32 listed here (up to iter_0000063) are the ones published so far. The remaining ones will be added later.

Checkpoints

32 checkpoints, saved every 2 iterations, from iter_0000001 to iter_0000063. Each lives in its own subfolder of this repo so you can compare points along the training curve:

iter_0000001, iter_0000003, iter_0000005, iter_0000007, iter_0000009, iter_0000011, iter_0000013, iter_0000015, iter_0000017, iter_0000019, iter_0000021, iter_0000023, iter_0000025, iter_0000027, iter_0000029, iter_0000031, iter_0000033, iter_0000035, iter_0000037, iter_0000039, iter_0000041, iter_0000043, iter_0000045, iter_0000047, iter_0000049, iter_0000051, iter_0000053, iter_0000055, iter_0000057, iter_0000059, iter_0000061, iter_0000063

Architecture	Qwen3.5 (Qwen3_5ForConditionalGeneration, hybrid linear/full attention + vision tower), 32 text layers, hidden 4096, 16 heads / 4 KV groups, vocab 248320
Precision	bfloat16
RL algorithm	GSPO (advantage_estimator=gspo), no KL penalty (kl_coef=0.0)
Learning rate	1.5e-6 (constant, min_lr=0)
Clip range	eps_clip=3e-3, eps_clip_high=4e-3
Rollouts	batch 16 prompts x 8 samples, global batch 64, temperature 1.0
Max response length	4096 tokens (sequence length 65536)
Parallelism during training	TP 2, PP 1, CP 8

Usage

Each training iteration is a subfolder of this repo, so pass subfolder= when loading:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "willamazon1/Qwen3.5-9B-smith-r2v1-iter145"
ckpt = "iter_0000063"   # any of the iterations listed below

tok = AutoTokenizer.from_pretrained(repo, subfolder=ckpt)
model = AutoModelForCausalLM.from_pretrained(
    repo, subfolder=ckpt, dtype=torch.bfloat16, device_map="auto"
)

msgs = [{"role": "user", "content": "What is 12*8?"}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
ids = tok(text, return_tensors="pt").input_ids.to(model.device)
out = model.generate(ids, max_new_tokens=256)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

To pull a single checkpoint without downloading the whole repo:

hf download willamazon1/Qwen3.5-9B-smith-r2v1-iter145 --include "iter_0000063/*" --local-dir ./Qwen3.5-9B-smith-r2v1-iter145

Conversion

Each subfolder was converted from a Megatron-LM torch_dist training checkpoint to HuggingFace safetensors using slime's tools/convert_torch_dist_to_hf.py, with the embedding padding stripped back to the tokenizer's vocab_size so tensor shapes match the upstream base model exactly. Weights are bfloat16; optimizer state is not included.

Every checkpoint was checked for NaN/Inf and for agreement between model.safetensors.index.json and the tensors actually on disk before upload.