willamazon1/Qwen3.5-9B-smith-r1v1-reportonly

Qwen3.5-9B Smith AENV round 1 (v1, report-only) — RL checkpoint series

Reinforcement-learning checkpoint series from the cpo_smith ... smith-round1-v1-reportonly-localds run: a report-only variant of the Smith agentic-environment task, trained against a local dataset.

The policy was warm-started from the smith-v5-gdpo-exact run, which also served as the reference model.

Checkpoints

73 checkpoints, saved every 2 iterations, from iter_0000001 to iter_0000145. Each lives in its own subfolder of this repo so you can compare points along the training curve:

iter_0000001, iter_0000003, iter_0000005, iter_0000007, iter_0000009, iter_0000011, iter_0000013, iter_0000015, iter_0000017, iter_0000019, iter_0000021, iter_0000023, iter_0000025, iter_0000027, iter_0000029, iter_0000031, iter_0000033, iter_0000035, iter_0000037, iter_0000039, iter_0000041, iter_0000043, iter_0000045, iter_0000047, iter_0000049, iter_0000051, iter_0000053, iter_0000055, iter_0000057, iter_0000059, iter_0000061, iter_0000063, iter_0000065, iter_0000067, iter_0000069, iter_0000071, iter_0000073, iter_0000075, iter_0000077, iter_0000079, iter_0000081, iter_0000083, iter_0000085, iter_0000087, iter_0000089, iter_0000091, iter_0000093, iter_0000095, iter_0000097, iter_0000099, iter_0000101, iter_0000103, iter_0000105, iter_0000107, iter_0000109, iter_0000111, iter_0000113, iter_0000115, iter_0000117, iter_0000119, iter_0000121, iter_0000123, iter_0000125, iter_0000127, iter_0000129, iter_0000131, iter_0000133, iter_0000135, iter_0000137, iter_0000139, iter_0000141, iter_0000143, iter_0000145

Architecture	Qwen3.5 (Qwen3_5ForConditionalGeneration, hybrid linear/full attention + vision tower), 32 text layers, hidden 4096, 16 heads / 4 KV groups, vocab 248320
Precision	bfloat16
RL algorithm	GSPO (advantage_estimator=gspo), no KL penalty (kl_coef=0.0)
Learning rate	1.5e-6 (constant, min_lr=0)
Clip range	eps_clip=3e-3, eps_clip_high=4e-3
Rollouts	batch 16 prompts x 8 samples, global batch 64, temperature 1.0
Max response length	4096 tokens (sequence length 65536)
Parallelism during training	TP 2, PP 1, CP 8

Usage

Each training iteration is a subfolder of this repo, so pass subfolder= when loading:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "willamazon1/Qwen3.5-9B-smith-r1v1-reportonly"
ckpt = "iter_0000145"   # any of the iterations listed below

tok = AutoTokenizer.from_pretrained(repo, subfolder=ckpt)
model = AutoModelForCausalLM.from_pretrained(
    repo, subfolder=ckpt, dtype=torch.bfloat16, device_map="auto"
)

msgs = [{"role": "user", "content": "What is 12*8?"}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
ids = tok(text, return_tensors="pt").input_ids.to(model.device)
out = model.generate(ids, max_new_tokens=256)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

To pull a single checkpoint without downloading the whole repo:

hf download willamazon1/Qwen3.5-9B-smith-r1v1-reportonly --include "iter_0000145/*" --local-dir ./Qwen3.5-9B-smith-r1v1-reportonly

Conversion

Each subfolder was converted from a Megatron-LM torch_dist training checkpoint to HuggingFace safetensors using slime's tools/convert_torch_dist_to_hf.py, with the embedding padding stripped back to the tokenizer's vocab_size so tensor shapes match the upstream base model exactly. Weights are bfloat16; optimizer state is not included.

Every checkpoint was checked for NaN/Inf and for agreement between model.safetensors.index.json and the tensors actually on disk before upload.