willamazon1/Qwen3-8B-tmax-aenv-r2v2

Qwen3-8B TMax AENV round 2 (v2, iter109-weighted) — RL checkpoint series

Reinforcement-learning checkpoint series from the Tmax-v39-round2-v2-iter109-weighted run (_r2v2): the second RL round, warm-started from iteration 109 of round 1 with a reweighted objective.

Round 1 (Tmax-v39-round1-v8-success-loop-mask, published as willamazon1/Qwen3-8B-tmax-aenv-r1v8) was used as the reference model for this round.

Checkpoints

22 checkpoints, saved every 5 iterations, from iter_0000004 to iter_0000109. Each lives in its own subfolder of this repo so you can compare points along the training curve:

iter_0000004, iter_0000009, iter_0000014, iter_0000019, iter_0000024, iter_0000029, iter_0000034, iter_0000039, iter_0000044, iter_0000049, iter_0000054, iter_0000059, iter_0000064, iter_0000069, iter_0000074, iter_0000079, iter_0000084, iter_0000089, iter_0000094, iter_0000099, iter_0000104, iter_0000109

Architecture	Qwen3, 36 layers, hidden 4096, 32 heads / 8 KV groups, vocab 151936
Precision	bfloat16
RL algorithm	GSPO (advantage_estimator=gspo), no KL penalty (kl_coef=0.0)
Learning rate	2e-7 (constant, min_lr=0)
Clip range	eps_clip=3e-4, eps_clip_high=4e-4
Rollouts	batch 8 prompts x 16 samples, global batch 64, temperature 1.0
Max response length	8192 tokens (sequence length 40960)
Parallelism during training	TP 2, PP 1, CP 4

Usage

Each training iteration is a subfolder of this repo, so pass subfolder= when loading:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "willamazon1/Qwen3-8B-tmax-aenv-r2v2"
ckpt = "iter_0000109"   # any of the iterations listed below

tok = AutoTokenizer.from_pretrained(repo, subfolder=ckpt)
model = AutoModelForCausalLM.from_pretrained(
    repo, subfolder=ckpt, dtype=torch.bfloat16, device_map="auto"
)

msgs = [{"role": "user", "content": "What is 12*8?"}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
ids = tok(text, return_tensors="pt").input_ids.to(model.device)
out = model.generate(ids, max_new_tokens=256)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

To pull a single checkpoint without downloading the whole repo:

hf download willamazon1/Qwen3-8B-tmax-aenv-r2v2 --include "iter_0000109/*" --local-dir ./Qwen3-8B-tmax-aenv-r2v2

Conversion

Each subfolder was converted from a Megatron-LM torch_dist training checkpoint to HuggingFace safetensors using slime's tools/convert_torch_dist_to_hf.py, with the embedding padding stripped back to the tokenizer's vocab_size so tensor shapes match the upstream base model exactly. Weights are bfloat16; optimizer state is not included.

Every checkpoint was checked for NaN/Inf and for agreement between model.safetensors.index.json and the tensors actually on disk before upload.