litupup/SceneGen-GRPO100

SceneGen GRPO100

Final checkpoint after 100 effective GRPO optimizer updates, initialized from SFT-1316. This is the pure GRPO B experiment: no credit assignment modulation.

• Shared policy weights and Adam state across Designer, Planner and Critic.
• Role schedule: Designer, Designer, Designer, Planner, Critic (DDDPC); only the selected role receives the training loss each round.
• Group size 8; 100 distinct training scenes; learning rate 1e-6.
• Outcome reward: mean of seven 0-10 quality grades divided by 10, minus 0.8 times requested designer operation units / 512. Failed trajectories receive -1.
• Independent frozen 27B terminal quality judge; measured reachability.
• Training: six-rank FSDP2, sequence parallel size 2. GPU3 served the embedding pool.
• Continued from step50 with model and full Adam state, with the original task ordering and seeds extended to 100 scenes.

Files and use

The repository root contains the Hugging Face inference model, tokenizer, processor and chat template. Use the root for inference with the compatible SceneGen runtime.

training_state/global_step_100/ contains all six FSDP2 model shards, Adam shards, extra state (including RNG/scheduler state), and FSDP configuration. After downloading the complete repository on Linux, restore the checkpoint layout:

ln -s ../.. training_state/global_step_100/huggingface

Use training_state/global_step_100 as the resume checkpoint with the matching SceneGen/VERL runtime (world size 6, sequence parallel size 2).

training_metadata.json records the Git commit, protocol hash, schedule hash and optimizer step. checkpoint-layout.json maps the original checkpoint paths. checksums.json records SHA-256 hashes and byte sizes.

Evaluation scope

Training and final evaluation completed at step100. Evaluation uses nine legacy bedroom development scenes with one fixed seed each; this is not a broad independent benchmark, and this model card does not claim a statistically established improvement over SFT.

Source: https://github.com/linjunxiong/SceneGen Experiment: https://swanlab.cn/@lit/SceneGen-GRPO/runs/43df9bdaa3b758c76c165

The upload contains model and training state plus training configuration metadata. Raw trajectories, prompts and credentials are excluded.