ALFWorld-Qwen3-8B-GiGPO

A Qwen/Qwen3-8B policy trained with GRPO on ALFWorld text-based embodied household tasks, with the GiGPO estimator: episode-level group advantages plus discounted step-level group advantages (gamma 0.95, step weight 1.0, mode mean_std_norm).

The repository root holds the final policy, at training step 400.

Trained 400/400 steps over 117 hours on 4 GPUs, finishing 2026-08-13. Training-rollout success rate averaged 86.8% over the last 50 steps, against 76.0% for the outcome-GRPO arm (W&B 0f5r5p0u, Alfworld_Qwen3-8B_outcome) over the same window. Per task type at step 400: pick_and_place 1.00, pick_heat_then_place 1.00, pick_cool_then_place 1.00, pick_clean_then_place 0.92, pick_two_obj_and_place 0.83.

Two caveats on that comparison. The outcome arm ran with max_prompt_length=2048 against this run's 4096 (it predates commit d3fa8424); neither run ever clipped a prompt, but filter_overlong_prompts=True may have dropped different samples. And gamma differs (1.0 vs 0.95), which is inherent to the estimator rather than a confound. n=1 per arm. The outcome arm has no checkpoints or rollouts on this host — it was trained on huawei-job-001 and survives here only as W&B history.

Length caps, recorded so the prompt-cap difference is not re-derived later: neither arm ever clipped a prompt. Max prompt_length/max over all steps is 1641 tokens for GiGPO (cap 4096) and 1772 for outcome (cap 2048), and prompt_length/clip_ratio is 0.0000 at every step of both. The standalone_eval alfworld_qwen3 preset evaluates at a 2048 prompt cap, which is therefore non-binding for both arms and does not favour either. Response truncation against the shared 2048 response cap is confined to early training — mean response_length/clip_ratio is 1.6% (GiGPO) and 1.4% (outcome) over steps 1-50, peaking near 17%, then 0.1% or below for the rest of both runs.

That truncated early regime does not contaminate this pair's GiGPO advantage, because on these arms the advantage is not an early one. Training-rollout success, GiGPO minus outcome, by window: 1-25 +1.7, 26-50 +1.2, 51-75 +5.0, 76-100 +9.6, 101-150 +5.6, 151-200 +3.0, 201-300 +0.1, 301-350 +5.9, 351-399 +10.9 points. The gap is ~1 point through steps 1-50 where truncation peaks, only opens from step ~76 onward once clipping has fallen to ~0.02%, closes to nothing over 201-300, and reopens late. Any claim that GiGPO helps early would need the truncation caveat; the claim supported here is that it helps late.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("wckwan/ALFWorld-Qwen3-8B-GiGPO")
tokenizer = AutoTokenizer.from_pretrained("wckwan/ALFWorld-Qwen3-8B-GiGPO")

# An intermediate checkpoint
model_step = AutoModelForCausalLM.from_pretrained("wckwan/ALFWorld-Qwen3-8B-GiGPO", subfolder="step_20")

Checkpoints

• step_20/
• step_40/
• step_60/
• step_80/
• step_100/
• step_120/
• step_140/
• step_160/
• step_180/
• step_200/
• step_220/
• step_240/
• step_260/
• step_280/
• step_300/
• step_320/
• step_340/
• step_360/
• step_380/

Raw FSDP checkpoints

fsdp/global_step_<N>/ holds the unmerged verl FSDP checkpoint (sharded fp32 model state, optimizer state and extra state) for step(s) 400. These are for resuming training, not for inference — use the merged exports above to load a policy.