π0.5 · RoboDojo cover_blocks · atomic-skill policy
π0.5 fine-tuned (full fine-tuning, JAX / openpi) on the atomic-skill (sub-task) split of the cover_blocks
augmentation data in N0ct1s/RoboDojo-Aug.
It is the low-level policy of an agent + π0.5 system: an agent decomposes the RoboDojo cover_blocks task, keeps the
memory (which colour is under which cup) and commands one atomic skill at a time with a sub-task instruction and a
target mask drawn on the camera images. It is not trained on the official whole-task instruction.
Not an official RoboDojo or Physical Intelligence model.
Checkpoints
| dir | step | notes |
|---|---|---|
10000/ |
10,000 | mid-training |
19999/ |
19,999 | final (LR fully annealed) |
Each holds params/ (Orbax, the EMA weights, ~12 GB) and assets/arx_x5_sim/norm_stats.json (the normalisation
statistics used in training). Optimizer state is not included. MANIFEST.sha256 lists every file's SHA-256.
Training
| item | value |
|---|---|
| base | gs://openpi-assets/checkpoints/pi05_base/params |
| openpi config | pi05_base_aloha_full_sim_arx-x5_seed_0 (XPolicyLab commit 10ab2651), with our data and mask transform |
| data | cover_blocks subtask split: 14,122 atomic-skill episodes, 1,236,808 frames, 15 sub-task instructions; generated from the released RoboDojo demonstrations (standard + random appearance, ~10 % corner cases), no evaluation layouts |
| inputs | 3 cameras (cam_high, cam_left_wrist, cam_right_wrist) at 224×224, 14-D joint state, sub-task instruction |
| target mask | exact simulator mask, perturbed like Grounded-SAM-2 output (incl. false alarms) and drawn on the images at load time — see train_code/mask_overlay.py (MaskAugParams defaults) |
| actions | 14-D ARX X5 joints (delta joints, absolute grippers), horizon 50 |
| steps / batch | 20,000 / 256 (8× A100-40GB, FSDP 8, ~20 h) |
| LR | warmup 1,000 → 2.5e-5, cosine to 2.5e-6 at step 20,000; AdamW, grad-clip 1.0, EMA 0.99 |
| norm stats | computed on 128,000 random training frames |
| train loss | 0.187 (step 0) → 0.0006 (10,000) → ~0.0002 (19,990) |
train_code/ has the training entry point (run.py, train.sh), the data-loading changes (lazy_masks.py: decode
the target mask of the sampled frame only) and the data config / mask overlay (configs.py, mask_overlay.py).
Use
Serve with openpi's Pi_05 policy from XPolicyLab (config pi05_base_aloha_full_sim_arx-x5_seed_0), pointing the
checkpoint at <step>/ and the norm-stats asset id arx_x5_sim at <step>/assets. At inference the agent supplies
the sub-task instruction and draws the target mask on the images the same way as mask_overlay.overlay().
License
The weights are a derivative of PaliGemma / π0.5 base and are distributed under the Gemma Terms of Use
(LICENSE_GEMMA.txt), including its use restrictions. Training data: Apache-2.0 (RoboDojo demonstrations and
N0ct1s/RoboDojo-Aug).