robocurve/pi05-yam-molmoact2

π0.5: YAM bimanual fine-tune on MolmoAct2 data

JAX/Orbax fine-tune of openpi pi05_base (full fine-tune, nothing frozen; PaliGemma backbone → Gemma license terms) for the bimanual YAM platform, trained on the AllenAI MolmoAct2-BimanualYAM dataset (124 LeRobot repos, 5,145 episodes, 11.35M frames). Independent replication of a reference fine-tune on the same data; full recipe at robocurve/pi05-yam-replication.

Intended use & safety

• Intended use: research and evaluation on I2RT YAM bimanual arms (2× 6-dof arms + grippers, three cameras) for the trained task families: block manipulation, box packing, cable charging.
• Out of scope: any other embodiment or rig without fine-tuning (VLA policies do not zero-shot transfer across embodiments); unattended operation; operation near people without a hardware e-stop and enforced workspace/torque limits.
• Validation status: open-loop MSE on one held-out recording session (see Losses & evaluation). No closed-loop sim or real-robot success rates reported. Users are responsible for safe integration (guardrails, e-stop, workspace limits) before deployment.

Training

Data	124 train repos / 5,145 episodes / 11.35M frames; no filtering (full dataset release); held-out val: the separate repo allenai/19012026-block-13 (session-level holdout)
Embodiment	YAM bimanual: 14-dof state/action, 2× (6 joints + gripper), absolute joint positions; cameras top + left/right wrists, 360×640
Image preprocessing	openpi yam_pi05 config defaults (see training repo); per-repo loading with 5 ms decode tolerance
Schedule	20,000 steps @ batch 512, cosine LR (peak 2.5e-5, warmup 1k), EMA 0.99, 8× B200 (FSDP), ~17.5 h on Modal
Checkpoint selection	best held-out val over milestone checkpoints every 1,000 steps → step 10,000
Headline curve	open-loop val MSE at selection: 0.00206 (reference release on the same protocol: 0.00259 released / 0.00204 best)

Losses & evaluation

• Training loss: π0.5's flow-matching action objective (action expert predicts the velocity field for noised 16-step action chunks conditioned on VLM features + state). Full fine-tune (nothing frozen); EMA 0.99 weights are the published ones.
• Eval regime: open-loop MSE, predicted vs ground-truth actions over 16-step chunks on the held-out session repo, n=20 trajectories, quantile-normalized action space; protocol and script in the training repo (eval_mse.py). Normalization parity: quantile (q01/q99) stats vendored from the reference release and used identically at train and eval time.
• Comparison: the reference fine-tune's released checkpoint scores 0.00259 (best 0.00204) on this same protocol; see the training repo for the reference details.
• Scope: open-loop MSE on a single near-distribution session (an action-prediction proxy), not a closed-loop success rate or task-generalization benchmark.

Provenance

Trained by	aris @ Robocurve, 2026-07-02 → 04
Training code	https://github.com/robocurve/pi05-yam-replication (private): openpi patches, data prep, train/eval scripts, norm stats
Framework	openpi @ 15a9616a00943ada6c20a0f158e3adb39df2ccac (JAX/Orbax, FSDP)
Compute provider	Modal, 8× NVIDIA B200 (FSDP)
Wall-clock	~17.5 h; preemptions: unknown (auto-resume was configured)
Total compute	≈140 B200 GPU-hours (8 × 17.5 h); FLOPs not estimated
Cost	not estimated
Experiment tracking	wandb (project per training repo config); run not publicly linked
Card authorship	original card by aris (first-hand); metadata + safety/losses/provenance sections added by the team's publishing session, reconstructed from the training repo and the original card

Format

Orbax checkpoint (JAX):

• params/: Orbax PyTree of model parameters (EMA weights, ~12 GB)
• assets/yam-bimanual-merged/norm_stats.json: normalization stats used at train time

Usage

With openpi patched per the training repo (registers the yam_pi05 config):

from openpi.training import config as oc
from openpi.policies import policy_config

train_cfg = oc.get_config("yam_pi05")
policy = policy_config.create_trained_policy(train_cfg, "<local_dir>")
action_chunk = policy.infer({
    "images": {"top": ..., "left": ..., "right": ...},  # HWC uint8
    "state": ...,                                        # (14,) absolute joint positions
    "prompt": "stack the blocks",
})["actions"]                                            # (16, 14)

• Observations: cameras top → base view, left/right → wrist views (HWC uint8, 360×640 source); state: (14,) absolute joint positions, 2× (6 joints + gripper).
• Actions: (16, 14) chunks of absolute joint positions at 30 fps; normalization stats in assets/yam-bimanual-merged/norm_stats.json (applied by the policy wrapper).

Data provenance & caveats

• All source repos are public AllenAI LeRobot v3 datasets (Apache-2.0); the fine-tune inherits Gemma terms via the PaliGemma backbone (see frontmatter license).
• The videos carry small timestamp jitter and occasional frame deficits vs their metadata; training used a 5 ms decode tolerance and per-repo loading (no physical merge; see the training repo README for why merged videos drift).
• Val split is a single held-out recording session of the block task: a near-distribution check, not a task-generalization benchmark. No closed-loop or real-robot results.

Versioning & contact

• main is stable (single published checkpoint, step 10,000). Pin the revision hash for exact reproduction. Superseding checkpoints will set new_version here.
• Issues: HF Discussions on this repo.