abdulrahimmirani/pi05-piperx-pick-cube-ft01

pi0.5 fine-tune — PiPER-X pick_cube (real teleop), ft01

π0.5 fine-tuned from pi05_base on 49 real-hardware teleop demonstrations of pick up the cube and put it in the cup, recorded on a PiPER-X (6-DOF + gripper) via XR-controller teleop at 30 Hz.

These are orbax checkpoints, not HF safetensors. There is no from_pretrained path — download and point openpi at the directory.

Checkpoints

step	epochs	train loss
2000	1.7	0.0074
4000	3.5	0.0059
6000	5.2	0.0045
8000	6.9	0.0039
9999	8.7	0.0040

Each contains params/ (weights, 5.8 GB), train_state/ (optimizer state — only needed to resume training, 3.0 GB) and assets/ (normalization statistics — required for inference).

Loss flattens after ~6000, so start by comparing 6000 and 9999. If they score the same, the back half of the run added nothing.

Serving

Requires the rahim-trc branch of The-Robotics-Company/openpi at commit f8bf156 or later — the config and transforms are not on main.

uv run scripts/serve_policy.py policy:checkpoint \
  --policy.config=pi05_piperx_teleop_expert \
  --policy.dir=<downloaded>/9999

Gotcha: pi05_piperx_teleop_expert has absolute paths baked in (assets_base_dir=/home/ubuntu/training/assets, checkpoint_base_dir=...) that point at the box it was trained on. Norm stats ship inside each checkpoint's assets/, but if config resolution fails on your machine, override those two fields.

Action space — read this before evaluating

State and action are 7-dim: joint1..joint6 (rad) + gripper.

1. The gripper is normalized, and your robot wants metres. Training used gripper = clip(1 - aperture_m / 0.07), so 0.0 = fully open, 1.0 = fully closed. Convert model output back before sending it to the arm:

aperture_m = (1.0 - gripper_norm) * 0.07

Getting this backwards drives the gripper inverted and the policy will look completely broken.

2. Binarization threshold needs re-tuning. RoboLabOutputs leaves the gripper continuous; the execution client binarizes it. The existing ~0.22 with hysteresis value was tuned on cuRobo planner sim data, not human teleop. Expect to re-tune it here.

3. Delta actions are handled internally. The model predicts joint deltas (gripper stays absolute), and AbsoluteActions converts back, so the served interface is absolute joint targets in rad. You do not need to integrate anything yourself.

Cameras

dataset key	model slot
observation.images.external (static)	base_0_rgb
observation.images.wrist (on the arm)	left_wrist_0_rgb
—	right_wrist_0_rgb — zeroed and masked

Native 640×480 is letterboxed to 224×224 (168×224 of real content, black bars top and bottom). Aspect ratio is preserved, not stretched.

Prompt string: pick up the cube and put it in the cup

Training recipe

• Expert-only: SigLIP tower and Gemma-2B frozen; 430.1 M of 3.353 B params trainable (12.8%) — the action expert plus the action/time projections
• action_horizon=30 (1 s at 30 Hz), action_dim=32 (7 real dims, zero-padded)
• batch 16, 10,000 steps (8.7 epochs over 18,431 frames)
• AdamW, warmup 400 → peak 2.5e-5 → cosine → 2.5e-6 at 10k
• EMA disabled (its full shadow param copy OOM-killed a 30 GB host)
• bf16 frozen params, 1× L40S (46 GB), 4 h 19 m
• Norm stats computed fresh — PiPER-X is not in the π0.5 pretraining mixture

Caveats

• No validation split. All 49 episodes are training data, so the loss curve cannot distinguish learning from memorizing. Hardware eval is the only signal.
• Training loss is not evidence of success. A previous π0.5 attempt on this same task scored 0/24 with absolute action targets and had a similarly clean loss curve. Delta actions were the identified fix and are used here, but that fix is unvalidated on hardware as of this upload.
• Flow-matching loss has no absolute scale; 0.0040 is not meaningful on its own.

Data

piper_x_pick_cube_v1 — 49 episodes / 18,431 frames / 614 s, 30 Hz, LeRobot v2.1, wrist + external RGB. 49 clean takes selected from 55 recorded on 2026-09-09. Verified: parquet rows, episodes.jsonl lengths and decoded frame counts agree for every episode and both cameras.