pi0.5 — relative actions, end-effector space (13-dim: xyz + 6D rotation + gripper + base)
Full fine-tune of lerobot/pi05_base on all ten
L5vel/*-eef-merged-v30 datasets, with actions expressed relative to the observation state at
the start of each chunk.
This is step 80K of a 100K-step run. Held-out loss here is 66% above the run's minimum, so these are not the best weights; they are published for step-matched comparison against the previous 30K round.
Held-out loss
5% of episodes per task are held out (77 of 1499 episodes, all 11 tasks).
| step | eval_loss |
|---|---|
| 5K | 0.0264 |
| 10K | 0.0245 min |
| 15K | 0.0249 |
| 20K | 0.0251 |
| 25K | 0.0259 |
| 30K | 0.0268 |
| 35K | 0.0274 |
| 40K | 0.0291 |
| 45K | 0.0304 |
| 50K | 0.0321 |
| 55K | 0.0343 |
| 60K | 0.0358 |
| 65K | 0.0377 |
| 70K | 0.0381 |
| 75K | 0.0400 |
| 80K | 0.0407 <- this repo |
The minimum is at 10K (0.0245). The curve rises after it: more steps make this model worse on held-out data, which is why the minimum rather than the endpoint is the checkpoint to prefer.
Action representation
use_relative_actions=true with relative_exclude_joints=["gripper"]. This is a processor
transform, not a data rewrite: RelativeActionsProcessorStep subtracts the anchor state at batch
time and AbsoluteActionsProcessorStep adds it back on the way out, so the policy emits absolute
actions and the conversion is identical in training, evaluation and inference.
action[t+k] -= observation.state[anchor] for all dims EXCEPT gripper
gripper absolute (it is a command, not a pose)
Action dim 13: eef_x, eef_y, eef_z, eef_xx, eef_xy, eef_xz, eef_yx, eef_yy, eef_yz, gripper, base_x, base_y, base_yaw — 12 of 13 dims relative, gripper absolute.
Normalisation is QUANTILES, computed from the relative offsets over chunks of 50
(meta/relative_action_provenance.json records this), not from the absolute targets. Absolute
statistics would put most targets far outside [-1, 1].
The end-effector rotation is the stored continuous 6D form (first two rotation-matrix
columns), not axis-angle. On this robot the gripper points down, so |rotvec| sits near pi where
the sign flips arbitrarily; a built axis-angle view carries frame-to-frame jumps of 2*pi.
Training data
11 language-conditioned tasks, 1499 episodes, 3,089,476 frames (17.2 h at 50 fps), one u850 arm on a mobile base, three cameras (left / right / wrist).
| task | episodes | frames |
|---|---|---|
| grab a drink from the fridge | 250 | 968,523 |
| pick up the bag on the ground and place it on the table | 200 | 477,058 |
| open the door and move inside | 200 | 506,473 |
| clean the table with the green towel | 199 | 534,608 |
| pick up the blue cup from the table | 102 | 75,870 |
| pick up the green bottle from the table | 100 | 69,748 |
| open the fridge door | 100 | 118,990 |
| place the blue cup on the table | 100 | 61,416 |
| pick up the grocery bag from the ground | 100 | 92,672 |
| move the croissant to the empty plate | 98 | 155,517 |
| place the green cup on the table | 50 | 28,601 |
The mixture is skewed: the four largest tasks are ~82% of frames. Sampling is uniform over frames, so it is not rebalanced.
Recipe
| init | lerobot/pi05_base |
| regime | full fine-tune (vision + language + action expert), 4.14B trainable |
| batch | 21/rank x 3 ranks = 63 effective |
| lr | 2.5e-5, cosine to 2.5e-6 over 100K, 1000 warmup; vision tower at 0.1x |
| chunk_size / n_action_steps | 50 / 10 |
| precision | bfloat16, gradient checkpointing on |
| seed | 1000 |
Loading
config.json here has had vision_encoder_lr_multiplier and state_in_prompt removed: both are
training-only fields from the fork this was trained in, and stock LeRobot rejects unknown config
keys. Inference is unaffected.
from lerobot.policies.pi05.modeling_pi05 import PI05Policy
policy = PI05Policy.from_pretrained("maskjp/pi05-relative-eef-all10-full-ft-80k")
meta/ ships the dataset metadata (~2 MB) for deployments that want it. Its info.json has
per-dimension names flattened to a plain list; the training datasets store them grouped as
{"motors": [...]}, which stock LeRobot resolves to the single key "motors" and which would
silently relativise the gripper.
Related
maskjp/pi05-relative-eef-all10-full-ft-10k— the same arm at 10K.maskjp/pi05-relative-eef-full-ft-30k— the previous round, 7 datasets / 949 episodes. Different training data; not a drop-in replacement.
- Downloads last month
- 21