PINlab/pi05-so101

pi0.5 SO-101 with a tactile skin

pi0.5 policies for a single SO-101 arm with a 5-chip magnetic tactile skin, fine-tuned from pi05_base. One folder per training run, one sub-folder per checkpoint step. None has been evaluated on the robot yet.

What is here

folder	config	cameras	prompt(s)	data	steps
grab_ball_skin/	pi05_so101_grab_ball_skin	2: base = fixed (train: fixed/side at random), wrist = handeye	"Grab the ball"	endoard/grab_ball_3cam_skin, first 100 episodes / 46,076 frames	10000, 20000, 29999
grab_ball_skin_3cam/	pi05_so101_grab_ball_skin_3cam	3: fixed, handeye, side	"Grab the ball"	endoard/grab_ball_3cam_skin, 150 episodes / 65,989 frames	10000, 20000, 29999
grab_cup_skin_3cam/	pi05_so101_grab_cup_skin_3cam	3: fixed, handeye, side	"Grab the blue cup"	endoard/grab_cup_3cam_skin, 166 episodes / 64,812 frames	10000, 20000, 29999
grab_3obj_skin_3cam/	pi05_so101_grab_3obj_skin_3cam	3: fixed, handeye, side	"Grab the red cube" / "Grab the green cube" / "Grab the green cylinder"	endoard/grab_{red_cube,green_cube,green_cylinder}_3cam_skin merged, 450 episodes / 161,918 frames, one task per source	10000, 20000, 28000
grab_cubes_skin_3cam/	pi05_so101_grab_cubes_skin_3cam	3: fixed, handeye, side	"Grab the red cube" / "Grab the green cube" (see Prompts)	endoard/grab_{red_cube,green_cube,red_cube_distractor_green_cube,green_cube_distractor_red_cube}_3cam_skin merged, 550 episodes / 191,324 frames	10000 (20000, 29999 in training)
lift_leave_cubes_skin_3cam/	pi05_so101_lift_leave_cubes_skin_3cam	3: fixed, handeye, side	"Lift the red cube" / "Lift the green cube" / "Lift the red cube, leave the green cube on the table" / "Lift the green cube, leave the red cube on the table" (see Prompts)	same 4 cube datasets merged, 550 episodes / 191,324 frames	10000, 20000 (29999 in training)

grab_3obj_skin_3cam is multi-task: the prompt selects the object. Its run was stopped at step 28,500 of 30k, so 28000 is its last checkpoint. Its held-out split is stratified per task (15 episodes each).

grab_cubes_skin_3cam and lift_leave_cubes_skin_3cam are trained on the same 550 cube episodes with different prompts; their held-out split is stratified per source (15/15/15/10 episodes).

Prompts

Every policy here was trained on a fixed set of text prompts. At inference send one of them verbatim as the observation prompt (same wording, case and punctuation); other phrasings are untested.

folder	training prompts (exact strings)
grab_ball_skin/, grab_ball_skin_3cam/	Grab the ball
grab_cup_skin_3cam/	Grab the blue cup
grab_3obj_skin_3cam/	Grab the red cube, Grab the green cube, Grab the green cylinder
grab_cubes_skin_3cam/	Grab the red cube, Grab the green cube
lift_leave_cubes_skin_3cam/	Lift the red cube, Lift the green cube, Lift the red cube, leave the green cube on the table, Lift the green cube, leave the red cube on the table

Which prompt each source dataset was trained with in the two cube runs:

source dataset (episodes)	scene	grab_cubes_skin_3cam	lift_leave_cubes_skin_3cam
endoard/grab_red_cube_3cam_skin (150)	red cube alone	Grab the red cube	Lift the red cube
endoard/grab_green_cube_3cam_skin (150)	green cube alone	Grab the green cube	Lift the green cube
endoard/grab_red_cube_distractor_green_cube_3cam_skin (150)	red cube + green cube	Grab the red cube	Lift the red cube, leave the green cube on the table
endoard/grab_green_cube_distractor_red_cube_3cam_skin (100)	green cube + red cube	Grab the green cube	Lift the green cube, leave the red cube on the table

So grab_cubes_skin_3cam uses one prompt per target colour whether or not the other cube is on the table, and lift_leave_cubes_skin_3cam also names the distractor when there is one.

Each step folder is an openpi checkpoint without optimizer state: params/ (EMA weights), assets/<asset_id>/norm_stats.json (quantile stats), _CHECKPOINT_METADATA, and a README.md with that checkpoint's details.

Download one checkpoint

from huggingface_hub import snapshot_download
d = snapshot_download("PINlab/pi05-so101", allow_patterns="grab_cup_skin_3cam/29999/*")
# serve with --policy.dir={d}/grab_cup_skin_3cam/29999

Serving

The checkpoints carry no config. Serve each one under the config in the table, which pins its asset_id:

python scripts/serve_policy.py policy:checkpoint \
    --policy.config=pi05_so101_grab_cup_skin_3cam \
    --policy.dir=<download>/grab_cup_skin_3cam/29999

The policy returns 16 actions spaced 2 frames apart (15 Hz), absolute joints (the delta is undone server-side). Replay at 15 Hz or interpolate; replaying at 30 Hz halves the speed. The 3-camera policies need all three cameras every step.

Recipe (all runs)

• State 21-D (padded to 32): 5 joints (deg), gripper (0-100), 5 skin chips x xyz (uT), raw.
• Action 6-D (padded to 32): delta on the 5 joints, absolute gripper (make_bool_mask(5, -1)).
• Images: base_0_rgb = fixed, left_wrist_0_rgb = handeye; right_wrist_0_rgb = side (mask on) for 3-camera runs, zeros with mask off for grab_ball_skin.
• action_horizon=16, action_stride=2, prompt from the dataset task string (remapped per source for the two cube runs, see Prompts).
• From pi05_base, 4 nodes x 1 A100-64GB, --fsdp-devices=4, batch 32, AdamW with cosine decay, EMA 0.99, seed 42, 30k steps, 10% of episodes held out.