act-pick-v2-6000

Second pick policy (v2), trained on a filtered set of the recordings; the first one is act-pick-*. ACT policy for the SO-101 "pick a piece from the crate and hold it up to the camera" task, checkpoint at 6,000 steps (of a planned 30,000). Inputs: front camera, wrist camera and the six joint positions; output: a chunk of 90 future joint targets at 30 fps.

Training data

Real recordings from MikelRubi/so101-pick-cube_* (leader/follower teleoperation, 30 fps, 640x480), filtered:

• only the 30 fps recordings (the 10 fps ones are left out);
• only episodes that start with one or two pieces in the crate (counted by eye on the first front frame);
• episodes 5 to 15 s long, start gap between leader and follower of at most 20 degrees, at most 3 s idle at either end, gripper opens at least once, and the known bad takes (wrong camera, people in view) removed.

278 training episodes (73,526 frames) from 44 recording sessions (266 from 24 Oct, 12 from 23 Oct). Held out: 15 episodes, 12 from so101-pick-cube_20261024_181939 (one piece) and 3 from so101-pick-cube_20261024_144537 (two pieces).

Recipe: LeRobot train(), batch 8, chunk 90, 30 action steps per replan, learning rate 1e-5, LeRobot's default image augmentation (colour, sharpness, small affine), seed 1000.

Offline check on the held-out episodes

Mean absolute joint error over the predicted chunk (lower is better), against a "stay at the current joint positions" baseline of 21.10:

checkpoint	policy	below baseline
6,000 steps (this one)	15.36	27%
12,000 steps	15.57	26%
18,000 steps	15.57	26%

The three checkpoints are within noise of each other on this set. Per joint at 6,000 steps, policy / baseline: shoulder_pan 6.9 / 9.7, shoulder_lift 14.0 / 30.3, elbow_flex 18.1 / 21.4, wrist_flex 14.9 / 21.1, wrist_roll 31.0 / 29.3, gripper 7.2 / 14.9. wrist_roll is not better than staying still.

This is an offline error on 15 episodes with one seed; it says nothing about whether the arm grasps the piece. It has not been tested on the robot.