ACT policy: SO-101 right arm picks up a watermelon plush
Imitation-learning policy (ACT, 52M parameters) trained with lerobot 0.6.1 on 30 teleoperated demos. The right SO-101 follower picks up a small watermelon plush that sits upright at one marked spot on the table, lifts it, and holds it.
Robot and observations
| Robot | SO-101 follower (so101_follower), lerobot calibration, joint positions in degrees (gripper 0-100) |
| Cameras | top: fixed overhead wide-angle, 640x480; wrist: on the follower's gripper, 640x480; both 30 fps |
| State / action | 6 joint positions (shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper) |
| Task text | "Pick up the watermelon plush" |
| Start pose | Arm folded in its rest pose, gripper slightly open; plush upright at the mark |
It only works in a scene very close to the original one (same camera placement, lighting and object position).
Training
| Data | 30 episodes, 6,718 frames at 30 fps (local dataset so101_watermelon_pick_clean) |
| Model | ACT: ResNet-18 (ImageNet init), 4 encoder / 1 decoder layers, width 512, 8 heads, dropout 0.1, VAE latent 32 |
| Actions | chunk of 100 steps, all 100 executed per model call, no temporal ensembling |
| Optimiser | AdamW, lr 1e-5 (backbone too), weight decay 1e-4, grad clip 10, no LR schedule |
| Schedule | 30,000 steps, batch 8 (about 36 epochs), seed 1000, no image augmentation |
| Hardware | Apple M-series GPU (MPS), about 7 h 40 min |
| Final loss | 0.077 total (L1 0.070 + 10 x KL 0.001), on normalised actions; no held-out split |
Use
pip install "lerobot[feetech]==0.6.1"
Load with ACTPolicy.from_pretrained("vimanyutaneja/so101-plush-pick"), or pass
--policy.path=vimanyutaneja/so101-plush-pick to lerobot tools. The saved config says device: mps;
on an NVIDIA GPU pass --policy.device=cuda.