mot-prog/so101_pick_up_wrist_pan_act

Model Card for act

Action Chunking with Transformers (ACT) is an imitation-learning method that predicts short action chunks instead of single steps. It learns from teleoperated data and often achieves high success rates.

Learn how to train and run it in the LeRobot act guide, or browse the full documentation.

---

Model Details

• License: apache-2.0
• Robot type: so_follower
• Cameras: wrist, shoulder_pan

Run the policy on your robot

lerobot-rollout \
    --strategy.type=base \
    --policy.path=mot-prog/so101_pick_up_wrist_pan_act \
    --robot.type=so101_follower \
    --robot.port=<your_robot_port> \
    --robot.id=my_awesome_follower_arm \
    --robot.cameras="{ wrist: {type: intelrealsense, serial_number_or_name: 130322273579, width: 640, height: 480, fps: 30}, shoulder_pan: {type: intelrealsense, serial_number_or_name: 130322271364, width: 640, height: 480, fps: 30} }" \
    --task="pick and place a blue cube in a wooden box"

Train your own policy

lerobot-train \
  --dataset.repo_id=${HF_USER}/<dataset> \
  --policy.type=act \
  --output_dir=outputs/train/<policy_repo_id> \
  --job_name=lerobot_training \
  --policy.device=cuda \
  --policy.repo_id=${HF_USER}/<policy_repo_id> \
  --wandb.enable=true

Writes checkpoints to outputs/train/<policy_repo_id>/checkpoints/.

---

Evaluation

Task: Pick up a cube from a random position and place it into a box. Variable: Position of the target box.

Box Position	Trials	Successes	Success Rate	Observations
Right	10	6	60%	Performance degrades when the cube is placed at extreme distances (too close or too far).
Top Right	10	4	40%	Prone to failure when the cube is placed too close to the robot or immediately adjacent to the box.
Top Left	10	5	50%	Exhibits the same spatial limitations as the Top Right configuration.
Left	10	1	10%	High grasping success, but spatial mapping is skewed (often attempts to place the cube in the top-left instead of the left).
Middle (Center)	10	7	70%	Highest success rate. The restricted space reduces possible spatial configurations, leading to smoother and more accurate placement trajectories.

Additional Notes on Difficulty:

• Spatial mapping errors significantly impact placement accuracy when the box is placed on the left side of the workspace.
• Depth perception or reachability issues occur at the boundaries of the workspace (extreme proximity or distance).

---

Citation

If you use this policy, please cite the method linked in the description above, along with LeRobot:

@misc{cadene2024lerobot,
    author = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Palma, Steven and Kooijmans, Pepijn and Aractingi, Michel and Shukor, Mustafa and Aubakirova, Dana and Russi, Martino and Capuano, Francesco and Pascal, Caroline and Choghari, Jade and Moss, Jess and Wolf, Thomas},
    title = {LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},
    howpublished = "\url{https://github.com/huggingface/lerobot}",
    year = {2024}
}