vis22/act_robotwin_putbd

Model Card for act

Action Chunking with Transformers (ACT) is an imitation-learning method that predicts short action chunks instead of single steps. It learns from teleoperated data and often achieves high success rates.

This policy has been trained and pushed to the Hub using LeRobot.

Learn how to train and run it in the LeRobot act guide, or browse the full documentation.

---

Model Details

• License: apache-2.0
• Robot type: unified_robot
• Cameras: cam_high, cam_left_wrist, cam_right_wrist

Inputs & Outputs

The policy consumes these observation features and produces these action features.

Inputs

Feature	Type	Shape
observation.state	STATE	(14,)
observation.images.cam_high	VISUAL	(3, 240, 320)
observation.images.cam_left_wrist	VISUAL	(3, 240, 320)
observation.images.cam_right_wrist	VISUAL	(3, 240, 320)

Outputs

Feature	Type	Shape
action	ACTION	(14,)

Training Dataset

• Repository: vis22/robotwin_demo_clean_aloha_agilex_v30
• Episodes: 30
• Frames: 19087
• Frame rate: 15 FPS
• Task(s): "Pick the medium plastic bottle with tapered neck and move it into the black garbage bin, repeat for the bottle for holding liquids and the black and red bottle.", "Use arms to pick the yellow bottle, the medium-sized water bottle, the smooth plastic bottle, and place them one by one in the black bin with white liner.", "Move the yellow bottle into the plastic dustbin, then do the same for the bottle with narrow neck and the bottle with curved body shape.", "Grab the green-striped cap on yellow bottle, place it in the black garbage bin, then grab the hand-sized bottle and the smooth plastic bottle for the black garbage bin.", "Grab the green-striped cap on yellow bottle and the smooth transparent plastic bottle, put them into the black garbage bin, and repeat for the drink bottle with slim neck.", "Use arms to pick the yellow liquid container with green top, the bottle with narrow neck, the hand-sized drinking bottle, and place them one by one in the dustbin with white inner bag.", "Transfer the medium plastic bottle with tapered neck, the transparent bottle, and the red and black beverage bottle to the dustbin with lid and bag located to the table's left.", "Take the bottle made of smooth plastic, the medium-sized water bottle, and the red bottle and place them into the plastic dustbin on the left.", "Grab each of the bottle with green cap, the bottle with narrow neck, and the hand-sized drinking bottle and drop them into the smooth black dustbin.", "Use arms to pick the green-striped cap on yellow bottle, the bottle for holding liquids, the rounded bottle with a white neck, and place them one by one in the medium black garbage container.", "Pick the smooth yellow bottle with ridged cap, drop it into the black dustbin, then move the bottle for holding liquids and the black soda bottle with white top into the black dustbin.", "Using arms, grab the yellow bottle and the blue-accented cylindrical bottle, then place them into the black bin with white liner along with the black bottle with red label.", "Using arms, grab the yellow liquid container with green top and place it in the plastic dustbin, do the same for the hand-sized bottle and the rounded bottle with a white neck.", "Grab the yellow bottle body with tapered neck, the water bottle with textured body, and the red and black beverage bottle sequentially, placing each into the plastic dustbin.", "Transfer the green-striped cap on yellow bottle, the bottle with narrow neck, and the red bottle into the black dustbin using your arms, one by one.", "Carry the bottle with green cap first, drop it into the dustbin with lid and bag, repeat with the transparent bottle and the drink bottle with slim neck.", "Place the yellow liquid container with green top, the cylindrical bottle, and the hand-sized drinking bottle in the rectangular black trash holder using the arms.", "Use the arms to move the bottle made of smooth plastic into the medium black garbage container, then repeat for the bottle with ribbed design and the bottle with curved body shape.", "Place the bottle with green cap, the medium-sized water bottle, and the black and red bottle in the dustbin with white inner bag using the arms.", "Use arms to pick the cylindrical yellow bottle for holding liquids, the water bottle with textured body, the black bottle with red label, and place them one by one in the plastic dustbin.", "Grab the bottle made of smooth plastic, drop it in the medium-sized black plastic dustbin, do the same for the bottle with narrow neck and the black soda bottle with white top.", "Using arms, grab the bottle made of smooth plastic and put it in the black dustbin, repeat for the water bottle with textured body and the red and black beverage bottle.", "Use arms to move the bottle made of smooth plastic, the cylindrical bottle, and the rounded bottle with a white neck to the dustbin with white inner bag on the table's left.", "Place the cylindrical yellow bottle for holding liquids, the hand-sized bottle, and the rounded bottle with a white neck, one at a time, into the trash bin.", "Grab the medium plastic bottle with tapered neck, drop it in the trash bin, do the same for the water bottle with textured body and the drink bottle with slim neck.", "Using arms, grab the yellow bottle body with tapered neck and place it in the rectangular black trash holder, do the same for the bottle for holding liquids and the black and red bottle.", "Use arms to pick the yellow bottle with smooth texture, the smooth transparent plastic bottle, and the rounded bottle with a white neck, dropping them into the rectangular black trash holder.", "Move the medium plastic bottle with tapered neck to the black dustbin, then the transparent bottle, and finally the hand-sized drinking bottle.", "Carry the bottle made of smooth plastic first, drop it into the dustbin with white inner bag, repeat with the bottle with turquoise cap and the rounded bottle with a white neck.", "Use the arms to transfer the cylindrical yellow bottle for holding liquids, the hand-sized bottle, and the smooth plastic bottle into the black dustbin one by one."

Training Configuration

Setting	Value
Training steps	100000
Batch size	8
Optimizer	adamw
Learning rate	1e-05
Seed	1000
LeRobot version	0.6.1

---

How to Get Started with the Model

New to LeRobot? These guides cover the full workflow:

• Install LeRobot — set up the lerobot package.
• Hardware setup — assemble, wire, and calibrate your robot and cameras.
• Record data & train a policy — the end-to-end imitation-learning walkthrough.
• CLI cheat-sheet — quick reference for the lerobot-* commands.

The short version to run and train this policy:

Run the policy on your robot

lerobot-rollout \
  --strategy.type=base \
  --robot.type=unified_robot \
  --robot.port=<your_robot_port> \
  --robot.cameras="{ <camera_1>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}, <camera_2>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}}" \
  --policy.path=vis22/act_robotwin_putbd \
  --task="Pick the medium plastic bottle with tapered neck and move it into the black garbage bin, repeat for the bottle for holding liquids and the black and red bottle." \
  --duration=60

Replace the remaining <...> placeholders with your own values: --robot.port and the camera names/indices are specific to your machine, and the camera names must match the observation keys this policy was trained on.

When --strategy.type=base is used the script doesn't record the episodes. Skipping duration will make the policy run indefinitely. For more information look at rollout documentation.

Train your own policy

lerobot-train \
  --dataset.repo_id=${HF_USER}/<dataset> \
  --policy.type=act \
  --output_dir=outputs/train/<policy_repo_id> \
  --job_name=lerobot_training \
  --policy.device=cuda \
  --policy.repo_id=${HF_USER}/<policy_repo_id> \
  --wandb.enable=true

Writes checkpoints to outputs/train/<policy_repo_id>/checkpoints/.

---

Evaluation

No evaluation results have been provided for this policy yet.

---

Citation

If you use this policy, please cite the method linked in the description above, along with LeRobot:

@misc{cadene2024lerobot,
    author = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Palma, Steven and Kooijmans, Pepijn and Aractingi, Michel and Shukor, Mustafa and Aubakirova, Dana and Russi, Martino and Capuano, Francesco and Pascal, Caroline and Choghari, Jade and Moss, Jess and Wolf, Thomas},
    title = {LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},
    howpublished = "\url{https://github.com/huggingface/lerobot}",
    year = {2024}
}