Hugging Face's logo
Activity Feed

AI & ML interests

Working with LeRobot SO101 and creating datasets using it, as well as finetuning both SmolVLA and Pi0.5 models using these.

Recent Activity
batencle  updated a dataset 4 days ago
Project-IRA/TPSoSe2026_Dataset_Collection_LeRobot_SO101
batencle  updated a dataset 4 days ago
Project-IRA/TPSoSe2026_Dataset_Fetch_Ball_LeRobot_SO101
batencle  updated a dataset 4 days ago
Project-IRA/TPSoSe2026_Dataset_Dice_Throw_LeRobot_SO101
View all activity

Organization Card
Community About org cards

Project-IRA — Interactive Robotic Arm

Vision-Language-Action policies and multi-task manipulation datasets for the SO-101 low-cost robot arm, built on LeRobot.

Code: https://github.com/Project-IRA/interactive-robotic-arm — the repositories here and that repo depend on each other; the code repo contains the recording, validation, merging, training and inference pipeline that produced everything in this organization.

Authors: Cleo Baten, Bela Keppler, Jonas Sapper

Start here

If you just want a policy that works on an SO-101:

Project-IRA/TPSoSe2026_Pi05_LeRobot_SO101_Finetuning_V7_Full_V2 — checkpoint 008000

This is the strongest model in the org: Pi0.5 fine-tuned on all four tasks, with image augmentation. Note its unusual internal directory layout, documented on its card.

Demo

Pi0.5 sorting lego bricks onto colour-matched plates on the physical SO-101.

Tasks

Task What the arm does
Sort Lego Color Sort lego bricks onto colour-matched plates
Desk Cleanup Clear objects from the desk into a bowl
Dice Throw Pick up a dice cup and tip the dice onto the table ("true random")
Fetch Ball Pick up a ball and place it into a waiting human hand

Models

Quality ratings are qualitative operator assessments from rollouts on the physical arm. There are no formal success-rate numbers.

Model Base Trained on Best checkpoint Quality
Pi05 V7 Full V2 Pi0.5 all 4 tasks 008000 Best overall — works really well
Pi05 V7 Full Pi0.5 all 4 tasks 010000 Works really well
Pi05 V4 Lego Pi0.5 Lego only 030000 Works really well (Lego only)
SmolVLA V6 Full SmolVLA all 4 tasks ~150000 (untested) OK-ish, across most tasks
SmolVLA V5 Full SmolVLA all 4 tasks 100000 Works, but worse than V6
SmolVLA V2 Lego SmolVLA Lego only 030000 OK-ish (Lego only)
SmolVLA V3 Desk Cleanup SmolVLA Desk Cleanup only — Does not work
SmolVLA V1 Misc SmolVLA misc early data — Does not work

The headline result: Pi0.5 substantially outperforms SmolVLA on this setup. The ~450M SmolVLA appears capacity-limited across four tasks, while Pi0.5 handles the full multi-task set well — and needs far fewer steps to get there (8k–10k vs 100k–200k).

Datasets

Dataset Episodes Content
Full Merged Final V1 930 The training set. All 4 tasks, 93 prompts, 844,208 frames
Lego 460 Colour sorting
Desk Cleanup 350 Desk clearing (250 of these went into the merge)
Dice Throw 120 Dice cup
Fetch Ball 100 Ball handover
Collection — Early miscellaneous data; superseded

Robot setup

Robot SO-101 follower arm (6-DOF), robot_type: so_follower
Teleoperation SO-101 leader arm
Control frequency 30 fps
State / action space 6-dim: shoulder_pan.pos, shoulder_lift.pos, elbow_flex.pos, wrist_flex.pos, wrist_roll.pos, gripper.pos
Camera observation.images.desk_view 800x600, h264 (recording)
Camera observation.images.wrist_left 640x480, h264 (recording)

Inference note: both cameras are run at 640x480 during inference, not at their recording resolutions, to reduce the payload sent to the inference server.

Prompt design

Language conditioning was varied deliberately: the prompt phrasing was changed roughly every 10 episodes during recording. The merged dataset therefore contains 93 distinct prompts across 10 episodes each, all in English. Full prompt lists are on each dataset card.

Licensing

All models and datasets here are CC BY-SA 4.0 — share-alike, so derivatives must stay open. Upstream LeRobot, SmolVLA and Pi0.5/openpi components are Apache-2.0; their copyright notices and licence text are retained as Apache-2.0 Section 4 requires.

Citation

@misc{project_ira_2026,
  title        = {Project-IRA: Interactive Robotic Arm},
  author       = {Baten, Cleo and Keppler, Bela and Sapper, Jonas},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Project-IRA}},
  note         = {Code: \url{https://github.com/Project-IRA/interactive-robotic-arm}}
}