MadmanMischief/so101_candy_rl

SO101 candy pick — RL policy and Isaac Lab twin

Task: teach an SO101 arm (leader/follower teleoperation) to pick the blue candies out of a mixed blue and green pile and drop them on a towel. Language prompt: "pick up the blue candy". Built during the B1S Hacktober Physical AI 2026 hackathon with LeRobot 0.6.2.

Robot: SO101 follower, 6 joints (pan, lift, elbow, wrist flex, wrist roll, gripper) at 30 Hz. Cameras: wrist (USB webcam on the gripper) and phone (Android phone beside the table).

This repo holds the reinforcement-learning side: a state-based PPO policy trained from scratch in an Isaac Lab task, plus the code that deploys it on the real arm and the Isaac Sim digital-twin scripts used for scripted demos and sim evaluation of the imitation models.

Contents

• isaaclab_task_so101_candy/ — Isaac Lab direct task Isaac-SO101-CandyPick-Direct (env, config, rsl_rl PPO config, real grasp poses). Drop into Isaac Lab 3.0 contrib/ and train with scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-SO101-CandyPick-Direct.
• runs/ — rsl_rl training runs (v1, v2 grasp shaping, v3 pre-grasp curriculum and resumes) with checkpoints, params and TensorBoard events.
• rl_deploy/ — real-arm deployment: policy_final.pt (exported final policy), deploy_rl.py, calibrate_table.py (8-point phone-to-table calibration), FK and URDF. See rl_deploy/README.md.
• twin/ — Isaac Sim 6.1 digital twin of the table, candies and cameras (twin_candy_sort.py, scripted_demos.py, camera fitting and eval scripts).

How it runs on the arm

The phone image gives the blue candy's pixel, the table calibration (2 to 15 mm error) turns it into a table position, and the policy maps that plus the joint angles to joint targets at 30 Hz, limited to 1° per step.

cd rl_deploy
python calibrate_table.py --points 8      # once per camera placement
python deploy_rl.py --policy policy_final.pt --go

Results

• Sim: over training episodes the final policy lifts the blue candy about 3.5 to 4.3% of the time and places it on the towel ~0–0.1%.
• Real arm: reaching transfers (the gripper ends up over the detected candy), grasping does not yet: in the recorded runs it hovered over or reached the blue candy but never closed the gripper, because it hasn't learned to close it in sim either.