MikelRubi/act-up-down-good

Up-down GOOD — ACT sorting policy for parts with pin (SO-101)

Carries a part WITH pin (GOOD) from the inspection camera to the green box and releases it. Stage 3a of the pipeline below. The sister policy for parts without a pin is MikelRubi/act-up-down-bad.

Recommended checkpoint

Checkpoint	up-down-good_v2/checkpoints/020000/pretrained_model
Why	Lowest error on the executed actions (7.88) and the only version that beats "copy the state" over the whole chunk (+16 %)
Status	Intermediate: on the executed window it does not yet beat "copy the state" (5.04). Not yet tested on the robot

The system

Built at the Robotekin LeRobot Hackathon 2026 (23–25 October 2026, Robotory, Vitoria-Gasteiz). One SO-101 arm runs a small quality-control cell for a metal bracket that may or may not carry a pin. Project page: MikelRubi/robotekin-so101-pin-sorting.

flowchart LR
    order["Order (voice / text)<br/>intent model → sorting rule"] --> tray["Part lies in the tray"]
    tray --> pick["1 · act-pick<br/>grasp the part according to its orientation,<br/>lift it, hold it up to the camera"]
    pick --> cls{"2 · pin-classifier<br/>front + wrist images"}
    cls -- "rule: usually pin (GOOD)" --> good["3a · act-up-down-good<br/>carry it to the green box, release"]
    cls -- "rule: usually no pin (BAD)" --> bad["3b · act-up-down-bad<br/>carry it to the white box, release"]

Stage	Repository	Model
1. Picking	MikelRubi/act-pick	ACT policy
2. Inspection	MikelRubi/pin-classifier	EfficientNet-B0 image classifier
3a. Sorting, GOOD	MikelRubi/act-up-down-good	ACT policy
3b. Sorting, BAD	MikelRubi/act-up-down-bad	ACT policy

Picking is a separate policy because the bracket's geometry makes the right grasp depend on its orientation; the two sorting policies start where the pick policy ends (part held in front of the camera).

Orchestration. The team's integrated demo, 30_full_demo_app.py, runs the loop on the real follower: an order (voice in Spanish, English or Basque, text, or Telegram) is read by the intent model IkerAzurmendi/bin-intent-gliner2; the arm waits until a part lies still in the tray; the pick policy lifts it to the camera; the classifier averages 1.5 s of frames; the order maps PIN / NO PIN to the green or the white box, and that box's policy carries the part there (policy_routes.json, run through run_policy.py). In the usual order GOOD parts go to the green box and BAD parts to the white box. During data collection the operator looked at the part and chose which sorting demonstration to record. The start/end-pose contract between stages is in act-up-down-bad/HANDOFF.md.

Naming note: good / bad describe the part, not the quality of a demonstration. GOOD = part with a pin → green box. BAD = part without a pin (defective) → white box. up-down-bad is a policy trained on successful demonstrations of carrying a defective part to the white box. No failed or negative demonstrations were intentionally used for training. Other informal names kept for compatibility: so101-pick-cube_* datasets contain the bracket, not a cube, and up-down_* datasets are the GOOD-box task (except the 23 October warm-up sessions).

Hardware

Component	What is known	Source
Arm	SO-101 follower, 6 joints: shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper (degrees; gripper 0–100)	dataset robot_type: so_follower, recording scripts
Teleoperation	SO-101 leader arm driving the follower (leader/follower); action = leader position	recording scripts
Cameras	Two USB RGB cameras: front (fixed overhead camera looking down at the tray; the part is lifted towards it for inspection) and wrist (on the gripper). 640×480 at 30 fps; the first sessions of 23 October used 1280×720 at 10 fps	dataset meta/info.json
Work cell	Orange tray with the parts, inspection position in front of the camera, green (big) box and white (small) box	recordings, tray_detect.py, demo_orders.py
Recording PC	Linux, arms on USB serial ports	team notes

Not documented yet: camera models and exact mounting positions, workspace dimensions and lighting, servo models, the part's dimensions, and the calibration procedure beyond LeRobot's standard lerobot-calibrate.

Repository layout

Folder	Original Training Studio run	Saved steps	Can resume from
up-down-good_v1/	G1-10k	2.5K · 5K · 7.5K · 10K	all
up-down-good_v1b/	Green1	2.5K	2.5K
up-down-good_v2/	G2-Green-all-20k	5K · 10K · 15K · 20K	all
up-down-good_v3/	G3-ShortChunk-Reg-10k	2.5K · 5K · 7.5K · 10K	10K
up-down-good_v4/	G4-G2cap-NoState-10k	5K · 10K	10K
up-down-good_v5/	G5-State-ShortChunk-G2Cap-10k	5K · 10K	10K

Each version holds checkpoints/<step>/pretrained_model/ (weights and normalisers, loaded with ACTPolicy.from_pretrained), dataset_features.json per checkpoint and, where resuming is possible, training_state/ (optimizer, scheduler and RNG). binpick_info.json describes the run configuration and evaluations/ holds its offline evaluations.

Training data

• v1 and v1b: a single dataset, MikelRubi/up-down_20261024_125916 (7 episodes: 5 for training, 2 for validation).
• v2 to v5: up-down-green-all, a local combination of the GOOD-box recordings (MikelRubi/up-down_* with the task "Carry the bracket from the camera to the good box and release it"): 213 episodes, 170 for training and 43 held out for validation. The exact list of source sessions was not recorded; the 32 GOOD-box datasets on the Hub hold 189 episodes, so the combined dataset is not a plain union of them.

Inputs: front camera (RGB 480×640) and, except in v4, the 6 joint positions (observation.state). Output: action, 6 absolute joint targets. Trained with the team's Training Studio (train_act.py, LeRobot 0.4.4).

Versions

Common base: ACT with an ImageNet ResNet18 backbone, VAE, batch 8.

Version	Data	Chunk / actions per prediction	State input	Learning rate	Regularisation	Steps
v1	1 dataset (5 ep.)	50 / 10	yes	1e-5	dropout 0.1	10K
v1b	1 dataset (5 ep.)	50 / 10	yes	1e-5	dropout 0.1	2.5K (partial)
v2	all (170 ep.)	50 / 10	yes	1e-5	dropout 0.1	20K
v3	all	20 / 5	yes	5e-6	dropout 0.2, weight decay 1e-3	10K
v4	all	50 / 5	no	1e-5	dropout 0.1	10K
v5	all	20 / 5	yes	5e-6	dropout 0.1	10K

Bold marks the change from the reference version: v1 for v2, and v2 for v3–v5.

Results (offline validation, 43 held-out episodes)

Same episodes and frames for every version (evaluations/*horizon5*.json or equivalent). Executed window: error of the 5 actions the robot executes before replanning ("copy the state" gives 5.04). Whole chunk: mean error over the full predicted chunk.

Version · step	Executed window (↓)	Whole chunk (↓)	Whole-chunk improvement over copying the state
v1 · 10K	25.54	26.21	−76 %
v2 · 20K	7.88	12.46	+16 %
v3 · 10K	15.05	15.48	−116 %
v4 · 10K	13.89	16.74	−13 %
v5 · 10K	8.35	9.18	−28 %

v1b has no evaluation (partial run). The whole-chunk improvement is only comparable between versions with the same chunk size, because the baseline depends on it: 14.85 with chunk 50 and 7.16 with chunk 20.

Reading: using all the data (v2) is what helps most, bringing the executed-window error from 25.5 down to 7.9. A shorter chunk with more regularisation (v3) or dropping the state (v4) makes it worse. v5, with a short chunk and the state, comes close to v2 without beating it.

Running it on the robot

Load pretrained_model/ with LeRobot and feed observations with the same names and units as in the recordings: front at 640×480 and 30 fps, plus the joint state except for v4. The start/end contract, stop condition and an example control loop are documented for the sister policy in act-up-down-bad (HANDOFF.md, tools/run_policy.py); these versions use only the front camera and need no velocity input.

Limitations

• Offline evaluation only; not yet run on the physical robot.
• No version beats "copy the state" on the executed window yet.
• Trained with LeRobot 0.4.4; loading with newer LeRobot versions has not been checked.

Acknowledgements

Created by Iker Azurmendi, Mikel Zulueta, Ander Autillo and Diego Suarez during the Robotekin LeRobot Hackathon 2026 (23–25 October 2026, Vitoria-Gasteiz). Thanks to Bravo Robots and to Robotekin for the hackathon, to Hugging Face for LeRobot, and to everyone who took part in it.