Rezakakooee/flux_hack_lora_erh_r8

FLUX 3 Action SO-101 LoRA: cups, Snickers, Snickers + sponge (3 tasks), run 8

A LoRA adapter for black-forest-labs/flux-3-action-so101, trained fresh on the team's teleop recording, 300 episodes of three tasks (100 each):

Task text (use it exactly)	Episodes
stack the paper cups on top of each other	100
Put all the Snickers bars on the plate	100
Place Snickers (brown wrapper) on the left plate and the sponge (green) on the right plate.	100

ERH Physical AI Hackathon, October 2026.

What you need

• LeRobot with FLUX 3 (the same install you use for the zero-shot run).
• A GPU with about 25 GB free.
• Access to the base model on Hugging Face (accept its license once).
• The same SO-101 arm as the recording.

Today's arm: which joint units file

The arm was re-calibrated during the recording (from episode 104 on it reads the wrist roll 97 degrees higher). This adapter was trained with that corrected, so one file works for every task on the arm as it is now: erh_r8/joint_units.json (wrist roll offset -14.077). Check the wrist roll at the folded rest before a rollout (about -14 in raw LeRobot units). If it reads about -111, the arm is back in the old calibration: ask us first.

Joint mapping (only if you do not use the plugin)

The plugin (so101_follower_flux with --robot.joint_units) does this for you. A robot client of your own must convert the joint values both ways, joint order shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper:

• robot -> model: state_model = sign * state_lerobot + offset, before the state goes into the model.
• model -> robot: action_lerobot = (action_model - offset) / sign, right before the command goes to the motors.
• Keep the model's own command history in model units (do not convert it back).
• Normalization: use the one saved in this adapter (policy_preprocessor* / policy_postprocessor* files), not the HF Space's q01/q99.

sign = [1, -1, 1, 1, -1, 1], scale 1. offset on the arm as it is now (wrist roll at the folded rest about -14 in raw LeRobot units):

Task	offset
all three tasks	[0, 92.901, 80.648, 0, -14.077, 0]

If the wrist roll at rest reads about -111 instead, the arm is back in the old calibration: ask us first.

Run it on the robot

1. Download this repo and install the robot plugin (no other dependencies):

   huggingface-cli download Rezakakooee/flux_hack_lora_erh_r8 --local-dir erh_r8
   pip install -e erh_r8/plugin/lerobot_robot_flux_units
   

2. Start the rollout (pick the task text and its joint units file from the table above):

   lerobot-rollout --strategy.type=base \
       --robot.type=so101_follower_flux --robot.port=<port> --robot.id=<arm id> \
       --robot.joint_units=erh_r8/joint_units.json \
       --robot.image_size="[480, 640]" \
       --robot.max_relative_target=10 \
       --robot.cameras="{scene: {..., width: 1280, height: 960}, wrist: {..., width: 640, height: 480}}" \
       --policy.path=erh_r8 \
       --task="Place Snickers (brown wrapper) on the left plate and the sponge (green) on the right plate."
   

Important

• --robot.image_size="[480, 640]" is required. The model needs both cameras at one size.
• Use the task text exactly as in the table.
• Keep a hand on the stop button.
• One model call takes about 1.6 s (1.0 s if the adapter is merged); one chunk of 32 actions lasts 1.07 s.

Offline results

Mean error (degrees) of the predicted vs the recorded commands over 256 steps, on recorded episodes that were not used for training (one episode per task, so the numbers are noisy; the robot is the real test):

Policy	Cups (episode 99)	Snickers (episode 199)	Sponge (episode 239 / 299)
This LoRA (9k)	19.4	92.8	– / 78.9
erh_r7	43.6	94.1	70.6 / –
erh_r6	22.4	91.6
erh_cups_r3	16.6
Zero-shot	95.1	168-180	83.8 / 69.1

Training

• BFL's LoRA recipe (r 32, batch 2 x 4, lr 1e-4), raw weights (not EMA), 9,000 microsteps on an A100, fresh from the base, on 285 of the 300 episodes (15 held out).
• The scene camera was resized from 1280x960 to 640x480 for training.
• Details: provenance.json.

License

FLUX Kommunity License v1.0: non-commercial research only. FLUX and its outputs must not be used to train another model with a similar function.