Rezakakooee/flux_hack_lora_erh_r6

FLUX 3 Action SO-101 LoRA: paper cups + Snickers (2 tasks), run 6

A LoRA adapter for black-forest-labs/flux-3-action-so101, trained fresh on the team's teleop recording, 130 episodes of two tasks:

Task text (use it exactly)	Episodes
stack the paper cups on top of each other	100
Put all the Snickers bars on the plate	30

ERH Physical AI Hackathon, October 2026. For the cups task only, see also flux_hack_lora_erh_cups_r3. A run with the third task (Snickers + sponge) follows.

What you need

• LeRobot with FLUX 3 (the same install you use for the zero-shot run).
• A GPU with about 25 GB free.
• Access to the base model on Hugging Face (accept its license once).
• The same SO-101 arm and calibration as the recording. joint_units.json holds this arm's joint offsets.

Joint mapping (only if you do not use the plugin)

The plugin (so101_follower_flux with --robot.joint_units) does this for you. A robot client of your own must convert the joint values both ways, joint order shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper:

• robot -> model: state_model = sign * state_lerobot + offset, before the state goes into the model.
• model -> robot: action_lerobot = (action_model - offset) / sign, right before the command goes to the motors.
• Keep the model's own command history in model units (do not convert it back).
• Normalization: use the one saved in this adapter (policy_preprocessor* / policy_postprocessor* files), not the HF Space's q01/q99.

sign = [1, -1, 1, 1, -1, 1], scale 1. offset on the arm as it is now (wrist roll at the folded rest about -14 in raw LeRobot units):

Task	offset
stack the paper cups on top of each other	[0, 92.901, 80.648, 0, -14.077, 0]
Put all the Snickers bars on the plate	[0, 92.901, 80.648, 0, -111.077, 0]

If the wrist roll at rest reads about -111 instead, the arm is back in the old calibration: ask us first.

Run it on the robot

1. Download this repo and install the robot plugin (no other dependencies):

   huggingface-cli download Rezakakooee/flux_hack_lora_erh_r6 --local-dir erh_r6
   pip install -e erh_r6/plugin/lerobot_robot_flux_units
   

2. Start the rollout (pick one task text):

   lerobot-rollout --strategy.type=base \
       --robot.type=so101_follower_flux --robot.port=<port> --robot.id=<arm id> \
       --robot.joint_units=erh_r6/joint_units.json \
       --robot.image_size="[480, 640]" \
       --robot.max_relative_target=10 \
       --robot.cameras="{scene: {..., width: 1280, height: 960}, wrist: {..., width: 640, height: 480}}" \
       --policy.path=erh_r6 \
       --task="Put all the Snickers bars on the plate"
   

Important

• --robot.image_size="[480, 640]" is required. The model needs both cameras at one size; without it the rollout stops with cameras must share (T, C, H, W).
• Use the task text exactly as in the table.
• Keep a hand on the stop button.
• One model call takes about 1.6 s (1.0 s if the adapter is merged), and one chunk of 32 actions lasts 1.07 s. Short pauses between chunks are expected.
• After a re-calibration of the arm, joint_units.json is no longer right.

Today's arm: which joint units file

The arm was re-calibrated during the recording: from episode 104 on it reads the wrist roll 97 degrees higher (at the folded rest about -14 in raw LeRobot units instead of about -111). This adapter learned the cups task in the old reading and most Snickers episodes in the new one. On the arm as it is now:

Task	--robot.joint_units=
Put all the Snickers bars on the plate	erh_r6/joint_units.json
stack the paper cups on top of each other	erh_r6/joint_units_current_arm.json (wrist roll offset -14.077)

Check the wrist roll at the folded rest before a rollout. If the arm is re-calibrated again, ask us first.

Offline results

Mean error (degrees) of the predicted vs the recorded commands over 256 steps, on recorded episodes that were not used for training (episodes 95 and 129; one episode per task, so the numbers are noisy):

Policy	Cups (episode 95)	Snickers (episode 109)
This LoRA	42.4	24.7
cups_r3 (cups only)	25.9	47.5
erh_r4 (10 Snickers episodes)	42.9	39.5
Zero-shot	93.4	147.6

Offline, this adapter is the best on the Snickers task and cups_r3 is the best on the cups task. The robot is the real test: please try both.

Training

• BFL's LoRA recipe (r 32, batch 2 x 4, lr 1e-4), raw weights (not EMA), 9,000 microsteps on an A100, fresh from the base, on 123 of the 130 episodes (7 held out).
• The scene camera was resized from 1280x960 to 640x480 for training.
• Details: provenance.json.

License

FLUX Kommunity License v1.0: non-commercial research only. FLUX and its outputs must not be used to train another model with a similar function.