act-so101-pickplace-dr — domain-randomization-trained ACT

ACT (~52M) trained from scratch on so101-sim-pickplace-dr — 150 successful SO-ARM100 pick-and-place demonstrations recorded under per-reset visual + physics domain randomization in MuJoCo. Same recipe as the nominal baseline (chunk 45, n_action_steps 15, batch 8, 60k steps); only the data differs, so every improvement below is attributable to the data.

Results

Robustness sweep, 15 eval episodes per randomization level, fixed eval seed:

	nominal	+ visual DR	+ physics DR	+ full DR
Scripted expert (privileged state)	100%	100%	100%	100%
ACT v1 (trained on nominal data)	60%	60%	53%	40%
ACT-DR (this model)	87%	87%	87%	100%

Standard 20-episode nominal protocol: 18/20 (90%) — vs the nominal-trained baseline's 65% (75% with temporal ensembling). Training on randomized data was not a robustness tax: it improved nominal performance by 27 points while staying flat across every distribution shift.

Measurement integrity note: the companion project's originally-published claim that the nominal policy "collapses to 0%" under randomization was an eval-harness artifact (a GL-context teardown bug fed the policy black frames); it was caught, fixed, and re-measured — full forensic in the sim2cell README.

Training

• LeRobot 0.6.0 lerobot-train, single Kaggle T4, AMP — 60k steps in 2h12m (~0.13 s/step), loss 9.39 → 0.036 (28.7 epochs of 150 episodes)
• --policy.type=act --policy.chunk_size=45 --policy.n_action_steps=15 --batch_size=8

Use

from lerobot.policies import make_pre_post_processors
from lerobot.policies.act.modeling_act import ACTPolicy

repo = "ahmedsohail2003/act-so101-pickplace-dr"
policy = ACTPolicy.from_pretrained(repo).to("cuda").eval()
pre, post = make_pre_post_processors(
    policy_cfg=policy.config, pretrained_path=repo,
    preprocessor_overrides={"device_processor": {"device": "cuda"}},
)
# batch: observation.state (1,6) + observation.images.front / .wrist (1,3,224,224) in [0,1]
action = post(policy.select_action(pre(batch)))

Provenance

Data recorded by a scripted expert (success-filtered) in the author's MuJoCo work-cell; robot model from MuJoCo Menagerie (trs_so_arm100, Apache-2.0).