lift_act_sim_v3 — the cell as the lab measured it

An action chunking transformer trained by behaviour cloning to lift a 2 × 6 × 2 cm red block 10 cm off the table of a simulated Dobot X-Trainer dual-arm cell — omn4i/Lift-v0 in Physical41/stack.

88.6% success on held-out seeds, against the scripted expert's 99.0%.

Which cell this is for — read this before using any version

	lift_act_sim_v1	lift_act_sim_v2	v3 (this one)
Cell	50 mm foam slab on the table	foam removed	geometry realigned to the lab's measured calibration
Trained at	pre-foam-removal tree	ebb10c8 (omn4i_envs)	648ecd5 (Physical41/stack)
Corpus	2847 episodes	1987 episodes	1985 episodes
Overall	93.8%	81.0%	88.6%
Left / right arm	93.8% / 93.8%	93.0% / 69.8%	96.3% / 81.2%

v1 and v2 are both for cells that no longer exist. Use v3 with current main.

What changed under v2, and what it cost

The simulation was reading its own numbers where the lab had measured them. Five corrections landed in Physical41/stack@7de2fcc; three of them move pixels:

	simulation had	the lab measured
table surface	a round −10 mm	+3.11 mm, from 48 fingertip contacts
arm separation	1060.0 mm	1065.4 mm, load-cell fit at 3.1 mm RMS
right arm riser	1069.4 mm	— it stood 9.4 mm off its own post
ee_pose frame	MuJoCo world	the controller's frame — a half turn and 223.4 mm apart

v2 scores 50.0% on this cell, against the 81.0% its own card reports on the cell it was trained for. Measured here on the same 500 seeds, same tree, same torch:

	overall	left	right
v2 on the realigned cell	50.0% (250/500)	53.7%	46.5%
v3 on the realigned cell	88.6% (443/500)	96.3%	81.2%

Attribution, by reverting one change at a time: the table and riser cost ~22 points, the 5.4 mm of arm movement ~8.7, and the ee_pose frame change exactly zero — which is a clean confirmation that this architecture reads only images and joint positions. The regression is entirely visual. Five millimetres of arm movement costing nearly nine points is the number worth remembering about behaviour cloning.

Results

500 held-out episodes (benchmark.TEST_SEEDS, range(10000, 10500)). None was trained on: the corpus is seeds 0–1999, from benchmark.DEV_SEEDS.

	success	95% Wilson	left arm	right arm
act (this model)	88.6% (443/500)	85.7% – 91.3%	96.3% (235/244)	81.2% (208/256)
classical (its teacher)	99.0% (495/500)	97.7% – 99.6%	—	—

Do not read 88.6% against v2's 81.0% as an improvement

They are different cells. v2's 81.0% was measured on the pre-realignment geometry, and no ablation has been run to attribute the difference. Everything controllable was held equal — 1985 episodes against 1987, 40,000 steps, the same hyperparameters and the same expert — but the environment itself moved, so the honest claim is the regression is repaired, not the model is better. The only controlled comparison on this page is 50.0% against 88.6%, on one cell.

The open problem, still open

A 15.1 point gap between the arms, and its cause is still not known. It narrowed from v2's 23.2 points, which is also unexplained — the same caveat applies, since the cell changed underneath it. The expert shows no such gap: it solves both arms at 99%. Whatever this is, it is the model's, not the demonstrations'.

Training

python sim/tools/record.py lift <corpus> --episodes 2000
python sim/tools/train_act.py <corpus> --steps 40000

Corpus	1985 of 2000 dev seeds succeeded, 104,678 frames, 5955 videos at 30 fps
Parameters	18.7 M
Steps / batch / chunk	40,000 / 48 / 32
Held-out mean absolute error	0.00714 rad
Wall clock	30 min recording, 37.5 min training, one RTX PRO 6000

Recorded and trained in a clean checkout of 648ecd5, so the commit in the table above describes exactly what produced these weights.

Inputs

Three cameras (overhead, left_wrist, right_wrist) at 240 × 320, and the 14 measured joint positions. It does not read ee_pose or depth. That is why the frame correction cost it nothing, and it is worth knowing before assuming a pose fix will reach it.

Using it

from omn4i_envs.baselines import act
policy = act.load("act.pt", replan=1)()

observation, info = env.reset(seed=10000)
policy.reset(observation, info)
action = policy(observation)

Or watch it:

python sim/examples/watch_task.py --task lift --checkpoint act.pt

which defaults to the test seeds, because a policy shown its own training seeds is demonstrating memory.