LW-Bench ACT 5K for X7s
LeRobot ACT policy trained from scratch on the audited X7s strict compositional split in LW-Compositional-Bench.
Training contract
- Dataset:
LightwheelAI/Lightwheel-Tasks-X7S - Train split: 91 clean tasks; tasks 7, 119, and 149 held out
- Inputs: three RGB cameras plus 25D robot state
- Output: native 21D X7s absolute action
- Hardware: 4 GPUs
- Seed: 1000
- Effective global batch: 64
- Checkpoint: 5,000 global optimizer updates
- Evaluation horizon: first 16 valid action steps
Offline 5K results
| Metric | Value |
|---|---|
| H16 MAE | 0.077566 |
| H16 RMSE | 0.163998 |
| Gripper balanced accuracy | 94.73% |
| Boundary-proxy MAE | 0.109228 |
| Interior MAE | 0.071501 |
| Boundary / interior MAE | 1.528x |
These results cover three strict held-out tasks and are descriptive. Offline action error does not replace closed-loop success evaluation.
Load with LeRobot
from lerobot.policies.act.modeling_act import ACTPolicy
policy = ACTPolicy.from_pretrained("Stevenshuqing/LW-Bench-ACT")
policy.eval()
Use LeRobot 0.4.3 and the included preprocessor/postprocessor files. The repository contains the complete pretrained_model directory produced by LeRobot.
Provenance
- Code and protocol: https://github.com/stevenqing/LW-Compositional-Bench
- Full report: https://github.com/stevenqing/LW-Compositional-Bench/blob/main/artifacts/model-eval/x7s/rotation-v0/budget-5k/report.md
- Code revision:
2d0e26dd2fe2e91046f416bd65bc369f8568d7f0 model.safetensorsSHA256:dffc7c1801b77a53c18fe3f78627aa742ec0d158e0ece5b78737ca4edab6dfa4
Limitations
This checkpoint is specific to the X7s observation/action schema. The benchmark is offline because the available A100 GPUs cannot provide Isaac Sim RT camera rendering. Transition boundaries are action-space proxies based on gripper sign changes and robust continuous-action peaks, not human semantic annotations.
- Downloads last month
- -