ataghof/molmoact2-so101nexus-lora-champion

MolmoAct2 x SO-101 sim, LoRA champion (v6)

MolmoAct2 is Ai2's open 5B vision-language-action model, shipped in LeRobot. This is a LoRA fine-tune of allenai/MolmoAct2-SO100_101 that picks up a red cube and places it on a blue target in a MuJoCo simulator of the SO-101 arm: from 0% zero-shot to 93% grasp in five days on one 24 GB GPU, with no human demonstrations.

Full story with videos: project page.

Results

Scored on 30 held-out cube positions, never seen in training:

Metric	v6
Grasp (cube held in the gripper)	93%
Strict success (placed on target, arm at rest)	9/30 (30%)
Loose (released within 6 cm of target)	50%

Recipe

• 500 scripted-expert demos in the sim, with injected noise and recovery (DART-style)
• Binary gripper (open/closed), success-state endings, slow release
• 10-step action chunks, absolute joint targets
• LoRA on the VLM plus full training of the action expert, one 24 GB GPU

The single biggest win was the binary gripper. Delta actions hurt, so v6 drops them.

Use it

from lerobot.policies.molmoact2.modeling_molmoact2 import MolmoAct2Policy

pol = MolmoAct2Policy.from_pretrained("ataghof/molmoact2-so101nexus-lora-champion")
pol.config.inference_action_mode = "continuous"  # not saved at train time
pol.to("cuda").eval()

The full eval harness (rollouts in the SO-101 sim, scoring, video) is in the repo.

Links

• Code, sim, collector, eval harness: https://github.com/ataghof/molmoact2-so101-sim
• Training dataset: https://huggingface.co/datasets/ataghof/so101nexus-cube500-binary

Built on MolmoAct2 (Ai2), LeRobot (Hugging Face), and so101-nexus (John Sutor).