ASethi04/MolmoAct2-BimanualYAM-oranges

MolmoAct2-BimanualYAM — "Put all oranges in the bowl" (5k steps)

Fine-tune of allenai/MolmoAct2-BimanualYAM on brandonyang/yam-vive-teleop (80 teleop episodes, 74,927 frames, bimanual YAM, 30 fps) for the task "Put all oranges in the bowl", which the base checkpoint does not solve.

A longer 12,000-step version is at ASethi04/MolmoAct2-BimanualYAM-oranges-12k and is expected to be the stronger policy.

Trainable params	727,296,544 / 5,591,928,304 (13%)
VLM	LoRA r=64, α=16, dropout=0.05 @ LR 5e-5
Action expert	fully fine-tuned @ LR 1.5e-4
Steps / epochs	5,000 / 4.3
Global batch	64 (8 GPUs × 8)
Optimizer	AdamW β=(0.9,0.95), ε=1e-6, wd=0, clip 1.0
Schedule	cosine, 250-step warmup, decay ratio 0.1
Precision	bfloat16 + gradient checkpointing
Action mode	both (discrete FAST + flow matching), 8 flow timesteps
Chunk / executed	30 / 30 (1 s @ 30 Hz)
Action space	14-D absolute joint pose
Cameras	observation.images.{top,left,right} @ 480×270
Normalization	quantile q01/q99; grippers raw
Split / seed	100/0 (all 80 episodes) / 1000

Final training step: step:5K smpl:320K ep:342 epch:4.27 loss:0.923 grdn:1.967 lr:5.0e-06 updt_s:2.404 data_s:0.075 smp/s:26 mem_gb:25.86 discrete_ce_loss:0.919 discrete_z_loss:0.000 action_flow_loss:0.004

Usage notes

1. Set inference_action_mode="continuous" — the saved config has None.
2. Do not pass norm_tag — normalization stats come from the fine-tuning dataset and are baked into the processor files.
3. Task string must match training exactly: Put all oranges in the bowl.

lerobot-policy-server \
  --policy.pretrained_name_or_path=ASethi04/MolmoAct2-BimanualYAM-oranges \
  --policy.inference_action_mode=continuous \
  --policy.model_dtype=bfloat16 --policy.device=cuda --host=0.0.0.0 --port=8081

Limitations

Single task, 80 demonstrations, no held-out validation set. Validate on hardware with a no-motion action probe before arming the robot.