anant9/molmoact2-so101-manipulation-step4000

MolmoAct2 SO100/SO101 — LoRA finetune (step 4000)

LoRA finetune of lerobot/MolmoAct2-SO100_101-LeRobot on anant9/manipulation-tasks (53 episodes / 76,951 frames / 39 tasks, SO101 arm, 2 cameras @ 224x224, 6-DoF absolute joint pose).

Training

Step	4000 of a planned 10,000
Trainable params	212.3M / 5.65B (3.8%)
LoRA	rank 64, alpha 16, dropout 0.05; VLM + action expert
Precision	bfloat16
Effective batch	48 (24 per GPU x 2)
Hardware	2x RTX PRO 6000 Blackwell
Last logged metrics (step ~5k)	loss 0.011, grad-norm 0.053, lr 3.1e-05

Normalization uses the base checkpoint's pretrained stats (norm_tag: so100_so101_molmoact2), not per-dataset stats — matching how MolmoAct2 is designed to normalize across SO100/SO101.

Important: weights are NOT merged

These are base weights plus LoRA adapter tensors (1194 adapter tensors), not a merged model. Loading therefore requires peft, and the config still references two external repos that are fetched at load time for the tokenizer and input packing:

• allenai/MolmoAct2-SO100_101
• allenai/MolmoAct2-FAST-Tokenizer

An offline machine needs those cached as well — this repo alone is not self-contained.

Requirements

• lerobot==0.6.1 with the molmoact2 extra (pip install 'lerobot[molmoact2]')
• config.json records device: cuda; override it if the target machine differs.