moonbot SmolVLA — three blocks stack, WITH force/torque
SmolVLA fine-tuned from lerobot/smolvla_base (SmolVLM2-500M backbone; full fine-tune: vision encoder and VLM trainable, as in the pi0 runs; cameras renamed front->camera1, eef->camera2, right->camera3), trained on gdiazsrl/three_blocks_stack_sep22 (296 episodes, 519,684 frames at 30 Hz, 9 language instructions; MoonBot 7-DOF arm, block stacking).
Input/output: 21-dim state = joint_read (8: 7 joints + gripper) + tip_pos (7: end-effector pose) + F_ee (6: force/torque); 3 RGB cameras (front, eef, right); 8-dim action (7 joint positions + gripper).
Comparison run. One of six policies comparing pi0 / SmolVLA / ACT with vs without force/torque, all trained with batch size 16, 20,000 steps, seed 1000 and no validation split; otherwise each uses its own LeRobot preset. The two datasets are the same episodes and frames, except that the 6 F_ee dimensions are removed from the state in the no-F/T one. The counterpart of this model without force/torque is moonbot_smolvla_no_tf.
| model | F/T | repo | W&B run |
|---|---|---|---|
| pi0 | with | moonbot_pi0_three_blocks_stack |
bkbszs91 |
| pi0 | without | moonbot_pi0_no_tf |
ahb0i6l9 |
| SmolVLA | with | moonbot_smolvla_tf |
rn1je6y9 |
| SmolVLA | without | moonbot_smolvla_no_tf |
9i3olvax |
| ACT | with | moonbot_act_tf |
uw4alfzk |
| ACT | without | moonbot_act_no_tf |
a7hpuoto |
Training
| initialization | lerobot/smolvla_base |
| parameters | 450M (393M trainable) |
| language input | yes |
| action chunk | 50 (all 50 executed) |
| optimizer | AdamW, lr 1e-4, betas (0.9, 0.95), weight decay 1e-10, grad clip 10 |
| LR schedule | cosine with 1,000 warm-up steps, decaying towards 2.5e-6 over 30,000 steps (stops at 20,000) |
| batch size / steps | 16 / 20,000 (about 0.62 epochs) |
| seed | 1000 |
| GPU | 1x NVIDIA RTX PRO 6000 Blackwell (96 GB) |
| training time | 2.8 h (GPU shared with another run) |
| final training loss | 0.030 (this policy's own objective; not comparable across algorithms) |
| framework | LeRobot 0.6.1 |
| training curves | https://wandb.ai/drmishra-space/lerobot/runs/rn1je6y9 |
Checkpoints
checkpoint-005000, checkpoint-010000, checkpoint-015000, checkpoint-020000 (final). Each checkpoint-XXXXXX/ folder holds the weights, config.json, train_config.json, the pre/post-processor files, this run's rosetta_contract.yaml and DEPLOY.md. checkpoint-020000/ also includes training_state/ for resuming. deploy/ holds the contract once.
Complete checkpoints, including the optimizer state, are also on the team OneDrive under DATA/Processed/smolvla_tf.
Deployment
Follow DEPLOY.md in the checkpoint folder and use this folder's rosetta_contract.yaml.
Not yet offline-ready: on first load SmolVLA fetches the HuggingFaceTB/SmolVLM2-500M-Video-Instruct config and tokenizer from the Hugging Face Hub, so the robot needs internet once (or a pre-filled HF cache). Note: config.json lists observation.state with shape [6] (inherited from smolvla_base); this is metadata only. The model was trained on, and normalizes, the full state described above (padded to 32).