SmolVLA - Stack (joint, 2-cam)

HuggingFace LeRobot SmolVLA fine-tuned on the ManiGuard stack base task (sim Franka Panda). Part of the ManiGuard VLA benchmark - SmolVLA vs pi0.5 vs GR00T on the same task families with identical data, cameras, and controller.

Model

Base: lerobot/smolvla_base - SmolVLM2 vision-language backbone + flow-matching action expert
Embodiment: Franka Panda, 8-D joint state/action (7 arm joints + 1 gripper), padded to SmolVLA's 32-D
Cameras (2): observation.images.top (overview = left view) + observation.images.wrist (256x256)
Action: absolute joint targets (fed straight to a JointController at eval); 50-step chunk
Tuning: SmolVLA default - vision encoder frozen, train the action expert (no LoRA)

Training

8-GPU accelerate DDP, global batch 512, 10360 steps (~2 epochs over 2,652,083 frames)
Data: IDEAS-Lab-Northwestern/datagen-stack-v1-joint-5cam - the 5-cam datagen dataset, prepared to a 2-cam standard-keyed copy (H.264 videos)

Usage

Load with SmolVLAPolicy.from_pretrained("IDEAS-Lab-Northwestern/smolvla-base-datagen-v1-stack-joint-2cam-yanZ") from LeRobot. The checkpoint carries the normalization stats.

WARNING - Convention (must match at eval): joint-space JointController (absolute joint targets) + 2 cameras (observation.images.top = the left overview, observation.images.wrist). A mismatched controller, camera set, or overview view silently feeds an out-of-distribution input.

Paper & Citation

Part of ManiGuard: paper (arXiv:2608.17386) · code · docs

@misc{peng2026maniguard,
  title         = {{MANIGUARD}: A Benchmark and Data Suite for Specification-Grounded
                   Safety Evaluation and Improvement of Robotic Manipulation},
  author        = {Peng, Yiyan and Wang, Philip and Zhan, Simon Sinong and Lyu, Yiqi
                   and Ni, Zhenyang and Yan, Jixin and Wong, Fiorelli and Jiao, Ruochen
                   and Yin, Hang and Cao, Xinyu and Shao, Huajie and Li, Manling
                   and Zhang, Ruohan and Zhu, Qi},
  year          = {2026},
  eprint        = {2608.17386},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2608.17386},
}

License

The base model lerobot/smolvla_base does not declare an explicit license for its weights (its SmolVLM2-500M backbone and the lerobot training code are both Apache-2.0). ManiGuard's fine-tuning contributions are released under Apache-2.0; the licensing status of the combined weights follows the upstream base model. This tag will be aligned if upstream declares a license.