InstructVLA: Vision-Language-Action Instruction Tuning from
  Understanding to Manipulation
Authors: name: Shuai Yang, name: Hao Li, name: Yilun Chen, name: Bin Wang, name: Yang Tian, name: Tai Wang, name: Hanqing Wang, name: Feng Zhao, name: Yiyi Liao, name: Jiangmiao Pang

To operate effectively in the real world, robots must integrate multimodal
reasoning with precise action generation. However, existing
vision-language-action (VLA) models often sacrifice one for the other, narrow
their abilities to task-specific manipulation data, and suffer catastrophic
forgetting of pre-trained vision-language capabilities. To bridge this gap, we
introduce InstructVLA, an end-to-end VLA model that preserves the flexible
reasoning of large vision-language models (VLMs) while delivering leading
manipulation performance. InstructVLA introduces a novel training paradigm,
Vision-Language-Action Instruction Tuning (VLA-IT), which employs multimodal
training with mixture-of-experts adaptation to jointly optimize textual
reasoning and action generation on both standard VLM corpora and a curated
650K-sample VLA-IT dataset. On in-domain SimplerEnv tasks, InstructVLA achieves
30.5% improvement over SpatialVLA. To evaluate generalization, we introduce
SimplerEnv-Instruct, an 80-task benchmark requiring closed-loop control and
high-level instruction understanding, where it outperforms a fine-tuned OpenVLA
by 92% and an action expert aided by GPT-4o by 29%. Additionally, InstructVLA
surpasses baseline VLMs on multimodal tasks and exhibits inference-time scaling
by leveraging textual reasoning to boost manipulation performance in both
simulated and real-world settings. These results demonstrate InstructVLA's
potential for bridging intuitive and steerable human-robot interaction with
efficient policy learning.