InstructVLA: Vision-Language-Action Instruction Tuning from
  Understanding to Manipulation
Authors: name: Shuai Yang; name: Hao Li; name: Yilun Chen; name: Bin Wang; name: Yang Tian; name: Tai Wang; name: Hanqing Wang; name: Feng Zhao; name: Yiyi Liao; name: Jiangmiao Pang
Published: 2025-07-23 13:57:06

Abstract: To operate effectively in the real world, robots must integrate multimodal
reasoning with precise action generation. However, existing
vision-language-action (VLA) models often sacrifice one for the other, narrow
their abilities to task-specific manipulation data, and suffer catastrophic
forgetting of pre-trained vision-language capabilities. To bridge this gap, we
introduce InstructVLA, an end-to-end VLA model that preserves the flexible
reasoning of large vision-language models (VLMs) while delivering leading
manipulation performance. InstructVLA introduces a novel training paradigm,
Vision-Language-Action Instruction Tuning (VLA-IT), which employs multimodal
training with mixture-of-experts adaptation to jointly optimize textual
reasoning and action generation on both standard VLM corpora and a curated
650K-sample VLA-IT dataset. On in-domain SimplerEnv tasks, InstructVLA achieves
30.5% improvement over SpatialVLA. To evaluate generalization, we introduce
SimplerEnv-Instruct, an 80-task benchmark requiring closed-loop control and
high-level instruction understanding, where it outperforms a fine-tuned OpenVLA
by 92% and an action expert aided by GPT-4o by 29%. Additionally, InstructVLA
surpasses baseline VLMs on multimodal tasks and exhibits inference-time scaling
by leveraging textual reasoning to boost manipulation performance in both
simulated and real-world settings. These results demonstrate InstructVLA's
potential for bridging intuitive and steerable human-robot interaction with
efficient policy learning.