Title: RoboAlign: Reinforcement Learning for Action-Aligned Multimodal Large Language Models
Primary area: applications to robotics, autonomy, planning
Keywords: MLLM, VLA, RL, Alignment, Embodied Reasoning

Abstract: In recent years, state-of-the-art vision–language–action models (VLAs) have been built upon pre-trained multimodal large language models (MLLMs). However, how to systematically train MLLMs to improve VLA performance remains an open problem. Although previous approaches primarily focus on strengthening embodied reasoning, improvement in language ability alone is insufficient to boost VLA performance without proper alignment between language and low-level actions. Motivated by this limitation, we propose RoboAlign, a novel training framework for MLLMs tailored for VLA learning. RoboAlign directly aligns MLLM representations with low-level action generation while simultaneously enhancing embodied reasoning and preserving general-purpose knowledge, thereby producing models well-suited for VLA. Specifically, we achieve action alignment through reinforcement learning, where the model generates action tokens via zero-shot reasoning in natural language. To validate the effectiveness of RoboAlign, we train VLAs by adding a diffusion-based action head on top of an MLLM backbone and evaluate them on major robotics benchmarks. Specifically, training base MLLMs with RoboAlign improves the performance on robotic tasks by 17.5%, 18.9%, and 106.6% on LIBERO, CALVIN, and real-world robotic environments, respectively. Moreover, RoboAlign outperforms models aligned only with language-described actions or with supervised fine-tuning based approaches such as ECoT, demonstrating its effectiveness and broad applicability.

TL;DR: RoboAlign directly aligns MLLM's representation with low-level action while simultaneously enhancing embodied reasoning and preserving general-purpose knowledge, thereby producing models well-suited for VLA