ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning
  and Online Reinforcement Learning
Authors: name: Hanyang Chen, name: Mark Zhao, name: Rui Yang, name: Qinwei Ma, name: Ke Yang, name: Jiarui Yao, name: Kangrui Wang, name: Hao Bai, name: Zhenhailong Wang, name: Rui Pan, name: Mengchao Zhang, name: Jose Barreiros, name: Aykut Onol, name: ChengXiang Zhai, name: Heng Ji, name: Manling Li, name: Huan Zhang, name: Tong Zhang
Organization: University of Illinois at Urbana-Champaign

Recent advances in embodied AI highlight the potential of vision language
models (VLMs) as agents capable of perception, reasoning, and interaction in
complex environments. However, top-performing systems rely on large-scale
models that are costly to deploy, while smaller VLMs lack the necessary
knowledge and skills to succeed. To bridge this gap, we present
Embodied Reasoning Agent (ERA), a two-stage framework that integrates
prior knowledge learning and online reinforcement learning (RL). The first
stage, Embodied Prior Learning, distills foundational knowledge from
three types of data: (1) Trajectory-Augmented Priors, which enrich existing
trajectory data with structured reasoning generated by stronger models; (2)
Environment-Anchored Priors, which provide in-environment knowledge and
grounding supervision; and (3) External Knowledge Priors, which transfer
general knowledge from out-of-environment datasets. In the second stage, we
develop an online RL pipeline that builds on these priors to further enhance
agent performance. To overcome the inherent challenges in agent RL, including
long horizons, sparse rewards, and training instability, we introduce three key
designs: self-summarization for context management, dense reward shaping, and
turn-level policy optimization. Extensive experiments on both high-level
planning (EB-ALFRED) and low-level control (EB-Manipulation) tasks demonstrate
that ERA-3B surpasses both prompting-based large models and previous
training-based baselines. Specifically, it achieves overall improvements of
8.4\% on EB-ALFRED and 19.4\% on EB-Manipulation over GPT-4o, and exhibits
strong generalization to unseen tasks. Overall, ERA offers a practical path
toward scalable embodied intelligence, providing methodological insights for
future embodied AI systems.