Foundation Models Meet Embodied Agents (2025)

Abstract: This tutorial will present a systematic overview of recent advances in foundation models for embodied agents, covering three types of foundation models based on input and output: Large Language Models (LLMs), Vision-Language Models (VLMs), Vision-Language-Action Models (VLAs)

## tutorial content an embodied agent is a generalist agent that can take natural language instructions from humans and perform a wide range of tasks in diverse environments. recent years have witnessed the emergence of large language models (llms) as powerful tools for building large agent models (lams), which have shown remarkable success in supporting embodied agents for different abilities such as goal interpretation @xcite @xcite @xcite @xcite , subgoal decomposition @xcite @xcite @xcite , action sequencing @xcite @xcite @xcite @xcite @xcite @xcite , and transition modeling (causal transitions from preconditions to posteffects) @xcite @xcite @xcite @xcite . however, moving from language models to embodied agents poses significant challenges in understanding lower-level visual details, and long-horizon reasoning for reliable embodied decision-making. we categorize the foundation models into large language models (llms), vision-language mod-els (vlms), and vision-language-action models (vlas). we investigate how embodied decisionmaking abilities differ between these models and how they scale as they get larger. this tutorial will present a systematic overview of recent advances in foundation models for embodied agents, covering three types of foundation models based on input and output: we compare these models and explore their design space to guide future developments, focusing on the following key aspect: