OmniEAR: Benchmarking Agent Reasoning in Embodied Tasks
Authors: name: Zixuan Wang; name: Dingming Li; name: Hongxing Li; name: Shuo Chen; name: Yuchen Yan; name: Wenqi Zhang; name: Yongliang Shen; name: Weiming Lu; name: Jun Xiao; name: Yueting Zhuang
Published: 2025-08-07 17:54:15

Abstract: Large language models excel at abstract reasoning but their capacity for
embodied agent reasoning remains largely unexplored. We present OmniEAR, a
comprehensive framework for evaluating how language models reason about
physical interactions, tool usage, and multi-agent coordination in embodied
tasks. Unlike existing benchmarks that provide predefined tool sets or explicit
collaboration directives, OmniEAR requires agents to dynamically acquire
capabilities and autonomously determine coordination strategies based on task
demands. Through text-based environment representation, we model continuous
physical properties and complex spatial relationships across 1,500 scenarios
spanning household and industrial domains. Our systematic evaluation reveals
severe performance degradation when models must reason from constraints: while
achieving 85-96% success with explicit instructions, performance drops to
56-85% for tool reasoning and 63-85% for implicit collaboration, with compound
tasks showing over 50% failure rates. Surprisingly, complete environmental
information degrades coordination performance, indicating models cannot filter
task-relevant constraints. Fine-tuning improves single-agent tasks dramatically
(0.6% to 76.3%) but yields minimal multi-agent gains (1.5% to 5.5%), exposing
fundamental architectural limitations. These findings demonstrate that embodied
reasoning poses fundamentally different challenges than current models can
address, establishing OmniEAR as a rigorous benchmark for evaluating and
advancing embodied AI systems. Our code and data are included in the
supplementary materials and will be open-sourced upon acceptance.