BEAR: Benchmarking and Enhancing Multimodal Language Models for Atomic
  Embodied Capabilities
Authors: name: Yu Qi, name: Haibo Zhao, name: Ziyu Guo, name: Siyuan Ma, name: Ziyan Chen, name: Yaokun Han, name: Renrui Zhang, name: Zitiantao Lin, name: Shiji Xin, name: Yijian Huang, name: Kai Cheng, name: Peiheng Wang, name: Jiazheng Liu, name: Jiayi Zhang, name: Yizhe Zhu, name: Wenqing Wang, name: Yiran Qin, name: Xupeng Zhu, name: Haojie Huang, name: Lawson L. S. Wong

Embodied capabilities refer to a suite of fundamental abilities for an agent
to perceive, comprehend, and interact with the physical world. While multimodal
large language models (MLLMs) show promise as embodied agents, a thorough and
systematic evaluation of their embodied capabilities remains underexplored, as
existing benchmarks primarily focus on specific domains such as planning or
spatial understanding. To bridge this gap, we introduce BEAR, a comprehensive
and fine-grained benchmark that evaluates MLLMs on atomic embodied
capabilities. BEAR comprises 4,469 interleaved image-video-text entries across
14 domains in 6 categories, including tasks from low-level pointing, trajectory
understanding, spatial reasoning, to high-level planning. Extensive evaluation
results of 20 representative MLLMs reveal their persistent limitations across
all domains of embodied capabilities. To tackle the shortfall, we propose
BEAR-Agent, a multimodal conversable agent that integrates pretrained vision
models to strengthen MLLM perception, 3D understanding, and planning
capabilities. It substantially enhances MLLM performance across diverse
embodied capabilities on BEAR, yielding a 9.12% absolute gain and a relative
improvement of 17.5% on GPT-5. Furthermore, our experiments indicate that
improving MLLM embodied capabilities can benefit embodied tasks in simulated
environments. Project website: https://bear-official66.github.io/