PointArena: Probing Multimodal Grounding Through Language-Guided
  Pointing
Authors: name: Long Cheng; name: Jiafei Duan; name: Yi Ru Wang; name: Haoquan Fang; name: Boyang Li; name: Yushan Huang; name: Elvis Wang; name: Ainaz Eftekhar; name: Jason Lee; name: Wentao Yuan; name: Rose Hendrix; name: Noah A. Smith; name: Fei Xia; name: Dieter Fox; name: Ranjay Krishna
Published: 2025-05-15 06:04:42

Abstract: Pointing serves as a fundamental and intuitive mechanism for grounding
language within visual contexts, with applications spanning robotics, assistive
technologies, and interactive AI systems. While recent multimodal models have
started to support pointing capabilities, existing benchmarks typically focus
only on referential object localization tasks. We introduce PointArena, a
comprehensive platform for evaluating multimodal pointing across diverse
reasoning scenarios. PointArena comprises three components: (1) Point-Bench, a
curated dataset containing approximately 1,000 pointing tasks across five
reasoning categories; (2) Point-Battle, an interactive, web-based arena
facilitating blind, pairwise model comparisons, which has already gathered over
4,500 anonymized votes; and (3) Point-Act, a real-world robotic manipulation
system allowing users to directly evaluate multimodal model pointing
capabilities in practical settings. We conducted extensive evaluations of both
state-of-the-art open-source and proprietary multimodal models. Results
indicate that Molmo-72B consistently outperforms other models, though
proprietary models increasingly demonstrate comparable performance.
Additionally, we find that supervised training specifically targeting pointing
tasks significantly enhances model performance. Across our multi-stage
evaluation pipeline, we also observe strong correlations, underscoring the
critical role of precise pointing capabilities in enabling multimodal models to
effectively bridge abstract reasoning with concrete, real-world actions.
Project page: https://pointarena.github.io/