Benchmarking Multimodal Mathematical Reasoning with Explicit Visual
  Dependency
Authors: name: Zhikai Wang, name: Jiashuo Sun, name: Wenqi Zhang, name: Zhiqiang Hu, name: Xin Li, name: Fan Wang, name: Deli Zhao

Recent advancements in Large Vision-Language Models (LVLMs) have
significantly enhanced their ability to integrate visual and linguistic
information, achieving near-human proficiency in tasks like object recognition,
captioning, and visual question answering. However, current benchmarks
typically focus on knowledge-centric evaluations that assess domain-specific
expertise, often neglecting the core ability to reason about fundamental
mathematical elements and visual concepts. We identify a gap in evaluating
elementary-level math problems, which rely on explicit visual
dependencies-requiring models to discern, integrate, and reason across multiple
images while incorporating commonsense knowledge, all of which are crucial for
advancing toward broader AGI capabilities. To address this gap, we introduce
VCBENCH, a comprehensive benchmark for multimodal mathematical reasoning with
explicit visual dependencies. VCBENCH includes 1,720 problems across six
cognitive domains, featuring 6,697 images (averaging 3.9 per question) to
ensure multi-image reasoning. We evaluate 26 state-of-the-art LVLMs on VCBENCH,
revealing substantial performance disparities, with even the top models unable
to exceed 50% accuracy. Our findings highlight the ongoing challenges in
visual-mathematical integration and suggest avenues for future LVLM
advancements.