VideoVista-CulturalLingo: 360^circ Horizons-Bridging Cultures,
  Languages, and Domains in Video Comprehension
Authors: name: Xinyu Chen; name: Yunxin Li; name: Haoyuan Shi; name: Baotian Hu; name: Wenhan Luo; name: Yaowei Wang; name: Min Zhang
Published: 2025-04-23 13:47:30

Abstract: Assessing the video comprehension capabilities of multimodal AI systems can
effectively measure their understanding and reasoning abilities. Most video
evaluation benchmarks are limited to a single language, typically English, and
predominantly feature videos rooted in Western cultural contexts. In this
paper, we present VideoVista-CulturalLingo, the first video evaluation
benchmark designed to bridge cultural, linguistic, and domain divide in video
comprehension. Our work differs from existing benchmarks in the following ways:
1) Cultural diversity, incorporating cultures from China, North America, and
Europe; 2) Multi-linguistics, with questions presented in Chinese and
English-two of the most widely spoken languages; and 3) Broad domain, featuring
videos sourced from hundreds of human-created domains. VideoVista-CulturalLingo
contains 1,389 videos and 3,134 QA pairs, and we have evaluated 24 recent
open-source or proprietary video large models. From the experiment results, we
observe that: 1) Existing models perform worse on Chinese-centric questions
than Western-centric ones, particularly those related to Chinese history; 2)
Current open-source models still exhibit limitations in temporal understanding,
especially in the Event Localization task, achieving a maximum score of only
45.2%; 3) Mainstream models demonstrate strong performance in general
scientific questions, while open-source models demonstrate weak performance in
mathematics.