ExpVid: A Benchmark for Experiment Video Understanding & Reasoning
Authors: name: Yicheng Xu; name: Yue Wu; name: Jiashuo Yu; name: Ziang Yan; name: Tianxiang Jiang; name: Yinan He; name: Qingsong Zhao; name: Kai Chen; name: Yu Qiao; name: Limin Wang; name: Manabu Okumura; name: Yi Wang
Published: 2025-10-13 16:45:28

Abstract: Multimodal Large Language Models (MLLMs) hold promise for accelerating
scientific discovery by interpreting complex experimental procedures. However,
their true capabilities are poorly understood, as existing benchmarks neglect
the fine-grained and long-horizon nature of authentic laboratory work,
especially in wet-lab settings. To bridge this gap, we introduce ExpVid, the
first benchmark designed to systematically evaluate MLLMs on scientific
experiment videos. Curated from peer-reviewed video publications, ExpVid
features a new three-level task hierarchy that mirrors the scientific process:
(1) Fine-grained Perception of tools, materials, and actions; (2) Procedural
Understanding of step order and completeness; and (3) Scientific Reasoning that
connects the full experiment to its published conclusions. Our vision-centric
annotation pipeline, combining automated generation with multi-disciplinary
expert validation, ensures that tasks require visual grounding. We evaluate 19
leading MLLMs on ExpVid and find that while they excel at coarse-grained
recognition, they struggle with disambiguating fine details, tracking state
changes over time, and linking experimental procedures to scientific outcomes.
Our results reveal a notable performance gap between proprietary and
open-source models, particularly in high-order reasoning. ExpVid not only
provides a diagnostic tool but also charts a roadmap for developing MLLMs
capable of becoming trustworthy partners in scientific experimentation.