VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large
  Video Language Models
Authors: name: Pritam Sarkar, name: Ali Etemad

Despite recent advances in video understanding, the capabilities of Large
Video Language Models (LVLMs) to perform video-based causal reasoning remains
underexplored, largely due to the absence of relevant and dedicated benchmarks
for evaluating causal reasoning in visually grounded and goal-driven settings.
To fill this gap, we introduce a novel benchmark named Video-based long-form
Causal Reasoning (VCRBench). We create VCRBench using procedural videos of
simple everyday activities, where the steps are deliberately shuffled with each
clip capturing a key causal event, to test whether LVLMs can identify, reason
about, and correctly sequence the events needed to accomplish a specific goal.
Moreover, the benchmark is carefully designed to prevent LVLMs from exploiting
linguistic shortcuts, as seen in multiple-choice or binary QA formats, while
also avoiding the challenges associated with evaluating open-ended QA. Our
evaluation of state-of-the-art LVLMs on VCRBench suggests that these models
struggle with video-based long-form causal reasoning, primarily due to their
difficulty in modeling long-range causal dependencies directly from visual
observations. As a simple step toward enabling such capabilities, we propose
Recognition-Reasoning Decomposition (RRD), a modular approach that breaks
video-based causal reasoning into two sub-tasks of video recognition and causal
reasoning. Our experiments on VCRBench show that RRD significantly boosts
accuracy on VCRBench, with gains of up to 25.2%. Finally, our thorough analysis
reveals interesting insights, for instance, that LVLMs primarily rely on
language knowledge for complex video-based long-form causal reasoning tasks.