MSR-ViR: Modularized Self-reflected Video Reasoner for Video Question Answering

Authors: Zihan Song, Zi Qian, Xin Wang, Hong Chen, Yaofei Wu, Longtao Huang, Hui Xue', Wenwu Zhu
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Video Question Answering, Multimodal LLM, Modular Network, Self-reflected Training

Abstract:
Recently, multimodal large language models (multimodal LLMs) have been applied to a wide range of video understanding tasks, particularly for Video Question Answering (VideoQA). However, existing multimodal LLMs suffer from the following challenge: the classic end-to-end training strategies of multimodal LLMs for VideoQA tasks are black-box, thus lacking interpretability as they can neither present a reasoning path nor indicate where the answer is derived from the video. To tackle this challenge, we propose MSR-ViR (Modularized Self-Reflected Video Reasoner), a self-reflected framework that introduces a Modularized Spatial-Temporal Grounding (MoST-Grounding) module to multimodal LLMs for VideoQA tasks. MoST-Grounding utilizes a question parser LLM to generate execution policies, which serve as a reasoning path from questions to answers providing interpretability for our VideoQA framework. Based on the execution policies, MoST-Grounding invokes various small modules to localize temporal segments and spatial regions in videos which provide multimodal LLMs with most relevant visual information, while presenting visual evidence of our final answers. To avoid the question parser LLM generating unreasonable policies, we further propose a reinforcement learning-based Alternate Self-reflection training strategy to optimize the Multimodal LLM and the question parser LLM. Experiments on VideoQA datasets (NExT-QA and STAR) and grounded VideoQA dataset (NExT-GQA) demonstrate that our method significantly improves video understanding capabilities of multimodal LLMs, while providing interpretable reasoning paths together with temporal and spatial localization evidence within the video.

Decision: reject
Reviewer scores: 5, 3, 6, 6

Reviews and discussion:
<thread>
creation date:
1729416768401

thread_type:
official_review

<thread_object>
creator: reviewer_1xgn

<content>
summary:
This paper addresses the black-box problem of multimodal large language models (MLLMs) in VideoQA by proposing MSR-ViR, a novel framework. MSR-ViR introduces two core components: (1) the MoST-Grounding module, which localizes relevant temporal segments and spatial regions in videos, and (2) an Alternate Self-Reflection Training Strategy, which iteratively enhances the question parser and the MLLM. Evaluations on datasets such as NExT-QA, STAR, and NExT-GQA demonstrate that MSR-ViR achieves competitive performance on VideoQA tasks and improves interpretability by providing visual evidence for answers.

strengths:
1) The paper is well written and the motivation is clear, with a strong focus on the interpretability challenge in VideoQA, making it highly relevant to the field.
2) The MoST-Grounding module integrates multiple submodules to effectively localize temporal and spatial regions, improving transparency in the input to the MLLM.
3) The Alternate Self-Reflection strategy introduces a novel reinforcement-based method to align the question parser and the MLLM, enhancing performance and consistency.

weaknesses:
1) The framework is relatively heavy, relying on multiple external tools for tasks and additional operations such as resizing and truncation, which increases computational overhead. Moreover, what if these external tools are unreliable, which can lead to further exposure bias? It's necessary to further investigate the choice of the sub-modules in the MoST-Grounding module.
2) While the approach improves the selection of input information, it does not make the internal reasoning process of the MLLM more interpretable. It still focuses on the process of 'input' to decide which information should be fed into the MLLM as soft prompts. 
3) The paper misses references with related works such as SeViLa [1] and GCG [2], which also focus on VideoQA with grounding elements. Including these baselines would strengthen the empirical validation.

[1] Yu et al. "Self-Chained Image-Language Model for Video Localization and Question Answering", 2023 NIPS
[2] Wang et al. "Weakly Supervised Gaussian Contrastive Grounding with Large Multimodal Models for Video Question Answering", 2024 ACM MM

questions:
See Weaknesses
</content>
</thread_object>

</thread>
<thread>
creation date:
1730171354764

thread_type:
official_review

<thread_object>
creator: reviewer_p6df

<content>
summary:
This paper aims to improve the interpretability of Multimodal LLMs in performing VideoQA. To achieve the goal, the authors design a modular self-reflection framework MSR-ViR. The framework primarily comprises a spatial-temporal grounding module and a self-refection learning mechanism based on DPO. MSR-ViR basically decouples video grounding from videoqa, enabling the interpretaion of intermediate results to understand the answers. The experiments on related datasets have demonstrated the strength of the approach in both accuracy and interpretability (grounded accuracy).

strengths:
1.	Successfully develop a ground-then-answer framework for interpretable video question parsing. The question parser policy is able to be optimized via answer feedback. 

2.	The approach is well presented and easy to understand.

weaknesses:
1.	The paper does not improve video grounding but just uses existing method UniVTG. According to table 2, the grounding performance in terms of IoP@0.5 is worse than previous VLMs (VGT and TempCLIP). This severely limits the improvements of QA performance. 
2.	According to the model ablation results in Table 3,  the global representation g_v (which opens back door for grounded QA) seems more crucial than other components. Such results slightly depart from the major claim of interpretable VQA where correct answers are anchored on correct visual content.
3.	Should compare with SeViLA which also finetunes a localizer on QVHighlight (like UniVTG) for grounded VideoQA.

questions:
see weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730476065507

thread_type:
official_review

<thread_object>
creator: reviewer_ndrp

<content>
summary:
The paper addresses the use of multimodal Large Language Models in understanding tasks across various multimodal scenarios, specifically focusing on applications in Video Question Answering. However, current Multimodal Large Language Models are largely black-box systems for VideoQA tasks, lacking the ability to provide an understandable reasoning path and, thus, suffer from limited interpretability. To address this, the authors propose MSR-ViR, which constructs a modular reasoning structure designed to generate reasoning paths, and incorporates a reinforcement learning framework to prevent the model from generating unreasonable reasoning paths.

While the proposed approach is interesting, it relies on the integration of four existing models. This ensemble-based structure shows only marginal performance improvements (1-2%), and the manuscript does not discuss crucial aspects such as reasoning time costs or memory overhead.

strengths:
1. The MoST-Grounding module introduces two standard modules—temporal localizer and spatial localizer—which can be flexibly assembled in sequence based on the question parser. This structure is robust and allows for reliable generation of reasoning paths.

2. The authors present a clear motivation: to create a framework for generating reasoning paths for the black-box nature of VideoQA tasks. The comprehensive visualization of reasoning paths further demonstrates the effectiveness of the model.

weaknesses:
1. The question parser and LLM reasoning components both rely on LLM structures, leading to high computational costs.

2. Both the temporal localizer and spatial localizer use existing models, specifically UniVTG and YOLO-World, which contribute to significant parameter overhead. As the complexity of VideoQA tasks increases, relying on these two models alone may not only limit predictive accuracy but also compromise the completeness of reasoning paths. Future work may need to explore additional modules to support diverse combinations (see [1] for reference).

3. The ablation study lacks comprehensiveness. While the authors assess model performance on QA and grounding tasks and provide an effectiveness analysis of each module, they do not evaluate inference speed, parameter count, or other metrics compared to end-to-end models. Given that the proposed framework integrates multiple existing large models, an analysis of inference speed is both important and currently missing.

[1]. Neural Module Networks

questions:
To generate reasoning paths for VideoQA, would it be more effective to design a Chain-of-Thought dataset and perform supervised fine-tuning (SFT)? The O1 model currently adopts this approach, achieving clear reasoning paths through an end-to-end structure.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732032927706

thread_type:
official_review

<thread_object>
creator: reviewer_gpww

<content>
summary:
This paper presents MSR-ViR (Modularized Self-Reflected Video Reasoner), a framework for Video Question Answering (VideoQA) that enhances interpretability by integrating multimodal large language models (LLMs) with spatial-temporal grounding and self-reflective training. Traditional multimodal LLMs struggle with interpretability, as they operate as black-box systems without revealing the reasoning process or the video segments informing their answers. MSR-ViR addresses this by using a MoST-Grounding module to localize relevant video segments and spatial regions based on policies generated by a question-parsing LLM, creating a clear reasoning path. To refine this process, the framework employs an Alternate Self-reflection Training Strategy, which jointly optimizes the multimodal LLM and the question parser through reinforcement learning, enabling mutual refinement based on feedback. Evaluations on popular VideoQA datasets (NExT-QA, STAR, and NExT-GQA) show that MSR-ViR surpasses traditional and grounding-based methods, demonstrating improved accuracy and the ability to localize relevant video segments, thereby providing visually grounded evidence for its answers.

strengths:
The paper presents several strengths in addressing Video Question Answering (VideoQA) tasks. Primarily, it enhances the interpretability of multimodal large language models (LLMs), which traditionally function as black-box systems. This framework, with its MoST-Grounding module, identifies relevant video segments and spatial regions, aligning them with text inputs to support answer derivation. The method also uses reinforcement learning to train the multimodal LLM and question parser LLM in tandem, represents an innovative approach that bolsters model transparency and clarity. 

The paper summarized grounded and modular videoQA methods, which aim to answer questions while identifying relevant regions in the video, have seen notable advancements through the integration of multimodal large language models (LLMs). 

The paper’s extensive experimental validation on datasets like NExT-QA and STAR showcases the method’s superior performance and its ability to provide visually-grounded evidence, setting it apart from existing models. This combination of interpretability, strategic training, and improved accuracy underscores the paper’s significant contributions to advancing VideoQA methodologies.

weaknesses:
The use of reinforcement learning (RL) in the paper, while different in facilitating the Alternate Self-reflection Training Strategy, has some limitations concerning novelty and implementation. RL is well-established for optimizing policies in non-differentiable tasks, and its application to train LLMs collaboratively is not entirely unprecedented, as other multimodal and modular frameworks have explored similar strategies. Additionally, the reinforcement learning process depends heavily on the quality of intermediate feedback provided by the multimodal LLM, which may propagate errors if the initial predictions are suboptimal. Furthermore, RL’s computational overhead and potential convergence issues in complex scenarios are not fully addressed, leaving questions about its scalability for larger datasets or more intricate VideoQA tasks. 

The paper claims the method is able to deal with complex tasks. Will you consider to add or discuss several recent video question answering tasks for comprehensive evaluations? (e.g., SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge,  Complex-TV-QA: A Study of Situational Reasoning for Traffic Understanding, etc.)

The experimental improvements reported in the paper, while demonstrating the effectiveness of the proposed method, appear limited when compared to other state-of-the-art methods, as highlighted in Table 2. Although the MSR-ViR framework outperforms baseline and grounding-based approaches, the margin of improvement is relatively modest, raising concerns about the practical significance of the gains. 

A concern regarding the paper is the absence of clear information about the availability of open-source code for review. It is crucial for verifying the implementation details, replicating experimental results, and evaluating the broader applicability of the proposed methods.

questions:
NA
</content>
</thread_object>

</thread>
<thread>
creation date:
1732609404386

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Explanation of the Paper Revisions

comment:
Thank all the reviewers for taking the time to review our paper and providing valuable and constructive feedback. Based on the reviewers' suggestions, we have made several modifications and additions to the paper. The main revisions are as follows:

1. **Table 1**: We have added SeViLa and GCG as grounding-based baselines, along with the experimental results of our Llava-Next version of MSR-ViR.
2. **Table 2**: Results of SeViLa and our Llava-Next version of MSR-ViR have been added.
3. Based on the revision of Table 1 and Table 2, adjustments have been made to the experiments setup descriptions in Section 4.1, as well as the descriptions of the experimental results in Sections 4.2 and 4.3.
4. **Appendix**: A new Section A.6 has been added, which includes experimental results on model parameters and inference speed, as well as an ablation study on the selection of temporal grounding models.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes a modular system for video quesiton-answering (VideoQA). The motivation for a modular framework is that it can be both more intepretable (by providing a reasoning trace) and accurate than a single-black box model. The authors employ a parsing LLM to generate programs from the input question, separate temporal- and spatial-grounding modules, and an answering LLM as well. The authors use DPO to train the parsing LLM, as an alternative to few-shot prompting of the LLM as used in prior works such as MoreVQA.

Reviewers appreciated the motivation of the approach, and the reasoning traces that can be produced by the system. Concerns were that the proposed approaches substantially increases the computaitonal cost (it takes about 6x longer to answer a quesiton), and the accuracy improvements do not justify the computational cost of the approach. Reviewer gPWW also pointed out that the authors have not evaluated their approach on more complex Video QA datasets where reasoning traces could be more useful.

The AC agrees that the experimental evaluation of the paper is weak: the datasets used by the paper (Next-QA, Next-GQA and STAR) are all outdated and created before LLMs started being widely used. And there are also recent VideoQA datasets which contain substantially longer and complex videos (such as Video-MME (which the authors cited in the rebuttal) and LVBench) which would be more suitable as they are more challenging, require more reasoning and grounding, and are not saturated like Next-QA, as they were created with (M)LLMs in mind. And although the authors have mentioned in their paper and rebuttal that a focus of the paper is improving interpretability of VideoQA models, the experiments in the paper mostly focus on accuracy. The ablations in the paper (Table 3; top part) only consider accuracy too.

The final decision is therefore to reject this paper. Authors are encouraged to improve the experimental evaluation, and to resubmit a revised version of this paper to a subsequent conference.