Divide and Conquer: Exploring Language-centric Tree Reasoning for Video Question-Answering
Authors: Zhaohe Liao
Jiangtong Li
Siyu Sun
Qingyang Liu
Fengshun Xiao
Tianjiao Li
Qiang Zhang
Guang Chen
Li Niu
Changjun Jiang
Liqing Zhang
Venue: ICML 2025 poster (2025)
Keywords: video question-answering
multi-modal large language model
Primary area: applications->computer_vision
TL;DR: We propose a Language-centric Tree Reasoning (LTR) framework that performs first-order reasoning on a structured logical tree for traceable, language-driven reasoning and enhances the reasoning transparency and accuracy in Video Question-Answering

Abstract: Video Question-Answering (VideoQA) remains challenging in achieving advanced cognitive reasoning due to the uncontrollable and opaque reasoning processes in existing Multimodal Large Language Models (MLLMs). To address this issue, we propose a novel Language-centric Tree Reasoning (LTR) framework that targets on enhancing the reasoning ability of models. In detail, it recursively divides the original question into logically manageable parts and conquers them piece by piece, enhancing the reasoning capabilities and interpretability of existing MLLMs. Specifically, in the first stage, the LTR focuses on language to recursively generate a language-centric logical tree, which gradually breaks down the complex cognitive question into simple perceptual ones and plans the reasoning path through a RAG-based few-shot approach. In the second stage, with the aid of video content, the LTR performs bottom-up logical reasoning within the tree to derive the final answer along with the traceable reasoning path. Experiments across 11 VideoQA benchmarks demonstrate that our LTR framework significantly improves both accuracy and interpretability compared to state-of-the-art MLLMs. To our knowledge, this is the first work to implement a language-centric logical tree to guide MLLM reasoning in VideoQA, paving the way for language-centric video understanding from perception to cognition.

Divide and Conquer: Exploring Language-centric Tree Reasoning for Video
Question-Asserring
Zhaohe Liao 14
Jiangtong Li 23 Siyu Sun
Qingyang Liu'
Fengshun Xiao Tianjiao Li* Qiang Zhang*
Guang Chen Li Niu Changjun Jiang?3 Liqing Zhang!
Abstract
1. Introduction
Video Orestion-Answerte (VideoQA) remains challenging in achieving advanced cognitive rea- soning due to the uncontrollable and opaque rea- soning processes in existing Multimodal Large Language Models (MLLMs). To address this is- sue, we propose a novel Language-centir centric Tree Reasoning (LTR) framework that targets on en- hancing the reasoning ability of models. In detail, it recursively divides the original question into logically manageable parts and conquers them piece by piece, enhancing the reasoning capa- bilities and interpretability of existing MLLMs. Specifically, in the first stage, the LTR focuses on language to recursively generate a language- centric logical tree, which gradually breaks down the complex cognitive question into simple per- ceptual ones and plans the reasoning path through RAG-based few-shot approach. In the second stage, with the aid of video content, the LTR per- forms bottom-up logical reasoning within the tree to derive the final answer along with the traceable reasoning path. Experiments across 11 VideoQA benchmarks demonstrate that our LTR framework significantly improves both accuracy and inter- pretability compared to state-of-the-aar MLLMs. To our knowledge, this is the first work to im- plement a language-centric logical tree to guide MLLM reasoning in VideoQA, paving the way for language-centric video understanding from perception cognition.
Video Question-Answering (VideoQA) has emerged significant research area with applications in multi-modal understanding, interactive artificial intelligence and cogni- tive science (Xiao et al., 2021; Li et al., 2022a; Wu et al., 2023; Li et al., 2023b; Mangalam et al., 2023; Li et al., 2024b; Chen et al., 2022: Wang et al., 2025; Chi et al.,
2024a: Li et al., 2025; Chi al., 2024b). The core chal- lenge lies in advancing language-centric video understand- ing from perception to cognition. Specifically, low-level perception involves understanding the spatiotemporal fea- tures of videos, such as recognizing objects, actions, and scenes. High-level cognition, on the other hand, requires comprehending the underlying logic of both the video con- tent and the posed questions, enabling the system to perform reasoning along logical structure and provide accurate an- swers. To address these challenges, recent advances extend Large Language Models (LLMs) to their multimodal vari- ants (MLLMs), such as Video-LLaMA (Zhang et al., 2023; Cheng et al., 2024) and Video-LLaVe (Lin et al., 2024) by integrating visual and textual information.
However, while they can provide certain explanations when answering questions, how achieve System-2 reasoning has not been fully explored. One major limitation is that the reasoning processes of these models are often uncon- trollable and lack of transparency. Such opacity makes it challenging to analyze their reasoning steps, rendering the results to be less trustworthy. For instance, when address- ing complex questions involving multiple temporal visual cues, the models may produce incorrect answers without revealing the reasoning path that led to those conclusions. Therefore, as users cannot trace back to pinpoint where the reasoning have gone astray, they cannot trust the results.
"quual contribution, MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai China: Key Labo- ratory of Embedded System and Service Computing, Ministry of Education, Shanghai, China: 3 School of Computer Science and Technology. Tongji University, Shanghai, China; Bilibili Inc, Shanghai, China. Correspondence to: Liqing Zhang <lq- zhang @aang@@ss.ee sjtu.edu.cn>.
To overcome such issues, we propose a novel training-free free, model-agnostie Language-centrie Tree Reasoning (LTR) framework that enhances model reasoning capabilities while improving the interpretability and verifiability of reasoning processes. Our framework utilizes language the central driving force for video understanding, starting from the log- ical structure inherent in the question itself. Initially, by integrating video content, we recursively generate a com-
Proceedings of the 4nnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Divide and Conquer: Exploring Langugge-centric Tree Reasoning for Video Question-Answering
Is safi for the man weating shirt to sil the roof of the car?
reasoning framework is designed on the language-centric logical tree, improving the verifiability and facilitating fur- ther errors analysis. As another approach, DSTN exploits Neural Modular Networks (NMNs) to generate a program and obtains the final answer through program execution. Although this approach enhances verifiability, it lacks er- ror tolerance, i.e., any mistake in the program invariably leads to an unrecoverable incorrect answer. In contrast, our framework offers a soft reasoning architecture, supple- menting logical reasoning with video information, ensuring explainable reasoning while increasing error tolerance.
Scage=1:
Dividde with Dop-down Recuseive ve Checking
Is safe for the man not wearing a shirt to on roof of the car
What the condition of the roof of the car?
What is the condition of the car?
the roof of the car sturdy?
Is the roof of the car
the car moving fast?
Is the car moving
flat?
left?
Stagu-2:
Conquex with Botcon-Ut Upp 7zee Besaoning
To validate the effectiveness of our LTR framework, we se- lect four existing MLLMs, i.e. VideoLLaMA3 (Zhang et al., 2025), VideoChat2 (Li et al., 2024b), Qwen2-VI. (Wang et al., 2024), and LLa VA-One Vision (Li al., 2024a) as baselines for comparison, and conduct experiments on 11 VideoQA benchmarks. Additionally, we perform ablation studies to analyze the effectiveness of each component within our framework and provide insights into the strengths and weaknesses of existing models based on our framework. Moreover, we provide case studies to demonstrate how our framework enables more error-tolerant, explainable, and controllable System-2 reasoning while enhancing prediction accuracy. Our contribution can be summarized as:
The roof condition is satisfled, but the car in jolt is still dangerous sit on the roof.
The roof's is both sturdy and flar.
The moving slowlly, but the road jolt.
Yes, the roof is stundy:
Yes, the is flat.
No, the car only
Yes, the moving towards the left.
slawly.
Figure 1. The example about the reasoning process of our LTR framework. In the first stage, LTR recursively divide the question into language-centrie logical tree. In the second stage, LTR con- quer the main question as bottom-up tree reasoning. Note that due
the assistance from video content, some flaw in logical tree can also be fixed during bottom-up tree reasoning.
prehensive language-centric logical tree from the given question. This tree has two signatures: first, its hierarchical structure explicitly represents the reasoning logic of the original question; second, the leaf nodes comprise simple perceptual questions that enable effective leveraging of ex- isting MLLMs' perceptual strengths. To generate such a logical tree, the MILM recursively divides the questions into simpler and logically reasonable sub-questions and de- cides whether the sub-questions are simple enough to be Icaf perceptual questions. Subsequently, to perform multimodal System-2 reasoning with the generated cenuuag--enttic log- ical tree, we begin by employing MILMS to answer all leaf node questions, establishing an evidentiary foundation for the bottom-up reasoning process. Then, aided by video con- tent, we perform logical bottom-up reasoning recursively within the tree, deriving node answers from the responses of their children while verifying consistency with visual evidence, ultimately obtaining both the original question's 's solution and a fully traceable reasoning path.
Motivation: We analyze the two main development stages of VideoQA, i.e., perception and cogmition, to explore a language-centric tree reasoning framework that achieves cognitive-level video understanding.
Framework: We propose novel training-free model- agnostic Language-centric Tree Reasoning (LTR) framework that utilizes language as the central driving force for video understanding, enhancing reasoning ability and interpretability of MLLMs.
Experiments: Extensive experiments on 11 bench- marks against baseline MLLMs demonstrate that the LTR framework improves reasoning accuracy, leading to more transparent and verifiable VideoQA system.
2. Related Work
2.1. Video Question-Answering Dataset
Some previous works have also explored interpretable VideoQA method, such as VoT (Fei et al., 2024a) and DSTN (Qian et al., 2022). VoT constructs a cognitive-level reasoning framework by providing detailed analysis at the object and action levels, followed by reasoning based on fine-grained video representations. While VoT enhances MLLM performance and offers additional reasoning cues, it still falls short in capturing the logical structure of questions and fully elucidating the reasoning process. Unlike VoT, our
By incorporating the temporal dimension, VideoQA natu- rally extends ImageQA, enabling the ability to answer ques- tions with dynamic visual content. In recent years, VideoQA datasets have evolved to encompass more complex tasks that require advanced reasoning skills, such as temporal reasoning (Choi et al., 2021), physical reasoning (Yi et al., 2020), evidence reasoning (Xiao et al., 2021; Wu et al., 2023), commonsense reasoning (Li et al., 2022a: 2023b), and long video understanding (Mangalam et al., 2023). To
2

---

Divide and Conquer: Exploring Language-centric Tree Reasoning for Video Question-Answering
enhance fine-grained video understanding, AGQA (Grunde- McL.aughlin et al., 2021) and AGQA-Decomp (Gandhi et al., 2022) utilize spatiotemporal scene graphs from Ac- tion Genome (Ji et al., 2020) to construct VideoQA datasets. With the recent advancements of MLLMs in video un- derstanding, test-only benchmarks such as MVBench (Li et al., 2024b), MMTTBench (Ying et al., 2024), and Video- MME (Fu et al., 2024) have been proposed to comprehen- sively evaluate MILM performance. However, the rea- soning logic in answer prediction remains under-explored. Our framework leverages the zero-shot reasoning ability of MLLMs to generate high-quality fine-grained logical trees when answering questions, reliefing the constraint caused by the absence of such datasets.
and better computational efficiency.
Despite these advancements, existing MLLMs still provide limited explanations for their answering process. To address these challenges, we propose the LTR framework which ex- plores language-centric logical tree reasoning in VideoQA, thereby enhancing the reasoning transparency.
2.3. Visual Reasoning
In the field of visual reasoning, the effectiveness of divid- ing complex questions into simpler sub-questions has been observed across various tasks, including ImageQA (Cao et al., 2018) and DocumentQA (Yang et al., 2018). In ear- lier research, most benchmarks (Hudson & Manning, 2019; Grunde-MLLaughlin et al., 2021) break down questions into modular programs defined within Neural Modular Networks (NMNs) (Qian et al., 2022) to generate answers. Among these, AGQA (Grunde-MLLughlin et al., 2021) introduces spatio-temporal scene graphs to represent reasoning pro- grams for VidcoQA. However, these reasoning programs from AGQA (Grunde-McLLuughli directly utilized by existing VideoQA methods. To address this limitation, AGQA- (Gandhi et al., 2022) trans- forms each reasoning program into multiple sub-questions and a compositional graph to evaluate the compositional consistency of VideoQA methods. Neural Modular Net- works (NMNs) have been extensively explored in both Im- ageQA (Andreas et al., 2016; Hu et al., 2017; Johnson et al., 2017; Mascharka et al., 2018) and VideoQA (Qian et al., 2022) for performing compositional reasoning in visual question answering. Chen et al. (2024) introduces general and universial meta-points for object represent, which effec- tively facilitate the reasoning on object interaction. These networks convert questions into modular sub- uub-programs and assemble these modules to execute the overall reasoning pro- cess. Although NMN- approaches offer interpretabil- ity, they rely on predefined modules, limiting the compati- bility with novel modules and reasoning frameworks.
2.2. Video Question-Annserring Methodology
Prior to the incorporation of Large Language Models (LLMs) in VideoQA, the architectures of VideoQA meth- ods primarily focused on aligning videos with questions. In early studies, this video-question alignment is achieved through the use of cross-modal attention (Li et al., 2019; Gao et al., 2019) or memory networks (Gao et al., 2018; Fan et al., 2019). Subsequently, graph reasoning (Jiang & Han, 2020; Park et al., 2021; Liu et al., 2021; Gu et al., 2021;
al., 2021) cannot be
Cherian et al., 2022) and hierarchical reasoning (Le et al., 2020; Guo et al., 2021; Xiao et al., 2022a; Peng et al., 2022: Xiao et al., 2022b) approaches gained popularity. Some research has explored VideoQA from causality perspec- tive. For example, IGV (Li et al., 2022c), EIGV (Li et al., 2022b), and TIGV (Li et al., 2024d) focus on distinguish- ing causal and environmental clips in video by utilizing a simple grounding indicator and promoting sensitivity to se- mantic changes in causal scenes. Furthermore, KPI (Li et al., 2023a) investigates front-door interventions knowledge proxy to mitigate the effects of dataset biases.
With the advancement of Large Language Models (LLMs), Video-MLLMs generally comprise pre-trained visual en- coder (Radford et al., 2021; Caron et al., 2021; Fei et al., 2024b) that encodes video frames into low-dimensional vi- sual features, a vision- langua- aligner to ensure that these visual representations are comprehensible to LLMs, and language decoder instruction-tune on multimodal data (Cai et al., 2024; Dubey et al., 2024) that generates text responses based on provided instructions and video content. Numer- ous efforts have been dedicated to designing effective visual alignment strategies. For example, Video-ChatGPT (Maaz et al., 2023) employs spatial and temporal pooling to in- dependently caption information. VideoLLaMA2 (Cheng al., 2024) presents the Spatial-Temppraa Convolution Con- nector to more efficiently capture spatial-temporal features with a manageable number of visual tokens. Furthermore, Video-LLaVA (Lin et al., 2024) aligns video and image data prior to projection, achieving unified visual representation
In the era of MLLMs, VoT (Fei et al., 2024a) endeavors to establish transparent VideoQA reasoning by offering de- tailed analyses of videos at the object and action levels and conducting reasoning based on fine-grained video represen- tations. However, it still fails to capture the logical structure of questions or fully elucidate the reasoning process. Logic- CheckGPT (Wu et al., 2024) addresses object hallucination and enhances visual understanding ability of MILMS by checking conflict consistency across multiple interrelated questions. However, this method neither explores hierarchi- cal question structures spanning cognitive reasoning to per- ceptual processing nor provides traceable reasoning paths. Therefore, we introduce the LTR framework, which first re- cursively generates a language-centre logical tree and then performs disual-assister reasoning along this logical tree. In
3

---

Divide and Conquer: Exploring Language-centrie Tree Reasoning for Video Question-Answering
Input
Output
Q: What is the person touching between opening closet and taking a box?
Based on the quesion and the Fell the lagunge-ommiii lugical mre is recursively genernted as Ar (in ISON format). The answer for all leaf questions are [I the QueeioonAnswer for All Leaf Question]] Affer that, perform logical reasoning along the tree. (List the Reasoning Provess for All Non leaf Questions). the orggnaa question "Deor"
Iherefore, the
Stage-1: 1: Divide with Top-down Recursive Checking
Stage-2: Conquer with Bottom-Up Tree Reasoning
to AdgA
taking scnnevise
keaching
Gii and lepe srscare Repine
(Shat the bouching betwens
and taking bostl undaple mac poropiad caoiimn?
lopel and voka oheece.
with
Questonl: that percee loodiing Qaeatond: Wust the peasos coanking secure beac
olsent?
What person soadhiu ofter opreing vieees
What persos soadieu after alouss rlous
Like coseertooo resuriig theefon Daey an Yearee*
Process Ansver Verification
Question Division
What persus wadhing ofler opening a closet
Goe IN lepan SEEEEE planchdy swatch siter
clooutt).
Band (suoquastion, Hy Incheueation. 21
Is the louching mwikh after operinng cleec?
we finlt. Mi, meeeee
Life Baadie affen
Like pleneioe locel bay bndiuse Saltiing scacl pecceptile?
unaiing dear afor opening closet
sppecciing ci parapellle?
Video Aided Logical Reasoning
No. incolres
quodion aeshes ruripde dlpectsee attion rool si Luther doaked.
Giric Mi sane [Doss
perzon wclats)
realiipee aigers ond auiion and sacdd be father draded.
Docn perw cssit
Das lotch doer?
Joes pinson cesit
Dop teech pardeh
Recursive Perceptual Checking
You fexlu sand is vidao and senching
Longusge-centrn Logical
Perceptual Leaf Question Answering
Figure 2. The illustration of Language-centric Tree Reasoning (LTR) framework. In the first stage, our LTR recursively divides complex cognitive questions to simpler question until they are perceptual questions. In the second stage, our LTR answers the perceptual Icaf questions and bottom-up reasoning toward the original question alone the language-centric logical tree. Best viewed when zoomed in.
this way, LTR method enhances both reasoning capabilities and interpretability, advancing towards System-2 reasoning.
able division approaches and ambiguity in determining perceptual simplicity. Therefore, we introduce retrieval- augmented generation (RAG)--asisted) question division. In detail, we the AGQA-Decomp (Gandhi al., 2022) dataset the retrieval database and embed all non-leaf question-subbuestionn tuple in it. Specifically, for each non-leaf question q E Q (where Q denotes the com- plete non-leaf question set), we encode it into a 256- dimensional feature vector fg E R255 using OpenAl's seeet--mmedding--g 3-large model, with this vector serving as the retrieval key. During the RAG process, we retrieve the top- most relevant question guestion-subquustiin subquestion tu- ples from AGQA-Decomp by computing cosine similarity between feature vectors. These retrieved tuples then provide few-shot exemplars for subquestion generation. To facilitate this process, we prompt the MLLM with:
3. LTR Framework
3.1. Divide with Top-down Recursive Checking
In this stage, our LTR recursively breaks down the cognitive question into perceptual ones to construct the language- centric logical tree with two steps. The first step divides non- perceptual questions into sub-questions, and the second step recursively checks whether the sub-questions are perceptual to determine the stop of question division.
3.1.1. QUESTION DIVISION
The Question Division step is motivated by the need to pro- gressively divide complex cognitive questions into simpler perceptual ones. However, directly dividing such questions based solely on textual content proves challenging due to their inherent reliance on comprchensive video understand- ing. To address this difficulty, this step leverages video To address this, the step strategically utilizes visual evidence to guide hicrarchical division. Specifically, given question and its corresponding video input, the MLLM recursively divides the original question into logically structured sub- questions that facilitate stepwise reasoning. Through this recursive process, LTR reduces cognitive complexity by transforming abstract queries into concrete perceptual tasks.
Instruction: Based the content of the given video, please help me to divide the given question into multiple more perceptual questions.
Sample: The similar samples for the question division are: {retrieval results in JSONJ.
Rule: The output should be formed in JSON format,
with SubQu; (sub-question) } as the key and value. Question: (original question).
Video:
1.2. RECURSIVE PERCEPTUAL CHECKING
However, directly asking the model to perform question division may introduce uncertainty, such as multiple vi-
The Recursive Perceptual Checking step is used to deter- mine whether a given question can be directly answered
4

---

Divide and Conquer: Exploring Language- Tree Reasoning for Video Question-Answering
with the perceptual capabilities of MLLMs. However, the feasibility of directly answering a certain question may vary as the video content changes. For example, the question "Is it safe to drive ahead?" can be directly answered if the video depicts an empty road, as the assessment of safety is straight- forward. In contrast, if the video contains a complex urban environment with multiple obstacles and dynamic elements, we need divide the question into several sub-questions focusing on specific objects and actions. Each sub-question would then be individually addressed to collectively form comprehensive answer. Therefore, we input both the ques- tion and the video into the MLLM. The MLLM analyzes the combined inputs to judge whether, for the video, the ques- tion qualifies as a perceptual query that can be answered directly. The perceptual questions will not be further di- vided. When all leaf questions are determined as perceptual, the top-down recursive checking stops, and the language- centric logical tree is generated accordingly. To facilitate this process, we prompt the MLLM with:
3.2.2. VIDEO-AIDED LOGICAL REASONING
The Video-aided Logical Reasoning step is designed infer the answer to parent question within the language-centri logical tree by utilizing the answers to its sub-questions, with the assistance of video information. The motivation behind this step stems from recognizing that potential vague may oceur during the question decomposition process, and the answers to sub-questions may also contain inaccura- cies. Therefore, relying solely on linguistic information for logical reasoning might not yield reliable results. By incor- porating video content, we enhance the reasoning process, allowing the MLLM to consider visual cues that may not have been explicitly addressed in the sub- uub-questions. For instance, as illustrated in Figure 1, even though the sub- questions do not include inquiries about the stability of the road, the MILM still accounts for road bumps when answer- ing about the "condition of the car". In practice, we input the video, the parent question, and the sub-questions along with their answers into MLLM to perform video-aided logi- cal reasoming. This approach ensures that the reasoning is grounded in both logic and video, mitigating the impact of potential errors in logical tree or sub-question answers. To facilitate this process, we prompt the MILLM with:
Instruction: Based on the given video, is the given question perceptible the video? The output should be "Yes/No Reason" (original question].
Rule:
Question:
Video:
{ (oideo_tokens}
Instruction: Given the video and the logical structure, please help me to derive the answer the main question. Rule: The output should first provide explanation from both logic and video, and then provide the final answer. Logical Structure: sir.illssssuctuee.nnSSSYY .ormatt. Video: {video tokens}
3.2. Conquer with Bottom-up Tree Reasoning
In the second stage, LTR performs bottom-up tree reason- ing through three steps to recursively derive a logical and visual coherent answer. In the first step, LTR answers all perceptual leaf questions based on video content. In the second and third steps, aided by video content, LTR reasons along the first-order logical structures and verifies answers, ensuring visual and logical coherent of answers.
3.2.3. IN-PROCESS ANSWER VERIFICATION
The In-process Answer Verification step is designed to en- sure the reliability of intermediate answers during the rea- soning process, which involves logical reasoning across multiple intermediate nodes. Before progressing further, it is essential to verify these intermediate answers to re- duce the propagation of errors. Specifically, our verification focuses two key aspects: (1) whether the answers and reasoning processes of intermediate node questions con- form to the logical reasoning from sub-questions to parent questions, and (2) whether the answers to intermediate node questions conflict with the video content itself. The first aspect emphasizes cognitive-level verification, assessing the logical consistency and coherence of the reasoning pro- cess. The second aspect concentrates on perceptual-level verification, evaluating the answers against the basic per- ceptual information derived from the video content. By integrating both cognitive and perceptual validations, we effectively ensure the accuracy and rationality of the final answers. Notably, when this step detects any cognitive or perceptual conflicts in the intermediate node answers, we
3.2.1. 1. PERCEPTUAL QUESTION ANSWERING
The Perceptual Question Answering step aims to address the leaf questions of the language-centric logical tree, thereby enabling further reasoning along the tree structure. Note that these leaf questions are simple enough be perceptible from the video, as depicted in Section 1.2. Therefore, they can be effectively solved by leveraging the strong perceptual capabilities of existing MLLMs. In this step, we adopt the conventional instruction used in MILMs for VideoQA. Specifically, we input the question and the video into the MLLM to generate the corresponding answer. To facilitate this operation, we prompt the MLLM with:
Instruction: Given the question and video, answer the question using several words or phrase.
Question: (original.qucction)
Video: (video. tokens) {video.tokens)
5

---

Divide and Conquer: Exploring Language-centric Tree Reasoning for Video Question-Answering
incorporate the conflicts as inputs and re-execute the Video- aided Logical Reasoning step to re-solve them. To facilitate this verification process, we prompt the MLLM with:
consistency, we utilize the DAG from the AGQA-Decomy test set for bottom-up tree reasoning. Regarding accuracy improvement, we observe more pronounced gains in sub- question compared to main questions. This is attributed to the relative simplicity of sub-questions, which facilitates more effective reasoning. Furthermore, the improvement in c is much larger than that in accuracy. This improvement is attributed to the Video-aided Logical Reasoning module, which exploits the logical relationships within the structure, enabeling the QA inforamtion in perceptual questions to propagate along the tree and help the model answer more cognitive questions, therefore enhances the compositional consistency between main and sub questions.
Given the video and the logical structure, help me to verify whether the [Reason and Answer] for the main question is logical and visual coherent.
Instruction:
Rule: The output should first evaluate the logical and visual coherency, and then judge the correctness. Logical Structure: struccrtt structure JSON format).
Video: {video.tokens}
4. Experiment
4.4. Comparisons on Zero-Shot Performance
4.1. Evaluation Protocols
In Tables 2 to 4, we present zero-shot performance compar- isons on three benchmarks: MVBench, with additional comparisons on other bench- marks provided in the Appendix. Overall, our framework significantly outperforms the baselines, which is attributed to the collaborative Divide with Top-down Recursive Check- ing and Conquer with Bottom-up Tree Reasoning stages.
We evaluate the LTR framework on 11 VideoQA bench- marks, including MSVD-QA (Xu et al., 2016), MSRVTT- QA (Xu et al., 2016), TGIF-QA (Jang et al., 2017), ActivityNet-QA (Yu et al., 2019), AGQA-Decomp (Gandhi et al., 2022), NEXT-QA (Xiao et al., 2021), Causal- VidQA (Li et al., 2022a), STAR (Wu et al., 2023), Ego- Schema (Mangalam et al., 2023), Video-MME (Fu et al., 2024), and MVBench (Li al., 2024b). For open-ended settings, we use GPT-3.5 to assess the generated responses, utilizing both accuracy and scoring metrics. For multiple- choice settings, we use the corresponding MLLM to select answers from the provided options based solely on the ques- tion and generated response. To illustrate improvements in compositional consistency, we evaluate compositional met- rics (cR, cP, e-F1) from VA (Liao et al., 2024) on AGQA- Decomp. Full experiment results are in the Appendix.
VidQA, NeXT-QA and
When comparing performance between simple perceptual grounding tasks and complex cognitive reasoning tasks, we find that LTR demonstrates larger improvements on rela- tively complex cognitive reasoning tasks. For example, Ta- ble shows more significant improvements on counterfac- tual and prediction tasks (between 2.4% and 4.2%) than on explanation and description tasks (between 0.9% and 2. 1%), as the former require more complex logical reason- ing capabilities that are systematically facilitated by our language-centric tree reasoning procedure.
4.2. Implementation Details
Specifically, the Divide with Top-down Recursive Check- ing stage guides MILMS to extract necessary perceptual information for complex reasoning, while the Conquer with Bottom-Uj Up Tree Reasoning stage gradually aggregates per- ceived visual clues through recursive logical reasoning to de- duce answers through step-by-step complex reasoning. The combination of these two stages (i.e., LTR framework) there- fore enhances complex reasoning capabilities of MLLMs while maintaining traceable reasoning processes.
In our LTR framework, we utilize four different MLLMs: VideoLLaMM3 (Zhang al., 2025), VideoChat2 (Li al., 2024b), Qwen2-VL (Wang et al., 2024), and LLaVA- One (Li et al., 2024a). We set the video resolution to 336x336 pixels and uniformly sample 16 frames from each video. The maximun new generated length is restricted to 2048 tokens. Other settings follow the recommended settings of zero-shot generations for each baseline model.
Similar patterns are also observed in MVBench (Table 2) and NeXT-QA (Table 4). Specifically, Table 2 reveals that improvements on geasoning-intensiin tasks (i.e., counter- factual inference (CI), episodic reasoning (ER), and action prediction (AP)) are more substantial than those on simple perceptual tasks (i.e., object existence (OE), action count (AC), and fine-grained pose (FP)). Furthermore, Table 4 demonstrates more significant improvements on causal and temporal questions compared to descriptive questions.
4.3. Comparisons on AGQA-Decomp
In Table 1, we compare the performance of LTR with 9 baseline methods on AGQA-Decomp. The columns marked with "main" and "sub" represents corresponding metries computed on root question and none-root questions in the lantuage-centric trees respectively. The experimental re- sults indicate that our framework significantly outperforms the baselines in terms of accuracy, score, and compositional consistency, which can be attributed to the collaborative compositional reasoning strategy. To evaluate compositional
These observations collectively confirm that our LTR frame-
6

---

Divide and Conquer: Exploring Language-centric Tree Reasoning for Video Question-Answering
Acc.
Score
Compositional Consistency
Method
main
sub
main
sub
cP
cR
cF1
Video-LLaVA (Zhang al., 2023)
57.5
65.0
2.7
3.3
58.7
59.6
59.1
LLaMA-VID (Li et al., 2024c)
58.2
63.2
2.8
3.2
56.1
58.8
57.4
Chat-UniVi (Jin et al., 2024a)
60.3
68.1
3.0
3.4
63.6
69.9
66.6
VideoChat (Li et al., 2023c)
56.4
63.7
2.7
3.3
57.4
56.7
57.1
MiniGPT4-Video (Ataallah al.. 2024)
54.5
61.8
2.4
2.1
55.6
52.8
54.1
VideoChat2 (Li et al., 2024b)
60.5
69.2
3.1
3.5
67.5
66.5
67.0
LTR
64.3+3.8
7
3.4+0.3
4.1
77.2+9.7
77.7+1 1.2
17.4+10.4
VideoLLaMA3 (Zhang et al., 2025)
69.2
76.7
3.7
4.1
72.2
69.7
69.4
+ LTR
72.7+3.5
82.7+6.0
4.0+0.3
4.5+0.4
80.4+8.2
82.1+12.4
81.2+11.8
LLavA-OneVision (Li et al., 2024a)
64.3
73.2
3.5
3,9
69.1
66.9
68.0
+ LTR
67.8+3.5
79.5+6.3
3.7+0.2
4.3+0.4
78.3+9.2
80.6+13.7
79.4+11.4
Qwen2-VL (Wang et al., 2024)
65.4
73.8
3.5
4.0
68.2
66.2
67.2
LTR
69.1
80.7+6.9
3.8+0.3
4.4+0.4
78.6+10.4
80.7+14.5
79.6+12.4
Table 1. Performance on AGQA-Decomp regarding the accuracy, score, and compositional consistency. The columns marked with "main" and "sub" represents corresponding metrics computed on root question and none-root questions in the language-centric trees respectively. The results in the blue reproduced by using their published model weights and instructions.
Method
Avg AS AP AA
FA
UA OE OI OS MD AL
ST AC MC MA
SC
FP
CO
EN
ER CI
Video-LLaVA (Zhang et al., 2023)
41.0 46.0 42.5 56.5 39.0 53.5 53.0 48.0 41.0 29.0 31.5 82.5 45.0 26.0
53.0
41.
33.5
41.
27.5
38.5 31.5
LLaMA-VID (Li et al., 2024c)
41.3 45.5 40.5 58.0 39.5 55.0 53.5 40.0 35.5 18.5 27.5 87.0 41.5 23.0 45.5 41.0 27.0
40.0 34.5
41.5
31.5
LLaMA-Adapler (Zhang et al., 2024a)
31.7 23.0 28.0 $1.0 30.0 33.0 53.5 32.5 33.5 25.5 21.5 30.5 29.0 22.5 41.5 39.5 25.0
31.5 22.5 28.0 32.0
Video-ChatGPT (Maaz et al., 2023)
32.7 23,5 26.0 62.0 22.5 26.5 54.0 28.0 40.0 23.0 20.0 31.0 30.5 25.5 39.5 48.5 29.0 33.0
29.5 26.0 35.5
VideoChat (Li et al., 2023c)
35,5 33.5 26.5 56.0 33.5 40.5 53.0 40.5 30.0 25.5 27.0 48.5 35.0 20.5 42.5 46.0 26.5 41.0 23.5 23.5 36.0
Lide (Zhang et al., 2023)
34.1 27.5 25.5 51.0 29.0 39.0 48.0 40.5 38.0
22.5 22.5 43.0
34.0
22.5 32.5 45.5
32.5
40.0 30.0
21.0 37.0
VidcoChat2 (Li et al., 2024b)
60.3 66.5 74.0 85.5 51.0 61.5 85.5 68.0 43.5 48.5 35.5 83.5 38.5 66.5 88.0 50.5 63.5 46.5 36.0 42.5 70.5
+LTR
62.4 69.5 78.0 86.0 52.5 62.0 85.5 73.0
45.0
49.5 35.5 86.0
40.0
68.5 87.0 55.0 64.5 47.0 38.0 49.5 75.0
VideoLLaMA3 (Zhang et al., 2025)
67.1 70.5 72.5 91.5 43.5 85.5 92.5 74.5 42.0
51.5 44.5 92.5 53.0
75.0
92.0 59.0
61.5 76.5 33,5 54.0 75.5
+LTR
69.0 74,0 77.5 91.0 45.0 85.0 91.5 79.0 43.0
52.0
45.0 94.0
54.0 77.5 92.5 63.0
62.0
77.0 35.5 60.5 81.5
Vnne-Onneisionn (Li al., 2024a)
57.3 73.0 69.5 79.0 46.0 79.5 63.5 76.5 37.5 21.5 40.0 92.0 47.0 46.0 68.5 52.0 55.0 63.5 34.5 51.5 50.0
+LTR
59.9 79.5 76.5 79.0 46.5 80.0 62.0 82.0 38.5 23.5 410 910 48.0 48.0 69.5 57.5 56.0
65.0
37.0
59.5 57.5
Qwen2-VL (Wang 2024)
65.7 77.0 80.5 81.5 49.5 75.0 93.5 72.5 39.0
47.0 47.5 93.5 46.0
81.5 94.5 46.5
58.5 68.0
41.5
55.0 66.5
+LTR
67.8 82.5 86.5 82.0 50.0 75.5 92.0 77.5 39.5 46.5 46.5 94.0 48.0 83.0 93.5 51.0 60.0 69.0 44.0 62.0 73.5
Table Experimental results on MVBench. The results in the white area are copied from the corresponding works or MVBench (Li et al., 2024b), and the results in the blue area are reproduced by us using their published model weights and instructions.
work enhances cognitive reasoning capabilities of MILMS more substantially than their perceptual abilities.
In the Logical Tree Reasoning Stage stage, we examine the effects of answer verification (line w/o A.V.), correct perceptual answers (line w/o P.L.Q.A) and video incorpora- tion (line R w/o video) in logical reasoning. Experimental results show that answer verification significantly impacts complex reasoning tasks by effectively identifying logical errors, while video incorporation is crucial since leaf ques- tions alone cannot provide sufficient information for tree reasoning. Moreover, the results indicate that relatively pre- cise perception is necessary for LTR framework. Although video-aided reasoning can provide some missing informa- tion during the process, it may not always be sufficiently accurate for deducing correct main answers. These findings highlight the importance of both answer verification and video incorporation in logical reasoning stage within the LTR framework, particularly for complex reasoning tasks.
4.5. Ablation Study
In this section, we study the effect of the two reasoning stages in Table The experiments are conducted on the VidSA and NeXT-QA with Qwen2-VL as baseline.
In the Logical Tree Generation Stage, we analyze the syn- ergistic impacts of integrating RAG (line w/o RAG) and video content (line w/o video) during hierarchical tree con- struction. Experimental results indicate that incorporating RAG provides essential gudiance on the quality of generated language-centric trees. Moreover, the existence of video also provides spatiotemporal grounding cues within the LTR framework; without it, the tree lacks video-specific logical structures, leading to degraded performance.
7

---

Divide and Conquer: Exploring Language-centric Tree Reasoning for Video Question-Answering
Video
2S
main Whet ox the netel objact that etshenary when video begis?? (Brown)
inter St When ofler the metri object whin the wdea begins (Brown and purple)
y de deded Lopical Resente
From lasical persacttve, llare are fronti and papple axdula is die sideo (feaf 2/, and the color of oljects whtnn vidoo begine gny, frows, purplc and grown (leef Thendler, the color ofjecm
lea_i What
abject?
purple)
Longugge meuugee-enntc Logiol
leaf_e What the abject when video begin? (Sroy, orean, parple and grees)
Froen viazi pereuctive, can find pumie
moving tewards browe Thereies, conclade dae staver rose cocatio "Shows puppee.
video bagine bnown purple
vidco begina.
22] Inter When oslor the
enatoonoyy
begies? (Rrewr, grly groes)
From logioe jereuernee, Legiall
onjoze stmoueey (lesf, Merrover, only the
styeche enery? (Eney green)
objons corn when the vidao bugins 3). Thrrsfor, the color statieeury objects whze the video begine Froen viani peripattve,, can that begazup of vidoo, boowe calric cplinaer and proas cylinder statomary, while brown cabie moves later So asseer leown, and Thoniirs, so oreclade thee the snower of wor quedion "itmrrn gny and greve"
breem, purple
2 1 What calor a the abject when the video baged (Sroy, arown, purple and grees)
and
Aidee-Aided Layical
From logiool perpeettvv, can and purple objers de
begning bagits legins (inkrr .it Morcove, bxown,, gny greas objects statioacy
begito (inio__)) Therefie,
parple
Thanciee,
Video
moiin How mony timax did the person show okjachs the camers? (Cernect: vis) (Pive)
Inher_ thene
chjnchs the conara? (Ves)
Lagical Reosamin
Frm logical perepactive fixily confirm
bo Doed porson evist? (Yer)
From igaiaa propective, we fine corfins pesson and somatinn
poneee (Jag."
comonn (oag Then, we conclade
penson damnning obrects
Longunge sontme Logieal
250 leof_ Is sumething show to the comers? (Yed)
peopective,
shjant
(Joaf /t, including two drinkag
the [wown] ofjinee om shown to cansors
leef_ thene showing comething? (Yar)
books.
alosss, [thred books, aible and those
Threefoee arschadz
lienee,
conciate that perees
shor the sbructs (ervon] cree
Inter_i How mony objects are shown to the comero? (Seven)
Video ded Legical Beceowing
per
visual pergrativv,
ABll leaf_ c How many drinking pisses ure shown the comera? (Two)
From logica perepattve, we can fins the ton drnkkig glane (inef C [ihree] books dood," it one tille (loa' and oue tioas boxes Hoaa shows
books
while
the
(three)
to the camera. by stikng them sugchar we concaale that anover "[avan]
by persoe's hund, tharffrr cannat
loa__ How tissus bones are shown to the comona?? (Gne)
leof How mony tables are shown to the comeno? (Onx)
From staal perpectivr, we car thet drikiigg glases and [three] books am beld by
Buhiid dwom thore Thorotrr oonclade the thene am jurve]] anject crown to cameers TheeGos, conclade dux srawer queston "Vos"
with thouse but
ottorrs (five) time to coreers
counted down objects, by excluding
fhese two oljjcts. the pesson show the
Thasefore, we coedlade faxt srower of
coot qucabie "Five"".
Roautt Recursivo Coerikng
Liet
Percestual Question
Percapted/Tre svenced
2
Need Diuace 75% Do Not Need Divaton
(Nicee loat Answant) Rendt st Ventidd dee- Aiddd Recsoring
[Wrong clees): Cluss in the outjst that locs to wrong snower
Figure 3. Qualitative examples on MVBench (Li et al., 2024b) generated by VideoChat2 (Li et al., 2024b) with our L.T framework.
5. Qualitative Results
in video could possibly be stationary at the beginning, it may move in other parts of the video. Thanks the visual- aided logical reasoning step, our framework successfully avoids this logical trap by recognizing that the brown cube is stationary at the beginning and starts moving later. Finally, for the main question, we conclude that the brown metal cube is stationary when the video begins.
In this section, we provide two extra qualitative results in Figure 3. The first case shows successful reasoning that uses perception results to derive the answer to the origi- presents failure scenario,
nal question. The second showcasing that although our visual-aided logical reason- ing can correct some errоrs in reasoning, severe perception incorrectness can also lead to unrecoverable failure.
In the second video, man shows drink glasses to the camera twice and shows books the camera four times sequentially. In this example, the sub-question inter O is further divided into three perceptual questions, confirming the existence of the person, the object, and the action "show". Therefore, we conclude that the person does show something to the camera. Besides, the counting sub-question inter. is di- vided into counting the different objects appearing in the video. The logic trap here is that some perceived objects serve as backgrounds and are not shown to the camera by the man. Therefore, the answer shall not be calculated by
Specifically, in the first video, a metal purple eube col- lides with metal brown eube the beginning, while all other objects not metal. For sub-question inter 0, we further divide it to inquire about the colors from different perspectives. Based on the answers of leaf_l and 2 our framework deduces the answer by finding the intersection of these two color sets, resulting in the correct answer "brown and purple". The sub-question inter_/ / is also processed in similar manner. However, an object that stationary
8

---

Divide and Conquer: Exploring Language-centric Tree Reasoning for Video Question-Answering
Method
Acc@
E Acc@P
Acc Acc
Causal-VidQA
NEXT-QA
Video-LLaVA (Zhang et al.. 2023)
73.7
74,4
47.7
51.5
61.8
Setting
Acc@ All
Acc@ All
Video- Video-ChatGPT ChaiGP (Maaz et al., 2023)
73.1
75.1
46.0
50.0
61.1
VideoChat (Li et al., 2023c)
72.9
73.9
45.9
45.8
59.6
Baseline
68.0
80.4
LaMA (Zhang et al., 2023)
69.2
71.0
44.4
45.0
57.4
Full
70.2
82.7
VoT (Fei et al,, 2024a)
81.2
83.0
54.7
58.6
69.4
w/o video
68.9
81.5
VideoChai2 (Li et al., 2024b)
66.8
75.1
45.8
38.6
56.6
First. Stage
w/o RAG
69.3
82.2
67.8
77.2
48.8
42.8
59.1
VideoLLLMA3 (Zhang et al., 2025)
79.1
79,9
55.9
43,8
64.6
w/o A.
69.7
82.0
+ LTR
80,0
80.8
58.3
47.5
66.6
Sccond Stage
w/o P.L.Q.A
66.1
79.1
VAV--neViisio (Li ct al., 2024a)
78.6
78.0
53.1
44,2
63.5
R. w/o video
52.6
61.1
+ LTR
79.8
79.2
56.1
48.5
65.9
VL (Wang et al., 2024)
80.3
81.5
59.8
50.3
68.0
+ LTR
81.4
82.3
62.3
54.8
70.2
Table S. Ablation studies. w/o video in first stage indicates that the video is not appear in first stage. w/o ver. in second stage indi- cates that delete the In-process Answer Verification. w/o P.L.Q.A indicats that Perceptual Leaf Question Answering provides miss- ing answers for leaf questions. R. w/o video indicates that no video is provided during the Video-aided Logical Reasoning.
Table Zero-shot performance on Causal ViuQAA D: description, E: explanation, P: prediction, C: counterfactual, A: all. Acc@E and Acc@C are reported in answer and reason setting. The results in the blue area are reproduced by us using their published model weights and instructions.
Method
A
Wcc
Acc Acc@ All
Video-LLLaA (Zhang et al., 2023)
75.9
63.8
67.7
66.3
6. Conclusion
ILAMA-VID (Li et al., 2024c)
Video-ChatGPT (Maaz et al., 2023)
75.7
64.1
66.9
64.4
In this work, we propose a novel two-stage Language- centric Tree Reasoning (LTR) framework that enhances the reasoning capabilities and transparency of MI.LMs. In the first stage, LTR recursively generates a language-centric logical tree, utilizing language as the central driving force to gradually transform the complex congitive questions into simple perceptual qustions. In the second stage, aided by video content, LTR performs bottom-up logical reasoning within the tree to recursively derive the final answer with a complete, traceable reasoning path. To enhace the logical division ability of MLLMs, we exploit retrieval- -aumeeeal- generation to guide question division. Extensive experi- ments across 11 VideoQA benchmarks demonstrate that the LTR framework significantly improves both accuracy and interpretability compared to sfate-of-the-ard MLLMs. Overall, this work implements a traceable tree reasoning framework, paving the way for future research on language- centric video understanding from perception to cognition.
Video-LaVIT (Jin al., 2024b)
VideoChat (Li et al., 2023c)
74.6
61.5
63.5
61.8
VideoLLaMA (Zhang et al., 2023)
72.3
57.4
59.2
60.6
VoT (Fei et al., 2024a)
83.3
74.6
75.8
76.0
VideoChat2 (Li et al., 2024b)
84.6
76.0
78.6
78.8
+ LTR
85.8
78.7
81.6
81.4
VideoL.LaMAA (Zhang et al., 2025)
87.7
81.0
82.7
83.0
+ LTR
88.5
82.9
84.8
84.8
LLaVA-OneVision (Li al., 2024a)
85.5
76.3
78.6
79.0
+ LTR
86.4
78.9
82.2
81.8
Qwen2-VL (Wang al., 2024)
85.7
78.2
80.1
80.4
+ LTR
86.7
80.3
82.8
82.7
Table Experimental results on NEXT-QA. D: descriptive, T: tem- poral, C: causal. The results in the white area are copied from the corresponding works or VoT (Fei et al., 2024a), and the results in the blue area reproduced by using their published model weights and instructions.
simply adding the numbers of sub-question answers. More- over, the MLLMs falsely count the number of books shown to the camera as three. These two challenges lead an incorrect answer of "seven" for this sub-question. However, in the higher-level logical reasoning (i.e., the visual-aided logical reasoning of the main question), the logic trap is par- tially avoided by the visual assistance since our framework finds that the table and tissue box are not interacted with by the man. Meanwhile, the perceptual error introduced by the MLLMs still prevents the model from concluding the correct answer, causing the incorrect answer "five". De- spite this incorrect answer, our framework still gives clear interpretable reasoning steps that identify the reason why the model is giving the wrong answer "five", and locates the root cause as the perceptual failure rather than incorrect reasoning. Such ability showcases that our LTR framework able to increase the interpretability of existing MI..Mss
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
A Acknowledgement
This work is supported by the National Natural Science Foundation of China (Grant No. 62402341, 62471287, 62372329), the Shanghai Municipal Science and Technol- ogy Major Project (Grant No. S0ZZXXX00)), the Postdoctoral Fellowship Program of CPSF (GZC20241225) and National Key Research and Development Program of China (Grant No. 2024 0024YFE02110 1000).

---

Divide and Conquer: Exploring Language-centric Tree Reasoning for Video -uestion-Answering
References
DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., Zhang. X., Yu, X., Wu, Y., Wu, Z. E., Gou, Z., Shao, Z., Li, Z., Gao, Z., Liu, A., Xue, B., Wang, B., Wu, B., Feng, B., Lu, C.,, Zhao C., Deng, C., Zhang, C., Ruan, C., Dai, D., Chen, D., Ji, D., Li, E., Lin, F., Dai, F., Luo, E. Hao, G., Chen, G., Li, G., Zhang, H., Bao, H.,, Xu, H., Wang, H.. Ding. H., Xin, H., Gao. H., Qu. H., Li. H., Guo, J., Li, J.,, Wang, J., Chen, J., Yuan, J., Qiu, J., Li, J., Cai, J. L., Ni, J., Liang, J., Chen, J., Dong, K., Hu, K. Gao, K., Guan, K., Huang, K. Yu, K., Wang, I.., Zhang, L., Zhao, L.,, Wang, L.,. Zhang, L., Xu, L., Xia, L., Zhang, M., Zhang, M., Tang, M., Li, M., Wang, M., Li, M., Tian, N., Huang, P., Zhang, P., Wang, Q. Chen, Q., Du, Q. Ge, R., Zhang, R., Pan, R., Wang, R., Chen, R. J., Jin, R. L., Chen, R., Lu, S., Zhou, S., Chen, S., Ye, S., Wang, S., Yu, S., Zhou, S., Pan. S., Li, S. S., Zhou, S., Wu, S., Ye, S., Yun, T., Pei, T., Sun, T., Wang, T., Zeng, W., Zhao, W., Liu, W., Liang, W., Gao, W. Yu, W., Zhang. W., Xiao, W. L., An, W., Liu, X., Wang. X., Chen, X., Nie, X., Cheng, X., Liu, X., Xic, X., Liu, X., Yang, X., Li, X., Su, X., Lin, X., Li, X. Q., Jin, X., Shen, X., Chen, X., Sun, X., Wang, X., Song, X., Zhou, X., Wang, X., Shan, X., Li, Y. K., Wang. Y. Q.. Wei, X., Zhang, Y., Xu, Y., Li, Y., Zhao, Y., Sun, Y., Wang, Y., Yu, Y., Zhang, Y. Shi, Y., Xiong, Y., He, Y., Piao, Y., Wang, Y., Tan, Y., Ma, Y., Liu, Y., Guo, Y., Ou, Y., Wang. Y., Gong, Y., Zou, Y., He, Y., Xiong. Y., Luo, Y., You, Y., Liu, Y., Zhou, Y., Zhu, Y. X., Xu, Y., Huang, Y., Li, Y., Zheng, Y., Zhu, Y., Ma, Y., Tang, Y., Zha, Y., Yan, Y., Ren, Z. Z., Ren, Z., Sha, 7.,, Fu, Z.,, Xu, Z., Xie, Z., Zhang, Z., Hao, Z. Ma, Z., Yan, Z., Wu, Z., Gu, Z., Zhu, Z., Liu, Z., Li,
Andreas, J., Rohrbach, M., Darrell, T., and Klein, D. Neural module networks. In CVPR, pp. 49, 2016.
Ataallah, K., Shen, X. Abdelrahman, E., Sleiman, E., Zhu, D., Ding, J., and Elhoseiny, M. MiniGPT4-Video: Advancing multimodal lims for video understanding with interleaved visual-textual tokens. arXiv preprint 2tt1:24004034113. 2024.
Cai, Z., Cao, M., Chen, H., Chen, K., Chen, K., Chen, X., Chen, X., Chen, Z., Chen, Z., Chu, P., al. InternLM2 technical report. arXiv preprint arXiv:2403.1729) 2024.
Cao, Q., Liang, X., Li, B., Li, G., and Lin, L. Visual question reasoning on general dependency tree. In CVPR, pp. 7249-7257, 2018.
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., and Joulin, A. Emerging properties in self-supervised vision transformers. In ICCV, pp. 9650- 9660, 2021.
Chen, J., Niu, L., Zhou, S.. Si, J., Qian, C., and Zhang, L. Weak-shot semantic segmentation via dual similarity transfer. Advances in Neural Information Processing Systems, 01332555-32536 2022.
Chen, J., Yan. J., Fang, Y., and Niu, L. Meta- learning and refining for category-agnoste pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 23534-23543, 2024.
Cheng, 7., Leng, S., Zhang, H., Xin, Y., Li, X., Chen, G., Zhu, Y., Zhang, W., Luo, Z., Zhao, D., et al. Videollama 2: Advancing spatial- -empoall modeling and audio under- standing in iidoo-Ilms arXiv preprint 24Xiv:2406.0744 2024.
Z., Xie, Z., Song. Z., Pan, Z., Huang, Z., Xu, Z., Zhang, Z., and Zhang, 7. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https: /tt////axvvvv// http://wttwrrraaa////////o// wwtp:///g/aab/aa/eeeee 2501. 12948.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The LLaMA herd of models. arXiv preprint arXiv:2407.21778 2024.
Cherian, A., Hori, C., Marks, T. K., and Roux, J. L. (2.5+1)D spatio-temporal scene graphs for video question answering. In AAAL, pp. 444 453,453.770 2022.
Fan, C., Zhang, X., Zhang, S., Wang, W., Zhang, C., and Huang, H. Heterogeneous memory enhanced multimodal attention model for video question answering. In CVPR, pp. 1999-2007, 2019.
Chi, H., Li, H., Yang. W., Liu, F., Lan, L., Ren, X., Liu, T., and Han, B. Unveiling causal reasoning in large language models: Reality or mirage? In Advances in Neural Information Processing Systems, 2024a.
Fei, H., Wu, S., Ji, W., Zhang, H., Zhang, M., Lee, M.- I.., and Hsu, W. Video-of thtoo-of-thought: Step-by-step video reasoning from perception to cognition. In ICML, 2024a.
Chi, H., Yang, W., Liu, F., Lan, L., Qin, T., and Han, B. Does confusion really hurt novel class discovery? International Journal of Computer Vision, 132(8):3191-3207 2024b.
Fei, H., Wu, S., Zhang, M., Zhang, M., Chua, T.-S., and Yan, S. Enhancing video-language representations with structural spatio-temporal alignment. JEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024b.
Choi, S., On, K., HcO, Y., Sco, A., Jang, Y., Lee, M. S., and Zhang, B. DramaQA: Character-centered video story understanding with hicrarchical QA. In AAAI, pp. 1166- 1174, 2021.
10

---

Divide and Conquer: Exploring Language-centric Tree Reasoning for Video Question-Answeriin
Fu, C.,, Dai, Y., Luo, Y., Li, L.., Ren, S., Zhang, R., Wang, Z.,, Zhou, C., Shen, Y., Zhang, M., et al. Video-MME: The first-ever comprehensive evaluation benchmark of 1lms in video analysis. arXiv preprint SKKi:22005.20077 2024.
Jin, Y., Sun, Z., Xu, K., Chen, L.,, Jiang, H., Huang, Q., Song, C., Liu, Y., Zhang, D., Song, Y., al. Video-LaVIT: Uni- fied video-language pre-training with decoupled visual- motional tokenization. arXiv preprint arXiv22402.03161, 2024b.
Johnson, J., Hariharan, B., Van Der Maaten, L.,, Hoffman, J., Fei-Fei, L., Lawrence Zitnick, C., and Girshick, R. Inferring and executing programs for visual reasoning. In ICCV, pp. 9989-9998, 2017.
Gandhi, M., Gul, M. O., Prakash, E., Grunde-MeLaughinn M.. Krishna, R., and Agrawala, M. Measuring compo- sitional consistency for video question answering. In CVPR, pp. 5046-50555 2022.
Le. T. M., Le, V., Venkatesh, S., and Tran, T. Hierarchical conditional relation networks for video question answer- ing. CVPR. pp. 9969-99797 2020.
Gao, J., Ge. R., Chen, K., and Nevatia, R. Motion- appearance co-memory networks for video question swering. In CVPR, pp. (576-6588 2018.
Li B., Zhang, Y., Guo, D., Zhang, R., Li, F., Zhang, H., Zhang, K., Li, Y., Liu, Z., and Li, C. Llava- onevision: Easy visual task transfer, 2024a. URLhttps:
Gao, L., Zeng, P., Song. J., Li, Y., Liu, W., Mci, T., and Shen, H. T. Structured two-stream attention network for video question answering. In AAAI, pp.
2019.
/ //rxivv... owgoorg//ao abs/////////// 2400 03326.
Crunde-MeLaughlee M., Krishna, R., and Agrawala, M. Agqa: A benchmark for compositional spatio-temporal reasoning. CVPR, pp. 11287-11297, 2021.
Li, H., Chi, H., Liu, M., Huang, W., Xu, L., and Yang, W. Transformer- spatial-temporal counterfactual outcomes estimation. In International Conference on Machine Learning, 2025.
Gu, M., Zhao, Z., Jin, W., Hong, R., and Wu, F. Graph-based multi-interaction network for video question answering. IEEE Trans. Image Process., 50:2758-2770 2021.
Li, J., Niu, I.., and Zhang, From Representation to Reasoning: Towards both evidence and commonsense reasoning for video question-answeriine answering. In CVPR, pp. 21241-21250, 2022a.
Guo, Z., Zhao, J., Jiao, L., Liu, X., and Li, L. Multi- scale progressive attention network for video question answering. In ACL, pp. 973-978, 2021.
Li, J., Niu, L., and Zhang, L. Knowledge proxy intervention for deconfounded video question answering. In ICCV, pp. 2782-279999 2023a.
Hu, R., Andreas, J., Rohrbach, M., Darrell, T., and Saenko, K. Learning to reason: End-to-end module networks for visual question answering. In ICCV, pp. 804-813, 2017.
Li, J., Wei, P., Han, W., and Fan, L. IntentQA: Context- aware video intent reasoning. In ICCV, pp. 11963-11974, 2023b.
Hudson, D. A. and Manning, C. D. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In CVPR, pp. 6700-6709, 2019.
Li, K., He, Y., Wang, Y., Li, Y., Wang. W., Luo, P., Wang, Y., Wang, L.., and Qiao, Y. VideoChat: Chat-centric video understanding. arXiv preprint arXiv:2305.06335 2023c.
Jang. Y., Song, Y., Yu, Y., Kim, Y., and Kim, G. TGIF- QA: Toward spatio-tempora reasoning in visual question answering. In CVPR, pp. 1359-1367, 2017.
Li, K., Wang, Y., He. Y., Li, Y., Wang, Y., Liu, Y.. Wang, Z., Xu, J., Chen, G., Luo, P., et al. MVBench: A compre- hensive multi-modal video understanding benchmark. In CVPR, pp. 22195-22206, 2024b.
Ji, J., Krishna, R., Fei-Fei, L., and Niebles, J. C. Action Genome: Actions compositions of spatio-temporal scene graphs. In CVPR, pp. 10236-10247, 2020.
Li, X., Song, J., Gao, L., Liu, X., Huang, W., He, X., and Gan, C. Beyond RNNs: Positional self-attention with co-attention for video question answering. In AAAI, pp. 8658-8665, 2019.
Jiang, P. and Han, Y. Reasoning with heterogeneous graph alignment for video question answering. In AAAI, pp. 11109-11116, 2020.
Li, Y., Wang, X., Xiao, J., and Chua, T. Equivariant and invariant grounding for video question answering. In ACM MM, pp. 4724-4722, 2022b.
Jin, P., Takanobu, R., Zhang, W., Cao, X., and Yuan, L. Chat- UniVi: Unified visual representation empowers large lan- guage models with image and video understanding. In CVPR, pp. 13700-13710, 2024a.
Li, Y., Wang, X., Xiao, J. Ji, W., and Chua, T. Invariant grounding for video question answering. In CVPR, pp. 2918-299 2022c.

---

Divide and Conquer: Exploring Language-centric Tree Reasoning for Video Question-Answering
Li, Y., Wang, C., and Jia, J. LLaMA-VID: An image is worth 2 tokens in large language models. In ECCV, pp- 323-340, 2024c.
Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., Fan, Y., Dang. K., Du, M., Ren, X., Men, R., Liu, D., Zhou, C., Zhou, J., and Lin, J. Qwen2-vl: Enhancing vision- -anguage model's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024.
Li, Y., Wang, X., Xiao, J., Ji, W., and Chua, T.-S. Transforee--eppoweee invariant grounding for video question answering. IEEE Transuctions on Puttern Anal- ysis and Machine Intelligence (TPAMI), 2024d.
Wang. Q. C., Xiao, Z., Mao. Y., Qu, Y., Shen, J., Lv, Y., and Ji, X. Model predictive task sampling for efficient and robust adaptation, 2025. URL https: /arxiv. org/ abs//b//////// 2501 11039.
Liao, Z., Li, J., Niu, L., and Zhang. L. Align and aggre- gate: Compositional reasoning with video alignment and answer aggregation for video question-answering, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13395-13404, 2024.
Wang. X.,, Liang, J.. Wang, C.-K., Deng, K., Lou, Y., Lin, M., and Yang, S. VLAP: Efficient video- languaa align- ment via frame prompting and distilling for video ques- tion answering. arXiv preprint arXiv:2312.08.0 2023.
Lin, B., Zhu, B., Ye, Y., Ning, M., Jin, P., and Yuan, L. Video-LLaVA, Learning united visual representation by alignment before projection. In EMNLP, 2024.
Wu, B., Yu, S.,, Chen, Z., Tenenbaum, J. B., and Gan, C. STAR: A benchmark for situated reasoning in real-world videos. In NeurlPS, 2023.
Liu, F., Liu, J., Wang, W., and Lu, H. HAIR: hierarchical visual-semantic relational reasoning for video question answering. In ICCV, pp. 1678-1687, 2021.
Wu, J., Liu, Q. Wang, D., Zhang, J., Wu, S., Wang, L., and Tan, T. Logical closed loop: Uncovering object hallucinations in large vision-language mod- els. In Findings of the Association for Computa- tional Linguistics: ACL 2024, pp. 6944-6962, Bangkok, Thailand, August 2024. Association for Computa- tional Linguistics. doi: 10. 10.116553/112022.fnnimm 414. URL https: /awalaaaantolooyyo...ooaa wrg.oooo..rrg////0 f findings-acl. 414/
Maaz, M., Rasheed, H., Khan, S., and Khan, S. Video- ChatGPT: Towards detailed video understanding via large vision and language models. arXiv preprint 000,:2306.05224, 2023.
Mangalam, K., Akshulakov, R., and Malik, J. EgoSchema: diagnostic benchmark for very long-form video lan- guage understanding. In NeurlPS, pp. 46212-46244, 2023.
Xiao, J., Shang. X., Yao, A., and Chua, T. NEXT-QA: Next phase question-answerrng to explaining temporal actions. CVPR, pp. 9777-9786, 2021.
Mascharka, D., Tran, P., Soklaski, R., and Majumdar, A. Transparency by design: Closing the gap between perfor- mance and interpretability in visual reasoning, In CVPR, pp. 4942 4950, 2018.
Xiao, J., Yao, A., Liu, Z., Li, Y. Ji, W., and Chua, T. Video as conditional graph hierarchy for multi-granular question answering. In AAAI, pp. 2804-2812, 2022a.
Park, J., Lee, J., and Sohn, K. Bridge To Answer: Structure- aware graph interaction network for video question an- swering. In CVPR, pp. 15526-15535, 2021.
Xiao, J., Zhou, P., Chua, T., and Yan. S. Video graph transformer for video question answering, In ECCV, 2022b.
Peng, M., Wang, C., Gao, Y. Shi, Y., and Zhou, X. Multi- level hierarchical network with multiscale sampling for video question answering. In IJCAI, pp. 1276-1282, 2022.
Xu, J., Mei, T., Yao, T., and Rui, Y. MSR-VTT: A large video description dataset for bridging video and language. 2016
In CVPR, pp.
Qian, Z., Wang, X., Duan, X., Chen, H., and Zhu, W. Dy- namic spatio-temporal modular network for video ques- tion answering. In ACM MM, pp.
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhut- dinov, R., and Manning, C. D. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In 2018.
2022.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In ICML, pp. 8748-8763, 2021.
EMNLP, pp.
Yi, K., Gan, C., Li, Y., Kohli, P., Wu, J., Torralba, A., and Tenenbaum, B. CLEVRER: collision events for video representation and reasoning. In ICLR, 2020.
Share. ShareGemini:
Scaling up video caption data for mul- timodal large language models, June 2024. URL ht tps: //gi thub. , com/ Shares re1 114 4 //shSaaeeeee iShareGeminn...
Ying, K., Meng, F., Wang, J., Li, Z., Lin, H., Yang, Y., Zhang, H., Zhang, W., Lin, Y., Liu, S., et al. Mmt-hench:
12

---

Divide and Conquer: Exploring Language-centric Tree Reasoning for Video Question-Answering
comprehensive multimodal benchmark for evaluating large vision-language models towards multitask agi. In ICML, 2024.
Yu, Z., Xu, D., Yu, J., Yu, T., Zhao, Z., Zhuang, Y., and Tao, D. ActivityNet-QA: A dataset for understanding complex web videos via question answering. In AAAI, pp. 9127-9134, 2019.
Zhang, B., Li, K., Cheng, 7.,, Hu, Z.,, Yuan, Y., Chen, G., Leng. S., Jiang, Y., Zhang, H., Li, X., Jin, P., Zhang, W., Wang, F., Bing, L.., and Zhao, D. Videollama 3: Fron- tier multimodal foundation models for image and video understanding. arXiv preprint 1301 2025. URL www.tttts//..m httptt:t///aateeeee //axivv..ivvvv org/ abs/
Zhang, H., Li, X., and Bing, L. Video-LLaMA: An instruction-ue tuned visual language model for video understanding. In EMNLP, pp. 543-553, 2023.
Zhang, R., Han, J., Liu, C., Zhou, A., Lu. Qiao, Y., Li,
H., and Gao, LLaMA-Adapter: Efficient fine-tuning of large language models with erro-initialized attention. In ICLR, 2024a.
Zhang. Y.-F., Wen, Q.. Fu, C., Wang. X., Zhang, Z., Wang, L.. and Jin, R. Beyond LLaVA-HD: Diving into high- resolution large multimodal models. arXiv preprint arXiv:2406.084877
13

---

Divide and Conquer: Exploring Language-centric Tree Reasoning for Video Question-Answeriig
In this appendix, we provide additional experiments on various benchmarks in Section A to support the generalization ability and effectiveness of our framework. We also provide additional visual examples showcasing more detailed analysis of how our LTR framework helps achieve successful video reasoning in ??. Moreover, provide discussions addressing potential common concerns in Section B. Finally, we conclude by discussing limitations and future work in Section C.
A. Exprements on More Datasets
In this section, we conduct more experiments to demonstrate the effectiveness of our LTR framework in VideoQA. Specifically, we evaluate benchmarks include MSVD-QA (Xu et al., 2016), MSRVTT-QA (Xu al., 2016), TGIF-QA (Jang et al., 2017), and ActivityNer-QA (Yu et al., 2019), which focus on relatively simple perceptual tasks like action and object recognition. The multiple-choice benchmarks comprise STAR (Wu et al., 2023), Ego-Schema (Mangalam et al., 2023), and Video-MME (Fu et al., 2024). Among them, STAR and Ego-Schema respectively target situated reasoning and long-video understanding, while Video-MME serves as an inference-only multifaceted VideoQA benchmark covering diverse tasks and video durations. We maintain identical evaluation protocols to our main experiments. For open-ended settings, GPT-3.5 assesses response quality using both accuracy and score metries. For multiple-choice settings, MLLMs select answers from provided options based solely questions and framework semnewwtr-geeerttedd responses.
framework under two settings: Le., open-ended and multiple-choice. The open-ended
A.1. Experiments on Open-Ended Benchmarks
In this section, we evaluate the MSRVTT-QA, TGIF-QA, and ActivityNet-QA datasets where answers are short phrases, with results presented in Table 6. For TGIF-QA, specifically assess the FrameQA, State Transition, and Repeating Action subsets. In the experimental results, we find that our model can improve the corresponding baseline MILMs by 1 1%-2% in terms of accuracy. Although our LTR framework primarily enhances reasoning through kanguage-cntrr Logical Trees, these perceptual task gains remain consistent as existing MLLMs already possess strong visual perception capabilities that our method strategically utilizes.
MSVD-QA
MSRVTT-QA
TGIF-QA
ActivityNet-QA
Method
Acc. Score
Acc.
Score
Acc.
Score
Acc.
Score
Video-LLaVA (Zhang al., 2023)
51.6
2.5
59.2
3.5
70.0
4.0
45.3
3.3
Video-ChatGPT (Maaz cl al., 2023)
49.3
2.8
51.4
3.0
35.2
2.7
Videol.LaMA (Zhang et al., 2023)
29.6
12.4
1.1
LLAMA-VID (Li et al., 2024c)
70.0
3.7
58.9
3.3
47.5
3.3
LLaMA-Adapter (Zhang et al., 2024a)
54.9
3.1
43.8
2.7
34.2
2.7
Chat-UniVi (Jin ct al., 2024a)
65.0
3.6
54.6
3.1
60.3
3.4
45.8
3.2
VideoChat (Li et al., 2023c)
56.3
2.8
45.0
2.5
34.4
2.3
2.2
Video-LaVIT (Jin et al., 2024b)
73.2
3.9
59.3
3.3
50.1
3.3
MiniGPT4- Video (Ataallah et al., 2024)
73.9
4.1
58.8
3.3
72.2
4.1
45.9
3.4
VidcoChat2 (Li al., 2024b)
70.0
3.9
53.8
3.3
72.6
4.0
49.5
3.3
LTR
71.6
3.9
55.6
3.4
73.9
4.0
51.1
3.3
VideoLLaMA3 (Zhang et al., 2025)
80.3
4.4
65.2
4.0
84.5
4.3
60.6
3.9
LTR
82.0
4.5
67.1
4.1
86.1
4.4
62.4
4.0
LLaVA-One Vision (Li et al., 2024a)
78.8
4.2
63.4
3.6
81.0
4.1
56.0
3.5
+ LTR
80.1
4.4
65.2
3.7
82.3
4.1
58.5
3.7
Qwen2-VL. (Wang et al., 2024)
79.6
4.3
3.8
82.4
4.0
56.6
3.6
LTR
81.0
4.4
65.8
3.8
84.6
4.0
58.4
3.6
Table Experimental results on MSRVTT-QA, TGIF-QA, and ActivityNet-QA. The results in the white area are copied from the corresponding works, and results the blue area are reproduced by using their published model weights and instructions. Ace. represents the accuracy.
A.2. Experiments on Multiple-Choice Benchmarks
In this section, we evaluate our LTR framework on multiple-choice datasets including STAR, Ego-Schema and Video-MME. Experimental results are presented in Table and Table These benchmarks adopt the multiple-choic format due to historical limitations in generative capabilities of MILMS,, making challenging to produce comprehensive and coherent
14

---

Divide and Conquer: Exploring Language-centric Tree Reasoning for Video Question-Answering
STAR
Method
Ego-Schema
Int.
Seq.
Pre.
Fea.
Avg.
Video-LLaVA (Zhang al., 2023)
64.3
67.0
56.5
50.1
59.5
38.4
LLaMA-VID (Li al., 2024c)
38.5
VLAP (Wang et al., 2023)
70.0
70.4
65.9
62.2
67.1
Video-LaVIT (Jin et al., 2024b)
37.3
VideoChat (Li et al., 2023c)
63.2
66.8
54.1
49.6
58.4
VoT (Fei et al,, 2024a)
71.5
72.6
66.6
62.7
68.4
VidcoChat2 (Li ct al., 2024b)
54.1
64.9
68.2
64.4
62.9
53.3
LTR
$7.2
71.7
67.5
66.1
54.7
VidcoLLaMA3 (Zhang et al., 2025)
60.1
68.8
68.7
62.6
65.0
56.7
LTR
63.1
72.2
72.2
66.4
68.5
58.1
LLaVA-One Vision (Li et al., 2024a)
60.0
69.1
72.3
64.5
66.5
59.2
LTR
62.8
72.4
75.2
67.9
69.6
61.0
Qwen2-VI. (Wang et al., 2024)
65.2
74.9
70.5
68.7
69.8
65.0
LTR
67.4
77.7
73.4
71.8
72.6
Table 7. Experimental results on STAR. "Int.", "Seq", "Pre.", "Fea.", and "Avg." stand for "Interaction", "Sequeecc", "Prediction", "Feasibility", and "Average" respectively. The results the white area are copied from corresponding works VoT (Fei et al., 2024a), and the results in the blue area reproduced by using their published model weights and instructions.
Method
Short
Medium
Long
Average
Video-LLaVA (Zhang et al., 2023)
45.3
38.0
36.2
39,9
Chat-UniVi (Jin et al., 2024a)
45.7
40.3
35.8
40.6
SliME (Zhang al., 2024b)
53.3
42.7
39.8
45.3
ShareGemini (Share, 2024)
49.1
41.3
39.1
43.2
VideoChat2 (Li al., 2024b)
50.0
39.3
40.3
43.2
+ LTR
53.8
42.2
42.4
46.1
VideaL.LaMA3 (Zhang al., 20255
76.6
62.8
54.6
64.6
LTR
80.3
64.2
56.7
VLaVA-OOeVision (l.i ct al., 2024a)
67.8
54.6
48.7
57.0
LTR
71.7
57.2
50.0
59,6
Qwen2-VL. (Wang et al., 2024)
69.8
58.6
51.8
60.0
LTR
73.7
61.8
53.9
63.1
Table Experimental results on Video-MME. The subtitle is not used during the evaluation. The results in white area are copied from the corresponding works Video-MME (Fu al., 2024), and the results in the blue area are reproduced by using their published model weights and instructions.
open-ended answers.
Overall, our LTR framework generally achieves 2%-44% improvements on these multiple-choice benchmarks, exceeding gains observed on open-ended benchmarks. This demonstrates particular effeetiveness in reasoning- intennive tasks, aligning with our module designs: Divide with Top-down Recursive Checking and Conquer with Bottom-up Tree Reasoning. In the experimental results on STAR, our framework exhibits more significant improvements temporal and causal reasoning tasks than that on descriptive tasks. This observation further demonstrates that framework excels in complex cognitive tasks, enhancing the ability of existing MLLMs in temporal and causal reasoning. For Ego-Schema, LTR achieves modest gains (1.4%-2.5%) primarily due to its long-video reasoning focus versus 16-frame input limitation. This limitation constrains the capacity of MILMS to perceive effective information from the video, causing more unreliable perceptual results in our Bottom-up Tree Reasoning stage, therefore resulting in a smaller performance boost. A similar trend is also evident in Video-MME, where the improvements on short and medium-length videos are more pronounced than that on long ones.
B. Discussions
In this section, we discuss the rationale behind the designation of LTR framework, its inherent trade-offs. and its implications for multimodal reasoning. We first contextualize the balance between increased computational complexity and gains in interpretability enabled by hierarchical decomposition. Next, we explore the advantages of prioritizing linguistic logic over vision-centric approaches for structured reasoning. Finally, we analyze how training- free modular integration preserves the generalization power of foundation models while enabling task-specific adaptability.
15

---

Divide and Conquer: Exploring Language-centric Tree Reasoning for Video Question-Answering
B.1. Complexity of Hierarchical Reasoning
The proposed two-stage LTR framework inherently introduces higher computational complexity compared to direct inference paradigms, yet this trade-off is strategically aligned with its advantages in test-time scalability and interpretability. By recursively decomposing questions into perceptual cub--roblems and derive answers through bottom-up reasoning, the LTR framework mimics human-like hierarchical reasoning. While the tree-based structure prolongs inference time due to iterative decomposition and aggregation, it enables localized error analysis and modular optimization, reducing the risk of cascading failures in end-to-end systems. Crucially, the explicit tree generation provides transparent intermediate outputs (e.g., sub-question perceptual checking and answer verification steps), allowing users to trace reasoning paths and diagnose failures. Though not ideal for latency-sensitive applications, this design ensures robustness in complex scenarios, as evidenced by consistent performance gains across diverse benchmarks. Future adaptations could incorporate adaptive depth control to dynamically balance precision and efficiency based on problem difficulty.
B.2. Language-centric Logical Reasoning Paradigm
Unlike conventional VideoQA framework that prioritize visual feature extraction, our framework explicitly anchors reasoning in linguistic logic. The tree-generation stage first divides questions into logically coherent sub-questions, enforcing semantic rigor before engaging visual perception. This language-centric approach mitigates the inherent bias of vision-centric models, where dominant visual cues (e.g., salient objects) may override abstract logical relationships. For instance, when resolving questions requiring multi-hop causality, the model establishes verifiable sub-goals through semantic decomposition rather than relying solely on visual correlations. While this paradigm sacrifices some efficiency in low-complexity perceptual tasks, it significantly enhances performance on benchmarks demanding structured reasoning, as logical constraints guide the visual search space. This aligns with cognitive principles where language scaffolds complex problem--olving, ensuring answers adhere to both visual evidence and contextual logic.
B.3. Training-Frid FFeee Generalization
By preserving pre-trained MILMs as fixed modules and integrating Retrieval-Augmented Generation (RAG), our framework achieves task-specific adaptability without compromising generalization. Traditional fine-tuning methods often degrade foundational model capabilities when specializing for niche tasks, whereas our pipeline delegates logical structuring and visual perception to separate stages, avoiding parameter updates. The RAG component further enhances Nomain-ssecifff reasoning by dynamically retrieving relevant knowledge (e.g., the first-order sub-question struction for question division) during inference, climinating the need for biased task-oriented training. Besides, the experimental results further underscore that systematic architectural innovation, rather than parameter optimization, can effectively balance specialization and generalization in multimodal reasoning systems.
C. Limitation and Future Works
In our LTR framework, we introduce a language-centric logical reasoning approach comprising two key stages: Divide with Top-down Recursive Checking and Conquer with Botton-up Tree Reasoning This framework generates and reasons over logical trees through language-structured representations. While demonstrating promising results, exhibits limitations requiring further investigation and potential enhancements. Below we detail two primary constraints and propose future research directions.
C.1. Detailed Understanding for Long Videos
Although we have observed that the accuracy on simple perceptual questions are generally higher than that of complex cognitive questions, it becomes difficult for the model to answer perceptual questions when facing long videos, since long videos may contain too much irrelevant information for answering simple perceptual questions. Such limitation is inherent in the baseline MLLMs due to their simple video-text alignment method based on frame-sampling.
In future work, improving the capability of MLLMs in understanding long videos is expected to enhance the performance of our LTR framework when facing complex long scenes, as such improvement would facilitate precise answering of perceptual questions and provide more accurate low-level information during the reasoning process.
Such limitation may be addressed through various directions. For example, dense video representation (instead of frame-
16

---

Divide and Conquer: Exploring Language-centric Tree Reasoning for Video Question-Answeriing
sampling based representation) could introduce detailed information that lost during frame sampling. Moreover, detailed spatio-temporal alignment from video question may also help reduce noisy information for answering perceptual questions.
C.2. System-2 Reasoning via Reinforcement Learning
Our LTR framework relies the zero-shot capability of MLLMs perform cognitive reasoning, which introduces two inherent constraints. First, while zero-shot methods avoid costly supervised training, they inherently depend the pre- trained knowledge and reasoning biases of the base MLLMs, potentially limiting adaptability to domain-specifie long-video video scenarios. Second, the lack of explicit feedback mechanisms in zero-shot paradigms hinders iterative refinement of reasoning chains, a critical feature for handling complex spatio-temporal dependencies in long videos.
Recent advances in reasoning frameworks, such as reinforcement learning based approaches (DeepSeek-A) Al al., 2025) that enhance System-2 reasoning without requiring reward model training, suggest viable solutions. A promising direction involves integrating such methods into multimodal scenarios through LTR framework. Furthermore, extending this learning paradigm to support curriculum learning strategies, progressing from short to long videos and from perceptual to cognitive tasks, could strengthen the hierarchical reasoning capabilities of MLLMs. This integration would not only improve accuracy in System-2 video reasoning but also enable more interpretable and auditable reasoning processes.
17


Reviews:
[{"review_id": "hEcPrNrVKo", "reviewer": "Reviewer_8P4e", "summary": "This paper introduces Language-centric Tree Reasoning (LTR), a framework to enhance the reasoning of MLLMs. It uses MLLMs to first hierarchically break down a question into sub-questions, then conquer the question by answering the sub-questions in a bottom-up way. In the experiments, LTR is applied to four state-of-the-art MLLMs and it can consistently improve their performance on 11 video question answering benchmarks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for carefully reviewing our paper, acknowledging its strengths, and providing valuable suggestions for improvement\nIf you have any further concerns, please feel free to raise them during the second-round rebuttal phase.\nAs recommended by the official FAQ, we provide all figures and tabels via the [anonymous link](https://anonymous.4open.science/r/ICML25-3189-7D31/README.md).\n\n### Comparison with More Baseline Methods.\nTo further validate the effectiveness of our LTR, we compare the it with three baselines (CoT, Major Voting, and Best of N), across AGQA-Decomp, MVBench, and CausalVidQA using VideoChat2 as the MLLM, as shown in [Table1](https://anonymous.4open.science/r/ICML25-3189-7D31/R-8P4e/Table1.png).\nFor fair comparison, the repetition count (N) for all baselines is set to match the number of MLLM executions in LTR for various questions.\nFor CoT, we carefully construct [five text-based examples](https://anonymous.4open.science/r/ICML25-3189-7D31/R-8P4e/Figure3.png) as prompts generated by GPT-4o.\nFor Best of N, VideoChat2 also serves as the reward model.\nThese comparisons will be included in the following version.\n\n\nIn [Table1](#table1), although CoT, Major Voting, and Best of N improve model accuracy on three datasets, their gains remain relatively moderate compared with our LTR, largely due to the lack of detailed information in these methods. \nSpecifically, the Major Voting and Best of N do not provide detailed sub-question analysis and the CoT offers only simple reasoning examples that constrain the reasoning ability of MLLMs, however, our LTR delivers a structured, comprehensive context and a detailed reasoning process that clearly demonstrates how the answer is deduced, thereby enhancing both accuracy and interpretability.\n\nFurthermore, the compared methods contribute little to the compositional consistency of models, as they lack intra-question information exchange. In contrast, our LTR employs a hierarchical information aggregation and logical inference procedure that yields a consistent and accurate reasoning process, ultimately achieving superior performance in accuracy and compositional consistency on several VideoQA benchmarks (***i.e.***, AGQA-Decomp, MVBench, and CausalVidQA) while also offering enhanced interpretability and controllability.\n\n### More Complex Qualitative Results and Module Analysis\nTo validate the effectiveness of our LTR framework, we provide two complex qualitative examples in [Figure 1](https://anonymous.4open.science/r/ICML25-3189-7D31/R-8P4e/Figure1.png) and [Figure 2](https://anonymous.4open.science/r/ICML25-3189-7D31/R-8P4e/Figure2.png) in our response, along with a detailed analysis for the first example.\n\nIn [Figure1](https://anonymous.4open.science/r/ICML25-3189-7D31/R-8P4e/Figure1.png), the video depicts a scenario where a man is struck in the crotch by a dog, causing him to bend over and hold his crotch. In Stage 1, the Divide with Top-down Recursive Checking successfully decomposes the main question into a language-centric logic tree with simpler sub-questions that precisely target perceptual aspects (e.g., detecting the man, the dog, and the actions performed). In Stage 2, the Video-Aided Logical Reasoning module integrates both logical cues and visual evidence to infer answers for each non-leaf node. Crucially, the In Process Answer Verification stage is applied to ensure consistency across intermediate responses; for instance, it corrects the answer for non-leaf question [inter_2] by cross-validating the reasoning with the visual content.\n\nFurthermore, we also update more qualitative results along with comprehensive discussion, covering both successful and failure cases, to better illustrate the impact of each module within our LTR framework.\nThe revised figures and detailed descriptions are available in our response to Reviewer DfhN. \nWe will include these examples and discussions in the following version.\n\n### Explanation of System-2 Reasoning\nThe term system-2 originates from psychology and cognitive science, where the dual process theory delineates human reasoning into two distinct processes: system-1, which is fast, intuitive, situational, and perceptual, and system-2, which is slow, logical, abstract, and cognitive [1]. \nNotably, CoT, Major Voting, and Best of N are also typical examples of system-2 reasoning models.\nWe will explain more on system-2 reasoning and its relevance to computer science in the follow version. \n\n### Minor Issue\nThe MLLM employed in the qualitative analysis is VideoChat, and all the typos will be corrected in the following version.\n\n[1] Evans, J. S. In two minds: dual-process accounts of reasoning. Trends in Cognitive Sciences, 7(10):454–459, 2003. ISSN\n1364-6613.", "claims_and_evidence": "1. The paper claims that the proposed LTR can improve the reasoning capability of MLLMs. The consistent improvements on four MLLMs across 11 benchmarks support this claim.\n2. The paper claims that the design choices in the LTR are essential to the overall performance. The ablation studies in Section 4.5 support this claim.", "methods_and_evaluation_criteria": "The proposed method is well-motivated.", "theoretical_claims": "This paper does not involve theoretical claims.", "experimental_designs_or_analyses": "To demonstrate the effectiveness of the LTR framework, it is applied on a few pre-trained MLLMs and it is shown that the performance can be consistently improved. However, the experiments do not compare LTR with other baseline methods that can improve the reasoning capability of MLLMs. The proposed LTR can be proved effective only when it can outperform the baselines. A few simple baselines include:\n1. Chain-of-Thought prompting [1], which prompts the models to \"think step-by-step\".\n2. LTR needs to execute an MLLM for multiple times to answer one question. Therefore, it is a test-time scaling approach. The following test-time scaling baselines should be considered:\n\n(a) Majority Voting [2], which uses the MLLM to sample N answers and selects the most frequent one as the response.\n\n(b) Best-of-N [2], which samples N answers and uses a reward model to score the answers. The one with the highest score is selected as the response. The reward model can be the evaluated MLLM itself.\n\nEspecially, the N used in the baselines above should be comparable to the average number of MLLM executions in LTR.\n\n[1] Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Jason Wei et al. NeurIPS 2022.\n\n[2] Let's Verify Step by Step. Hunter Lightman et al. ICLR 2024.", "relation_to_broader_scientific_literature": "The previous MLLM reasoning works do not capture the logical structure of questions. The LTR framework proposed in this work leverages the logical structure and improves the MLLM performance on video question answering benchmarks.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "**Other Strengths:**\n1. The proposed LTR framework is training-free and can be directly employed on pre-trained MLLMs.\n2. Multiple up-to-date MLLMs are evaluated in the experiments. \n3. The paper writing is clear and easy to follow.\n\n**Other Weaknesses:**\n1. The question of the qualitative example in Figure 3 is too easy. The answer can be easily guessed even without the video provided. To demonstrate the effectiveness of the proposed method, a more challenging question should be used as a qualitative example.", "other_comments_or_suggestions": "**Typos:**\n1. L82-L83: off -> of\n2. L380-L381: existace -> existence\n\n**Writing:**\n1. In L46-L47, the concept of \"System-2 reasoning\" should be explained.\n2. In Section 4.5, qualitative examples are highly appreciated to demonstrate how each module affects the reasoning of the model.\n3. In Section 4.6, the base model used in the qualitative example should be mentioned.\n\n**Rating justification:** The proposed method is well-motivated and can effectively improve the performance of MLLMs. However, due to the lack of comparison with any existing baseline that can improve MLLM reasoning, I cannot vote for the acceptance of this paper. I am open to adjust my rating after rebuttal.", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HK4NKUpJXD", "reviewer": "Reviewer_DfhN", "summary": "This paper proposes Language-centric Tree Reasoning (LTR), a training-free, model-agnostic framework that enhances reasoning capabilities and interpretability in Video Question Answering (VideoQA) by using MLLMs. LTR addresses the limitations of existing MLLMs, such as opacity and lack of controllability in their reasoning processes. The framework operates by recursively generating a language-centric logical tree based on the input question and incorporating video content to create leaf nodes. These leaf nodes represent simple perceptual questions that can be answered by MLLMs. LTR then performs bottom-up reasoning through the tree, leveraging MLLM responses to the leaf node questions and verifying consistency with visual evidence. This process culminates in an answer to the original question and a traceable reasoning path. Experiments on 11 VideoQA benchmarks using four different MLLMs demonstrate that LTR improves reasoning accuracy and provides a more transparent and verifiable VideoQA system. Ablation studies analyze the effectiveness of individual components within the framework, and case studies showcase its enhanced error tolerance and explainability.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for carefully reviewing our paper, acknowledging its strengths, and providing valuable suggestions for improvement\nIf you have any further concerns, please feel free to raise them in the rebuttal comment.\nAs recommended by the official FAQ, we provide all figures via the [anonymous link](https://anonymous.4open.science/r/ICML25-3189-7D31/README.md).\n\n### Automated Pipeline\nWe agree that automation is critical for practical applications. Our two-stage approach, first generating the language-centric logical tree, then performing bottom-up reasoning through tree validation, handles VideoQA automatically.\nFor each question, the system recursively decomposes it into simpler sub-questions until all leaf nodes are perceptive, using retrieved few-shot examples as guidance.\nThe resulting tree is processed bottom-up to analyze both the video and question, yielding interpretable answers.\nFurthermore, while LTR is designed for full automation, it also allows for human intervention in sensitive scenarios, enabling users to adjust intermediate responses for better accuracy.\n\n### Applicable Question Types\nWe provide two figures to illustrate how LTR framework handles summarization and negation in multiple-choice scenarios. For the summarization question([Figure1](https://anonymous.4open.science/r/ICML25-3189-7D31/R-DfhN/Figure1.png)), LTR naturally decomposes the video into parts and generates a Language-centric Logical Tree that prompts a summary for each segment, leading to a comprehensive summary of the main question. In the case of negation in multiple-choice questions([Figure2](https://anonymous.4open.science/r/ICML25-3189-7D31/R-DfhN/Figure2.png)), the framework breaks down the question by checking each statement individually and then integrates the results to determine which statement is false, closely mirroring human reasoning.\n\nHowever, for questions that are linguistically simple yet require complex cognitive visual reasoning, such as “Is there a thief?”. LTR may struggle to generate a reasonable Language-centric Logical Tree. In these cases, the question is often misclassified as perceptual due to its simplicity, even though answering it requires detailed motion analysis and a comprehensive understanding of the video content.\nWe will include these discussions in the following version.\n\n### More Qualitative Results\nWe further provide three cases to illustrate LTR’s performance in different settings.\nIn [Figure3](https://anonymous.4open.science/r/ICML25-3189-7D31/R-DfhN/Figure3.png), for “Questions unanswered with LTR,” the counting sub-question misidentify background objects due to perceptual miscounting and a logic trap, resulting in an incorrect final answer despite a transparent reasoning process.\nIn [Figure4](https://anonymous.4open.science/r/ICML25-3189-7D31/R-DfhN/Figure4.png), for “Questions correctly answered with LTR and incorrectly answered by baseline,” LTR accurately deduced the stationary object’s properties by intersecting responses from multiple sub-questions and leveraging visual-aided reasoning, whereas the baseline relied solely on perceptual cues.\nIn [Figure5](https://anonymous.4open.science/r/ICML25-3189-7D31/R-DfhN/Figure5.png), for “Questions correctly answered by baseline and incorrectly answered with LTR,” the baseline produce a correct answer without any explanation. However, an erroneous leaf response in LTR led to an incorrect final answer, even though the intermediate video content analysis remained robust.\n\n### Errors in Intermediate Nodes\nErrors or hallucinations in reasoning are inevitable, but prior study [1] show that providing more context or sampling multiple answers can help suppress them. In LTR, detailed sub-question decomposition offers comprehensive context that reduces hallucinations and minimizes cascading errors, as evidenced by improved sub-question accuracy. So, an error at an intermediate node is less likely to cascade and adversely affect the final answer.\n\nThe In-Process Answer Verification stage further mitigates errors by prompting the model to check both reasoning logic and consistency between answers and visual content.\nAlthough no system can fully eliminate errors, this verification step increases the reliability of intermediate responses, as confirmed by experiments in Table5 of the submission.\nWe will include these discussions in the following version.\n\n[1] Fei, H. et.al. Video-of-thought: Step-by-step video reasoning from perception to cognition. In ICML, 2024\n\n### Performances on Closed-Source Models.\nTo valid the effectiveness of LTR on closed source models, we conduct experiments using GPT-4o on EgoSchema and MVBench, with 16 uniformly sampled frames per video.\nAs summarized in [Table1](https://anonymous.4open.science/r/ICML25-3189-7D31/R-DfhN/Table1.png), LTR yield significant accuracy improvements on these benchmarks, demonstrating its generalizability across various models.\n\n### Minor Issue\nAll typos will be revised in the following version.", "claims_and_evidence": "The evidence presented largely supports the claims made in the submission, offering a convincing case for the effectiveness of the LTR framework.", "methods_and_evaluation_criteria": "The proposed methods for enhancing VideoQA through language-centric tree reasoning are reasonable to the problem.", "theoretical_claims": "This submission does not present any theoretical claims requiring formal proofs. The focus is on the empirical evaluation of the proposed framework.", "experimental_designs_or_analyses": "The experimental design appears generally sound, encompassing a suitable range of evaluations. Testing the LTR framework with four different open-source MLLMs across 11 benchmarks, covering both open-ended and multiple-choice question types, provides a comprehensive assessment of its effectiveness. The chosen evaluation metrics are appropriate for VideoQA tasks. Furthermore, including a thorough ablation study allows for a detailed analysis of the contributions of individual components within the LTR framework. However, certain specific questions regarding the experimental setup and results are raised in the \"Other Strengths and Weaknesses\" section.", "relation_to_broader_scientific_literature": "This work addresses the limitations of current MLLMs in VideoQA, particularly their lack of transparent System-2 reasoning. It proposes a novel language-centric reasoning framework, offering a potential solution to the interpretability challenges faced by existing approaches like VoT and DSTN.", "essential_references_not_discussed": "The paper provides a comprehensive overview of related work.", "other_strengths_and_weaknesses": "Strengths:\n  - The language-centric tree reasoning framework appears novel and offers a promising approach to improving interpretability and controllability in VideoQA.\n  - The proposed framework's training-free and model-agnostic nature enhances its practicality and broad applicability.\n\n\nWeaknesses:\n  - The LTR framework, as described, involves multiple steps and inferences to process a given question and generate the reasoning tree. While this is acceptable for research purposes, practical applications require a fully automated pipeline integrated with the model, eliminating manual intervention. The paper needs to address the feasibility of such an automated pipeline and discuss how the process of generating the tree can be seamlessly integrated into a real-world VideoQA system. The lack of a clear automation strategy limits the scalability and generalizability of the proposed approach. \n  - The current description of the LTR framework suggests its applicability might be limited to specific question types. It remains unclear how the framework would handle more complex or nuanced questions, such as those requiring summarization (e.g., \"Summarize what happened in this video\") or those involving negation in multiple-choice scenarios (e.g., \"Which of the following statements is false?\"). The authors should clearly define the types and scope of questions that the LTR framework can handle and explicitly address these limitations. A discussion of the limitations regarding question types should also be included in the limitations section.\n  - While the reported experimental results show improvements across various metrics, the paper would benefit significantly from richer qualitative analysis. Including more detailed case studies would provide valuable insights into the LTR framework's practical impact. Specifically, the authors should provide examples of:\n    - Questions correctly answered with LTR that were previously answered incorrectly by the baseline MLLMs.  \n    - Questions that remain unanswered even with LTR.\n    - Questions answered correctly by the baseline MLLMs but incorrectly answered after applying LTR.\n  \n    These case studies would provide the community with a deeper understanding of the LTR framework's strengths and weaknesses, facilitating further research and development in this important area.", "other_comments_or_suggestions": "- On page 2, line 83, left column, \"oof\" appears to be a typographical error and should be corrected to \"of.\"\n  - In section 3.2.2, \"Video-Aided Logical Reasoning,\" the provided example refers to \"Figure 2 (red box).\" However, the explanation seems to correspond to Figure 1, not Figure 2. The authors should verify the figure reference and correct it accordingly for clarity and accuracy.", "questions_for_authors": "- The proposed LTR framework presents a compelling approach to incorporating reasoning into VideoQA. However, the authors should elaborate on how the process, particularly the In-Process Answer Verification stage, ensures the correctness of intermediate responses within the reasoning tree. What mechanisms are in place to prevent and handle potential errors during this stage, such as incorrect verification by the model itself? How would the framework perform if an incorrect answer is generated at an intermediate node? Could this lead to a cascading effect, ultimately resulting in an incorrect final answer?\n  - The experiments utilize open-source models. Have the authors explored applying the LTR framework to any closed-source models?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ieN11oESAn", "reviewer": "Reviewer_AFv7", "summary": "The paper introduces Language-centric Tree Reasoning, a framework for VideoQA that hierarchically decomposes complex questions into a logical tree. It first recursively splits questions into perceptual sub-questions using linguistic cues and retrieval-augmented generation (RAG). Then, answers are aggregated bottom-up, guided by video content for verification. Experiments across multiple benchmarks show improved accuracy and interpretability over existing MLLMs.\n\n## update after rebuttal\n\nThank the author for the rebuttal. I will keep my original rating, which was already positive.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for carefully reviewing our paper, acknowledging its strengths, and providing valuable suggestions for improvement\nIf you have any further concerns, please feel free to raise them during the second-round rebuttal phase.\nAs recommended by the official FAQ, we provide all figures and tables via the [anonymous link](https://anonymous.4open.science/r/ICML25-3189-7D31/README.md).\n\n### Domain Adaptability and Finetuning on Domain Specific Tasks\nIn general, finetuning MLLMs in a specific domain significantly enhances performance by providing a more solid basis for reasoning. We further detail finetuning for different modules as follows:\n* Divide with Top-down Recursive Checking: Finetuning is generally unnecessary for this stage since question decomposition primarily relies on language understanding, with visual content serving as a supplementary element where coarse understanding is sufficient for most scenarios. However, when encountering different question distributions or videos with distinct characteristics (e.g., 360-degree videos or extremely long videos), finetuning may offer significant benefits.\n* Perceptual Leaf Question Answering: This step can be readily finetuned using any VideoQA dataset, which would improve the accuracy of leaf question answers and, consequently, enhance subsequent reasoning stages.\n* Video-Aided Logical Reasoning and In-Process Answer Verification: Although finetuning these steps can benefit the framework, it requires additional effort to construct corresponding reasoning data for the specific domain.\n\nTo further validate the effectiveness of finetuning at different stages, we conducted experiments on AGQA-Decomp by finetuning our entire framework, leveraging the compositional graph and detailed answers provided in AGQA-Decomp. In [Table1](http://anonymous.4open.science/r/ICML25-3189-7D31/R-AFv7/Table1.png), we compare the performance of Qwen2-VL and VideoChat2 on AGQA-Decomp with and without finetuning. For the main question, our LTR framework improves the baseline MLLMs by 3%–4%, and the finetuning strategy further boosts LTR by an additional 3%–4%, leading to a total improvement of 7%–8% in accuracy. For sub-questions, the improvement is even larger, 12%–13%, because the Perceptual Leaf Question step is more amenable to enhancement in a specific setting. Moreover, the improvement in terms of $c$-$F_1$ is approximately 15%–16%, largely attributable to the increased sub-question accuracy and the hierarchical information aggregation and logical inference in Video-Aided Logical Reasoning. We thank the reviewer for this suggestion and will incorporate these experimental results and discussions in the following version of the paper.\n\nWe thank the reviewer again for the detailed reviewing, and hope our rebuttal have solved the proposed concerns and would strengthen the reviewer's confidence in judging positively for this paper.", "claims_and_evidence": "LTR enhances reasoning accuracy and transparency in VideoQA by leveraging a hierarchical, language-driven decomposition. This claim is supported by:\n- Quantitative gains of 1%–2% on open-ended tasks and 2%–4% on multiple-choice tasks.\n- Improved compositional consistency metrics.\n- Qualitative examples showing traceable reasoning steps that diagnose errors.", "methods_and_evaluation_criteria": "Methods:\n- Divide Stage: Recursively decompose a complex question into a language-centric logical tree using RAG to ensure semantic coherence.\n- Conquer Stage: Perform video-aided bottom-up tree reasoning to aggregate sub-question answers, verify intermediate results, and derive the final answer.\n\nEvaluation Criteria:\n- Open-ended benchmarks assessed via GPT-3.5 (accuracy and scoring).\n- Multiple-choice tasks evaluated by having MLLMs select from provided options.\n- Detailed ablation studies and qualitative error analyses further validate each component.", "theoretical_claims": "- Hierarchical Reasoning: Mimicking human System-2 reasoning by breaking down complex questions improves logical consistency.\n- Language-Centric Approach: Anchoring the reasoning process in linguistic logic prior to engaging visual evidence reduces bias from overly salient visual cues.\n- Training-Free Adaptability: Integrating pre-trained MLLMs with RAG allows for task-specific reasoning without additional fine-tuning, preserving generalization.", "experimental_designs_or_analyses": "- Dataset Coverage: Experiments conducted on a wide range of VideoQA datasets, including MSVD-QA, MSRVTT-QA, TGIF-QA, ActivityNet-QA, STAR, Ego-Schema, and Video-MME.\n- Ablation Studies: Analysis of each component (e.g., RAG integration, video guidance, answer verification) to show their contributions.\n- Qualitative Examples: Detailed reasoning paths for both successful cases and failure scenarios (highlighting where perceptual errors occur).", "relation_to_broader_scientific_literature": "- The work builds on previous VideoQA methods that utilize visual feature extraction and neural modular networks but distinguishes itself by focusing on interpretability.\n- The authors compare against recent state-of-the-art models (e.g., VideoLLaMA, VideoChat2, Qwen2-VL) and position LTR as an approach that provides a clear reasoning trace.\n- Aligns with literature on hierarchical reasoning and the cognitive basis for multi-hop question answering.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Weaknesses:\nHeavy reliance on MLLMs’ zero-shot capabilities with no finetuning, which may limit adaptability in domain-specific tasks.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Would incorporating fine-tuning for domain-specific scenarios help overcome the limitations of zero-shot reasoning in complex, long-video contexts?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: After the rebuttal, two reviewers have increased their ratings, and all reviewers provide positive ratings (3 Weak Accepts). The mentioned concerns have been addressed. On the other hand, there is no reviewers who "strongly" support this submission, and one reviewer is not an expert in this area. Thus, we recommend **Weak Accept** for this submission if there is room in the program.
Decision: Accept (poster)