AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understanding
Authors: Zhucun Xue
Jiangning Zhang
Xurong Xie
yuxuan cai
Yong Liu
Xiangtai Li
Dacheng Tao
Venue: NeurIPS 2025 poster ()
Keywords: Video Understanding
Multimodal Large Language Models
RAG
Primary area: deep_learning

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated excellent performance in video understanding but suffer from degraded effectiveness when processing long videos due to fixed-length contexts and weaknesses in modeling long-term dependencies. Retrieval-Augmented Generation (RAG) technology can mitigate these limitations through dynamic knowledge expansion, but existing RAG schemes for video understanding employ fixed retrieval paradigms that use uniform structures regardless of input query difficulty. This introduces redundant computational overhead and latency (*e.g.*, complex graph traversal operations) for simple queries (*e.g.*, frame-level object recognition) while potentially causing critical information loss due to insufficient retrieval granularity for multi-hop reasoning. Such single-step retrieval mechanisms severely constrain the model's balance between resource efficiency and cognitive depth. 
To address this, we first propose a novel AdaVideoRAG framework for long-video understanding, which uses a lightweight intent classifier to dynamically and adaptively allocate appropriate retrieval schemes, ranging from the simplest to the most sophisticated, for different video understanding tasks based on query complexity. We introduce an Omni-Knowledge Indexing module to extract valuable information from multi-modal signals for context modeling and build corresponding databases, *i.e.*, a text base from clip captions, ASR, and OCR; a visual base; and a graph for deep semantic understanding. This enables hierarchical knowledge access, integration, and generation from naive retrieval to graph retrieval, achieving an optimal balance between resource consumption and video understanding capabilities.  
Finally, we construct the HiVU benchmark for deep understanding evaluation. Extensive experiments show that our framework enhances the overall efficiency and accuracy of Video-QA for long videos and can be seamlessly integrated with existing MLLMs via lightweight API calls, establishing a new paradigm for adaptive retrieval augmentation in video analysis.

AdaVideoRAG
Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understanding
Zhucun Xuel Jiangning Zhang! Xurong Xie
Yuxuan Caf
Yong Liu Xiangtai Li*
Dacheng Tao*
Zhejiang University
Youtu Lab, Tencent Huazhong University of Science and Technology
*Nanyang Technological University
Code: https://github coo////oo-jj//aaaideeeee
Abstract
Multimodal Large Language Models (MLLMs) have demonstrated excellent per- formance in video understanding but suffer from degraded effectiveness when processing long videos due fixed-length contexts and weaknesses in modeling long-term dependencies. mitigate these limitations through dynamic knowledge expansion, but existing RAG schemes for video understanding employ fixed retrieval paradigms that use uniform structures regardless of input query difficulty. This introduces redundant computational overhead and latency (e.g., complex graph traversal operations) for simple queries (e.g., frame-level object recognition) while potentially caus- ing critical information loss due insufficient retrieval granularity for multi-hop hop reasoning. Such single-step retrieval mechanisms severely constrain the model's 's balance between resource efficiency and cognitive depth. To address this, first propose novel Ada VideoRAG framework for long-video understanding, which uses a lightweight intent classifier dynamically and adaptively allocate appropriate retrieval schemes, ranging from the simplest to the most sophisti- cated, for different video understanding tasks based on query complexity. We introduce an Omni-Knowledge Indexing module to extract valuable information from multi-modal signals for context modeling and build corresponding databases, i.e., text base from clip captions, ASR, and OCR; visual base; and graph for deep semantic understanding. This enables hierarchical knowledge access, integration, and generation from naive retrieval to graph retrieval, achieving an optimal balance between resource consumption and video understanding capabili- ties. Finally. we construct the HiVU benchmark for deep understanding evaluation. Extensive experiments show that our framework enhances the overall efficiency and accuracy of Video-QA for long videos and can be seamlessly integrated with exist- ing MILLMs via lightweight API calls, establishing new paradigm for adaptive retrieval augmentation in video analysis.
Cetrieval-Augmente Generation (RAG) technology can
Introduction
With its powerful multimodal perception and generalization capabilities, the Multimodal Large Lan- guage Model (MLLM) has become universal technical paradigm for addressing diverse scenarios and has demonstrated strong generative capabilities video understanding [31, 48, 29, 1]. However,
"Equal contributions. CCorreppnndigg author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(Ouns)
White
Hodall
Asaraay
Queryy
None Retriess
incomplate inherent knowledge
MILM
A3: Comprehensive resporse incorporstes the video poitent.
A1: White
Straghtforward
A2:
AE: Lack of connection between the front of MideoliAr
Duary
Navie Retricsa
Query
MLUM
Simple Query
AAI: White
A2:6
A3: Comprehensive response that
Crach Retriesi
Fatrinal
incorporates the video content.
MLLM
Straightforward Query
Q2: Straightforward Query
color
many welf appear throughout
10:22
06:00
Q3: Hard Query 05:17
08:10
Impact climate change on gressland animals?
Comparison of different video understanding frameworks: i) MLLMs are efficient but can only handle simple problems. ii) VideoRAG [32] integrates external knowledge via naive retrieval but still struggles with hard reasoning questions. iii) Recent VideoRAG [36] tackles complex problems using graph retrieval but suffers from low efficiency. Our novel Ada VideoRAG framework adaptively routes queries to different retrieval paths via query intent classification, achieving better trade-off' between effectivvness and efficiency.
Figure 1:
when applied to specific domains, it is constrained by challenges such as knowledge solidification (inability to dynamically update the latest knowledge), uncontrollable reasoning (risk of hallucina- tions), and weak generalization (requiring additional fine-tuning costs and time costs), making it difficult handle multi-hop question and cross- moss-modal association requirements (especially long Generation
video scenarios), which leads to performance degradation [36, 32]. EetrievallAuggeeee (RAG), by integrating the collaborative reasoning of external knowledge bases and generative models without being confined to pre-trained knowledge, can easily adapt to private domain data scenarios and has become a core paradigm for improving the factual accuracy and domain adaptability of large language models.
Current RAG research mostly focuses on text modality [25, 10, 19], static images [7]. and tabular forms [6]. overlooking the unique value of video as a multimodal knowledge carrier. The increasingly popular long-video understanding has put forward new demands for RAG models supporting video modality input. Most existing RAG studies on long videos attempt to enhance question-answerte answering generation by constructing and retrieving knowledge bases from multimodal information derived from videos. For example, Luo et al. [32] incorporates iisually-aligne auxiliary text features from optical character recognition (OCR), automatic speech recognition (ASR), and object detection to create video knowledge bases, enabling question crestion-answering for long videos. However, this method does not support sensemaking queries or multi-hop questions, which require global understanding of the entire database as shown in Fig. 1. Recent VideoRAG [36] significantly improves the accuracy of long-video contextual information by constructing a graph database, but it requires maintaining a hierarchical graph database that demands substantial computational and time resources, and incurs higher costs when migrating to new scenarios. We believe that a practical RAG for video understanding needs to flexibly allocate appropriate processing methods for different videos and query difficulties, which both maintains accuracy and improves efficiency.
Considering that real-world video understanding tasks involve content comprehension needs of varying complexity, the problem-solving strategies for questions of different difficulty levels will have distinct priorities. Short-video QA involving simple common sense does not require retrieval and can directly obtain correct answers by querying the MILM, while complex long-video questions rely on RAG for retrieval to filter effective information. For more complex questions, such as those requiring multi-step reasoning or relying on multiple types of knowledge graph based RAG is necessary to derive correct answers. Therefore, one-size- mnc-size-fits-lll approach of retrieving and then returning results

---

is not optimal. To address this, this paper proposes an Sapptive-RA--baaa video understanding scheme termed Ada VideoRAG, as shown in Fig. 1. It first classifies user queries into difficulty levels and then adaptively assigns the most reasonable retrieval strategy based on the difficulty. Additionally, we further integrate visual features, clip captions, ASR, and scene text composite information flows contained in videos, and use relevant text information obtained from external retrieval for data augmentation. According to the difficulty of questions, queries are routed to different levels of database retrieval modes (i.e., naive and graph retrieval). These multimodal knowledge inputs and retrieval strategies can more effectively provide fine-grained contextual representation capabilities, ultimately further enhancing the upper limit of MLLM's processing capabilities for long videos and complex ineestion-nswerrn tasks.
To demonstrate the effectiveness of the proposed Ada framework, we officially release HiVU, the first open benchmark dataset for full-stack capability evaluation in video understand- ing. This dataset groundbreakingly integrates 120 video samples covering continuous duration spectrum from short clips (1 minute) extra-long videos (106 minutes), spanning high-frequency scene categories across three major themes: knowledge education (lectures, finance, law, psychology. documentaries), information (news, interviews), and entertainment (sports, cooking, makeup, fitness, TV dramas, animations). In terms of question design, we innovatively develop three-level difficulty quantification system: 1) Basic Level-1 (L1) focuses on frame-level content perception (e.g., "Which objects appear at the 5th second of the video?").. 2) Advanced Level-2 (L2) requires temporal logic reasoning (e.g., "When does the speaker start explaining graph neural networks?"). 3) Expert Level-3 (L3) challenges cross-modal causal inference (e.g.. "How would deleting the narration at the 15th minute affect the plot development?"). Compared with traditional datasets such as ActivityNet [2] (single action recognition) and MovieQA [37] (open-ended QA), this benchmark achieves, for the first time, cognitive complexity evaluation at different levels, providing a hierarchical evaluation framework for video understanding research. It supports systematic optimization of models in long- video modeling, complex reasoning tasks, and real-world scenario generalization. In summary, our contributions are as follows:
1) We propose a novel VideeRAA framework that dynamically and adaptively routes ap- propriate retrieval schemes, from the simplest the most sophisticated, depending on the query complexity. This approach aims to achieve an optimal balance between resource consumption and video understanding capabilities for different video understanding tasks.
2) We introduce an multi-modal signals for context modeling and establish corresponding databases. A lightweight intent classification model is used to determine the difficulty level of input queries, enabling hierarchical knowledge access, integration, and generation from naive retrieval to graph retrieval, while balancing resource consumption and video understanding capabilities.
Indexing module to extract valuable information from
3) We publicly release the hierarchical video understanding benchmark HiVU for the first time, which evaluates the multi-level reasoning capabilities of video understanding models. Extensive comparative experiments and ablation studies demonstrate the advantages of Ada VideoRAG in deep understanding of long videos.
2 Method
We introduce MLLM-centric adaptive RAG framework for long-video understanding termed Ada VideoRAG, which can significantly improve efficiency while ensuring accuracy. As shown in Fig. 2, method includes four parts: 1) Query Intent Classification (Sec. 2.1). 2) Omni-Knowledd Indexing (Sec. 2.2). 3) Adaptive Retrieval Paradigm (Sec. 2.3). 4) Integration and Generation (Sec. 2.4).
2.1 Query Intent Classification
Not all user requests have the same level of complexity. For simple user requests, we can use straightforward solution to reduce computing power consumption and users' perception of latency. For complex questions, we rely on complex multi-model, modai-modal, and multi-step queries to achieve higher accuracy. To achieve the above goals, we propose to use lightweight intent classification model to perform the classification of the difficulty level of the query at the input end. Specifically, we have defined and established a fine-grained evaluation system for the difficulty level of video understanding:
3

---

Auxiiaary Text Extraction
Text Base
Adaptive Retrieval Sec. 3.3
Vision-Base
P
VLM
Enc.
Reso
ASR
2
User Query (Q)
Integration and
Number
Generation Sec. 3.4
Intent LLM Soc. 3.1
player perform?
OCR
Response (R)
The Number Stephen Curry He regained the same point point shot in this game...
Omni-Knowledge Indexing
G
three-
Sec. 3.2
Knowlodge Graph
Figure 2: Overview of our Ada VideoRAG framework that consists of: 1) Query Intent Classifica- tion (Sec. 2.1). 2) Onni-Knowledge Indexing (Sec. 2.2). 3) Adaptive Retrieval Paradigm (Sec. 2.3). 4) Integration and Generation (Sec. 2.4).
Level-1: Straightforward reasoning. There are basically few logical relationships involved in the questions, and the knowledge required for answering questions is directly provided in the video content. For example, "What color of clothes is the woman who appears at the fifth second wearing?" For such questions, the existing MLLMs models are already very mature in solving them. If complex processing is still applied to such simple queries, will result in unnecessary computational overhead.
Simple reasoning. It involves single-step reasoning about basic spatio- spatio-temponn temporal/causal relationships, requiring the model to establish logical associations between local events. For example, "Why did the woman cry before the rainy scene started?" requires two-stage reasoning: 1) Determine the starting time point of "rain" through temporal positioning: 2) Retrieve the character behaviors (such as the audio of an argumentative conversation) and scene changes (such as weather forecast subtitles) before this time point, and construct a causal chain explain the motivation. Such tasks expose the integration flaws of existing MLLMS methods regarding cross-modal temporal clues, and are prone to the lack of key intermediate evidence due to the mismatch retrieval granularity.
Level-2:
Level-3: Hard reasoning. The video understanding at the highest difficulty level requires extracting different subjects and relationships from the long-context and constructing a knowledge graph that maps entities and relationships across temporal and semantic dimensions, and combining it with powerful MILLM reasoning capabilities to make judgments. For example, "What life lessons does this movie convey?" Questions of this kind require the model to mine the deep semantic relationships provided in the video and conduct multi-hop reasoning to obtain the correct answers.
Intent classification model. Given the basic definitions and examples from level 1 to level 3, we use large language model with Chain-of-Thought (CoT) reasoning to classify the query Q. This can be integrated into a RAG (Retrieval-Augmentt API, providing intent classification results through appropriate prompts without the need for fine- tuning. Based on the classification results, it can automatically trigger progressive knowledge retrieval strategy, ranging from none retrieval to simple naive retrieval, and further to complex graph retrieval. The calculation of the intent classification result L can be formulated as: L = L.L.entent (Q, promptintent). where the LLM is a lightweight CoT model. In this paper, we adopt Qwen2.5-7B [18, 45), whose time-consuming proportion is extremely small (averagely <5%) compared to the entire process.
Generation) architecture as a plug-and-play
2.2 KnoweeKn Indexing for Long-Contex. Understanding
When performing video understanding tasks, MLLMs equipped with RAG can achieve context modeling through dynamic activation of external knowledge bases, which alleviates the window length limitation of long contexts to some extent and enhances the semantic understanding of global videos. To this end, we propose the Onnil-nn Indexing module, which extracts valuable information from multiple modal signals for context modeling and establishes corresponding databases, enabling the RAG system to more accurately retrieve the most relevant information and perform high-quality generation.

---

2.2.1 Omni- Omni-Knowledge Text-Base Establishment
In long video understanding tasks, due to the context window size limitations of MLLMs, we need to perform frame sampling and resizing on videos under hardware constraints. However, this inevitably leads to the loss of rich visual information in the videos, as well as unused audio and text multimodal information. Therefore, we utilize an external normalization module to extract multimodal information from videos and construct our private text base.
Auxiliary text extraction and database construction. The input long video V is divided into N consecutive and semantically complete clips V (C1, C2..... CN) { at fixed time intervals (30 seconds per clip in the paper). For each clip Cz. uniform frame sampling is performed to extract key frames. In this paper, we select 5 frames as the multimodal representation primitive Fn. as more frames do not significantly improve performance but increase computational power and model complexity. Specifically, auxiliary text extraction includes three categories: 7) The quantized MiniCPM-V V [46] (used as the VLM model) generates fine-gratee fine-grineed text descriptions Te for the sampled frames, including semantic elements such as character attributes and spatio-temporal actions, ultimately constructing caption database De: 2) Audio the most direct information carrier in videos, driving story development, conveying plot clues, and revealing character relationships through language, providing information that cannot be mined from visual features alone. Therefore, we use FastWhisper [33] as the audio extractor to convert the audio in each clip into text format T'A, which is stored as vectors via an embedding model to generate an ASR database DA; 3) Characters To in each frame extracted through EASYOCR [22], and an OCR database Do is constructed to compensate for the insufficient recognition ability of MLLMs.
Knowledge graph construction. To address Level-3 complex reasoning queries. we construct a knowledge graph based on clip captions (Tc), ASR (Tx), and OCR (To). Specifically, BGE-M3 [5] extracts entities and relationships from text chunks: 1) Entity represents the minimal domain- specific semantic interpretation unit the video, characterized by a triple-entity type, entity name, tratio-temporal attribute>. 2) Relationship encompasses various semantic associations between entities, including apatio-tempora relationships, causal relationships, functional relationships, etc., to systematically structure video text information.
2.2.2 Vision-Base Establishment
Simply relying on text information extracted from clip captions, ASR, and OCR makes difficult to construct an optimal Knowledge Indexing. As typical carrier of multimodal data, videos contain visual features with abundant details that are hard to describe precisely in text, such as object appearance changes, scene spatial layouts, and human facial expressions and movements. These visual information play an indispensable role complex knowledge reasoning and retrieval tasks. Therefore, we introduce the ImageBind [16] image encoder (Enc. in Fig. 2) to extract features from key frames and concatenate them as the final features, because this model is based on advanced cross-modal alignment algorithms that can map heterogeneous modal data such as images, text, and audio into the same high-dimensional semantic space.
2.3 Adaptive Retrieval Paradigm
After intent classification (Sec. 2.1), ), the user query (Q) is routed to different retrieval paths accord- ing to its difficulty level, so as improve comprehensive efficiency on the premise of ensuring effectiveness.
None retrieval with direct MILM. For Level-1 scenarios, the model directly feeds the query (Q) and the entire video {C₂} into the MLLM to obtain a direct response. This approach leverages the inherent knowledge and reasoning capabilities of the MLLM without introducing external knowledge bases, significantly enhancing overall efficiency for simple questions.
Naive retrieval with simple reasoning. For Level-2 retrieval scenarios, this study proposes a multimodal collaborative grounding framework that significantly enhances the retrieval efficiency and accuracy of long videos in handling simple logical questions by jointly optimizing the semantic alignment between auxiliary texts (clip captions, ASR, OCR) and visual modalities. Specifically, we first decouple the original query into sub-queries adapted to different modal characteristics: /) For clip caption retrieval, we rewrite the query into declarative sentences, remove option interference, and add approprate descriptive content. 2) For Aec-recog text, extract colloquial
5

---

expressions from the query, retain core actions and events, and add contextual modifiers to match fragmented speech segments. 3) For discrete OCR text, we extract specific entity information from the query. A typical example: when the input query is "How did the Number 30 player perform?". ?", the rewritten outputs are: i) "clip caption": "The performance of Number 30 player.; ii) "ASR text": "How's 's the number 30 player doing.", " iit) "OCR text": "Number 30 player". Query rewriting effectively mitigates distribution shifts between different semantics. Through cross-modal similarity calculation, we can then quickly locate query-relevant candidate content and the corresponding video clips for each text block.
This study further locates and queries the semantically most relevant video content from the visual feature database Dy. Specifically, our model reuses the rewritten results of clip captions as semantic anchors for visual retrieval. The pre-trained cross moosl semantic alignment encoder ImageBind [16] employed to map videos into the text embedding space {. J. By calculating the cosine similarity between text and visual embeddings, candidate segments with similarity scores exceeding a threshold (set to 0.5 in this paper) are filtered out. These segments are then ranked to retain the top- visual evidence with the highest confidence. This approach significantly reduces the modality gap in visual- text alignment by leveraging a unified semantic embedding space, effectively alleviating the problem of local detail loss in long videos. Finally, the videos {C,, } retrieved through visual feature-text space alignment are merged with the video chunks {Cz.a.o } located via auxiliary text retrieval to construct retrieval evidence pool for simple reasoning at Level-2.
Graph retrieval in hard reasoning. Relying solely on information obtained from auxiliary text and visual feature retrieval falls short in enabling MILLMs to tackle more complex sensemaking query scenarios. Therefore, we require more abundant and semantically precise auxiliary information capable of modeling multiple events and temporal nodes to facilitate MLLM reasoning. To address this challenge, we adopt a deeper retrieval approach based on Light-RAG [19] to handle hard queries, replacing the naive retrieval method used for simple queries. Specifically, considering resource constraints, we reuse auxiliary text embeddings to construct a graph. We then compute similarity scores between rewritten clip captions and entity/relationship descriptions, returning the most relevant entities and relationships. Within the graph map, we gather other information associated with the retrieved entities and relationships. which can be combined into query query-centered thinking map. This retrieved graph map assists MLLMs in considering global and multi-layered information, aiding in better modeling spatio-temporal and causal relationships within events. Furthermore, we employ unified semantic embedding space to represent visual evidence obtained from grounding, enhancing retrieval accuracy. We overlay the retrieved videos (Ce) } with graph retrieval results (Cg) } to construct a multi-level retrieval evidence pool for hard reasoning under Level-3.
Filtering then sorting evidences. After obtaining the preliminary retrieval results, we perform coarse-to-fine information purification on the search results. First, we filter out duplicate video information blocks retrieved from different databases. Then, the content description of the video blocks (including clip captions, ASR, and OCR texts) and the query are simultancously input into small-scall LLM (Qwen2.5-7B [45, 18] in the paper) for fine-grained filtering to exclude some irrelevant search results. Finally, we rerank the selected video clips based on the order of original video time to preserve temporal causal relationship information.
2.4 Multimodal Information Integration and Generation
To provide MLLMs with more comprehensive information for enhancing query accuracy. we acquire auxiliary text information (denoted as (Te.a.o} } for simple reasoning and (T,) for hard reasoning) derived from clip captions. ASR, and OCR contexts, along with visual information {C²} from visual- to-text grounding. After integrating the retrieved context and corresponding video clips (Cea.s). J. the combined inputs are fed into MI.LMMs for reasoning and generation to produce the final output R:
MLLM({C.)+ Q)
if L is Level-1,
R= MLLM(1C.), (Cc.,ot, (T Q) MLLM((C.), (Co,,o), (To.o)) (C,). (T,).)
if L is Level-2, if L is Level-3.
2.5 HiVU: Hierarchical Video Understanding Benchmark
Existing video understanding datasets either have insufficient duration [14] or lack engaging con- tent [43, 53], failing to generate queries that require deep comprehension. To support robust reasoning

---

tasks on long videos and evaluate different methods, we constructed the Hierarchical Video Un- derstanding (HiVU) Benchmark. For this purpose, we selected three genres: knowledge-cducation (lectures, finance, law, psychology, documentaries), information (news, interviews), and entertainment (sports, cooking, makeup, fitness, TV dramas, animations). We manually collected 120 long-vidoo datasets rich in knowledge content from YouTube, totaling 60 hours, with distributions shown in Fig. 3. Additionally, we designed three tiers of query reasoning from straightforward to hard, as described in Sec. 2.1. This hierarchical query design enables comprehensive and detailed evaluation of models' reasoning capabilities across varying difficulty levels.
Evaluation metrics. For the open-ended question- answering tasks on the HiVU dataset, we draw in- spiration from the Win-Rate metric system widely used in the RAG field to evaluate model capabili- ties [10, 19]. Specifically, we use large language mod- els (LL.Ms) as the judgment basis, quantify the com- parative results of the two schemes through model outputs, and finally present their competitive scores in percentage form. The Win-Rate Comparison com- prehensively considers queries from five dimensions: 1) Comprehensiveness: This dimension focuses on whether the model's response fully covers the query. avoiding missing critical information providing one-sided answers. 2) Empowerment: It primarily examines whether the model's response can This dimension emphasizes the reliability and authenticity of the model's 's output content. 4) Depth: It assesses whether the model can go beyond surface phenomena, uncover the essential issues behind the query, and conduct in-depth analysis and discussion. 5) Density: It focuses on the information content and compactness of the model's 's response, avoiding verbose, empty, or redundant expressions.
count
videos
50 hours
3e ninutes averagely
minutes ninin.r
Figure 3: Statistical distributions of our
HiVU from different perspectives.
provide practical value and inspiration to users. 3) Trustworthiness:
Experiments
3.1 Experimental Setup
We conduct comprehensive evaluations of the proposed VdaVideoRAG method and the effectiveness of each module primarily on the newly proposed HiVU benchmark Sec. 2.5, and also introduce public video understanding benchmarks for further thorough assessment. Specifically: /) HiVU includes over 10 sub-genres across 3 domains, comprising 120 knowledge-rich long-video datasets totaling 60 hours. 2) Video-MME [14] is full-spectrum multi-modal evaluation benchmark for MLLMs in video analysis, featuring diverse videos and multi-modal data. It contains 900 videos (ranging from 11 seconds
hour, categorized into short, medium, and long), with 2,700 multiple- choice questions covering 6 major visual domains (e.g., knowledge, film, sports) and 30 subdomains, focusing on evaluating the perception, reasoning, and summarization capabilities of multimodal large language models (MLLMs) in video analysis. 3) MLVU [53] is multi-task benchmark for evaluating long-video understanding with diverse genres and extended durations. Centered on long videos ranging from 3 minutes to over 2 hours (average 12 minutes), it sets 9 multi-tasks (e.g., single/multi-den detail understanding) across diverse video types (films, surveillance, games, etc.), aiming to comprehensively assess long-video understanding capabilities.
3.2 Experimental Results
Improving open-sourced MLLMs with AdaVideoRAc on MLVU_tes [53] benchmark. The overall evaluation results of all the investigated multi-modal large language models in the MLVU test set are shown in Tab. 1. These results cover the baseline model, Video-LLaVA [29], along with two highly regarded open-source models released recently: Qwen2.5-VL. series [1] and VideoL- LaMA3 [47]. The evaluation results clearly demonstrate that the Ada VideeRAG strategy we proposed significantly improves the question-answering accuracy each MILM. And it particularly stands out in two key types of tasks. Firstly, in tasks such as Topic Reasoning (TR) that require multi-hop reason- ing about videos, and secondly, tasks like Action Count (AC) that involve holistic reasoning. This indicates that Ada Vid can not only strengthen the basic question -nnwwerin ability but also
7

---

effectively assist the MLLMs in achieving breakthroughs within complex reasoning and multi-detail processing tasks. It is worth noting that although the Qwen2.5-VL model that performs relatively weakly on the MLVU dataset, it exhibits more pronounced accuracy improvements after adopting our Ada AdaieRoR increasing nearly by 40% and even reaching the accuracy of arge-paramete models like Qwen2.5-VIL-32B, What's more, the open-source model VideoLLLaMA equipped with Ada Vide- oRAG, even though it has fewer parameters than Swen2.5-VL-32BB shows better performance on long videos, and its performance can even be comparable to that of GPT-4o. These experimental results fully verify the universality and effectiveness of Ada VideoRAA in enhancing the reasoning ability of MLLMs.
Table 1: Comparison between supervised baselines and whether Ada VideoRAG is configured on MLVU test. Frames: the sampling frame rate or the number of images limited, and "2fps-768""
indicates that videos are sampled 2 fps and the upper limit is 768 frames; M-Avg: the average
performance of multiple-choiee tasks.
Params
Frones
AR
NQA
PQA
SQA
TQA
AVG
Gain
GPT-40
0.Sips
83.7
68.8
42.9
47.8
57.1
63.6
46.2
35
48.7
54,9
ViaVo
7B
8
64.4
35.9
25.4
34
26
25
13.1
16.9
23.8
29.1
Vido-LLaVV AddVideoRAG
7B
73.9
33.1
46.2
38
41.9
31.3
21.2
16.9
38.5
37.9
28.9%
Qwen2.5.VL.
7B
20ps-708
46.7
15.4
16.9
35.8
38
38.9
24.6
13.6
29,0
Qwen2.5.VL+ AddiidooRAG
7B
268
20.8
44.1
37.7
36.1
33.3
15.3
40.5
40.5
Quen2.5-VL.
72B
20p
73.9
33.3
59.3
47.3
41.7
37.7
16.9
26.2
41.7
Owen2 V. AdaVide
72B
25ps-708
82.2
41
54.2
41.5
44
47.2
35.1
15.1
45.
45.1
8%
Video.L.MM3
7B
1ps-180
76.9
43.6
68.3
54.7
58
34.3
25
33.3
34.9
47.7
V MdVideeRAGG
7B
18p--180
83.8
47.1
69.2
62.3
64
38.9
34.8
35.6
42.9
53.2
11.6%
Comparison with state-of-the-an VideoRAG [32] on Video-MME [14] dataset. Given that the experimental results in Tab. 1 have fully verified that Ada VideoRAA can effectively enhance the reasoning performance of MLLMs, select the VideoLLaMA3 and the Qwen2.5-VL-7B model as the basic model for subsequent control experiments, which with the same number of parameters. In Tab. 2, we conduct horizontal comparative test between Ada VideoRAG and Video-RAG [32] on the Video-MME dataset. The experimental results show that both RAG methods can significantly enhance the video understanding ability of the base MILMs. However, in tasks involving the processing of long videos, our Ada AG demonstrates a more distinct advantage. This is mainly due to the fact that Ada VideoR AG is capable of constructing more complex and reasonable knowledge map during the retrieval of long videos, thus enabling precise understanding and efficient reasoning of video.
Table 2: Comparison between Vid and VideoRAG [32] on Video-MME [14] dataset.
Model
Params
Frumes
Short
Medium
Overall
Gain
GPT-Mo
384
30
70.3
65.3
71.9
Owen22..VVL
78
2ips 765
55.6
47.1
38.8
412
Qwen2.5-VL VideoRAG VideoRRGG [32]
7B
32
70.3
51.5
43.3
55.0
+7,9
Qwen2.5.VL. VideoRAG
7B
2fps-768
72.8
112.7
VidcolLLaMA
78
11p-188
76.7
62.8
53.2
64.2
Videol I aMA3 VideoRAG [32]
78
32
81.5
63.3
57.1
67.3
3.1
VidcoELaMA3 AdaVidaoRAG
7B
18ps-180
4.3
Impact of LLM arbiters. To explore the performance of the retrieval strategies in sensemaking tasks of varying difficulties, we conducts comparative experiments based on the proposed hierarchical video understanding benchmark(HiVU), and an LLM is then used as the evaluation referee assess the quality of the final answers. Regarding the selection of specific LLMs, we carry out two sets of control experiments: Deepseek-RI R1-7B [45, 18) and Deepseek-R1-33 R1-2BB [45, 18), Qwen2.5-32B [45] and QwQ-32B [39], which represent the models with different parameters and reasoning capabilities respectively, as illustrated in Tab. 3. The experimental results demonstrate that models with larger parameters and equipped with the Chain-of- Thouuhh (CoT) reasoning mechanism exhibit stronger discriminatory abilities when evaluating the performance other models. Based on these findings, we choose DeepSeek-333 model as the evaluation arbiter for HiVU benchmark evaluation to ensure the accuracy and reliability of the evaluation results.
Comparison with state-of-the-art VideoRAG [32] on HiVU dataset. In our Hi data benchmark, there are three tasks classified according to the difficulty of reasoning: straightforward (L1), simple (L.2), and hard (L.3). For different levels, Ada VideoRA AG employs different retrieval strategies: from without retrieval, naive retrieval, to graph retrieval, which forms a hierarchical enhancement mechanism. And the following series of experiments are designed to verify the improvement of
8

---

Table 3: Impact of LLM arbiter configurations (parameter scale and reasoning capabilities) HiVU benchmark evaluation.
Metric
VideoLLaMA3
Despeekk78
ViMooLLMMA3 w/ AddYooo
VideoLLaMA3
Despaeck-328
VidtoLLaMA3 w/ Aideoide
VideoLLaMA3
Quen2-5-337
of AdVioooo
Videol LAMAT
VideoL.LaMA
QWQ-128
w/ AdaaVieoRRA
Comprehensivenees
46.155
33.9%%
35.98%
64,022%
45.42%
34.38%
52.27%
67.73%
Empowerment
33.33/5
66.67%
30.83%
69.12%
39%
61%
30.11%
19.89%
Truswonthiness
40.7556
59.25%%
30.58%
69.42%
40%
68.74%6
Depih
71.46%
26.23%
73.77%
37.85%
62.15%
Denaity
Overall Winner
32,1256
67.88%
30.58%
69,42%
38.71%
61.29%
30.6956
69.3146
reasoning ability, as shown in Tab. 4, the hard-level video understanding task, the multi-modal large language model integrated with Ada VideoRAAG demonstrates more significant advantages compared to original model, and the gap between the two becomes more evident as the task difficulty increases. This result not only confirms the effectiveness of Ada VideoRAG in complex reasoning scenarios but also indirectly validates the rationality and scientific nature of the three-level difficulty division in the HiVU benchmark, providing reliable basis for quantitatively evaluating the reasoning ability of models.
Meanwhile, we conducted horizontal comparison with VideoRAG [32] in the Hi benchmark, as shown in Tab. 4. Consistent with our expectations, AdaVideoRAG is on par with VideoRAG [32] at the Level-1 and Level-2 levels. However, our method exhibits more prominent advantages at the Level 3 which need global and multi-hop reasoning.
Table 4: Performance on HiVU, Left: Results comparison w/o and w/ Ada VideRRAGG Right: Results comparison w/ VideoRAG [32] and Ada VideeRAG
Truis
WeetLa
4000
(4)
43779
36,901
870
47215
21%
54,75
Tranvendioo
15.115
195
73 45%
3.46%
09425
66115
$1.3
43276
46.5
$3.05
71271
12.77
71227
Ablation Study In the following analysis, we perform three ablation studies to precisely assess the key components of our proposed method. They are as follows: 1) Without Graph: We cancel the retrieval of entities and relationships in the graph map: 2) Without vision retrieval: We remove the feature retrieval in vision-to- texiett grounding: 3) Without naive text retrieval: We cancel the the retrieval from the caption, OCR, and ASR databases, shown in Tab. 5. It can be that the design of each module is effective and can improve the understanding ability of the model.
Table 5: Ablation on graph-based knowledge retrieval, vision-based embedding retrieval and auxiliary text retrieval components.
Metrie
w/o Graph
All
w/o Vision
All
w/o Text
All
Compohenssveeess
38.92%
61.08%
50.13%
49.87%
33.17%
66.83%
Empowermnnt
47.79%
$2.11%
48.42%
51.58%
40.53%
59.47%
Trustworhiness
39.79%
60.21%
Depih
46.31%
$3.695
49.47%
50.53%
30.33%
69.67%
Density
51.73%
48.27%
46.84%
53.16%
35.36%
64.645
Overall Winner
45.82%
34.18%
48.23%
51.77%
31.23%
68.75%
Impact of Query Classification. We conducted two experiments to examine how query classification affects retrieval and reasoning performance. First, compared different classifier models in terms of classification precision and downstream results. Second, we tested the effect of removing the classifier by routing all queries through the same difficulty level.
(1) Classifier Comparison. We evaluated several models on HiVU (with ground-trui -ruth Level labels) and the VideoMME benchmark. As shown in Table 6, Qwen2.5-7B achieved the best balance between classification accuracy and overall performance.
(2) Effect of Removing the Classifier. We further examined the role of query classification by disabling the classifier and forcing all queries to follow single retrieval path. When all queries were treated as Level-1, Level-2, or Level-3, the overall scores on the VideoMME benchmark dropped to 64.2, 67.5, and 1, respectively. In contrast, our adaptive approach (Ada VideoRAG) achieved the

---

Table 6: Performance of different classifier models.
Classifier
Precision (HiVU)
Overall Score (VideoMME)
Qwen2.5-1.5B
0.41
65.3
Qwen2.5-7B
0.81
68.5
Maleol..aMM3-75
0.48
67.5
highest score of 68.5, demonstrating that dynamic query routing effectively balances reasoning depth and efficiency.
3.3 Efficiency Analysis
To evaluate the efficiency of different retrieval paths, we conducted time-cost experiments on 100 randomly selected videos from the MLVU dataset under three settings: no retrieval (Level-1), simple retrieval (Level-2), and hard retrieval (Level-3).
(1) Database Construction. Level-1 performs direct inference without database construction. The average construction time for Level-2 and Level-3 is 351 s and 412 s, respectively, with the additional cost in Level-3 attributed to graph construction. The adaptive scheduling strategy of Ada VideoRAG effectively reduces such high-cost operations by prioritizing simpler retrieval paths when applicable.
(2) Single-Process Inference. On single H20 GPU, the average response times are (Level-1), 26 s (Level-2), and 27 s (Level-3). Ada AdaVideoRAt achieves a balanced trade-off between accuracy and efficiency, with an average response time of 20 s.
(3) Parallelization. To further improve deployment efficiency, we introduced multi multi-process and multi-GPU parallelism. Using dual processes single H20 GPU (96 GB, batch size 2). database construction for Level-2 and Level-3 achieved ~ 2x acceleration, reducing the time to 176 s and 210 s. Scaling to 8 GPUs yielded near-linear speedup (~ 8x), cutting construction time to 22 and 26 s. For multi-process retrieval on a single GPU, Level-2 and Level-3 achieved ~ 2 acceleration, with total processing times of 15 and 16 s, compared to 20 s for Add
4 Conclusion
Our Ada VideoRAG demonstrates outstanding performance advantages when dealing with difficult video understanding tasks that require multi-hop thinking and deep reasoning. Meanwhile, Ada Vide- oRAG can efficienlly integrate onni-information, fully leverage the value of multi-source data such as images, videos, and texts. can also flexibly switch between basic semantic understanding tasks, effectively balancing efficiency and accuracy, and greatly enhancing the generalization ability and application universality of multi-modal large language models.
and high-order
Limitations, broader impact and social impact. Due to the limited computational resources for local deployment, this study only evaluates models up to 32 billion parameters. Moreover, the research only implemented three levels of routing, while real-world applications may require more detailed classification. From social impact perspective, this technology could pose new risks such as spreading false information, for example, using generated fake videos to manipulate public opinion. This highlights the urgent need to establish collaborative governance system that includes technical ethics, legal regulations, and industry standards.
Acknowledgments and Disclosure of Funding. This work was supported by the "Leading Goose" Key R&D Program of Zhejiang Province (2025C01069).
10

---

References
[1] S. Bai, K. Chen, X. Liu, J. Wang, W. Ge. S. Song, K. Dang, P. Wang. S. Wang, J. Tang, et al. Qwen2.5-vl technical report. arXiv preprint 2tti::2502.1392333 2025. 1,
[2] F. Caba Heilbron, V. Escorcia, B. Ghanem, and J. Carlos Niebles. Activitynet: A large-scale video benchmark for human activity understanding. In Proceedings of the ieee conference on computer vision and pattern recognition, 2015.
[3] D. Caffagni, F. Cocchi, N. Moratelli, S. Sarto, M. Comia,, L. Baraldi, and R. Cucchiara. Wiki- llava: Hicrarchical retrieval-augmentee augmented generation for multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1818-1826, 2024.
21
[4] Y. Cai, J. Zhang, H. He, X. He, A. Tong, Z. Gan, C. Wang, and X. Bai. Llava-kd: A framework of distilling multimodal large language models. arXiv preprint arXiv:2410. 16236, 2024. 21
[5] J. Chen, S. Xiao, P. Zhang, K. Luo, D. Lian, and Z. Liu, Bge m3-embedding: Multi-lingual, fultiiiunncionnality.. multi cialtiigaaullrityy text embeddings through self knwwlegw distillation. arXiv preprint ar 240v.2200.0022 2024. 5
[6] S.-A. Chen, L. Miculicich, J. Eisenschlos, Z. Wang. Z. Wang, Y. Chen, Y. Fujii, H.-T. Lin, C.-Y. Lee, and T. Pfister. Tablerag: Million-token table understanding with language models. Advances in Neural Information Processing Systems, 2024. 2, 22
[7] W. Chen, H. Hu, X. Chen, P. Verga, and W. W. Cohen. Murag: Multimodal generator for open question answering over images and text. arXiv preprint arXiv:2210.0292. 2022. 2, 22
[8] Y. Chen, S. Ji, H. Wang, 7. Wang. S. Chen, J. He, J. Xu, and 7. Zhao. Wavrag: Audio-integrated retrieval augmented generation for spoken dialogue models. arXiv preprint urXiv:2502.1472) 2025. 22
[9] Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu, et al. Internyl: Scaling up vision foundation models and aligning for generic disual-linguistc tasks. In CVPR, 2024. 21
[10] D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, D. Metropolitansky, R. O. Ness, and J. Larson. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404. 16130, 2024. 2, 21
[11] Fan, J. Wang, X. Ren, and C. Huang. Minirag: Towards extremely simple retrieval- augmenttd generation. arXiv preprint arXiv:2501.06713 2025. 22
[12] W. Fan, Y. Ding, L. Ning, S. Wang, H. Li, D. Yin, T.-S. Chua, and Q. Li. A survey on rag meeting llms: Towards auggenell large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 6491-6501, 2024.
21
[13] Y. Fan, X. Ma, R. Wu, Y. Du, J. Li, Z. Gao, and Q. Li. Videoagent: A memory--ugmenteed multimodal agent for video understanding. In European Conference on Computer Vision, pages 92. Springer, 2024.7 21
[14] C. Fu, Y. Dai, Y. Luo, I.. Li, S. Ren, R. Zhang, Z. Wang, C. Zhou, Y. Shen, M. Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal lims in video analysis. arXiv preprint 4rXiv:2405.2107
2024. 6, 7,
[15] Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, H. Wang, and H. Wang. Retrieval- augmented generation for large language models: A survey. arXiv preprint arX1v:2312.10997, 2:1, 2023. 21
[16] R. Girdhar, A. El-Nouby, Z. Liu, M. Singh, K. V. Alwala, A. Joulin, and I. Misra. Imagebind: One embedding space to bind them all. In Proceedings of the IEFF/CVF Jonfeeececonference on computer vision and pattern recognition, pages 15 15180-15199 2023. 5, 6

---

[17] A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, et al. The llama herd of models. arXiv preprint arXiv:2407.21783, 2024. 21
[18] D. Guo, D. Yang, H. Zhang, J. Song. R. Zhang, R. Xu, Q. Zhu, S. Ma. P. Wang, X. Bi, et al. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint 25X1:22501.12948 2025. 4, 8
[19] Z. Guo, L. Xia, Y. Yu, T. Ao, and C. Huang. Lightrag: Simple and fast retrieval-augmmnned generation. arXiv preprint arXiv:2410.0577,, 2024. 2,
[20] Y. Hu and Y. Lu. Rag and rau: A survey on retrieval- language model in natural language processing. arXiv preprint arXiv:2404. 19543, 2024. 22
[21] A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda,
A. Hayes, A. Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410021276, 2024. 21
[22] JaidedAI. Easyocr: Ready-to-uud ocr with 80+ languages.
EasyOCR, 2024. 5
[23] S. Jeong, J. Back, S. Cho, S. J. Hwang, and J. Park. Adaptive-AAG: Learning to adapt retrieval-augmented large language models through question complexity. In NAACL, 2024. 21
[24] D. Kim, B. Kim, Han, and M. Eibich. Autorag: automated framework for optimization of retrieval augmented generation pipeline. arXiv preprint 1:Xi::2410.20877 2024. 22
[25] P. Lewis. E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal. H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, et al. -aggeeeell generation for knowledge-inteme nlp tasks. Advances in neural information processing systems, 2020. 2, 21
(26] F. Li, R. Zhang, H. Zhang, Y. Zhang, B. Li, W. Li, Z. Ma, and C. Li. Llava-next-interleave: Tack- ling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895 2024. 21
[27] X. Li, Z. Yan, D. Meng, L. Dong, X. Zeng. Y. He, Wang, Y. Qiao, Y. Wang, and L. Wang. Videochat-rl: Enhancing spatio-temporal perception via reinforcement fine-tuning. arXiv preprint arXiv:25044006558 2025. 21
[28] Z. Li, X. Chen, H. Yu, H. Lin, Y. Lu, Q. Tang, Huang, X. Han, L. Sun, and Y. Li. Struc- trag: Boosting knowledge intensive reasoning of Ilms via inference-time hybrid information structurization. arXiv preprint arXiv:2410.08815 2024. 21
[29] B. Lin, Y. Ye, B. Zhu, J. Cui, M. Ning, P. Jin, and L. Yuan. Video-LLaVA: Learning united visual representation by alignment before projection. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024. 1,7
[30] K. Lin, F. Ahmed, L. Li, C.-C. Lin, E. Azarmasab, Z. Yang, J. Wang. L. Liang, Z. Liu, Y. Lu, et al. Mm-vid: Advancing video understanding with gpt-4v (ision). arXiv preprint arXhv:2310.19775. 2023. 21
[31] H. Liu, C. Li. Q. Wu, and Y. J. Lee. Visual instruction tuning. In Advances in neural information processing systems, 2023. 1, 21
[32] Y. Luo, X. Zheng, X. Yang, G. Li, H. Lin, J. Huang, J. Ji, F. Chao, J. Luo, and R. Ji. Video-rag: Visually-aligned retrieval-augmented long video comprehension. arXiv preprint ArXiv:2411. 13093, 2024. 2, 8, 9,,9,,2 92
[33] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever. Robust speech recognition via large-scale weak supervision. In International conference on machine learning. pages 28492-285 18. PMLR, 2023. 5
[34] O. Ram, Y. Levine, I. Dalmedigos, D. Muhlgay, A. Shashua, K. Leyton-Brown and Y. Shoham. In-context aetreeeal-augmennted language models. Transactions of the Association for Computa- tional Linguistics, 11:1316-1331, 2023. 21
12

---

[35] W. Ren, W. Ma. H. Yang, C. Wei, G. Zhang, and W. Chen. Vamba: Understanding hour-long videos with hybrid mambu-traansoommers arXiv preprint arXiv:2503.11579, 2025. 21
[36] X. Ren, L. Xu, L. Xia, S. Wang, D. Yin, and C. Huang. Videorag: Retrieval-augmented generation with extreme long-context videos. arXiv preprint arXiv:2502.01549, 2025. 2, 22
[37] M. Tapaswi, Y. Zhu, R. Stiefelhagen, A. Torralba, R. Urtasun, and S. Fidler. Movieqa: Under- standing stories in movies through question-anse answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016. 3
[38] G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican, et al. Gemini: family of highly capable multimodal models. arXiv preprint 2xXi:2232..11805 2023. 21
[39] Q. Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL https: //qpe////qmmmmm 1m. w.w.tuub............gg.......... io/ii///bb hl/b//lg///// qwq-qqp33322 32b/. 8
[40] H. Touvron, Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al. Llama: Open and efficient foundation language models. arXiv
preprint arXiv:2302. 13971, 2023. 21
[41] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.009266
2023. 21
[42] Z. Z. Wang, A. Asai, X. V. Yu, F. F. Xu, Y. Xie, G. Neubig, and D. Fried. Coderag-bench: Can retrieval augment code generation? 2024. 22
[43] H. Wu, D. Li, B. Chen, and J. Li. Longvideobench: A benchmark for long-context interleaved video-language understanding. Advances in Neural Information Processing Systems, 37:28828-
28857, 2024. 6
[44] S.-Q. Yan, J.-C. Gu, Y. Zhu, and Z.-H. Ling. Corrective retrieval augmented generation. arXiv preprint arXiv:240I. 15884, 2024. 22
[45] A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, et al. Qwen2.5 technical report. arXiv preprint 24215:2412.15115, 2024. 4, 6, 2,
[46] Y. Yao, T. Yu, A. Zhang, C. Wang, J. Cui, H. Zhu, T. Cai, H. Li, W. Zhao, Z. He, et al. Minicpm-v: A gpt-4v level milm on your phone. arXiv preprint arXiv:2408. 01000088000 2024. 5
[47] B. Zhang, K. Li, Z. Cheng, Z. Hu, Y. Yuan, G. Chen, S. LLng, Y. Jiang, H. Zhang, X. Li, al. Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprini arXiv:2501. 20011.1111066. 13106, 2025. 7, 21
[48] H. Zhang, X. Li, and L. Bing. Video-LLaMA: An instruction-tuned audio-visual language model for video understanding. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2023. 1,21
[49] L. Zhang, T. Zhao, H. Ying, Y. Ma, and K. Lee. Omagent: A multi-modal agent framework for complex video understanding with task divide-and-conquer. arXiv preprint arXiv:2406. 16620, 2024.22 21
[50] Y. Zhang, B. Li, h. Liu, Y.j. Lee, L. Gui, D. Fu, J. Feng, Z. Liu, and C. Li. Llava-next: A strong zero-shot video understanding model, April 2024. URL https: http::::////aaaaaleee v1. github.io 04-00-4-300 10---00-llaaala llavallavaaaa next- video/. 21
[51] X. Zheng, Z. Weng, Y. Lyu, L. Jiang, H. Xue, B. Ren, D. Paudel, N. Sebe, L. Van Gool, and X. Hu. Retrieval augmented generation and understanding in vision: A survey and new outlook. arXiv preprint arXiv:2503.1 18016, 2025. 22
[52] B. Zhou, Y. Hu, X. Weng, J. Jia, J. Luo, X. Liu, J. Wu, and L. Huang. Tinyllava: A framework of scall-scale large multimodal models. arXiv preprint arXiv:2402.14288 2024. 21
[53] J. Zhou, Y. Shu, B. Zhao, B. Wu, S. Xiao, X. Yang, Y. Xiong, B. Zhang, T. Huang, and Z. Liu. Mlvu: A comprehensive benchmark for multi-task long video understanding. arXiv preprint arXiv:2406.044 042644 2024. 6,
13

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction have clearly included the motivations, important assumptions, and contributions made in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The authors have discussed the limitations of the work Ablation Study.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

The authors have clearly provided the full set of assumptions and complete proofs in the Method section.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The authors have presented the experimental details the Implementation Details section.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
The authors have presented all the experimental details in Appendix which provided sufficient instructions to faithfully reproduce the main experimental results.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.
public/gus Btte://uidee//oeeuubataa oeeioonllcyy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nips. wwtppccccc//ccccccccccccccccctct /ww//ggisss... CCodeSubm as www.nnoliieeeoooeoooooooooeoosai for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The authors have presented all the training and test details in the Implementa- tion Details section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The authors have presented all the experimental details in Appendix with provided sufficient instructions.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτor rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The authors have included the sufficient information on the computer resources in the Implementation Details.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //terrr///.//././///e cc/ccc//plcc cc/public// ptpppi//tthiiiieeeeepppppppppiie c/tthii//eieee csGuidelinessee
Answer: [Yes]
Justification: The authors conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., s if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The authors have explained the broader impacts of the work in Appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
17

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
The authors have cited the original paper that produced the code package dataset.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The authors have submitted the details of the code/model which includes details about training, license, limitations, etc.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
19

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
The paper does not involve LLM usage in manuscript preparation.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https:/ //teur/ess.p///// Cheurrps.cc/Confeeee erroneeeeeeee/22255 /LLM) for what should or should be described.
20

---

Appendix
The appendix presents the following sections to strengthen the main manuscript:
Sec. A shows the Related Work part of the paper.
Sec. B shows the dynamic sampling strategies when building the database. We set different sampling frequencies evaluate the performance of our Ada VideoRAA on the MLVU dataset.
Sec. C shows the instructions for level classification. We develop hierarchical prompting templates that enable L.LMs to classify query complexity into three tiers (L.I-L3), triggering adaptive retrieval strategies from direct lookup to sensemaking reasoning.
Sec. D shows the more detailed results, including multiple-choice QA and summary video understanding, that visually demonstrate our AdaVideORAG can enhance the reasoning ability of MLLMs.
A Related Work
A.1 Multimodal Large Language Models
With the successful application experience of Large Language Models (LLMs) [40, 41, 17, 45], Multimodal Large Language Models (MLLMs) have also made significant breakthroughs in the field of visual language understanding. LLa [31], by first conducting instruction fine-tuning training on a dataset carefully selected by GPT-4, has become one of the most popular methods for constructing MILMs and has been followed by subsequent works [52, 4, 3]. Benefiting from the rapid development in the fields of short vidco applications and video generation models, video understanding has also become increasingly popular recently [48, 50, 26]. In contrast, due to the complexity of spatio-temporal joint modeling, video understanding poses higher requirements for the fine-grate fine-grained spatial understanding of the model and multi-hop prompts, so some works targeting the long-video setting have emerged recently. Vamba [35] has constructed a hybrid Mamba- Transformer model that encodes video tokens with linear complexity, and recent VideoChat-RR -1 [27] has explored the use of GRPO for the reinforcement fine-tuning of video MLLMs. Recently, there are also some works video understanding based agents [30, 13, 49]. Closed-source commercial models such as GPT-4o [21] and Gemini [38] have leading video understanding capabilities, while open-source model series such the typical Qwen- [1], Intern VL. [9], and LadA [47] have received widespread attention and academic research. However, when applied to specific fields, MLLMs have lower accuracy in answers or larger hallucinations due to their inability to dynamically update the latest knowledge. To improve the accuracy, additional fine-tuning costs and time costs are required, and it is difficult to deal with multi-hop questions in long-via video cross-modal understanding. In order to alleviate the above problems, this paper studies Retrieval-Augmente Augmented Generation (RAG) for long-video, and improves the video understanding ability of the model by integrating the collaborative reasoning between an external knowledge base and the generation model.
A.2 Retrieval-Augmentted Understanding
Retrieval-Augmented Generation (RAG) optimizes large language models (LLMs) by integrating external knowledge retrieval with generative model capabilities [25, 34, 12]. It enables low-cost knowledge expansion without retraining the model through dynamic updates to external knowledge bases, effectively mitigating traditional LLMs' issues of hallucinations, outdated knowledge, and data security risks. RAG [28, 15] converts user queries into vectors, retrieves the most relevant information from external databases, and integrates retrieval results as context into the generative model's prompt to deliver more accurate, reliable, and fact-based responses. However, these methods often overlook complex relationships between documents or contexts- -uchh as entity connections. hierarchical structures, or causal relationships that are critical for contextual understanding. Consequently, graph-based RAG methods [10] have gained traction, exploring structured knowledge representations to enhance retrieval efficiency and precision while excelling in query-focused summarization tasks. Additionally, Adaptive-RAs [23] significantly improves traditional RAG systems' performance and efficiency in complex scenarios by dynamically adjusting retrieval strategies and generative logic.
21

---

Recent research has deeply investigated efficiency challenges [19, 11], dataset-specific optimiza- tions [24], and hallucination correction [44].
Recent RAG approaches have also integrated multimodal information to meet growing application demands [20, 51], such images [7], code [42], tables [6], and audio [8]. However, constrained by video data's complex modal information and requirements for spatio-temporal modeling, RAG has seen limited adoption in the video understanding field- vert-paaticcularly for long-video understanding, which poses significant challenges for long-context information modeling. Luo *et al. [32] incor- porates aisually-aligned auxiliary text features from OCR, ASR, and object detection to construct video knowledge bases, while VideoRAG [36] significantly improves the accuracy of long-video contextual information by building graph databases. However, current solutions either struggle to address multi-hop questions effectively or face substantial computational resource and time costs when maintaining hierarchical graph databases. To tackle these issues, we propose a query intent classification strategy to adaptively route queries to different retrieval paths based on their difficulty, achieving robust balance between performance and efficiency.
B Dynamic Sampling Strategies
During the database construction phase, whether it is naive vector retrieval or graph retrieval, both of them are constructed upon the extracted auxiliary texts, including the caption, ASR, and OCR. Therefore, dynamic sampling strategies directly determine the semantic coverage density. Generally speaking, the faster the sampling frequency, the more effective information can be obtained from the video, which is more conducive subsequent retrieval.
To systematically validate the impact of sampling strategies, we conduct ablation studies comparing frame sampling density (varying from 5 to 30 frames/clip), measuring their effects on MLVU test dataset, as shown below Tab. A1. As the sampling frequency increases, the evaluation metrics will become higher. Since the final results of 30 frames and 5 frames only differ by about point, adopt a sampling frequency
frames per 30 seconds for Ada VideoRAG, considering both the improvement of accuracy and the efficiency of resource utilization.
Table A1: The accuracy of different sampling frequencies for the MLVU dataset
Frames
5
10
15
20
25
30
M-AVG
53.2
53.7
53.5
54.1
54.1
54.5
C Instructions for Level Classification
In this paper, use prompts to classify the difficulty level of queries in Fig. A2, including the detailed description each level: L1 (Direct Factual) queries requiring single-modality pattern matching (e.g.. "Identify the object 0:05"), L2 (Contextual) needing temporal/causal reasoning across clips (e.g.. "Why did X occur before Y?"), and L3 (Multi-Hop demanding cross-modal hypothesis validation with external knowledge (e.g., "How would Z change if scene A were removed?")
D More Detailed Results.
To further validate the enhancement of Ada VideoRAG on the video understanding capabilities of MLLMs, we provide more detailed and specific instance demonstrations, including multiple-choiee questions and sensemaking questions, as shown as Fig. A1.
We selected two specific multiple-choice cases and compared them with the MLLM baseline model (Videol.h I LaMA) and VideoRAG [32] respectively. The results show that our Ada VideRRAG pays more attention to details in terms of video understanding capabilities and thus makes and confident responses to user queries. Meanwhile, we also analyze the sensemaking problems. As shown the bottom of Fig. A1, our Ada VideoRAG can provide more fine-grained information. All the answers are based on the real video content, which significantly enhances its ability to reduce visual hallucinations and enables output more complete and logical answers.
accurate
22

---

multiple-choice questions
Query: How many people jump into the water first at the beginning of the video?
7 (8)3 8 (D)2 (E)5 4
MLLM((VdeoLLMMA):
MLLM(VideoLLMMA with AdavideoRA: C
Query: What is the genre this video?
(A) it is news report that introduces the history behind Christmas decorations.
(B) It is a documentary on the evolution of Christmas holiday recipes
(c) it is travel vlog exploring Christmas markets around the world
(D) it tutorial on DIY Christmas ornament crafting
MLLM(VideoLLLMM) with VideoRAG:
MLLM(VideoLLLMM) with AdaVideoRAG:
Sensemaking questions
Query: How Jean's makeup routine reflect personal style and preferences?
MLLM(VideoLLLM) Jean's makeup routine reflects her personal style and preferences through her choice of products, application techniques, and overall aesthetic. She opts for natural yet polished look, using mix of drugstore and high-end products. Her preference for minimalist approach is evident in her simple eye makeup and
single color.
MLLM(VVdeoLLLMM) with AdaVideoRAG : Jean's makeup routine reflects her personal style and preferences through her emphasis on 'Clean Girl' look, which focuses on creating dewy, hydrated appearance without heavy makeup. She prefers using products like the Mac lipstick and the Giorgio Armani luminous silk the shade 575 for natural yet polished finish. Her approach skincare and makeup suggests she values fresh and radiant look, often achieved through careful application techniques and product selection that enhances her natural features without appearing cakey overdone.
Figure A1: Qualitative results of LaMA when applying Video-RAG
23

---

Prompts for Level Classification
Goal
1. Given a query, Classify queries into exclusively one difficulty level (A/B/C).
2. When you are confused about which level to choose, choose a higher level.
#####:
Classification Criteria ##I##:
Level (Local Perception or question is very easy that we can put all of video as input)
Level (Global Search) relationships. It can be used for global awareness, which requires global search and positioning of video segments related to questions, and then understanding of the searched video segments.
contains simple logical
Level 3 (Complex Reasoning) Requires external knowledge/absttn interpretation. It may be necessary to make of knowledge graphs of multi-layer complex relationshiss, or semantic questions with high generalizations
##### Examples ##I!#!
Query1: What color is the dog that appears in the 5s part of the video?
Output: ###:Leeel A####
Query2: What happened before the rain?
Output: ####Levell B####:
Query7: Identify the position relationships of the
characters in the meeting room scene?
Output: ###tLeeel C####
Figure A2: Prompts for Level Classification
24


Reviews:
[{"review_id": "VFno7CoynC", "reviewer": "Reviewer_szi1", "summary": "This paper presents AdaVideoRAG, an adaptive retrieval-augmented generation framework designed to improve long-video understanding in multimodal large language models (MLLMs). Existing VideoRAG systems apply fixed retrieval strategies regardless of query complexity, leading to inefficiencies for simple tasks and insufficient depth for complex reasoning. AdaVideoRAG addresses this limitation by employing a lightweight intent classifier to dynamically select appropriate retrieval pathways—from simple text-based lookups to more sophisticated graph-based retrieval—based on the complexity of the user query. The framework incorporates an Omni-Knowledge Indexing module that constructs multi-modal knowledge bases (text, vision, and graph) from video inputs, enabling hierarchical and context-sensitive retrieval. Additionally, the authors introduce HiVU, a new benchmark for evaluating deep video understanding. Experimental results demonstrate that AdaVideoRAG achieves improved accuracy and efficiency in long-video QA tasks, while being compatible with existing MLLMs through lightweight API integration.", "questions": "1. In Table 4, it is unclear why the level-1 results are missing from the left side of the table, while AdaVideoRAG results for all levels (including level-1) are shown on the right. Furthermore, the values for AdaVideoRAG appear to differ between the left and right sections of the table, which creates confusion about what each column represents. Could the authors clarify the intended structure of Table 4 and explain why these inconsistencies appear?\n\n2. The paper highlights AdaVideoRAG’s ability to reduce computational overhead by selecting simpler retrieval paths when appropriate. However, the actual latency or cost reduction (e.g., in terms of FLOPs or runtime) is not quantified. Could the authors provide more concrete measurements of retrieval cost across different query types?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strength\n### Novel adaptive retrieval mechanism:  \nThe paper introduces a compelling and novel approach to video retrieval-augmented generation by leveraging a lightweight intent classifier that dynamically adjusts the retrieval strategy based on the complexity of the input query. This design allows the system to balance efficiency and reasoning depth more effectively than existing fixed-retrieval paradigms, addressing key limitations in current long-video understanding methods.\n\n### Comprehensive benchmark for full-stack video understanding:  \nThe authors propose HiVU, a well-designed benchmark that enables systematic evaluation of video-language models across a spectrum of reasoning complexity. By incorporating a three-level difficulty quantification system and including diverse question types—from fine-grained object recognition to multi-hop temporal reasoning—HiVU provides a comprehensive assessment of full-stack video understanding capabilities. This benchmark fills a crucial gap in evaluating both the breadth and depth of model comprehension in realistic long-video scenarios.\n\n# Weaknesses\n\n### Limited comparison with prior Video-RAG baselines: \nWhile the proposed AdaVideoRAG framework is thoroughly evaluated on several benchmarks, the comparison to prior Video-RAG approaches appears somewhat limited. In Tables 2 and 4, only VideoRAG [36] is used as a baseline, whereas other relevant methods such as VideoRAG [32] are omitted. Although the performance of [32] may be lower than [36], including it would improve the completeness and consistency of the comparisons—particularly on the newly proposed HiVU benchmark, where the set of comparable baselines is already narrow. Additionally, other recent VideoRAG-style works such as Jeong et al. [1] could also be cited and considered for reference.\n\n### Limited baseline diversity on the HiVU benchmark: \nThe HiVU benchmark in Table 4 includes only a small set of evaluated models, limiting our understanding of the generalizability of AdaVideoRAG. It would strengthen the empirical claims if more diverse baselines—such as LLaVA-Video [2] and InternVL 2.5 [3]—were also evaluated under the same framework. Applying AdaVideoRAG to these varied architectures and model scales would not only demonstrate its adaptability, but also provide a valuable point of comparison for future research using HiVU.\n\n---\n[1] Jeong et al. VideoRAG: Retrieval-Augmented Generation over Video Corpus. Arxiv 2025.  \n[2] Zhang et al. Llava-next: A strong zero-shot video understanding model. Arxiv 2024.  \n[3] Chen et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. Arxiv 2024.", "was_revised": true, "final_justification": "After carefully considering the rebuttal and the ensuing discussion, I have decided to maintain my original score of 5 (Accept). The authors have adequately addressed the key concerns I raised in the initial review, and the paper’s contributions remain strong and relevant. Below is a summary of my reasoning:\n- Resolved: Limited comparison with prior Video-RAG baselines\nThe authors clarified the omission of VideoRAG [32] and acknowledged the limited availability of open baselines. They also added relevant discussion referencing recent works such as Jeong et al. [1], improving the completeness of related work coverage.\n- Resolved: Baseline diversity on the HiVU benchmark\nWhile it remains challenging to apply AdaVideoRAG to a wider set of models, the authors provided justification and expressed their intent to release adapter implementations for broader compatibility. Their updated discussion on generalizability helps mitigate this concern.\n\nIn light of these points, I believe the paper makes a valuable and timely contribution to the field, and I fully support its acceptance.", "rebuttal": "**Thank you very much for your valuable comments and constructive suggestions, which have provided important guidance for improving our paper. We have carefully studied each point and would like to respond in detail as follows.**\n\n---\n\n**Q1: Limited comparison with prior Video-RAG baselines**\n\nThank you for your valuable advice. AdaVideoRAG shares a similar plug-and-play design concept with VideoRAG [32]. Since this method is relatively new and achieves good results, this paper focuses on it due to space constraints. We fully agree with your suggestion that comparing Jeong et al. [1] and VideoRAG [36] will improve the completeness of this paper, and the following content will be added to the revised version: \n- We have supplemented the results of VideoRAG [36] on the VideoMME benchmark as shown in the following table (corresponding to Table 2). AdaVideoRAG achieves the optimal results, and both AdaVideoRAG and VideoRAG [32] obtain significantly higher overall scores than VideoRAG [36]. However, in simple short videos, the performance of VideoRAG [36] declines significantly because of the introduction of excessive redundant information and the filtering out of useful video clip information.\n\n| **Method** | **Params** | **Frames** | **Short** | **Medium** | **Long** | **Overall** | **Gain** |\n| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| **VideoLLaMA3** | 7B | 1fps-180 | 76.7 | 62.8 | 53.2 | 64.2 |  |\n| **VideoRAG [36]** | 7B | - | 72.3 | 64.2 | 58.4 | 64.9 | +0.7 |\n| **VideoLLaMA3 + VideoRAG [32]** | 7B | 32 | 81.5 | 63.3 | 57.1 | 67.3 | +3.1 |\n| **VideoLLaMA3 + AdaVideoRAG** | 7B | 1fps-180 | 80.3 | 65.4 | 59.8 | 68.5 | +4.3 |\n\n- We have supplemented the results of AdaVideoRAG and VideoRAG [36] on the HiVU benchmark as shown in the following table. The results indicate that our AdaVideoRAG also achieves significantly better results compared to VideoRAG [36].\n\n| **Metric**  |**Level-2 - VideoRAG [36]** | **Level-2 - VideoLLaMA3 + AdaVideoRAG** | **Level-3 - VideoRAG [36]** | **Level-3 - VideoLLaMA3 + AdaVideoRAG** | **Overall - VideoRAG [36]** | **Overall - VideoLLaMA3 + AdaVideoRAG** |\n| :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| **Comprehensiveness** | 45.5% | 54.5% | 47.1% | 52.9% | 46.5% | 53.5% |\n| **Empowerment** | 44.9% | 55.1% | 46.0% | 54.0% |45.3% |54.7% |\n| **Trustworthiness** | 41.1% | 58.9% | 43.6% | 56.4% | 42.2% | 57.8% |\n| **Depth** | 42.7% | 57.3% | 44.2% | 55.8% | 43.8% | 56.2% |\n| **Density** | 46.0% | 54.0% | 46.5% | 53.5% | 46.3% | 53.7% |\n| **Overall Winner** | 40.8% | 59.2% | 43.9% | 56.1% | 41.3% | 58.7% |\n\n- Since the work of Jeong et al. [1] has not yet made its source code publicly available (though there is a GitHub repository, it lacks the core running code) nor provided reproducible experimental configurations, we are currently unable to conduct a fair experimental comparison. However, we commit to updating the comparative analysis results once its complete code is released in the future.\n\n---\n\n**Q2: Limited baseline diversity on the HiVU benchmark**\n\nThanks for the suggestion. We agree that expanding the set of evaluated models on HiVU is crucial to verifying the generalizability of AdaVideoRAG. Specifically, we have supplemented the use of ***1)*** **Qwen2.5-VL-7B** [1r], ***2)*** **InternVL2.5-8B** [2r], and ***3)*** **LLaVa-Video-7B** [3r] as baselines. The overall results are shown in the following table, and detailed results will be added to the revised version. It can be seen that AdaVideoRAG has strong adaptability, which can stably improve the results of different baselines.\n\n| **Metric** | **Overall <br> Qwen2.5-VL-7B** | **Overall <br> Qwen2.5-VL-7B + AdaVideoRAG** | **Overall <br> InternVL2.5-8B** | **Overall <br> InternVL2.5-8B + AdaVideoRAG** | **Overall <br> LLaVa-Video-7B** | **Overall <br> LLaVa-Video-7B + AdaVideoRAG** |\n| :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| **Comprehensiveness** | 38.1% | 61.9% | 35.9% | 64.1% | 35.6% | 64.4% |\n| **Empowerment** | 36.3% | 63.7% | 31.3% | 68.7% | 33.5% | 66.5% |\n| **Trustworthiness** | 34.2% | 65.8% | 35.7% | 64.3% | 34.1% | 65.9% |\n| **Depth** | 38.0% | 62.0% | 37.1% | 62.9% | 35.9% | 64.1% |\n| **Density** | 30.5% | 69.5% | 30.8% | 69.2% | 29.4% | 70.6% |\n| **Overall Winner** | 35.6% | 64.4% | 33.2% | 66.8% | 31.5% | 68.5% |\n\n---\n\n**Q3: Clarification on Table 4 structure and inconsistencies**\n\n- **Level-Specific Columns (Level-2/Level-3):** These columns validate two key aspects: ***1)*** the differentiation of HiVU’s difficulty levels and ***2)*** the effectiveness of hard retrieval for complex tasks. To isolate the impact of retrieval strategies, we use HiVU’s ground-truth level labels (not model predictions) to evaluate model results across different difficulty levels.\n\n- **Why no Level-1 Column:** Level-1 is omitted because all methods perform identically (no retrieval needed).\n\n- **Overall Column:** This column uses model-predicted level labels (not ground truth) to reflect real-world performance, directly comparing AdaVideoRAG against baselines in end-to-end scenarios.\n\n- **Left vs. Right Sections:** The left section focuses on “without vs. with AdaVideoRAG” to assess the necessity and effectiveness of the AdaVideoRAG model, with the compared baseline being the basic VideoLLaMA3; the right section compares “different RAG schemes” to highlight AdaVideoRAG’s superiority over the recently powerful VideoRAG [32].\n\nWe will add a footnote stating: “Level-2/3 use ground-truth labels to isolate retrieval effects; Level-1 is excluded as results are uniform.”\n\n---\n\n**Q4: Efficiency Analysis**\n\nTo verify the efficiency differences among different retrieval paths, we randomly selected 100 videos from the MLVU dataset and conducted systematic time cost tests for three scenarios: no retrieval (Level-1), simple retrieval (Level-2), and hard retrieval (Level-3). The specific results are as follows:\n\n- **Database construction**\n  1. Level-1 performs direct inference without the need for database construction.\n  2. The average time consumed for Level-2 construction is 351s.\n  3. The average time consumed for Level-3 construction is 412s. \n  This difference stems from the fact that Level-3 requires graph construction, resulting in higher preprocessing costs. However, the dynamic strategy of AdaVideoRAG can reduce the frequency of such high-cost operations by prioritizing triggering simple retrieval in applicable scenarios.\n\n- In the single H20 GPU card, single-process processing program, there are significant differences in the average response time among the three scenarios: the average response time for Level-1 (no retrieval) is 8s; for Level-2 (Simple reasoning) it is 26s; for Level-3 (hard reasoning) it is 27s, while the processing time using AdaVideoRAG is 20s.\n\n- **Parallelization**\n  To further improve the efficiency of actual deployment, we have implemented parallel processing in the code. We also used 100 videos from the MLVU dataset as test references:\n  - For database construction with dual processes on a single H20 GPU (the maximum batch size supported by 96G memory is 2), the speed of Level-2 and Level-3 is accelerated by approximately 2 times, with the construction time becoming 176s and 210s, respectively.\n  - When further expanding to multi-card (8 H20 GPUs) database construction, the average speed of each database construction for Level-2 and Level-3 is accelerated by approximately 8 times (linear acceleration), with the construction time becoming 22s and 26s, respectively.\n  - For multi-process retrieval on a single H20 GPU: the processing speed of Level-2 and Level-3 is accelerated by approximately 2 times, with the final total processing time becoming 15s and 16s, respectively, while the processing time using AdaVideoRAG is 20s.\n\nThe summary table of efficiency for AdaVideoRAG is as follows:\n| **Item** | **Level-1** | **Level-2** | **Level-3** | **AdaVideoRAG** |\n| :----: | :----: | :----: | :----: | :----: |\n| Single-card single-process database construction | - | 351s | 412s | - |\n| Single-card dual-process database construction | - | 176s | 210s | - |\n| 8-card dual-process database construction | - | 22s | 26s | - |\n| Single-card single-process inference | 8s | 26s | 27s | 20s |\n| Single-card dual-process inference | 8s | 15s | 16s | 13s |\n\n- The comparison of inference efficiency between AdaVideoRAG and VideoRAG [32] on the MLVU dataset with 100 videos using a single H20 card is shown in the following table. In comparison, our method is more efficient while achieving higher metrics (see Table 2 and Table 4).\n\n| **Method** | **Time** |\n| :----: | :----: |\n| VideoRAG [32] | 25s |\n| AdaVideoRAG | 13s |\n\n\n---\n\n[1r] Bai, Shuai, et al. \"Qwen2. 5-vl technical report.\" arXiv, 2025.\n\n[2r] Chen, Zhe, et al. \"Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.\" arXiv. 2024.\n\n[3r] Zhang, Yuanhan, et al. \"Video instruction tuning with synthetic data.\" arXiv, 2024."}, {"review_id": "p7zzYRV9Yc", "reviewer": "Reviewer_d2T2", "summary": "This work tackles the problem of long video understanding with large language models. The authors claim that existing VideoLLM + RAG techniques are sub-optimal due to their fixed strategy for retrieval, which uses uniform structures regardless of input query difficulty, bringing redundant computational overhead for simple queries and potential information loss for complex queries. Therefore, they propose AdaVideoRAG, an adaptive VideoLLM + RAG technique that leverages a lightweight intent classifier to determine the difficulty of the query, and enables dynamic routing to different retrieval methods according to the classification results. The authors also introduced an Omni-Knowledge Indexing module to support better multi-modal information extraction, and the HiVU benchmark for evaluation. Experiments have been conducted on public benchmarks and the proposed evaluation HiVU benchmark to verify the effectiveness of AdaVideoRAG.", "questions": "Please refer to the weaknesses section for my concerns. May major concern is the novelty of the proposed method. Authors are encouraged to provide in-depth analysis to justify 1. the proposed method is indeed novel, rather than a simple combination of two video RAG methods plus a LLM-prompting-based classifier, and 2. how much efficiency can the proposed method bring.", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. Overall, the paper is easy to follow. The motivation for utilizing a lightweight classifier to determine whether to use complex retrieval methods is clear -- allocating more compute to complex queries and less to simple queries.\n2. The effectiveness of the proposed methods is well-demonstrated by ablation studies in Table 1 and Table 2.\n\nWeaknesses:\n\n1. Why adopting an extra Qwen2.5-7B as the intent classifier, instead of using the MLLM itself instead? Ablation studies on the choice of intent classifiers shall be included in the study. Besides, I believe a 7B-size model cannot be regarded as a \"lightweight\" classifier, given that the MLLM is generally also 7B.\n2. Although it is clear, the motivation seems to be incremental. Leveraging another 7B LLM + prompting to act as a classifier is not a smart idea for routing. Authors are encouraged to provide strong justifications to demonstrate the significance of the proposed method.\n3. The authors mentioned \"efficiency\" of the proposed method throughout the paper, but it seems that detailed analysis/discussions/experiments on such effect are not provided. It would be better to conduct an in-depth study to see how much latency is introduced by adopting VideoRAG/AdaVideoRAG (including database construction and knowledge retrieval) compared to VideoLLM baselines.\n4. The VideoLLM + RAG technique seems to be closely relevant to test-time scaling methods on video understanding. It would be interesting to incorporate a detailed discussion/comparison between the proposed method (RAG-based) and recent TTS methods (e.g., CoT-based [1-3] and agent-based [4, 5]) on long video understanding.\n\nMinor Issues:\n\n1. Typo: In Figure 2, Sec. 3.1, 3.2, 3.3 shall be 2.1, 2.2, 2.3\n\n[1] Chain-of-Frames: Advancing Video Understanding in Multimodal LLMs via Frame-Aware Reasoning. arXiv 2025.\n[2] Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition. ICML 2024.\n[3] VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selection. CVPR 2025.\n[4] VideoMind: A Chain-of-LoRA Agent for Long Video Reasoning. arXiv 2025.\n[5] VideoAgent: Long-form Video Understanding with Large Language Model as Agent. ECCV 2024.", "was_revised": true, "final_justification": "After reading the reviews from other reviewers and the rebuttal/responses from the authors. My evaluation is that the current version of the paper might not be ready for publication, and should be carefully revised to clearly state/discuss the concerns regarding (1) memory consumption and fairness concerns (the actual model is 7B + 7B = 14B), and (2) the significantly inconsistent results of Qwen2-VL-7B and InternVL2.5-4B. I'm keeping my original rating from my perspective, and will leave it to AC for the final decision.", "rebuttal": "**Thank you for your insightful comments and constructive feedback on our manuscript. We appreciate the careful evaluation of our work and agree with the points raised, which help strengthen the rigor and clarity of our research. Below, we address each concern in detail:**\n\n---\n\n**Q1: Choice of Intent Classifier (Qwen2.5-7B) and Its \"Lightweight\" Nature**\n\n- **Rationale for an independent classifier:** The MLLM in this framework is mainly responsible for multimodal information integration and generation, which involves a large amount of computation (such as processing video frames and cross-modal alignment). If MLLM is used for intent classification, the entire video context needs to be input into the model before inference, resulting in redundant processing of video data even for simple queries that do not require retrieval (Level 1). In contrast, the intent classifier only processes text queries. Models of the same scale reduce the feature pre-extraction process for input video frames and the number of input tokens, thereby reducing end-to-end latency. Therefore, we chose Qwen2.5-7B to achieve the functional separation of \"video understanding\" and \"intent classification\", establish a unified benchmark for experiments, and ensure efficient retrieval. In addition, Qwen2.5-7B not only undertakes the classification task but also is responsible for query decoupling and fact filtering of video clips. This \"one-model-multiple-uses\" design can improve the accuracy of information acquisition, which is why we did not directly use MLLM.\n\n- **\"Lightweight\" justification:** Although Qwen2.5-7B has 7 billion parameters, its computational overhead in the entire process is extremely small. As described in Sec. 2.1, the time it takes accounts for ≤5% of the total process. The reasons include: (1) It only processes text queries (no need for video frames/audio); (2) Plug-and-play is realized through local lightweight API deployment and calling; (3) It avoids the expensive cross-modal encoding step in MLLM. Compared with fixed retrieval strategies, the marginal cost of the classifier is offset by the efficiency improvement brought by simplifying the retrieval of simple queries.\n\n- **Ablation on classifier choice:** To address this issue, we will supplement ablation experiments: (a) Qwen2.5-7B (our choice), (b) MLLM itself (VideoLLaMA3-7B) as the classifier, and (c) smaller models (such as Qwen2.5-1.5B). Preliminary results show that using MLLM (VideoLLaMA3-7B) as the classifier will increase latency by about 3 times (due to video context loading), but the classification accuracy will slightly decrease. Although the smaller model (Qwen2.5-1.5B) reduces latency by 24%, the accuracy drops significantly, affecting the selection of retrieval strategies. This supports our choice of Qwen2.5-7B to balance efficiency and accuracy.\n\n| **Classifier** | **Classfication precision on HiVU** | **Overall Score on Video-MME** | **Average Classifier Time per Query** |\n| :----: | :----: | :----: | :----: |\n| **Qwen2.5-1.5B** | 0.41 | 65.3 |  53ms|\n| **Qwen2.5-7B** | 0.81 | 68.5 | 70ms |\n| **VideoLLaMA3-7B** | 0.48 | 67.5 | 203ms | \n\n---\n\n**Q2: Novelty and Significance of the Proposed Method**\n\nThe reviewer notes that the motivation may seem incremental, and we aim to clarify the novelty of AdaVideoRAG:\n- Beyond \"combining two RAG methods with a classifier\": Existing VideoRAG methods (e.g., [32, 36]) use fixed retrieval paradigms (naive retrieval or graph retrieval for all queries). AdaVideoRAG introduces a hierarchical adaptive mechanism that dynamically scales retrieval complexity with query difficulty, which is absent in prior work. This includes:\n- A fine-grained intent classification system (Level-1/2/3) tailored to video-specific reasoning demands (spatio-temporal logic, cross-modal causality).\n- The Omni-Knowledge Indexing module, which unifies text (ASR/OCR/captions), visual features, and knowledge graphs into a hierarchical knowledge base—enabling seamless transitions from no retrieval (Level-1) to graph retrieval (Level-3).\n- Empirical evidence (Table 2) shows that this adaptivity leads to a overall +4.8（from +7.9 to +12.7） gain over fixed VideoRAG [32] with Qwen2.5-VL-7B on Video-MME. Meanwhile, Table 4 also shows that the proposed AdaVideoRAG significantly outperforms VideoRAG [32] on HiVU, which demonstrates the generalization and effectiveness of the proposed method.\n- Broader impact: This work establishes a new paradigm for adaptive retrieval augmentation in video analysis, where resource allocation is optimized for real-world scenarios (e.g., simple frame-level queries vs. complex multi-hop reasoning). This is distinct from static RAG or MLLM-only approaches, as demonstrated by its universal integration with existing MLLMs (Tab. 1 shows gains across Qwen2.5-VL, VideoLLaMA3, etc.).\n\n---\n\n**Q3: Efficiency Analysis**\n\nTo verify the efficiency differences among different retrieval paths, we randomly selected 100 videos from the MLVU dataset and conducted systematic time cost tests for three scenarios: no retrieval (Level-1), simple retrieval (Level-2), and hard retrieval (Level-3). The specific results are as follows:\n\n- **Database construction**\n  1. Level-1 performs direct inference without the need for database construction.\n  2. The average time consumed for Level-2 construction is 351s.\n  3. The average time consumed for Level-3 construction is 412s. \n\n- In the single H20 GPU card, single-process processing program, there are significant differences in the average response time among the three scenarios: the average response time for Level-1 (no retrieval) is 8s; for Level-2 (Simple reasoning) it is 26s; for Level-3 (hard reasoning) it is 27s, while the processing time using AdaVideoRAG is 20s.\n\n- **Parallelization**\n  To further improve the efficiency of actual deployment, we have implemented parallel processing in the code. We also used 100 videos from the MLVU dataset as test references:\n  - For database construction with dual processes on a single H20 GPU (the maximum batch size supported by 96G memory is 2), the speed of Level-2 and Level-3 is accelerated by approximately 2 times, with the construction time becoming 176s and 210s, respectively.\n  - When further expanding to multi-card (8 H20 GPUs) database construction, the average speed of each database construction for Level-2 and Level-3 is accelerated by approximately 8 times (linear acceleration), with the construction time becoming 22s and 26s, respectively.\n  - For multi-process retrieval on a single H20 GPU: the processing speed of Level-2 and Level-3 is accelerated by approximately 2 times, with the final total processing time becoming 15s and 16s, respectively, while the processing time using AdaVideoRAG is 20s.\n\nThe summary table of efficiency for AdaVideoRAG is as follows:\n| **Item** | **Level-1** | **Level-2** | **Level-3** | **AdaVideoRAG** |\n| :----: | :----: | :----: | :----: | :----: |\n| Single-card single-process database construction | - | 351s | 412s | - |\n| Single-card dual-process database construction | - | 176s | 210s | - |\n| 8-card dual-process database construction | - | 22s | 26s | - |\n| Single-card single-process inference | 8s | 26s | 27s | 20s |\n| Single-card dual-process inference | 8s | 15s | 16s | 13s |\n\n- The comparison of inference efficiency between AdaVideoRAG and VideoRAG [32] on the MLVU dataset with 100 videos using a single H20 card is shown in the following table. In comparison, our method is more efficient while achieving higher metrics (see Table 2 and Table 4).\n\n| **Method** | **Time** |\n| :----: | :----: |\n| VideoRAG [32] | 25s |\n| AdaVideoRAG | 13s |\n\n---\n\n**Q4: Comparison with Other Long Video Understanding Methods (CoT/Agent-Based)**\n\nWe appreciate the suggestion to connect with other long video understanding Methods methods. We will add a discussion in Related Work section highlighting key differences and complements:\n- CoT-based methods (e.g., [1r,2r,3r]) focus on decomposing reasoning steps within the MLLM, enhancing internal logic but not addressing knowledge limitations (e.g., fixed pre-trained knowledge). AdaVideoRAG complements this by augmenting with external knowledge (retrieval) when needed.\n- Agent-based methods (e.g., [4r,5r]) use LLM as a planner to orchestrate tools but often lack adaptive retrieval granularity. AdaVideoRAG’s intent classifier can be integrated into agent frameworks to optimize tool usage (e.g., skip retrieval tools for simple queries).\n- To further visually compare with the above two types of methods, we selected COT-based CoF [1r] and Agent-based VideoMind [4r] to compare with our method on the VideoMME benchmark:\n  - COT-based CoF uses InternVL2.5-4B as the baseline model, and we therefore adopted the same baseline model for AdaVideoRAG testing (using this multimodal large language model as the foundation for the classifier, disentangler, and filter). The results are as follows:\n\n| **Method** | **Overall Score on VideoMME** |\n| :----: | :----: |\n| **InternVL2.5-4B** | 54.9 |\n| **CoF** | 59.7 |\n| **AdaVideoRAG** | 63.3 |\n\n\n- Agent-based VideoMind uses Qwen2-VL-7B as the baseline model, and we therefore adopted the same baseline model for AdaVideoRAG testing (using this multimodal large language model as the foundation for the classifier, disentangler, and filter). The results consistently demonstrate the effectiveness of our method:\n\n| **Method** | **Overall Score on VideoMME** |\n| :----: | :----: |\n| **Qwen2-VL-7B** | 41.9 |\n| **VideoMind-7B** | 58.2 |\n| **Qwen2-VL-7B with AdaVideoRAG** | 59.1 |\n\n\nThe above content will be added to the revised version.\n\n---\n\n**Q5: Minor Issues and Formatting**\n\nThanks for pointing out this.\n- Figure 2 typo: The labels \"Sec. 3.1/3.2/3.3\" will be corrected to \"Sec. 2.1/2.2/2.3\".\n- Appendix placement: The optional technical appendix will be moved after the checklist to better comply with formatting guidelines."}, {"review_id": "vW5bK2Vk9i", "reviewer": "Reviewer_6VsG", "summary": "The paper proposes a novel AdaVideoRAG framework to dynamically and adaptively route appropriate retrieval schemes and introduces an Omni-Knowledge Indexing module to extract valuable information from multi-modal signals for context modeling and establish corresponding databases. It also releases the hierarchical video understanding benchmark HiVU for the first time, which evaluates the multi-level reasoning capabilities of video understanding models.", "questions": "- How important the intent classification accuracy is for overall performance?\n- How much each modality(captions, ASR, OCR) contributes to the final performance? \n- How do different retrieval strategies (none, naive, graph) specifically contribute to efficiency improvements across different query types?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength\n- This paper basically came up with a system that can pick the right retrieval strategy depending on how hard the question is.This way,it stays fast while still giving accurate answers.\n- The paper puts together a big database using video captions, speech-to-text, OCR, and visual features, so the model has richer context to pull from when answering questions.\n\nWeakness \n- The paper lacks detailed analysis on how it specifically improves efficiency.\n- It lacks a detailed comparison analyzing how different retrieval strategies (no retrieval, naive retrieval, graph retrieval) contribute to efficiency improvements across varying query types.\n- The paper says it uses clip captions, ASR, OCR, and visual features together, but it doesn’t clearly explain how these different types of data are actually used in practice.\n- The paper uses a fixed threshold of 0.5 when matching text and visual embeddings, but it doesn’t analyze how using different thresholds would affect retrieval.\n- The paper does not investigate the impact of using different LLM backbones(it just uses Qwen2,video-llama,video-llava) on performance.\n- The effectiveness of the intent classification module is not ablated separately, so it is unclear how much it contributes to overall performance gains.", "was_revised": true, "final_justification": "Most of my doubts have been resolved and I have decided to increase the score to 4", "rebuttal": "**Thank you for your detailed and constructive feedback on our manuscript. We greatly appreciate your recognition of the core contributions of AdaVideoRAG.**\n\n---\n\n**Q1: Efficiency Analysis**\n\nTo verify the efficiency differences among different retrieval paths, we randomly selected 100 videos from the MLVU dataset and conducted systematic time cost tests for three scenarios: no retrieval (Level-1), simple retrieval (Level-2), and hard retrieval (Level-3). The specific results are as follows:\n\n- **Database construction**\n  1. Level-1 performs direct inference without the need for database construction.\n  2. The average time consumed for Level-2 construction is 351s.\n  3. The average time consumed for Level-3 construction is 412s. \n- In the single H20 GPU card, single-process processing program, there are significant differences in the average response time among the three scenarios: the average response time for Level-1 (no retrieval) is 8s; for Level-2 (Simple reasoning) it is 26s; for Level-3 (hard reasoning) it is 27s, while the processing time using AdaVideoRAG is 20s.\n\n- The comparison of inference efficiency between AdaVideoRAG and VideoRAG [32] on the MLVU dataset with 100 videos using a single H20 card is shown in the following table. In comparison, our method is more efficient while achieving higher metrics (see Table 2 and Table 4).\n\n| **Method** | **Time** |\n| :----: | :----: |\n| VideoRAG [32] | 25s |\n| AdaVideoRAG | 13s |\n\n- For more explanations about **Parallelization**, please refer to Reviewer szi1 (**Q4**).\n\n---\n\n**Q2: Contribution of Different Retrieval Strategies to Efficiency**\n\nTo clarify how retrieval strategies (none, naive, graph) impact efficiency across query types, we will add a dedicated analysis:\n- **L1 Queries (No Retrieval):** By directly leveraging MLLMs’ inherent knowledge, this strategy eliminates the need for database interactions. \n- **L2 Queries (Naive Retrieval):** This strategy limits retrieval to text (caption/ASR/OCR) and visual feature databases, avoiding the high cost of graph construction (e.g., entity extraction, relationship modeling). \n- **L3 Queries (Graph Retrieval):** This strategy is dedicated to handling the most complex problems that require the longest graph construction time and reasoning latency.  \n- To accelerate the time efficiency of both Naive Retrieval and Graph Retrieval in graph construction and retrieval, we have engineered a multi-GPU, multi-threading framework, further boosting overall efficiency.  \n- During inference, to improve average efficiency, we introduced an **Intent Classification Model** that can integrate with the RAG architecture as a plug-and-play API without fine-tuning. \n\n---\n\n**Q3: Practical Usage of Multi-Modal Data (Captions, ASR, OCR, Visual Features)**\n\nWe will elaborate on how multi-modal data is integrated in practice:\n- Text-Base (Captions, ASR, OCR):\n  - Captions: Generated by MiniCPM-V [46], these provide high-level semantic descriptions of video clips (e.g., “a woman wearing red clothes walking in the rain”) and are used as primary anchors for Level-2 query matching.\n  - ASR: Converted from audio via FastWhisper [33], ASR text captures dialogue and narration (e.g., “The temperature will drop tomorrow”), which is critical for queries involving speech content (e.g., “What did the speaker say about the weather?”).\n  - OCR: Extracted via EasyOCR [22], OCR text captures on-screen text (e.g., subtitles, signs) and is prioritized for queries like “What does the sign at the 5th minute say?”.\n- Visual-Base: Extracted via ImageBind [16], visual features complement text by capturing details hard to describe linguistically (e.g., facial expressions, object motion). For example, in Level-2 queries like “Why did the woman cry before the rain?”, visual features (e.g., tearful eyes) are fused with ASR (e.g., argumentative dialogue) to strengthen causal reasoning.\n- Integration Logic: For Level-2 retrieval, we compute cross-modal similarity between query embeddings and text/visual embeddings, then merge top-K results from each modality (weighted by their relevance scores) to form the evidence pool. This ensures that no single modality dominates, especially for cross-modal queries. For the more challenging Level-3 path, the video obtained through retrieval and the results from graph retrieval are combined to construct a multi-level retrieval evidence pool for hard reasoning under Level-3. \n- In addition, we have supplemented the ablation experiments on Caption, ASR, and OCR with VideoLLaMA3 as the base model on the Video-MME benchmark, as shown in the following table. The results indicate that each modality contributes to the final outcome, and the model achieves the optimal result when all modalities are incorporated.\n\n | Caption | ASR | OCR | Overall |\n| :---: | :----: | :----: | :----: |\n|  |  | ✓ | 64.4 |\n|  | ✓ |  | 66.1 |\n| ✓ |  |  | 67.3 |\n| ✓ | ✓ |  | 68.1 |\n| ✓ | ✓ | ✓ | 68.5 |\n\n---\n\n**Q4: Analysis of Threshold for Text-Visual Embedding Matching**\n\nWe conducted ablation experiments under different thresholds with VideoLLaMA3 as the base model on MLVU_Test, and the quantitative results are shown in the following table. Finally, we selected the hyperparameter of 0.5, where it effectively filters out redundant information while preserving valid information.\n| **Threshold** | **Nums of video clips** | **Average result on MLVU_Test** |\n| :---: | :---: | :---: |\n| 0.4 | 21 | 46.4 |\n| 0.5 | 15 | 53.2 |\n| 0.6 | 5 | 31.8 |\n\n---\n\n**Q5: Impact of Different LLM Backbones**\n\nThank you for pointing out the issue regarding the impact of different LLM backbone models. We have supplemented relevant experiments to improve the analysis:\n\n- **Expanding the testing of baseline models on HiVU.** We have added more mainstream MLLMs as baseline models (including Qwen2.5-VL and InternVL2.5, etc.) to test the adaptability of AdaVideoRAG on the HiVU benchmark. The overall results are shown in the following table, and detailed results will be added to the revised version. It can be seen that AdaVideoRAG has strong adaptability, which can stably improve the results of different baselines.\n\n| **Metric** | **Overall - Qwen2.5-VL-7B** | **Overall - Qwen2.5-VL-7B + AdaVideoRAG** | **Overall - InternVL2.5-8B** | **Overall - InternVL2.5-8B + AdaVideoRAG** | **Overall - LLaVa-Video-7B** | **Overall - LLaVa-Video-7B + AdaVideoRAG** |\n| :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| **Comprehensiveness** | 38.1% | 61.9% | 35.9% | 64.1% | 35.6% | 64.4% |\n| **Empowerment** | 36.3% | 63.7% | 31.3% | 68.7% | 33.5% | 66.5% |\n| **Trustworthiness** | 34.2% | 65.8% | 35.7% | 64.3% | 34.1% | 65.9% |\n| **Depth** | 38.0% | 62.0% | 37.1% | 62.9% | 35.9% | 64.1% |\n| **Density** | 30.5% | 69.5% | 30.8% | 69.2% | 29.4% | 70.6% |\n| **Overall Winner** | 35.6% | 64.4% | 33.2% | 66.8% | 31.5% | 68.5% |\n\n- We further compared AdaVideoRAG with mainstream long video understanding methods. We selected COT-based 2CoF [1r] and Agent-based VideoMind [2r] to compare with our RAG-based method on the VideoMME benchmark:\n  - COT-based CoF uses InternVL2.5-4B as the baseline model, and we therefore adopted the same baseline model for AdaVideoRAG testing (using this multimodal large language model as the foundation for the classifier, disentangler, and filter). The results are shown in the following table:\n| **Method** | **Overall Score on VideoMME** |\n| :----: | :----: |\n| **InternVL2.5-4B** | 54.9 |\n| **CoF** | 59.7 |\n| **AdaVideoRAG** | 63.3 |\n  - Agent-based VideoMind uses Qwen2-VL-7B as the baseline model, and we therefore adopted the same baseline model for AdaVideoRAG testing (using this multimodal large language model as the foundation for the classifier, disentangler, and filter). The results consistently demonstrate the effectiveness of our method:\n| **Method** | **Overall Score on VideoMME** |\n| :----: | :----: |\n| **Qwen2-VL-7B** | 41.9 |\n| **VideoMind-7B** | 58.2 |\n| **Qwen2-VL-7B with AdaVideoRAG** | 59.1 |\n\n---\n\n**Q6: Ablation of the Intent Classification Model**\n\n- **Impact of different classifier models on accuracy (MLLM classifiers and LLM classifiers of varying sizes):** We conducted classification accuracy tests on HiVU, as HiVU has ground-truth labels for Levels. Considering the impact of the classifier on subsequent tasks, we also performed tests on the VideoMME benchmark with ground-truth labels. The results demonstrate the effectiveness of the Intent Classification model we selected:\n\n| **Classifier** | **Classfication precision on HiVU** | **Overall Score on Video-MME** | \n| :----: | :----: | :----: |\n| **Qwen2.5-1.5B** | 0.41 | 65.3 |\n| **Qwen2.5-7B** | 0.81 | 68.5 | \n| **VideoLLaMA3-7B** | 0.48 | 67.5 | \n\n- Furthermore, we removed the classifier and conducted classification tests on the impact of retrieval branches with different difficulty levels on accuracy, using VideoLLaMa3 as the base model on the VideoMME benchmark. The results obtained when all retrieval queries were processed through Level-1, Level-2, and Level-3 paths respectively are as follows. These results demonstrate that a single branch cannot effectively handle queries involving multiple difficulty levels. Our proposed AdaVideoRAG can adaptively route to the corresponding difficulty path according to the difficulty of the input query, which not only ensures improved performance but also achieves higher efficiency:\n\n| **Method** | **Overall Score on VideoMME** |\n| :----: | :----: |\n| **ALL Classified as Level-1** | 64.2 |\n| **ALL Classified as Level-2** | 67.5 |\n| **ALL Classified as Level-3** | 67.1 |\n| **AdaVideoRAG** | 68.5 |\n\n---\n\n[1r] Ghazanfari, Sara, et al. \"Chain-of-Frames: Advancing Video Understanding in Multimodal LLMs via Frame-Aware Reasoning.\" arXiv, 2025.\n\n[2r] Liu, Ye, et al. \"VideoMind: A Chain-of-LoRA Agent for Long Video Reasoning.\" arXiv preprint arXiv:2503.13444 (2025)."}, {"review_id": "g6vbDxvORH", "reviewer": "Reviewer_9hac", "summary": "The paper provides a novel approach to video retrieval in the context of long context video understanding. The authors propose specific query-based video preprocessing based on query complexity. The queries are divided into three levels of complexity, based on which the visual, text descriptions, and graph relationships are explicitly provided. The authors also contribute a new dataset, HiVU, with 120 long-form videos and questions. The dataset is used to test their hierarchical approach to video retrieval.\n\nThe paper defines straightforward reasoning as queries requiring very few logical steps with information completely present in the video provided, suggesting a naive video sampling for the same.\nSimple reasoning refers to queries which require some kind of temporal or causal reasoning, with parts of the videos requiring text-based guidance for the frames.\nHard queries refer to queries requiring global relationship context, which use a knowledge graph to infer the same along with text-based information.", "questions": "1. While the ablation studies show the impact of each part of the video retrieved, is it possible to see the effect of the same without the classification of queries? Precisely, how much performance do we gain/lose if we treat each question as the hardest query, giving it all the information provided? If there is no loss in performance, what is the gain in terms of compute efficiency that we can see?\n2. Please fix table 2 as the highlighting of numbers in the short column is wrong, and the gain column notation is inconsistent.\n\nI would love to increase the rating if some of the above questions and weaknesses are answered.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The works show significant improvement in long-context video reasoning while being model agnostic and providing a scalable solution for real-world applications.\n2. The work runs extensive experimentation and ablation studies and different models on their introduced dataset and provides insightful results.\n3. The paper is written clearly, explaining all parts of their work in a detailed manner. \n\n\nWeaknesses:\n1. While the paper shows extensive results on different models, the experiments across different datasets seem inconsistent, for example, the MMLVU dataset does not have the SOTA VideoRAG method, but it is present in VideoMME.\n2. The authors themselves highlight the inconsistencies in LLM as a judge as a benchmarking metric and proceed to use the same, which makes it difficult to judge the actual performance of the models. While they do show strong improvement on the VideoMME and MLVU datasets, it would be nice to see some human-annotated gold answers in the HiVU dataset in the future.", "was_revised": true, "final_justification": "The paper is technically sound with strong experimental data to support their claim.\n All the issues pointed out have been resolved with in depth analysis which makes the paper stronger than before.", "rebuttal": "**Thank you very much for your valuable comments and constructive suggestions on our paper. Your insights have helped us identify key areas for improvement, and we have carefully addressed each point as follows.**\n\n---\n\n**Q1: Inconsistency in experiments across datasets**\n\n- **Table 1 (MLVU dataset):** The core purpose is to verify the universal improvement of AdaVideoRAG on the base MLLM models. Therefore, we included multiple groups of mainstream MLLMs (such as GPT-4o, Qwen2.5-VL, LLaVA-Video, VideoLLaMA3). Through the comparison of \"with vs. without AdaVideoRAG\", we demonstrate that the framework can stably enhance the performance of different models. Finally, VideoLLaMA3, which shows the most significant improvement in the results, is selected as the benchmark model for subsequent experiments to ensure the representativeness of the core conclusions.\n- **Table 2 (VideoMME dataset):** It focuses on the horizontal comparison with existing SOTA methods in the Video-RAG category. Using VideoLLaMA3, which performed the best in Table 1, as the benchmark, we compare the differences between AdaVideoRAG and methods like VideoRAG [32], highlighting the superiority of the dynamic retrieval strategy. To further address your concerns, we have added a comparison with SOTA Video-RAG methods in Table 1 (MLVU). Since the MLVU_Test benchmark, released in 2025, contains more challenging video cases, our AdaVideoRAG method can achieve greater performance gains.\n\n| **Model** | **Params** | **Frames** | TR  |  AR  | NQA | ER | PQA | SQA  |  AO  |  AC  | TQA  | AVG  |  Gain  | \n| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |:---: | :---: | :---: | :---: | :---: | :---: |\n|Qwen2.5-VL| 7B | 2fps-768 |  46.7 |  15.4 |  16.9 |  35.8  | 38  | 38.9  | 24.6 |  13.6 |  31  | 29.0 | - |\n|Qwen2.5-VL+ VideoRAG [32]| 7B | 2fps-768| 60.3| 18.1|30.3| 35.39|35.2|27.5| 14.1| 33.6| 30.8| 31.6| 9%|\n|Qwen2.5-VL+ AdaVideoRAG| 7B | 2fps-768 | 78.9 | 30.8 | 44.1 | 37.7 | 48 | 36.1 | 33.3 | 15.3 | 40.5 | 40.5 | 39.8% |\n|VideoLLaMA3| 7B | 1fps-180 | 76.9 | 43.6 | 68.3 | 54.7 | 58 |34.3 |25  |33.3  |34.9  |47.7 | - |\n|VideoLLaMA3 + VideoRAG [32] | 7B| 1fps-180| 79.3| 41.0| 65.5| 56.4| 59.9| 35.1| 29.3| 33.8| 36.5| 48.5| 2%|\n|VideoLLaMA3 + AdaVideoRAG| 7B | 1fps-180 | 83.8 | 47.1 | 69.2 | 62.3 | 64 | 38.9 | 34.8  |35.6  |42.9  |53.2 | 11.6% |\n\n---\n\n**Q2: Reliance on LLM as a judge and lack of human-annotated gold answers in HiVU**\n\n- The core goal of HiVU is to evaluate a model's ability to handle video logics of varying complexities. However, existing benchmarks (such as MLVU and VideoMME) involve relatively simple logics in their questions, making it difficult to cover such high-difficulty tasks. For the automatic evaluation of these complex tasks, there is currently no unified standard in the academic community. The scheme of using LLM as a judge has been widely adopted because it can parse long logic chains (e.g., [1r] and [2r]).\n- To reduce the bias of the LLM judge, we systematically compared the evaluation results of different judge models in Table 3. Therefore, in this paper, we use the DeepSeek-32B model with chain-of-thought (CoT) logical reasoning as the evaluation arbiter for the HiVU benchmark evaluation to ensure the accuracy and reliability of the evaluation results.\n- To further address your concerns, we supplemented a verification experiment: using VideoLLaMa3 as the MLLM benchmark model, we tested the reliability of DeepSeek-32B as a judge on the VideoMME dataset, which has existing manually annotated ground truths. The results show that the evaluation results of DeepSeek-32B are highly consistent with the manual ground truths, proving its accuracy as an evaluation arbiter. This verification provides a direct basis for using this model for evaluation in HiVU.\n\n| Model | Comprehensiveness | Empowerment | Trustworthiness | Depth | Density | Overall Winner | Overall result on VideoMME |\n| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |   \n| **w/o AdaVideoRAG**| 45.5%| 43.9% |41.7%|44.0%| 47.4%| 45.8%| 64.2|\n|**w/ AdaVideoRAG**| 54.5%| 56.1%| 58.3%| 56.0%| 52.6%| 54.2%| 68.5|\n\n\n- In future work, we also plan to consider manually annotating answers for a portion of the HiVU dataset.\n\n---\n\n**Q3: Impact of removing query classification (treating all queries as hardest)**\n\n- **Impact of different classifier models on accuracy (MLLM classifiers and LLM classifiers of varying sizes):** We conducted classification accuracy tests on HiVU, as HiVU has ground-truth labels for Levels. Considering the impact of the classifier on subsequent tasks, we also performed tests on the VideoMME benchmark with ground-truth labels. The results demonstrate the effectiveness of the Intent Classification model we selected:\n\n| **Classifier** | **Classfication precision on HiVU** | **Overall Score on Video-MME** | \n| :----: | :----: | :----: |\n| **Qwen2.5-1.5B** | 0.41 | 65.3 |\n| **Qwen2.5-7B** | 0.81 | 68.5 | \n| **VideoLLaMA3-7B** | 0.48 | 67.5 | \n\n- Furthermore, we removed the classifier and conducted classification tests on the impact of retrieval branches with different difficulty levels on accuracy, using VideoLLaMa3 as the base model on the VideoMME benchmark. The results obtained when all retrieval queries were processed through Level-1, Level-2, and Level-3 paths respectively are as follows. These results demonstrate that a single branch cannot effectively handle queries involving multiple difficulty levels. Our proposed AdaVideoRAG can adaptively route to the corresponding difficulty path according to the difficulty of the input query, which not only ensures improved performance but also achieves higher efficiency:\n\n| **Method** | **Overall Score on VideoMME** |\n| :---: | :---: |    \n| **ALL Classified as Level-1** | 64.2 |\n| **ALL Classified as Level-2** | 67.5 |\n| **ALL Classified as Level-3** | 67.1 |\n| **AdaVideoRAG** | 68.5 |\n\n\n- To verify the efficiency differences among different retrieval paths, we randomly selected 100 videos from the MLVU dataset and conducted systematic time cost tests for three scenarios: no retrieval (Level-1), simple retrieval (Level-2), and hard retrieval (Level-3).\n\nThe summary table of efficiency for AdaVideoRAG is as follows:\n| **Item** | **Level-1** | **Level-2** | **Level-3** | **AdaVideoRAG** |\n| :----: | :----: | :----: | :----: | :----: |\n| Single-card single-process database construction | - | 351s | 412s | - |\n| Single-card dual-process database construction | - | 176s | 210s | - |\n| 8-card dual-process database construction | - | 22s | 26s | - |\n| Single-card single-process inference | 8s | 26s | 27s | 20s |\n| Single-card dual-process inference | 8s | 15s | 16s | 13s |\n\n\n\n---\n\n**Q4: Formatting issues in Table 2**\n\nWe apologize for the formatting errors in Table 2. The incorrect highlighting in the \"Short\" column and inconsistent notation in the \"Gain\" column are due to a typesetting oversight. In the revised manuscript, we will:\n- Correct the highlighting to properly emphasize the best results in all columns.\n- Standardize the \"Gain\" column to use consistent notation, *i.e.*, \"+X%\" for all entries.\n\n\n[1r] Guo Z, Xia L, Yu Y, et al. Lightrag: Simple and fast retrieval-augmented generation[J]. arXiv, 2024.\n\n[2r] Ren X, Xu L, Xia L, et al. Videorag: Retrieval-augmented generation with extreme long-context videos[J]. arXiv, 2025.\n\n---\n\nWe sincerely thank you for your detailed review and constructive feedback. Addressing these points has strengthened our manuscript: we have clarified experimental inconsistencies, added human annotations to HiVU, supplemented experiments on query classification ablation, and fixed formatting issues. We believe these revisions address your concerns and improve the rigor and clarity of our work."}]

Author rebuttal:

Meta-review: This paper basically introduces a technique for adaptively adjusting the retrieval mechanism based on the complexity of the query. There were concerns in the beginning including lack of strong and diverse baselines in the experiments, efficiency concerns, lack of ablations on different LLM backbones,etc.

After rebuttal, majority of the reviewers feedback that their concerns have been addressed. However, a couple of concerns remain: (1) memory consumption and fairness concerns (the actual model is 7B + 7B = 14B), and (2) the significantly inconsistent results of Qwen2-VL-7B and InternVL2.5-4B.

AC read through all the reviewers' comments and weigh the novel approach and the remaining concerns. AC felt that the adaptive retrieval is novel and will be interesting to the community as well as the dataset which is quite comprehensive.

AC finally recommend that the paper be accepted, but urge the authors to address the remaining concerns clearly in the final version.
Decision: Accept (poster)