VideoSAVi: Self-Aligned Video Language Models without Human Supervision
Authors: Yogesh Kulkarni
Pooyan Fazli
Venue: COLM 2025 (2025)
Keywords: Video understanding
Self-alignment
Video-language models
Direct preference optimization
Self-critiquing
TL;DR: VideoSAVi introduces a self-aligning approach that enables video-language models to generate high-quality preference pairs from their own outputs, achieving state-of-the-art performance without external supervision.

Abstract: Recent advances in video-large language models (Video-LLMs) have led to significant progress in video understanding. Current preference optimization methods often rely on proprietary APIs or ground-truth captions to generate preference data (i.e., pairs of model outputs ranked based on their quality or alignment with human judgment), which is then used to train models for video-language alignment. This approach is both costly and labor-intensive. To address this limitation, we introduce $\textbf{VideoSAVi}$ ($\underline{\textbf{S}}$elf-$\underline{\textbf{A}}$ligned $\underline{\textbf{Vi}}$deo Language Model), a self-training pipeline that enables Video-LLMs to reason over video content without external supervision. Our approach includes a self-critiquing mechanism that identifies reasoning errors in the model's initial responses and generates improved alternatives, creating preference pairs directly from video content. VideoSAVi then applies Direct Preference Optimization (DPO) to iteratively train the model using the preference data, thus enhancing its temporal and spatial reasoning for video understanding. Experiments show that VideoSAVi delivers significant improvements across multiple benchmarks, including a +4.2 percentage point gain on MVBench, +3.9 on PerceptionTest, and +6.8 on the challenging EgoSchema dataset compared to baseline models. Our model-agnostic approach is computationally efficient, requiring only 32 frames, offering a promising direction for self-aligned video understanding without reliance on external models or annotations.

Published as a conference paper at COLM 2025
VideoSAVi: Self-Aligned Video Language Models without Human Supervision
Yogesh Kulkarni
Pooyan Fazli
Arizona State University
[ykulka18, pooyan Jeasu. edu nttps://people-roee robots, gi thub. 1oVVidosAA//
Abstract
Recent advances in video-large language models (Video-LLMss) have led to significant progress in video understanding. Current preference optimiza- tion methods often rely proprietary APIs human-annotated captions to generate preference data (i.e., pairs of model outputs ranked by quality alignment with human judgment), which is then used to train models for video-language alignment. This approach is both costly and labor-intensive. To address this limitation, introduce VideoSAVi (Self-Aligned Video Language Model), a self-training pipeline that enables Video-LI.Ms to learn from video content without external supervision. Our approach includes self-critiuuing mechanism that identifies reasoning errors in the model's initial responses and generates improved alternatives, creating preference pairs directly from video content. VideoSAVi then applies Direct Preference Optimization (DPO) iteratively train the model using the preference data, thus enhancing its temporal and spatial reasoning for video understanding Experiments show that VideoSAVi delivers significant improvements across multiple benchmarks, including +4.2 percentage point gain on +3.9 on Perception Test, and +6.8 on the challenging EgoSchema dataset compared to baseline models. Our model-agnostic approach is computa- tionally efficient, requiring only 32 frames, offering a promising direction for self-aligned video understanding without reliance external models or annotations.
1 Introduction
Vision-languaae models (VLMs) (Liu et al., 2023; Li al., 2023a; Radford et al., 2021; Wang et al., 2024b; Chen et al., 2024; Li et al., 2024a) have made significant strides by integrating visual perception with the reasoning capabilities of large language models (LLMs) (OpenAi, 2024; Dubey et al., 2024; Ouyang et al., 2022). These models excel in interpreting and generating contextually relevant responses through the combination of image encoders and language generation techniques. Building on this foundation, recent video-large language models (Video-LLMs) (Zhang al., 2023; Lin et al., 2024; Li et al., 2024b; Zhang et al., 2024d; Wang et al., 2022; 2024e) incorporate temporal information by converting video frames into tokens that LLMs can process, allowing them to better analyze video content. While Video-LIMs show impressive capabilities, they generally rely on large, high-quality annotated datasets, which are resource-intensive to obtain and limit scalability.
Instruction tuning has been pivotal in advancing both VLMs and Video-LLMs (Liu et al., 2023; Brown et al., 2020; Xu et al., 2023; Wei et al., 2021; Wang et al., 2024b; Chen et al., 2024; Zhang et al., 2024c), but creating large-scale video instruction datasets is costly (Deng et al., 2024). Recent efforts to generate massive instruction datasets (up to 1.3M pairs) by distilling knowledge from proprietary models like GPT-4V (Zhang et al., 2024d) have yielded only marginal improvements. This reliance on extensive annotated data and proprietary models limits the adaptability of Video-I.LMs, creating barriers to wider adoption.

---

Published as a conference paper at COLM 2025
Alignmentt-ased approaches have shown promise for improving video understand- ing (Kulkarni & Fazli, 2025a). LLaVA-Hound (Zhang et al., 2024b) uses Direct Preference Optimization (DPO) (Rafailov et al., 2023) with text-based rankings from proprietary mod- els, while TPO (Li et al., 2025b) focuses on temporal preference optimization by sampling negatives from video segments outside the target clip. However, both methods rely heavily on proprietary APIs or ground-truth captions, which limits their accessibility and scalability.
This raises a critical research question: How can we train Video-LLMs generate high-quulltyy outputs without relying on proprietary models, ground-trute captions, or expensive human annola- tions while maintaining robust temporal and spatial reasoning?
To address this challenge, we propose VideoSAVi, a novel framework requiring no external supervision beyond the model itself. Our approach focuses on post-training refinement through a four-stage process: (1) generating diverse reasoning questions and initial answers about video content, (2) self-critiqung these answers to identify factual errors and reasoning flaws, (3) revising the responses based on the sell--gneeated feedback, and (4) leveraging these pairs for DPO to align the model towards improved reasoning capabilities.
Unlike prior approaches that depend on proprietary models or extensive annotations, VideoSAVi uniquely generates high-quality preference pairs through self-critique, identify- ing specific reasoning failures in both spatial relationships and temporal sequences. This self-supervision creates subtle yet unambiguous learning signals that, when optimized through DPO, enable efficient post-training refinement of capabilities already present in the model. By directly addressing the model's own weaknesses rather than conforming to exter- nal judgment, our approach delivers significant improvements across diverse benchmarks without the high computational cost of full retraining.
Our main contributions are follows:
1. We introduce VideoSAVi, a novel self-training framework that enables Video-LLMs to reason over video content without external supervision, while maintaining computational efficiency.
2. We develop quality-aware self-critique mechanism that generates preference pairs by focusing on both temporal and spatial reasoning, enabling comprehensive video understanding.
3. Through extensive experiments, we demonstrate that VideoSAVi delivers significant im improvements across multiple benchmarks, including +4.2 percentage points on MVBench, +3.6 on NeXTQA, +3,9 on PerceptionTest, and +6.8 on EgoSchema. Moreover, VideoSAVi consistently improves various model architectures, with an average gain of +2.1 percentage points across all benchmarks.
2 Related Work
Video-LLMs. Recent Video-LLMs have made remarkable progress in multimodal under- standing (Chen et al., 2024; Wang et al., 2024b; Li et al., 2024a;b; Zhang et al., 2024d). However, these models typically require massive instruction-tuning datasets and extensive computational resources for pre-training. Even with substantial training data, they often struggle with proper grounding in visual content (Fu et al., 2024; Zhou et al., 2024; Liu et al., 2024b; Kulkarni & Fazli, 2025b), particularly for videos (Wang et al., 2024a; Yao et al., 2024; Wang et al., 2024e; Song et al., 2024). While some approaches attempt to address these challenges through architecture modifications (Liu et al., 2024a; Shu et al., 2024; Li et al., 2024d; Wang et al., 2024d), specialized training objectives (Lee et al., 2024; Lan et al., 2024; Liu et al., 2024c), or inference-time adaptations (Yang et al., 2024; Hu et al., 2025; Xu et al., 2024), they typically focus specific aspects rather than holistic alignment. In contrast, VideoSAVi introduces a post-training refinement strategy that avoids training from scratch or adding complex components. Instead of introducing entirely new capabilities, our method identifies, amplifies, and improves reasoning skills already present in the model. By leveraging self-alignment through preference optimization, VideoSAVi enables models to correct their own ertors and strengthen existing temporal and spatial reasoning, without relying on external supervision or large new datasets.
Lecett
2

---

Published as a conference paper at COLM 2025
Learning from AI Feedback. Recent work adapts preference learning techniques to video- language models to improve aligmment. LLaVA-Hound-DPPS (Zhang et al., 2024b) applies Direct Preference Optimization (DPO) (Rafailov et al., 2023) to video understanding, but operates mainly at the text level, without incorporating visual context. It relies on preference pairs from proprietary models like GPT-4, introducing external dependencies. Similarly, Temporal Preference Optimization (TPO) (Li et al., 2025b) focuses solely on temporal rea- soning, ignores spatial relationships, and requires video captioning as an intermediate step, adding computational overhead. In contrast, VideoSAVi generates preference pairs directly from the model's own assessment of correctness, eliminating the need for external supervi- sion or auxiliary tasks like captioning. This self-alignment approach simultaneously targets both temporal and spatial reasoning. By critiquing its own responses, the model produces training signals that directly address its specific reasoning failures rather than conforming to external judgments. This results in a more efficient and focused refinement process, targeting the model's s actual weaknesses rather than pursuing general improvements.
Self- Soll-Training. Self-training is a powerful method for improving language model perfor- mance (Gulcehre et al., 2023; Singh et al., 2024; Huang et al., 2023; Zelikman al., 2022; Yeo et al., 2024; Wang et al., 2024c). The core idea is for models to generate their own training data and use it to refine their performance iteratively, with notable success in enhancing reasoning and task-specific capabilities. Recent advances extend self-training to vision- language models (VLMs) (Sun et al., 2025; 2024; Deng et al., 2024) and Video-LLMs (Zohar et al., 2024). However, applying self-training to the video domain introduces unique chal- lenges due to the complex spatiotemporal nature of videos. Existing methods often rely on expensive teacher models (Sun et al., 2024), require ground truth labels (Zohar et al., 2024), or focus on isolated aspects of video understanding. Moreover, they struggle to generate high-quality preference data that captures nuanced relationships between visual elements across time (Yin et al., 2024). Our work addresses these limitations by introducing a novel self-alignment framework tailored for video understanding. We integrate self-critiquing mechanism with preference- learning, enabling Video-LLMs to automatically detect and correct their own reasoning errors across spatial and temporal dimensions.
3 VideoSAVi
We presentt VideoSAVi, a novel self-aligned framework that enhances video-language models without relying external supervision. VideoSAVi leverages existing Video-LLMs to generate challenging questions, produce and critique answers, and refine them into high- quality outputs. This self-training pipeline addresses the high costs associated with human annotations and proprietary models, while also improving the model's temporal and spatial reasoning capabilities. VideoSAVi follows four-stage process: (1) generating reasoning questions and initial responses based on video content, (2) identifying temporal and spatial reasoning errors through self-critique, (3) revising responses based on the critique, and (4) optimizing model performance using DPO with the improved responses. Specifically, the framework prompts the baseline model, Intern VL2.5 (Chen et al., 2024), along with the video input, to produce diverse questions that probe complex temporal dynamics and spatial details. The model then answers these questions, critiques its own reasoning, and generates refined responses to guide further learning. Figure 1 illustrates the overall architecture and iterative self-training pipeline of VideoSAVi.
3.1 Question Generation and Initial Responses
For self-alignment to be effective, generated questions must target specific reasoning skills that can be refined through preference optimization. To this end, we prompt the baseline model using an instruction template (see Appendix, Figure 8) across two core reasoning dimensions:
Spatial Reasoning Questions. We use prompts such as "Generate questions that require identifying spatial relationships between objects" and "Generate questions about visual details that might be easily overlooked." " The resulting questions focus on spatial layouts, fine-grained visual details, and contrastive reasoning.
3

---

Published as a conference paper at COLM 2025
VM
Queetion Generation
Inftall Resporses
Temporal hropens
Temporal Responsec moving
Responses
Selt-Criique
Preferenceee Optimization (DPO)
Revised Spatial Respense: The ecult is standing bohlidd the chil, positoned to the right side from cemmra stest.
Spatiel Critique: Visual ovidence showa adult is standing behind the child, not beside The acult is positiondd to provide supportwthh tands.
Preforenci Generation
Temperal Responser moving down posttions
Temporal Ciitiuu:: woving
Rovisd
the
while pushos from behind, stationary.
(Digrollmed)
(Prteoed
stili.
Figure 1: Overview of VideoSAVi. Our iterative self-training pipeline consists of four key stages: (1) generating diverse spatial and temporal reasoning questions and initial responses about video content, (2) applying self-critique mechanism identify reasoning errors and inconsistencies, (3) refining responses based on the critique feedback, and (4) training the model through DPO using paired initial (dispreferred) and revised (preferred) responses. This self-supervised cycle progressively improves VideoSAVi's reasoning capabilities with- out external supervision.
Temporal Reasoning Questions. We use prompts such as "Generate a question about the temporal sequence of events" and "Generate questions that require understanding what happened before or after specific key moments." " The resulting questions challenge the model to track objects across frames and infer cause-and-effect relationships.
Despite advancements in Video-LLMs, their initial responses to such questions often re- veal reasoning errors in spatial relationships, temporal sequencing, and object tracking. VideoSAVi addresses these limitations through a self-critiquing mechanism that allows the model to evaluate and improve its own reasoning without external supervision.
3.2 Self-Critique Mechanism
Critique Generation. The self-critique process begins once the model generates an initial response do to a question-video pair (v,q). It then assumes the role of its own critic by prompting itself with a specialized template (see Appendix, Figure 9) designed to elicit critical evaluation of its reasoning:
c c= Scritique (v, q, (,;4;; 8).
(1)
The critique function analyzes the initial answer for inconsistencies by comparing its claims against visual evidence from the video frames. For spatial reasoning, it evaluates object positions, visibility, and attributes. For temporal reasoning, it examines event sequencing, causality, and transitions. The critique prompt encourages counterfactual reasoning by asking the model to consider alternative interpretations that may better align with the video content. This self-assessment mirrors human introspection, where evaluating one's own conclusions leads to refinement. Through this process, the model learns to balance confidence in its initial assessment with appropriate skepticism, particularly in the face of ambiguous visual input.
Error Identification. The critique identifies specific errors (e1, en) in the initial re- sponse. Rather than being learned through separate training process, the severity assess- ment function & is implicitly encoded in the model's parameters via preference optimiza- tion:
4; =
(2)
The model implicitly learns to assess error severity by developing an understanding, refined through preference optimization, of how different types of mistakes affect response quality.
4

---

Published as a conference paper at COLM 2025
It categorizes errors either critical (e.g., factual inaccuracies, logical contradictions) or minor (egg, imprecisions, stylistic issues) based on their influence on overall correctness.
The critique specifically targets common failure modes in video understanding, including hallucinations of non-existent objects or actions, errors in temporal ordering that misrepre- sent event sequences (Li et al., 2025a), incorrect causal attributions, misinterpretations of spatial relationships, and failures in tracking entities across frames.
3.3 Response Refinement
The refinement function is implemented via prompt template rather than a separately parameterized module. The initial response ag and the generated critique are combined into a new prompt (see Appendix, Figure 10) that instructs the model to produce an improved answer:
a1 (v
This refinement prompt includes instructions such as "Consider the following critique of your previous answer and provide an improved response that addresses these issues." " The model then generates a refined answer aj that incorporates the feedback from the self-critique.
Several principles guide the refinement process: maintaining factual correctness by ensuring all claims are directly observable in the video, preserving useful information from the original response while correcting errors, avoiding the introduction of new speculation that is not supported by visual evidence, and ensuring logical consistency throughout the response.
3.4 Preference Pair Creation and Optimization
The initial (dispreferred) response ao and the revised (preferred) response ai form pref- erence pair for DPO. These pairs are then used to train the model with the following loss function:
(D) - logor Blog xo(ailv.4) Teer(aylv,g) - B Blog No(aglv, 4)
]
(4)
Tret(aolv,
where za is the model policy being trained, Æyet is a reference policy derived from the initial model, and ß scaling hyperparameter. This objective encourages the model to assign higher probability to refined, more accurate responses (a1) over the initial, flawed ones (ap).
Through repeated iterations of the entire pipeline (i.e., initial response generation, self- critiquing, refinement, and preference optimization), VideoSAVi progressively improves its ability to generate accurate responses and identify and correct its own reasoning errors. This iterative process creates self-improving cycle that enhances both spatial and temporal reasoning capabilities without requiring external supervision.
4 Experiments and Evaluations
For training data, we sample diverse collection of 4,000 videos from three sources: 2,000 from Star (Wu et al., 2021), and 1,000 each from Vidal (Zhu et al., 2024) and WebVid (Bain et al., 2021). Our self-supervised pipeline yields comprehensive dataset of 24,000 prefer- ence pairs (6 pairs per video: spatial reasoning + 3 temporal reasoning x 4,000 videos). To ensure fair evaluation, we decontaminate the training data by removing any videos that overlap with the evaluation benchmarks. VideoSAVi uses state-of-the-art Intern (Chen et al., 2024) as its backbone and is optimized using the SWIFT (Zhao et al., 2024) framework. We employ LoRA (Hu et al., 2021) with a = (and rank = 8), and DPO with temperature 0.1. All training and evaluation conducted two NVIDIA L40S GPUs (48GB), with

---

Published as a conference paper at COLM 2025
TempCompass
PerceptionTst
NeXTQA
MVBlench
EgoSchema
LongVideollench
(1) Rnceliur Middls
V125 (Chen et, 2024)
68.3
62.2
77.0
69.8
52,0
57.8
-SFT
68.5
63.0
77.5
53.
58.
68.7
64.5
78.3
54.5
58.1
Hound-DPO (Zhang et al,, 2014b) (L. et al, 2125b)
66.8
61.0
74,8
64.2
48.5
54.3
68.2
62.0
77.2
68.8
52.8
58.1
(2) wfaull-gg--lha///1 Models
Lade (Cheng al,, 2024)
43.4
51.4
54.6
517
Kangaroo" (Liu 2024a)
LLaVA-NeTT-Vidae (Zhang et al., 2024))
33.0
48.8
53.5
53.1
49.1
LLaVA-NeXT Inteellleeeeeeeeeee (Lá al., 2024b)
54.1
51.2
67.0
46.5
51.0
44.8
Qwen2-VI (Wang 2024h)
62.3
25.7
99.2
85.6
11aVA- 11aVA.OneVision OneVison (Li et.,
64.5
29.3
64.0
56.3
LLaVA-Video (Zhang al, 2024d)
66.4
67.9
74.2
58.6
37.6
58.2
(3) Pwwpfforenne-pptiiiiie Molels
LlVVA-Huun--DPO (Zhang et, 2024b)
35.5
45.1
61.6
36.1
36.7
:-KT (Ahn et 2024)
56.0
47.0
63.0
36.3
46.2
38.2
LLVAA-Video-TVe TPO (Li et al., 20256)
66.6
66.3
77.8
56.7
36.0
56.3
VideoSAVi
69.1.aa
66 1,39
80.6.3a
74.0.4.
58.8.a
59.8,2
Table 1: Comprehensive evaluation of VideoSAVi against leading video understanding models. Best scores are in bold, and second-best scores are underlined. For VideoSAVi, performance improvements over InternVL. 2.5 are shown as subscripts (reported as absolute gains in percentage points). All results except those marked with t are reproduced using LMMs-Eval (Zhang et al., 2024a).
a maximum of 32 frames per video to avoid CUDA out-of-memory errors. The full training process consists of four iterations of self-training. Each iteration takes approximately 48 GPU hours (24 hours for generating preferences and 24 hours for DPO training), resulting in total of 1192 GPU hours across all iterations. For evaluation, we use L.MMs-Eval (Zhang et al., 2024a) to ensure consistent comparison with prior work. Additional experiments and analyses are provided in the appendix, including: iteration-wise performance (SA.1), pref- erence data composition (SA.2), training data composition (SA.3), DPO training dynamics (SA.4), qualitative examples (SA.6), dataset samples (SA.7), critique examples (SA.8), and prompt design ($A.9).
Benchmarks. We evaluate VideoSAVi on a range of general video understanding bench- marks, each targeting specific reasoning skill: MVBench for multi-task reasoning (Li et al., 2024c), Perception Test for visual perception (Patraucean et al., 2024), TempCompass for temporal understanding (Liu et al., 2024b), and NeXTQA for compositional reasoning (Xiao et al., 2021). To assess performance on long-form video understanding, we use EgoSchema, which features 3-minute egocentric videos (Mangalam al., 2023), and LongVideoBench, which includes hour-long videos across diverse tasks (Wu et al., 2024).
4.1 Results
We compare VideoSAVi with (1) baseline models built on InternVL2.5 (Chen et al., 2024), (2) current state-of-the-ad models, and (3) models enhanced through preference optimization. Table presents the evaluation results.
VideoSAVi achieves substantial gains over foundation models across all benchmarks. Compared to its foundation model, In2ern (Chen al., 2024), VideoSAVi shows significant improvements across all benchmarks: +0.8 percentage points on TempCompass, +3.9 on Testeption +3.6 on NeXTQA, +4.2 MVBench, +6.8 EgoSchema, and +2.0 on Long VideoBench. Supervised fine-tuning (SFT) on VInstruct (Maaz et al., 2023) and our aligned SFT* with preferred responses show modest improvements, but neither matches our model's generalization capabilities. Notably, VideoSAVi overcomes the limitations of Hou (Zhang et al., 2024b), which demonstrates that purely text-based ranking of preferences is fundamentally inadequate for video understanding, leading to severe performance degradation on MV MVBench (-5.6) and EgoSchema (-3.5).
VideoSAVi outperforms previous state-of-the-art key benchmarks through self- alignment. Our approach surpasses Qwen2-VL (Wang et al., 2024b) by +9.1 percentage points on MVBench achieving 74.0% accuracy, demonstrating significant improvements

---

Published as a conference paper at COLM 2025
Error Type
Iterl
Iter2
Iterr
Iter4
Test Condition
VideoSAVi TPO
H-DPO
Factual Inaceuracy
287
198
127
73
Training Dist.
74.5
71.2
69.8
Temporal Ordering
243
162
108
Cross-Question
70.2.4.
62.5.g.7
56.2.13.6
Spatial Relationship
208
147
92
48
Cross-Vided
69.8.4.7
61.8.9.
54.1.15.7
Object Hallucination
169
118
71
33
Full OOD
68.7-5.8
59.4.11.
51.3.18.5
Causal Reasoning
132
91
58
27
Adversarial
67.9-6.6
57.8.13.4
48.5.21.3
Detail Omission
118
87
44
16
Compositional
68.3.6.2
58.6.12.6
49.2.20.6
Total
1157
803
500
249
Avg. Gen. Gap
-6.4
-13.5
-17.9
Table Reasoning Errors Across Iterations.
Table 3: Generalization capability across
increasingly challenging test conditions.
over existing methods. On NeXTQA, VideoSAVi outperforms the previous best, LLaVA- VneVision (Li et al., 2024a), by +1.3 percentage points. While LLaVA-Video (Zhang al., 2024d) maintains an edge on PerceptionTest (67.9% vs. 66.1%), and LLaVA-OneVins leads on EgoSchema (64.0% vs. 58.8%), VideoSAVi demonstrates stronger generalization across the full set of benchmarks.
VideoSAVi redefines preference optimization for robust video understanding. Previ- ous preference-optimizee approaches show inconsistent performance across benchmarks. LLaVA-Hound DPO (Zhang et al., 2024b) and i-SRT (Ahn et al., 2024) suffer large perfor- mance drops, and TPO (Li et al., 2025b) fails improve temporal understanding despite its specific focus. In contrast, VideoSAVi delivers consistent improvements, outperforming TPO by +1.5 percentage points on LongVideoBens and reaching 59.8%. These findings challenge the assumption that preference optimization necessarily compromises consistency. Our self-alignment approach demonstrates robust performance across all dimensions of video understanding, maintaining strong capabilities in both temporal reasoning and spatial comprehension, where previous methods show significant variability.
4.2 Ablation Studies
Reasoning Errors Across Iterations. To address concerns about potential error reinforcement in our self-critique approach, we conduct error propagation analysis across four iterations of VideoSAVi (Table 2). We use 1200 video-question pairs generated by iteration 1 and have each subsequent iteration (2, 3, and 4) independently answer the same questions without access to previous responses. GPT-4o (Hurst et al., 2024) evaluates all answers using the prompt provided in Figure 12 (see Appendix). This setup tests for the risk of "selffdelusion," " where self-improving method might reinforce its own incorrect beliefs instead of correcting them. The results show a consistent and substantial reduction in errors, with total mistakes decreasing by 78.4%, from 1157 in iteration 1 to 249 in iteration 4. This reduction closely aligns with our iteration-wise performance improvements (Appendix SA.1), confirming that VideoSAVi learns to correct its reasoning rather than reinforcing errors.
Memorization vs. Generalization. Table evaluates VideoSAVi's ability to generalize be- yond its training distribution. We design comprehensive test bed (detailed methodology in Appendix $A.5) with increasingly challenging conditions: novel question formulations (Cross-Quie (Cross-Question) unseen videos (Cross- (Cross-Video), new domains (Full OOD), deliberately chal- lenging inputs (Adversarial), and cases requiring integrative reasoning (Compositional) VideoSAVi demonstrates exceptional robustness with an average generalization gap of only -6.4 percentage points. This resilience is particularly evident in the most challenging scenarios, in which VideoSAVi maintains 68.3% accuracy on compositional questions and 67.9% on adversarial examples. In contrast, TPO (Li et al., 2025b) shows a larger average gap of -13.5 percentage points, particularly struggling with adversarial reasoning (-13.4), while Hound-DPO (H-DPO) (Zhang et al., 2024b) shows the largest generalization gap (-17.9 percentage points), with performance dropping by nearly 21 percentage points on adversarial examples. The small performance variance across conditions confirms that our self-alignment approach develops genuine reasoning capabilities rather than memorizing patterns from training data.
7

---

Published as a conference paper at COLM 2025
Critic Type
MVB
ES
LVB
Model
MVB
ES
LVB
In2er (Baseline)
69.8
52.0
57.8
Smaller Models with Preference Learning
InternVL (1B)
63.5
39.6
45.4
SFTHound-DPO
64.2
48.5
54.3
+ VideoSAVi
64.8,13
43.8,42
48.3+2.9
SFTTPO
68.8
52.8
58.1
Intern VL (2B)
65.9
44.7
48.0
GPT-4o
72.0
57.0
59.0
Parameter Scaling Comparison
VideoSAVi
74.0
58.8
59.8
InternVL (2B)
65.9
44.7
48.0
VideoSAVi 67.5+1.6
49 49.4.4.7
52.3+4.3
Table 4: Effect of different critics on model performance. MVB: MVBench, ES: EgoSchema, LVB: LongVideobe BideoBecn.nc
Intern VL. (4B)
68.5
55.3
51.9
Table 5: Preference Learning vs. Model Scaling. MVB: MVBench, ES: EgoSchema, LVB: LongVideoBenct
Externally Trained Critics vs. Self-Critiquing. Table compares self-critiquing approach to external cri- tique methods. We evaluate: (1) SFT with GPT-4o- generated critiques of preference pairs from existing methods (SFTHound-DrO and SFTTRO), (2) GPT-4o cri- tique, and (3) VideoSAVi's self-critiquing framework. De- spite leveraging sophisticated external critique genera- tion (prompt in Appendix, Figure 11), both SFT variants underperform. Houndddd suffers significant drop (- 5.6 percentage points on MVBench) while SPO shows only minimal improvement. GPT-4o-based critiquing per- forms better (+2.2 percentage points on MVBench), yet VideoSAVi's self-critiquing consistently achieves superior results across all benchmarks (+4.2 percentage points on MVBench, +6.8 EgoSchema). While GPT-4o achieves strong results overall (Table 6), its critiques introduce out-of- distribution, off-policy preferences that misalign with the model's training distribution, diminishing DPO's effectiveness due to KL-divergence constraints, In contrast, VideoSAVi's on-policy critiques remain within the model's natural output distribution, enabling more stable and effective preference learning. These results highlight the advantage of integrating critique generation directly into the learning loop over relying on external critics.
Benchmark
GPT-4o
TempCompass
73.8
Perception
72.1
NeXTQA
88.1
MVBench
64.6
EgoSchema
66.2
Long Bong
66.7
Table 6: GPT-4o performance on video understanding bench- marks.
Parameter Scaling Preference Learning. Table 5 illustrates the comparative efficiency of preference learning vs. parameter scaling for enhancing video understanding, The results show that VideoSAVi consistently delivers substantial improvements across all model sizes and benchmarks. While larger models generally achieve better performance, VideoSAVi offers complementary and cost-effective improvements. For instance, 1B parameter model with VideoSAVi (48.3% on BongVideeBench) achieves comparable performance to baseline model (48.0%). This efficiency advantage becomes particularly significant when considering the computational resources required for scaling parameters vs. our approach, which requires only self-generated preference pairs.
Human Evaluation. To validate the quality of self gelf-generated preference pairs, we conduct rigorous human evaluation with 6 external evaluators across 516 videos (261 temporal and 255 spatial). Evaluators are shown videos along model-generated questions and preference pairs without revealing the reasoning category. A pair is considered correct if and only if: (1) the question is answerable from video content, (2) the preferred response is accurate, and (3) the dispreferred response contains clear errors. This strict protocol reveals strong performance for both spatial relationships (71.3%) and temporal ordering (67.1%), with an overall quality of 69.2%. These results are particularly notable given that the preference learning process is fully self-supervised, operating without human intervention or external models. This confirms that our self-critiquing mechanism identifies genuine reasoning errors rather than arbitrary preferences, demonstrating that self-alignment can generate high-quality training signals for complex video understanding tasks.
8

---

Published as a conference paper at COLM 2025
Model
TempCompass
PerceptionTest
NeXTQA
MVBench
EgoSchema
LongVideoBench
VaVeo (I et, 2024)
34.3
42.6
58.4
34.1
18.8
39.1
VideoSAVi
36.8 +2.5
43,9
38.7 -46
23.5 .47
41.8 +2.7
-etet-ee (Li al., 2024b)
53.2
51.0
67.3
46.5
51.0
44.8
VideoSAVi
55.6
514 414
712-39
50.3
51.6 -16
47.9
HaaVA-OheVision (I.etal,, 2024a)
64.1
57.5
79.3
56.7
64.0
56.3
VideoSAVi
66.3 +22
60.8 .a3
80.9 -16
99.9-32
65.7 -1.
58.7 -24
Owen2 VL (Wang ctal., 2024b)
68.9
62.1
75.6
64.9
39.2
55.6
VideoSAVi
68.4 -0.
63.4 .33
78.8. .32
68.3 3.4
62.8 -38
58.1 12.5
Owen2.5VL (Bai et al., 2025)
72.3
68.6
75,8
63.2
60.9
60.7
+VideoSAVi
72.4
68.9
73.9
63.4
61.0
60.7 +0.0
Qwen22550mml (Xu 2025)
69.6
66.6
73.2
63.0
61.8
56.9
+VideoSAVi
69.9 v8.
66.9 40.3
75.6 .04
64.3 -13
61.9 -0.1
57.2 .0.
Table 7: Model-agnostic improvements from VideoSAVi.
60.5
81
80
68.0
67.5
76
75
30
40
50
80
202
20
60
70
80
90
100
30
40 50
100
Parceddgge Preference Parrs
Perce
of Proferance Parm
Percartage Profemance Palm
(a) TempCompass
(b) Perception
(c) NeXTQA
14
72-
poteech
70
66
66
120
(d) MVBench
(e) EgoSchema
Long
Figure 2: Impact of preference data size model performance.
Model-Agnostic Improvements. Table 7 demonstrates the broad applicability of our ap- proach across diverse architectures. VideoSAVi consistently enhances performance, deliver- ing substantial improvements on nearly all benchmarks, regardless of the base model. The gains are particularly notable for VideoLLa (Lin et al., 2024), with improvements of +4.8 percentage points on NeXTQA, +4.7 on EgoSchema, and +4.6 on MVBench, showing that models with greater headroom benefit most from our preference optimization. For mid-tier models like LLaVA-NexT-Innteleaw (Li et al., 2024b), VideoSAVi provides well-balanced improvements (+3.9 on NeXTQA, +3.8 on MVBench, +3.6 on EgoSchema), strengthening both spatial and temporal reasoning capabilities. Notably, VideoSAVi also enhances top- (Li et al., 2024a) by +1.6 on NeXTQA,
performing models, improving LLa LaVA--Oneissio reaching an impressive 80.9% and boosting Qwen2-VL (Wang et al., 2024b) across five bench- marks. For state-of-the-ate models like Qwen2.5VL (Bai et al., 2025) and Qwen2.5-Omni (Xu et al., 2025) that have already undergone post-training refinements using DPO, VideoSAVi still delivers consistent but more modest improvements, with notable gains MV Bench (+0.2 and +1.3 respectively), demonstrating our method's effectiveness even when applied to already-optimizedd models. These consistent arossterchee improvements, averaging +2.1 percentage points across all models and benchmarks, confirm that our self-aligning preference learning framework addresses fundamental challenges in video understanding rather than exploiting model-specific characteristics.
Impact of Preference Data Size. Figure 2 demonstrates how VideoSAVi's self-alignment pipeline consistently improves performance across all benchmarks as the percentage of pref- erence pairs from the final (fourth) iteration of self-training increases. In contrast, TPO (Li et al., 2025b) shows minimal improvements or plateaus (particularly on FgoSchema), and

---

Published as a conference paper at COLM 2025
Ability
Benchmark Base + VVieeoSAi
Model
MVB
ES
LVB
Multimodal
MMMU
53.1
53.2
InternVL2.5
69.8
52.0
57.8
Document
DocVQA
91.3
90.9
Noisy Pref. 71.2
57. 57.1
58.2
OCR Text
TextVQA
79.2
79.8+0.6
Reasoning
GSM8k
77.4
76.9-0.5
VideoSAVi 74.0
58.8
59.8,2.0
Video Cap.
VidChat
2.73
3.0.0.27
Hallucination POPE
90.1
90.3.0.2
Table 9: Robustness to preference data noise. MVB: MVBench, ES: EgoSchema, LVB: Long BengeBBen.....
Table 8: VideoSAVi preserve core capabili-
ties.
Hound-DPO (Zhang et al., 2024b) experiences notable performance drops after initial im- provements. VideoSAVi's robustness and steady progress throughout training stems from its self-critiquing mechanism that targets challenging reasoning cases by effectively identifying subtle temporal inconsistencies and spatial reasoning errors. The steep improvement curve on EgoSchema (+6.8 percentage points) highlights how our approach excels at complex ego- demonstrate
centric understanding tasks, while the consistent gains on Long effective temporal alignment even when competing methods struggle with longer-form content.
Preservation of Core Capabilities. To address concerns about potential overfitting during self-training, we evaluate VideoSAVi's performance across a diverse set of benchmarks, assessing the backbone model's pre-training performance (Chen et al., 2024). As shown in Table 8, VideoSAVi largely maintains or improves performance across all domains, pre- serving multimodal understanding (MMMU (Yue et al., 2024): +0.1 percentage points), document comprehension (DocVQA (Mathew et al., 2021): +0.6), and OCR reasoning (TextVQA (Mathew et al., 2021): +0.6), while enhancing video captioning (VideoChat- GPT (Maaz et al., 2024): +0.27) and reducing hallucination (POPE (Li et al., 2023b): +0.2). This preservation achieved by frezznn the vision encoder and projector while applying low-rank LoRA adapters (rank 8) to update only 0.1% of language model parameters, re- taining original visual feature extraction while enhancing language model reasoning over visual features.
Robustness to Noisy Preferences. To test VideoSAVi's robustness to noisy preference data, we deliberately flip the labels of 30% of our 24,000 preference pairs, introducing substantial incorrect training signals. Table 9 shows that despite this significant noise, VideoSAVi with noisy preferences still consistently outperforms the baseline (+1.4 percentage points MVBench, +5.1 EgoSchema, +0.4 on LongVideo pipeline maintains superior performance. This robustness stems from DPO's inherent regularization through KL-divergence constraints and the on-policy nature of our self- generated preference pairs, which stay within the model's natural output distribution and avoid instability associated with external preference data.
while the full VideoSAVi
5 Conclusion
VideoSAVi is novel self-aligned framework that employs a self-critiquing mechanism to detect and correct spatial and temporal reasoning failures in video-language models. It gen- erates high-quality preference pairs directly from video content without requiring external supervision. Extensive experiments show substantial performance improvements over base- line models on a comprehensive set of benchmarks, demonstrating robust generalization across diverse test conditions. The method is computationally efficient and model-agnostic,, enabling smaller models to outperform larger baselines through remarkable parameter effi- ciency. Future work will explore self-rewarding mechanisms and automatic hallucination detection frameworks, further enhancing video understanding reliability without reliance external annotations or proprietary models.
10

---

Published as a conference paper at COLM 2025
Reproducibility Statement
To ensure the reproducibility of our research, we rely publicly available datasets and frameworks. Our training videos are sourced from open datasets, including Star (Wu et al., 2021), Vidal (Zhu et al., 2024), and WebVid (Bain et al., 2021). We implement our approach using the SWIFT training framework (Zhao et al., 2024) and evaluate it using the LMMs- Eval (Zhang et al., 2024a), both of which open source. Further, will release our final trained model checkpoints and the complete set of model-generated preference pairs. A list of all prompts used in our implementation can be found in Appendix $A.9.
Acknowledgments
This research was supported by the National Eye Institute (NEI) of the National Institutes of Health (NTH) under award number RO1 RO1EY034562. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
References
Daechul Ahn, Yura Choi, San Kim, Youngjae Yu, Dongyeop Kang, and Jonghyun Choi. i-srt: Aligning large multimodal models for videos by iterative self-retrospective judgment. arviv preprint Xix:::2406.11 11280, 2024.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. Xiv preprint ar 2rXi::2502.13923, 2025.
Max Bain, Arsha Nagrani, Güi Varol, and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pp. 1728-1738, 2021.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, and Jared D Kaplan. Lan- guage Models are Few-Shot Learners. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), volume 33, pp. 1877-1901. Curran Associates, Inc., 2020.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling, xiv preprint arXit:2412.002211 2024.
Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. Videollama 2: Advancing spattal-temporal modeling and audio understanding video-Ilms. arXiv preprint ar XiX:v:2406 07476, 2024.
Yihe Deng, Pan Lu, Fan Yin, Ziniu Hu, Sheng Shen, James Zou, Kai-Wei Chang, and Wei Wang. Enhancing large vision language models with self-training on image comprehen- sion. arXiv preprint arXio:2405.1 19716, 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Letman, and et al. The Llama 3 Herd of Models, August 2024.
Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever com- prehensive evaluation benchmark of multi-modal llms in video analysis. arXiv preprint arXio:2405.21075, 2024.
Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, and Lotte et al. Weerts. Reinforced self-training (rest) for language modeling. arXio preprint Xii::2308.0899: 2023.
11

---

Published as a conference paper at COLM 2025
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint ar Xiv:2106 09685, 2021.
Jian Hu, Zixu Cheng, Chenyang Si, Wei Li, and Shaogang Gong. Cos: Chain-of-shot prompting for long video understanding. arviv preprint ar Xii::2502200428, 2025.
Jiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large Language Models Can Self-Improve. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1051-1068, December 2023.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. Xiv preprint arXiv:2410.21276, 2024.
Yogesh Kulkarni and Pooyan Fazli. Avatar: Reinforcement learning to see, hear, and reason over video. Xiv preprint wxXv:2508.03100 2025a.
Yogesh Kulkarni and Pooyan Fazli. Videopasta: 7k preference pairs that matter for video-llm alignment. arv preprint or Xio:22504.1499 2025b.
Xiaohan Lan, Yitian Yuan, Zequn Jie, and Lin Ma. Vidcompress: Memory-enhanced tem- poral compression for video understanding in large language models. ar preprint arXit:2410.11417, 2024.
Seon-Ho Lee, Jue Wang, Zhikang Zhang, David Fan, and Xinyu Li. Video token merging for long-form video understanding. ar ar preprint arit::2400.2378,, 2024.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint Xix::24088.03. 03326, 2024a.
Chaoyu Li, Eun Woo Im, and Pooyan Fazli. Vidhalluc: Evaluating temporal hallucinations in multimodal large language models for video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp- 13723-13733, June 2025a.
Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleawe Tackling multi-image, video, and 3d in large multimodal models. ar arvivv preprint ar Xit::240700789. 2024b.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping Language-
Image Pre-training with Frozen Image Encoders and Large Language Models. In Proceed- ings of the 40th International Conference on Machine Learning (ICML), pp- 19730-19742, July
2023a.
Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, and et al. Mvbench: A comprehensive multi-modal video understanding benchmark. In CVPR, pp. 22195-22200 IEEE, 2024c.
Rui Li, Xiaohan Wang, Yuhui Zhang, Zeyu Wang, and Serena Yeung-Levy. Temporal prefer- ence optimization for long fong-form video understanding. arXiv preprint arXio:2501.13919, 2025b.
Xinhao Li, Yi Wang, Jiashuo Yu, Xiangyu Zeng, Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan He, Chenting Wang, et al. Videochat-fllshh Hierarchical compression for long-context video modeling. arXiv preprint Xio:2501.00054 2024d.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 295-305, December 2023b.
12

---

Published as a conference paper at COLM 2025
Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. In EMNLP, PP- 5971-5984. Association for Computational Linguistics, 2024.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS, 2023.
Jiajun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu, Xiaoqi Ma, Xiaoming Wei, Jianbin Jiao, Enhua Wu, and Jie Hu. Kangaroo: A powerful video-language model supporting long-context video input. Xiv preprint arXiv:2408. 15542, 2024a.
Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. TempCompass: Do video LLMs really understand videos? In Findings of the Association for Computational Linguistics (ACL), pp. 8731-8772. Association for Computa- tional Linguistics, August 2024b.
Zhijian Liu, Ligeng Zhu, Baifeng Shi, Zhuoyang Zhang, Yuming Lou, Shang Yang, Haocheng Xi, Shiyi Cao, Yuxian Gu, Dacheng Li, et al. Nvila: Efficient frontier visual language models. arXiv preprint ar arXi::2412.0446 2024c.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video- chatgpt: Towards detailed video understanding via large vision and language models. Xiv preprint Xit:2306..05424 2023.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Video-ChatGPI: To- wards detailed video understanding via large vision and language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volune 1: Long Papers) (ACL), pp. 22585-12600 August 2024.
Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. EgoSchema: A diagnostic benchmark for very long-form video language understanding. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems (NeurIPS), volume 36, pp. 46212-46244. Curran Associates, Inc., 2023.
Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Pp. 2200-2209, 2021.
OpenAl. GPT-4 Technical Report, March 2024.
Long O Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, and Zhang- Training language models to follow instructions with human feedback. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 35:27730- 27744, December 2022.
Viorica Patraucean, Lucas Smaira, Ankush Gupta, Adria Recasens, Larisa Markeeva, Dylan Banarse, Skanda Koppula, Mateusz Malinowski, Yi Yang, Carl Doersch, et al. Perception test: A diagnostic benchmark for multimodal video models. In NeurlPS, volume 36, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, and Sandhini Agarwal. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the International Conference on Machine Learning (ICML), pp. 8748-8763, July 2021.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct Preference Optimization: Your Language Model Secretly a Reward Model. In Thirly-Sevent Seventh Conference Neural Information Processing Systems (NeurIPS), November 2023.
Yan Shu, Peitian Zhang, Zheng Liu, Minghao Qin, Junjie Zhou, Tiejun Huang, and Bo Zhao. Video-xl: Extra-long vision language model for hour-scale video understanding. arv preprint ar Xiv::2400 14485, 2024.
13

---

Published as a conference paper at COLM 2025
Avi Singh, John D Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, and et al Garcia. Beyond Human Data: Scaling Self Training for Problem- Solving with Language Models. Transactions on Machine Learning Research (TMLR), apr 2024.
Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, et al. Moviechat: From dense token to sparse memory for long video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18221-18232, 2024.
Guohao Sun, Can Qin, Huazhu Fu, Linwei Wang, and Zhiqiang Tao. Stllava-med: Self- training large language and vision assistant for medical. arXiv preprint ar arXio:2406.19973, 2024.
Guohao Sun, Can Qin, Jiamian Wang, Zeyuan Chen, Ran Xu, and Zhiqiang Tao. Sq- llava: Self-questioning for large vision-language assistant. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 156-172, 2025.
Jiawei Wang, Liping Yuan, Yuchen Zhang, and Haomiao Sun. Tarsier: Recipes for training
and evaluating large video description models. arXiv preprint arXiv:2407.00634 2024a.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024b.
Tianduo Wang, Shichen Li, and Wei Lu. Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoning. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pp. 11917-11928, August 2024c.
Xiao Wang, Qingyi Si, Jianlong Wu, Shiyu Zhu, Li Cao, and Liqiang Nie. Retake: Reducing temporal and knowledge redundancy for long video understanding. arXiv preprint arXiv:2412.20504 2024d.
Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, et al. Internvideo: General video foundation models via generative and discriminative learning. Xiv preprint ar arXio2211.003191 2022.
Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Jilan Xu, Zun Wang, et al. Internvide02: Scaling video foundation models for multimodal video understanding. In ECCV, 2024e.
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc Le. Finetuned Language Models are Zero-Shot Learners. In Proceedings of the International Conference on Learning Representations (ICLR), 2021.
Bo Wu, Shoubin Yu, Zhenfang Chen, Joshua B Tenenbaum, and Chuang Gan. Star: A benchmark for situated reasoning in real-world videos. In Proceedings of the Conference on Neural Information Processing Systems Datasets and Benchmarks Track (NeurIPS), 2021.
Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-context interleaved video-language understanding. arXiv preprint ar arXio:2407.15754 2024.
Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. Next-qa: Next phase of question- answering to explaining temporal actions. Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9777-97866 June 2021.
Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, et al. Qwen2. 5-omni technical report. arXiv preprint 200:2250,, 2025.
Mingze Xu, Mingfei Gao, Zhe Gan, Hong- You Chen, Zhengfeng Lai, Haiming Gang, Kai Kang, and Afshin Dehghan. Slowfast-llava: A strong training-free baseline for video large language models. ar Xiu preprint ar X10:2207.118811, 2024.
14

---

Published as a conference paper at COLM 2025
Zhiyang Xu, Ying Shen, and Lifu Huang. Multilnstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the Annual Meeting of the Association for Computational Lin- guistics (ACL), pp. 11445-11465, July 2023.
Zeyuan Yang, Delin Chen, Xueyang Yu, Maohao Shen, and Chuang Gan. Vca: Video curious agent for long video understanding. arXiv preprint ar XrXi::2412.10471, 2024.
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level milm on your phone. Xiv preprint ar Xiv:24008.0000 01800, 2024.
Wei Jie Yeo, Teddy Ferdinan, Przemyslaw Kazienko, Ranjan Satapathy, and Erik Cam- bria. Self-training large language models through knowledge detection. arXiv preprint ar arXie:240..1127,, 2024.
Yueqin Yin, Zhendong Wang, Yujia Xie, Weizhu Chen, and Mingyuan Zhou. Self-augmented preference optimization: Off-policy paradigms for language model alignment. arXiv preprint ar Xri::2405.20830 2024.
Zhou Yu, Dejing Xu, Jun Yu, Ting Yu, Zhou Zhao, Yueting Zhuang, and Dacheng Tao. Activitynet-qa: A dataset for understanding complex web videos via question answering. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), volume 33, pp. 9127- 9134, 2019.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9556-9567, 2024.
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. STaR: Bootstrapping Reasoning With Reasoning. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 35:15476-15488, December 2022.
Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding. In Proceedings of the Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP), PP. 543-553, 2023.
Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, et al. Lmms-eval: Reality check on the evaluation of large multimodal models. arXiv preprint arXiv:2407.11 X.X1:2207..1272,, 2024a.
Ruohong Zhang, Liangke Gui, Zhiqing Sun, Yihao Feng, Keyang Xu, Yuanhan Zhang, Di Fu, Chunyuan Li, Alexander Hauptmann, Yonatan Bisk, et al. Direct preference optimization of video large multimodal models from language model reward. arXiv preprint ar Xiv:2404.000 01258,,1.8....1 2024b.
Yuanhan Zhang, Bo Li, haotian Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. Llava-next: A strong zero-shot video understanding model, April 2024c. URL
gi git.u....... ho/blog///24/////////////// 04- 30- 1o--llava-neee dediva-next-viie -vieeo/.
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. ar Xrv preprint arXie:2410.02711, 2024d.
Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, et al. Swift: a scalable lightweight infrastructure for fine-tuning. xrv preprint ar 2tt::2400005177 2024.
Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. Mlvu: A comprehensive benchmark for multi-task long video understanding. ar Xiv preprint ar 2006::2406.04264, 2024.
15

---

Published as a conference paper at COLM 2025
Bin Zhu, Bin Lin, Munan Ning, Yang Yan, Jiaxi Cui, and et al. Languagebind: Extending video-language pretraining to n-modality by language-based semantic alignment. In Proceedings of the The International Conference on Learning Representations (ICLR), 2024.
Orr Zohar, Xiaohan Wang, Yonatan Bitton, Idan Szpektor, and Serena Yeung-Levy. Video- star: Self-training enables video instruction tuning with any supervision. arXiv preprint (iii::207700688, 2024.
16

---

Published as a conference paper at COLM 2025
A
Appendix
A.1
Iteration-wise Analysis
Figure demonstrates the iteration-wise performance improvements of VideoSAVi across six video understanding benchmarks. Each iteration represents complete cycle of our self-alignment pipeline, where the model (1) generates questions targeting specific reason- ing capabilities and produces initial responses, (2) self-critiques these responses to identify reasoning errors, (3) creates revised responses based on critique feedback, and (4) uses the revised and initial responses to form preference pairs for DPO training. The consis- tent upward trajectory across all benchmarks validates our hypothesis that self-critiquing enables effective self-training without external supervision. Most notably, we observe sub- stantial gains on tasks requiring complex reasoning: NeXTQA (+3.6 percentage points), MVBench (+5.1), and EgoSchema (+6.8). These improvements are particularly significant since EgoSchema features ego-centric videos with complex temporal relationships, while MVBench demands both fine-grained spatial understanding and temporal reasoning. The performance gains exhibit different patterns across benchmarks. TempCompass shows more modest improvements (+0.8), likely because our baseline model already performs competitively on this benchmark. In contrast, Perception Test demonstrates steady improve- ment (+3.9), indicating that VideoSAVi's self-critiquing effectively addresses perceptual reasoning errors. For long-form videos (Long VideoBench), observe +2.0 improvement, confirming that our approach can enhance extended temporal reasoning capabilities. The iteration-wiss analysis reveals that the most significant improvements occur in earlier iter- ations (especially iterations 1-2). This pattern suggests that VideoSAVi quickly identifies and corrects the most critical reasoning errors, followed by more minor refinements in subsequent iterations. Importantly, unlike methods that rely on external supervision or pro- prietary models, VideoSAVi achieves these improvements entirely through self-refinement, demonstrating the usability of self-aligned video understanding.
A.2 Preference Composition Analysis
The decomposition of preference types in Table 10 reveals distinct task-specific effectiveness patterns. Spatial-only preferences lead to significant improvements on perception- -nnteniiee
10.3
21
69.2
M03100 Scere 00
Pwotten
Broutho
(a) TempCompass
(b) PerceptionTest
(c) NeXTQA
Soon
59.5
22-
57.5
Brndiin
Iaridon
Baraden
(d) MVBench
(e) EgoSchema
(l Long VidhoBench
Figure 3: Ierationn-ise performance improvement of VideoSAVi across six video un- derstanding benchmarks. Each iteration refines the model through self-critiquing and preference optimization, demonstrating consistent performance gains without external supervision.
17

---

Published as a conference paper at COLM 2025
Benchmark Performance
Preference Type
TempCompass
Perception Test
NeXTQA
MVBench
EgoSchema
Long VideoBench
Baseline
In2ernVL2.55
68.3
62.2
77.0
68.9
52.0
57.8
Proference Composition
Spatial Only
68 88.8.05
65.5,33
78.9-19
72.2.s.
55.3,33
58.4.06
Temporal Only
69.0,a7
63.1,09
79.5.25
70.5.16
56.1.41
59.2,14
Full (Sepoial-Temprra)
69.1,a8
66.1,39
80.6.36
74.0.51
58.8.6.8
59.8.2.0
Table 10: Preference Composition Analysis of VideoSAVi. We evaluate the contribution of spatial and temporal preference types across six benchmarks. Improvements over the baseline are shown as subscripts. While each type contributes to performance gains, their complementary combination in the full VideoSAVi yields consistently superior results, demonstrating the importance of addressing both spatial and temporal reasoning capabili- ties for comprehensive video understanding. Notably, spatial preferences provide larger gains on perception-focused tasks, while temporal preferences excel on longer-form video understanding, but the integration of both creates the strongest overall results.
benchmarks (PerceptionTest: +3.3 percentage points, MVBench: +3.3), demonstrating that targeting visual reasoning errors substantially enhances scene understanding capabilities. These gains align with the inherent spatial reasoning demands of these benchmarks, which require precise object localization and attribute recognition.
Temporal-only preferences demonstrate complementary strengths, excelling on benchmarks with extended temporal dependencies (EgoSchema: +4.1, Long, VidgoBdeeenchh: +1.4, NeXTQA: +2.5). The effectiveness EgoSchema is particularly noteworthy, as egocentric videos con- tain complex action sequences requiring precise temporal ordering and causality inference. Temporal preference learning effectively addresses common temporal reasoning failures, including sequence ordering errors and causal misattributions.
The integration of both preference types yields strong results several benchmarks. For MVBench, the full model achieves a +5.1 improvement, exceeding the sum of individual components (+3.3 spatial, +1.6 temporal). This non-linear gain suggests that joint optimiza- tion addresses compound reasoning errors that span both dimensions, particularly for tasks requiring spatiotemporal reasoning (e.g., tracking object state changes over time).
Analysis of performance differentials reveals benchmark-specific optimization patterns. On perception-focused tasks (PerceptionTest. MVBench), spatial preferences contribute disproportionately to the full model's gains, while on temporally complex benchmarks (EgoSchema, Bengeid) temporal preferences provide complementary strengths that enhance the combined model. This task-specific contribution pattern validates our approach of targeting both reasoning dimensions simultaneously rather than optimizing for single aspect.
A.3 Training Dataset Composition Analysis
To investigate the impact of dataset diversity self-alignment performance, we conducted an ablation study using individual datasets while maintaining consistent pipeline parame- ters across four iterations. As shown in Table 11, VideoSAVi demonstrates dataset-agnostic improvements, achieving gains over the baseline InternVL2.5 model regardless of the source dataset. Star (Wu et al., 2021) videos particularly enhance complex reasoning bench- marks (NeXTQA +2.1 percentage points, EgoSchema +3.5), likely due to their rich situated contexts. Vidal (Zhu et al., 2024) contributes balanced improvements across benchmarks, tasks
while WebVid (Bain et al., 2021) exhibits stronger contributions to perception-oriente (PerceptionTest +2.8). Notably, the performance gap between individual datasets and the combined approach (gap of 1.5-3.3 percentage points across benchmarks) demonstrates that VideoSAVi does not merely overfit to specific dataset preferences but rather benefits from diverse visual contexts. This finding reveals critical insight: effective preference learning for video understanding requires exposure to varied temporal dynamics, visual compositions, and reasoning scenarios that single-source datasets cannot provide in isola-
18

---

Published as a conference paper at COLM 2025
Benchmark Performance
Dataset Source
TempCompass
PerceptionTest
NeXTQA
MVBench
EgoSchema
Long Bidco
Raseline
Intern V1.2.5
68.3
62.2
77.0
68.9
52.0
57.8
Single Datasef (1,000 videos)
Star Only
68.7.04
64.1,19
79.1.21
71.3.2.4
55.5.35
58.9.1.1
Vidal Only
68.6.03
64.5,23
78.5.15
70.2-1.3
54.6.2.
58.5.0.7
WebVid Only
68.5.02
65.0.28
78.4.14
70.7
53.8.
58.7.0.9
All Datasets
69.1,08
66.1,39
80.6,3.6
74.0.5.1
58.8,68
59.8,2.0
Table 11: Dataset Ablation Analysis of VideoSAVi. We evaluate the performance impact of using only a single source dataset (1,000 videos each) vs. the full combination across six benchmarks. Improvements over the baseline shown subscripts. While each individual dataset contributes to performance gains, their combination yields consistently superior results, demonstrating the importance of diverse video sources for comprehensive understanding. Star (Wu et al., 2021) videos particularly enhance reasoning benchmarks, Vidal (Zhu et al., 2024) shows balanced improvements, and WebVid (Bain al., 2021) con- tributes strongly to perception tasks, but the integration of all sources creates the strongest overall results.
tion. The complementary nature of different video sources enables the model to develop more robust reasoning capabilities applicable across diverse benchmarks, confirming that dataset diversity serves as an implicit regularization factor in self-supervised preference optimization.
A.4 DPO Training Dynamics
The DPO training dynamics in Figure 4 reveal critical insights into VideoSAVi's self- alignment process. The reward distribution exhibits clear bifurcation pattern, with preferred responses achieving consistently higher rewards (μ 1.5) than dispreferred alternatives (μ 0.5). This substantial separation (average gap 1.0) confirms the model's ability to differentiate quality even without external supervision. Gaussian smoothing (o =5) was applied to mitigate the inherent stochasticity in neural network training while preserving underlying trends. The rapid increase in classification accuracy (reaching 0.8 by iteration 1000) demonstrates that the model quickly learns to distinguish between response qualities, with accuracy stabilizing despite continued training, indicating robust generalization rather than overfitting. Most notably, the reward gap's consistent growth throughout training suggests that the model continuously refines its understanding of quality differences rather than merely amplifying initial biases. The training loss profile exhibits three distinct phases: rapid early descent (iterations 0-500), steady intermediate refinement (iterations 500-1500), and convergence with minor oscillations (iterations 1500+). This pattern aligns with theoretical expectations for preference optimization and demon- strates that our self-alignment approach achieves parameter convergence without external reward signals. This validates the viability of purely self-supervised preference learning for improving video understanding.
A.5 Generalization Test Bed Details
To rigorously evaluate the generalization capabilities of video understanding models, we construct comprehensive test bed spanning multiple dimensions of generalization diffi- culty. This section details our methodology, data sources, and evaluation protocols.
A.5.1 Dataset Composition
Our test bed comprises videos from diverse sources:
19

---

Published as a conference paper at COLM 2025
2.5
1.0
Accurazy
2.0
1.5
0.8
Accuracy
Reward
1.0
0.7
0.6
0.0
Preferred
Disprelerned
2000
3000
500
2000
Training Iterations
Training Iterations
(a) Reward Values During Training
(b) Preference Classification Accuracy
2.0
Reward Gap
0.22
DPO Loss
Separation
Reward Diference
1.5
Loss Value
0.10
0.46
0.5
0.0
0.12
500
1000
1500
2000
2500
2000
500
1000
1500
2000
2500
3000
Training
Training
(c) Reward Gap (Preferred Dispreferred)
(d) DPO Training Loss
Figure 4: Direct Preference Optimization training dynamics for VideoSAVi. The visual- izations show: (a) clear separation between preferred and dispreferred response rewards; (b) increasing preference classification accuracy that stabilizes at approximately 80%; (c) growing reward gap demonstrating effective preference learning, and (d) consistent loss reduction indicating stable convergence. These metrics confirm that our self-alignment approach successfully produces high-quality preference pairs that enable effective model optimization without external supervision.
1. Training Distribution: 100 videos each from Star (Wu et al., 2021), Web WibVid (Bain et al., 2021), and Vidal (Zhu et al., 2024) datasets (300 total), representing the model's original training domain.
2. Cross-Videe 150 additional unseen videos from the same datasets.
3. Full OOD: 150 videos from ActivityNet (Yu et al., 2019), representing a completely different domain with more complex activities.
For each video, we generate rich set of questions using GPT-4o (Hurst et 2024) according to the prompt shown in Figure 13.
A.5.2 Test Condition Categories
We design six distinct test conditions with progressively increasing difficulty:
1. Training Distribution. Standard format questions applied to training distribution videos. These serve as our baseline for measuring generalization gaps.
2. Cross-Question. Questions with complex syntax and uncommon vocabulary ap- plied to fraining distribution videos. This tests linguistic generalization while keeping visual content consistent.
20

---

Published as a conference paper at COLM 2025
3. Cross-Video. Standard format questions applied to unseen videos from the same datasets. This tests visual generalization while keeping question patterns consistent.
4. Full OOD. Standard format questions applied to ActivityNet videos, testing both visual and domain generalization.
5. Adversarial. Deliberately challenging questions applied to training distribution videos, requiring focus on subtle non-obvious elements.
6. Compositional. Multi-step reasoning questions applied to training distribution videos, requiring integration of multiple reasoning types.
A.5.3 Evaluation Protocol
For each test condition, we evaluate models using multiple-choice accuracy, with each question accompanied by four possible answers (generated by GPT-4o). We maintain consistent evaluation protocols across all models to ensure fair comparison. To quantify each test condition, (2) the drop
generalization capability, we calculate (1) performance in performance from the training distribution to each other condition, and (3) the average generalization gap across all challenging conditions.
A.5.4 Quality Check
To ensure the quality of our test bed, we conduct verification step where manually review a randomly sampled subset of 200 question-answer pairs across all conditions. The verification confirms that:
97. 97.5% of questions are answerable from video content.
94.0% are categorized correctly according to their intended test condition.
98.0% have single unambiguously correct answer.
This verification step validates the quality and reliability of our generalization test bed. The distribution of questions across different reasoning types (spatial, temporal, object-centric, action-centric) was balanced to avoid bias toward any particular reasoning capability.
A.6 Qualitative Analysis
Figure 5 presents detailed qualitative comparison between our VideoSAVi and the baseline In2ernVL2.5 (Chen et al., 2024) model across three distinct video understanding scenarios. Each example demonstrates how our proposed self-critique and preference optimization pipeline effectively addresses different types reasoning errors, highlighting the improve- ments made in both temporal and spatial reasoning through our method.
In the first example (cooking scenario), the baseline model shows a common temporal hallucination by generating non-existent microwave heating step. This represents critical factual error in temporal reasoning where the model invents an action sequence not supported by visual evidence. VideoSAVi correctly identifies and generates an accurate representation of the action sequence, properly recognizing the transfer of the mixture between containers before dishwashing begins.
The second example (brick-making) highlights spatial reasoning deficiencies in the baseline model. The baseline model incorrectly positions the finished bricks in "vertical columns behind" the person and misidentifies the clay pit as being "to the right" of the work area. VideoSAVi produces refined response that accurately captures the grid pattern arrangement to the left side of the person and the correct positioning of the clay pit directly in front of the person.
The third example (car cleaning) demonstrates object hallucination, where the baseline model incorrectly claims the person is using "soap and brush" rather than identifying the pressure washer visible in the video frames. VideoSAVi not only correctly identifies the pressure washer but also accurately describes additional contextual elements like the outdoor setting and the organization of cleaning supplies.
21

---

Published as a conference paper at COLM 2025
The fourth example (dog competition) showcases VideoSAVi's ability to connect multiple events within a video. The baseline model hallucinates specific details about treats, judges, obstacle arrangement, and reward ribbons, none of which appear in the video. VideoSAVi produces a response that precisely captures the actual course elements (yellow-framed ob- stacles, jump bars, blue tunnel) and correctly identifies the spatial arrangement of spectators and tents.
These examples illustrate how our self-critique mechanism systematically identifies reason- ing errors across both spatial and temporal dimensions, creating high-quality preference pairs that enable the model to learn from its own mistakes without external supervision. The consistent pattern of error correction across diverse scenarios demonstrates the general- izability of our approach, showing strong improvements in eliminating hallucinated objects, correcting spatial relationships, and accurately representing temporal sequences.
A.7 Dataset Samples
Figure 6 showcases representative examples from our preference pair collection, illustrating how VideoSAVi effectively refines its reasoning capabilities. The examples reveal critical distinctions in temporal reasoning, where the model transitions from generating non- existent events (athlete celebrating) to precise temporal sequencing (immediate action following bar clearance) and from reversing causal sequences (blending after squeezing) to establishing correct procedural order. Similarly, for spatial reasoning, the preference signal guides the model from imprecise localization (beneath/ partially covered) toward accurate spatial relationships (centrally positioned/ surrounded) and from incorrect reference points (behind the leftmost cup) to precise positional understanding (underneath the middle cup). These preference pairs exemplify how our self-critiuuing mechanism isolates specific reasoning failures without requiring human annotation or proprietary model distillation. The resulting preference signal contains subtle yet unambiguous distinctions that, when optimized through DPO, enable VideoSAVi to develop robust spatial-temporas reasoning capabilities grounded directly in visual evidence. This demonstrates the effectiveness of our approach in generating high-quality training signals through purely self-supervised means.
Dheneat/ppartialyy
A.8 Example of Self-Critiqinng Mechanism for Improving Temporal and Spatial Reasoning
Figure 7 demonstrates VideoSAVI's self-critiquing mechanism across diverse video under- standing scenarios. Our approach enables the model to identify and correct reasoning failures without external supervision. The examples shown are from the fourth iteration of self-training, where reasoning capabilities have been significantly enhanced through repeated preference optimization.
As illustrated in Figure 7, our self-critique pipeline systematically addresses both temporal and spatial reasoning errors. For temporal reasoning (first example), the critique identifies when responses are too general and lack specific temporal details about actions like the precise movements of table tennis player during a rally. For spatial understanding (second example), the pipeline enhances detail about relative positions of subjects within scenes, such as specifying the exact seating position of person in a green shirt. The hamburger drawing example (third) shows how the critique mechanism detects when response fails to mention critical preparatory actions, while the food preparation example (fourth) highlights the correction of significant spatial-tempon
inaccuracies in hand coordination.
By framing these critiques as preference pairs (in the next step), we align the model specifi- cally to these dimensions while avoiding the introduction of biases that may arise when using external or proprietary models for critiquing. This approach mitigates failure modes that commonly affect video-LLMs: temporal hallucinations (generating non-existent se- quences), spatial mistepresentations (incorrectly positioning objects), and detail omissions (missing critical visual evidence).
The advantage of our approach lies in maintaining end-to-end differentiability throughout the preference learning process. When the model identifies that response is too general or
22

---

Published as a conference paper at COLM 2025
contains significant inaccuracies, it generates precise corrective signals focused on the specific reasoning dimension requiring improvement. Through iterative DPO refinement, these self-generated preference pairs enable VideoSAVi to progressively strengthen its reasoning across spatiotemporal dimensions without requiring external labels or feedback.
Unlike approaches that rely on external critique models, our self-alignment mechanism preserves internal video representations throughout the refinement process, avoiding the information loss that typically occurs when passing through different model architectures. This technical design choice enables more efficient learning from fewer examples, as demon- strated in the food preparation example where hand positioning and action synchronization are precisely captured.
A.9 Prompt Design for Self-Aligning Video-LLMs
The effectiveness of VideoSAVi relies critically on carefully designed prompts that guide each stage of the self-alignment pipeline. Our prompt templates are written to elicit specific types of reasoning, generate high-quality self-critiques, and produce refined responses that address identified shortcomings without external supervision.
Reasoning- Question Generation. Figure 8 presents our template for generating diverse and challenging questions that target specific reasoning capabilities. This prompt is structured to balance spatial and temporal understanding requirements with explicit instructions to focus on relationships that require careful attention. The spatial reasoning component emphasizes object positioning, visual details, and scene composition, while the temporal reasoning component focuses on event sequencing, cause-effect relationships, and state transitions. We deliberately avoid questions with trivial answers by instructing the model to prioritize aspects requiring deeper visual analysis. This prompt enables us to create an initial dataset of challenging questions and preference pairs, which are then used to train the model through DPO.
Self-Critique Mechanism. The core innovation of VideoSAVi lies in its ability to critique its own responses. Figure 9 shows our template for generating comprehensive assessments of reasoning errors. The prompt is carefully designed to evaluate four distinct aspects: spatial reasoning accuracy, temporal reasoning correctness, cross-modal consistency between claims and visual evidence, and overall response quality. For each identified issue, the model must specify the problematic statement, explain why it is incorrect, provide contradicting visual evidence, and assess the severity of the error. This structured approach ensures that critiques are specific and actionable rather than vague or general, enabling precise, targeted improvements in subsequent iterations.
Response Refinement. Figure 10 illustrates our template for generating improved responses through self-critique. This prompt guides the model to carefully balance preserving accurate information while correcting identified errors. We emphasize factual correctness, grounding in visual evidence, and logical consistency to prevent the introduction of new hallucina- tions during refinement. The prompt specifically targets factual errors, temporal ordering mistakes, spatial relationship misrepresentations, and reasoning flaws identified in the cri- tique stage. This targeted refinement approach enables VideoSAVi to generate high-quality preference pairs that effectively isolate and improve specific reasoning capabilities.
External Critique Generation. For a fair comparison with external critique methods, we design the prompt shown in Figure 11. This template guides GPT-4o in analyzing response pairs to identify specific errors in dispreferred responses compared to preferred alternatives. The structured critique format highlights issues in spatial and temporal reasoning, factual inaccuracies, and missing critical details. This prompt serves as benchmark for evaluating the effectiveness of VideoSAVi's self-critique approach against externally generated critiques.
Error Analysis. Figure 12 presents our error categorization template, used with GPT-40 to evaluate model improvements across iterations. This prompt defines various error types, in- cluding factual inaccuracies, temporal ordering mistakes, spatial misrepresentations, object hallucinations, causal reasoning failures, and detail omissions. By enforcing a structured output format, the template enables quantitative tracking of error reduction across iterative
23

---

Published as a conference paper at COLM 2025
refinements, providing objective metrics for VideoSAVI's self-improvement process. This experiment is essential in verifying that our approach genuinely corrects reasoning errors rather than reinforcing existing patterns.
Generalization Assessment. To rigorously evaluate generalization capabilities, we design the prompt in Figure 13. This template generates progressively more challenging ques- tion variants, including standard training distribution questions, complex cross-question variants, deliberately challenging adversarial questions, and multi-step compositional ques- tions. By structuring the difficulty progression, we can measure VideoSAVi's generalization gap across different reasoning dimensions and compare it with alternative approaches. The significant reduction in the generalization gap (-5.2% for VideoSAVi vs. -16.3% for Hound-DPO) validates the effectiveness of our method in achieving robust generalization.
24

---

Published as a conference paper at COLM 2025
What sequence of actions does the person perform with the yellow mixture before washing dishes?
Query:
InternVL2.5:
The person stirs the yellow mixture, places it in the microwave to heat it, then returns to take it before washing dishes the sink. The heating step occurs between handling the mixture dishwashing.
VideoSAVi:
The person moves to wash dishes in the sink. The yellow mixture remains in the bowl until dishwashing begins.
yellow mixture a bowl, transfers some of another container, and then
Query:
How the finished bricks arranged in relation the person's work and the clay pit?
The finished bricks are stacked in vertical columns behind the person. The clay pit is to the right the work area, and the person moves between the pit and the brick arrangement area.
InternVL2.5:
VideoSAVI:
The finished bricks are arranged grid pattern to the left side of the person. They are positioned away from the clay pit, which directly in front of the person.
Query:
What cleaning task is the person performing this video?
InternVL2.5; The person cleaning the car's floor mats using soap and brush. The person scrubs each mat by hand after removing them from the vehicle.
The person is cleaning the car's floor mats with a pressure washer. They remove the black rubber mats from the vehicle, spray them with water in outdoor area, and have their cleaning supplies organized in shopping cart nearby.
VideoSAVi:
Query: Describe the video.
InternVL2.5:
The video shows dog agility competition where the person is using treats to guide the dog through the course. The obstacles arranged randomly. There are five judges observing from different angles, and the event concludes with dogs receiving colored ribbons based on their performance.
The video shows dog competition taking place in an outdoor park setting. There yellow- framed obstacles forming a course, including jump bars blue tunnel. Muttiple handlers guide dogs through the course while spectators watch from the sidelines where blue tents are set up. The black and white dog is seen navigating between obstacles with its handler.
VideoSAVi:
Figure 5: Qualitative examples demonstrating VideoSAVi's self-critique pipeline for improving video understanding. Baseline model responses reveals reasoning errors: hal- lucinated details (judges, treats), incorrect spatial relationships (stacked columns, right side), and fabricated details (soap, scrubbing). Through our self-critiquing mechanism, VideoSAVi corrects these errors by: (1) generating accurate temporal sequences (yellow mixture handling), (2) precisely identifying spatial configurations (brick arrangement), and (3) eliminating object hallucinations (pressure washer brush). Red text indicates errors in baseline responses and green text highlights accurate details in VideoSAVi responses.
25

---

Published as a conference paper at COLM 2025
Query: Did the athlete lie down on the mat immediately after clearing the bar?
Dispreferred Response: The athlete first stood celebrated before lying down on the mat.
Preferred Response: The athlete immediately lay down on the mat after clearing the bar.
MEAT
Query: Was the mixture blended before or after squeezing lime into the container?
Dispreferred Response: The mixture was blended after squeezing lime into the container.
Preferred Response: The mixture was blended before squeezing lime into the container.
Query: How is the yellow material positioned relative to the pink petals in the final flower?
Dispreferred Response: The yellow material is positioned beneath, partially covered the pink petals.
Preferred Response: The yellow material positioned centrally, surrounded by the pink petals.
Query: Where does the person place the blue object relative to the white cups?
Dispreferred Response: The blue object is placed behind the leftmost white cup.
Preferred Response: The blue object is placed underneath the middle white cup.
Figure 6: Examples of preference pairs from our dataset showing how VideoSAVi corrects reasoning errors through self-critiquing. The top two examples demonstrate temporal reasoning improvements: accurately identifying action sequences (athlete immediately lying down) and event ordering (mixture blending before squeezing). The bottom two showcase spatial reasoning refinements: correctly representing positional relationships (yellow material centrally positioned versus underneath) and precise object localization (cup placement). These subtle yet critical distinctions form the training signal that enables VideoSAVi to learn fine-grained video understanding without external supervision. Red text indicates errors in dispreferred responses and green text highlights accurate details in preferred responses.
26

---

Published as a conference paper at COLM 2025
Query: What happens after the player in red serves?
Initial Response: After the red player serves, the opponent returns the ball and brief rally occurs. The red player then makes winning shot.
Self-Cre Critique: The response is too general and lacks specific temporal details. doesn't clearly indicate the exact movements of the red player during the rally.
Refined Response: After the player serves, he steps back, shifts right, receives his opponent's return, and returns forehand that his opponent fails return properly, ending the rally.
Query: How does the man in the green shirt's position change the video?
Initial Response: The man in the green shirt is sitting on the leather couch and later he standing in the middle of the room.
Setf-Critique: The response needs more spatial details about his relative position to the other people and objects. The temporal sequence could be more clearly described.
Refined Response: Initially, the man in the green shirt seated left sidee the brown couch. Later, has moved to standing position in the center the room with his right hand extended outward while facing the man blue.
Query: What actions does the person take before placing the real hamburger on the drawing paper?
The person writes "Hamburger Master" at the top of the page before placing real hamburger on the drawing paper. They are holding a pen and creating the title text as the initial step in their drawing process.
Initial Response:
The response is too general and lacks specific temporal details. doesn't clearly identify all the observable actions that occur before the hamburger placement. The response fails to mention the exact positioning of the paper and how the person prepares the workspace.
Self-Critique:
Before placing the real hamburger the paper, the person first positions the blank white paper, then writes "Hamburger Master" at the top of the page, completing the title. They hold the pen in their right hand while steadying the paper with their left hand.
Refined Response:
Query: How person coordinate their right
preparing the
Initial Response: During the cutting process, the person their right hand to hold the knife while their left hand rests the counter. They cut with the right hand dowmward motion while occasionally adjusting the food with both hands.
Self-Critique: The response contains significant spstial-temporal inaccuracies, It incorrectly claims the left hand rests the counter during cutting, contradicting visual evidence. It fails describe the specific positioning and movement of both hands in relation to each other and to the food.
The person begins by positioning the food on the cutting board using both hands. During the actual cutting, the left hand stabilizes the food item by pressing down from above, while simultaneously the right hand grips the knife for cutting. This coordinated hand positioning creates a spatial relationship where the left hand serves as support white the right hand performs the cutting action.
Refined Response:
Figure 7: Examples of VideoSAVi's self-critique pipeline for various video understanding tasks. Red text indicates errors in initial responses, cyan text shows critique identified errors and green text highlights revised details VideoSAVi responses.
27

---

Published as a conference paper at COLM 2025
Reasoning-Focused Question Generation Prompt
You tasked with generating challenging and diverse questions about video content that test deep understanding of both spatial and temporal relationships. These questions should target specific reasoning capabilities that will help improve video understanding models.
Spatial Reasoning Questions: Generate questions that require:
Identifying precise spatial relationships between objects abjects/people in the scene.
Detecting subtle visual details that might be easily overlooked.
Comparing and contrasting different regions of the frame.
Recognizing object attributes (color, size, shape, texture) and their spatial arrangement.
Understanding occlusion, perspective, and relative positioning,
Reasoning about visual composition and scene layout.
Example spatial questions:
"What is the spatial relationship between the adult and child in the scene?"
"How are the objects on the table arranged relative to each other?"
"What visual details in the top-right corner distinguish it from the bottom- left?"
"What is the configuration of people and furniture the room?"
Temporal Reasoning Questions: Generate questions that require:
Tracking the sequence and order of events.
Understanding cause-effect relationships between actions.
Identifying what happens before or after specific key moments.
Reasoning about duration, speed, and timing of actions.
Detecting transitions between states or scenes.
Analyzing how entities change or move over time.
Example temporal questions:
"What happens immediately before the person picks up the cup?"
"What sequence of events leads to the child falling?"
"How does the arrangement objects change from the start to the end?"
"What causes the dog to start running?"
Guidelines for Question Generation:
Create questions that are answerable solely from the video content.
Focus on aspects that require careful attention and reasoning.
Avoid questions with trivial or immediately obvious answers.
Ensure questions target both easy-to-observe and subtle elements.
Generate questions that challenge understanding without requiring special- ized knowledge.
Balance questions across different areas of the frame and timepoints in the video.
For each video, generate 3 spatial reasoning questions and 3 temporal reasoning questions that probe different aspects of understanding.
Figure 8: Prompt template used to generate diverse spatial and temporal reasoning questions that target specific reasoning capabilities for self-alignment.
28

---

Published as a conference paper at COLM 2025
Self-Critique Generation Prompt
You are acting as a critical evaluator for response video-based query. Your task is to meticulously analyze the response for errors, inconsistencies, and potential improvements.
Given the video content and the initial response, thoroughly examine the following aspects:
Spatial Reasoning Assessment: Analyze whether the response accurately represents spatial relationships visible in the video. Check object positions are correctly de- scribed relative to another. Verify that all significant visible objects are accounted for without hallucination. Examine if object attributes (color, size, shape) are accu- rately represented. Assess if spatial descriptions are precise and unambiguous. Temporal Reasoning Assessment: Evaluate whether the sequence of events is cor- rectly ordered. Check cause-effect relationships between actions are logically sound. Verify that temporal markers (before, during, after) accurately reflect video content. Assess if the duration of actions events realistically represented. Exam- ine if transitions between states scenes are accurately described.
Cross-Modal Consistency: Analyze if claims made in the response are directly observable in the video. Identify any statements that contradict visual evidence. Check for speculative content not grounded in the video. Verify that the response addresses the specific query without tangential information.
Response Quality: Evaluate the answer is appropriately comprehensive for the query. Check for logical contradictions within the response itself. Assess if the response maintains an appropriate level of detail. For each identified issue, clearly specify:
1. The exact problematic statement omission.
2. Why it is incorrect or problematic.
3. The relevant visual evidence from the video that contradicts or is missing,
4. The severity of the error (critical or minor).
Conclude your critique with a concise summary of the most significant issues that should be addressed in a refined response. Focus particularly on factual errors rather than stylistic concerns.
Figure 9: Prompt template for the critic model to generate comprehensive assessment of spatial, temporal, and logical errors in initial responses.
29

---

Published as a conference paper at COLM 2025
Response Refinement Prompt
You are tasked with refining a response to video-based question based on detailed critique. Your goal is to produce an improved answer that addresses all identified issues while maintaining accuracy.
Review the following materials carefully:
The original question about the video.
Your initial response to this question.
A detailed critique identifying specific errors and issues.
Refinement Guidelines:
Maintain factual correctness by ensuring all claims are directly observable in the video.
Preserve useful and accurate information from the original response.
Correct all errors and inconsistencies identified in the critique.
Avoid introducing new speculation not supported by visual evidence.
Ensure logical consistency throughout your refined response.
Address the specific question directly and comprehensively.
Important: Focus on addressing the specific issues raised in the critique, particularly factual errors, temporal ordering mistakes, spatial relationship misrepresentations, or reasoning flaws.
Consider the provided critique of your previous response and provide an improved response that addresses these issues.
Figure 10: Prompt template for response refinement, which instructs the model produce an improved answer that incorporates feedback from the self-critique while maintaining factual accuracy and logical consistency.
30

---

Published as a conference paper at COLM 2025
External Critique Generation Prompt
You are analyzing pairs of responses to video-based questions, where one response (Preferred) has been judged as superior to the other (Dispreferred), Your task is to generate a detailed critique of the Dispreferred response, identifying specific deficiencies compared the Preferred response. Given Information:
Video.
Question about the video.
Preferred response (higher quality).
Dispreferred response (lower quality).
Critique Guidelines: Generate detailed critique of the Dispreferred response that: Identifies specific factual errors or inaccuracies.
Highlights failures in spatial reasoning (object positions, relationships, at- tributes).
Points out temporal inconsistencies (event ordering, causality, transitions).
Notes missing details that appear in the Preferred response.
Identifies logical flaws or contradictions.
Analyzes how and why the Dispreferred response fails to address the ques- tion.
Structure Your Critique:
1. Summary of Core Deficiencies: Briefly summarize the main problems (1-2 sentences)
2. Spatial Reasoning Issues: Identify specific spatial errors misunderstand- ings
3. Temporal Reasoning Issues: Highlight coquence/causalitg problems
4. Factual Errors: List specific incorrect claims or hallucinations
5. Comparative Analysis: Note key elements present in the Preferred response but missing or incorrect in the Dispreferred
6. Improvement Recommendations: Suggest specific changes to correct the identified issues
Be specific, precise, and reference exact details from both responses and the video. Focus on substantive issues rather than stylistic differences. Where possible, explain why certain errors would be particularly problematic for user understanding.
Figure 11: Prompt used with GPT-4o to generate external critiques of dispreferred responses from existing preference datasets.
31

---

Published as a conference paper at COLM 2025
GPT-4o Evaluation Prompt for Error Analysis
You are an expert evaluator assessing responses to video-based questions. Your task is to identify and categorize errors in the provided response. You will be given:
1. A video.
2. A question about the video.
3. A model-generated response to evaluate.
Evaluation Categories:
Analyze the response for the following enror types:
Statements that directly contradict what visible the video. Claims about objects, actions, events that do not appear in the video. Temporal Ordering: Incorrect sequencing of events (e.g., claiming happened before B when the reverse is true). Misrepresentation of causal relationships between actions.
Factual Inaccuracy:
Spatial Relationship: Incorrect descriptions of object positions arrangements. Misrepresentation of relative locations (left/ /right,, above bblow, etc.).
Object Hallucination: Mentioning objects entities that not present in the video. Attributing incorrect properties to objects that do exist.
Causal Reasoning: Incorrect inferences about why events occurred. Unsupported claims about motivations or intentions.
Detail Omission: Failing to mention critical elements necessary answer the question. Overlooking important visual details that change the interpretation. Instructions:
1. For each error you identify, specify the error type from the categories above.
2. Quote the problematic text from the response.
3. Explain why it's an error based on the video description.
4. Count the total number of errors each category.
Output Format:
1. Error counts by category.
2. Brief examples of each error type found.
3. An overall assessment of response quality.
Be thorough in your analysis, as this evaluation will be used to measure model improvement across iterations.
Figure 12: Prompt used for GPT-4o to evaluate and categorize errors in model responses across iterations.
32

---

Published as a conference paper at COLM 2025
Test Condition Question Generation Prompt for Generalization Assessment
You will generate diverse question types to test video understanding model's generalization capabilities across increasingly challenging conditions. For each video, create questions in the following categories: Training Distribution Questions: Create standard format questions that follow typical patterns:
"What object appears after the person sits down?"
"What is the spatial relationship between [object A] and [object BJ?"
"How many people are visible in the scene?"
Cross QuestiQu Variants: Reformulate standard questions using novel phrasing and more complex language:
"Describe the causal relationship between the sitting action and subsequent object appearance"
"Elaborate on the configuration of entities in relation to the central figure"
"What sequential patterns of movement can be discerned among the visible actors?"
Cross-Vides Questions: Apply standard question formats to unseen videos from the same datasets:
Use the same question templates as Training Distribution.
Test visual generalization with consistent question complexity.
Focus on transferring learned patterns to new visual content.
Adversarial Questions:
Create questions with deliberately misleading cues or that require focusing on non-obvious elements:
"What is the main action occurring in the background?" (when foreground action prominent)
"What subtle change occurs to the leftmost object while attention would naturally focus on the center?"
"Ignoring the primary movement, what secondary action follows the initial event?"
Compositional Questions: Formulate questions requiring multiple reasoning steps or integration of different reasoning types:
"Compare the temporal order of actions on the left versus right sides of the frame"
"How does the spatial configuration change from the beginning to the end of the sequence?"
"What causal chain connects the initial object arrangement to the final state?"
For each video, generate 8 questions (2 per category) with 4 answer choices and correct choice. Ensure questions are answerable from the video content, specific, and aligned with their respective category definitions. For adversarial questions, identify what makes them challenging (e.g., distraction, subtlety, misdirection).
Figure 13: Prompt used to generate diverse test conditions for generalization assessment. GPT-4o created questions across standard, cross-question, adversarial, and compositional categories to systematically evaluate VideoSAVi's ability to generalize beyond its training distribution.
33


Reviews:
[{"review_id": "6aaK6ysJnM", "reviewer": "Reviewer_vG49", "summary": "### Quality\n\n* The manuscript is well‑written.\n* Tables and figures clearly illustrate the benefits of the proposed method.\n\n### Clarity\n\n* The presentation flows logically and is easy to follow.\n* All necessary technical details are provided.\n\n### Originality\n\n* The approach is moderately original.\n* Self‑training / thought‑refinement to replace human annotation or external models has been studied extensively, but its application to **video understanding** remains relatively under‑explored.\n\n### Significance\n\n* The method shows consistent, model‑agnostic gains over baselines on several video benchmarks.\n* However, it is **not** compared with the latest SOTA on some datasets—for example, *Video Instruction Tuning with Synthetic Data* (LLaVA‑Video‑7B, Oct 2024) scores **83.2** on NEXT‑QA and **67.9** on PerceptionTest, outperforming the results reported here.\n\n---\n\n**Overall:** The paper is clear and of good quality, with modest originality and solid—though not state‑of‑the‑art—significance.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. **Timely exploration of self‑training for video LLMs**\n\n   * Self‑training has been richly studied for text‑only LLMs, but **video understanding remains under‑explored**.\n   * The authors’ method outperforms established alignment-based baselines such as **TPO** and **HOUND‑DPO**, demonstrating that a carefully designed, *self‑critique* loop can yield larger gains.\n   * This work therefore advances the frontier of *video* self‑training and provides a concrete recipe others can build on.\n\n2. **Consistent, model‑agnostic improvements on multiple benchmarks**\n\n   * On **InternVL 2.5**, the proposed framework delivers a non‑negligible improvement of\n\n     $$\n       \\Delta = \\text{+}3.6 \\text{ points (NEXT‑QA)}, \\quad\n       \\text{+}6.8 \\text{ points (EgoSchema)}\n     $$\n\n     over the baseline (exact numbers in Table 2). And it get new SOTA on MVBench with 74.0.\n   * Similar gains transfer to other Video-LLMs, including **Video‑LLaVA** and **LLaVA‑OneVision**, indicating the approach is *model‑agnostic* and not tailored to a single architecture.\n   * The breadth of benchmarks and backbones constitutes a **comprehensive experimental setup** that convincingly demonstrates robustness.\n\n3. **Clear and accessible presentation**\n\n   * The paper is *well‑written* and logically structured; key details (algorithm, training schedule, hyper‑parameters) are easy to locate.\n   * Figures and tables neatly illustrate the self‑training pipeline and quantitative gains, making the contribution **simple yet effective** for readers to reproduce.\n\n*In sum, the work offers a timely and rigorous exploration of self‑training for video LLMs, shows consistent cross‑model improvements, and is presented with exemplary clarity.", "reasons_to_reject": "1. **Missing comparison with current SOTA**\n\n   * The recent baseline *Video Instruction Tuning with Synthetic Data*—**LLaVA‑Video‑7B** (Oct 2024)—achieves\n\n     $$\n       83.2 \\text{ on NEXT‑QA}, \\qquad 67.9 \\text{ on PerceptionTest},\n     $$\n\n     clearly outperforming all results reported in this paper.\n   * Omitting such a strong, publicly available model prevents readers from gauging true progress and risks overstating the contribution.\n\n2. **No scalable mechanism to prevent hallucinations in self‑training pairs**\n\n   * The pipeline relies on the **same \\~7 B base model**—with different prompts—to\n\n     1. generate questions,\n     2. draft answers,\n     3. self‑critique, and\n     4. produce revised answers.\n   * These *(initial ✕ revised)* pairs are fed directly into a DPO stage. Although a human study over 516 videos finds **≈30 %** of the pairs are low‑quality, this filter is *not* applied to the full training set, leaving substantial noise unchecked.\n   * Small video‑LLMs are prone to hallucinating fine‑grained visual details; without automatic verification, the preferred labels may be unreliable and could degrade rather than improve the model.\n\n> **Overall:** By omitting the current SOTA and lacking a scalable quality‑control mechanism, the paper’s empirical claims remain unconvincing.\n\n---", "questions_to_authors": "1. Do you have plans to report results that **directly compare** your method with the current SOTA **LLaVA‑Video‑7B** on NEXT‑QA and PerceptionTest?\n2. What automated strategy can you adopt to **detect or filter hallucinated revisions at scale** to ensure the reliability of the DPO training signal?\n3. Did you perform **decontamination** of the videos used for generating training data (e.g., removing overlaps with benchmark datasets)? If so, how was this done?\n4. Have you evaluated the approach on **weaker video‑LLMs**—particularly those that follow instructions poorly or hallucinate more? Does the self‑training help or harm such models, and is there a performance threshold where it becomes detrimental?", "extra_scores": {}}, {"review_id": "b4fYCtQSuN", "reviewer": "Reviewer_CBX8", "summary": "This paper proposes a framework to align video large language models (video-LLM) with human preference without human supervision. Using a self-critiquing mechanism, the framework gets feedback from other AIs and refine the response. Then, fine-tune/align video-LLM with the refined response using a standard post-training method direct preference optimization (DPO). The proposed method achieves strong performance on various video benchmarks, including MVBench, EgoSchema, and PerceptionTest.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* **Emerging topic.** Self-training is an emerging approach that can improve the performance of multimodal large language models without the need for additional annotations.\n\n* **Strong empirical results.** The proposed method achieves state-of-the-art performance and demonstrates significant improvement over vanilla LLMs. Its effectiveness is validated across various benchmarks, including MVBench, NeXTQA, PerceptionTest, and EgoSchema.\n\n* **Robustness analysis.** The robustness of the proposed method is analyzed in Table 3, demonstrating that under challenging test conditions, it exhibits the smallest performance degradation compared to baseline methods.", "reasons_to_reject": "* (minor) **Overclaim.** The authors claim that self-training is more effective than increasing model parameters when discussing the results in Table 6. However, except for two cases with marginal gains 0.3 and 0.4, in most cases larger models achieved better performance. “These findings confirm that addressing … greater performance gains than merely increasing model capacity” should be toned down. \n\n* (minor) **Manual efforts and comparison with longer prompts.** In the appendix, the authors provide actual self-critique generation prompt. The proposed method requires implicit human supervision. The human manually designed the prompts while monitoring the quality of responses. Also, as existing methods, manual/semi automated filter seems needed for DPO with refined responses. So, in some sense, this pipeline also requires human supervision. Self-training is good but the training with refined responses may not be perfect. Instead, using the criteria, chain-of-thoughts or conversation style generation is possible. With long, elaborated prompts through multiple iterations (CoT/conversation w/ good prompts), it might achieve similar or even better performance  than the proposed method. Do you have analysis regarding this?", "questions_to_authors": "- **Side effect/overfitting of self-training.** The idea of self-training idea is intriguing but as with other post-training/fine-tuning methods, it may negatively impact abilities originally learnt during pre-training. Did you evaluate the performance of the self-trained models on the tasks that they previously handle well?\n- **Computational cost of experiments.** In the experiment section, the experiments are conducted on two GPUs (NVIDIA L40S 48GB). How many GPU hours were required to conduct your experiments?\n- Key difference between self-training and chain-of-thought?", "extra_scores": {}}, {"review_id": "du4mNw8zcX", "reviewer": "Reviewer_XAqh", "summary": "This work proposes VideoSAVi, a self-training video understanding pipeline that reasons over the model’s initial responses to generate improved ones, creating preference pairs. These pairs are then used for DPO training, enhancing the model’s performance without human supervision.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Significant improvements are demonstrated across four video QA benchmarks, with an average gain of +3.4 points over six benchmarks.\n- The memorization and generalization experiments (Table 3) indicate robustness in OOD cases, validating the method’s generalizability.\n- The reward value study (Figure 4) shows that self-constructed preference pairs stabilize DPO training, avoiding collapse.", "reasons_to_reject": "While the experiments are comprehensive, some evaluation settings lack rigor, raising concerns:\n\n- Insufficient baseline details: Missing key information on data sources, scalability, and training setups, which are crucial for fair comparison.\n- Limited data scaling in analysis experiments: While the main experiments use 4,000 videos and 24,000 preference pairs, some ablation studies and appendix analyses rely on only 100–300 videos, weakening their reliability.\n- Ill-formed analysis in Table 5: GPT-4o is used as a critique model, but its own performance is not reported, making it difficult to assess the critique prompt’s quality across models.", "questions_to_authors": "Have you explored directly applying critique and improvement prompts to the model (without preference pair construction and training) to optimize benchmark performance? How do the results compare?", "extra_scores": {}}, {"review_id": "W4oZ8dO1zL", "reviewer": "Reviewer_nQcD", "summary": "The paper introduces VideoSAVi, a self-aligned video language model that improves video understanding without human supervision. It uses a self-critiquing mechanism to identify reasoning errors and generates preference pairs for training, achieving state-of-the-art performance on multiple benchmarks.", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed self-training framework with the quality-aware self-critiquing mechanism is novel and sound.\n\n2. The proposed Direct Preference Optimization (DPO) enhances temporal and spatial reasoning for video understanding.\n\n3. The experimental results confirm that VideoSAVi achieves state-of-the-art performance on several benchmark datasets.\n\n3. The paper is well-written, easy to follow, and well-organized.", "reasons_to_reject": "1. More state-of-the-art baseline models and benchmark datasets should be included to further confirm VideoSAVi's effectiveness.", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper presents a contribution to the field of video understanding by proposing VideoSAVi, a robust and effective self-training framework for Video-LLMs. The core idea of self-alignment through a self-critiquing mechanism and DPO is well-executed. The empirical results are strong and consistently demonstrate improvements across various benchmarks and backbone models. The authors' comprehensive responses to reviewer questions, including additional experiments and clarifications, should be added to the revised version, particularly regarding comparisons with state-of-the-art models, robustness to noise, and potential overfitting.
Decision: Accept