$\mathcal{V}ista\mathcal{DPO}$: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models
Authors: Haojian Huang
Haodong Chen
Shengqiong Wu
Meng Luo
Jinlan Fu
Xinya Du
Hanwang Zhang
Hao Fei
Venue: ICML 2025 poster (2025)
Keywords: Direct Preference Optimization
Large Video Models
Primary area: theory->optimization

Abstract: Large Video Models (LVMs) built upon Large Language Models (LLMs) have shown promise in video understanding but often suffer from misalignment with human intuition and video hallucination issues. 
To address these challenges, we introduce **VistaDPO**, a novel framework for Video Hierarchical Spatial-Temporal Direct Preference Optimization. VistaDPO enhances text-video preference alignment across three hierarchical levels: 
i) **Instance Level**, aligning overall video content with responses; 
ii) **Temporal Level**, aligning video temporal semantics with event descriptions; 
and iii) **Perceptive Level**, aligning spatial objects with language tokens. 
Given the lack of datasets for fine-grained video-language preference alignment, we construct **VistaDPO-7k**, a dataset of 7.2K QA pairs annotated with chosen and rejected responses, along with spatial-temporal grounding information such as timestamps, keyframes, and bounding boxes. Extensive experiments on benchmarks such as Video Hallucination, Video QA, and Captioning performance tasks demonstrate that VistaDPO significantly improves the performance of existing LVMs, effectively mitigating video-language misalignment and hallucination.

VistaDPO: Video Hierarchical PaatialTeempora Direct Preference Optimization for Large Video Models
Haojian Huang"
Haodong Chen 2 Shengqiong Wu Meng Luo
Jinlan Fu?
Xinya Du
Hanwang Zhang' Hao Fei
Abstract
V7
Large Video Models (LVMs) built upon Large Lan- guage Models (LLMs) have shown promise in video understanding but often suffer from misalignment with human intuition and video hallucination issues. To address these challenges, we introduce VistaDPO, a novel framework for Video Hierarchical Spatial- Temporal Direct Preference Optimization. VistaDPO enhances text-video preference alignment across three hierarchical levels: i) Instance Level, aligning over- all video content with responses; ii) Temporal Level, aligning video temporal semantics with event de- scriptions; and iii) Perceptive Level, aligning spa- tial objects with language tokens. Given the lack of datasets for fine- grained lideo-language preference alignment, we construct VistaDPO-7k, dataset of 7.2K QA pairs annotated with chosen and rejected re- sponses, along with spatial-temporal grounding infor- mation such as timestamps, keyframes, and bounding boxes. Extensive experiments on benchmarks such as Video Hallucination, Video QA, and Captioning performance tasks demonstrate that VistaDPO signif- icantly improves the performance of existing LVMs, effectively mitigating video-language misalignment and hallucination. The code and data are available at VistaDPO Repository.
3/u
y/1
V
y/u
(a) Textual DPO
(b) Previous Multimodal DPO
D
V
yeu
yt
Y/u
VistaDPO (c)
"What object did the person down in the video?"
"The person put down
video."
placed some clothes og ground."
3t
musical note instead."
16
5
99 Peresptive Leve!
Temporal Level
O
Iusance Level
(d) VistaDPO Overview
Figure 1. (a) Traditional textual DPO overlooks multimodal infor- mation, limiting video-language tasks. (b) Existing multimodal DPO methods rely coarse alignment, missing rich temporal and perceptual details. (c&d) VistaDPO overcomes these limitations with hierarchical spatiotemporal preference optimization frame- work, enabling fine-grained ladeo-language alignment and precise reasoning over video dynamics. Here, Vw the preferred response over yt, and vw the visual input more likely to produce it than uy.
1. Introduction
cant research attention. Built upon Large Language Models (LLLMs) (Touvron et al., 2023; Bai al., 2023; Peng et al., 2023; Dubey et al., 2024), LVMs leverage the powerful in- telligence of LLMs in language, achieving unprecedented understanding of video content. However, increasing stud- ies reveal that LVMs encounter critical issues, such as video understanding that deviates from human intuition (Zhou et al., 2024a: Fei et al., 2024a: Cheng al., 2024a; Hu et al., 2024) or the phenomenon of video hallucination (Wang et al., 2024; Sahoo et al., 2024; Yuan et al., 2024), where the model outputs content that does not align with the input, egg. user instructions, video content. The root of these issues lies in the inherent nature of current LVM architectures (Yan et al., 2021; Cheng et al., 2024b; Lin et al., 2023), where most LS integrate a video encoder (e.g., ViT) into text-
Achieving human-like reasoning capabilities for videos is critical research topic in the field of AL. In recent years, Large Video Models (LVMs) (Li et al., 2023; Zhang et al., 2023a; Lin et al., 2023; Li et al., 2024c; Wu et al., 2024a; Cheng et al., 2024b; Fei et al., 2024b; Jin et al., 2024; Qian et al., 2024; Li et al., 2025) have garnered signifi-
'Equal contribution 'The University of Hong Kong "The Hong Kong University of Science and Technology "National Uni- versity of Singapore "University of Texas Dallas Nanyang Technological University. Correspondence to: Hao Fei <haofei37 @ eds.edu.sg>.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Vidco Hierarchical Spatial-Temporal Preference Optimization
oriented LLMs through a connector to achieve video signal interpretation. Since backbone L.Mss undergo extensive pre-training on large-scale language data while video en- coders lack peer capability, this gap leads LLMs to produce overly confident outputs based on biased or even incorrect perceptions of video content from the encoder. While the su- pervised fine-tuning (SFT) with video-language pairs (Wang et al.. 2024; Leng et al., 2024; Yuan et al., 2024) can par- tially improve the alignment between the two modalities in LVMs, fundamentally addressing the issue requires reliance on extremely large-scale data.
with event descriptions, enabling temporal reasoning. Perceptive Level: Aligning video spatial objects (Ze., regions of interest) with objective tokens or phrases in the language at fine- line-graieed semantic level. To implement such fine-grained preference optimization, we construct a large-scale spatial-temporally grounded video dataset called VistaDPo-7k. We manually anno- tate 3,878 videos with spatial-temporal groundings in a video QA format, providing high-quality labels for halluci- nated and non-hallln non-hallucinated answers, along with timestamps, keyframes, and bounding boxes of relevant semantics.
Recently, Direct Preference Optimization (DPO) (Rafailov et al., 2024) has been proposed as a promising alternative to SFT. trains L.MMs to prefer responses chosen by evalua- tors over rejected ones when presented with user query. By identifying which response better aligns with human pref- erences rather than requiring precise target outputs, DPO significantly alleviates dependence on annotated data while enhancing alignment with human values and effectively ad- dressing hallucination issues. Some follow-up studies (Xie et al., 2024; Liu et al., 2024d; Zhou et al., 2024b; Fu et al., 2025b) have extended DPO from textual multimodal LLMs, facilitating cross-modal alignment and improving the generalization capabilitiess of the models. Most recently, Hound-DPO (Zhang et al., 2024b) pioneers a video DPO, demonstrating that tailored rewards through DPO can signif- icantly enhance the performance of LVMs. Unfortunately, we find that this work straightforwardly applies the DPO
We conduct extensive evaluation on benchmarks includ- ing Video Hallucination, Video QA, Captioning Tasks, by post-training existing popular LVMs with the proposed Vis- taDPO. The results show that VistaDPO consistently im- proves baseline LVMs, achieving significant average im- provements of 26.42% over PLLaVA and 53.92% over Video-LLaVA respectively. Through in-depth analysis, we show that VistaDPO effectively and comprehensively cap- tures the dynamic interactions between video content and texts, thanks to its hierarchical spatial-tempora alignment strategy. To summarize, this work contributes in threefold: Propose a novel Video Hierarchical DPO (VistaDPO) mechanism, a more fine-grained DPO strategy to optimize the alignment between video and language in LVMs.
Construct and release large-scale (7.2K) high-quality annotated QA pairs dataset, which can serve as a valuable resource for follow-up video DPO research.
strategy designed for image-ter text LLMs to languaee preference alignment (as shown in Figure 1), which intro-
Empirically, VistaDPO significantly improves the gener- alization capabilities of existing LVMs, effectively miti- gating video-language misalignment and hallucination.
duces two critical limitations. First, Zhang et al. (2024b) fails to adequately consider the temporal characteristics of videos. Unlike static images, videos always require both spatial semantic understanding and dynamic temporal rea- soning (Fei et al., 2024c), necessitating a comprehensive modeling of the spatial-temporal attributes of videos. Sec- ond, their work focuses solely on coarse-grained alignment between video and language (response text) at the instance level, which may lead to suboptimal preference alignment (Zeng et al., 2024; Gunjal et al., 2024). We emphasize that achieving proper alignment between two modalities requires a fine- mee-grained preference aligmment. Intuitively, dynamic videos correspond to paired text at multiple hierarchical levels.
2. Related Work
By building on powerful LLMS and integrating various mul- timodal encoders, researchers have developed MLLMs (Liu et al., 2024a; Fu et al., 2025a; Yin et al., 2024; Wu et al., 2024b) and LVMs (Li et al., 2023; Zhang et al., 2023a; Lin et al., 2023; Li et al., 2024c; Cheng et al., 2024b; Jin et al., 2024: Li al., 2025). Through necessary SFT on visual instruction-tuning data, MLLMs and LVMs have not only developed robust multimodal understanding capabilities but have also significantly enhanced human-computer interac- tion, making cross-modal interactions more intuitive and seamless. Unfortunately. inheriting the intrinsic halluci- nation issues of LLMs, LVMs also frequently suffer from hallucinations (Liu et al., 2024b; Zhang et al., 2024b; Li et al., 2024a; Sahoo et al., 2024) fail to align their under- standing of visual content with human values. Increasing the volume of multimodal SFT data has been shown to al- leviate these issues to some extent (Ahn et al., 2024; Tan et al., 2024; Jiang et al., 2024; Chen et al., 2024). However, this approach is often accompanied by higher annotation
To address these challenges, we propose novel framework, Video Hierarchical Temporall Direct Preference Optimization (namely VistaDPO), aiming to strengthen LVMs. VistaDPO improves tixt- preference alignment across hierarchical granularities. Specifically, we design three levels of alignment (as shown in Figure 1):
Instance Level: Matching the overall video content with the most appropriate response for semantic alignment. Temporal Level: Aligning video temporal semantics
2

---

Vidco Hierarchical Spatial-Temporal Preference Optimization
costs and computational expenses. This challenge is partic- ularly pronounced in video scenarios, where LVMs demand significantly larger datasets and higher training costs.
log-likelihood loss, DPO derives the objective function:
To To (x)
To (oulla (z:)
u(æ, yw, vi) = 8 Blog
Tree (Bwl (z) B G
Trer (z) (4)
Subsequently, the community has introduced the DPO tech- nique (Rafailov et al., 2024), where preference alignment aligns LLMs with human values, reducing hallucinations by guiding the model's ' adjustments using pairs of preferred and rejected data. Multimodal preference alignment, as an exten- sion of preference alignment techniques to visual and textual inputs, has been widely applied to MLLMs to improve cross- modal alignment (Liu et al.. 2024d; Xie et al., 2024; Zhou et al., 2024b) as shown in Table 5. Recently, Hound-DPO, pioneered by Zhang et al. (2024b), successfully applies mul- timodal DPO to LVMs, improving video understanding and addressing hallucination issues. However, it overlooks the preference alignment of visual inputs. In this paper, we aim to further enhance the effectiveness of DPO in video sce- narios by modeling fine-grained alignments between video and language. To achieve this, we propose a hierarchical preference optimization framework that efficiently captures dynamic spatial-temporal dependencies in video tasks.
LDpo = (a....u. [log a (u(z, Mu 31)),, where the action score with yi denotes the i-th token of the response y can be formulated as:
log Ttyax) = Σ log plyil,,,,e
(5)
D:EN
4. VistaDPO-7k: A Spatial-temporal Grounded Video DPO Dataset
Existing LVMs often suffer from limited spatial-temporal perception, leading to video-language misalignment and hallucination issues (Lan et al., 2024). We propose Vis- taDPO with spatial-temporal DPO to achieve fine-grained alignment between video and language modalities. To sup- port this, we construct a spatial-temporaa grounded dataset, VistaDPo-7k, by integrating data from 14 prevalent video datasets and systematically designing QA pairs to evaluate and mitigate hallucinations. These hallucinations are categorized into two major dimensions: Perception (e.g.. Object, Static/Dynamic Attribute, Static Relation, OCR) and Temporal (e.g., Action, Dynamic Relation, Sequence), cov- ering both static and dynamic aspects of video understand- ing. The dataset provides chosen and rejected responses, along with fine-grained temporal dependencies that include key timestamps, frames, and bounding boxes, enabling mod- els to better capture spatial-temporal interactions, as can be shown Figure 2(a). Vi VistaDPO-77 7k supports multi- level preference optimization across Temporal, Perceptive, and Instance levels, offering a robust benchmark to reduce hallucinations and enhance the spatial-temporal reasoning capabilities of LVMs. Please refer to Appendix &B for more details on dataset construction and specifications.
3. Preliminaries
Direct Preference Optimization (DPO) (Rafailov et al., 2024) aligns language models with human preferences, re- moving the need for explicit reward modeling or reinforce- ment learning (RL). Given a model To (the target model) and a reference policy Tref (from supervised fine-tuning), the RL objective in reinforcement learning with human feedback (RLHF), initialized with mg = Tyef, is expressed as: max (.yonoo)) [r(z,)) y)] We
5KKL [To(y] (t) || Teery [=]]
where r(z: y) denotes the reward function with x as the input instruction and y as the response. DPO establishes a mapping between the reward model and the optimal policy under the reverse KL divergence, obtaining representation of the reward function concerning the policy:
5. Methodology
To tackle the spatiotemporal complexities in jideo-language tasks, we propose VistaDPO, which implements hierarchical preference optimization across three aspects: (i) Instance- wise Semantic Preference Optimization, aligning prefer- ences at response and video levels; (ii) Temporal Action- Event Preference Optimization, capturing overlooked tem- poral dynamics; and (iii) Perceptive Spatial-Object Pref- erence Optimization, enabling fine-grained alignment be- tween tokens and objects. Figure 2(b) illustrates the overall architecture of VistaDPO.
To <)(x)
r(z, v) = :ogg
Teet (ylz) + Blog 7(x),
(2)
where B is coefficient for the reverse KL divergence penalty, and Zxx) is the partition function.
Given the chosen response Yu. preferred over the rejected response Yı, DPO aligns with human preference using the Bradley-Terry model for pairwise comparisons:
exp(r(æ, Mw))
PBr > ye (z:) exp(r(æ, yw)) + exp(r(x.
(3)
5.1. Instance- Semantic Preference Optimization Effective video video-language alignment hinges on distinguish- ing preferred (chosen) from non-preferred (rejected) re- sponses while capturing global video content. To address hallucinations and misalignments caused by spatiotempo-
By substituting Eq. into Eq. 3 and leveraging the negative
3

---

Video Hierarchical -empoolll Preference Optimization
Object
17%
10%
Dynamic
Number
Temporal (44%)
Attrbune
7%
of
Perception (96%))
Which object did the open video?
Location
219
Dynamic Relation
Instance
Temporal
Percepaive
T
11%
Statie
*
Visual Encoder
Relation
7%
3%
Color
Sequence
+
Projection
Embedding
OCR
WWhich ooject did the cerzon zif on during the scene?
Large Language Model
LoRA
Shuppinz cart
Apwar
Duagons
Copo, Copo C
(Dppo
Cppo
CDoO
Tokens
Tokens
20
SALE
277 MER
70A A 460)
In the video, the opened the window, revealing a bright sunny day outside.
The person in the video seemedi to handling clothes, folding neatly table.
Ho ooT on cheir...
3/u
y
(a) Metadata of VistaDPO-7k
(b) Illustration of VistaDPO
Figure (a) The metadata of VistaDPO-78 7 highlights its focus on fine-graine aee-ggained video-language tasks, emphasizing temporal (44%) and perceptual (56%) reasoning. yi" and yf" denote the irrelevant and relevant non-preferred responses respectively. (b) VistaDPO introduces a hierarchical spatiotemporal preference optimization framework. Instance (v") and perceptive (e/) levels align global-to-local semantics with spatial visual features, leveraging both text-relevant and irrelevant rejected responses for robust cross-modal interaction. Temporal (v") level aligns clip-level semantics with temporal dynamics, enabling precise reasoning across spatial and temporal dimensions.
ral complexities and over-reliance text, propose response-level alignment to refine preference differentia- tion and video-level alignment to enhance instance-wise semantic understanding.
2) failing capture the fine- tne-grained spatial-temporal depen- dencies of the correct content in the video. To mitigate this, we further introduce two types of non con-preferred responses into the optimization process:
Response-Level Alignment. LVMs often face challenges in maintaining global consistency when generating responses. While these models effectively capture the general context of video input and prompt z, they frequently struggle to distinguish user-preferred responses Yu from non coneerefrreed responses at the response level, leading to suboptimal alignment with user intent. To promote overall consistency by encouraging the model to align its response-level prefer- ences with human expectations, the objective function can be formulated as:
log
Te (11l,,
Σ
B, log
Te (yf x)
Tref (Mn
He (V) (r,
(E(r,r)r)
where yf" denotes the relevant non- non-pre -refffrerred for these are semantically relevant to the video content but contain spatial or temporal inconsistencies, e.g., incorrect temporal order- ing, wrong actions, or misinterpreted spatial locations. In contrast, y/" denotes the irrelevant non non-preferred responses, which are entirely unrelated the video content, intro- ducing noise by hallucinating events or objects with no connection to the actual video.
LDPO, E (w. [log (u,(U,, x, yw v1)),,
Video-Level Alignment. Unlike most prior DPO works, which focus exclusively on textual optimization, we intro- duce video-level preference optimization for the first time to reduce LVMs' overreliance on language. At the video level, the model needs to understand the preference rela- tionships of the entire video as a coherent semantic unit. However, since LVMs are prone to hallucinations involving irrelevant video content, we optimize the model to recog- nize global discrepancies among videos. To this end, we construct video-level preferred and non-preferred sample pairs, denoted as uy and vy" Thus u, (v", of x, vw) within LDPO, can be formulated
where
To To(ywlu,2) 3 log Trer (v)
To To( x)
uy = log Tree (we 2,,
Here, log ग (ylu, æ) is defined as:
log (v)0,
Σ logr(ulo,,, E:Ey
(8)
The existing method of Hound-DPO (Zhang et al., 2024b) directly adopts the above approach, focusing solely on align- ing the chosen response with the prompt. Nevertheless, the complex spatial-temporal dependencies rejected re- sponses are completely neglected. Intuitively, intrinsic hal- lucinations in generative models typically arise from: 1) roneously inferring content that does not exist in the video;
Te bi x)
log To(uluy, (10) (Te 1oj. z))
Up = log
Tret (yw luy x) where of is sampled from the mini-batch that unrelated

---

Vidco Hierarchical Spatial-Temporal Preference Optimization
to the query in this work.
align them coherently to form a consistent response. The sequential KI. divergence can be defined as:
5.2. Temporal Semantic Preference Optimization
( (eDse (I, v Ywi Trei (Tor) CDPO. sg
Clip-Level Alignment. While previous multimodal DPO methods have mainly focused on the spatial aspects of visual samples (as shown in Table 5), unlike static images, videos require both spatial semantic understanding and dynamic temporal reasoning. This necessitates a comprehensive mod- eling of the spatial-temporal attributes of videos.
(12)
| SeqkiL (2, v/ yt; Trof | o))
where sg represents the stop-g graddggadientoooo operator, ensuring that gradients are not propagated through the reference policy Trof, and Dsqg is the sequence-level KL divergence:
T
DSegkL = Σ B vce)). (13)
At the temporal level, the model must distinguish between time segments in the video that are relevant to the prompt and those that are irrelevant. To align video temporal seman- ties with event descriptions provided in the prompt, we treat time segments related to the prompt as preferred clips oy and time segments unrelated the prompt as non- con-preferred clips off as shown in Figure 2. Following Eq. (10), the clip-level objective function can be defined as:
1=1
Overall, after incorporating instance-wise, temporal, and perceptive- preference optimization, the overall loss function for VistaDPO is formulated as follows:
Li LDPO, + LDPO,
Instance
(14)
+ XCDPo, + PLDPo. + pLDro,
LDPO, ~ log a (M of, F. )s)))
(11)
Temporal
Perceptive
5.3. Perceptive Spatial-Object Preference Optimization While instance-wiss alignment captures global semantics, fine-grained perceptual alignment is crucial for precise video- video-language interaction. Videos inherently involve com- plex spatial relationships, where objects, actions, and re- gions dynamically interact over time. Language, in turn, encodes these interactions through specific tokens, making it essential to establish detailed alignment between spatial objects and their corresponding linguistic references.
where X, μu and p represent the loss weights.
6. Experiments
In this section, empirically investigate the effectiveness of VistaDPO in reducing hallucinations.
6.1. Experimental Settings
Baselines. We apply VistaDPO to two different 7B-size LVMs: Video-LLaVA (Lin et al., 2023) and PLLaVA (Xu et al., 2024). For Video-LLaVA, it employs Language- Bind (Zhu et al., 2023) encoder for visual inputs, and Vicuna-7B v1.5 (Chiang et al., 2023) as the LLM back- bone. For PLLaVA, the visual input is processed through VIT-I. (Radford et al., 2021) and MM projector, with Vicuna as the LLM backbone. While other LVMs cannot be directly compared due to differences in base models, preference data, and alignment strategies, we provide these results for ref- erence: VideoChatGPT (Maaz et al., 2023). VideoChat2 (Li et al., 2024c), LLaMA-VID (Li et al., 2025), LLaMA- Adapter (Zhang al., 2023b), and Video-LLaMe (Zhang et al., 2023a).
Object-Level Spatial Alignment. At the spatial level, the model needs to capture the key locations and states of ob- jects within the video. However, LVMs are often prone to hallucinations in spatial layouts, leading to incorrect object placements or misinterpretations of scene context. To address this, we strengthen the model's understanding of spatial information through object-level preferred and non-preferred sample design. Specifically, we select the keyframe relevant to the prompt I as the preferred instance oy as shown in Figure. 2. For the non-preferred sample of', we further apply a masking operation to the key regions within the selected frame, thereby focusing the model's 's at- tention on the relevant spatial content while reducing the influence of irrelevant regions. Accordingly, the object-level loss LoPO, can be defined in a manner similar to Eq. (11).
Evaluations. To evaluate the effectiveness of VistaDPO, we adopt benchmarks for three aspects: (1) Video Halluci- nation: VideoHallucer (Wang et al., 2024) and EventHallu- sion (Zhang et al., 2024a): (2) General Video QA: MSVD- QA (Xu et al., 2017), MSR-VTT-QA (Xu et al., 2017), TGIF-QA (Jang et al., 2017), and ActivityNet- (Yu et al., 2019); and (3) Captioning Performance: VideoChaiGPT- Bench (Maaz et al., 2023). For ablation studies and analysis, we mainly employ our VistaDPO on Video-I.LaVA.
Token-Level Alignment. While response-level optimiza- tion enhances global consistency, lacks the gramularity re- quired to address loken-specific errors, such misattributed objects or incorrect temporal markers (e.g.. "after" vs. "be- fore"). Token-level optimization ensures that the model aligns its preferences at finer granularity, thereby reducing hallucinations in object-action relationships. Inspired by TDPO (Zeng et al., 2024), we implement token-level opti- mization to evaluate preferences for individual tokens and
Implementation Details. We train the Video-LLaVA 7B (Lin et al., 2023) and PLLaVA 7B (Xu et al., 2024) with VistaDPO for epochs, with learning rate of 5e 7 and a
5

---

Vidco Hierarchical Spatial-Temporal Preference Optimization
Table Main results video hallucination benchmarks. Bold values indicate the best performance and A denotes the corresponding improvement percentages over the baselines (i.e. PLLaVA and Vico-LLaVA), "T" denotes higher is better.
VideoHallucer
EventHallusion
Models
Basict
Hallucinated? Overall
Entire
Mix
Misleading
Overall
Binaryt
Desc.t
Binaryt
Desc.
Binaryt
Binaryt
Desc.
VidcoChatGPT (Maaz et al., 2023)
92.8
10.4
14.9
5.5
57.0
3.6
21.6
36.4
4.3
VideoChat2 (Li al., 2024c)
29.7
25.8
7.8
16.7
4,6
12.4
1.6
22.6
16.1
2.6
LLaMA-VID (Li et al., 2025)
89.9
26.6
21.0
30.7
16.5
73.6
7.8
43.1
54.0
10.9
PLLaVA (Xu et al., 2024)
75.1
55.5
38.1
45.6
16.5
58.5
3.1
81.4
60.6
6.1
+ Hound-DPO (Zhang et al., 2024b)
69.3
58.
36.2
47.4
19.3
24.9
4.1
83.3
45.7
9.8
+ VistaDPO (Ours)
82.5
72.1
57.8
55.3
23.6
62.2
6.2
97.1
68.9
12.7
A%
9.9
29.9
51.7
21.3
42.7
6.3
100.0
19.3
13.7
108.2
Video-LLaVA (Lin al.,, 2023)
95.1
20.3
17.8
30.7
8.3
57.5
7.3
41.2
45.9
7.6
+ Hound-DPe DPO (Zhang et al., 2024b)
83.4
43.0
29.5
35,9
9.8
15.5
9.3
63.7
33.3
9.5
+ VistaDPO (Ours)
98.2
64.4
54.3
50.9
14.9
62.2
10.4
95.1
67.2
12.1
A%
3.3
217.2
205.1
65.8
79.5
8.2
42.5
130.8
46.4
59.2
Table 2. Main results video QA and captioning benchmarks. Symbols follow the definitions in Table
Question-Answer
Captioning
Models
MSVD1
MSR-VTT
TGIF
Act.Nett
Correct
Detailt
Context
Temporalt
Consist
VideoChatGPT (Maaz et al., 2023)
64.9
49.3
51.4
35.2
2.4
2.5
2.6
2.0
2.4
LLaMA-Adapter (Zhang et al., 2023b)
54.9
43.8
34.2
2.0
2.3
2.3
2.0
2.2
Video-LIaMA (Zhang et al., 2023a)
$1.6
29.6
12.4
2.0
2.2
2.2
1.8
1.8
PLLaVA (Xu et al., 2024)
76.6
62.0
77.5
56.3
3.2
2.9
3.6
2.3
2.9
Hound-DPO (Zhang et al., 2024h)
82.3
73.1
79.9
54.7
3.2
2.8
3.4
2.4
2.7
VistaDPO (Ours)
86.4
80.2
84.3
59.1
3,9
2.8
A%
12.8
29.4
8.8
5.0
9.4
3.5
8.3
21.7
0.0
Video-LLaVA (Lin al, 2023)
71.8
59.0
48.4
45.3
2.8
3.4
2.6
Hound-DPO (Zhang et al., 2024b)
80.7
70.2
61.4
40.9
3.0
2.7
3.3
2.0
2.6
VistaDPO (Ours)
85.3
76.9
74.1
$5.0
3.4
2.9
3.6
2.6
2.9
A%
18.8
30.3
53.1
21.5
5.9
11.5
Table Ablation study level losses on VideoHallucer. Hound- DPO (Zhang et al., 2024b) employs the same strategy as DPO
against the previous video DPO strategy, specifically Hound- DPO, based on LVMs PLLaVA (Xu al., 2024) and Video- LLaVA (Lin al., 2023). As shown in Table 1, we adopted two video hallucination benchmarks, VideoHallucer (Wang et al., 2024) and EventHallusion (Zhang et al., 2024a). The results indicate that VistaDPO significantly alleviates hal- lucination issues compared to Hound-DPO. Notably, while Hound-DPe improved hallucination--elatee performance, they introduced undesirable trade-offs, such as reduced ac- curacy in addressing fundamental categories like the "Basic" class in VileoHallucer. Furthermore, Hound-DPO led to decline in the model's descriptive capabilities and accu- racy, as observed in the "Desc. (Descriptive)" category of EventHallusion. These limitations highlight the shortcom- ings of prior methods and underscore the superiority of our VistaDPO framework and the accompanying VistaDPO-7K dataset. To provide a comprehensive assessment of LVMs' performance post-training, evaluate both their general and captioning capabilities in the following sections.
(Rafailov et al., 2024), hut based on its own constructed dataset.
Methods
Basict
Hallu.
Over:f
VistaDPO
98.2
64.4
54.3
w/o LDPo.
97.8
62.3
53.0
w/o LDPO.
98.1
62.0
52.8
w/o LDPO,, LDPO:
97.6
61.5
49.4
w/o LDPO,, LDPO Lppo.
97.2
60.1
46.6
only LDPO,
95.8
52.3
39.8
Vanilla DPO w/ VistaDPO-7K Hound-DPO
95.4
50.8
38.1
83.4
43.0
29.5
batch size of 8 on H100 GPUs. For training, we followed Zhang et al. (2024b) to set the hyperparameter 3 = 0.1 and followed Zeng et al. (2024) to set p = 0.1 for Cppo. As for hyperparameters of LDro, and Cpro., we set X = 0.4 and μ = 0.2 respectively. Moreover, we set 3re = 0.7 and 3rr 0.3 for the relevant and irrelevant pre-frefedee
responses respectively for LpPO.-
6.2. Main Results
Video Question-Answeriin
In addition to assessing the effectiveness of our VistaDPO in addressing hallucination is- sues, evaluating the model's 's general performance is equally critical. To this end, we conducted evaluations on four com- monly used open-ended general drestion-answeriig bench- marks in a zero-shot setting, illustrated the left side of Table 2. VistaDPO consistently outperforms HoundDPO
We compare VistaDPO with Hound-DPO (Zhang et al., 2024b) on video hallucination, video QA, and captioning benchmarks to verify the effectiveness of our approach.
Video Hallucination. To benchmark VistaDPO, we fo- cused on the model hallucination problem that DPO post- training aims mitigate and compared its performance
6

---

Vidco Hierarchical Spatial-Temporal Preference Optimization
Enfire -
Mix
Misleading
Overall
100
Bimary Accuracty (%6)
302
Blinary Accuracy (%)
400
Bii Pac.
Birary Accuracy
Bimary Accuracy
80-
00
60
80-
0.0
0.1
0.1/0/9
0.3007
0.570.5 Value
0.700.3
0.201
Value
Value
Value
Figure 3. Ablation study of hyperparameters on EventHalluusin..
Halluc
Hallciinted
VistaDPO explicitly captures spatial-temporee temporal dependencies through object-level (LDro.,) and clip-level (Lnro.) opti- mization, enabling the model to better understand localized temporal and spatial relationships. (ii) Implicitly, it encodes apatial-tempord information via response-leve levell (CDpo,) preference alignment, which incorporates both relevant (yf*) and irrelevant (vf") non-preferred responses. These results highlight the importance of fine- fine-grained apatial-temporal dependencies in video understanding, enabling more robust and effective video-language alignment. @ Impact of Comprehensive High-quality Dataset. Under the vanilla DPO strategy, post-training with VistaDPO-7KK outperforms Hound-DPO, which uses a less comprehensive dataset. This demonstrates that a richer and higher-quality dataset im- proves generalization, enhances performance, and effec- tively mitigates hallucinations. 4 Impact of Hyperparam- eters. Additionally, we conduct hyperparameter ablation study (i.e. Figure 3). Specifically, we analyzed the impact of two hyperparameter sets on VistaDPO performance: @ Loss Weights: The optimal weights for all three levels balance the model's ability to capture temporal (clip-level A), spatial (object-level μ)) and fine-grained token dependencies (token- level p). Too low weight for any level weakens the model's s ability to capture relevant dependencies, while excessively high weights disrupt the balance, leading to overfitting to specific details and loss of broader context. @ Weights for Relevan//rrelevant Responses: The combined weight for both non con-preferred samples (v/", yf" helps the model cap- ture spatial-temporal relationships the textual level, which also highlights the need for careful hyperparameter tuning to effectively capture spatial-temporal relationships.
130
TCD
-105-
-00
-10
-103
130
PG1 (c) Niesiio
(9) Videod LaVA
(b) -Hund-DO
Figure 4. T-SNE visualization of representation. (a) Video-LLaVA shows substantial overlap between hallucinated (orange) and non- hallucinated (green) representations. (b) With Hound-DD DPO, there is no distinct improvement in the separation of the two clusters. (c) With VistaDPO, the representations achieve clear clustering, highlighting its superior discriminative capability.
and demonstrates significant performance improvements both base models. These results indicate that VistaDPO not only mitigates hallucination issues to large extent but also enhances its ability to comprehend video content and generate accurate responses to questions.
Captioning Capability. We further evaluate the captioning capabilities of the model using the video-based text gen- eration benchmark proposed by Maaz et al. (2023), which assesses five critical dimensions: Correctness, Detail Ori- entation, Contextual Understanding, Temporal Understand- ing, and Consistency. As shown on the right of Table 2, VistaDPO consistently outperforms Hound-DPP across all dimensions on two base models. These results highlight Vis- taDPO's s ability to generate contextually relevant, detailed, and temporally accurate text from video inputs. Moreover, the findings demonstrate that the post-training process with VistaDPO-7K preserves the model's captioning capabilities, avoiding the degradation observed in Hound-DPO
7. Analyses and Discussions
We now take one step further, providing comprehensive analyses to demonstrate VistaDPO's s superiority.
6.3. Ablation Studies
To evaluate the contributions of each level and their combina- tions, we conduct ablation studies on VistaDPO using Video- LLaVA (Table 3). The key findings are as follows: O Ef- fectiveness of Hierarchical Preference Optimization. The hierarchical optimization strategy significantly improves per- formance, demonstrating its effectiveness in capturing multi- level preferences for better learning and task alignment. @ Importance of Spatial-Temporal Dependencies. Spatial- temporal preference optimization, both explicit and implicit, plays a critical role in enhancing DPO performance: (i)
7.1. Enhanced Video-Language Representation
To empirically demonstrate the effectiveness of VistaDPO, we conduct an analysis from a representational perspec- tive, as illustrated in Figure 4. Specifically using 95 sam- ples (video, non-hallucinated captions, and hallucinated captions) from the "misleading" subset of EventHallusion (Zhang et al., 2024a), evaluated the alignment of vi- sual and textual embeddings. Video-LLaVA exhibits over- lapping features and weak modality alignment, struggling
7

---

Vidco Hierarchical Spatial-Temporal Preference Optimization
70
70
70
67.2
Dieckress Rindom Mask
67.2
67.2
66.0
65
Reteenu
Landom Mane
Reveree
64.8
RGI Mess
101 Move
64.3
63.6
62.3
Segnent
60.9
61.1
Accuraty
80
65
54.3
53.1
$1.4
59.2
54.3
52.0
52.0
50.7
50.2
60-
48.8
45
VideoHtallucor
Event--iilusion
VideoHallucer
Event-iallssonn
VideoHallucer
Eventi-ialuusion
(a) Video-lovel
(b) Clip-level
(c) Object-eevel
Figure S. Ablation study of visual non-preferred samples on two video hallucination benchmarks.
Video LLaVA
+Hound DPO
-only wio Care.
+VistaDFO
logoth
P0/ X1
Origin MMonn
Advensrntl Moan
06.0
10.0
94.8
SWigP 1.86
Skigh 2.26
Accurntyy (96)
72.0
79.5
80)
(A) Video-LLaVA
(3) Rous&
20
Skip 2.42
Shif 3.85
Absolute
femporal Heabive
Longth Felabive
Figure 6. Adversarial temporal testing on VideoHallucer. The gray regions indicate the performance drop under adversarial scenarios for each method.
Gyre,
Figure 7. Kernel Density Estimation (KDE) of log-likelihood dif- ferences in adversarial masking experiments. The log-likelihood difference measures the separation between original and adversar- ial distributions, with the shift representing the mean difference. Larger shifts indicate greater model robustness.
to distinguish hallucinated from non non-hallucinated captions. With Hound-DPO, this issue is partially mitigated through vanilla DPO, but a significant gap between textual and video embeddings remains. In contrast, with VistaDPO, which incorporates hierarchical fine-grained preference model- ing, the alignment is significantly improved by narrowing the distance between visual and textual modalities and dis- tinctly separating hallucinated from -anllllucinated cap- tions. These results underscore VistaDPO's s superior capabil- ity to unify modalities and effectively reduce hallucination. 7.2. Analysis of Visual Non-preferred Samples
clip, and object levels on model performance. At the video level, the Reverse method achieves the highest overall ac- curacy (67 667.2%) significantly outperforming Randommess (54.3%), Blackness (50.2%), and Random Mask (52. 15.1%) 1%). This suggests that disrupting temporal order provides more informative negative samples compared to random sampling or masking strategies, which fail to introduce sufficient semantic contrast. At the clip level, Relevant Segments yields the best performance (64.8%), surpassing Random- ness 153.1%). Blackness (52.9%), Reverse (61.1%), and Random Mask (62.6%). This highlights that using event- irrelevant segments as negatives more effectively challenges the model to focus on event-specific semantics, whereas random or blackened clips lack meaningful contrast. At the object level, ROI Move achieves the highest accuracy (66.0%). outperforming ROI Mask (64.3%), Randommess (54.3%), and Blackness (53.7%). This indicates that spa- tially disrupting key objects introduces more challenging and informative negative samples compared to masking or random sampling. Overall, these results emphasize that well- designed, semantically targeted negative samples- such as those disrupting temporal order, leveraging event irrel- evance, altering spatial relationships are crucial for enhancing the model's ability to distinguish fine-grained video- Iido-language alignments.
The quality of preference samples depends on the rejection visual samples and the gap between rejection and chosen samples. We explore strategies for constructing rejection samples at the video, clip, and object levels, while keeping the chosen samples (original video, event segment, and keyframe) unchanged for each level as shown in Figure 5.
Select a random sample from the minibatch. (ii) Blackness: Set all RGB values of the chosen sample to O. (iii) Reverse: Reverse the order of all frames in the chosen sample. (iv) Random Mask: Mask half the frames in the chosen sample.
Video-level: (i) Randonness:
Clip-level: (i) Randomness. (ii) Blackness. (iii) Reverse. (iv) Random Mask. (v) Relevant Segments: Use segments where the event does not occur.
Object-level: (i) Randomness. (ii) Blackness. (iii) ROI Mask: Mask the key object in the chosen sample. (iv)
ROI Move: Move the key object to disrupt its original spatial relationships.
7.3. Adversarial Temporal Testing
As demonstrated in Figure 5, we observe the following per- formance trends: Figure 5 demonstrates the impact of dif- ferent negative sample construction strategies across video,
To evaluate the robustness of VistaDPO, we conducted ad- versarial temporal testing using the "Temporal" subset of VideoHallucer (Wang et al., 2024), which includes three cat-
8

---

Vidco Hierarchical Opatial-Tempore Preference Optimization
Does man tries to stop a man from shooting a basket ball" happee earlier than The ball is pessed across the room and men chose it across the court"?
Does the girl play with her pet in the video?
boes a dog run night a person in the video?
Yes.--
Wi Hamm--PP Viss
w/ VanuDPo) Yes...
Yes,.. Hound DPO Yes... w/ FlaaDPo Yes,-.
Mc Hound-DPO Yess..
wi PisetPo
Ves,
dag stand by a person in the video?
Hound-DPO
FivadPP
Does the girl play with her pet the videa?
dog walk right a person the vides?
the
her
vides.
dog right past person
Does
happen earlier than ...??
teawing the dog and playing with the souch.
wc
Hound-DOO Ves, dog meen we milking alorge the person video, indicating franing exercise ceronnstration of control and obedience.
Yes," hespers carlier i....
Me Hound-DDP) Yes,, the girl oppears to play fully interact with
the pet, encouroging it to play and move around.
Hound DRO Yes, haspen carlier than...
only wio Lano,
the girl appears play and interect that sprowled beige couch.
Jonly wio Coro,) Yes, walks right porson the vides. fooisteps.
following
only w/o Cypo, Yes, hoppant eorlier than
nr VisaPPO))
No. 's mas tries to stop a man from sheoting a
w'' Kinbal/P No. there is no intersction or play between the gift and her get in the video There is no direct contact between the grl and her pet throughout.
Visaro
No, is specifically a dog nurring fowcrds a person in the video rather walking rewards thet person.
Sanket boll' hoppare ofter than tthe ball is passnd
ocress the room and non chase it acreas the court
(5) Temporal adversurial testing demonstration
(b) Spatial adversarial testing demonstration
Token adversarial lesting demonsttation
Figure& 8. Case Studies of Adversarial Testing for VistaDPO: We conduct case studies from three perspectives: (a) Temporal adversarial
testing, which examines whether the model can infer the correct sequence of events by introducing reversed temporal order through video playback. (b) Spatial adversarial testing, which evaluates the model's ability to understand subject-object interactions by masking frames or pixels related to the target object. (c) Token adversarial testing. which tests the model's sensitivity to subtle linguistic differences by introducing similar action descriptions (e.g.. contrasting "run" with "stand" and "walk"). Each test compares VistaDPO with baselines (i.e. Video-I.La and Hound-DPO) and corresponding ablated versions to assess the impact of key components.
egories of video-based QA tasks: (i) Temporal Absolute, focusing on when an event occurs; (ii) Temporal Relative, addressing the order of two events; and (iii) Length Rela- tive, comparing the duration of two events. For adversar- ial testing, we reversed all videos and adjusted answers to align with the reversed timeline (as shown in Figure 8(a). As shown in Figure 6, the base model (Video-LLaVa, and prior work (Hound-DPa) -PO) suffer significant performance drops across all three adversarial scenarios, revealing their inabil- ity to effectively model temporal hallucinations and vul- nerability to timeline modifications. In contrast, VistaDPO shows minor degradation, demonstrating better temporal awareness and robustness against adversarial challenges.
highlighting the importance of the proposed spatial-object preference optimization. These show VistaDPO's s superior ability to capture subtle semantic differences and enhance adversarial robustness.
7.5. Adversarial Token Testing
As shown in Figure 8(c), we conduct adversarial token test- ing to evaluate model robustness, For the original question, "Does dog run right a person in the video?", all models answered correctly. When "run" was replaced with "stand" (a significant semantic shift), most models maintained ac- curate responses. However, with an adversarial sample re- placing "run" with "walk" (a subtle semantic change), only VistaDPO correctly captured the nuanced difference. This underscores VistaDPO's s robust token-level understanding. capturing tne-ggraneed semantic shifts and ensuring precise video-language alignment under adversarial conditions.
7.4. Adversarial Spatial Testing
To evaluate spatial adversarial robustness, we test with video and the question, "Does the girl play with her pet in the video?" As shown in Figure 8(b), all models correctly respond to the original video (upper side). However, in the adversarial version (lower side), where frames are masked to ensure the girl and pel never uppear logether, only Vis- taDPO correctly identifies the absence of interaction. To further assess adversarial discriminative capability, we use Kernel Density Estimation (KDE) on the VideoHallucer dataset to visualize how model representations shift when reasoning over noisy (adversarial) samples (see Figure 7). Video-LLaVA achieves a shift value of 1.86, showing lim- ited ability to distinguish between original and adversarial samples. Adding Hound-DPO slightly reduces the shift to 1. 1.26, indicating no improvement. VistaDPO achieves the highest shift value of 3.85, significantly outperforming other models. Removing CPO,, reduces the shift to 2.42,
8. Conclusion
In this paper, we propose VistaDPO, a novel framework for Video Hierarchical Optimization, which enhances the alignment between text and video preferences across three hierarchical levels: in- stance, temporal, and perceptive. To support fine-grained preference alignment, we introduce VistaDPO-7k, dataset of 7.2K QA pairs with annotations for chosen/rejected re- sponses and spatial-temporal groundings. Extensive eval- uations on tasks, i.e., Video Hallucination, Video QA, and Captioning benchmarks demonstrate that VistaDPO signifi- cantly improves existing LVMs, addressing laneo-language misalignment and hallucination issues.
Direct Preference

---

Vidco Hierarchical Spatial-Temporal Preference Optimization
Impact Statement
Cheng, S., Fang, K., Yu, Y., Zhou, S., Li, B., Tian, Y., Li, T., Han, I.., and Liu, Y. Videgothink: Assessing egocentric video understanding capabilities for embodied ai. arXiv preprint arXiv:2410.11660 2024a.
This paper presents work whose goal is to advance the field of Machine Learning, particularly in the domain of video- language alignment and large video models (LVMs). By in- troducingVis sttpvssstDDO framework for hierarchical spatial- temporal direct preference optimization, and constructing the VistaDPO-77 7k dataset, we aim to improve the align- ment between video content and human preferences, mit- igating issues such as hallucination and misalignment in video-language tasks.
Cheng. Z., Leng. S., Zhang, H., Xin, Y., Li, X., Chen, G., Zhu, Y., Zhang. W., Luo, Z.,, Zhao, D., al. Videollama 2: Advancing spatial-temporal modeling and audio under- standing in video-Ilms. arXiv preprint arXiv:2406.0742 2024b.
Chiang. W.-L., Li, Z., Lin, Z.., Sheng, Y., Wu, Z., Zhang, H., Zheng, L.,, Zhuang, S., Zhuang, Y., Gonzalez, J. E., et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicua. Imsys. org (accessed 14 April 2023), 2(3):6, 2023.
The potential societal impact of this work includes enhanc- ing the robustness and reliability of Al systems in appli- cations such as video analysis, autonomous systems, and multimedia content understanding. While these advance- ments could contribute positively to fields like education, accessibility, and cntertainment, they also raise ethical con- siderations, including potential misuse in surveillance or biased decision-making the models are not carefully eval- uated for fairness and accountability.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint urXiv:2407.21726 2024.
We have taken steps to ensure that the dataset and method- ology are designed to reduce biases and hallucinations, and we encourage future researchers to apply these methods responsibly. Beyond these considerations, there are no im- mediate societal consequences of this work that require specific attention.
Ethayarajh, K., Xu, W., Muennighoff, N., Jurafsky, D., and Kiela, D. Kto: Model alignment prospect theoretic optimization. arXiv preprint arXiv: arXiv:2402.01306 2024.
Fei, H., Wu, S., Ji, W., Zhang, H., Zhang. M.. Lee, M.- L., and Hsu, W. Video-of-thie thouhht:: Step-by-step video reasoning from perception to cognition. In Forty-first International Conference on Machine Learning, 2024a.
References
Fei, H., Wu, S., Zhang, H., Chua, T.-S., and Yan, S. Vit- ron: A unified pixel-level vision Ilm for understanding, generating, segmenting, editing. 2024b.
Ahn, D., Choi, Y., Yu, Y., Kang, D., and Choi, J. Tun- ing large multimodal models for videos using rein- forcement learning from ai feedback. arXiv preprint arXiv:2402.03746, 2024.
Fei, H., Wu, S., Zhang, M., Zhang, M., Chua, T.-S., and Yan, S. Enhancing video-language representations with structural spatio-temporal alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024c.
Azar, M. G., Guo, Z. D., Piot, B., Munos, R., Rowland, M., Valko, M., and Calandriello, D. A general theoretical paradigm understand learning from human preferences. In International Conference Artificial Intelligence and Statistics, pp. 4447-4455 PMLR, 2024.
Fu, C., Dai, Y., Luo, Y., Li, L., Ren, S., Zhang, R., Wang, Z., Zhou, C., Shen, Y., Zhang, M., et al. Video-mme. The first-ever comprehensive evaluation benchmark of multi-modal Ilms in video analysis. arXiv preprint 2024.
Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al. Qwen technical report. arXiv preprini arXiv:2309. 16609, 2023.
Fu, C., Lin, H., Wang, X., Zhang, Y.-F., Shen, Y., Liu, X., Li, Y., Long, Z., Gao, H., Li, K., et al. Vita-1.5: Towards gpt- 4o level real-time vision and speech interaction. arXiv preprint arXiv:2501.0195: 2025a.
Chen, D. and Dolan, W. B. Collecting highly parallel data for paraphrase evaluation. In Proceedings of the 49th annual meeting of the association for computational lin- guistics: human language technologies, pp. 190-200, 2011.
Fu, J., Huangfu, S., Fei, H., Shen, X., Hooi, B., Qiu, X., and Ng, S.-K. Chip: Cross-modal hierarchical direct prefer- ence optimization for multimodal llms. arXiv preprint arXiv:2501.16622 2025b.
Chen, H., Huang, H., Dong, J., Zheng, M., and Shao, D. Finecliper: Multi-modal fine-graine clip for dynamic fa- cial expression recognition with adapters. In Proceedings of the 32nd ACM International Conference on Multime- dia, pp. 2301-2310, 2024.
Gunjal, A., Yin. J., and Bas, E. Detecting and prevent- ing hallucinations in large vision language models. In
10

---

Video Hierarchical Spatial-Temporal Preference Optimization
Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 38, pp. 18135-18143, 2024.
Lin, B., Ye, Y., Zhu, B., Cui, J., Ning, M., Jin, P., and Yuan, L., Video-Ilava: Learning united visual represen- tation by alignment before projection. arXiv preprint 2rXiv:2311.1011 2023.
Hu, G., Xin, Y., Lyu, W., Huang, H., Sun, C., Zhu, Z., Gui, L., Cai, R., Cambria, E., and Seifi, H. Recent trends of multimodal affective computing: A survey from nip perspective. arXiv preprint arXiv:2409.07360 2024.
Liu, H., Li, C., Wu, Q. and Lee, J. Visual instruction tun- ing. Advances in neural information processing systems, 36, 2024a.
Jang. Y., Song, Y.. Yu, Y., Kim, Y., and Kim, G. Tgif- qa: Toward patio-ttmppora reasoning in visual question answering. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2758-2766, 2017.
Liu, H., Xue, W. Chen, Y., Chen, D., Zhao, X., Wang, K., Hou, L., Li, R., and Peng, W. A survey on halluci- nation large vision- language models. arXiv preprint arXiv:2402.00255 2024b.
Jiang, X., Ge, Y., Ge, Y., Shi, D., Yuan, C., and Shan, Y. Supervised fine-tuning in turn improves visual foundation models. arXiv preprint arXhv:2401.10222 2024.
Liu, Y., Li, S., Liu, Y., Wang, Y., Ren, S., Li, L.., Chen, S., Sun, X., and Hou, L. Tempcompass: Do video Ilms really understand videos? arXiv preprint arXiv:2403.000 240.00403.0 2024c.
Jin, P., Takanobu, R., Zhang, W., Cao, X., and Yuan, L. Chat-univi: Unified visual representation empowers large language models with image and video understanding. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 13700-13710, 2024.
Liu, Z., Zang, Y., Dong, X., Zhang, P., Cao, Y., Duan, H., He, C., Xiong, Y., Lin, D., and Wang, Mia- dpo: imati-image augmented direct preference optimiza- tion for large vision- language models. arXiv preprint arXiv:2410. 17377, 2024d.
Lan, W., Chen, W., Chen, Q., Pan, S., Zhou, H., and Pan, Y. A survey of hallucination in large visual language models. 2024.
Lu, J., Li, J., An, S., Zhao, M., He, Y., Yin, D., and Sun, X. Eliminating biased length reliance of direct preference optimization via down-sampled kl divergence. arXiv preprint arXiv:2406.10957 2024.
arXiv preprint
Leng, S., Xing, Y., Cheng, Z., Zhou, Y., Zhang, H., Li, X., Zhao, D., Lu, S., Miao, C., and Bing, L. The curse of modalitmoda Evaluating hallucinations of large multimodal models across language, visual, and audio. arXiv preprint arXiv:2410.12758 2024.
Maaz, M., Rasheed, H., Khan, S., and Khan, F. S. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXhv:2306.05424 2023.
Li, C., Im, E. W. and Fazli, P. Vidhalluc: Evaluat- ing temporal hallucinations in multimodal large lan- guage models for video understanding. arXiv preprint 24XIv:2412.03335, 2024a.
Maaz, M., Rasheed, H., Khan, S., and Khan, F. Videogpt+: Integrating image and video encoders for enhanced video understanding. arXiv preprint arXiv:2406. 09100. 2024.
Li, F., Zhang, R., Zhang, H., Zhang, Y., Li, B., Li, W., Ma. Z., and Li, C. -nnteeetiinteree Tackling multi- image, video, and 3d in large multimodal models. arXiv preprint ar arXiv:2407.00 400.0999. 2024b.
Mosig, J. E., Mchri, S., and Kober, T. Star: A schema- guided dialog dataset for transfer learning. arXiv preprint arXiv:2010.11883 2020.
Li, K., He, Y., Wang. Y., Li, Y., Wang, W., Luo, P., Wang, Y., Wang, L.,, and Qiao, Y. Videochat: Chat-centric video understanding. arXiv preprint 230i:23305006355 2023.
Park, R., Rafailov, R., Ermon, S., and Finn, C. Disentan- gling length from quality in direct preference optimiza- tion. urXiv preprint arXiv:2403. 19159, 2024.
Li, K., Wang, Y., He, Y., Li, Y., Wang. Y., Liu, Y., Wang, Z., Xu, J., Chen, G., Luo, P., al. Mvbench: A compre- hensive
Peng, B., Li, C., He, P., Galley, M., and Gao, J. Instruc- tion tuning with gpt-4. arXiv preprint arXiv:2304.03277, 2023.
video understanding benchmark. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 22195-22006, 2024c.
Pi, R., Han, T., Xiong, W., Zhang, J., Liu, R., Pan, R., and Zhang. T. Strengthening multimodal large language model with bootstrapped preference optimization. In European Conference on Computer Vision, pp. 382-398. Springer, 2025.
Li, Y., Wang, C., and Jia, J. Llama-vid: An image is worth 2 tokens in large language models. In European Conference on Computer Vision, pp. 323-340 Springer, 2025.

---

Video Hierarchical Spatial-Temporal Preference Optimization
Qian, L.,, Li, J., Wu, Y., Ye. Y., Fei, H., Chua, T.-S., Zhuang, Y., and Tang, S. Momentor: Advancing video large lan- guage model with fine-graines temporal reasoning. arXiv preprint arXiv:2402. 11435, 2024.
Wu, S., Fei, H., Qu, L., Ji, W., and Chua, T.-. NExT- GPT: Any-to-any multimodal LLM. In Proceedings of the International Conference Machine Learning, pp. 53366-53397, 2024a.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8766 PMLR, 2021.
Wu, Z., Chen, X., Pan, Z., Liu, X., Liu, W., Dai, D., Gao, H., Ma, Y., Wu, C., Wang, B., et al. Deepseek-v12, Mixture- of-experts vision-language models for advanced multi- modal understanding, arXiv preprint arXiv:2412.10300 2024b.
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. Er- mon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Ad- vances in Neural Information Processing Systems, 36, 2024.
Xiao, J., Shang, X., Yao, A., and Chua, T.-S. Next-qa: Next phase of question-anse answering to explaining temporal actions. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 9777-9786, 2021.
Sahoo, P., Meharia, P., Ghosh, A., Saha, S., Jain, V., and Chadha, A. A comprehensive survey hallucination in large language, image, video and audio foundation models. Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 11709-11724, 2024.
Xie, Y., Li, G., Xu, X., and Kan, M.-Y. V-dpo: Mitigating hallucination in large vision language models via vision- guided direct preference optimization. arXiv preprint arXiv:2411.02712 2024.
Sarkar, P., Ebrahimi, S., Etemad, A., Beirami, A., Anı,, S. Ö., and Pfister, T. Mitigating object hallucination via data augmented contrastive tuning. arXiv preprint arXiv:2405. 186544 2024.
Xu, D., Zhao, Z., Xiao, J., Wu, F., Zhang, H., He, X., and Zhuang, Y. Video question answering via gradually refined attention over appearance and motion. In Pro- ceedings of the 25th ACM international conference on Multimedia, pp. 1645- 2017.
Shangguan, Z., Li, C.,, Ding. Y., Zheng, Y., Zhao, Y., Fitzger- ald, T., and Cohan, A. Tomato: Assessing visual temporal reasoning capabilities in multimodal foundation models. arXiv preprint ar arXiv:2410.23266 2024.
Xu, J., Mei, T., Yao, T., and Rui, Y. Msr-vtt: A large video description dataset for bridging video and language. In Proceedings of the IEEE conference computer vision and pattern recognition, pp.
2016.
Tan, Z., Yang, X., Qin, L., Yang. M., Zhang, C., and Li, H. Evalalign: Supervised fine-tuning multimodal lims with human-alignee data for evaluating text-to-image models. arXiv preprint arXiv:2406. 16562, 2024.
Xu, L., Zhao, Y., Zhou, D., Lin, Z., Ng. S. K., and Feng, J. Pllava: Parameter- frume llava extension from images to videos for video dense captioning. arXiv preprint arXiv:2404 16994, 2024.
Tom, G., Mathew, M., Garcia-Bordils, S., Karatzas, D., and Jawahar, C. Reading between the lanes: Text videoqa on the road. In International Conference on Document Analysis and Recognition, pp. 137-154. Springer, 2023.
Yan, W., Zhang, Y., Abbeel, P., and Srinivas, A. Videogpt: Video generation using vq-vae and transformers. arXiv preprint arXiv:2104.1000 10157, 2021.
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A,, Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F. et al. Llama: Open and efficient foundation lan- guage models. arXiv preprint arXiv:2302.13997 2023.
Yang, K., Liu, Z., Xie, Q. Huang, J., Min, E., and Ananiadou, S. Selective preference optimization via token-level reward function estimation. arXiv preprint arXiv:2408. htth:2408.13111 13518, 2024.
Wang, X., Wu, J., Chen, J., Li, L., Wang. Y.-F., and Wang, W. Y. Vatex: A sarge-scale, high-quality multilingual dataset for video-and-language research. In Proceedings of the IEEE/CV international conference on computer vision, pp. 4581 1581-4591, 2019.
Yi, K., Gan, C., Li, Y., Kohli, P., Wu, J., Torralba, A., and Tenenbaum, J. B. Clevrer: Collision events for video representation and reasoning. arXiv preprint arXiv:1910.01 10l.:00900. 2019.
Wang. Y., Wang, Y., Zhao, D.. Xie, C., and Zheng, Z. Videohallucer: Evaluating intrinsic and extrinsic halluci- nations in large language models. arXiv preprint arXiv:2406. 16338, 2024.
Yin, S., Fu, C., Zhao, S., Xu, T., Wang, H., Sui, D., Shen, Y., Li, K., Sun, X., and Chen, E. Woodpecker: Hallucination correction for multimodal large language models. Science China Information Sciences, 1712222201 2024.
12

---

Video Hierarchical Spatial-Temporal Preference Optimization
Yu, Z., Xu, D., Yu, J.. Yu, T., Zhao, Z., Zhuang. Y., and Tao, D. Activitynet-qa: A dataset for understanding complex web videos via question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 9127-9134, 2019.
Zhu, B., Lin, B., Ning, M., Yan, Y., Cui, J., Wang, H., Pang, Y., Jiang, W., Zhang, J., Li, Z., et al. Language- pretraining to n-modality
bind: Extending by language-basse based semantic alignment. arXiv preprint arXiv:2310.01852, 2023.
Yuan. Y., Zhang, H., Li, W. Cheng, Z., Zhang, B., Li, L., Li, X., Zhao, D., Zhang, W., Zhuang, Y, et al. Videorefer suite: Advancing spatial-temporal object understanding 2024.
with video Ilm. arXiv preprint arXiv:2501.000
Zeng, Y., Liu, G., Ma, W., Yang, N., Zhang, H., and Wang, Token-level direct preference optimization. arXiv preprint ar arXiv:2404. 1999, 2024.
Zhang. H., Li, X., and Bing, L. Video-Ilama instruction- tuned audio-visual language model for video understand- ing. arXiv preprint vrXiv:2306.02855 2023a.
Zhang, J., Jiao, Y., Chen, S., Chen, J., and Jiang, Y.-G. Eventhallusion: Diagnosing event hallucinations in video llms. arXiv preprint ArXi::2409. 16597, 2024a.
Zhang, R., Han, J., Liu, C., Gao, P., Zhou, A., Hu, X., Yan, S., Lu, P., Li, H., and Qiao, Y. alama-adapterr Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303. 16199, 2023b.
Zhang, R., Gui, L., Sun, Z., Feng, Y., Xu, K., Zhang, Y., Fu, D., Li, C., Hauptmann, A., Bisk, Y., et al. Di- rect preference optimization of video large multimodal models from language model reward. arXiv preprint arXiv:2404.0123 2024b.
Zhao, M., Li, B., Wang, J., Li, W., Zhou, W., Zhang, L., Xuyang, S., Yu, Z., Yu, X., Li, G., et al. Towards video text visual question answering: Benchmark and baseline. Advances in Neural Information Processing Systems, 35: 35549-35562, 2022.
Zhao, Z., Wang, B., Ouyang, L., Dong, X., Wang, J., and He. C. Beyond hallucinations: Enhancing lylms through hallucination-awanl direct preference optimization, arXiv preprint arXiv:231 16839, 2023.
Zhou, I., Xu, C., and Corso, Towards automatic learning of procedures from web instructional videos. In Proceed- ings of the AAAI Conference Artificial Intelligence, volume 32, 2018
Zhou, T., Chen, D., Jiao, Q., Ding, B., Li, Y., and Shen, Y. Humanvbench: Exploring human-centric video under- standing capabilities of milms with synthetic benchmark data. arXiv preprint 1rXiv:2412.17577 2024a.
Zhou, Y., Cui, C., Rafailov, R., Finn, C., and Yao, H. Align- ing modalities in vision large language models via pref- erence fine-tuning. arXiv preprint arXiv:2402.11411 2024b.
13

---

Video Hierarchical Spatial-Temporal Preference Optimization
A. Limitation and Future Work
While VistaDPO excels at aligning video and language with fine-grained precision, its performance on long-duration videos with complex temporal dependencies leaves room for improvement. Such scenarios pose unique challenges for any alignment framework. Building on our strong spatial-temporal modeling foundation, future work could explore hierarchical architectures or nemory-augmentt mechanisms further enhance the ability to capture long-term interactions, extending the reach of our method to even more complex video-language tasks.
B. More Details of Data Annotation
Table 4. Summary of Hallucination Types, Sample Counts, and Data Sources.
Hallucination Type
Sample Count
Data Source
Object
1,200
MSR-VTT, STAR, VATEX
Number
500
ActivityNet -A, MSR- VTT. NEXT-QA, VATEX
Location
500
MSR-VTT, NEXT-QA, VATEX
Color
500
ActivityYet-QA, CLEVRER, MSR-VTT, VATEX
Static Relation
800
ActivityNet-QA,, MSR-VTT, VATEX
OCR
500
RoadTextVQA, ViteVQA
Action
1,200
MSR-VTT, MSVD. STAR, VATEX
Dynamic Auribute
300
TempCompass. Tomato
Dynamic Relation
1,500
MSR-VTT, NEXT-QA. STAR, VATEX, VCGBench--iveene Vidco-MME, YouCook2
Sequence
200
Datasets Sources. We constructed dataset by sampling from the validation sets of 14 existing datasets in Table specifically MSR-VTT (Xu et al., 2016), STAR (Mosig et al., 2020), VATEX (Wang et al., 2019), ActivityNet-QA (Yu et al., 2019), NExT-QA (Xiao et al., 2021), CLEVRER (Yi et al., 2019), RoadTextVQA (Tom et al., 2023), ViteVQA (Zhao et al., 2022), MSVD (Chen & Dolan, 2011), TempCompass (Liu et al., 2024c), Tomato (Shangguan et al., 2024), VCGBench-Diverse (Maaz et al., 2024), Video-MME (Fu et al., 2024), and YouCook2 (Zhou et al., 2018), encompassing tasks such as binary QA, multiple-choice QA, and captioning-Q2 QA. To define hallucination within the context of video-based QA, we categorized it into two dimensions: Perception and Temporal, and generated corresponding chosen and rejected responses.
Specifically, the Perception dimension evaluates the model's ability recognize static information in videos. This includes object recognition, identifying static attributes (e.g., number, color, position), understanding spatial relationships between objects, and extracting other elements such as OCR. In contrast, the Temporal dimension assesses the model's ability
Question: Which object did the person sit on during the scene?
Chosen:
The person sat chair during the
Rejected1: In the scene, the person chose to sit in shopping cart. (Trrelevant)
(Relevant but Wrong)
Rejected2: During the scene, the person sat at the table instead.
(b) Augmented Chesen/Rejected QA-pairs
Question:
Which object did the person sit on during the scene?
Answer:
The person sat on chair during the scene.
(a) Original QA Pairs from Exsiting Prevalent Datasets
shopping cart (Felse
VIDEO?
table (True)
Appear (22.63s)
(26.694)
Disuppear (34.134) f..t
(-))
(c) Manually Annotated Temporal Information
Figure 9. Illustration of datasct pipeline for constructing augmented video-language QA pairs. (a) Original QA pairs are extracted from existing prevalent datascts, providing basic QA pairs. (b) These pairs are augmented by introducing chosen and rejected answers, where rejected answers include both irrelevant responses (e.g., "shopping cart") and relevant but incorrect ones (e.g., "table"). (c) To enhance spatiotemporal understanding, manual annotations are added, specifying object appearances. spatial coordinates (e.g., bounding boxes). and temporal dynamics (e.g.. appearance and disappearance timestamps). This pipeline ensures richer, more nuanced data for hierarchical preference optimization in video-language tasks.
14

---

Vidco Hierarchical Spatial-Temporal Preference Optimization
comprehend dynamic temporal information, such as recognizing actions, identifying subtle dynamic attributes (e.g.. movement direction, speed, shape), understanding event relationships, and perceiving action sequences within the video. By leveraging the prompt structure illustrated in Figure 9, we expanded the original QA data into a dataset suitable for DPO training with chosen and rejected responses. During the construction of rejected response, we carefully considered whether the core semantics of the question were present in the video, generating both relevant and irrelevant rejected responses. This approach aims to enhance the model's global understanding and robustness at the response level.
To explicitly strengthen the model's spatiotemporal perception capabilities, we first identified all objects involved in the video. Subsequently, we manually annotated keyframes in which at least 30% of the object's contours appeared or disappeared in the frame, as well as any keyframes directly relevant answering the question. For each annotated keyframe, we labeled the bounding box coordinates (i.e., (I,y, y, w, h)) of the objects.
Quality Control. To ensure annotation quality, all annotators were PhD students from universities who underwent standardized training and utilized a unified annotation tool. Each video was annotated independently by two annotators, and cross-validation was performed. Samples with annotation discrepancies were discarded to maintain high data quality.
C. More Discussions on Related Work
Table S. Comparison among different DPO strategies.
Base Model
DPO
Textual
Visual
Method
LLM
(7B, not specified)
Text
Image
Video
Granularity
Dimension
DPO (Rafailov et al., 2024)
Text
2.8
X
X
Sentence
IPO (Azar et al., 2024)
Text
Pythia-2.8B
X
Sentence
KTO (Ethayarajh al., 2024)
Text
Llama-3-8B & Qwen-3B-Instruet
Sentence
R-DPO (Park et al., 2024)
Text
Pythia-2.8B
X
Sentence
SamPO (Lu et ul., 2024)
Text
Tulu2-13B-SFT & Llama3-8B-Inniee
X
X
Sentence
SePO (Yang et al., 2024)
Text
LLaMA2-Chat & Pythia-SFT--699
X
Sentence & Token
TDPO (Zeng et al., 2024)
Text
GPT-2-Large
Sentence & Token
HA-DPO (Zhao et al. 2023)
Image
LLaVA-vI.5 MiniGPT-4
Sentence
BPO (Pi et al., 2025)
Image
ILaVA-vI.5
X
Sentence
FDPO (Gunjal et al., 2024)
Image
InstructBI.IP IIB
X
X
Sentence
HALVA (Sarkar et al., 2024)
Image
ILaVA-v1.5
X
Sentence & Token
POVID (Zhou et al., 2024b)
Image
LLaVA-v1.5
Sentence
Spatial Spatial Spatial
MIA-DPO (Liu et al., 2024d)
Image
LLaVA-v1.5 & IntemLM-XC2.5
Sentence
V-DPO (Xic et al., 2024)
Image
LLavA-v1.5
Sentence
NeXi-DPO (Li el al., 2024b)
Video
LLaVA-Nexi
X
Sentence
Hound-DPO (Zhang al., 2024b)
Video
Video-LLaVA
Sentence
VistaDPO (Ours)
Video
Video-LLaVA & PLLaVA
Sentence & Token Spatial & Temporal
To highlight our contributions, we detail in Table 5 how our proposed VistaDPO differs from previous DPO strategies. Two critical distinctions are summarized as follows:
TpeporllTemporal Video Preference Optimization: Previous DPO methods predominantly focused on language-level alignment. With advancements in the field, the focus gradually shifted from language models to laiiunla models. While some works incorporated image-level visual alignment, these approaches remained limited to static images. Recent works like LLaVA-Next-DPO (Li et al., 2024b) and -LaVA-Hound-DPO (Zhang al., 2024b) extended DPO strategies to video-language models. However, these methods only applied vanilla DPO strategies, optimizing alignment exclusively at the language level, with no explicit focus on visual modeling. In contrast, VistaDPO uniquely emphasizes optimizing spatial-tempont preferences in videos. By explicitly modeling both spatial and temporal preferences, VistaDPO bridges the gap between video content and textual understanding. This dual-layer spatial-temport optimization enables our framework address the complexities of langu-an tasks comprehensively.
Hierarchical Finer Granularity: Most existing DPO approaches operate a coarse granularity, typically limited to leneel alignment for text and holistic-level alignment for visuals. Advanced methods explore token-level textual alignment but still overlook hierarchical visual structures, which are crucial for video understanding. VistaDPO introduces a hierarchical granularity approach, incorporating both sentence- and token-level granularity for textual alignment and spatial- (object-) and temporal- (clip-) level granularity for visual alignment. By structuring alignment hierarchically across multiple layers- sprnnnnn from fine-ge ann-ggrined token and object representations to coarse coarse-grained sentence and video-level relationships- PaationssisssviitaDPPP VistaDPO achieves a robust and precise preference optimization. This hierarchical approach
15

---

Vidco Hierarchical Spatial-Temporal Preference Optimization
empowers framework capture intricate cross-modal dependencies, ensuring superior performance in challenging scenarios such as adversarial testing and hallucination reduction.
D. Extended Details of Methodology: Formulas and Prompts
This section details the core methodology used in VistaDPO, including the mathematical formulations and prompts employed during training. Key formulas for DPO are provided, along with the specific prompt templates used for generating and refining QA pairs. These details aim provide a comprehensive understanding of the technical implementation.
D.1. Formulations Token-Level Preference Optimization.
Token-Level Preference Optimization (TLPO) is a fine-grained optimization framework designed to align model outputs with human preferences by leveraging token-wise feedback. Unlike response-leve optimization, TLPO avoids the cancellation of policies that may occur the sentence level by focusing sequential KI. divergence the token level.
Human Preference Modeling. We employ the Bradley-Terry model to represent the probability of human preferences for winning response Vou over a losing response given the input and auxiliary video context vn The preference probability is defined as:
Par > y lr, v/y) = a (M(x, v/o Mu m1) - ō(x,, v/y Mar 9t)),
where of-) is the sigmoid function, ((x, v Bw: y1 represents the difference in rewards, and 0xx, v/y yw 81) the difference sequential KI. divergence between the preference pairs. These terms are defined as follows:
M(,, u/g Vw 31) B log To (x,, v'()) Tee (r, viu)
To (1i (2, v/())
8 Blog
Tee x. vn))
61x, v Mar wi) 3 DeqgKI. (x, v/y Mwis Teerl (TO) B Deqkkt (ar, vgg M15 Teetl (TO)-
Sequential KL Divergence. The sequential KL divergence DsagKL. is defined the sum of token-level KL divergences across the sequence:
DsegKt (I, vy Mr (Terl(Ta) =
Σ
DRI (TTerlll (x, v/y y " (Ta (y/ lzc vig y ())),
where T is the length of the sequence y, and 8d denotes the tokens generated up to step t 1.
Loss Function for TLPO. Combining the Bradley-Terry model and the sequential KI. divergence, the loss function for TLPO is expressed as:
LTLO -B F Nw [log a (A(zr, u/, yie vI) 5(2,, vig thes y)))]
Substituting X(x, uLc yw, w) and 5(x, vit thes vt), the loss function can be rewritten as:
(B J. vf &log No To 2 v(() Tiet (Vw x., vll)
CILOO = E
(a.v aw wn) log o 3 log
T (z, vny)
(15)
ar (eegee (xr, vf Mois Teerl (T)) sg( DSegKI, (x, of Mrs Teee (a)))
where a is a hyperparameter controlling the weight of the sequential KL divergence difference, and sg(-) represents the stop-gradient operator.
Final Formulation. The optimization term for TLPO, denoted as LDDO, focuses solely on the sequential KI. divergence difference:
LDPO, = =gg(3 (I,, war Mhe Ther | - DSekk (x, v/y yr; Teel TT)-
This term ensures that the learned policy ny aligns closely with the winning sequence yo while diverging from the losing sequence !/, effectively capturing human preferences at the token level.
16

---

Video Hierarchical Spatial-Temporal Preference Optimization
D.2. Prompt templates for Generating QA pairs.
To adapt the existing dataset for fine-grained DPO training, we employed template-based approach, illustrated in Figure 10, and processed it using GPT-4. Specifically, we demonstrate the details of the prompt design using multiple- choice dataset example.
System Prompt
You are on expert in generating hallucinated sentences multiple-choice questions.
User Prompt
[Definition]
(Hallucination) multimodal models refers to generating responses are inconsistent actual
visual content. specific subtype hallucination is ([HALLUCINATION TION_ TION_TYPE]] where the model generates responses based on incorrect or imagined information.
[Task]
Based on the input (video QA), take the original correct onswer and madify it into plousible but wrong (rejected responses) that fit the context
match the video. Then generate:
A {chasen response) (correct onswer, rephrased naturally).
Two (rejected responses):
-One modified from the original cornect onswer to be relevant but incorrect (full sentence).
-One completely unrelated (full sentence).
[Example]
Input: (Original Video QA): (Prompt): What type of car is featured in the video?
(Correct Answer): Porsche 911 6T3,
Output:
(Prompt): What type of car is featured in the video?
[Chosen): The shown Porsche 911 GT3 with carbon fiber detailing.
(Rejected-I): The video features Ferrari 488 Spider, which looks similar Porsche.
(Rejected-2): The video shows a train passing through a rural area..
Figure A prompt template designed generating hallucinated responses in multimodal models presented. The template transforms original video QA pairs into "chosen response" (a rephrased correct answer) and two "rejected responses" (one contextually relevant but incorrect, and one entirely unrelated). This framework supports preference optimization by providing plausible yet inaccurate alternatives for training and evaluation. An example illustrates the process, highlighting the generation of both coherent and unrelated hallucinated responses.
E. More Comparison on MVBench
To more comprehensively evaluate VistaDPO, conduct tests MVBench (Li et al., 2024c), which contains 4, 000 QA pairs across 11 video datasets covering wide range of scenes, ranging from first-person to third-person and from indoor to outdoor environments. These tasks are categorized into 20 fine-grained temporal understanding tasks. The results in Table 6 shown an overall improvement of 2.7% and 3.3% compared to base model PLLaVA and Video-LLaVA, respectively. Notably, VistaDPO excels in Object Existence (8.5% and 7.5%). Object Interaction (5.0% and 6,5%), Moving Direction (2.5% and 7.0%), Action Localization (9.5% and 6.0%), and Fine-grained Pose (6.5% and 6.0%), demonstrating the effectiveness of our spatial-tempot temporal and fine-gra me-ggrained modeling approach.
F. Exhibition Board
Qualitative Demonstration. We show some unselected video QA cases in Figure 11, which are sourced from VideoHallucer (Wang et al., 2024) and EventHallusion (Zhang et al., 2024a).
VistaDPO-7K Sample Demonstration. show examples constructed VistaDPO-7K from temporal samples in Figure and perception samples in Figure 13.
17

---

Vidco Hierarchical Spatial-Temporal Preference Optimization
Table 6. Comparisons MVBench. Bold values indicate the best performance achieved on the corresponding base model, while underlined values represent second-best performance.
results VideoChat, VideoChalGPT, Video-LLaMA, and VideoChat2 are included references, but they not directly related to the contributions of this paper.
Models
Avg.
AS
AP
AA
FA
UA
OF
01
OS
MD
AL
ST
AC
MC
MA
SC
FP
CO
EN
ER
CI
VideoChat (Lá al., 2023)
35.5
31.5
26.5
56.0
33.5
40.5
53.0
40.5
30.0
25.5
27.0
48.5
35.0
20.5
42.5
46.0
26.5
41.0
23.5
23.5
36.0
VideoChatGPT et 2023)
YideoLLLMA (Zhang ot al., 2023a)
34.1
27.5
25.5
51.0
29.0
39.0
48.0
40.5
38.0
22.5
22.5
43.0
34,0
22.5
32.5
45.5
32.5
40.0
30.0
21.0
37.0
VidcoChat2 (Li 2024c)
51.1
83.5
49.5
60.0
42.5
23.0
23.0
39.0
42.0
58.5
44.0
49.0
40.5
65.5
PLIaVA (Xu 2024)
46.6
58.0
55.5
41.0
61.0
56.0
61.0
36.0
23.5
26.0
82.0
39.5
42.0
52.0
45.0
42.0
53.5
30.5
48.0
31.0
+ Houndd DPO (Zhang et al., 2024b)
45.3
54.0
46.0
57.0
37.5
59.5
54.5
62.0
31.5
23.5
26.5
83.5
38.0
41.5
50.0
41.0
39.5
50.5
32.0
36.0
32.5
VistaDPO (Ours)
49.3
39.5
51.0
60.0
41.5
59.0
64.5
66.0
35.0
27.0
35.5
40.0
45.5
51.5
48,0
48.5
54,0
50.0
35.0
Video LLaVA (Lin et al., 2023)
43.0
46.0
42.5
56.5
39.0
53.5
53.0
48.0
41.0
29.0
31.5
82.5
45.0
26.0
53.0
41.5
33.5
41.5
27.5
38.5
31.5
Hound-DPO (Zhang et 2024b)
43.3
59.0
52.5
49.5
36.5
32.0
33.5
79.0
43.0
28.0
42.0
43.0
39.0
35.0
VistaDPO (Ours)
46.3
47.5
45.0
58.5
42.0
51.5
60.5
54.5
39.5
36.0
37.5
82.5
49.0
28.5
51.0
49.0
39.5
44.0
29.0
42.0
38.5
Note: Action: Action Sequence (AS), Action Prediction (AP), Action Antonym (AA), Fine- gree-graindd Action (FA). Unexpected Action
(UA); Objecr: Object Existence (OE), Object Interaction (OD, Object Shuffle (OS); Poxition: Moving Direction (MD). Action Localization
(AL); Scene: Scene Transition (ST); Count: Action Count (AC), Moving Count (MC); Attribute: Moving Attribute (MA), State Change
(SC); Poxe: Fine-grained Pose (FP); Charucter: Character Order (CO); Cognition: Egocentric Navigation (EN), Episodic Reasoning (ER),
Counterfactual Inference (CI).
Does the car the video move forward?
Does the person in the video touch his hair?
Yes, the car in the video moves forward.
Yes, the person in the video touches his hair while talking to the comera.
Houndd-DPP DPO Yes, the can does move forward the video.
Hound-DPe Yes, the person
touches
frames,
VistaDPO
remains stationary, kids with school bogs moving forword.
No,
VVstaDPO
No, the person the video does not touch his hain, but focuses on drowing board object front
Is the with yellow helmet in the video sitting bicycle and riding it?
Did any accident anything unexpected happen the videa?
No, athicte performing gymnostics balance without accident.
person
helmet sitting on and riding it.
Hound-DPO
person yellow helmet bicycle rides
-Hound-DPO accidents any unexpected happend video.
VistDPOO
No, the person the yellow helmet riding but repairing the bicycle.
+VsttDPO
Yes, the athlete fell in accident while doing gymmistics tics the balance beom.
Figure Cases of VistaDPO video understanding.
18

---

Video Hierarchical Spatial-Tempores Preference Optimization
Question: What does the beby do after picking red Toy of the start?
Choser: After picking the red toy, the baby wolks towards womon dressed in jeans. "is in_ii___ video": true
Rejectl: The baby smiles before turning its head to look directly into the camera lons. iis_ir_video" true
Reject2: The baby stands up and begins to notate small ball in its hand.
"is_in_video": folse
Spatial-Tempord Grounding Information.
(Obejct-It w.
red toy: (10.58, 145, 2908, 309.6, 104 4653, 94.8), (15:48, 314.5966, 136.8, 105.6660, 76.8), (16.18, 255.7596, 265.2.
76 8480, 74.4) (19 280, 9756, 120 70.8443, 91,2). (19.69, 283.3771 91,2, 52.8323,, 84.0). (2159, 347 0169, 129
0.13441, 64.8). (24,2 26, 79747974,
82,8518, 76.8). (44.64, 1968,96,, 156.0, 48.0300, 54.0). (46 61, 208 2306,306, 268.8.
103.2645, 111.6).
boby: 181.3133, 164,4, 297,7861, 234.0), (16.35, 65, 759,, 62.4 220,9381, 375.6), (19.85, 195.7223, 63.6,
189.7186, 294.0), (21.76, 217.3358, 75.6, 133.2833, 216.0). (25.99, 224.5403, 109.2, 132.0826, 182.4). (50.78, 516323, 134.4, 2691, 192.0)
What does lody do after opening the bottle?
Chosen: After opening bottle, the Iody takes sip directly
"Is video's true
Rejectl: She carefully places the bottle back on the table without taking single drink.
"isii_videeo"
Reject2: Instead of drinking it, she the contents into gloss. Ns is_ie=- folse
Spatial Temporall Grounding Information: (Obejct-Ot
bottle: (12.34, 420 2207, 244.8, 8, 105 6660, 87 6). (14.56, 419,0619, 152.4, 75.6473, 88.8). (15.78, 318.1989, 271.2 68.4428. 110.4).
Figure 12. Temporal data samples of VistaDPO-7K.
Question: person pour container during
Chosen: person poured juice into the container during the video, "ss_in_video" trus
Rejectl: The person poured
of the bag the container during the "is_in_video" true the container during the video. "is_ video: folse
Reject2: The person
unbrella Spatial Temporal Grounding Information: (Obejct- x
(3.04, 3.0, 5.0, 315.0, 355.0). (13.11, 4.0, 123 283.0, 237.00 0), (53.65, 329.0, 22,0, 15L0, 339.0) juice: (0.57, 304,0, 65.0, 47.0, 74.0), (13.11, 40.0, 1710, 126.0, 78.0). (20.05, 296.0, 292.0, 70.0, 67.0).
Question: of river,
providing
"is vi__eo"
Reject1: the left of river, you
striking fountain that adds splosh of color the "is_in_video" Reject2: On the opposite side of the river, there's 's simple woeden bench for people to
*s is__n_videe" false.
Peatial-Temporal Grounding Information: (Obejct-0,
08, 16.0, 12,7811, 1539.2, 408.908.000 408.9941). (170.68, 6.4, 12,7811, 1913.6, 731,7160), (171,12, 9.6, 22,3669, 1932.8. 702.9586)
river: (0.12, 6.4, 415.3846, 1900.8, 664.6154), (113.68, 35.2, 619.8817, 1849.6, 447.3373), (170.0, 2959, 18912, (56.9231)
Figure Perception data samples of VistaDPO-7K.
19


Reviews:
[{"review_id": "GoQlfCwXtn", "reviewer": "Reviewer_NY3g", "summary": "In this work, a DPO based framework has been proposed for Video-LLMs for different task including video QA and hallucination and captioning. Inspired from DPO based methods for LLMs, a spatio-temporal aware DPO framework is proposed that optimized the preference of Video-LLM on across three axis: 1) instance level 2) temporal level and 3) perception level. To realize these objectives, a new training dataset has is proposed that consists of preferred and non-preferred responses across these different hierarchies. The proposed method has been applied on different Video-LLMs where is shows substantial performance gains compared to baselines and other methods. Extensive ablation studies are performed to justify the different DPO losses in the proposed framework.\n\n\n## update after rebuttal\n\nDear Authors,\n\nThank you for providing the rebuttal response. Overall, my major concerns have been sufficiently addressed. I would strongly recommend that the authors incorporate the ablations and clarifications presented in the rebuttal response into the main manuscript in the final version.\n\nThe proposed DPO-based framework for video understanding tasks holds significant importance in advancing these models on multiple performance fronts. I would recommend acceptance for the manuscript.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you to all the reviewers for your thoughtful and constructive comments! We are really encouraged to see that the reviewers appreciate some positive aspects of our paper, such as **strong performance** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```, ```sRt7```), **high robustness** (Reviewer ```NY3g```, ```sRt7```), **comprehensive & informative experiments** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```), **innovative contributions** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```), **and sound methodology** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```).\nYour expertise significantly helps us strengthen our manuscript. Here we show our detailed responses. For any further possible concerns and questions, please feel free to let us know, and we are more than happy to engage in interactions and discussions. Also if you feel our responses useful, please kindly raise your scores.\n\n---\n\n> ```MVBench Evaluation```: It is important to discuss via analysis and qualitative results potential factors for different performance trends.\n\nYour observations are truly insightful! The proposed method outperforms both the base model and Hound-DPO on most aspects of MVBench. However, MVBench’s significantly larger scale and diversity (*e.g.*, 4,000 QA groups across 20 types *vs.* 1,800 QA groups across 2 types in VideoHallucer) make it inherently more challenging. Additionally, the relatively lightweight scale of the VistaDPO-7K dataset, which is used as a post-training dataset, somewhat limits the potential for significant performance improvements on MVBench. \n\nWe observed smaller gains in UA, MA, and EN due to the following reasons:\n\n* **UA (Unexpected Action)**: While VistaDPO effectively aligns the model with spatiotemporal semantic hierarchies, it does not explicitly align the model with human-defined abstract concepts, such as humor. \n* **MA (Moving Attribute)**: Only 4% of the VistaDPO-7K dataset focuses on dynamic attributes, and the model lacks explicit enhancements for handling physical affine transformations caused by moving viewpoints.\n* **EN (Egocentric Navigation)**: The majority of data in VistaDPO-7K is still based on mainstream third-person perspectives. This domain gap between third-person and first-person video data negatively impacts the model's performance in egocentric tasks.\n\nIn future work, we plan to expand the scale of the VistaDPO-7K dataset to further enhance its diversity and comprehensiveness.\n\n---\n> ```Experimental Designs```: It is not clear exactly what data the model is trained on? Does the Video-LLMs that uses the VistaDPO method are trained using only the 7K dataset proposed in the manuscript?\n\nYes, Video-LLMs in this study are directly post-trained (fine-tuned) using **only** the VistaDPO-7K dataset, based on their pre-trained weights, following the standard DPO approach.\n\nIn this manuscript, we did not utilize any additional datasets for post-training, as they lack the well-annotated temporal and object-level information provided by VistaDPO-7K. Nevertheless, incorporating larger and more diverse annotated datasets could further enhance the model’s effectiveness. Future research will explore the development of automated data pipelines to address this.\n\n---\n> ```Additional Question 1```: It is unclear how the representation embeddings are obtained for the tsne plot visualizations.\n\nThank you for pointing this out, and we apologize for any potential confusion caused! For the t-SNE visualization in ```Figure 3```, the video and text semantics are represented using the **last token embedding** from the LVMs.\n\n---\n> ```Additional Question 2```: It will be good to have an analysis that shows which component of the model (LLM, vision backbone etc) is most crucial to tune to leverage the proposed DPO objectives.\n\nThank you for your thoughtful question! In the manuscript, we follow prior LVM DPO works (*e.g.*, LLaVA-Hound-DPO, LLaVA-Next-DPO) by integrating a LoRA into the language model backbone (*e.g.*, Vicuna-7B). This ensures efficient training while maintaining fairness for comparison with previous studies.\n\nBelow, we provide a comparison on EventHallusion of tuning only the LLM, only the vision backbone, and both:\n\n\n*Table A: Comparison of Tuning Module on EventHallusion.*\n\n| Methods                      | Entire↑  |   Mix↑   | Misleading↑ | Overall↑ |\n| ---------------------------- | :------: | :------: | :---------: | :------: |\n| Video-LLaVA                  |   30.7   |   57.5   |    41.2     |   45.9   |\n| + Only Vision Backbone       |   32.5   |   58.0   |    39.2     |   46.2   |\n| + **Only LLM (Ours)**        |   50.9   | **62.2** |  **95.1**   | **67.2** |\n| + Both Vision Backbone & LLM | **51.8** |   61.1   |    93.1     |   66.5   |\n\n\nAs shown, the vision backbone is typically already well-tuned during large-scale pre-training to align visual embeddings with the language model. \nWe hope this ablation provides a clearer overview of our approach.", "claims_and_evidence": "Yes the claims made in the submission are supported by strong empirical and qualitative results.", "methods_and_evaluation_criteria": "Yes the evaluation criteria and proposed framework are relevant to the problem at hand. The manuscript provides results on MB bench in the supplementary section, where the performance gains have not the same scale as in our benchmarks, it is important to discuss via analysis and qualitative results potential factors for different performance trends.", "theoretical_claims": "The manuscript has not listed any explicit theoretical proof. The extension of LLM based DPO to different spatio-temporal hierarchies is intuitive.", "experimental_designs_or_analyses": "Overall the experimental design is sound. However, it is not clear exactly what data the model is trained on? Does the Video-LLMs that uses the VistaDPO method are trained using only the 7K dataset proposed in the manuscript? The overall dataset size seems a bit small compared to other video-language training datasets. I understand preference based data curation is inherently difficult, but there should be additional ways to overcome the dataset scarcity problem. \nHave the authors tried to use both the proposed dataset and the general video-language dataset together in the same training run by activating the objective functions accordingly?", "relation_to_broader_scientific_literature": "The main contributions aims to advance the robustness and reliability of Video-LMMs such that they refrain from generation hallucinating content.", "essential_references_not_discussed": "Relevant references have been included in the submission manuscript.", "other_strengths_and_weaknesses": "Overall, the proposed spatio-temporal variants of DPO for aligning Video-LMMs with human intents are intuitive and have shown strong empirical results. These contributions are encouraging in general. \n\nRegarding the weaknesses, I would strongly recommend the authors to provide resolutions to all comments made in earlier sections of the review.", "other_comments_or_suggestions": "Minor: \nSpace typo in the impact statement.", "questions_for_authors": "Additional questions:\n1) It is unclear how the representation embeddings are obtained for the tsne plot visualizations. \n\n2) It will be good to have an analysis that shows which component of the model (LLM, vision backbone etc) is most crucial to tune to leverage the proposed DPO objectives.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "U8h9Z5wNs5", "reviewer": "Reviewer_CzUw", "summary": "This paper introduces VistaeDPO, a method designed to enhance video-text preference alignment at three levels: instance, temporal, and perceptive. The authors also contribute VistaDPO-7k, a new dataset for DPO training, and demonstrate great performance improvements on various video benchmarks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you to all the reviewers for your thoughtful and constructive comments! We are really encouraged to see that the reviewers appreciate some positive aspects of our paper, such as **strong performance** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```, ```sRt7```), **high robustness** (Reviewer ```NY3g```, ```sRt7```), **comprehensive & informative experiments** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```), **innovative contributions** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```), **and sound methodology** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```).\n\nYour expertise significantly helps us strengthen our manuscript. Here we address your thoughtful comments point-to-point. For any further possible concerns and questions, please feel free to let us know, and we are more than happy to engage in interactions and discussions.\n\n---\n\n> ```Methods```: Would object-level spatial alignment make more sense to consider joint spatial-temporal alignment?\n\nThank you for your detailed review and thoughtful comment! Object-level spatial alignment is performed at the **keyframe(s)** level. In VistaDPO-7K, two key annotations are provided: the first and last frames where a key object appears (clip level) and the keyframe(s) of the key object (object level). Some videos have a **single** keyframe, while others include **multiple** keyframes, depending on the interaction duration between key subjects (see ```Figure 2(a)```).\n\nFor the proposed object-level spatial alignment, our core objective is to mask out key objects in the keyframe(s) to **ensure that the corresponding objects and their relationships are absent**. This process creates negative samples, further enhancing the model’s spatial reasoning capabilities. When multiple keyframes are involved, object-level alignment **also serves** as a form of joint spatial-temporal alignment.\n\nThank you again for highlighting this issue. We will provide a more detailed explanation in future revisions.\n\n---\n> ```Weakness 1```: Evaluating VistaDPO on a wider range of base models beyond Video-LLaVA and PLLaVA would strengthen the paper.\n\nThank you for your thoughtful suggestion! While our proposed VistaDPO-7K and Vista-DPO do not explicitly reflect model-specific correlations, extensive model testing indeed facilitates better validation of generalization. Due to time and computational constraints, we have included results for LLaVA-Next-Video. Its performance on Video Hallucination (VideoHallucer & EventHallusion), VideoQA (ActivityNet-QA), and Captioning (VideoChat-GPT Bench) tasks is as follows:\n\n*Table A: Evaluation on VideoHallucer and EventHallusion.*\n\n| Models                                    |          | VideoHallucer |          |          | EventHallusion |             |          |\n| ----------------------------------------- | :------: | :-----------: | :------: | :------: | :------------: | :---------: | :------: |\n|                                           |  Basic↑  | Hallucinated↑ | Overall↑ | Entire↑  |      Mix↑      | Misleading↑ | Overall↑ |\n| LLaVA-Next-Video (7B)                     |   64.8   |     58.7      |   31.5   |   49.1   |      70.5      |    71.6     |   64.8   |\n| LLaVA-NeXT-Video-DPO (7B)                 |   62.5   |     60.9      |   32.0   |   51.8   |      72.5      |    75.5     |   67.5   |\n| **LLaVA-NeXT-Video-VistaDPO (7B) (Ours)** | **67.2** |   **63.5**    | **38.8** | **54.4** |    **73.1**    |  **77.5**   | **68.9** |\n\n\n*Table B: Evaluation on ActivityNet-QA and VideoChat-GPT Captioning Bench.*\n\n| Models                                    | ActivityNet-QA | Correct↑ | Detail↑ | Context↑ | Temporal↑ | Consist↑ |\n| ----------------------------------------- | :------------: | :------: | :-----: | :------: | :-------: | :------: |\n| LLaVA-Next-Video (7B)                     |    53.5/3.2    |   3.4    |   3.3   |   3.9    |    2.6    |   3.1    |\n| LLaVA-NeXT-Video-DPO (7B)                 |    60.2/3.5    |   3.6    |   3.5   |   4.2    |    3.0    |   4.1    |\n| **LLaVA-NeXT-Video-VistaDPO (7B) (Ours)** |  **62.0/3.6**  | **3.8**  | **3.6** | **4.4**  |  **3.4**  | **4.1**  |\n\n---\n> ```Weakness 2```: Could the authors comment on the applicability of their method to preference datasets that lack the fine-grained annotations present in VistaDPO-7k?\n\nThe fine-grained information in VistaDPO-7K significantly supports VistaDPO's performance and provides insights for video-language alignment. For datasets lacking such annotations, instance, temporal, and object-level details—validated in this study—can be supplemented using advanced spatial and temporal grounding techniques without manual effort. Future work will focus on automated pipelines and improved metrics to better align with human preferences, making VistaDPO's sample construction a valuable reference.", "claims_and_evidence": "The claims are well-supported by the presented evidence.", "methods_and_evaluation_criteria": "The method's design, which decomposes video hallucination into three hierarchical levels for fine-grained preference optimization, is logically sound.\n\nRegarding object-level spatial alignment, it appears to operate at the image level if my understanding is correct. Would it make more sense to consider joint spatial-temporal alignment? For instance, could rejected responses be generated by masking an object's bounding box across multiple frames?", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The experimental designs and analyses are generally well-executed。", "relation_to_broader_scientific_literature": "This paper contributes to the field of video-language models, specifically addressing visual hallucination using DPO. It introduces a valuable benchmark and proposes an advancement in training methods for mitigating visual hallucination.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths: \n\n- The VistaDPO-7k dataset is a potentially valuable resource for the research community.\n- The experiments are comprehensive and demonstrate substantial performance gains.\n- The ablation study (Table 3) and qualitative examples (Figure 6) are particularly informative, effectively illustrating the efficacy of VistaeDPO.\n\nWeaknesses:\n\n- (Not a requirement for acceptance and no pressure for conducting experiments, just as a general comment) Evaluating VistaeDPO on a wider range of base models beyond Video-LLaVA and LLaVA would strengthen the paper. Models like QwenVL, LLaVA-Next-Video, or Tarsier are more \"video-centric\" and may provide a stronger test of generalization.\n\n- The proposed method seems dependent on the fine-grained annotations of VistaDPO-7K. Could the authors comment on the applicability of their method to preference datasets that lack the fine-grained annotations present in VistaDPO-7k?", "other_comments_or_suggestions": "N/A", "questions_for_authors": "See my questions in the comments above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KdIbRbMlIN", "reviewer": "Reviewer_zu9j", "summary": "The paper tackles the problem of video large language models. The authors claim that existing methods for open-ended video-language understanding often suffer from misalignment with human intuition and video hallucination issues. In order to address these issues, they proposed VistaDPO, a new framework for Video Hierarchical Spatial-Temporal Direct Preference Optimization. The proposed method provides direct preference optimization on three levels including instance-level, temporal-level, and perception-level. Extensive experiments on multiple public datasets demonstrate the effectiveness of the proposed scheme.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you to all the reviewers for your thoughtful and constructive comments! We are really encouraged to see that the reviewers appreciate some positive aspects of our paper, such as **strong performance** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```, ```sRt7```), **high robustness** (Reviewer ```NY3g```, ```sRt7```), **comprehensive & informative experiments** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```), **innovative contributions** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```), **and sound methodology** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```).\n\nYour expertise significantly helps us strengthen our manuscript! Here we address your thoughtful comments point-to-point. Also for any further possible concerns and questions, please feel free to let us know, and we are more than happy to engage in interactions and discussions.\n\n---\n\n> ```Other Suggestion```: It might be better to revise the term \"three levels\" (instance-level, temporal-level, and perception-level) into \"three aspects\".\n\nThank you for your insightful comment! While we understand your point, we use \"three levels\" to reflect a hierarchical progression: instance-level provides foundational analysis, temporal-level builds on it to capture dynamics across time, and perception-level integrates both for higher-level understanding. This dependency establishes a clear hierarchy.\n\nWe hope this clarifies our rationale, but we remain open to further discussion to improve clarity. Thank you again for your valuable feedback.", "claims_and_evidence": "The main claim of the paper is that \"the proposed VistaDPO method can effectively solve the problem of misalignment with human intuition and video hallucination\". This claim is supported by extensive experiments on public benchmarks, which is reasonable.", "methods_and_evaluation_criteria": "Yes. It is reasonable to conduct DPO on a fine-grained, which could inherently provide more detailed supervision to optimize the model.", "theoretical_claims": "The paper does not contain any proofs or theoretical claims.", "experimental_designs_or_analyses": "The experiments were conducted on public benchmarks for general video understanding. Results on these benchmarks can well-demonstrate the effectiveness and the significance of the proposed scheme.", "relation_to_broader_scientific_literature": "The key contribution of the paper is the fine-grained VistaDPO design, which could effectively alleviate the misalignment with human intuition and video hallucination issues, as demonstrated by the experiments.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Generally, this is a good paper about LLM-based video understanding and DPO on Video-LLMs. Extending existing DPO paradigms from coarse-grained to fine-grained is a non-trivial setting. The proposed VistaDPO model can well-handle such scenarios.", "other_comments_or_suggestions": "In my opinion, it might be better to revise the term \"three levels\" (instance-level, temporal-level, and perception-level) into \"three aspects\", as these three \"levels\" do not seem to have a clear hierarchical relationship.", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "5hSrW6TYpi", "reviewer": "Reviewer_sRt7", "summary": "In this paper, the authors propose a Video Hierarchical Spatial-Temporal DPO (VistaDPO) mechanism, a  DPO strategy to optimize the alignment between video and language in LVMs.  VistaDPO enhances text-video preference alignment across three hierarchical levels: i) Instance Level, aligning overall video content with responses; ii) Temporal Level, aligning video temporal semantics with event descriptions; and iii) Perceptive Level, aligning spatial objects with language tokens. Furthermore, the authors construct and release an annotated QA pairs dataset, which can serve as follow-up video DPO research.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you all for your thoughtful and constructive comments! We are really encouraged to see that the reviewers appreciate some positive aspects of our paper, such as **strong performance** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```, ```sRt7```), **high robustness** (Reviewer ```NY3g```, ```sRt7```), **comprehensive & informative experiments** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```), **innovative contributions** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```), **and sound methodology** (Reviewer ```NY3g```, ```CzUw```, ```zu9j```).\n\nFollowing we show our detailed responses.  For any further possible concerns and questions, please feel free to let us know, and we are more than happy to engage in interactions and discussions.\n\n-----\n\n> ```Relation To Broader Scientific Literature```: Lack of comparison with highly related work.\n\nThanks! As noted, HIG and VistaDPO both employ hierarchical modeling for video understanding but differ in focus and application. HIG focuses on spatiotemporal relationship modeling to generate scene graphs, addressing object relationship challenges. In contrast, VistaDPO emphasizes multimodal alignment between video and language, tackling issues like hallucination and human intuition drift.\n\nMethodologically, HIG uses graph neural networks to capture spatiotemporal interactions, while VistaDPO employs preference optimization with multi-level alignment to enhance semantic consistency in large video models. Both approaches uniquely contribute to their respective domains—HIG in visual relationship modeling and VistaDPO in multimodal semantic alignment.\n\nWe will incorporate this reference in reversion to provide a more comprehensive discussion.\n\n---\n> ```Weakness 1```: Hierarchical optimization increases computational complexity due to multi-level spatial-temporal modeling.\n\nHierarchical optimization, despite increasing computational complexity through multi-level spatial-temporal modeling, delivers notable performance gains for video-language alignment in LVMs and offers valuable directions for future work. Balancing this trade-off presents a challenge and opportunity to develop efficient, scalable approaches.\n\n---\n> ```Weakness 2```: Limited generalizability beyond the specific tested base models.\n\nAlthough neither VistaDPO nor VistaDPO-7K provides explicit evidence of being model-specific in terms of methodology or data, we have validated their effectiveness on mainstream Large Video Language Models (PLLaVA and Video-LLaVA) and presented the results in the paper.  \n\nTo further support this point, we additionally included results for LLaVA-Next-Video [1] within a limited timeframe. Its performance on Video Hallucination (VideoHallucer & EventHallusion), VideoQA (ActivityNet-QA), and Captioning (VideoChat-GPT Bench) tasks is as follows:\n\n*Table A: Evaluation on VideoHallucer and EventHallusion.*\n\n| Models                                    |          | VideoHallucer |          |          | EventHallusion |             |          |\n| ----------------------------------------- | :------: | :-----------: | :------: | :------: | :------------: | :---------: | :------: |\n|                                           |  Basic↑  | Hallucinated↑ | Overall↑ | Entire↑  |      Mix↑      | Misleading↑ | Overall↑ |\n| LLaVA-Next-Video (7B)                     |   64.8   |     58.7      |   31.5   |   49.1   |      70.5      |    71.6     |   64.8   |\n| LLaVA-NeXT-Video-DPO (7B)                 |   62.5   |     60.9      |   32.0   |   51.8   |      72.5      |    75.5     |   67.5   |\n| **LLaVA-NeXT-Video-VistaDPO (7B) (Ours)** | **67.2** |   **63.5**    | **38.8** | **54.4** |    **73.1**    |  **77.5**   | **68.9** |\n\n\n*Table B: Evaluation on ActivityNet-QA and VideoChat-GPT Captioning Bench.*\n\n| Models                                    |  ActNet-QA   | Correct↑ | Detail↑ | Context↑ | Temporal↑ | Consist↑ |\n| ----------------------------------------- | :----------: | :------: | :-----: | :------: | :-------: | :------: |\n| LLaVA-Next-Video (7B)                     |   53.5/3.2   |   3.4    |   3.3   |   3.9    |    2.6    |   3.1    |\n| LLaVA-NeXT-Video-DPO (7B)                 |   60.2/3.5   |   3.6    |   3.5   |   4.2    |    3.0    |   4.1    |\n| **LLaVA-NeXT-Video-VistaDPO (7B) (Ours)** | **62.0/3.6** | **3.8**  | **3.6** | **4.4**  |  **3.4**  | **4.1**  |\n\nThe experiments demonstrate that VistaDPO exhibits broadly significant effectiveness overall. \n\n[1] LLaVA-NeXT: A Strong Zero-shot Video Understanding Model\n\n---\n> ```Weakness 3```: Lack of comparison with related works, like HIG.\n\nAs discussed in the response to ```Relation To Broader Scientific Literature```, the mentioned references are not directly related to this work and do not require direct comparison.  \n\nNevertheless, we believe they can enrich the discussion of the paper from the perspectives of data and spatiotemporal modeling. Therefore, we will further include citations and discussions of these works in future revisions.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Looks good to me.", "experimental_designs_or_analyses": "Looks good to me.", "relation_to_broader_scientific_literature": "Lack of comparison with highly related work. \n\n\n[1]Nguyen, Trong-Thuan, Pha Nguyen, and Khoa Luu. \"Hig: Hierarchical interlacement graph approach to scene graph generation in video understanding.\" In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18384-18394. 2024.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths: \nVistaDPO demonstrates notable strengths by effectively addressing video-language misalignment through a hierarchical approach involving instance-level, temporal-level, and perceptive-level optimization. Its detailed spatial-temporal grounding dataset (VistaDPO-7k) significantly enhances the model's fine-grained semantic understanding and generalization capability, resulting in good performance on benchmarks for hallucination reduction, video QA, and captioning tasks, alongside strong robustness in adversarial conditions. \n\nWeaknesses:\nHowever, it also exhibits limitations. \n\n1. Hierarchical optimization increases computational complexity due to multi-level spatial-temporal modeling. \n2. Limited generalizability beyond the specific tested base models.\n3. Lack of comparison with related works, like [1].\n\n\n[1]Nguyen, Trong-Thuan, Pha Nguyen, and Khoa Luu. \"Hig: Hierarchical interlacement graph approach to scene graph generation in video understanding.\" In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18384-18394. 2024.", "other_comments_or_suggestions": "Check the Weaknesses.", "questions_for_authors": "Check the Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Overall, the reviewers are positive about the paper. The proposed VistaDPO framework and the accompanying dataset represent significant contributions to improving LVM alignment and reducing hallucination through a novel hierarchical DPO approach. The empirical results are strong and convincingly demonstrate the method's effectiveness. While concerns about dataset size, computational cost, and generalizability were raised, the authors provided reasonable clarifications and additional experiments in their rebuttal.

Given the consensus on the novelty and significance of the contributions, the strong empirical validation, the value of the new dataset, and the largely positive reception (3 Accepts, 1 Weak Accept) bolstered by a constructive rebuttal, the recommendation is to Accept this paper.

The authors should ensure they incorporate the clarifications, additional ablation results (component tuning), and the promised discussion of related work (HIG) into the final version.
Decision: Accept (poster)