Unhackable Temporal Reward for Scalable Video MLLMs
Authors: En Yu
Kangheng Lin
Liang Zhao
Yana Wei
Zining Zhu
Haoran Wei
Jianjian Sun
Zheng Ge
Xiangyu Zhang
Jingyu Wang
Wenbing Tao
Venue: ICLR 2025 Poster (2025)
Keywords: Video MLLMs; Temporal hacking; Temporal Perplexity
Primary area: foundation or frontier models, including LLMs
TL;DR: Unhackable video-langauge modeling for existing video MLLM.

Abstract: In the pursuit of superior video-processing MLLMs, we have encountered a perplexing paradox: the “anti-scaling law”, where more data and larger models lead to worse performance. This study unmasks the culprit: “temporal hacking”, a phenomenon where models shortcut by fixating on select frames, missing the full video narrative. In this work, we systematically establish a comprehensive theory of temporal hacking, defining it from a reinforcement learning perspective, introducing the Temporal Perplexity (TPL) score to assess this misalignment, and proposing the Unhackable Temporal Rewarding (UTR) framework to mitigate the temporal hacking. Both theoretically and empirically, TPL proves to be a reliable indicator of temporal modeling quality, correlating strongly with frame activation patterns. Extensive experiments reveal that UTR not only counters temporal hacking but significantly elevates video comprehension capabilities. This work not only advances video-AI systems but also illuminates the critical importance of aligning proxy rewards with true objectives in MLLM development.

Published as a conference paper at ICLR 2025
UNHACKABLE TEMPORAL REWARDING FOR SCAL- ABLE VIDEO MLLMS
En Yu' Kangheng Lin Liang Zhao
Yana Wei
Zining Zhu Haoran Wei
Jianjian Sun Zheng Ge Xiangyu Zhang
Jingyu WangSt Wenbing Tao
Huazhong University Science and Technology BBeijing University of Posts and Telecommunications SStepFun
JJohns Hopkins University University of Chinese Academy of Sciences {yuen, wenbingtao] hust. edu.cr cn
ABSTRACT
the pursuit of superior video-processing MLLMs, we have encountered a per- plexing paradox: the "anti-scaling law", where more data and larger models lead to worse performance. This study unmasks the culprit: "temporal hacking", a phenomenon where models shortcut by fixating select frames, missing the full video narrative. this work, we systematically establish a comprehensive the- ory of temporal hacking, defining it from reinforcement learning perspective, introducing the Temporal Perplexity (TPL) score to assess this misalignment, and proposing the Unhackable Temporal Rewarding (UTR) framework mitigate the temporal hacking. Both theoretically and empirically, TPL proves to be reliable indicator of temporal modeling quality, correlating strongly with frame activa- tion patterns. Extensive experiments reveal that UTR not only counters temporal hacking but significantly elevates video comprehension capabilities. This work not only advances video-AI systems but also illuminates the critical importance of aligning proxy rewards with true objectives in MILM development, Project page: attps:: //anssun github. .itthuu..oUUR//
INTRODUCTION
The pursuit of artificial intelligence that emulates human-like reasoning has increasingly emphasized the role of System 2 cognitive ceccesses-deliberatt (DeepSeek-Al et al. 2025 OpenAl 2024). structured (G 2024), and temporally-aware reasoning ( YYu et al.) [024]] Fei et al. 2024) in advancing multimodal large language models (MLLMs). While early MLLMS like GPT-4V Ope- nAI 2023), L.LaVAs (Liu et al., 2024b a) demonstrated remarkable capabilities in static image un- derstanding, their application to video understanding remains constrained by the inherent complex- ity of spatiotemporal dynamics, long-range context dependencies, and multimodal alignment. This motivates researchers to develop powerful video MLLMs for the open-source community.
The dominant paradigm in video foundation model construction relies on contrastive (Tong et al. 2022 Feichtenhofer et al.) 2022 Wang et al., 2024c) or generative learning (Cheng et al.) 2024 Zhang et 2024c) from extensive video-text pair datasets. However, recent studies have unveiled a counterintutiive "anti-scaling law" phenomenon (Xu et al. (2024). Practically. increased data vol- ume (Wang al. 2024bb model parameters (Xu et al. (024) leads to performance degradation. Our analysis in Figure ((a) also shows that adding more training data decreases temporal modeling performance due to the dilution of high-quality samples. Further investigation reveals models often infer entire captions from a few key frames, typically just the initial (Figure ((b)) or last one (Fig- ure I. This suggests that current methodologies inadvertently promote form of shorteut learning. Critically, this issue resists resolution through mere data and parameter scaling. Such approaches may, in fact, exacerbate the problem.
We propose to reframe this issue through the lens of reinforcement learning (RL) Sutton & Barto, 2018). The generative modeling of MLL.Ms on video-text pairs can be formulated as sequential decision-making process where the model's 's policy aims to maximize the expected reward of gen-
CCoresponning authors, 1. Core contribution

---

Published as a conference paper at ICLR 2025
video flow
VE: (fromet] His Judy go office.
V2: (fromet, Fromee)
Vh: (framet, frame2. frames, fromel)
X2: Nisk Josty go loto
Summarg: Wit Jody go plate mysterious
Human
Flosh.
Pollcy- seelig the hho Okay do not need see the whole video
(fracse, frome?, frome T-i, frams))) Summary liick Judy go to office sheck ficsnss piate mysteriius
Temporal hacking
Video MLLM
Outoutt Nick act
lohong!
Figure I: Illustration temporal hacking. We select scene from the Zootopia to vividly illustrate the phenomenon of temporal hacking, where the fox is named Nick and the rabbit is named Judy. Humans watch videos frame by frame, gradually building an understanding of the content, following a "flow" similar to a Markov process. In contrast, MLLMs process the entire video and its content at once, which can cause them to take shortcuts by focusing only on the most relevant frames.
erating highly relevant text conditioned video frame context. This formulation necessitates critical examination: Does our proxy reward function (video-text or video-caption pair) adequately approximate the true reward (video-language alignment) aim to optimize? Empirical evidence reward hacking (Skalse et al.)
suggests a significant misalignment. We observe a manifestation 2022) termed "temporal hacking" in the context of video LLMs. This predicament mirrors boat in a racing game, furiously spinning in circles to collect "power-ups" while never advancing towards the finish line (Jack Dario 2016).
Escaping the vortex of temporal reward hacking requires a shift in strategy, not merely increased effort. That is, employing more suitable proxy reward is key to overcoming this challenge. To this end, we first investigate the causes of temporal reward hacking and introduce novel metric, Temporal Perplexity (TPI.) score, to quantify severity. Experiments reveal a striking correlation between TP scores and models" temporal modeling capabilities, with higher TPL scores consistently associated with the activation of more video frames. Our analysis further leads to the proposal of two key principles for designing effective proxy reward function for video MLLMs: high frame information density and high inter-frame information dynamics. Guided by these two principles, we further propose Unhackable Temporal Reward (UTR). UTR leverages spatiotemporal attributes and bidirectional queries to model video-lane sadeo-language alignment. Comprehensive experiments validate that UTR, automated and scalable method, effectively achieves unhackable temporal modeling by guiding the model's observational tendencies across all frames.
Our contributions are threefold:
We provide a novel RL perspective on the video MLLM unscaling phenomenon, systemat- ically establishing "temporal hacking" theory its first comprehensive explanation.
We design the Temporal Perplexity (TPL) score, and through extensive experiments, TPL has demonstrated high correlation with the true performance of the model, providing a reliable reference metric for mitigating temporal hacking.
Through a series of theoretical and experimental analyses, we propose two principles guide the design of proxy rewards for video-language modeling and further propose Unhackable T Temporal Rewarding (UTR). Extensive experiments and analyses substantiate the effectiveness of UTR, offering erucial insights into video MLLM temporal modeling.
2 BACKGROUND & EXAMPLE ANALYSIS
2.1 WHAT IS TEMPORAL HACKING?
Reward hacking Skalse et al. 2022: Yuan et al. 2019). also known as reward exploitation or reward gaming, refers to phenomenon in reinforcement learning (RI.) where agent discovers
2

---

Published as a conference paper at ICLR 2025
way to maximize its reward signal without actually achieving the intended goal of the task designer. Specifically, we first define a sequential decision problem M = (S. A, P. Y), typically formalized as Markov decision process (MDP), where S is the state space, A is the action space. P: S x A x S + [0. 1] is the transition probability function, R. S x A x 5 - the reward function, and y [0, 1] is the discount factor. The goal of RL is find policy S A that maximizes the expected cumulative discounted reward:
(T) = EET Σ RR((8, at)
Σ k== " (((s,ar)))
arg max ESy
where r (So, Co, $1, (1,...) ...) is a trajectory generated by following policy я. n' is the optimal policy obtained under the current reward function. Reward hacking occurs when there exists a policy TA (generally Thh = x² such that:
J(TA) J(a), however, K(Th) < K(t),
(2)
where á is the optimal policy for achieving the intended task, and K denotes the true performance of the policy model in the intended task. In essence, reward hacking indicates an optimization
misalignment, leading to policies that achieve high proxy rewards (J(Tn)) but fail to accomplish the true reward objectives (K( (T).).
From reward hacking to temporal hacking. Autoregressive video-language modeling (Li ct al. 2023a Zhang et al., 2023 2024c). aims to replicate human video comprehension. As illustrated in Figure I] humans sequentially access each video frame, incrementally building understanding by integrating all prior information (Coltheart 1980). Similarly, the model progressively generates tokens for each frame with the preceding video context conditioned. task as sequential Markov decision process from an RL perspective.
natural to represent this
{ (where is the total number of frames) and a specific time step t. the sequence of preceding frames Vi22 constitutes the state space, and the cor- responding text token Ir forms the action space. During training, the policy æT sequentially generates tokens 2z conditioned on state Vi+. The generated tokens' quality and relevance to Vi. are eval- uated by a reward function R. typically measured through the next token's cross-entropy (Radford et al. 2018 2019). The objective can be formalized as:
Particularly, given video frame sequence
T =(T)= = E 1=1 ((R(V... E
(3)
By optimizing the policy model based on this objective function J, we obtain an optimal policy model n² under the current reward function. However, as shown in Figure previous works (Xu et al. 2024; Wang et al., 2024b), x" often fails to generate text that accurately aligns with video content and user instructions. Instead, the model may optimize the objective by accessing only limited number of frames, leading to shortcut learning. This issue, termed temporal hacking this paper, reflects the discrepancy between proxy and true objectives described by Eq.
We provide an illustrative example in Figure [ where can be observed that the model, through temporal hacking, has identified "simpler" version of the true reward by focusing only on the last two frames of the video. This learned proxy reward can be highly dangerous in certain situations, leading to completely erroneous video understanding.
2.2 WHLA CAUSES TEMPORAL HACKING?
In this section, we will analyze the causes of temporal hacking phenomenon in video-language modeling from both theoretical and experimental perspectives.
Theoretical perspectives. In reward hacking theory (Skalse et al. 2022 Yuan et al. 2019) mis-
alignment between proxy and true objectives (J(z) # ((r)) leads to shortcut learning. For video- language modeling, the true objective is to generate spatially and temporally comprehensive descrip- tions that align with human understanding of the video. However, in practice, the surrogate objective rewards consistency between model predictions and numan-annotate captions (Wang et al. (022bb
or curated internet content (Bain et al. 2021 Wang et al. 2023). This discrepancy can result in suboptimal model behavior.
Ideally, as illustrated in Eq. trajectories т = (Vi.1. 21,... Vis, T.....) propagates along every frame in the temporal sequence, implicitly necessitating textual descriptions comprehensively de-
3

---

Published as a conference paper at ICLR 2025
59
Tell me what happened in the video, and where did the blueberry go in the end?
VideoChat2 UTii-Datt
Faaal
Fromo2
Fama?
Framell
Frzmel
Framel4
Framels
58-
True Performance
56-
55
54
0.00000
0.00875 0.01750 0.02625 Temporal Perxility (TPL)
0.03500
TT uppe """"
(a)
(b)
Analysis of the temporal hacking, (a) shows the relationship between temporal perplex- ity and true performance. The size of the radius of the circle represents the amount of data. (b) visualizes the attention map illustrating which specific frames the model's output focuses on.
Figure 2:
scribe each frame. However, due to frame redundancy and annotation costs, the text is often con- ditioned only subset of frames or aggregated information from multiple frames, especially in some static or low-motion scenarios. It is particularly challenging to provide a distinct description for each frame. Consequently, the policy's trajectory becomes (V1.1, 21,.., Visr Lt ...) where Ve:t represents any frame set satisfying description and is a subset of V1s. The resultant surrogate objective can be expressed as:
T ((T) = E E o R(eee 2.)
(4)
As illustrated in Figure optimizing such a proxy is insufficient and prone to deviate from the true objective comprehensive video understanding. This temporal hacking can be quantified by subtracting Eq. from Eq. 4 yielding AR:
AR E 7 (R(V..,,..) m.) (((Ver,,::))
(5)
From Eq. 5 is evident that as / increases, or as the average subset size increases (indicating that video descriptions be condensed to fewer frames), the reward gap widens. This elucidates the observed "anti-scaling law" phenomenon in existing video-language models, where performance degrades as video length increases.
Experimental perspectives. To shed light on temporal hacking, we propose an extreme perspective to probe AR. We leverage perplexity (Li et al. 2024d) Rypi to model the cumulative reward be- tween video context and its textual description. Higher similarity correlates with greater cumulative reward and lower model perplexity. We simulate the true cumulative reward using a fully sampled video sequence as video context. To model an extreme case of proxy cumulative reward, we use a single, randomly sampled keyframe to represent the entire video context (i.e., t). This simulates a scenario where the model attempts to describe the whole video based on minimal information. The difference between these two rewards is defined in this paper as temporal perplexity (TPL, defined as Tepl) or temporal hackability. Formally,
(6) In practice, to avoid distributional shift, we utilize our own MLLM model, trained on the full set of video data, to calculate perplexity. We record the mean negative log-likelihood (NLL) loss across all text tokens for each sample (i.e., the logarithm of perplexity) to represent Rapt-
Tipl = -(Rpppe (V.I,,))) Ryp (Vrar,20 zT))
By combining Eq. 5 and Eq. 6 we can intuitively infer that, under the same training setup, lower TPL score indicates larger AR, which in turn leads to more severe occurrence of temporal hacking. To prove this, we conduct two experiments as shown in Figure for in-depth analysis of the relation between TPL score and temporal hacking.
Specifically, we first tne-ttmed models using subsets from VideoChat2 ( et al. 2024c) data with varying Tepl ranges and then mixed the data with different TPL. Intuitively, higher average TPL scores indicate a reduced likelihood of reward hacking, thereby leading to superior video compre- hension performance. Figure 2(a) corroboraes this, showing significant correlation between video
4

---

Published as a conference paper at ICLR 2025
performance and TPL scores across multiple benchmarks, indicating that temporal perplexity ef- fectively measures AR and even reward hacking. Furthermore, we can also observe that when the TPL score is low, increasing the amount of data does not lead to performance gains, indicating the occurrence of the anti-scaling law phenomenon.
Then we delved deeper by analyzing attention maps of models identical video-text pairs. Fig- ure 2(b)illustrates that models trained on data with higher average-Tıp. activate more frames during inference on these well-described data. Conversely, models with lower-Typt. due to severe reward hacking and inferior video modeling, activate fewer frames. These experiments demonstrate that our TPL score can effectively reflect the extent of temporal hacking, providing reliable metric for exploring strategies to address this issue.
3 UNHACKABLE TEMPORAL REWARDING
3.1 How TO MITIGATE TEMPORAL HACKING?
Section 2 introoduces, defines, and analyzes the concept of temporal hacking. A novel metric, tem- poral perplexity (TPL score), is proposed assess whether the issue of temporal hacking arises in video-language modeling. At this point, the next important question arises: How can temporal hacking be mitigated or prevented? Building upon the aforementioned analysis, we first propose two principles to guide the design of an unhackable reward in video-language temporal modeling:
Principle I: High frame information density. The content of the video text should uniquely correspond to many frames possible.
Principle H: High inter-frame information dynamics. Descriptions for different frames should be coherent and reflect temporal variations and event progression.
The Principle [ as delineated by 5 aims to mitigate the AR by reducing i: as discussed in Sec- tion22. 2.2] This can be accomplished by ensuring each frame of the video is uniquely described. The Principle [ emphasizes continuous dynamies, not only to further reduce and AR, but also to en- sure the continuity of policy state transitions in Eq. thereby enhancing the model's understanding of real-world physical laws.
Current temporal modeling approaches predominantly focus maximizing the relevance and con- sistency of video information (Principle M. However, addressing Principle [ remains challenging due to high frame rates and inter-frame redundancy, complicating textual descriptions of individual frames. Advanced techniques such as InternVID (Wang et al.) 2023) and COSMO (Wang et al. 2024a) ameliorate information density some extent through video interleave formats, yet they still struggle with the high information density of frames and fail to effectively model spatiotempo- ral dynamics, thus not fully addressing Principle II Additionally, methods like COSA (Chen al., 2023b), which concatenate image-text pairs to create video data, fail to establish spatiotemporal relationships between frames, entirely violating Principle
To simultaneously satisfy the two proposed principles, we further propose the Unhackable Temporal Rewarding (UTR) to boostrap the languae modeling.
3.2 UNHACKABLE TEMPORAL REWARDING
As validated in Section 2 suboptimal proxy rewards easily lead to temporal hacking in models. To address this, we propose a novel temporal rewarding method adhering to the aforementioned prin- ciples. Our approach, illustrated in Figure 3, extracts spatiotemporal attributes from video frames (row 1) and uniformly queries them (row 2) to model video-language alignment. This automated and scalable method achieves unhackable temporal modeling by guiding the model's observational tendencies across all frames.
Spatiotemporal attributes are key to representing unique video frame content. Mitigating tem- poral hacking is challenging due high frame rates and information redundancy in videos men- tioned before. We propose extracting spatiotemporal attributes (c.g., trajectory, identity, action) to capture relatively independent information from each frame. This approach offers two advantages:
5

---

Published as a conference paper at ICLR 2025
1
Video
Expert Models
Altributions
Tracking Algorithm
Attribution Trajectories
Nick
Grounding DINO
Judu
Track Anything
Grounded-SAM SAM2
ByteTrack
Kalman Filter
GRIT
DetCLIP
Nick is griniing on The side, sceres.
DeepSort
Traeet Neos us fios is grimiing as the Juds coaing shess and halk stored"
a
2
Task Conversation
Querying Attribution Trajectory
Select Query Altribution
Plecse
Sponial
Temporal
say Leatt nsponss thi vide??
and carrespanale while mroecting By
nome: Jodg
Nike
Identity <ldi>> / cid2>
coordinate: <borts
1
grlnving the its
Trackl: Wiaeettd--frmme--aaae </Idl>
locolion: on the side
time: 2.0 - 3.5
mnjettry This shows W.T and so gt the office usk Flush
ffrmmer <//d2>
appaorance raboit, scored
oction: lasking
Palley: Oil need see euch Trams the video !
Good job
Figure 3: Overall pipeline of Unhackable Temporal Rewarding (UTR), UTR begins by using a mixture of expert models to extract unique spatiotemporal attributes and employs tracking al- gorithm to construct multiple subject trajectories based on confidence levels (data modeling, top). then performs bidirectional querying of temporal and spatial attributes to generate dialogue data (lask modeling, bottom), thereby learning spatiotemporal dynamics.
Frame-to-frame variations in attributes, especially positional coordinates, enable modeling of frame-specific information, increasing information density (aligning with Principle These attributes function queries link information across the video. facilitating learning of spatiotemporal dynamics (aligning with Principle M7y.
Specifically, given video frame sequence V = = [we)T with the same meaning in Eq. we extract the attribute information of subjects from each frame as follows:
X; (x xz 2t
app act F(ve).
where 2t0c ,x, app .xtet indicate the location, appearance, and action information of subjects in frame ve, respectively. Function I' extracts this information, using labeled data or specialized models such
as GRiT (Wu et al., 2022) and Grounding DINO (Liu et al. 2023a). We then organize this subject information into trajectories corresponding to each subject:
{Y = { v.d. act )==1]== N = A((x,)
(8)
where Y, is the trajectory subject and N is the number of subjects the video. To be specific, vi, Mi. id vft indicate the trajectory, identity, and action information of subject i in frame Us, re- spectively. Function associates subjects across frames form trajectories and identities, typically using tracking algorithms like Byte'Track (Zhang et al. 2022
Bidirectional querying explicitly models spatiotemporal dynamies. Previous methods (Wang et al. 2023 2024a) modeled relatively dense information by interleaving text with selected frames, yet they neglected the critical spatiotemporal dynamics. Inspired by Merlin (Yu et al. 2024), we propose bidirectional querying mechanism that uses any temporal or spatial attribute to query global spatiotemporal attributes. This approach offers two benefits:
Explicit modeling of spatiotemporal attributes forces the model to read each frame, aligning with Principle
The arbitrariness of querying across time and space enhances the model's understanding of spa- tiotemporal dynamics, and the stronger this arbirraiiees,, the deeper the understanding, aligning with Principle 11
Particularly, we randomly sample the information of one more subjects as query attributes and select several frames as query frames. The model must predict the complete subject information based on the provided query data. Formally,
P,Y)V,Y,) P (H (ye.,,
(9)
where { C (1,2....,A) N) represents sampled subject identities, (t, 1 C (1,2, T indicates selected query frames, and ys.,, denotes attribute information of selected subjects sampled from Y, which can be location, appearance, and action description.

---

Published as a conference paper at ICLR 2025
Notably, the random selection of query frames [t, ensures the model utilizes query information from any part of the video- beginnin.. middle, or end- as cues to trace the entire trajectory. This approach not only compels the model to fully observe and comprehend the entire vidco, avoiding shortcuts like relying solely on initial or final frames, but also enhances its understanding of time- dependent physical laws. By necessitating the model infer states across various temporal intervals, it implicitly learns concepts such as momentum, velocity, and acceleration, thereby strengthening its grasp of fundamental spatiotemporal dynamics.
4 EMPIRICAL RESULT DETAILS
4.1 EXPERIMENT SETTINGS
We primarily construct UTR-Data using several existing open-source video datasets, (Miech et al. 2019), MeViS (Ding et al. 2023), and LaMOT et. 2024e). To extract subject attributions from each video frame, we use the region-to-tee detector GRiT (Wu al. 2022). Subsequently, we apply the ByteTraack (Zhang et al.) 2022) tracking algorithm to construct attribution trajectories. Further details can be found in the Appendix
Datasets.
namely
Implementation Details. To apply our UTR modeling strategy within the current video MLLM, we have developed a novel video MLLM, i.e., Video-UTR. For the specific Video-UTR pipeline, we follow the general architecture in LLa V.VANeXT-Vidde (Zhang al.) (024c), which consists of a vision encoder, SigLIP-L (Zhai et al. 2023), a large language model, QWn--2 (Yang et al. 2024), and modality alignment projector, 2- layer GeLU-MLP. The training process consists of two stages. (1) Stage 1: Modality alignment. where only the projector is trained using the 558K LLaVA (Liu et al. 2024b) dataset. (2) Stage II: Multi-task joint training, where the LLM is trained with various task datasets including video instruction-folloiig data. Here, we mainly apply our UTR in the Stage II, which combines the constructed task data based on UTR with LLaVA-NeXT SFT data. Further details about the training settings can be found in the Appendix B
4.2 GENERAL COMPREHENSION EVALUATION
To showcase the generality and effectiveness of the proposed paradigm, we evaluated Video-UTR across various understanding benchmarks. Using the standard MLLM evaluation framework and the LLMs-Eval tool (Zhang et al.)
2024a). we assessed major image and video understanding tasks. Results are shown in Tables 1 and 2 For video understanding, we focused on three general bench-
marks: MVBench (I.i et al. 2024c). TempCompass (Liu et al., (024c), and VideoMME. (Fu et al.)
2024). as well as four video QA benchmarks: MVSD-QA (Xu et al. (017). MSRVTT-QA (Xu ct al. 2016). TGIF-QA (Jang et al. 2017), and ActivityNet-QA (Caba Heilbron et al., 2015). For
image understanding, we reported scores from popular benchmarks like MM Vet Yu et al. 2023), MMBench (Liu et al. 2023b), MMMU (Yue et al. 2024), MME (Fu et al. 2023). VA-VA (Liu
et al.
2024b). SEED (e et al.] 2023b), A12D (Kembhavi al. 2016). and RealWorldQA (AAl
2024
For fairness, we used results from original papers.
Video Understanding. Table 1 shows that Video-UTR outperforms other video MILMSs most benchmarks, ranking first in 4 out of 7 tasks, highlighting its strong video understanding capabili- ties. Its high scores on MVBench (58.78%). TempCompass (59.67%), and VideoMME (52.63%) demonstrate its ability to handle complex tasks like temporal reasoning, identifying differences, lo- cating objects, tracking motion, and interpreting dynamic scenes. Additionally, its performance on four video QA benchmarks reflects exceptional understanding, particularly in managing temporally sensitive information. Remarkably, Video-UTI achieves these results using only about 1.1 video samples, much smaller dataset compared to other models of similar performance, showcasing the efficiency and effectiveness of our UTR approach.
Image Understanding. Table 2 shows that Video-UTR, despite being video MILM, delivers highly competitive performance compared to image-level MLLMs. For instance, on key bench- marks, Video-UTR matches or outperforms top image MLLMs like LLaVA-1.5 (Liu et al. 2024b) (39.6% vs. 35.4% on MM-Vet) and the stronger LLaVA-NeXT-Img (Zhang et al. 2024c) (76.6% 72.1% on MMBench). It also performs well on hallucination benchmarks, achieving 88.9% on POPE, and excels in image QA, with 63. 7% on RealWoridQA, showing its ability to avoid misiden- tification and misalignment with irrelevnt image details. These results demonstrate that UTR not
7

---

Published as a conference paper at ICLR 2025
Table 1: General Video Understanding Performance Comparsion on benchmarks, Video-UTR outperforms competitors in 4 out of 7 benchmarks and ranks second on the others, despite these competitors using larger training datasets more parameters. Several benchmark names are abbre- viated due to space limits, TempC: Tempcompass, ANet-QA: Activity Accuracy. The best results are bold and the second-best results are underlined. * indicates metrics reproduced by ourselves for evaluation cause original paper does not report.
And Acc indicates
MSVD-QA MSRVVT-QA TGIF-QA
ANet-QA
Methods
LLM
MVBench Tempt' VideoMMEE
Scale
Azc
Scure
Acc
Score
Ace
Score
Ace
Score
VideoChar
2023a
Vicuna-78
766K
35,5
56.3
2.8
45.0
2.5
34,4
2.3
2.2
VidcoChatZ
2024c
Vicuna-7B
1.9M
51.1
38.5
70.0
3.9
54.1
3.3
49.1
3.3
Video-Chntii
2024
Vicuma-7B
765K
32.7
31.8
51.6
2,5
29.6
1.8
12.4
1.1
Video-LLaVA
2023
Vicuna-78
765K
34,8
39.9
64.9
3.3
49.3
2.8
51,4
3.0
35.2
2.7
Vid2
X124
LLaMA2-78
PLLaVA 2024
LLaMA2-7B
1M
46.6
76.6
4.1
62.0
3.5
77.5
4.1
56.3
3,5
NL.VANNeXTTVaa Video 2024c
Qwen2-7B
880A
54.6
33.7
67.8
3.5
53.5
3.2
LLaVA. LLaVA-One Visions 2024n
Qwen2-7B
1.6.
56.2
50.0"
58.2
65.3*
3.82
43.3*
3.0"
52.8*
3.4
56.6"
3.3"
Video UTR (Ours)
Qwen2-7B
1.1M
58.8
59.7
52.6
73.5
4.1
58.3
3.6
56.4
3.6
55.0
3.2
Table 2: General Image Understanding Performance Comparision on 9 benchmarks, Video- UTR achieves performance comparable to, surpassing, that of pure image-level MLLMs. LLaVA W; LLaVA in the wild. The best results are bold and the second- best results are underlined.
Methods
LLM
MM-Veet
MMMU
LLaVAW
RealWorldQAA
Image-feud MLL.M
InstrctBLIP
2024
Vicuna-7B
33.1
36.0
30.6
1137.1
59.8
86.1
53.4
40.6
36,9
Qwen-VL-Ch
2023b
Qwen-7B
47.3
60.6
37.0
1467 8
67.7
74.9
58.2
63.0
49.3
LLaVA-v1.5-77 2024a
Vicuna-7B
30.5
64.3
35.7
1510.7
61.8
86.1
58.6
55.5
54.8
I.
3B
Viuuna1338
35.4
67.7
37.0
1531.3
66.1
88.4
61.6
61.1
55.3
ShareGPT4v
2023a
Vicuna-7B
37.6
68.8
37.2
1567.4
72.6
86.6
69.7
58.0
54.9
LLaVA-NeXT-Bne
2024e
LLaMA3-8B
44.4
72.1
41.7
1551.5
63.1
60.0
Vileo-tevei MLLM
LLIMA-VID 2023b
Vicuna-BB
1521.4
59.9
VL
7B
73.1
LLaVA-NeXT- Vidco 2024c
QWen2-7B
42.6
1580.1
75.9
60.1
Viden-UTR (Ours)
Qwen2.7B
39.6
76.6
43.4
1583.6
69.4
88.9
74.7
72.1
63.7
only helps video MILMS overcome temporal hacking but also enhances their ability to analyze and understand images effectively.
4.3 ABALATION STUDY ABOUT UTR
Effectiveness of each component of UTR. In this ablation study, evaluate the impact of remov- ing the two key components of UTR: data modeling (UTR-Data) and task modeling (Bidirectional Querying) from Video-UTR. We focus three major video and image understanding benchmarks. As shown in Table removing UTR-Data and Bidirectional Querying leads to significant drop in performance on video understanding tasks, emphasizing their importance in handling complex video reasoning tasks. Notably, removing UTR-Data causes a more consistent and pronounced de- cline across all benchmarks, including both image and video tasks. This underscores the critical role of data modeling in UTR, as it directly aligns with the two principles we proposed.
At the same time, eliminate the potential influence of video data volume, we also add an equivalent amount of VideoChat2 (Li et al., 2024c) data. It can be observed that the additional video data did not result in further gains, which further underscores the importance of data modeling. Video data constructed in improper manner will inevitably lead to temporal hacking, thus hindering the improvement of true video understanding performance.
Ablation on the scalability of Video-UTR. In Section 2.2 we identify that the "anti-scaling law" phenomenon observed current video MLLMs is due to the issue of temporal hacking. To address this, we propose UTR mitigation strategy. In this experiment, video MLLMs, with the integration of UTR, can exhibit scalability. As shown in Table and Ta- 5 thanks the incorporation of UTR, Video-UTR demonstrates certain degree of scalability in
will demonstrate whether
8

---

Published as a conference paper at ICLR 2025
Table Abalation study of Video-UTR on both video and image understanding benchmarks.
Ablution Setting
Duta Scale
MVBench
TGIF-QA
ANet-QA
MMVet
MMBench
POPE
Video-UTI UTR
1.MM
58.78
56.44
55,00
39.59
76.63
88.86
Task Modeling
1.0
58.45
56.11
54.21
37.33
76,37
89,29
-atta Modeling
780K
54.63
54.74
54.15
42.20
75.77
89.13
More VideoChat2
1.M
57.65
53.39
53.65
36.56
75.95
88.76
Table 4: Scalability of video data size.
Table 5: Scalability of frame length.
UTR Data Size
MVBench
TempCompas
VideoMME
Frame Length
MVBench
TempCompass
VideoMME
54.63
58.88
50.93
56.28
52.56
180K
58.45
58.47
52.30
50.08
56.14 14
52.81
$8.78
59.67
32
$1.40
56.11
52.07
the size of video data. Specifically, the larger the volume of video data, the better the model's perfor- mance. Additionally, we observe that under the condition of unchanged video content, an increased number of video frames does not negatively impact the model's performance. This scalability is advantageous for further exploring the better performance of Video-UTR in the future.
4.4 SPAPIAL-TTEMORAT UNDERSTANDING OF VIDEO-UTR
Spatial and temporal comprehension are equally important for multimodal video understanding. Here, we evaluate Video- UTR's performance in these two areas us- ing the latest benchmark, MM-ID, in zero-shot setting. MM-ID tests a model's s ability to recognize identities across four increasingly complex levels, focusing on matching and locating objects with differ- ent identities across frames. As shown in Table 6 Video-UTR achieved highly competitive scores on both the match- ing and location sub-metrics without any MM-ID training data. Moreover, it out- performed methods using significantly larger datasets, further demonstrating the strength of the UTR approach. By lever- aging spatiotemporal attribute modeling, UTR effectively enables the model to
Table 6: Zero-shot spatial-temporal understanding performance on MM-ID (Ji ct al. 2024).
Methods
Matching
Location
Q&A
Caption
Open-source Modeis
MMICL 2003a SEED
3.53
3.18
2025H
3.19
3.58
QwenNL 2123h
0.504
3.63
2.65
Closed-source APIs
OwenVL-Phas 2023b
0.313
0.187
3.87
3.79
QwenVL-Maa
20140
0.224
0.301
4.64
4.23
Gemini
4.97
GPT-4V (X144)
0.627
0.244
4.77
4.67
Video-UTIE UTR (Ours)
0.277
0.328
4.36
3.62
Table 7: Zero-shot long-range video understanding performance on MMBench- Video (Fang al. 2024).
Methods
Overall
Perception
Reasoning
Clidde-3.--Some
1.35
VidkoChat2-HD
20z4
1.23
0.44
1.23
PLLAVA-34B
KT44
1.16
1.I
LLaVA-NeXT Video
2024c
1.13
0.58
1.03
learn both spatial and temporal aspects.
Video-1/TR-7B (Ours)
In addition, we rigorously assess the performance of our Video-UTR on the recently introduced and exceptionally demanding long-range video understanding benchmark, MVBench-Videe As illustrated Table 7 our Video-UTR exhibits remarkable competitiveness on the performance of video perception and reasoning. surpassing a multitude of models with 34B parameters and even larger architectures, despite its compact 7B parameter size.
4.5 IN-DEPTH ANALYSIS ABOUT THE TPL. SCORE
we design a novel metric, temporal perplexity (TPL score), measure the aligmment degree between the proxy reward and the true reward. In this part, we aim to elucidate the correlation between TP scores and true rewards more intuitively from the perspective of video data quality. Specifically, we randomly select 100 video-text pairs from WebVid (Bain al., 2021) and calculate their temporal perplexity based on the definition in Eq. Then we pick two representative examples to illustrate the relationship between temporal perplexity (TPL) and the quality of video-text pairs.
In Section 2.2
As shown in Figure it can be observed that higher TPL score indicates a higher information density in the video more detailed description. In this scenario, the model struggles to describe the entire video using just a single frame. Conversely, if the model's performance based on a single frame is nearly good when using all frames, it either suggests that the video's 's dynamics

---

Published as a conference paper at ICLR 2025
Caption: "Agapanthus flowers at ile Brehat, Bretagne France"
TPL Score: O. 0.0018573
Caption: "Happy
shopping online, watching photos on loptop together"
TPL Score: 0.5056167
Figure 4: Quantitative comparison of the video-text pair with different temporal perplexity
negligible, making it almost like image, the textual description is sparse that additional video information does not significantly improve modeling. The result aligns with our discussion in Section 2.2 and the analysis Figure This case study demonstrates that the TPL score can serve as useful metric for filtering high-quality video-text pair data. Please refer to Appendix [ir more in-depth investigation. Furthermore, for a more comprehensive and in-depth analysis and discussion of the UTR methodology, please refer to Appendix E
RELATED WORK
Multimodal video foundation models. Recently, vision-language models (Liu et al. 2024b Zhu
al. 2023 Zhao al. 2023b Wei al. 2024b have demonstrated versatile visual understanding through visual instruction tuning (Liu et al.) 2024b; Zhu et al. 2023). However, real-world video comprehension in multimodal models still in its early stages. Due the absence of powerful video
encoders in the community, existing mainstream video MLLMs (Zhang et al. 2024cc 2023 Cheng al. 2024) still rely on established images encoder, i.e., CL.IP (Radford et al. 2021), to extract visual information frame by frame. Subsequently, they integrate temporal modeling techniques, egg., Q-former (Zhang et al. 2022). 3D Conv (Cheng et al. 2024), and Pooling (Zhang et al. 2024c
Xu et al. 2024). to compress the visual tokens before feeding them with language tokens into LLMs.
In addition to advancing the design powerful temporal modules, recent works have increasingly acknowledged the pivotal role of video-language modeling in video comprehension. Some works try to design filtering mechanisms (Wang et al., (002bb) to obtain high-quality video data with fine- grained description. while others aim to construct appropriate data structures (Wang et al. 2023 2024a Chen et al., 2023b) and task formats (Yu et al. 2024) to enhance modeling performance. In this work, systematically present how to design effective video-language modeling from reinforcement learning perspective and propose guiding principles along with example frameworks.
Reward hacking theory was firstly introduced in the field of RL as special case of Goodhart's Law Goodhart 1984; Leike et al. 2018), and later explored in the context of AI alignment (Leike 2017). DeepMinnd] 2018) formalizes reward hacking by identifying types of reward mis- specifications that lead to it. Subsequent works (Pan et al. 2022] Laidlaw et al. (2024) try to deal with reward hacking from different aspects.
other hand, reward hacking is not exclusive to reinforcement learning, it also occurs in the optimization of pretrained visual generation models, where approaches often optimize towards a reward model by directly backpropagating gradients from differentiable reward model (Li et al. 2024b; Zhang et al. (022bb). In this work, we transfer the concept reward hacking to video-language modeling and establish a novel temporal hacking theory to explain the shortcut learning in the existing video MLLM.
CONCLUSION
In this work, we propose the theory of temporal hacking from a reinforcement learning perspective to explain shorteut learning in video MLLMs. We introduce novel metric, Temporal Perplexity (TPL), to quantify the severity of temporal hacking. Through extensive experiments, we use the TPL score to analyze the causes and features of temporal hacking, leading to the development two guiding principles for ladeo-language modeling. We further propose Unhackable Video- Language Modeling (UTR) and build a powerful video MLLM, i.e., Video-UTh We hope this work offers a new perspective and insights help the community build more robust video-Al systems.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This work was supported by the National Natural Science Foundation of China under Grant 62176096. This work was also supported by the National Natural Science Foundation of China under Grant 62071067.
REFERENCES
Anthropic.
Claude 3.5 sonnet.
claude 5-
3-5-----nnn-- sonnet 2024.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, al. Qwen technical report. arXiv preprint wwxiv:2339.166/0 2023a.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308. 1283008..9666 /2966, 2023b.
Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. In IEEE International Conference on Computer Vision, 2021.
Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles. Activitynet: A large-scale video benchmark for human activity understanding. In Proceedings of the ieee conference on computer vision and pattern recognition, pp. 961-970, 2015.
Lin Chen, Jisong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. arXiv preprint arXiv:2311.12799, 2023a.
Sihan Chen, Xingjian He, Handong Li, Xiaojie Jin, Jiashi Feng, and Jing Liu. Cosa: Concatenated sample pretrained lision-language foundation model. arXiv preprint orXiv:2306.09000 2023b.
Xi Chen, Josip Djolonga, Piotr Padlewski, Basil Mustafa. Soravit Changpinyo, Jialin Wu, Car- los Riquelme Ruiz, Sebastian Goodman, Xiao Wang, Yi Tay, et al. Pali-x: On scaling up multilingual vision and language model. arXiv preprint arXiv:2305 18565, 2023c.
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24185-24199 2024.
Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-Ilms. arXiv preprint arXiv:2406.07411 2024.
Max Coltheart. The persistences of vision. Philosophical Transactions of the Royal Society of London. B, Biological Sciences, 290(1038):57-69) 1980.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang. Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards general-purpose vision- language models with instruction tuning. Advances in Neural Information Processing Systems, 36, 2024.
Deep
Specification URL
gaming:
The flaw in the reward,
2018.
DeepSeek-AL, Daya Guo, Dejian Yang. Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu. Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan,
11

---

Published as a conference paper at ICLR 2025
Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang. Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. I.. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, panpan Huang, Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye. Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang. Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang. W. I.. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nic, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xi- aosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. LI, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi. Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang. Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha. Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song. Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforce- ment learning. 2025.
/ttp/vaiivvv..///../........a... org/ abs/ 2501 12948
Henghui Ding, Chang Liu, Shuting He, Xudong Jiang, and Chen Change Loy. Mevis: A large-scale benchmark for video segmentation with motion expressions. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 2694 2703, 2023.
Rumpei Dong, Chunrui Han, Yuang Peng, Zekun Qi, Zheng Ge, Jinrong Yang, Liang Zhao, Jianjian Sun, Hongyu Zhou, Haoran Wei, Xiangwen Kong, Xiangyu Zhang, Kaisheng Ma, and Li Yi. DreamL.M: Synergistic multimodal comprehension and creation. In The Twelfth International Conference on Learning Representations, 2024a.
Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, et al. ditemmlm-xcomposer/ Mastering free- form text-image composition and comprehension in vision-language large model. arXiv preprint 2411v:2401.164200 2024b.
Xinyu Fang, Kangrui Mao, Haodong Duan, Xiangyu Zhao, Yining Li, Dahua Lin, and Kai Chen. Mmbench-video: A long-form multi-shot benchmark for holistic video understanding. arXiv preprint arXiv:2400. 14515, 2024.
Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, Meishan Zhang, Mong-Li Lee, and Wynne Hsu. Video-of-thought. Step-by-step video reasoning from perception to cognition. In Forty-first International Conference on Machine Learning, 2024.
Christoph Feichtenhofer, Yanghao Li, Kaiming He, et al. Masked autoencoders as spatiotemporal learners. Advances in neural information processing systems, 35:35946-35958, 2022.
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang. Xiawu Zheng, Ke Li, Xing Sun, and Rongrong Ji. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint --Xiv:2306.13394 2023.
Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evalua- tion benchmark of multi-modal lims in video analysis. arXiv preprint arXiv:2405.21075 2024.
Yuying Ge, Sijie Zhao, Ziyun Zeng Yixiao Ge, Chen Li, Xintao Wang, and Ying Shan. Making Ilama see and draw with seed tokenizer. arXiv preprint arX1v:2310.0/2/8. 2023a.
12

---

Published as a conference paper at ICLR 2025
Yuying Ge, Sijie Zhao, Ziyun Zeng, Yixiao Ge, Chen Li, Xintao Wang, and Ying Shan. Making Ilama see and draw with seed tokenizer. arXiv preprint arXiv:2310001218, 2023b.
Charles Albert Eric Goodhart. Monetary theory and practice: The uk experience. (No Title), 1984.
GPT-4o. Hello gpt-4o, 2024. URL StLLttts://ppnaaii httpp//ww.......ooae//eeellll/o/
Agrim Gupta, Piotr Dollar, and Ross Girshick. Lvis: A dataset for large vocabulary instance segmen- tation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5356-5364, 2019.
Clark Jack and Amodei Dario. Faulty reward functions in the wild, 2016 URL https: 7/openai. hetpm/iidexeeeee fnde://falttyy-eearr funetiinnnnii//// December 21, 2016.
Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, and Gunhee Kim. Tgif-qa: Toward spatio- temporal reasoning in visual question answering. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2758-2766, 2017.
Yatai Ji, Shilong Zhang, Jie Wu, Peize Sun, Weifeng Chen, Xueleng Xiao, Sidi Yang, Yujuu Yang, and Ping Luo. Ida-vim: Towards movie understanding via id-aware large vision-language model. arXiv preprint arXiv:2407.07577 2024.
Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. In Computer -ccinnEECCV 2016: 14th Euro- pean Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV 14, pp. Springer, 2016.
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting lan- guage and vision using crowdsourced dense image annotations. jininnatoonal.uurna of computer vision, 123:32-73, 2017.
Cassidy Laidlaw, Shivam Singhal, and Anea Dragan. Preventing reward hacking with occupancy measure regularization. arXiv preprint arXiv:2403.03/885 2024.
Jan Leike, Miljan Martic, Victoria Krakovna, Pedro A Ortega, Tom Everitt, Andrew Lefrancq. Lau- rent Orseau, and Shane Legg. Ai safety gridworlds. arXiv preprint arXiv: 100983,. 2017.
Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, and Shane Legg. Scalable agent alignment via reward modeling: research direction. arXiv preprint 2018.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.0333 033266 2024a.
Jiachen Li, Weixi Feng. Wenhu Chen, and William Yang Wang. Reward guided latent consistency distillation. arXiv preprint arXiv:2403. 11027, 2024b.
KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.0635. 2023a.
Kunchang Li, Yali Wang, Yinan He, Yizhuo Li. Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mvbench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22195 22206, 2024c.
Ming Li, Yong Zhang, Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang, Ning Cheng, and Tianyi Zhou. Superfiltering: Weak-to-strong data filtering for fast instruction-tuning, arXiv preprint arXiv:2402.00530, 2024d.
Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models. arXiv preprint arXiv:2311. 1.11433,74,, 2023b.
13

---

Published as a conference paper at ICLR 2025
Yunhao Li, Xiaoqiong Liu, Luke Liu, Heng Fan, and Libo Zhang. Lamot: Language-guider multi- object tracking. arXiv preprint arXiv:2406.08324 2024e.
Bin Lin, Bin Zhu, Yang Ye, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311.1012 2023,
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Eision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- fion, pp.
2024a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuiigg. Advances in neural information processing systems, 36, 2024b.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren. Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499 2023a.
Yuan Liu, Haodong Duan, Yuanhan Zhang. Bo Li. Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: your multi-modal model an all-around player? arXiv preprint arXivv2307.06281, 2023b.
Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou, Tempcompass: Do video Ilms really understand videos? arXiv preprint 2rXiv:2403.00476, 2024c.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint 2.Xiv:2306005424.. 2023.
Meta. Llama3.2-vision, 2024. URL httth:tt////wwwt 11ama. com/
Antoine Miech, Dimitri Zhukov, Jean-Baptiss Alayrac, Makarand Tapaswi, Ivan Laptev, and Josef Sivic. Howtol00m: Learning text-video embedding by watching hundred million narrated video clips. In Proceedings of the IEEE/CVF international conference on computer vision, pp- 2630-2640, 2019.
OpenAl. Gpt-4v(ision) system card, Card. pdf 2023.
OpenAl. Learning to reason with llms. September 2024. URL
learning-to- resoooreaooo-- wih
Alexander Pan, Kush Bhatia, and Jacob Steinhardt. The effects reward misspecificaaion: Mapping and mitigating misaligned models. arXiv preprint 2mXiv:2201.03544, 2022.
Yuang Peng, Yuxin Cui, Haomiao Tang, Zekun Qi, Runpei Dong, Jing Bai, Chunrui Han, Zheng Ge, Xiangyu Zhang, and Shu-Tao Xia. Dreambench++: A human-alignen benchmark for personalized image generation. arXiv preprint arXiv:2406. 16855, 2024.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, al. Improving language under- standing by generative pre-training, article, 2018.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. PMLR, 2021.
14

---

Published as a conference paper at ICLR 2025
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06334 2017.
Joar Skalse, Nikolaus Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and character- izing reward gaming. Advances Neural Information Processing Systems, 35:9460- 00019947, 2022.
Richard Sutton and Andrew G. Barto. Reinforcement learning: introduction. MIT press, 2018.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, lean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:23122 1805, 2023.
Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Videomae: Masked autoencoders are data- efficient learners for self-supervised video pre-training. Advances in neural information process- ing systems, 15:10078-1003,, 2022.
Alex Jinpeng Wang, Linjie Li, Kevin Qinghong Lin, Jianfeng Wang, Kevin Lin, Zhengyuan Yang, Lijuan Wang, and Mike Zheng Shou. Cosmo: Contrastive streamlined multimodal model with interleaved pre-training. arXiv preprint (rxXi:2441.00844 2024a.
Jiawei Wang, Liping Yuan, and Yuchen Zhang. Tarsier: Recipes for training and evaluating large video description models. arXiv preprins arXiv:2407.00634 2024b.
Yi Wang. Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: A large-scale video-text dataset for multimodal understand- ing and generation. arXiv preprint arXiv:2307.06942 2023.
Yi Wang, Kunchang Li, Xinhao Li. Jiashuo Yu, Yinan He. Guo Chen, Baoqi Pei, Rongkun Zheng, Jilan Xu, Zun Wang, al. Internvide02: Scaling video foundation models for multimodal video 2024c.
understanding. arXiv preprint arXiv:2403.
Haoran Wei, Lingyu Kong, Jinyue Chen, Liang Zhao, Zheng Ge, Jinrong Yang. Jianjian Sun, Chun rui Han, and Xiangyu Zhang. Vary: Scaling up the vision vocabulary for large vision-language model. In European Conference Computer Vision, pp. 408 428-424. Springer, 2024a.
Haoran Wei, Lingyu Kong, Jinyue Chen, Liang Zhao, Zheng Ge, En Yu, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. Small language model meets with reinforced vision vocabulary. arXiv preprint arXiv:2401. 12500,, 2024b.
Jialian Wu, Jianfeng Wang, Zhengyuan Yang. Zhe Gan, Zicheng Liu, Junsong Yuan, and Lijuan Wang. Grit: A generative region-to-tens transformer for object understanding. arXiv preprint
122000221,.002880 2022.
Weijia Wu, Yuzhong Zhao, Zhuang Li, Jiahong Li, Hong Zhou, Mike Zheng Shou, and Xiang Bai. A large cross-modal video retrieval dataset with reading comprehension. Pattern Recognition, 157:110818, 2025.
xAI. Grok, 2024.
Dejing Xu, Zhou Zhao, Jun Xiao, Fei Wu, Hanwang Zhang, Xiangnan He. and Yueting Zhuang. Video question answering via gradually refined attention over appearance and motion. In Pro- ceedings of the 25th ACM international conference on Multimedia, pp. 1645-1653, 2017.
Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Msr-vtt: A large video description dataset for bridging video and language. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 5288-5296, 2016.
Lin Xu, Yilin Zhao, Daquan Zhou, Zhijie Lin, See Kiong Ng. and Jiashi Feng. Pllava: Parameter-fir free Ilava extension from images to videos for video dense captioning. arXiv preprint arXiv:2404. 16994, 2024.
15

---

Published as a conference paper at ICLR 2025
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024.
En Yu, Liang Zhao, Yana Wei, Jinrong Yang, Dongming Wu, Lingyu Kong, Haoran Wei, Tiancai Wang, Zheng Ge, Xiangyu Zhang, et al. Merlin: Empowering multimodal ulms with foresight minds. In European Conference on Computer Vision, pp. 425 443. Springer, 2024.
Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308. 02490, 2023.
Yinlong Yuan, Zhu Liang Yu, Zhenghui Gu, Xiaoyan Deng, and Yuanqing Li. A novel multi-step reinforcement learning method for solving reward hacking. Applied Intelligence, 012874-2888, 2019.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, al. Mmmu: A massive multi-discipline multi- modal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 9556-9567, 2024.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 11975-11986, 2023.
Hang Zhang, Xin Li, and Lidong Bing. Video-Ilama: An instruction-tuner audio-visual language model for video understanding. arXiv preprint arXiv:2306. 022858 2023.
Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang. Chunyuan Li, et al. Lmms-eval: Reality check on the evaluation of large multimodal models. arXiv preprint arXiv:2407.12772 2024a.
Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu, Fucheng Weng, Zehuan Yuan, Ping Luo, Wenyu Liu, and Xinggang Wang. Bytetrack: Multi-object tracking by associating every detection box. In European conference on computer vision, pp. 1-21. Springer, 2022.
Yinan Zhang, Eric Tzeng, Yilun Du, and Dmitry Kislyuk. Large-scale reinforcement learning for diffusion models. arXiv preprint arXiv:2401. 12244, 4, 2024b.
Yuanhan Zhang, Bo Li, haotian Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. Llava-next: A strong zero-shot video understanding model, April 2024c. URL
Haozhe Zhao, Zefan Cai, Shuzheng Si, Xiaojian Ma, Kaikai An, Liang Chen, Zixuan Liu, Sheng Wang, Wenjuan Han, and Baobao Chang. Mmicl: Empowering vision-language model with multi-modal in-context learning, arXiv preprint arXiv:2309.077 07791,, 2023a.
Liang Zhao, En Yu, Zheng Ge, Jinrong Yang, Haoran Wei, Hongyu Zhou, Jianjian Sun, Yuang Peng, Runpei Dong, Chunrui Han, et al. Chatspot: Bootstrapping multimodal llms via precise referring instruction tuning. arXiv preprint arXiv:2307.09444, 2023b.
Luowei Zhou, Chenliang Xu, and Jason Corso. Towards automatic learning of procedures from web instructional videos. In Proceedings the AAA/ Conference on Artificial Intelligence, volume 32, 2018.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models, 2023.
Zining Zhu, Liang Zhao, Kangheng Lin, Jinze Yang, En Yu, Chenglong Liu, Haoran Wei, Jianjian Sun, Zheng Ge, and Xiangyu Zhang. Perpo: Perceptual preference optimization via discriminative rewarding. arXiv preprint arXiv:2502.04377. 2025.
16

---

Published as a conference paper at ICLR 2025
A APPENDIX
In this appendix, we provide additional details about temporal hacking and our Unhackable Temporal Rewarding (UTR), which were omitted due to the 10-page limit of the main paper. Specif- ically, Section B elaborates the dataset and training settings of Video-UTR. Section C presents additional experiments to analyze UTR's s characteristics. Section[joffees more qualitative examples to demonstrate the capabilities of Video-UTR, and Pection/Eproviiies further discussion of existing approaches.
B ADDITIONAL DETAILS ABOUT EXPERIMENTAL SETTING
Additional information of the datasets. In Section 3.2 of the manuscript, introduced how established the unhackable temporal rewarding (UTR) including data modeling (UTR-Data) and task modeling (Bidirectional Querying). Now, in this section, we go into greater detail about how we collected and built the UTR-Data and how we constructed task conversation. To start, we provide an overview of our collected data in Table & and then dive into the step-by-step process of how was constructed.
Table 8: Training Data Statistics. We first build our UTR-Data mainly based on sampled HowTol00M McViS, and LaMOT. Then mix UTR-Data with several existing video conver- sation data, Le, LLaVA-NeXT-SFT and VideoChat2.
Modality
Dataset
Original
Used
Ratio %
Training Stage
10wTo100M
Miech et al.
2019
100M
50K
0.05%
Stage
MeVIS Ding et al.
2023
443K
90K
20.3%
Stage II
Video-Text
LaMOT Li
2024e
2.44M
225K
10.5%
Stage II
VideoChat2 et al.
2024c
100K
Stage
BLIP-558K
al.,
2024b
558K
S58K
100%
Stage
Image-Text
LLaVA-NeXT-SFT
Zhang et ul.
2024c
790K
790K
100%
Stage II
Vision-Language
Total
106.231M
1.813M
1.71%
Stage I & II
Specifically, we follow the steps below pre-process the raw video data to construct UTR-Data
(1) Randomly sample the fixed number (16, 24 or 32) frames at a certain frame (gap = 3,4 or 5) or random interval to form video clip each time.
(2) Extract all spatiotemporal attribution trajectories containing their category, identity, action and bounding boxes in each video clip. This can be accomplished through expert models, e.g., GRiT (Wu
et al. 2022]. Grounding DINO (Liu et al., (023a), and ByteTrack (Zhang et al. 2022) or directly obtained from the annotations provided by datasets.
(3) Remove the trajectory containing too small objects (smaller than 1/32 of the image size).
(4) Random select observation (spatial or temporal attributions in the randomly selected frame) as the query to conduct bidirectional querying task modeling.
(5) Compose the task format the following:
Question: System prompt + query question.
Answer: query answer, catl<idi>Praner <box>; .t.ox>;...//.....
where <query question, query answer> is the question-answer pair that designed based on the selected querying attributes.
Additional Training Setting Details, As stated the manuscript, Video-UTR follows a two-stage training procedure. In this part, will provide detailed overview of our training settings, includ- ing the hardware used for training, the duration, and the training hyperparameters. All information recoderd in Table
17

---

Published as a conference paper at ICLR 2025
Table 9: Training hyperparameters of Video-UTR. The hyperparameter placed in the middle indicates that this hyperparameter is used in both stages.
Configuration
Stage I
Stage II
Machine
NVIDIA Tesla A800 80GB GPU 64
Training hours
hour
20 hours
VIT init.
SigLIP-so400m-ppaae 14-384
Video-UTRR Stage
LLM init.
Twen2-7B-Instruus
Video-UTR Stage I
Projection init.
random
Video-UTR Stage
Image resolution
384²
3842
ViT sequence length
2048
2048
L.M sequence length
32K
32K
Video Frame length
32
Optimizer
Adam
Optimizer hyperparameter
B2 0.95, eps 1e-8
Peak learning rate
Vision Tower: 2e -;
LLM: 1e
Minimum learning rate
ViT learning rate decay
0.9
VIT Drop path rate
0
Learning rate schedule
cosine decay
Weight decay
0.05
Gradient clip
1.0
Training steps
lk
5k
Warm-up ratio
0.003
0.003
Global batch size
512
256
Gradient Acc.
Numerical precision
bfloat16 X
Optimizer sharding
Activation checkpointing
X
Model parallelism
Pipeline parallelism
X
Table 10: Video benchmark evaluation setting. We report some detailed setting during evaluation. MCQ: Multi-choice question. QA: ireestion-aswwer
Benchmark
Evaduation type
Prompts
Input framee
Answer selection
MOQ
Questionn "Plosse dinectly give the best option."
32
Comprss
Temoral
Caption Matching
Yes or No
Question "Flease dinectly give the best optiom!"
Question + "Please answer yes or no:"
32
32
GPT
Capiiniig
Queat:ion
32
MVBench
Video MCQ
"Question" Quesstion + "Option:" Options "Only give the best option."
32
Option matching
'hhee frames of video. Select best
following multiple
VileMME
Valeo-MC1
question bused on video. Respond with only the
C, comrect option.
32
Oytion matching
MSVD
Video QA
Question
32
GPT Score
NesQAA
Video
question msng sinple phrase."
GPT
TGIFQA
Video
Question
ViteoChuriPT
QA
Question
GPT
Additional Testing Setting Details, In the inference and evaluation phase, we essentially follow the settings PL (Xu et al. 2024) and LLLVA-NeeTT-Viee (Zhang et al. 2024c), including the system prompt for inference, the number of frames extracted, and so on, while conducting eval- uations on the existing video benchmark. Specifically, as illustrated in Table IO we mainly use the uniform frame sampling for frame selection. For answer selection based on GPT score, mainly use the gpt-3.5-turbo-066 version to evaluate the responses of our model.
C ADDITIONAL EXPERIMENTAL ANALYSIS
18

---

Published as a conference paper at ICLR 2025
Question: Tell me what happened in the video, and where did the blueberry go in the end?
Low TPL: In the
Frame1
Frame2
Frame7
Frame10
Frame13
Frame14
Frame16
video, person is seen picking up blue berries from bowl
and placing them in another bowl. The
blue berry ends up in the second bowl of the video.
Sheite
High TPL: In the video, person seen picking up blue berries from bowl and placing into another bowl. The blue berry placed on plate, and finally, it ended up in white bowl
shemte
then
PARRE
Figure 6: Output attention visualization. We compute the average output layer attention of the tokens generated by the model each frame the QA task and visualized the results.
More analysis about tem- poral perplexity (TPL). In the Section 4.5 present a case study to illustrate the relationship between the pro- posed TPL score and data quality, where higher TPL score indicates better data quality. In this part, we fur- ther present the relationship between TPL and data qual- ity from quantitative sta- tistical perspective. Specifi- cally, we calculate the TPL score different data sub- sets in VidcoChat2 (Li et al.] 2024c) and computed their average values. The results are shown in Figure 5 that the TPL. distribution for the YouCook2 (Zhou al. 2018) and TextVR (Wu al. 2025) sub- set is relatively high. This suggests that these two data subsets are of relatively high quality. As we know, these datasets, such YouCook2, contain large amount of first-person perspective and high- motion video data. These videos are rich in high information density and dynamic content, which beneficial for the model's temporal modeling. The results further prove that TPL provides a ref- erence for selecting nigh-quality data from VideoChat2. Based on the TPL distribution, sampling more reasoning data likely be more beneficial for achieving better video-language modeling.
1.30
8.20
8.15
6.89
messaiinnn an dtl
neroale dday 2onz
clrspiication voscattoooo
ne mateniigcoon
coserration
Figure 5: Quantitative TPL statistic of VideoChat2.
We can observe
More attention visualization analysis. In Figure 2(b) we present the attention map visualizations of frame tokens under model responses at different TPL. levels. In this part, we further provide more detailed attention analysis. As shown in Figure 6 & 7 we conduct two forms of attention visualization. The first involves video QA, visualizing the attention values between the answer content and the tokens of each video frame. The second form calculates the self-altention when inputting the video-text pair into the model simultaneously. From both visualization results, we can observe that Video- UTR, while achieving higher TPL score, clearly attends to more frames, thereby avoiding the loss of crucial details the video and making the answers more accurate and detailed.
19

---

Published as a conference paper at ICLR 2025
Video
Text
blusberry was token of bowl. The person initially intended to put the blueberry into cup, but instead of doing so, he placed the blueberry info white bowl.
Figure 7: Video-text input attention visualization. The left is the attention map of the model with low TPL while the right is the attention map with high TPL score.
D ADDITIONAL QUALITATIVE ANALYSIS
In this section, intuitively demonstrate the unhackable capability of Video-UTR, present several subjective video Q&A cases, as shown Figure Compared to our baseline, LLaVA-Next- Video, our Video-UTR demonstrates more accurate video understanding capability, specifically by better comprehending user queries, focusing on more video details, and providing more precise and less hallucinated responses. These results further validate the effectiveness of our proposed UTR modeling.
E MORE DISSCUSIONS
E. DEPENDENCY ON EXPERT MODELS FOR UTR
As introduced Section 3.2 of the main text, UTR leverages existing expert models to extract spatiotemporal attribute cues, which serve the foundation for data modeling and task modeling. Therefore, the capability the selected expert models and the quality the extracted attributes are critical variables that significantly influence the effectiveness of UTR modeling. In this part, we will delve into the significance of selecting expert models, the selection criteria, the details attribute extraction, and the validation of attribute quality.
The importance and rationale behind selecting expert models for attribute extraction. The use of expert models to support MLLM training has become widely adopted strategy in the current development stage. Notable implementations include models such as PaLI-X (Chen et al. (023c), Qwen-Vr VI. (Bai al., 2023a), Intern VI. (Chen al., 2024), and I.L.MA3.2 3.iiiin (Meta 2024). which integrate domain-specific expert models spanning areas like detection, grounding, and OCR scale up training data annotation. The effectiveness of this approach has been well-validated through extensive empirical studies. Fundamentally, these pipelines operate as a distillation pro- cess, transferring knowledge from expert models MLLMS enhance specific capabilities, such as fine- -oeee-rrinndddddd perception. In line with this paradigm, our proposed UTR framework employs expert models extract spatiotemporal attributes from video data, thereby strengthening the spatiotempo- ral perception abilities of video MI..Ms. This improvement is substantiated by the empirical results presented in Table 4.4 of our manuscript.
Extraction and filtering of high- attributes. To select specific expert models, we conducted a systematic evaluation based on existing benchmarks, e,g., COCO [Ln et al. 2014). Lvis (Gupta et al.] 2019). VG (Krishna et al. 2017), etc., of the performance of various options, such as GRiT (Wu et al. (002)) and Grounding (Liu et al. 2023a). to identify the most suitable candi- dates. For the proposed spatiotemporal attributes including bounding boxes, captions, identities, and actions, as illustrated in Figure 3 of our manuscript- we implemented a multi-stage selection and filtering process. First, we filtered the attributes based on the confidence scores provided by
20

---

Published as a conference paper at ICLR 2025
Human: Tell what happened in the vidco, and where did the blueberry go the end?
LLa LLaVA-NeXT- -A-NeeTT-Vide: In video, a person is seen picking up blue berries from bowl placing them in another bowl. The blue berry ends in the second bowl of the video.
Ours: the video, person seen picking up blue berries from a bowl and placing it into another bowl. The blue berry was then placed plate, and finally, it ended up in white bowl
Human: Please describe the video detail.
LLaVA-NeXT-Video:
standing hallway, and the holding white bag. They be having conversation, it's unelear what they's saying. The is wearing blue shirt, and the man is wearing black shirt. There green behind them, and window the wall. The man appears be looking something off-camerua, while the woman looks directly at camera.
The and
the vidco, man is scen in hallway and entering room. He then proceeds to walk towards the woman who is getting out the room. The man appeurs to be playful mood as he starts making funny faces at the woman, seemingly trying to entertain her. Overall, the video depicts light-hearted moment between two individuals who seem to be having good time together.
Ours:
Human: Tell me what is the man doing with the woman? Please describe detail.
LLLaV--NeT--V VeXTT-iide: The man and woman are standing in the dark, the man holding hand. They seem to be enjoying each other's company as they stand close together. The video shows them from different angles, but it clear that they are sharing a romantic moment.
the video, man woman seen walking down street night. The man wearing black jacket, they both appear to their late teens or early twenties. They continue walking for while, the man occasionally glancing over her. Eventually, they stop and stand facing cach other, and the man leans in to kiss her. The woman reciprocates the kiss, and they stand there for moment before pulling away from each other. Overall, the video depicts romantic moment between young people who seem to have strong connection.
Figure 8: Qualitative examples visualization of Video-UTR. Please note that we only display the most important frames from the full video (32 frames) to conserve space.
the expert models. Next, we applied multi-object tracking algorithm, i.e., ByteTrack (Zhang et al., 2022), analyze contextual correlations within the video content. This analysis included examining factors such the Intersection over Union (IoU) of bounding boxes across frames and trajectory continuity metrics, ensuring that trajectory lengths exceeded predefined thresholds. This compre- hensive process ensures the reliability and consistency of the extracted attribute trajectories, thereby enhancing their overall quality and utility.
21

---

Published as a conference paper at ICLR 2025
Human validation of the extracted at- tributes. To further validate the effec- tiveness of the extracted spatiotemporal attributes from video data, we conducted human evaluation experiment. Specif-
Table 11: Human validation of extracted attributes.
Validation
Location
Description
Consistency
Human
2.98
2.23
2.57
ically, 100 data samples generated using our UTR pipeline were randomly selected for assessment by human evaluators. Human annotators will score these data based on three criteria: the accuracy of the subject bounding box, the correct- ness of the attribute descriptions, and the consistency of the attribute trajectories, using a scoring range of I to 3. The results is shown in Table [1 We can observe that the average quality score of the extracted attributes is quite high, indicating a strong level of reliability. The results of this evaluation highlight the robustness and high quality of both the extracted spatiotemporal attributes and the constructed data, confirming the reliability of our pipeline.
E.2 CONSISTENCY OF TPL WITH HUMAN JUDGMENT
Table 12: Consistency between TPL score and human judgment.
High
Medium
Low
Validation
Richness
Relevance
Richness
Relevance
Richness
Relevance
TPL level
3
3
2
2
Human
2.85
2.76
2.15
1.85
1.61
1.64
In Section 4.5
we point out that TPL not only reflects the degree of temporal hacking in the video- language modeling process, but it can also serve a high-order metric to indicate the quality of video-text pairs. In this part, we plan to further explore this issue by examining the consistency of TPL with human judgment, highlighting the reliability of TPL score data filtering metric.
Specifically, we first randomly select 100 video-text pairs from VideoChat2 (Li et al. 2023a) and calculate their temporal perplexity based on the definition in Eq. 6 Next, we sort the data by their TPL values and divide it into three groups: high, medium, and low. We then invite several hu- man annotators rate these sampled video-text pairs on scale of to The criteria for scoring includes two aspects, i.e., the richness of the video-text information (considering both information density and dynamics) and the relevance of the video the text. Based on the annotators' scores, the consistency can be evaluated based on the average human ratings and their alignment with the level categories. Before the human annotators begin annotation process, we provide a detailed an- notation guideline, which explains the scoring criteria and standards comprehensively and includes relevant references. Specifically, the two composite metrics, i.e., richness and relevance, are defined as follows:
Richness:
Frame information density, referring to the degree to which each frame corresponds to inde- pendent description.
Level of descriptive detail, referring to the richness of details included from the video.
The richness of motion information, including the extent of motion in both the subject and the scene.
Relevance:
The relevance between the video and the text, specifically, the degree to which the description corresponds to the video content.
The relevance of the video context, specifically, the extent which the descriptions of relevant subjects in the video reflect dynamic changes.
Follow the above standards, annotators conducted the validation of the sampled cases and ultimately summarized their assessments into scores for two composite metrics. The results is shown in Ta-
12 We can observe that the groupings based on TPL scores and those based on human judgments are generally consistent. This indicates that our proposed TPL score is a reliable metric for filtering high-quality video-text pair data.
22

---

Published as a conference paper at ICLR 2025
34
BRIEF
PATRICK
NILLION
HISTORY
IRELAN
IRISH
Question: What happened when Irish imiggrants brought the tradition of St. Patrick's Day to America?
Options: A. They began
around drinking and celebrating the festival. B. St. Patrick moved to America.
C. They celebrated St. Patrick's birth instead.
D. They attended the church and gather for feasts.
Model output: B
dd oweff
Question: What is the shape of the paper shown in the video?
Options: A. Cirole
Rectangle
C. Square
D. Triungle
Model output: D
Figure 9: Failure case visualization of Video-UTR. We select two representative failure cases from the VideoMME (Fu et al. 2024) benchmark.
E.3 FAILURE CASE ANALYSIS OF UTR
Although our proposed UTR significantly mitigates temporal hacking from both data modeling and task modeling perspectives, it still has limitations in some situation, and we identify several rep- resentative examples on the VideoMME (Fu et al. 2024) benchmark. As illustrated in Figure the top case shows that Video-UTR does not perform as well on certain knowledge-orient MCQ tasks. This type of question tests the inherent knowledge base of large language models, so our UTR method does not result in significant improvement. The bottom case illustrates that in scenarios where the answer can be determined by analyzing a single frame or a few frames, our UTR method does not demonstrate a significant advantage. Placing more emphasis on the overall video content does not provide notable benefits in addressing such questions.
Video
The aforementioned failure cases analysis also highlights the need to design better video under- standing benchmarks that can more reasonably and reliably evaluate the ability of video MILMs to observe and comprehend the overall video content, rather than relying heavily on the inherent capabilities of LLMs.
E.4 LIMITAION AND FURTURE WORK.
Limitation of Unhackable Temporal Hacking. Although our proposed UTR significantly miti- gates temporal hacking from both data modeling and task modeling perspectives, has notice- able limitation in terms of its reliance expert model accuracy. Since UTR modeling is based on extracted subject attributes, the quality of these attributes- such as positional accuracy, pre- cise descriptions of the subject's appearance and actions, and the accuracy of trajectory associa- tions-directie impacts the overall performance of the final model. Therefore, improving the quality of these extracted subject attributes represents a highly valuable direction for future improvement.
Future work. On the other hand, seamlessly integrating the constructed attribute trajectorics into dialogues poses yet another challenging issue. Exploring whether single multimodal large lan- guage model (Yu et al., 2024; Dong et al. 2024at Peng et al.) (024) can be utilized to handle the entire data processing and task construction pipeline is highly promising research direction. In addition, the establishment a more explicit temporal rewarding mechanism represents a valuable research direction. Leveraging reinforcement learning algorithms, e.g., DPO (Rafailov al. 2024 Zhu et al.) 2025). PPO (Schulman et al., 2017) and GRPO (DeepSeek-Al et al. 2025). for post- training enhancement of the model's 's video comprehension and reasoning capabilities constitutes key focus for future research.
23


Reviews:
[{"review_id": "ZaJxhOYdOS", "reviewer": "Reviewer_PyRQ", "summary": "This paper addresses a critical problem in MLLMs for video processing: \"temporal hacking,\" where models simplify the task by focusing on select frames, missing broader context. The authors propose a framework called UTR that mitigates temporal hacking by emphasizing consistent frame activation across the entire video, leading to more accurate video comprehension. This framework incorporates TPL to quantify misalignment in temporal modeling and guides the model through two key principles: high frame information density and inter-frame information dynamics.", "questions": "1. How is equation 6 used in loss function?\n2. For equation 6, why using T:T instead of T-t:T for the second term?\n3. Is that possible to get the TPL of your baselines comparing with Video-UTO?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1.  The paper introduces temporal hacking as a new perspective on reward hacking in RL and effectively adapts this concept to video MLLMs.\n2. TPL score is a notable contribution, providing a quantitative way to assess temporal misalignment, a challenge often overlooked in video modeling.\n3. The approach shows promise for scalability with additional data, making it versatile for future applications and datasets.\n4. The results are good comparing to various larger models.", "weaknesses": "1. UTR relies on high-density, high-dynamic frame descriptions. Several datasets is lack of this, I am wondering how are the other methods performance could be improved after simply data modeling.\n2. The ablation study is confusing, does table 3 \"-Data Modeling\" means no data modeling and no task modeling or just no data modeling? \nThe comparison data scale is different which is unfair to compare.\n3. For ablation result, the Video-UTO is not the best without Data Modeling on MMVet and also UTR-Data size of 180K results are sometimes worse than 0K is also confusing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7gYUQfeYoI", "reviewer": "Reviewer_4fw2", "summary": "This paper thoroughly analyzes the phenomenon of temporal hacking, where video MLLM learns to spuriously correlate its answer to query prompts to a subset of selected frames without considering the whole video context. The author introduces temporal perplexity (TPL) to assess this misalignment, showing a good correlation with model performance. Moreover, the author proposes the Unhackable Temporal Rewarding (UTR) frameworks and empirically demonstrates their effectiveness by comparing their methods with several open-sourced MLLM.", "questions": "1. There is an inconsistency between Table 1 and the text in Line 376 regarding the data scale used for Video-UTR. Table 1 lists the data scale as 1.1M, whereas Line 376 claims the performance is achieved with approximately 700K video data points. Which number is correct?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The proposed $\\mathcal{T}\\_{tpl}$ is interesting and correlates well with the model performance.\n\n2. This paper systematically analyzes reward hacking from the RL perspective and proposes two guiding principles for designing unhackable rewards.\n\n3. The proposed method Video-UTR outperforms several baseline methods on both video and image understanding tasks.", "weaknesses": "- **Over-complicated Presentation**: I feel the presentation of the paper is unnecessarily complicated and does not explain the details sufficiently. The paper spends too much space introducing the concept of **temporal hacking**, which can be efficiently summarized as *existing training data and objectives make the learned MLLM find shortcuts when answering the query prompt by spuriously correlating the answers with only a subset of video frames, ignoring the whole video frame context.* In contrast, the author is supposed to provide more details of their proposed methods in Section 3.2 instead of putting them in the Appendix.\n\n- **Lack of detailed description of the proposed TPL $\\mathcal{T}\\_{tpl}$**: Equation 6 defines $\\mathcal{T}\\_{tpl}$, which requires query the function $\\mathcal{R}\\_{ppl}$. However, the paper does not provide details about $\\mathcal{R}\\_{ppl}$ other than citing the original paper developed the $\\mathcal{R}\\_{ppl}$. For example, does calculating the $\\mathcal{R}\\_{ppl}$ require access to a specific MLLM?\n\n- **Empirical results**: comparison between the proposed Video-UTR and LLaVA-OneVision is insufficient. The paper only reports LLaVA-OneVision’s results on 3 out of 7 benchmarks, where it outperforms Video-UTR on 2 out of those 3. Besides, the authors should underline PLLaVA's accuracy for ANet-QA rather than underlining their Video-UTR. Additionally, the authors should include results for MLLMs that are also finetuned from QWen2-7B in Table 2. Without these results, it is difficult to assess the source of the performance improvements.\n\n- **Regarding the RL perspective**: Reward hacking is not exclusive to reinforcement learning, as RL is just one approach to maximizing a reward function. In the field of aligning pretrained visual generation models, several methods optimize toward a reward model by directly backpropagating gradients from a differentiable reward model. These approaches are also susceptible to reward hacking, as discussed in [1, 2]. Reward hacking generally occurs when the learning objective fails to fully capture the intended model behavior. I recommend revising this statement and including a citation for [1,2] to provide a broader context.\n\nMinor:\n\n- **Lack of detailed figure captions**: The authors should thoroughly explain each figure within the main text rather than leaving readers to infer their meanings.\n\n- **Missing citations**: Various benchmarks have recently been proposed to evaluate MLLM's video understanding capabilities, e.g., [3, 4]. The authors should consider evaluating their methods on these benchmarks and cite them in the related works sections.\n\nOverall, the presentation of the paper can be significantly improved.\n\n[1] Li et al., Reward Guided Latent Consistency Distillation. TMLR 2024\n\n[2] Zhang et al., Large-scale Reinforcement Learning for Diffusion Models.\n\n[3] Fang et al., MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding. NeurIPS 2024\n\n[4] He et al., MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C6EGcXdRL0", "reviewer": "Reviewer_Edyo", "summary": "The paper addresses a paradox observed in training video-processing Multimodal Large Language Models (MLLMs), termed the \"anti-scaling law,\" where increasing data and model size leads to decreased performance. The authors identify \"temporal hacking\" as the culprit—a phenomenon where models focus on select frames instead of capturing the full video narrative, effectively shortcutting temporal understanding. I believe this behaviour is analogous to posterior collapse in VAEs, where models find ways to ignore part of the input while still optimizing the objective, suggesting a fundamental challenge in enforcing comprehensive information utilization in deep learning models. The authors present a theoretical framework from a reinforcement learning perspective to explain temporal hacking and introduce the Temporal Perplexity (TPL) score as a metric to assess misalignment in temporal modelling. To mitigate temporal hacking, they propose the Unhackable Temporal Rewarding (UTR) framework, which enhances video comprehension by aligning proxy rewards with true objectives through spatiotemporal attribute extraction and bidirectional querying. This solution addresses the root cause: the misalignment between training objectives and desired behaviour in video understanding. Empirical results show that TPL correlates strongly with temporal modelling quality and that UTR significantly improves video MLLMs' performance across multiple benchmarks, while also demonstrating scalability with increased data and frame lengths.", "questions": "1. Could the authors provide more detailed explanations of the TPL score calculation and its computational overhead? How does it scale with longer video sequences?\n2. How does the computational complexity of UTR compare to existing video MLLM approaches? Is the additional overhead from attribute extraction justified by the performance gains?\n3. Have the authors explored how the quality and computational cost of expert models affect the final performance? Is there a minimum accuracy threshold needed for these models, and how does their use impact the overall training time?\n4. Are there any limitations or potential drawbacks to the UTR approach in scenarios with highly redundant or static video content?\n5. Could the authors provide more details about how TPL correlates with human judgment of video understanding quality?\n6. Can the framework be applied to other domains involving sequential data, such as speech or time-series analysis?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Originality**: The paper introduces the novel concept of temporal hacking in video MLLMs and frames it within a reinforcement learning context, offering a fresh perspective on the anti-scaling issue.\n- **Theoretical foundation**: The reinforcement learning perspective provides clear insights into the problem and guides the development of the solution.\n- **Practical metric**: The introduction of the Temporal Perplexity (TPL) score provides a quantifiable measure of temporal misalignment, which correlates with model performance and can guide future research.\n- **Proposed solution**: The Unhackable Temporal Rewarding (UTR) framework is a constructive approach to mitigate temporal hacking, showing significant improvements in video comprehension tasks.\n- **Comprehensive evaluation**: The paper presents extensive experiments across multiple benchmarks and includes detailed ablation studies, demonstrating the effectiveness of the proposed methods.\n- **Practical impact**: The proposed solution is implementable and shows strong empirical results, indicating its potential applicability in real-world video MLLMs.", "weaknesses": "- **Clarity and organization**: My main issue is that the paper's presentation suffers from unclear explanations and a lack of cohesion in some sections. Technical terms are sometimes introduced without sufficient context, making it hard to follow, at least to me. I even found Figure 1, which aims to provide a clear illustration of the phenomenon, relatively unclear. \n- **Limited discussion of computational overhead**: While the supplementary material provides some training details, the paper lacks a thorough analysis of the computational cost introduced by the UTR framework, particularly regarding the extraction of spatiotemporal attributes using expert models. This makes it difficult to assess the practicality and scalability of the approach compared to existing methods.\n- **Dependency on expert models**: The method relies heavily on the accuracy of expert models for attribute extraction, which may limit its applicability if these models are not sufficiently accurate or if they introduce significant computational overhead.\n- **Theoretical depth**: The reinforcement learning framework and the concept of temporal hacking could be more rigorously developed. Definitions and theoretical justifications need strengthening.\n- **Related work**: There is a lack of thorough comparison with existing methods addressing similar issues, which limits understanding of the paper's novelty and significance in the broader context.\n- **Limited exploration of failure cases**: The paper could benefit from more analysis of scenarios where UTR might not perform well, such as with highly redundant or static video content.\n- **Reproducibility**: Details on experimental settings, such as hyperparameters, data preprocessing, and model configurations, are insufficient, hindering the ability to replicate results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ckAYZI1zUM", "reviewer": "Reviewer_ezXT", "summary": "The paper introduces Unhackable Temporal Rewarding (UTR), a framework to improve multimodal large language models (MLLMs) by addressing a phenomenon called \"temporal hacking\". This phenomenon occurs when MLLMs, which process video frames sequentially to understand narratives, exploit shortcuts by focusing on only a few frames (sometimes just the first and the last), thereby missing the entire context. This issue has led to an anti-scaling paradox where increasing model size or data volume degrades performance. The authors propose UTR to counteract this problem by enforcing comprehensive temporal processing, ensuring the model engages with the full video content.", "questions": "Please see weaknesses above.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-written, with the key contributions listed clearly in the introduction. The authors also identified relevant literature in this line of research and highlighted their contributions in the paper. The authors introduce very fundamental principles that should be used to design a video captioning metric, which are high frame information density and high inter-frame information dynamics. They construct a metric by extracting the location, appearance, and action information of subjects in a frame using pre-trained models. They construct datasets based on 3 open-source video datasets, from which they extract subject attributions from each frame in the video.", "weaknesses": "1. Since the proposed metric relies on subject attributions extracted from pre-trained models, the errors from those models might propagate into the extracted metadata and subsequent make the metric unreliable. Can the authors comment on whether this is a viable concern?\n2. Have the authors measured the reliability of extracted metrics in the presence of multiple subjects in one frame?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)