VUDG: A Dataset for Video Understanding Domain Generalization
Authors: Ziyi Wang
Zhi Gao
Boxuan Yu
Zirui Dai
Peiyao Wang
Yuxiang Song
Qingyuan Lu
Jin Chen
Xinxiao Wu
Venue: ICLR 2026 Poster (2026)
Keywords: Video Understanding
Dataset
Domain Generalization
Primary area: datasets and benchmarks

Abstract: Video understanding has made remarkable progress in recent years, largely driven by advances in deep models and the availability of large-scale annotated datasets.
However, the robustness of these models to domain shifts encountered in real-world video applications remains a critical yet underexplored problem, limiting their practical reliability.
To address this problem, we introduce \textbf{V}ideo \textbf{U}nderstanding \textbf{D}omain \textbf{G}eneralization (\textbf{VUDG}), the first dataset designed specifically for evaluating domain generalization in video understanding.
VUDG contains videos from 11 distinct domains that cover three types of domain shifts, and maintains semantic consistency across different domains to ensure fair and meaningful evaluation. We propose a multi-expert progressive annotation framework to efficiently annotate videos with structured question-answer pairs designed for domain generalization.
Extensive experiments on 9 representative Large Vision-Language Models (LVLMs) and several traditional video question answering methods show that most models (including state-of-the-art LVLMs) suffer performance degradation under domain shifts. 
These results highlight the challenges posed by VUDG and the difference in the robustness of current models to data distribution shifts. We believe VUDG provides a critical resource to benefit future research in domain generalization for video understanding.

Published as a conference paper at ICLR 2026
VUDG: A DATASET FOR VIDEO UNDERSTANDING Do- MAIN GENERALIZATION
Ziyi Wang's ! Zhi Gao Boxuan Yu', Zirui Dai Peiyao Wang', Yuxiang Song', Qingyuan Lu', Jin Chen' Xinxiao Wu 1.2
'Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science & Technology, Beijing Institute of Technology
"Guangdong Laboratory of Machine Perception and Intelligent Computing, Shenzhen MSU- -BT University
https: /tap:://VUDG-VVV -innDD--Viieee git.hiithi... i
ABSTRACT
Video understanding has made remarkable progress in recent years, largely driven by advances in deep models and the availability of large-scale annotated datasets. However, the robustness of these models to domain shifts encountered in real-world video applications remains a critical yet underexplored problem. limiting their prac- tical reliability. To address this problem, we introduce Video Understanding Domain Generalization (VUDG), the first dataset designed specifically for eval- uating domain generalization in video understanding. VUDG contains videos from 11 distinct domains that cover three types of domain shifts, and maintains semantic consistency across different domains to ensure fair and meaningful evalu- ation. We propose a multi-expert progressive annotation framework to efficiently annotate videos with structured question question-ansser answer pairs designed for domain general- ization. Extensive experiments on representative Large Vision-Language Models (LVLMs) and several traditional video question answering methods show that most models (including state-of-the-ae the-ttrrtt LVL.Ms) suffer performance degradation under domain shifts. These results highlight the challenges posed by VUDG and the difference in the robustness of current models to data distribution shifts. We believe VUDG provides critical resource to benefit future research in domain generalization for video understanding.
INTRODUCTION
Video understanding has achieved remarkable progress tasks like action recognition and video question answering (VideoQA) (Chen & Ho 2022] Lin et al.] 2022 Li et al.] 2022). However, most models assume the training and testing data have the same distribution, leading significant performance degradation when encountering distribution shifts in real-world applications. This issue becomes more and more critical large languaaee models (LVLMss) increasingly fine-tuned for specific video applications. When deploying in the real world, model's ability to handle unseen domains is essential for safety and reliability,
impossible cover all potential data distributions during the fnn-ttunin phase. This problem could be formulated as the task of domain generalization (DG) (Lin et al., 2023b; Paaaaaaii Spyrou 2024, Chen et al.) [023] Wang et al. 2026) in video understanding, where the model trained on the training data (source domain) expected to perform well on the unseen testing data (target domain) with different distributions. Given the aforementioned challenges, evaluating the generalization of LVL.Ms after fine-tuning is an essential and valuable step to ensure their robustness in real-world scenarios.
Although there exist some benchmarks (Jang al., 2017 Li ct al. 2024a Fu et al. 2024) of video understanding across different domains, they are not suitable to evaluate the DG performance, as their semantic spaces across domains are different, and the performances of models will be affected not only by the domain shifts but also by the semantic space differences. Hence, the robustness of models to the domain shifts, i.e., the model's 's generalization, may be evaluated properly. This
"Corresooniigg author: Zhi Gao.

---

Published as a conference paper at ICLR 2026
Domains
Quession-nsswerin
Evaluation
Viewpoint
Condition
Action Recognition G: What the doing? sitling chair.
Multiple
Test
FOOD
Carbbon
Attribute identificttin
A: They yellw.
Q: What color the lighi the unbreela??
Rainy
Snowy
Night
Single
Nig1
Object identification
Q: What is placed on the table?
Shary
A of cheese.
Pre-insined LVLMs
Rairy
Temporal Understanding
Test
Q: What happened after the man sat down?
Domaine
Sarvilance
A He tumed on the TV.
Virtual
Snewy
Figure
Overview of the proposed VUDG dataset.
highlights critical gap: the lack of dedicated dataset to rigorously and properly evaluate the domain generalization capabilities of video understanding models.
To bridge this critical gap, we formally identify and tackle the problem of domain generalization in video understanding. We introduce Video Understanding Domain Generalization (VUDG), the first dataset specifically designed evaluate the performance of domain generalization in video understanding. VUDG comprises 11 domains, with videos sourced from diverse online platforms and open-source datasets (Wang et al., 2019: Ugai et al. 2024 Wang al. 2024a Chen al. 2024). This collection strategy aims to maximize the diversity of videos while preserving inter-domain semantic consistency, a core requirement
evaluate the domain generalization properly. These domains exhibit variations in visual styles (e.g., cartoon, game, movie/TV show, virtual environment), viewpoints (e.g., egocentric, surveillance, shaky), and environmental conditions (e.g.. foggy, night, rainy, snowy), making VUDG comprehensive generalization evaluation in video understanding. Note that we collect both the training and testing data for each domain, training and testing data are from distinct data sources, and the testing data is from testing splits of existing datasets or online platforms, to avoid the potential data leakage issue in evaluating LVLMs.
To construct VUDG, we introduce progressive multi-expert annotation framework specifically designed for the DG task. A key feature of our framework ensuring semantic consistency across all 11 domains by pre-defining shared space of daily human activities. The framework then employs cascade distinct large models for automated QA generation and verification. This design mitigates seff-reinforcement bias and streamlines the final human review process ensure data quality. This process yields high-quality open-ended and multiple-choice QA pairs, making VUDG a versatile dataset to rigorously evaluate model generalization. The dataset is designed support various DG protocols, including multi-source (Wang et al. (202I, single-source (Chen et al. 2023). and zero-shot (Lu et 2024) domain generalization.
We conduct extensive experiments on VUDG with representative VideoQA models and 9 state-of-the- art LVLMs under multi-source, single-source, and zero-shot DG settings. The results reveal that all models struggle to transfer into the target domains with unseen and different distributions, showing significant performance drop compared to their performance on source domains. These experiments validate the challenges posed by VUDG and highlight the limitations of current models in handling domain shifts, motivating future work on developing more generalizable video understanding models.
In summary, key contributions
follows:
We identify the problem of domain generalization in video understanding and introduce VUDG, the first dataset specifically designed evaluate it. VUDG contains 11 different domains from varying visual styles, viewpoints, and environmental conditions, while sharing the same semantic space.
We propose progressive multi-expert annotation framework designed for DG that leverages a cascade of models for annotation generation and verification, minimizing model bias and human workload while ensuring high-quality open-ended and multiple-choiee QA pairs.
We conduct comprehensive experiments on 9 state-of-the-ar LVLMS and several VideoQA baselines, revealing the challenge of VUDG large gaps across domains and highlighting the limitations of current models under distribution shifts.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
2.1 VIDEO UNDERSTANDING BENCHMARKS
The development of video understanding datasets has fueled recent advancements in video under- standing. Datasets such as ActivityNet (Caba Heilbron et al. 2015) and Kinetics (Kay et al. (017) have been pivotal action recognition tasks, providing millions of labeled video clips across diverse activities. Later, Charades (Sigurdsson et al. 2016). TVQA (Lei et al. 2018), MSVD-QA (Xu et al. 2017). and MSR-VTT-QA (Xu et al. 2016) expand the scope of video understanding include VideoQA tasks, introducing videos paired with textual questions that require reasoning about the video content. However, these datasets ignore scenarios where the distributions of the training and testing data are inconsistent, leaving the problem of domain generalization underexplored. Although some recent datasets (e.g. Video Vista (Li et al. 2024b), Video-MME (Fu et al. 2024)) include videos from multiple categories such as HowTo, Film, and Cartoon, the semantic disparities between these categories are often too large. Therefore, they are not ideal for isolating the effects domain shifts, making it difficult to fairly evaluate model's generalization ability across domains in downstream tasks.
2.2 DOMAIN GENERALIZATION IN VIDEO TASKS
Previous works Wang al. 2024b Zhang et al. 2023 Lin et al., 2023b] Yao al. 202T) have explored domain generalization (DG) in video tasks to enhance robustness under unseen distributions. VideoDG (Yao et al.) 2021) introduces an adversarial pyramid network and constructs three DG settings based on difterent dataset sources, different action consequences, and different camera viewpoints for video classification generalization. Ani-GIFs (Majumdar et al. (2022) presents the first synthetic DG dataset using animated GIFs and real videos to study domain shift in action recognition. ARGOIM (Plizzari et al.) 2023) samples egocentric clips from Ego41) (Grauman et al. 2022) across diverse scenarios and locations evaluate cross-context generalization. MDVAD (Flaboreal et al. 2023) aggregates six surveillance video datasets to benchmark anomaly detection under environment and camera shifts. In contrast these datasets that focus on domain generalization
in video classification, anomaly detection, or action recognition, our dataset is tailored for video understanding, which poses richer visual reasoning challenges and aligns closely with the rapid development of LVLMs.
Table The comparison existing video understanding datasets involves several key aspects: total number of videos (Videos) and video clips (Clips), number of QA pairs (QA Pairs), annotation method (Anno., where M/A indicates manual/automatic). whether the videos contain diverse domains (Dom.), and whether the semantic spaces across domains are consistent (Sem.).
Datasets
Vidcos
Clips
QA
Dom.
Scm.
MSRVIT-OA A Xu
2017
2,990
2,990
72,821
X
MSVD-QA Xu al. 2017
504
504
13,157
X
NerivityNer-OS QA Ya al. 2019
800
800
8,000
X
EgoSchema Mangalam al
2023
5,063
5,063
5,063
ARM
X
TGIFQA
9,575
X
MVBench
(L al.
3024g
3,641
3,611
4,000
V
X
BedchBBanch
Ving et
2023
5,917
5,917
17.036
A&M
Y
X
TempCompass Liu
2024
410
5000
7,540
A&M
X
Vidco-MME
900
X
VideoVista
20246
894
3,402
3,402
X
VUIXi (Ours)
7,899
7,899
36,388
A&M
VUDG DATASET
We introduce the VUDG dataset that contains 11 distinct domains, including videos with different visual styles, viewpoints, and environmental conditions. To ensure high-quality data, we propose progressive multi-expert annotation framework that leverages multiple large models, followed by human review for question- answer pairs generation and filtering. Importantly, we incorporate different large models in the generation and verification stages mitigate the bias stemming from fixed model that tends to validate its own outputs, thereby improving the diversity, objectivity, and robustness of the collected QA pairs. The annotation pipeline consists of four key stages: (a) Video
3

---

Published as a conference paper at ICLR 2026
Collection, (b) Open-Ended QA Pairs Generation, (c) Multiple-Choics QA Pairs Generation, and (d) QA Pairs Screening and Reviewing. The overall workflow is illustrated in Figure
Video Collection
Open Ended QA Generation
Muti-Choice QA Generation
QA Pairs Screening and Reeie
Visuo
Filaring
Open-Enn Erded GA Paim
GA Pains
Video
Exsiiig
Dataneta
DeepSeck-V3
Domeon Lal
Action Lid
Gemini 2.0
Gemigi..d0io
Human Experts
Viceo Contest
Multple-cholon
Portorms
Fitzring
Filteiing
Fiiral
Figure 2: The pipeline diagram of the proposed multi-expert annotation framework.
3.1 VIDEO COLLECTION
We define 11 domains for video collection, including cartoon, game. movie/TV show, virtual envi- ronment, egocentric, surveillance, shaky, foggy, night, rainy, and snowy. Then, collect videos from various data sources based on the domain name. We employ Qwen2.5-VL.-7B (Bai et al. 2025) to filter irrelevant videos that do not belong the predefined domains. To ensure semantic consistency across domains, we manually define a list of daily human activity scenes (e.g. reading books or documents, riding bicycle, feeding pet etc.) and utilize Qwen2.5-VL-1 to select videos belonging to this predefined activity list. The activity list and the prompts that are used to select videos are detailed Appendix
To climinate potential data leakage (since LVLMs are pre-trained on large amount of data), we create training and testing sets for each domain and ensure clear separation between them by collecting them from different data sources.
Training Set: The training set constructed exclusively from the training sets of existing open- source datasets, including Intern Vid (Wang al. 2024a), ShareGPT4Video (Chen et al. (024), VideoInstruct (Maaz et al., 2024). and MMDI. (Ugai et al. 2024). We have checked that no data from these sources overlaps with existing benchmarks (used as the testing set), ensuring striet separation between the training and testing data in VUDG.
Testing Set: The testing is primarily derived from the testing sets of existing open-source video datasets, benchmarks, and videos crawled from online platforms. Specifically, we use the test splits of VATEX (Wang al. 2019), ActivityNer (Caba Heilbron et al. 2015), VideoVista (Li et al.. 2024b) and MMDL (Ugai al.) 2024). Furthermore, leverage diverse user- serr-genertted content from online video platforms such as YouTube, Douyin, and Bilibili. These platforms host rich and varied videos from different domains, enabling
collect broad spectrum of videos. Table 2 shows the proportion of each source of the test set videos.
Table 2: Proportion of video sources in the test set.
Dutasets
VATEX
ActivityNet
MMDL
VidcoVista
Collect
Proportion
15.17%
22.02%
11.78%
1.41%
49.62%
All subsequent processes, such as QA pairs generation and filtering, are applied uniformly to all the collected videos, without further distinguishing between the training set and the testing set.
3.2 QUESTION AND ANSWER GENERATION
The question and answer generation process comprises three steps: open-ended QA pairs generation, multiple-choice QA pairs generation, and QA pairs screening and reviewing.
Design of Question Category: The generated QA pairs expected to enable thorough and comprehensive evaluation of video understanding. Figureelllusttttttttrr four types of questions: (1) Action Recognition, which focuses identifying action categories, requiring the model to accurately
Billustrate
4

---

Published as a conference paper at ICLR 2026
Multiple Domains
Question Categorios
Epecentric
whet does the do their hand pahm?
(1) Action Recegnition
(3) Objject Edcutification
The hise guiturboeyy
semonce daing
Q: dhey poaring osto mocdles?
yellow anslrocus?
Mavie/TV Shew
Curting yellow nusterons
A: Chopped scallions.
What acticn the
white perform after eitting
kep.
Attribbte Idencification
Temperal Understandng
Q. What color are dee sax yollo frying Vellow.
Q Aller the socdles are cooked, what
What azion dees Aninaal perfom on the drums!
added them!
N. Aitiial Res the drans, eytthals, and moves.
A: muderoms egg.
Night
Or Wist nation the mas perferiing the soccer Geld!! A: He playing visliz.
Figure 3: Overview of various domains and question types in our VUDG dataset.
recognize the action occurring at a specific time point in the video; (2) Attribute Identification, which assesses the model's ability to perceive visual attributes such color, shape, and position of simple objects; (3) Object Identification, which tests the model's ability to recognize specific objects; (4) Temporal Understanding, which involves the temporal ordering of actions and requires the model to accurately identify the sequence of events, i.e., the event that occurs before or after specific action.
Open-Ended QA Pairs Generation: To generate open-ended QA pairs, first leverage Gemini- 2.5-Flash, a fast and cost-effective multimodal model with strong video understanding capabilities. This model is used to generate initial questions and open-ended answers for each video. We design two distinct prompts for question categories (1)-() and question category (4), respectively, since they focus different information cues. Detailed examples of the two prompts can be found in Appendix J.T For each video, we generate one question for each of the question categories (1)-(3) and two questions for the question categories (4).
For the generation of multiple-choice QA Pairs, utilize DeepSeek-v3 to generate five plausible but incorrect options for each open-ended QA pair. These options are conditioned on the original question and the correct answer. Afterward, the options are randomized to ensure a balanced distribution of all six choices. Examples of prompts for multiple-choice QA generation be found in Appendix J.2
Multiple-Choice QA Pairs Generation:
QA Pairs Screening and Reviewing: Despite the structured pipeline used for QA pair generation, issues such as ambiguous phrasing, semantically overlapping options, and factual inaccuracies in open-ended answers may still arise, potentially compromising the quality of the generated pairs. To address these challenges, we introduce a hybrid screening process that integrates both automated model-based evaluation and human-expert review. First, employ Gemini-2.5-Prre a more advanced multimodal model, to perform a thorough review of each QA pair with access to the original video context. This model classifies each QA pair into one of three categories: (a) correct QA pairs, (b) partially flawed answers with fixable issues, and (c) invalid questions. These automated classifications serve as the basis for further manual inspection. Then, human experts revise or remove QA pairs that are flagged as problematic ((b) or (c)) by Gemini-2.5-PPre ensuring that the final dataset maintains high standards of clarity and accuracy. Crucially, this mandatory human-in-the-loow refinement, combined with our multi-model cascade strategy, effectively breaks the potential circularity loop and mitigates the risk of LLM dependence in dataset construction. Detailed prompts for Gemini-2. 2.5-ni--..-Po can be found in Appendix J.3
3.3 STATISTICS
The training set comprises 6,337 video clips and 31,685 QA pairs. The distribution of videos across domains is illustrated in Figure 4a To reduce memory usage during training, all training videos are limited maximum duration of ten minutes. The duration distribution shown in Figure 5a
The testing set contains 1,532 video clips and 4,703 QA pairs, and the distribution of video numbers in each domain is demonstrated in Figure Compared to the training set, the testing set includes longer videos to better evaluate each model's 's ability to handle complex and extended temporal contexts. The duration distribution is shown in Figure 5b
5

---

Published as a conference paper at ICLR 2026
BABK
IawieTV Jtew
FEKOIEX 1ide
Rpreetric raka
rdee
m0
mahr
Amall
1arvellace
(a) VUDG Training Set
(h) VUDG Testing Set
Figure 4: Statistics showing the number of QA pairs across different domains in (a) the VUDG training set and (b) the VUDG testing set.
Duration
Training
Testing Set
1500
3144
2000
1500
300
1880
219
109
1.3
5-10
10-15
13-20 Over
30
Daration (ntiaule)
Duration (mrinste))
(a) VUDG Training Set
(b) VUDG Testing Set
Figure 5: Statistics illustrating the distribution of video durations across (a) VUDG training set and
(b) VUDG testing set.
3.4 EVALUATION METRIC
We adopt two evaluation protocols widely used in domain generalization research: Leave-One Domain-Out for multiple-domain generalization and generalization, following prior works (Jo & Yoon 2023 Chen al. 2023).
Satee--DmainOOu for single-domain
For multiple domain generalization, one domain is used the target domain using its testing set, while the training sets of the remaining N domains are treated as source domains for training.
The final performance is averaged over all domains, formulated as Avg" = Σ P,, where P; denotes the accuracy of the i-th domain used as the target domain, and N is the number of domains in each setting.
For single domain generalization, the training of one domain is used the source domain for training, and the left 1 domains are treated as target domains using their testing sets. The performance this domain is calculated the average of the accuracy on the left N 1 domains, formulated as Avg" + Ei ( Σ Pj), where Pj denotes the accuracy of the j-th domain with the i-th domain as the source domain.
For zero-shot generalization, models are directly tested on the full testing sets without any training. For multiple-choice questions, each model receives the video input alongside textual indications, including the question and the candidate options. We then compute the accuracy of a model on each domain and quantify the final performance by averaging questions, we use DeepSeek-V. V3 to automatically evaluate the answers. Specifically, we adopt task- specific evaluation protocols to evaluate from two aspects, with a total score of 5 points for each aspect and maximum score of 10 for each question. For Q1, Q2 and Q3 (Action Recognition, Attribute Identification, and Object Identification), answers are evaluated based on factual accuracy and relevance to the question. For Q4 and Q5 (Temporal Understanding), which emphasize temporal reasoning, answers are assessed based on temporal accuracy and question relevance. The final score for question is Score Stace + Srel where Suace S E 50. Evaluation prompts for all question types are detailed in Appendix J.4, while training configurations and model-specific hyperparameters for each evaluated model are included in Appendix
results on all domains. For open-ended

---

Published as a conference paper at ICLR 2026
4 EXPERIMENTS
4.1 SETTINGS
Baseline: For domain generalization setting, evaluate L.L.-based and non-L.LM-based methods. HBI (Jin et al.
2023) and EMCL4QA (Jin et al. 2022) are representative VideoQA methods that do not rely on LLMs, while VideoLLaMA2-711 (Cheng al. 2024) and Qwen2.5VL-3B (Bai et al.) 2025) are popular LVLMs using LLMs. We evaluate their performances on VUDG with the multiple and single domain generalization settings. For zero-shot evaluation, we evaluate nine large-scale video understanding models, including
(Maaz et al. 2024), MiniGPT4-Video (Ataallah) ett al. 2024), VideoChar2-7B (Li et al. 2023), Video-LLaVA-73 (Lin et 2023a), LadeoLLaMAA-
7B (Cheng et al. 2024), MPLGG-OW---G
O-B--B- YYe et al. 2025). Tideo-CCAMa (Fei et al.
2024).
VideoLLaMA3-77 (Zhang et al. 2025), Tarsier2-7B
Yuan et al. 2025 and Qwen2.5VL- 7-B (Bai
ct al. 2025
For training on the VUDG training set, we apply full-parane parameter fine- tuning non-L- methods and Low-Rank Adaptation (LoRA) (Hu et al., 2022 to LVLMMs to ensure training efficiency. For LoRA, we set the rank to 128 and the scaling factor to 256. Further training details of each model can be found in Appendix E For the zero-shot evaluation setting, we set all LVLMs to the official default configuration. Regarding models employing fixed frame sampling, we adopt their default official settings (e.g., Video-LLaMA2 uses 16 frames per video). Regarding models evaluated under fixed FPS settings, we uniformly set the frame rate to I FPS to ensure consistency.
Implementation Details:
4.2 DOMAIN GENERALIZATION RESULTS
In the following experiments, we use abbreviations for different domains: CA (Cartoon), GA (Game), MO (Movie/TV), VI (Virtual), EG (Fgocentric), SU (Surveillance), SH (Shaky), FO (Foggy), NI (Night), RA (Rainy), SN (Snowy), and D-Avg (Domain-wise Average). We first provide the fully finetuning results on all domains, which can serve as upper bound for the performance of different methods on VUDG, The results are presented in Table 3
Table 3: Performance of full fine-tuning across all domains on multiple-choice QA.
Model
CA
GA
MO
Style
Avg
EG
SU
Viewpoint SH
Avg
FO
NI
Condition
RA
SN
Avg
D-Avg
VadeoLLaMA2-7B
Cheeg vl 2024
65.4
6144
72.5
71.1
67.6
74.7
68.8
68.5
70.6
69.4
69.7
62.8
73.1
68.7
68.8
5VL Bai
71.8
62.1
71.5
70.8
69.0
76.2
69.2
66.1
70.5
66.3
67.6
65.8
70.2
67.5
68.9
We first evaluate HBI (Jin et al. 2023), EMCL4QA (Jin et al. (022), Vide (Cheng et al., 2024), and Qwen2.5Vi. (Bai et al. (2025) in single domain generalization setting under the domain shift caused by the difference of environment condition, i.e., the Environmental Condition split of the VUDG dataset. As shown in Table 4 the performance of VideoLLaMA2 degrades 15.4 percentage points compared with that of full fine-tuning, indicating that single domain generalization poses a great challenge to these models.
Table 4: Single domain generalization results on multiple-choice QA under different domain shifts.
Visual Style
Viewpoint SH
Eav, Condition
Model
CA
GA
MO
VI
Avg'
BG
SU
Avg'
PO
NI
RA
SN
Avg'
D-Avg
EMCL4QQ Jin et al
2023
1844
16.7
18.5
17.9
17.9
19.0
17.4
16.4
17.6
18.8
18.2
17.1
1688
17.7
17.7
HRIllin
189
16.9
17.9
18.2
18.0
17.9
19.3
18.6
18.2
17.9
1o ent hene at 2024
55.2
48.7
48.6
52.5
53.1
56.6
58.3
53.4
5VL Ba af
63.3
66.5
66.1
64.6
65.1
59.8
61.7
63.4
61.6
57.2
58.4
57.3
58.4
57.8
61.5
Table 5 summarizes the performance of four VideoQA methods in multiple domain generalization setting. We test these methods under three types of domain shifts and record the average results of each type of domain shift. As shown in Table 5 we have several findings. Firstly, non-I.L.M based methods (HBI (Jin et al. and EMCL4QA (Jin et al.) (022)) achieve poor performance, indicating their limited generalization ability when exposed to -iitt-ddistriuttion domains. Secondly, LLM--ased methods show significantly better generalization compared to non-LLM-based methods. Among them, VideoLLaMA2-7B outperforms all others with the highest average accuracy across Visual Style (66.5%), Viewpoint (66.2%). and Environmental Condition (68.2%). Qwen2.5VL-3B shows competitive results but suffers notable degradation under harsh environmental conditions (e.g.. only 55.9% NI and 55.5% SN), suggesting vulnerability to visual noise and degradation.
7

---

Published as a conference paper at ICLR 2026
Despite VideoLLaMA227B'ss strong performance under multiple DG settings, clear gap remains compared to models fin-ttued across all domains (Table 3). Meanwhile, Qwen2.5-VL-33 shows worse performance multiple DG fine-tuning than its fully fine-tuning counterpart. These results suggest that current large vision-language models (Ls) require more robust training or fine-tuning strategies enhance domain generalization when adapting downstream tasks.
Table 5: Multiple domain generalization results on multiple- choice QA under different domain shifts.
Model
GA
MO
Avg"
SU
Vieupoint SH
Aug"
FD
Condition RA
SN
Avg"
D-Ang
HBBI Jin al 2025
14.9
182
17.2
16.4
16.7
17.4
16.7
18.5
17.5
17.7
18.9
16.9
17.6
17.8
17.3
EMC Tagx bireral m 5023
17.7
18.7
16.8
17.7
17.7
17.4
16.7
19.6
17.9
19.1
18.4
18.8
19.3
18.7
18.1
Qwen2.VV 56 ner 2005
70.5
61.7
62.0
66.2
65.0
65.6
61.2
57.7
61.5
61.0
55.9
59.5
55.5
58.0
61.
Vidool LMA2 75
2024
61.6
64.4
68.7
70.1
66.5
66.1
62.9
69.6
66.2
69.6
69.1
64.9
69.2
68.2
66.9
4.3 ZERO-SHOT RESULTS
Table 6: Multiple-choice zero-shot test results VUDG. Performance across 11 domains.
Visual Style
Vieupoint
Env. Condition
Avg
Avg
Avg
D-Avg
Malco-CanGPT-7 Muaz Musz al. 2024
14.1
12.5
14.4
9.7
12.7
12.9
11.6
14.1
12.9
14.5
16.3
13.8
15.8
15.1
13.6
MiniGPT4-Viel
Almmiinn 2014
13.6
12.9
13.9
14.2
13.7
12.7
13.2
13.8
13.2
15.9
15.7
13.0
13.0
14.4
13.8
VideoChat2-773 7B eL F0re3
16.2
14.1
10.3
12.6
14.6
13.8
13.4
13.9
16.7
15.4
17.8
11.6
15.4
14.0
Video-LLaVA-72
VodcoLLaMA2271
Clene 2024
31.5
34.0
31.7
304
31.9
34.6
34.5
39.6
36.2
33.4
34.7
30.5
32.2
32.7
33.4
mPL.UGG-Dw/3-78
Yeelal 20755
50.0
508
49.7
61.0
5299
53.5
46.8
56.7
$2.3
51.0
49.2
48.7
48.2
49.3
5144
Viden-CCAM-77
Fer 2012
58.6
40.9
60.0
52.7
$2.3
54,8
47.0
57.1
53.0
51.0
51.0
48.3
47.8
49.5
$1.5
VideoLLaMMA-775 12simi
Tarsier2-78 Yuse ml
64.6
56.8
59.7
75.4
64.1
66.0
64.01
63.8
64.6
57.7
63.
50.5
60.5
60.3
62.8
Qven2.5V1--7 (50) m1. 2025
71.3
61.4
72.3
75.4
70.1
79.8
73.3
68.1
73.7
77.2
69.7
71.81
73.7
72.9
72.1
GPT-40 (16 frames)
64.6
64.0
68.6
73.2
67.6
66.5
60.8
68.8
65.4
61.3
62.0
61.0
59.8
61.0
64.6
In this section, we report the zeoo evaluation results on the testing set of VUDG. Domain-specific performance on multiple-choice QA pairs is presented in Table 6
We provide detailed results as well as domain- wiie averages (D-Avg) across 11 domains. Table presents the accuracy of multiple-choice questions categorized by question type.
Multiple-Choice QA: As shown in Table 6 Qwen2.5VL-77B achieves the highest average accuracy (72.1%) across 11 visual domains, demonstrating strong generalization. Meanwhile, the advanced closed-source model GPT-4o yields a lower accuracy of 64.6%, indicating that massive pre-training alone does not fully mitigate the challenge of domain shifts. Notably, models like Video-LLaMAA-78 show moderate performance but struggle on viewpoint shifts such as Surveillance (SU) and Shaky (SH) scenes. Earlier models like Video-ChatGPT-7B and MiniGPT4- Video exhibit much lower performance overall, suggesting limited robustness to various distribution shifts.
Table 7: Various question categories of multiple-choice test results on VUDG. Q1 (Action Recogni- tion), Q2 (Attribute Identification), Q3 (Object Identification). Q4 & Q5 (Temporal Understanding).
Model
Q1
Q2
Q3
Q4 & Q5
Overall
VadooChaiGGTT-P Mauz al 2024
11.6
13.4
12.7
14.7
13.6
VideoChar2-7D (L)
p02s)
15.3
13.7
13.7
MiniGiPA4VVidll Aiariin el 2024
14.4
11.8
14.9
14.1
13.8
Vidoo-LaVV-77 Li 20133
30.1
28.2
22.1
21.0
24,1
VideoLLaMA2-73 henng et
35,3
36.2
30.2
33.3
ValcoCCAMM-7B
(For 2024
53.8
55.0
58.0
47.1
$1.5
mPLUGG-OW337
Te nL. 2012
55.2
53.8
58.9
46.8
51.6
Vidaol.I
64.3
75.2
65.4
Qwen2.5VL-7B Bai ct 2005
73.7
77.3
80.8
67.7
72.7
GPT (16 frames)
65.9
71.9
75.0
57.8
64.7
Performance Across Question Categories: To further understand model behavior, Table breaks down performance by question category. Qwen2.5VL-78 achieves the best performance across all categories, with particularly strong results in Object Identification (Q3) and Temporal Understanding (Q4 & Q5). In addition, the performance of most models in action recognition, attribute identifica- tion, and object identification is generally better than that in temporal understanding. This pattern suggests that current L.LMMs are better at static, appearance-baal based reasoning than dynamic temporal understanding, revealing a challenge for video-based generalization.
Open-Ended QA:
We also evaluate the zero-shot performance of different methods open-ended QA. As shown in Table Video-CCAM achieves the highest domain average (6.84), followed closely by Qwen2.5VL-7B and mPL.UG-OW1337B. Compared to the multiple-choice setting, performance
8

---

Published as a conference paper at ICLR 2026
gaps among models narrow in the open-ended QA setting, suggesting increased difficulty in generating free-form answers under domain shifts.
Table 8: Open-ended zero-shot test results on VUDG. Performance across 11 visual domains.
Model
CA
Visual Style MO
V1
Viewpoint
SU
St1
FO
NI
Condition Fan.
SN
Avg
D-Avg
MiniCiP4-VVaknn
at
2024
4.47
4.42
3.91
4.56
4.47
4.32
Video ChaiGPT
p021
VideoChu2-7B Li patr 2074
5.22
5.71
5.82
5.52
6.12
544
5.88
5.85
5.77
5.96
5.17
5.88
5.70
3.67
Videol
2025
5,57
5:35
5.80
3.91
5.66
3.77
5,64
5.3
5:78
5.61
3.75
5.30
5.83
5.07
5.70
Video11 1V
5.30
5.22
5.81
6.20
5.66
6.32
5.66
6.11
6.33
5.80
5.99
5.42
6.03
5.61
581
VideoLLLMMA2-11
Ce 2024
5,80
5.9
6.32
6.68
6.18
6.88
6.05
6.65
6.53
6.46
6.55
6.12
6.45
6.40
6.35
mPLUG-OO(13-7::
6.07
6.10
6.44
6.98
6.40
7.32
6.07
6.92
6.77
6.75
6.84
6.33
6.76
6.67
6.60
Owen2.5VL-2B
6.53
7.02
6.66
6.79
6.79
Vika-CCAM-77
4.4 ANALYSIS ON TEXTUAL DOMAIN SHIFT
Beyond our primary focus on visual shifts, we conduct an auxiliary experiment to probe for textual domain shifts arising from different question formats. We evaluate Videol.LaMA2 on our multiple- choice (MC) test set under two fine-tuning conditions: using our open-ended (OE) training data versus MC training data. As shown in Table | the model fine-tuned on OE data achieves 50.0% accuracy, a substantial 18.8 percentage point drop compared to the 68.8% from the model trained on in-domain MC data. This performance gap confirms the presence of significant textual domain shift, demonstrating that linguistic variations alone can hinder model generalization.
4.5 HUMAN STUDY:
To validate our annotation quality, we conduct a human study. Since the multiple-choice QA pairs are generated from open-ended QA, we randomly sample 300 open-ended QA pairs and task independent evaluators with rating them on 5-point scale (I=Poor, 5=Excellent) for two key criteria: Relevance and Correctness. For comparison, we establish two control groups: (1) "random match" denotes randomly pairing questions with answers from other QA pairs, and (2) "filtered" represents QA pairs that were generated but subsequently discarded during the human review phase. As shown in Table 10 our annotations achieve substantially higher scores than both control groups, affirming the quality and reliability of the VUDG dataset.
Table 9: Performance Vid2 on the multiple- choice (MC) test set under different training conditions.
Table 10: Human evaluation of annotation quality.
Method
Accuracy (%)
QA Pairs Type
Relevance
Correctness
Lade (zcro-shot)
33.4
Random-Matched
1.23
1.45
LaM2 (fine-tuned on OE)
50.0
Filtered
2.36
3,58
VideoLLuMA2 (fine-tuned on MC)
68.8
Ours
4.78
4.62
4.6 VISUALIZATION
We visualize the embeddings of video frames, questions, and ground-truth answers using CLIP- B/16 (Dosovitskiy et al. 2021) for image encoding and BGE-V1.5 (Xiao et al.] (024) for text encoding. As shown in Fig. [6, the CLIP frame features (a) exhibit relatively clear clustering by domain, indicating significant visual differences across domains and validating the challenge of domain shifts in VUDG. In contrast, the distributions of question embeddings (b) and answer embeddings (c) are more uniformly mixed, suggesting that the semantic content remains consistent across domains. This supports our design principle of preserving cross-domain semantic consistency while introducing realistic visual shifts.
CONCLUSIONS
We present VUDG, a novel dataset for evaluating domain generalization in video understanding while maintaining semantic consistency across 11 diverse domains, enabling fair and challenging evaluation. To construct high-quality QA pairs specifically for domain generalization, we develop a progressive multi-expert annotation pipeline that benefits from multiple large models together with human expert refinement. Through extensive experiments, we observe that current models struggle to address domain shifts when adapted to downstream tasks, resulting in suboptimal generalization. As for zero-shot evaluation, the performance of different LVLMs varies greatly, while even state-of-the-an L.LMS exhibit inconsistent performance between domains. We believe VUDG provides valuable

---

Published as a conference paper at ICLR 2026
cartoon
right
movie/TV
surve lance
egocentric
shaky
rainy
geme
foggy
(a) GLIP frame fectures
(b) Question embeddings
(o) GT answer embeddings
Figure 6: The t-SNE visualization of CLIP features for video frames, and BGE embeddings for questions and ground-truth answers across different domains.
foundation for advancing generalizable video understanding. However, our dataset currently only focuses on visual domain shifts. In the future, we plan to explore textual domain shifts and incorporate additional modalities such audio to enable more comprehensive multimodal generalization.
ACKNOWLEDGEMENTS
This work was supported by the Natural Science Foundation of China (NSFC) (Grant No. 62406009), the Shenzhen Science and Technology Program (Grant No. 022202222221105444500 Science Foundation of Shenzhen (Grant No. JCYJ202308071 (22700000)))) the Guangdong Provincial Key Area Project of General Universities (Grant No. 20242DDZX0017), and the Opening Project of the State Key Laboratory of General Artificial Intelligence, BIGAI/Peking University, Beijing, China (Grant No. SKLAGI20250P03, SKLAGI2025PPS8,
the Natural
REPRODUCIBILITY STATEMENT
The VUDG dataset and the codehase to reproduce experiments are made publicly available.
REFERENCES
Kirolos Ataallah, Xiaoqian Shen, Eslam Abdelrahman, Essam Sleiman, Deyao Zhu, Jian Ding, and Mohamed Elhoseiny. Minigpt4-video: Advancing multimodal llms for video understanding with interleaved visual-textual tokens. arXiv preprint arXiv:2404.0344 2024.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report. arXiv preprint 2rXiv: 2025.
Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles. Activitynet: A large-scale video benchmark for human activity understanding. In Proceedings of the ieee conference on computer vision and pattern recognition, pp. 961-970, 2015.
Jiawei Chen and Chiu Man Ho. Mm-vit: Multi-modal video transformer for compressed video action recognition. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 1910-1921, 2022.
Jin Chen, Zhi Gao, Xinxiao Wu, and Jiebo Luo. Meta-causal learning for single domain generalization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. (683-7692, 2023.
Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Zhenyu Tang, Li Yuan, et al. Sharegpt4video: Improving video understanding and generation with better captions. Advances in Neural Information Processing Systems, 37:19472 19495, 2024.
Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, and Lidong Bing. Videollama 2: Advancing spatial-temporia modeling and audio understanding in video-Ilms. arXiv preprint arXiv:2406.07476 2024. URL fttps://aaxiis wtg//bs////a /240 07476
10

---

Published as a conference paper at ICLR 2026
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICIRR, 2021.
Jiajun Fei, Dian Li, Zhidong Deng, Zekun Wang, Gang Liu, and Hui Wang. Video-ccam: Enhancing video-language understanding with causal cross- -ooss-tttnntion masks for short and long videos, 2024. URL ponttps://arxtt wrg. abs /taaa///2000 2408 1 14023
Alessandro Flaborea, Luca Collorone, Guido Maria D' Amely Di Melendugno, Stefano Arrigo, Bardh Prenkaj. and Fabio Galasso. Multimodal motion conditioned diffusion model for skeleton- based video anomaly detection. Proceedings of the IEEE/CVF international conference on computer vision, pp. 10318-10322 2023.
Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang. et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal Ilms video analysis. arXiv preprint arXiv:2405.21075. 2024.
Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 18995-19012, 2022.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
Yunseok Jang. Yale Song, Youngjae Yu, Youngjin Kim, and Gunhee Kim. Tgif-qa: Toward spatio- temporal reasoning visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2758-2766, 2017.
Peng Jin, JinFa Huang, Fenglin Liu, Xian Wu, Shen Ge, Guoli Song, David A. Clifton, and Jie Chen. contrastive learning for compact video-and- video-and-language language representations. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances Neural Information Processing Systems, volume 35, pp. 30291-30306. 2022.
Peng Jin, Jinfa Huang, Pengfei Xiong, Shangxuan Tian, Chang Liu, Xiangyang Ji, Li Yuan, and Jie Chen. Video-text as game players: Hierarchical banzhaf interaction for cross cross-modal representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 2472-2482, 2023.
Sang-Yes Yeong Jo and Sung Whan Yoon. Poem: polarization of embeddings for domain- invariant repre- sentations. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), volume 37, pp. 8150-8158, 2023.
Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green. Trevor Back, Paul Natsev, al. The kinetics human action video dataset. arXiv preprint wwXiiv/705.0690 2017.
Jie Lei, Licheng Yu, Mohit Bansal, and Tamara Berg. Tvqa: Localized, compositional video question answering. arXiv preprint arXiv:/809.01696, 2018.
KunChang Li. Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wan, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprini arXiv:2305.06355. 2023.
Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mvbench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference Computer Vision and Patlern Recognition, pp. 22195-22206, 2024a.
Yicong Li, Xiang Wang, Junbin Xiao, Wei Ji. and Tat-Seng Chua. Invariant grounding for video question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2022.
11

---

Published as a conference paper at ICLR 2026
Yunxin Li, Xinyu Chen, Baotian Hu, Longyue Wang, Haoyuan Shi, and Min Zhang. Videovista: A versatile benchmark for video understanding and reasoning. arXiv preprint arXi::2466.11303 2024b.
Bin Lin, Bin Zhu, Yang Ye, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311. 2023a.
Kevin Lin, Linjie Li, Chung-Ching Lin, Faisal Ahmed, Zhe Gan, Zicheng Liu, Yumao Lu, and Lijuan Wang. Swinbert: End-to-end transformers with sparse attention for video captioning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
17949-17958, 2022.
Kun- Yu Lin, Jia-Run Du, Yipeng Gao, Jiaming Zhou, and Wei-Shi Zheng. Diversifying spatial- temporal perception for video domain generalization. Advances in Neural Information Processing Systems, 36:56012-56026, 2023b.
Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. Tempcompass: Do video ilms really understand videos? arXiv preprint .rXiv:2403.004476 2024.
Ziqian Lu, Fengli Shen, Mushui Liu, Yunlong Yu, and Xi Li. Improving zero-shot generalization for clip with variational adapter. In European Conference on Computer Vision (ECCV), pp. 328-344. Springer, 2024.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models, 2024. URL https
7/arxiv.
05424
Shoumik Sovan Majumdar, Shubhangi Jain, Isidora Chara Tourni, Arsenii Mustafin, Diala Lteif, Stan Sclaroff, Kate Saenko, and Sarah Adel Bargal. Ani-gifs: A benchmark dataset for domain generalization of action recognition from gifs. Frontiers in Computer Science, 4:876846, 2022.
Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. Egoschema: diagnostic benchmark for very long-form video language understanding. Advances in Neural Information Processing Systems, 36:46212-466 462444 2023.
Munan Ning, Bin Zhu, Yujia Xie, Bin Lin, Jiaxi Cui. Lu Yuan, Dongdong Chen, and Li Yuan. Video-bench: A comprehensive benchmark and toolkit for evaluating video-based large language models. arXiv preprint arXiv:2311.16105 2023.
Antonios Papadakis and Evaggelos Spyrou.
multi- egocentric activity recognition approach towards video domain generalization. Sensors, 24(8):2491, 2024.
Chiara Plizzari, Toby Perrett, Barbara Caputo, and Dima Damen. What can cook in italy teach mechanic in india? action recognition generalisation over scenarios and locations. Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 13656-13666, 2023.
Gunnar A Sigurdsson, Gül Varol, Xiaolong Wang, Ali Farhadi, Ivan Laptev, and Abhinav Gupta. Hollywood in homes: Crowdsourcing data collection for activity understanding. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part I 14, pp. 510-526. Springer, 2016.
Takanori Ugai, Kensho Hara, Shusaku Egami, and Ken Fukuda. Multimodal datasets and bench- marks for reasoning about dynamic ppatio-temporalityy in everyday environments. arXiv preprint arXiv:2408. 11347, 2024.
Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, and Tao Qin. Generalizing to unseen domains: A survey domain generalization. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (ICAII, pp. 4627-4635. International Joint Conferences on Artificial Intelligence Organization, 2021.
Xin Wang, Jiawei Wu, Junkun Chen, Lei Li. Yuan- Fang Wang, and William Yang Wang. Vatex: A large-scale, high-quality multilingual dataset for aedoo-and-lanuuse research. In Proceedings of the IEFF/CVF international conference on computer vision, pp. 4581-4591, 2019.
12

---

Published as a conference paper at ICLR 2026
Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang. Ping Luo, Ziwei Liu, Yali Wang, Limin Wang, and Yu Qiao. Internvid: A large-scale video-text dataset for multimodal understanding and generation. In The Twelfth International Conference on Learning Representations, 2024a.
Zhiqiang Wang, Xiaojing Gu, Huaicheng Yan, and Xingsheng Gu. Domain generalization for video anomaly detection considering diverse anomaly types. Signal, Image and Video Processing, 18(4): 3691-3704, 2024b.
Ziyi Wang, Zhi Gao, Jin Chen, Qingjie Zhao, Xinxiao Wu, and Jiebo Luo. Simulate, refocus and ensemble: An attention-refoccsing scheme for domain generalization. IEEE Trunsuctions on Multimedia, pp. 1-13, 2026.
Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. C-pack: Packed resources for general chinese embeddings. In Proceedings of the 47th International ACM SIGIR Conference Research and Development in Information Retrieval (SIGIR), 2024.
Dejing Xu, Zhou Zhao, Jun Xiao, Fei Wu, Hanwang Zhang, Xiangnan He, and Yueting Zhuang. Video question answering via gradually refined attention over appearance and motion. In Proceedings of the 25th ACM international conference on Multimedia, pp. 1645-1653, 2017.
Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Msr-vtt: A large video description dataset for bridging video and language. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 5288-5296. 2016.
Zhiyu Yao, Yunbo Wang, Jianmin Wang, Philip S Yu, and Mingsheng Long, Videodg: Generalizing temporal relations in videos novel domains, IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11) (11111)9999-00000 2021.
Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. mPLUG-ow13: Towards long image-sequence understanding in multi-modal large language models. In The Thirteenth International Conference Learning Representationss, 2025.
Zhou Yu, Dejing Xu, Jun Yu, Ting Yu, Zhou Zhao, Yueting Zhuang, and Dacheng Tao. Activitynet-qa: A dataset for understanding complex web videos via question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 9127-9134, 2019.
Liping Yuan, Jiawei Wang. Haomiao Sun, Yuchen Zhang, and Yuan Lin. Tarsier2: Advancing large Dision-language models from detailed video description to comprehensive video understanding. arXiv preprint orXiv:2501.0777 .07888, 2025.
Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, Peng Jin, Wenqi Zhang, Fan Wang, Lidong Bing, and Deli Zhao. Videollama 3: Frontier multimodal foundation models for image and video understanding, 2025. URL htttst:tts///trt
.rg.i.oorgraaaee. /bba/ /200 13106
Shengyu Zhang, Xusheng Feng. Wenyan Fan, Wenjing Fang, Fuli Feng, Wei Ji, Shuo Li, Li Wang, Shanshan Zhao, Zhou Zhao, et al. Video-audio domain generalization via confounder disen- tanglement. Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 15322-15330,, 2023.
13

---

Published as a conference paper at ICLR 2026
A
DOMAIN DEFINITIONS
The definitions of all domains are shown in Table
Table 11: Domain definitions grouped by distribution shift types, along with descriptions.
Shift Type
Domain
Description
Cartoon
Stylized animation with exaggerated motion simplified texture
Scenes from video games often rendered real-time
Visual Style
Movie/TV
Professionally shot narrative content with cinematic framing
Virtual
Fully simulated environments, often from virtual production
Egocentric
First-persoo pist-pprson perspective, often from head-mounted
Viewpoint
Surveillance
Static, wide-angle views from mounted cameras in public/private spaces
Shaky
Handheld, unstable footage with dynamic camera motion
Foggy
Reduced visibility due to simulated natural fog
Night
Low-light nighttime scenes, often under artificial lighting
Environmental Condition
Rainy
Ouldoor scenes with visible rain, wel surfaces, and overcasi skies
Snowy
Scenes snowfall, snow-covered ground, and diffused light
B DATA COLLECTION
We present the prompt for Qwen2.5-VL VL-7BB used to select videos that belong to the specific activity list in Figure 7 and the types of daily actions that the VUDG dataset focuses shown in Table
Prompts For Download
"Below, will provide you set of phrases describing scenes of people's daily actions, Please check the video includes any scenes from the provided set. If so, reply 'yes', otherwise answer 'no'. Do not answer any additional content."
Figure 7: Prompts for download.
The list of human daily activity scenes containing 37 items that the VUDG dataset focuses on is detailed in Table 12
Table 12: Human Daily Activity Scenes List with Descriptions
Activity
Description
Sitting
Sitting on couch chair
Using Laptop
Using laptop or other electronic devices (including typing on a key-
board)
Writing/Drawing
Writing or drawing in a notebook (including painting)
Cooking
Cooking or preparing food (chopping, cutting, mixing, stirring)
Cleaning Dishes
Cleaning dishes or household items; organizing spaces Gardening or plant care (watering, tending, pruning) Exercising stretching (yoga, home workouts, outdoor) Brushing or styling hair (combing, etc.)
Gardening
Exercising
Hair Styling
Reading
Reading books documents
Hygiene
Washing hands or doing hygiene routines (brushing teeth) Attending meetings presentations (online in-person) Taking notes during meeting
Meeting
Taking Notes
Drinking
Drinking water, coffee, tea, other beverages
Preparing Meals
Preparing meal or snack (lunch, dinner)
Using Phone
Using phone or tablet (checking notifications)
14

---

Published as a conference paper at ICLR 2026
Activity
Description
Walking Outdoors
Walking in a park or natural area Adjusting or fixing household items
Fixing Items
Feeding Pets
Feeding a pet
Art/Crafting
Engaging
or crafting activities
Device Setup
Setting up or adjusting electronic devices Checking or updating calendar or planner Watching television or screens (incl. gaming) Resting or lying down Preparing tools or items for a task Cleaning or organizing room (tidying clutter) Sorting laundry folding clothes Taking a break from a task or activity
Checking Calendar
Watching Screens
Resting
Tool Preparation
Organizing
Laundry
Taking a Break
Conversation
Engaging in discussions conversations
Shopping
Shopping or browsing items Driving
Driving
Eating
Eating a meal
Playing Toys
Playing with a toy
Makeup
Applying makeup
Dancing
Dancing
Biking
Riding bicycle
Music
Playing musical instrument
Commuting
Commuting via public transport
We also visualize the distribution of the number of QA pairs across different activity scenes in the test set Figure
Activity Scanes distribution
250
Namber of
and pottggmnn coceeettin coafee
Phowing
Besting
satii the
Aetivity Seuney
Figure 8: Distribution of the number of QA pairs across different activity scenes.
C ANALYSIS ON STATIC BIAS
We conducted an in-depth analysis of language bias and visual bias in our constructed VUDG dataset, evaluating the Tarsier2-7B model under three input settings: text-only, one-frame sampling, and full-video input. Experimental results in Table 13 show that when relying solely question text, the model achieves only 20.6% average accuracy, close to random guessing (theoretical baseline: 16.7%), indicating that the dataset contains minimal exploitable linguistic cues, effectively mitigating language bias. Meanwhile, performance under the one- one-frame pre-frame setting (40.3%) is notably inferior to that of full- video input (62.8%), highlighting the critical role of temporal dynamics and contextual information
15

---

Published as a conference paper at ICLR 2026
across video frames. This underscores the dataset's design emphasis rich spatiotemporal reasoning and confirms its resistance to visual simplification bias.
Table 13: Multiple-choice zero-shot test results of Tarsier2-7B Yuan et al. 2025 different settings: text-only, me-frrmme and full-video.
on VUDG under
Visual Style
Viewpoint
Env. Condition
Model
D-Avg
CA
GA
MO
VI
Avg
EG
SU
SH
Avg
FO
NI
RA
SN
Avg
text-only
23.3
21.5
20.7
18.3
20.9
21.0
22.6
22.8
22.1
20.6
20.3
17.8
17.9
19.2
20.6
one-frame
34.9
42.6
37.7
43.9
39.8
41.4
45.5
47.3
44.7
37.6
38,9
37.9
35.2
37.4
40.3
full-video
64.6
56.8
59.7
75.4
64.1
66.0
64.0
63.8
64.6
57.7
63.4
59.5
60,5
60.3
62.8
D SEMANTIC SPACE COMPARISON
As shown in Figure b). Video-MME exhibits strong domain separation, particularly for "Sports Competition" and "Artistic Performance," which form distinct clusters far from other domains. This suggests the dataset encodes domain-specinee biases that may encourage models to overfit to visual stylistic cues rather than learn generalizable reasoning, making less suitable for evaluating domain generalization. In contrast, VUDG (Figure aa) maintains semantic consistency across domains, better reflecting real-world cross-domain video understanding challenges.
Cattonn
HEW benoo
FIGHT
PMEK In Weformance
Shalty
Rooy
(8) VUDG
(b) Video-MME
Figure 9: Comparison of t-SNE visualization of BGE embedding for different domain problems between VUDG and Video-MME.
E IMPLEMENTATION DETAILS
All the experiments are conducted on 8 NVIDIA RTX 4090D GPUs, each with 24GB of graphics memory.
E. HBI
We train HBI for epochs with a global batch size of 32. On the model side, we cap captions at frames, employ 2D linear patch mode, the slice-frame position to 2, keep all layers trainable (no freezing), and enable "loose" interaction modeling. Optimization is carried out with a base learning rate of le-4, higher coefficient learning rate of 1e-3 for specialized submodules, and weighted loss terms of 2 for the KL divergence and for the symmetric KL (SKL) divergence.
32 words and videos
E.2 EMCL4QA
We integrate the EMCL module into CLIP-based video-question-answeriie backbone and train for 5 epochs with a global batch size of 128, clipping videos to 12 frames and questions to 32 tokens. During each forward pass, EMCL performs T = 9 routing iterations over K 32 subspaces using Gaussian kernel (o = 1) and updates its bases via moving-average average momentum a = 0.9, then fuses reconstructed and original features with scale factor 3 0.5. We optimize end-to-end with Adam
16

---

Published as a conference paper at ICLR 2026
and 10% linear warmup, applying learning rate of lc-7 to the CLIP encoders and le-4 to the EMCL module and QA head, training against an InfoNCE-busee cross-entropy loss with temperature 0.01.
E.3 VIDEOLLAMA2
We fine-tune Vid22LLaMA2 with a LoRA rank of r = 128 and an alpha of 256, inserting STC Connector modules as multimodal projectors learned at 2e-5. We train with a global batch size of 64 (per device 1, gradient accumulation 8), optimize via AdamW at 2e-5 with no weight decay and 0.03 warmup ratio, and apply cosine annealing schedule throughout. To enable efficient large- training, we leverage DeepSpeed with ZeRO Stage 3 optimization, which allows for memory--eficient distributed training without compromising performance.
E.4
QWEN
We finetune Qwen2.5VI-33B using LoRA with rank of 64 and LoRA alpha of 64, targeting key projection modules (q-proj, k_proj, v-proj. proj) within the model architecture. We adopt global batch size of 32, achieved by setting per-device batch size of 4 and gradient accumulation steps of Optimization is performed using the AdamW optimizer with learning rate of 2e-7, no weight decay. and a warmup ratio of 0.03. A cosine learning rate scheduler is employed throughout training. To enable efficient large-scale training, we leverage DeepSpeed with ZeRO Stage optimization, which allows for memory-efficient distributed training without compromising performance.
F THE USE OF LARGE LANGUAGE MODELS (LLMS)
Large Language Models (LLLMs) were utilized as tools exclusively for dataset creation and evaluation, and not for research ideation. Our progressive multi-expert annotation framework employed a cascade of models for specific tasks: Qwen2.5-VL-7B was used for video selection from source datasets; Gemini-2.5-Flas generated initial open-ended question-answer (QA) pairs; DeepSeek-v3 converted these into multiple-choice questions; and Gemini-2.5-Pru performed an automated quality check. This multi-mode) approach was adopted to mitigate the hallucination, bias issues, and the risk of circularity (i.e., evaluating models on data generated by models with similar architectures or relying solely LLMs for benchmarking). Crucially, all manual review and correction by human annotators to ensure high quality. For evaluation, we also used DeepSeek-v3 as an automated judge score the open-ended QA responses.
annotations underwent final
G ANALYSIS OF DOMAIN DISCREPANCY
To quantitatively verify that the domains within our dataset are sufficiently distinct, we measure the visual discrepancy between them using Jensen-Shannon (JS) divergence. We first extract frame-level visual features for videos in each domain using pre-trained CLIP ViT-B/16 model. Then, for each pair of domains, we compute the JS divergence between their feature distributions. A higher JS divergence value indicates a larger gap and less similarity between the two domains.
Table 14 presents the results for several representative domain pairs that belong to the same high-level shift type. For instance, the game vs. virtual pair yields a high divergence of 0.441, and the egocentric vs. surveillance pair also shows significant gap (0.333). These quantitative results confirm that our dataset contains meaningful and non-trin non-trivial domain shifts, providing a solid basis for evaluating domain generalization.
H SOCIETAL IMPACTS
Our work aims to improve the robustness and generalization of video-language models in real- world scenarios by introducing a dataset focused on domain generalization. This has potential positive societal impacts in enhancing the reliability of AI systems used in safety-critical or diverse environments, such as surveillance, autonomous vehicles, or assistive technologies, where domain shifts are inevitable.
17

---

Published as a conference paper at ICLR 2026
Table 14:
(JS) divergence between feature distributions of representative domain pairs. Higher values indicate greater dissimilarity.
Domain Pair
JS Divergence
0.312 game vs. surveillance
game vs. cartoon
game vs. egocentric
0.304
0.289
game vs, virtual
0,441
egocentric surveillance
0.333
Similar to many existing datasets, VUDG includes some publicly available videos collected from online platforms. While we take care to avoid personally identifiable or sensitive content, the use of web-soureed data may still raise concerns regarding content ownership or individual rights. To mitigate such risks, our dataset is released strictly for non-commercial, academic research purposes only. We encourage responsible use and further discussion on ethical data curation and model deployment practices in this field.
QUESTION TYPES
In this section, we present illustrative examples for four question categories: Action Recognition (Table T3), Attribute Identification (Table T6), Object Identification (Table (7) and Temporal Under- standing (Table
Table 15: Examples of Action Recognition Questions.
Question Type
No.
Question Options
Answer
Q: What action does the character do upon exiting the door?
E
A. The character waves at friend.
B. The character jumps into a car.
C. The character runs down the street.
D. The character checks their watch.
E. The character falls their knees.
F. The character starts singing loudly.
Q: What action does the person perform in the convenience store?
F
Action Recognition
A. pays for the items the counter.
B. picks up a dropped item from the floor.
C. asks the clerk for assistance.
D. reads a magazine near the entrance.
E. drinks soda near the fridge.
F. puts items in the basket.
Q: What action is the woman performing at the checkout counter?
B
A. The woman is signing receipt.
B. The woman talking to the cashier.
C. The woman is packing her groceries.
D. The woman is searching for her wallet.
E. The woman counting her change.
F. The woman is looking at her phone.
18

---

Published as a conference paper at ICLR 2026
Table 16: Examples of Attribute Identification Questions.
Question Type No.
Question & Options
Answer
Q: What is the relative position of the child and the ladder?
C
A. The child is climbing up the ladder.
B. The child is sitting on top of the ladder.
C. The child is standing below the ladder.
D. The child is leaning against the ladder.
E. The child holding the ladder steady.
F. The child is jumping off the ladder.
Q: What is the quantity of full bookshelves behind the per- son?
F
Attribute Identification
A. There are five full bookshelves.
B. There are three full bookshelves.
C. There is one full bookshelf.
D. There are four full bookshelves.
E. There no full bookshelves.
F. There are two full bookshelves.
Q: What the state of the fountains at the end of the video?
E
A. The fountains are overflowing.
B. The fountains are frozen.
C. The fountains are turned off.
D. The fountains are being cleaned.
E. The fountains are spraying water.
F. The fountains completely dry.
Table 17: Examples of Object Identification Questions.
Question Type
No. Question & Options
Answer
Q: What is the tool attached to the end of the hose?
D
A. It is a green nozzle.
B. is a black brush.
C. is a blue sprayer.
D. It
red baster. E. It is white filter.
F. It is yellow funnel.
Q: What item is present near the stone henge area?
E
A. A bucket is present near stone henge.
Object Identification
B. A flag is present near stone henge.
C. A rope is present near stone henge.
D. A mirror is present near stone henge.
E. Bells are present near stone henge.
F. A ladder is present near stone henge.
Q: What kind of artwork hangs on the left wall?
F
A. photograph of mountain.
B. print of flower.
C. a drawing
river.
D. sketch of tree.
E. a poster of city.
F. painting of tower.
19

---

Published as a conference paper at ICLR 2026
Table 18: Examples of Temporal Understanding Questions.
Question Type
No.
Question & Options
Answer
Q: What does father do right after entering the kitchen?
E
A. He checks the time on the wall clock.
B. He washes his hands at the sink.
C. He opens the refrigerator grab drink.
D. He sits down at the kitchen table.
E. He plays tricks and kisses Mom.
He starts cooking dinner immediately.
2
Q: What the order of ingredients placed on the bread?
E
A. Ham, lettuce, tomato, then cheese.
Temporal Understanding
B. Lettuce, ham, tomato, then cheese.
C. Tomato, ham, lettuce, then cheese.
D. Cheese, ham, tomato, then lettuce.
E. Lettuce, tomato, ham, then cheese.
F. Tomato, lettuce, ham, then cheese.
3
Q: In what order does the main character demonstrate biking surfaces?
A
A. Clean pavement + Icy pavement - Snowy path.
B. Snowy path - Clean pavement - Icy pavement.
C. Snowy path Icy pavement Clean pavement.
D. Clean pavement + Snowy path + Icy pavement.
E. Icy pavement Clean pavement- - Snowy path.
F. Icy pavement + Snowy path - Clean pavement.
J PROMPTS FOR GENERATION AND EVALUATION
J.1 PROMPTS FOR OPEN-END QA PAIRS GENERATION
Prompts for open-ended QA pairs generation are shown in Figure and and Figure II
J.2 PROMPTS FOR MULTIPLE-CHOICE OPTIONS GENERATION
Prompts for multiple-choice options generation are shown in Figure and Figure 13
J.3 PROMPT FOR REVIEWING QA PAIRS
Prompts for reviewing QA pairs using Gemini-2.5-Pro shown in Figure 14 and Figure 15
J.4 PROMPTS FOR OPEN-ENDED EVALUATION
Prompts for open-ended evaluation using DeepSeck-Vi V3 are shown in Figure 16 and Figure 17
20

---

Published as a conference paper at ICLR 2026
Prompts For Open-ended generation Q123
Please generate three open-ended questions and their answers based on the following video content. Follow the requirements below:
1. The first question should be related to action in the video. It should ask about specific action in the video.
2. The second question should be related to the attributes of objects in the video, such as their color, size, shape, position, distance, quantity, motion, material, state, orientation, or texture.
3. The third question should related an object in the video. It should ask
about the identification, description, role of a specifie object in the video.
4. Fach question should be precise and unambiguous, ensuring that the answer is clear and directly related to the video content.
5. IMPORTANT: The questions and answers must be based solely on the visual
content of the video. Do not use any information from the audio track, even if is present the video.
6. Don't generate any other content except three QA pairs.
7. Make your answers simple as possible, try use no more than 10 words.
Output Format:
Q1: [Question about action]
A1: [Answer to Q1]
Q2: [Question about attributes]
A2: [Answer to Q2]
Q3: [Question about object]
A3: [Answer Q3]
Example Output:
Q1: Which did the man perform in the cinema?
A1: Hc ate popcorns.
Q2: What the color of the woman's hat?
A2: Brown.
Q3: What the man's hand?
A3: An ice cream.
Figure 10: Prompts for generating open-ended answers for Q1, Q2 and Q3.
21

---

Published as a conference paper at ICLR 2026
Prompts For Open-ended generation Q45
Please generate two open-ended questions and their answers based on the following video content. Follow the requirements below:
1. Both questions must related to temporal information or sequential relationships in the video, such as the order which events occur or the sequence which objects appear or reasoning about temporal information.
2. Each question should be precise and unambiguous, ensuring that the
answer is clear and directly related to the video content.
3. IMPORTANT: The questions and answers must be based solely the
visual content of the video. Do not use any information from the audio track, even if it is present in the video.
4. Don't generate any other content except two open-ended QA pairs.
5. Make your answers as simple as possible while keeping correct and
precise, try to use no more than 10 words.
Output Format:
Q1: [Question about temporal sequential in information]
A1: [Answer to Q1]
Q2: [Question about temporal or sequential information]
A2: [Answer to Q2]
Example Output:
Q1: Which event happens first for the characters in the video?
Al: Waking up from bed.
Q2: What is the correct order of events in the video?
A2: Event A - Event B - Event C.
Figure 11: Prompts for generating open-ended answers for Q4 and Q5.
22

---

Published as a conference paper at ICLR 2026
Prompts For Options Generation Q123
Based on the following questions and their open- ended answers, generate four multiple-choice options for each question. Follow the requirements below:
Example:
Q1: What action does the person perform
immediately after entering the room?
A) sits the chair.
B) picks up book from table.
each question, generate six options (A, B, C, D, F, F), with only one correct option that matches the open-ended answer exactly.
C) turns on the TV.
D) opens the window.
E) stands on the sofa.
The other five options should be relevant to the question but significant different from the open- ended answer, und should be umbiguous confusing.
F) jumps on the desk. Answer: D
Q2: What color of the woman's hat?
The correct option should be marked with the corresponding letter (e.g.. "A)").
A) red.
B) yellow.
4. Ensure that the length of all six options is as similar pussible. Avoid making the correct option the longest Remember simplify the content in the correct option make it shorter without changing the meaning.
C) blue.
D) pink.
E) orange.
F) cyan.
Answer: B
Ensure that semantic differences between the options are significant. Avoid creating options that are similar in their attributions, such as color, size, applications, actions, objects etc. The output should be in the following format:
Q3: What object the person holding during the entire scene?
A) phone.
B) coffee mug.
Q1: [Question 1]
C) book.
A) [Option A]
D) a pen.
B) [Option B]
E) pencil.
C) [Option C]
F) hat.
D) [Option D]
Answer: C
E) [Option L]
[Option
Questions and Answers:
Answer: [Correct Option Letter]
Q1: {questions(O];
Al: topen answers[O]]
Q2: [Question 2]
Q2: [questions]]]
A) [Option A]
A2: topen answers[[]]
B) [Option B]
Q3: [qucstions[2]]
C) [Option C]
A3: (open answers[2]]
D) [Option D]
E) Option
[Option
Answer: [Correct Option Letter]
Q3: [Question 3]
A) [Option A]
B) [Option B]
[Option
D) [Option D]
E) [Option E]
F) [Option F]
Answer: [Correct Option Letter]
Figure 12: Prompts for generating options for Q1, Q2 and Q3.
23

---

Published as a conference paper at ICLR 2026
Prompts For Options Generation Q45
Based following questions and their open-ended answers, generate four multiple-choice options for each question. Follow the requirements below:
Q2: [Question 2]
A) [Option A]
B) [Option B]
C) [Option CJ
For each question, generate six options (A, C, F), with only one correct option that matches open-ended answer exactly.
D) [Option D]
[Option E]
F) [Option FJ
2. The other five options should be relevant to the problem but significant different from the open-ended answer. especially in timing, and should not be ambiguous confusing.
Answer: |Correct Option Letter]
Example:
Q1: What does person perform
Ensure the options are clear, concise, and directly related to the question, but the temporal sequence of four options should not be similar.
immodiately after entering the room?
A) sits on the chair.
B) picks up book from the table.
The correct option should be marked with the correspouding letter (e.g.. "')""..
turns the TV.
D) opens the window.
Ensure that length of all six options as similar as possible. Avoid making the correct option the longest one. Remember to simplify the content in the correct option make shorter without changing the meaning.
E) stands on the sofa.
F) jumps on the desk.
Answer: D
Q2: What the color of the woman's har?
Ensure that semantic diffcrences between the options are significant. Avoid creating options that are similar in timing. You may need to avoid lemporal
A) red.
B) yellow.
C) blue.
similarities between options.
D) pink.
The output should the following format:
orange.
Q1: [Question
cyan.
A) [Option A]
Answer: B
B) [Option B]
C) [Option C]
Questions and Answers:
D) [Option
Q1: [questions(0]:
E) [Option E]
A1: (open [0]
F) [Option F]
Q2: [questions[1]:
Answer: [Correct Option Letter]
A2: Sopen_answers[11
Figure 13: Prompts for generating options for Q4 and Q5.
24

---

Q2: {42] Published as a conference paper at ICLR 2026
Prompts For Checking Q123
You strict video content auditor. Carefully examine each QA pair below based strictly on the visual information in the video.
Evaluation Criteria (MUST check all aspects):
1. Action Correctness: Are described actions consistent with the video?
Attribute Accuracy: Are object attributes (colorisize/position accuratc?
Object Presence: Do mentioned objects actually exist in the video?
4. Action Sequence: Is the order of actions correctly described?
Temporal Consistency: Does the timing match the video progression?
Validation Rules:
- If BOTH Q and A are fully correct - Output TRUE
- -Iff Q is factually wrong (asks about non-existent content) - Output FALSE
Q is valid but A is incorrect - Provide corrected answer
If uncertain due ambiguous video Output FALSE
QI: {ql}
Al: {alopen)
A2: (a2open]
Q3: {q3) A3: {a3open]
Required Output Format (STRICT):
Alresult: TRUE/ FALSE/ [Corrected Answer]
A2result: TRUE/ FALSE/
[Corrected Answer]
A3result: TRUE/ FALSE
[Correeted Answer]
Example output:
Alresult: TRUE
A2result: FALSE
A3result: The Sky should be blue.
Critical Requirements:
ONLY use visual evidence from the video frames, do not consider any audio
information.
Be EXTREMELY strict assume FALSE when uncertain
3. For corrections: do not point out the original error, but provide the correct
answer directly
4. Never explain your reasoning
Figure 14: Prompts for Checking Q1, Q2 and Q3.
25

---

Published as a conference paper at ICLR 2026
Prompts For Checking Q45
You are a strict video content auditor. Carefully examine each QA pair below based strictly on the visual information in the video.
Evaluation Criteria (MUST check all aspects):
1. Action Correctness: Are described actions consistent with the video?
2. Attribute Accuracy: Are object attributes color/size/positionn accurate?
3. Object Presence: Do mentioned objects actually exist in the video?
Action Sequence: Is the order of actions correctly described?
Temporal Consistency: Does the timing match the video progression?
Validation Rules:
If BOTH Q and A are fully correct - Output TRUE
- If Q is factually wrong (asks about non-existent content) - Output FALSE
If Q is valid but A is incorrect - Provide corrected answer
uncertain due to ambiguous video - Output FALSE
Q1: {ql}
A1: {alopen]
Q2: (q2] A2: [a2open]
Required Output Format (STRICT):
Alresult: TRUE/ FALSE/ [Corrected Answer]
A2result: TRUE/ FALSE/ [Corrected Answer]
Example output:
Alresult: TRUE
A2result: FALSE
Critical Requirements:
ONLY use visual evidence from the video frames, do not consider any audio
information.
2. Be EXTREMELY strict assume FALSE when uncertain
3. For corrections: do not point out the original error, but provide the correct
answer directly
4. Pay more attention on Temporal Consistency. Never explain your reasoning
Figure 15: Prompts for Checking Q4 and Q5.
26

---

Published as a conference paper at ICLR 2026
Prompts for Open-ended Evaluation of Q123
Evaluate the answer on TWO dimensions: Accuracy and Relevance. Each score must be an integer from 0 to 5.
1) Accuracy:
O: Completely incorrect; contradicts the video's content.
Mostly incorrect; only minor correct elements.
2: Some correct elements; significant inaccuracies present.
3: Generally correct, minor inaccuracies.
4: Mostly correct; very minor errors.
5: Completely accurate; aligns perfectly with the video's content.
2) Relevance:
O: Entirely irrelevant; does not address the question.
Minimally relevant; largely off-topic.
2: Partially relevant; includes significant unrelated information.
3: Mostly relevant; some extraneous details.
4: Highly relevant; minor non-essential details.
5: Fully relevant: directly addresses the question without deviation.
Instructions:
Analyze the respondent's answer in the context of the video. Scores need to be given with sufficient reasons.
Assign accuracy sccuracy_score score (0-5) based on the correctness of the information.
3. Assign relevance score (0-5) based how well the answer addresses the question.
4. The answer's phrasing does not need to match the expected wording exactly meaning and content alignment are what matter.
Output Format (JSON):
"jsonn {
"accuracy_score" <int 0-5>, selevance_score"" <int 0-5>
H1
Example:
Question: What is someone doing with a knife to yellow mushrooms?
Ground Truth: A person is shown using a knife to cut yellow mushrooms off log in the forest
Respondent's Answer: Someone is cutting up yellow mushrooms with knife. Expected JSON Output: Output Format (JSON): "'json !!
"accuracy serrs_scorre 4, "relevance_scon" score":
Figure 16: Prompts for Open-ended Evaluation of Q1 Q3(cction Recognition, Attribute Identifica- tion, and Object Identification)
27

---

Published as a conference paper at ICLR 2026
Prompts for Open-ended Evaluation of Q45
Evaluate the answer on TWO dimensions: Sequence Correctness and Relevance. Each score must be an integer from 0 to 5.
1) Sequence Correctness:
O: Completely incorrect order.
1: Mostly incorrect order.
Some correct order; significant errors.
Generally correct order; minor errors.
4: Mostly correct order, very minor errors.
5: Perfect sequence.
2) Relevance:
O: Entirely irrelevant; does not address the question.
1: Minimally relevant: largely off-topic.
Partially relevant; includes significant unrelated information.
Mostly relevant; some extrancous details.
lighlly relevant; minor non-essential details.
Fully relevant; directly addresses the question without deviation.
Instructions:
1. Analyze the respondent's answer in the context of the video; provide brief justification for each score.
2. Assign sequence sequencceccorrectness correctness score (0-5) based on how accurately the answer reflects the temporal order of events.
Assign relevance score (0-5) based on how directly the answer addresses the question's 's focus.
The answer's phrasing does not need to match the expected wording exactly meaning and order aligument are what malter.
Output Format (JSON):
"json
{{
"sequence cotrectnssssee score": <int 0-5>,
"relevance_score" score": <int 0-5>
31
Example:
Question: In what order do the girl's expressions change?
Ground Truth: The video shows girl listening through safe, looking confused, then later happily taking shower.
Respondent's Answer: The girl's expression is confused as she listens through the safe, then she is happy when she takes a shower.
Model Answer: Surprise, sadness, and then happiness.
Expected JSON Output:
"jsonn
"sqquence_corrreeness correctness sorrectness_sccree: 2, "relevance_score 5
...
Figure 17: Prompts for Open-ended Evaluation of Q4 and QS(Temporal Understanding)
28


Reviews:
[{"review_id": "L9PrxCqvpi", "reviewer": "Reviewer_xqBB", "summary": "Authors introduce VUDG, a video-understanding domain generalization benchmark spanning 11 domains with a shared semantic space. QA pairs are built via a multi-expert (model-assisted + human) annotation pipeline. Across multi-source, single-source, and zero-shot protocols, both classic VideoQA methods and state-of-the-art LVLMs show sizable accuracy drops under domain shifts, revealing uneven robustness and a clear gap from in-domain fine-tuning.", "questions": "refer to weaknesses", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Innovative Benchmark Design\n\nThe paper introduces a novel benchmark specifically crafted to test how well video understanding models generalize across domain shifts, filling an important gap in current evaluation practices.\n\nThorough and Systematic Evaluation\n\nIt provides a comprehensive assessment of multiple domain shift scenarios (style, viewpoint, weather, lighting) using both traditional VideoQA models and modern LVLMs, offering a well-rounded perspective on robustness.", "weaknesses": "Insufficient Distractor Analysis\n\nWhile the paper reports aggregate counts and durations (Fig. 4–5), it provides no fine-grained QA/distractor analyses - e.g., answer-length distributions, length-matching of options, distractor similarity, so on. For example, the prompt states to have distractors be the same length but from my experience LLMs might just ignore it completely and there is no statistics to verify if it is the case. Further investigation into questions and answer options would strengthen the benchmark’s validity\n\nLLM dependence and possible circularity\n\nThe pipeline relies on LLMs for generation, screening, and even as an automated judge for open-ended scoring, which can introduce model bias and evaluation circularity despite a final human pass.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PjvBU07Pmz", "reviewer": "Reviewer_M1bM", "summary": "This paper introduces the Video Understanding Domain Generalization (VUDG) dataset, a dataset designed to evaluate domain generalization of video understanding models. The authors key motivation is that existing video understanding benchmarks either do not measure domain generalization, or are ineffective at measuring it due to large semantic gap between domains. VUDG aims to address this by collecting videos from 11 distinct domains while keeping the underlying semantic content consistent through a shared set of daily human activities. the authors employ a progressive multi-expert annotation framework that combines multiple large vision-language models for automated question generation, verification, and filtering, followed by a human verification. This process yields a training dataset of 31k QA pairs, and a testing dataset of 4k QA pairs. The authors additionally benchmark existing video understanding models on VUDG.", "questions": "What evidence exists to motivate the community to evaluate the domain generalization ability of their models with VUDG, instead of a benchmark like Video-MME that also contains multiple video domains? Can this semantic shift be quantified and shown to hurt a benchmarks ability to assess domain generalization? Or can it be qualitatively shown?\n\nHow can the authors be certain that the MCQs in VUDG do not suffer from visual or language bias? Are there any failsafes in the data curation to mitigate this?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The proposed benchmark is the first large-scale benchmark designed to measure domain generalization in VLMs, consisting of a dedicated training/testing set and clear protocols for evaluation\n\nThe choice of domains and corresponding videos is high quality: The domains selected in VUDG are broad and diverse and a large portion of the videos used for testing are newly collected by the authors, reducing the risk of data leakage with existing VLM training data", "weaknesses": "The reviewer understands the primary motivation behind VUDG is to unify the semantic space across domains, and this intuitively makes sense to the reviewer. However, it is not scientifically shown why existing benchmarks fail to assess domain generalization due to the lack of cross-domain semantic alignment\n* What evidence exists to motivate the community to evaluate the domain generalization ability of their models with VUDG, instead of a benchmark like Video-MME that also contains multiple video domains?\n* The t-SNE in Figure 6 does show the semantic alignment between domains in VUDG, but it is not compared with other benchmarks\n\nIt is unclear if the testing dataset exhibits language or visual bias in the MCQ questions, which is a common pitfall of LLM generated MCQs (Zohar et al., Apollo: An Exploration of Video Understanding in Large Multimodal Models, CVPR 2025). The authors mention there is a human verification step, but do not explicitly state that it addresses this bias.\n* Additionally there are no qualitative examples of MCQs in the paper. As this is a dataset paper the reviewer expected to see many more examples of QA pairs in the dataset\n\nThe choice of VLMs used for evaluation in the full fine-tuning and single-domain generalization settings is relatively shallow, reducing the insight gained from these settings", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Cj7Tq64cpK", "reviewer": "Reviewer_RsGD", "summary": "The paper presents a new dataset to evaluate model capability for domain generalization in a video comprehension task. Videos are filtered using a multimodal LLM to ensure they fall into certain domains. The videos are then filtered based on the actions again, using a multimodal LLM so that videos in the different domains have similar semantics. QA pairs are automatically generated for each video, which are then reviewed by human experts. The dataset offers both training and test sets. The paper provides evaluation results over multiple models in multi-domain fine-tuning, single-domain fine-tuning, and zero-shot settings. Human evaluation is also provided to show the relevance and correctness of the automatically generated WA pairs.", "questions": "I want to see some discussion on the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. I believe the paper offers a useful dataset (once it is published online), which should contribute to the community. \n2. The paper also offers comprehensive details of how the dataset is created.", "weaknesses": "1. This may be a minor concern, but since the dataset only offers limited semantics, the evaluation capability may also be limited. It would be nice if the paper could provide some discussion on this point (e.g., whether the set of actions is sufficient for evaluating general performance, and why). \n2. As domain generalization can involve fine-tuning, it is desirable to evaluate possible bias in QA pairs and semantics. This may be done by training a language model with only QA pairs and some tokens to represent semantics. \n3. It’s hard for me to see what the human evaluation results in Table 10 mean. Human-annotated QAs and some randomization (e.g., randomly pairing up a video and QA in different domains but in the same semantics, or just random video and QA pairs) may help understand it. \n4. I also don’t see why human evaluation uses a 5-point scale. Relevance and correctness sound more like binary.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zK0nQjomNN", "reviewer": "Reviewer_Qtow", "summary": "This paper proposes a new dataset, VUDG, to test domain generalisation in video understanding models. It includes 11 domains across three kinds of shifts: **semantic** (e.g., cartoon), **viewpoint** (e.g., egocentric), and **environmental conditions** (e.g., foggy). VUDG has the following qualities:\n1. There is semantic consistency across the domains, i.e., they show similar actions.\n2. The annotation is based on a generating QA pairs from a cascade of LLMs with a final human review component.\n3. The videos are sourced from existing datasets as well as from online video sites.\n\nA suite of video models are evaluated on VUDG across 4 settings:\n1. Full domain (train on all domains, test on all domains)\n2. Multi-domain (train on all-but-one domain, test on the one remaining domain)\n3. Single-domain (train on a single domain, test on all other domains)\n4. Zero-shot\n\nSome key findings include:\n* Video models perform worse in settings (2) and (3) compared to (1).\n* LLM based models are much superior than traditional domain generalisation methods.\n* Qwen2.5VL-7B is very strong in zero-shot performance", "questions": "1. Domain generalisation is well studied for images. What about videos is special that makes it interesting in this context? If the authors can motivate this better, it can be useful.\n\n2. it is not clear how you get data for different domains. The source datasets are stated but how does one get videos from different domains?\n\n3. \"Attribute identification\" task may be more relevant for images than videos. Can you provide examples for attribute identification from videos?\n\n4. Most existing video benchmarks suffer from static biases (a single frame or even without any visual frames, models can get reasonable accuracies). Have you done any analysis in this regard?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Domain generalisation is an interesting problem to study in video context (especially for something like exo-to-ego generalisation).\n2. The use of a cascade of strong LLMs to obtain QA annotations seems scalable and can possibly help scale up the proposed dataset. Including newer LLMs can also improve the current dataset.\n3. The inclusion of \"Temporal understanding\" in the designed questions (e.g., related to temporal ordering of actions) is valuable and worth studying in context of domain generalisation.", "weaknesses": "1. **General concern about relevance of domain generalisation in LLM era**. Given the increasingly massive sizes and extent of the training datasets for modern video LLMs (e.g., Tarsier 2 is trained on 40 million videos), the gap between what we deem as training and testing is shrinking. This is likely why Qwen2.5VL-7B zero-shot (~72%) is stronger than any fine-tuned models. I am sure the closed models are even stronger just zero-shot. Thus, I am not sure if studying domain generalisation (and this dataset in particular) brings a lot of novel insights. \n\n2. **Some stronger models can be included in evaluation.** An open model like Tarsier/Tarsier2 is shown to be quite a strong video LLM. It may already achieve very strong zero-shot performance, so it should be included in my view. Likewise, no closed models are included. While I understand the cost aspect, at least one model should be included because I suspect it may already be very strong zero-shot. \n\n3. **Value added by this dataset.** While domain generalisation in video context sounds interesting, I did not gather what new, surprising insights this dataset would bring to the community. Studying it for special applications like medical imaging or private videos is valueable, but this dataset still has domains that are likely present in the train dataset of some of these big models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)