VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation
Authors: Hritik Bansal
Clark Peng
Yonatan Bitton
Roman Goldenberg
Aditya Grover
Kai-Wei Chang
Venue: ICLR 2026 Poster (2026)
Keywords: physical commonsense
semantic adherence
video generation
benchmark
auto evaluator
Primary area: datasets and benchmarks
TL;DR: We introduce VideoPhy-2, an action-centric dataset for evaluating physical commonsense in generated videos.

Abstract: Large-scale video generative models, capable of creating realistic videos of diverse visual concepts, are strong candidates for general-purpose physical world simulators. However, their adherence to physical commonsense across real-world actions remains unclear (e.g., playing tennis, backflip). Existing benchmarks suffer from limitations such as limited size, lack of human evaluation, sim-to-real gaps, and absence of fine-grained physical rule analysis. To address this, we introduce VideoPhy-2, an action-centric dataset for evaluating physical commonsense in generated videos. We curate 4000 diverse and detailed prompts for video synthesis from modern generative models. We perform human evaluation that assesses semantic adherence, physical commonsense, and grounding of physical rules in the generated videos. Our findings reveal major shortcomings, with even the best model achieving only $47.7\%$ joint performance (i.e., high semantic and physical commonsense adherence) on the hard subset of VideoPhy-2. We find that the models particularly struggle with conservation laws like mass and momentum. Finally, we also train VideoPhy-2-AutoEval, an automatic evaluator for fast, reliable assessment on our dataset. Overall, VideoPhy-2 serves as a rigorous benchmark, exposing critical gaps in video generative models and guiding future research in physically-grounded video generation. The data and code is available at \url{https://videophy2.github.io/}.

Published as a conference paper at ICLR 2026
VIDEOPHY-2: A CHALLENGING ACTION-CENTRIC PHYSICAL COMMONSENSE EVALUATION IN VIDEO GENERATION
Hritik Bansal
Clark Peng"1
Yonatan Bitton
Roman Goldenberg Aditya Grover Kai-Wei Chang!
University of California Los Angeles
Google Research
ABSTRACT
Large-scale video generative models, capable of creating realistic videos of di- verse visual concepts, are strong candidates for ceneral-purpos physical world simulators. However, their adherence to physical commonsense across real-world actions remains unclear (e.g.. playing tennis, backflip). Existing benchmarks suffer from limitations such as limited size, lack human evaluation, sim-to-real gaps, and absence of fine-gra fine-grained physical rule analysis. To address this, we introduce VIDEOPHY-2, 2- an cetion-centric dataset for evaluating physical commonsense generated videos. We curate 4000 diverse and detailed prompts for video synthe- sis from modern generative models. We perform human evaluation that assesses semantic adherence, physical commonsense, and grounding of physical rules in the generated videos. Our findings reveal major shortcomings, with even the best model achieving only 47. 47.7% joint performance (i.c., high semantic and physical commonsense adherence) on the hard subset of VIDEOPHY-2, We find that the models particularly struggle with conservation laws like mass and momentum. automatic evaluator for
Finally, we also develop VIDEOPHY-22 2-ATOVVVLL fast, reliable assessment on our dataset. Overall, VIDEOPUY-2 serves as a rigorous benchmark, exposing critical gaps in video generative models and guiding future research in physically- video generation. The data and code available at ht llttps::/videopph:: hwt.::i...ittuu.....
Model Performance VideoPhy2 Dataset
SVD-2V
All
Figure 1: Performance VIDEOPHY-2 dataset using human evaluation. We evaluate the physical commonsense and semantic adherence text conditioning prompts for diverse real-world actions. We observe that the best-performing model W (27B total, 14B active params) achieves 47.7% on the hard subset of the data, created using Cog VideoX-5B reference model. represents evaluation on small subset of the dataset.
the
Hard
VideoCrafter2
HunyuanVideo-15
Ray2
Sora*
Cosmos-Dift-78
CogVideoX:5B
B-14
Wan2.2-278-A148
10
20
30
40
60
70
80
90 100
Accuracy Based on Human Evaluation (%)
INTRODUCTION
Recent advancements in large-scale video generative modeling offer the potential to simulate the
physical world accurately [3] 58). In particular, this capability enable learning general-purpose
visuomotor policies [36 [9], autonomous driving [Л], and game playing [5]] 21[5
5]. In daily life,
humans rely their sophisticated physies intuition interact with the world 20 (e.g., predicting

---

Published as a conference paper at ICLR 2026
lennii
person lernis hitting
"The ball moves through the air form"
Playing Tennis
Prompt
the racquet
player serves, the ball lssvas
Video Generation
Caption
ball person weerr ing while shorts and shoes. The court is green with white lines marking boundarios.
yellow-green
Phys. Rules
high speed,
Seed
lands
"The bell changes
box.
service
00:01
00:02
00:03
position after
being hit"
AM
Does the video follow physical commonsense? (Unlkely)
Text Description:
fennis player serves, the bell leaves the racquet high speed, and lands the service box.
Physical Commonsense and Rules
Is the physical rule valid for the given video?
Semantic Adherence
Followed Ruia(a): The moves thoouhh the an are
Violsted Rule(s): The tonnis ball changes position after being hit
Does the video follow the caption
(l.e., is the text description
Additional human-written violations
grounded in the video)? (Very Unlikaly)
The bail moves toward the player but reacts as if were hit without contact.
Figure 2: VIDEOPHY-2 pipeline. We generate a text prompt from the seed action using an L.M,, create a video with a video model, and caption it with a VLM to extract candidate physical rules. Then, humans rate the video's physical likelihood, verify rule violations, suggest missing rules, and assess prompt adherence.
the trajectory of football after being hit). However, the extent to which existing video models can generate physically likely worlds across diverse real-world actions remains unclear.
An approach evaluating generated videos is compare them with ground-truth physical simulations 1252. Furthermore, there is a lack of mature methods for rendering diverse real-world materials 49 and for accurately simulating complex physical interactions [40]. For instance, simulating a scenario like 'a child kicking ball against a wall' requires precise estimation of the foot's pose, and considerations of the ball's air pressure and material properties. While we focus on evaluating the physical likelihood of generated videos, an assessment that can often be made by humans without formal physics education by relying on their world experience.
R 31
Recent work such as Physics-IQ [88] conditions video models on the first few frames of real videos and evaluates their similarity by comparing predicted videos with ground-truth completions. However, this approach faces several challenges: (a) the extent to which it agrees with human judgment remains unclear, and (b) extending to more complex scenarios depicting multiple events is non-trivial. Another work PhyGenBench [40] curates small set of 160 manually crafted prompts, which is not scalable. Additionally, their evaluation approach simplifies the problem by designing text prompts that explicitly associate with single physical law (e.g., 'A stone placed on the surface of water pool' is linked to law of Buoyancy). Although, this strict one-to-one association between a prompt and a physical law is problematic, as video models often exhibit imperfect semantic adherence. For instance, a video model might generate a video that does not strictly follow the prompt but still adheres physical commonsense (e-g., producing video where 'a stone is dropped from height into the pool', where gravity is more crucial than buoyancy). We note the difference between VIDBOPHY-2 and several existing work in Table
To address these gaps, we propose VIDEOPUY-22, challenging physical commonsense evaluation dataset for woal-world actions. Specifically, we curate a list of 197 actions across diverse physical activities (e.g.. hula- mula-hooping, gymnastics) and object interactions (e.g.. bending an object until it breaks). Then, we generate 3940 detailed prompts from these seed actions using a large language model (LLM).. Further, these prompts are used to synthesize videos with modern video generative models. Finally, we compile a list of candidate physical rules (and laws) that should be satisfied in the generated vidcos, using dision-languag models in the loop. For example, in video of sportsperson playing tennis, physical rule would be that ball should follow a parabolic trajectory under gravity. For gold-standard judgments, we ask human annotators to score each video based on semantic adherence and physical commonsense, and to mark its compliance with physical rules.
In experiments, find that the best perffrrero model, Wan2.2-27B-AL44 [60], achieves joint performance score (high semantic adherence and physical commonsense) of only 55.4% while Wan2.1-14B achieves score of 32.6%. To further increase the dataset's difficulty, create hard subset where the performance of 2ann2.227B-A-. respectively. Moreover, our fine-grained analysis of human-annotated physical rule violations shows that video models struggle the most with conservation laws (e.g., mass and momentum).
and Wan2.1-14BB drops to 47.7% and 22%,
While human evaluation serves as the gold standard for real-worlh physical commonsense judgment, it is expensive and difficult to scale. To address this, we train an automatic evaluation model,
2

---

Published as a conference paper at ICLR 2026
Table Comparison between VIDEOPHY-2 and several prior work. We highlight salient features of the VIDEOPHY-2 and show that its unique contributions. For instance, it is one of the largest datasets for physical commonsense evaluation, along with violated physical rule (and law) annotations.
Feature
VRench 23
Gen
PhysiosI
ExalCrafter
ViRKOPY
VIDEOPHY-2 (Ouns)
Num of captions
1746
160
396
700
655
3910
Gold homam evaluation
Physical
Plysical
annotations
Real-world aclioe-oentric
I Long (dense) cigxions
X
Hard sibset
Automatic evalnatce
videos unnotations
Human feedback type
Pairwise
Raiing
Rasing (1-5)
Rating (0-1)
Rating (1-5)
VIDEOPHY-2-- AUTOEVAL, capable of performing a wide range of tasks including scoring semantic adherence, physical commonsense. and classifying physical rule grounding in the generated video. In our experiments, we find that SIDEOPHY-2-AUTOEN foundation model, Semmini-2.0-Flsh--e 236% on the semantic adherence and physical commonsense tasks, respectively, on the unseen prompts. Overall, demonstrate that VIDEOPHIY-2 is high-quality dataset that poses a formidable challenge for modern video models.
outperforms a capable multimodal [8], with relative correlation improvement of 81% and
2 VIDEOPHY-2 DATASET
We present the steps for data construction (Figure below:
Seed Actions (Stage 1): We curate set of actions relevant to physical commonsense evaluation. Specifically, we compile a diverse list of over 600 actions from popular video datasets that capture wide range of real-world activities, particularly those involving sports, physical activities, and object interactions. These datasets include Kinetics [6]. UCF-101 [35], and SSv2 [22). Next, we divide the student authors, with undergraduate or more degree in STEM, into two groups, each of which independently reviews the list and marks actions deemed relevant for physical commonsense evaluation. Our goal is to include actions that test various physical laws (e.g., gravity, elasticity, buoyancy, reflection, conservation of mass and momentum). Importantly, we filter out actions that do not clicit significant motion or are unlikely to be compelling for physical commonsense evaluation in videos (c.g.. typing, applying cream, arguing, auctioning, chewing, playing instruments, petting a cat). Finally, we retain only the actions deemed relevant by both groups of annotators. After this filtering process, obtain list of 232 actions, which we further refine using Gemini-2.0-Flann F.assh--EE to remove semantic duplicates, resulting in a final set of 197 actions. Among these, 143 and 54 actions belong to object interactions and physical activities category, respectively. We present the list in Appendix Table 9
LLM-Generated Prompts (Stage 2): In this stage, we query the Gemini-220-Flaashe Exp LLM to independently generate 20 prompts for each action in our dataset. In particular, we focus on the depiction of multiple events within a prompt to increase the challenge for modern video generation models (e.g., encourage the LLM to generate 'An archer draws the bowstring back to full tension, then releases the arrow, which flies straight and strikes a bullseye on paper target' instead of a simpler prompt 'An archer releases the arrow'). ). Our prompt generation template is presented in Appendix F In total, we curate 3940 prompts covering a wide range of actions. Since the modern video models understand long video deseriptions, we also generate dense captions from the original captions using the Mistral-NeMo-11 12B--Intrutc prompt upsampler from [. In particular, these dense captions add more visual details the original caption without changing its semantic meaning (e.g., main characters and actions)
Candidate physical rules and laws (Stage 3): In this stage, we aim generate candidate physical rules and associated laws that could be followed (or violated) in the generated video. Since video models often struggle to adhere to conditioning text prompts, we do not derive physical rules directly from them. Instead, we first generate videos using generative models conditioned on prompts from
We present some of the generated captions and underlying actions in Appendix Table 13 and the upsampled captions in Appendix Table
3

---

Published as a conference paper at ICLR 2026
the VIDEOPHY-2 dataset. Then, we create captions for these videos using the strong video captioning capabilities of Sammini-2.0-Flasse
This ensures that the physical rules are constructed based on details grounded in the video itsclf Subsequently, we ask 2..mini-2.0-Flashh a set of three physical rules (and laws) that should be followed for a given video. Since a video may violate physical rules that not covered in the pre-defined rules, we further ask the human annotators to write additional violated rules during physical commonsense evaluation. We present the rule generation prompt in Appendix Table
to generate
Construction of the Hard Subset (Stage 4): While we collect diverse and lengthy captions to make the task more challenging, we further employ model-based strategy to identify subset of particularly difficult actions. Specifically, we generate videos using a strong open video model, Cog VideoX-5B 63]. conditioned on captions from the VIDEOPHY-2 dataset. From this, we select 60 actions (out of 197) for which the model fails to generate videos that accurately adhere the prompts and follow physical commonsense (Appendix Table On examination, we find that these actions focus on physics-rich interactions (e.g., momentum transfer in throwing discus or passing football), state changes (e.g., bending something until it breaks), balancing (e.g., tightrope walking), and complex motions (e.g., backflip, pole vault, and pizzatossing). In total, we designate 1200 prompts making the dataset more challenging. We present the list of hard actions in Appendix Table 10
Data Analysis: We present the dataset statistics in Appendix Table 7 Specifically, VIDEOPHY-2
contains 3940 captions, which is 5.72x more than those in the VIDEOPHY dataset. Additionally, the average lengths of original and upsampled captions are 16 and 138 tokens, rexpectively-11.8 1.88 x and 16.2x longer than those in VIDEOPHY. Furthermore, VIDEOPHY-22 includes 110K human annota- tions across various video generative models and their semantic adherence, physical commonsense, and physical rule annotations. Finally, we show the distribution of the root verbs and direct nouns in the original captions of VIDEOPHY-2 in Appendix Figure 6 demonstrating the high diversity of the dataset. We also illustrate the diversity of multiple captions for an action in Appendix Figure 7
EVALUATION
3.1 METRIC
In practice, generated videos must satisfy several constraints, including high video quality [41].
temporal consistency (28), and entitybaccggound consistency 77. While many of these metrics intertwined, it is essential evaluate each independently to gain clearer understanding of model's 's capabilities. In this work, we focus on assessing the extent to which a generated video (1) adheres to the input text prompt and (2) follows physical commonsense. To quantify these aspects, we employ a rating-based evaluation using a 5-point Likert scale, well-established methodology for capturing human judgments across domains ranging from psychology 37 to large language model evaluation [50]. This approach has also been adopted in video model evaluation B 40 41). Unlike ranking- based feedback, which only reflects relative preferences between two outputs, our rating system measures the absolute degree of model's success or failure. Moreover, the 5-point scale provides more fine-grained feedback than binary labels (e.g.. plausible/implauiibl)) enabling a more nuanced analysis of model performance.
Since human evaluation inherently involves subjectivity, we implemented a rigorous protocol to ensure reliability. All annotators underwent structured training guided by detailed rubric with clear examples, ranging from "very unlikely" (1) to "very likely" (5), to anchor their judgments and establish shared understanding of the scale. To further reduce individual bias and capture stable consensus, each video was evaluated by three annotators. This process yielded a high inter-annotator agreement of 80% (comparable to agreement scores reported in prior work g consistency and validity of our framework.
confirming the
We observe that prompting Gemini-2.0 2.0- Flash-Exp to generate physical rules directly from the video did not. yield high-quality outputs. Therefore, we prefer a two-step process: captioning followed hy rule generation. We note that similar model-based strategy is also adopted in recent work like Humanity's 's I Last Exam 511 and ZeroBench [4 collect hard instances for model evaluation.
4

---

Published as a conference paper at ICLR 2026
Semantic Adherence (SA): Here, we aim to assess whether the input text prompt is semantically grounded in the generated video. Specifically, studies whether the entities, actions, and relationships described in the prompt are accurately depicted in the video (c.g., person visibly jumping into the water). To measure semantic adherence, annotators rate each video on a 5-point scale, selecting from the following options: (SAA E Very Unlikely (I), Unlikely (2), Neutral (3), Likely (4), Very Likely (5))- In this case, very unlikely indicates that the video does not match the prompt at all, and very likely highlights the video fully adheres the prompt with no inconsistencies.
Physical Commonsense (PC): Here, our goal is to assess whether the generated video follows the physical laws of the real-world intuitively (c.g., the football should start moving after impact in accordance with newton's 's first law). We note that the physical commonsense evaluation is independent of the underlying video generating text prompt. Since a video can follow (or violate) numerous laws, we are concerned with the holistic sense of the video's 's physical commonsense. In particular, the annotators rate each video on a 5 5-point scale, selecting from the following options: (PC E Very Unlikely (I), Unlikely (2), Neutral (3), Likely (4), Very Likely (5)). Here, very unlikely that the video contains numerous violations of fundamental physical laws, and very likely indicates that the video demonstrates a strong understanding of physical commonsense with no violations.
Similar to [8]. we compute joint performance as the main evaluation metric, which measures the fraction of videos that both adhere closely to the text prompt (SA > 4) and follow physical commonsense to a high degree PC 2 4). We do not report the posterior score (PC' >= 4|SA >= 4) since bad model can game
Physical Rules (PR): A key feature of the VIDEOPHY-2 dataset is the collection of candidate physical rules (and their associated laws) that humans evaluate as being followed or violated in the generated video (e.g., 'the ball should go down' is a physical rule associated with the law of gravity). These rules enable a fine-grained assessment of the video model's capabilities. Specifically, we determine whether candidate physical rule is violated followed (I), or cannot be determined (CBD) (2) in the generated video Further, we ask human annotators to note more physical rule violations to ensure comprehensive coverage.
3.2 SCORING
Human Evaluation. In practice, human evaluation serves as a gold standard for assessing the quality of generative foundation models [43] 64]. In particular, we collect judgments using the Amazon Mechanical Turk (AMT) platform from group of 12 human annotators, which were selected after passing a qualification test. Since physical commonsense is independent of the generated video- prompt alignment, evaluate semantic adherence and physical commonsense (including rule-based judgment) as separate tasks for human annotators. This differs from prior work in VIDEOPHY (8), which treats semantic adherence and physical commonsense assessment single task. It may introduce evaluation bias, as annotators have access to the prompt while conducting the physical commonsense evaluation, a scenario we explicitly avoid in this work.
We present the annotation UI for the semantic adherence task in Appendix Figure 17 where the input consists of a text prompt and the corresponding generated video. Note that human annotators were shown the original prompt (not the upsampled prompts) to ensure a fair comparison between video models, regardless of their ability to handle short or long prompts. In the following task, human annotators are asked to evaluate only the generated video and with regard to adherence to specific physical rules followed/viiglted//BB)
overall physical commonsense, and observable behaviors that violate physical reality The annotation interface is shown in Appendix Figure 18
Automatic Evaluation. While human judgments serve as the gold standard, automating the evalua- tion process crucial for faster and more cost-effective model assessments. In this study, we evaluate
'A video model can adhere to the prompt for only out of 1.000 prompts in the dataset. Now, assume that this vidco is also physically realistic. In this case, the posterior performance of the model will be reported as 1 100%, which is quite mislcading for the model builders.
'We include CBD category because L.M --gegnerrtedd physical rules may not be grounded the video. In our instructions the annotators, explicitly clarify that the overall physical commonsense judgments should extend beyond the predefined physical rules listed in the task.
5

---

Published as a conference paper at ICLR 2026
several video-language foundation models (e.g., Gemini-2.0-Fllase
VideoScore 23 on two tasks: semantic adherence and physical commonsense scoring. Specifically, we prompt the models to score generated videos based these two criteria and then normalize their predictions to 5-point scale. We provide more details about score computation in Appendix Kj Additionally, we introduce a classification task to determine whether a given physical rule is followed, violated, or CBD in the generated video, leveraging video-language models such as VideoLLaVA [38]. Here, we prompt the model classify each video-rule pair into one of three categories: followed, violated, or CBD,
4 SETUP
Video generative models. In this work, we evaluate diverse range of state-of-the-ar text-to-video generative models. Specifically, we assess seven open models and two closed models, including CogVideoX-5B [63]. VideoCrafter2 [17]. Hunyuan Video- 138 (34], Cosmos-Diffasion-7H [1] Stable Video Diffuusio (SVD-12V) [1] Wan2. 1-14B (60, Wan2.2-722 2-27B--48 (MoE with 27B total and 14B active params) [60]. OpenA/ Sora [13] and Luma Ray2 44] We prompt these models with the upsampled captions, except for those that do not support long (dense) captions by design i,e., Hunyuan- 13B and VideoCrafier2. For SVD-I2V, we first generate an image using Stable Diffusion and then use it as a conditioning variable to SVD. Additionally, we generate short videos (less than 6s) they easier evaluate and effectively highlight challenges the VIDEOPHY-2. The model inference details are provided in Appendix M
Wan222-112-227--1444
Dataset setup. Similar to [8] we take a data-driven approach and use human annotations across multiple tasks to train the automatic evaluator. We split the VIDEOPHY-2 dataset into test set for benchmarking and a training set for training the VIDEOPHY-2-AUTORN training and testing prompts consist of 3350 (197 actions x 17 captions per action) and 590 (197 actions x 3 captions per action) prompts, respectively.
model. Specifically, the
Benchmarking, For every tested model, we generate one video per each test prompt, that is, 591 videos per model 3 After generating the videos, we ask three annotators to evaluate them based on semantic adherence, overall physical commonsense, and violations of various physical rules. Annotators can also suggest additional physical rules that may be missing from our list. For every generated video, we compute the SA and PC scores (1-5) by averaging the three annotators scores and rounding to the nearest integer. Following this, the joint score is computed to assess the quality of the generated video. We use the majority voting for determining whether the listed physical rule (and law) is followed, violated, or cannot be grounded in the generated video. Additional human-written violations are converted to a statement of a physical rule (and law) using strong reference model, choose hard subset of 60 actions for which it
2.0nini-.00-Flasie Exp.
With Cog V5d achieved a zero joint performance. In our experiments, we observe that this hard subset leads to big drop in performances in comparison to the entire data across video models.
Entouvaluation Our experiments reveal that existing video-language models struggle to achieve strong agreement with human annotators. This discrepancy primarily arises due to their limited un- derstanding of physical commonsense and rules, as well as the complexity of the prompts. Hence, we supplement our benchmark with a video-language model Specifically, we aim to provide more accurate predictions for the generated videos along three axis semantic adherence score (1-5), physical commonsense score (1-5), and physical rule classification (0-2). We follow data- duta-driven approach to distill human knowledge into foundation model for these tasks. Specifically, we fine-tune a lideo-language model Srreeooon-Pysiceee g on 50K human annotations acquired for these tasks. We train a mult-task model to solve the three tasks using shared backbone, to allow the task knowledge transfer. We provide the templates and setup used for model finetuning in Appendix Mand Appendix[] respectively.
(7B parameters).
Training set for HIDEOPHY-2-AUTHT 2-AUTOEVAL. Within a limited data collection budget, sample I video per caption from one of the three video models including Hunyuan Video- 13B, Cosmos-
We exclude other closed models due lack of APT access (c.g., Veo2 [88] Kling 52)
'For Sora, however, we generate subset of 60 videos (randomly selected from 591). manually, using Sora playground (https: / /openai..... i. com/ /ssrr/////// / due to the lack of an official API, and 394 videos (2 prompts per action) for Ray2 due the limited API budget.
6

---

Published as a conference paper at ICLR 2026
Table 2:
Human evaluation results on VIDEOPHY-2. present joint performance that focuses on high semantic adherence and high physical commonsense in the generated videos. PA, OI refer to the physical activities, and object interactions subsets of the data, respectively. We mark the best performing models in each column by blue and second best by yellow.
Model
Class
All
Hard
PA
OI
Wan2.2-27B-A144B [60]
Open
55.4
47.7
54.5
58.6
Wan2.1-14B [60]
Open
32.6
21.9
31.5
36.2
Cog VogViccXX-5B [63
Open
25.0
0.0
24.6
26.1
Cosmos-Dift-7B
Open
24.1
10.9
22.6
27.4
Hunyuan- 13B [4
Open
17.2
6.2
17.6
15.9
VideoCrafter-2 17
Open
10.5
2.9
10.1
13.1
SVD-I2V [17
Open
6.0
3.3
5.2
8.7
Ray2
Closed 20.3
8.3
21.0
18.5
Sora 13
Closed 23.3
5.3
22.2
26.7
Diffusion-74 7B, and Cog VogVieoX--55 from the training set, of size 3350. All other video models are used to study the generalization capabilities of the auto-rater. Subsequently, we perform human annotations in the same way as the benchmarking process i.c., aggregating semantic adherence, physical commonsense and rule judgments across the three annotators. total, we collect roughly 50K human annotations across the three tasks, and spend $3515 USD collecting the training data. against several baselines on
Post- tes--trainnng we compare the performance of 2--HOOHY---AUTEEEE the semantic adherence and physical commonsense judgments using Pearson's 's correlation between the ground-truth and predicted scores. Further, we compare the joint score prediction accuracy and F1 score between auto-rater and selected baselines. In addition, compare the physical rule classification accuracy between the ViDEOPHY-2-AUTOEVA and baselines.
EXPERIMENTS
Here, we present the benchmarking results and the fine-grained analysis ($5.1). Then, we note the usefulness of our auto-rater against modern video-language models 5.2).
5.1 MAIN RESULTS
Performance on the dataset. We compare the joint performance of various open and closed text-to-video generative models the VIDEOPHY-2 dataset in Table _ Specifically,, we present their performance on the entire dataset, the hard split, and subsets focused on physical activities/sports (PA) and object interactions (OI). Even the best-performing model, Wan2.2-27B-AL B-A4B, achieves 55.4% and 47.7% (14% relative reduction) the full and hard splits of our dataset, respectively. On the other hand, we find that the second-best model, Wan2. achieves only 32.6% and 21.9% (33% relative reduction) the full and hard splits, respectively. This highlights at the effectiveness of higher model capacity in Wan2.2 in comparison to Wan2.1 without increase in inference cost using mixture- of-experts for different denoising timesteps. Furthermore, we observe that closed models, such as Ray2, perform worse than open models like 210.2227B-A144 and Cog VideoX-5-BB This suggests that closed models not necessarily superior to open models in capturing physical commonsense. Additionally, we find that performance on physical ac- tivities (sports) is generally lower than on object interac- tions across different video models. This suggests that future data curation efforts should focus on collecting high-quality sports activity videos (c.g., tennis, discus throw, baseball, cricket) to improve performance on the VIDEOPHY-2 dataset. Finally, we present the correlation between SA and PC judgments and other video metries, including aesthetics (measured using the LAION classi- fier 335 and motion quality (measured using optical flow from RAFT (56)), in Table 3 Our results reveal that phys- ical commonsense is not well-correlated with any of these video metrics. This indicates that a model dataset simply by optimizing for aesthetics and motion qual-
Table 3: Correlation analysis between mantic adherence and physical sense with aesthetics and motion video metrics.
Aesthetics
Motion SA
SA PC
0.1
0.02
0.09
0.002
0.14
cannot achieve high performance
7

---

Published as a conference paper at ICLR 2026
Ray2, Two joakis silt-by-siie, thiir wakes colliding spray
Ilunyuan. sledgshammer fonce post, casiing the ground.
(0 Cosmos- -.00, javelis thoon from high lands visible dust.
below,
Figure 3: Comparison of Wan2.2 with other models. The top row shows videos generated by Wan2.2: (a) For Ray2, the jetski on the left lags behind the other jetski and then starts moving backward. (b) For Hunyuan-13BB the sledgehammer deforms after the swing, and broken wooden board appears out of nowhere. (c) For Cosmos-78 the javelin expels sand before it even hits the ground.
(a) A yo-yo
hanging rope; yo-yo briefly soings back and forth before falling.
(h) person inflates
carefallly pushes Iny inside before gying it off.
Figure 4: Illustration Wan2.2's bad physical commonsense. Even the best- pest-performing model, Wan2.2, may struggle to correctly capture physical laws, leading to the generation of unnatural vidcos. Examples of such artifacts include: (a) A spinning yo-yo, deforming it spins, its strings disappearing from view. (b) A balloon deflating due external force despite it being tied up.
ity; rather, requires dedicated efforts incorporate physical commonsense into video generation. Overall, our findings suggest that VIDEOPHY-2 presents a significant challenge for modern video models, with substantial room for improvement future iterations.
Fine-grained Analysis. our human annota- tions, create list of physical rules (and as- sociated laws) that are violated in each video of the VIDEOPHI--2 dataset. We then analyze the fraction of instances in which physical law is violated gain fne-grained insights into model behavior. For example, 100 physical rules associated with the law of gravity and 25 of them are violated, the violation score would be 25%. We present the results of physical law violations in Figure 5) We observe that the con- servation of momentum and mass are among the most frequently violated physical laws, with violation scores of 40%, in the videos from the VIDEOPHY-2 dataset. Conversely, we find that reflection and buoyancy are relatively mastered with violation scores less than 20%
Physical Law Analysis on VideoPhy-2
Reflection
Bueyancy
Inertia
Gravity
Friction
Elasticity
Conserv. of Momentum
Conserv. Mass
Violations (%)
Figure 5: Physical laws violation analysis. We present the violation scores for diverse physical laws based on human annotations collected from various video genera- models on VIDEOPHY-2.
Qualitative Analysis. We perform qualita- tive analysis to provide visual insights into the model's mode of failures. We present qualitative examples in Figure model, Man2.2-27B-A143 with other video models. Notably, we observe violations of physical com- monsense, such as jetskis moving unnaturally reverse and the deformation of solid sledgehammer, defying the principles of elasticity. However, even Wan lacks physical commonsense, shown in Figure 4 In this case, we highlight that a yo-yo shown to deform drastically as it spins, defying its material properties. Further, cover model-specifer poor physical commonsense instances along the caption and human-judged physical violations in Appendix O For example, we show that the
Saureeuu to compare the best herf-performing
We conduct an agreement analysis across different physical laws that are derived from physical rules. We found that inter-annotator agreement scores ranged from 70% to 80%, with lower agreement for laws like elasticity, and higher agreement (up 80%) for laws such as reflection.
8

---

Published as a conference paper at ICLR 2026
Table Auto-rater evaluation results. present the pearson's 's correlation (x (x100) between the predicted scores and ground- bouund-ruuth scores (1-5) on the unseen prompts and unseen video models.
Unseen prompts
Unseen video models
Avg.
SA
PC
Avg.
SA
PC
Salcooon-Physiee
28.5
32.0
25.0
26.5
27.0
26.0
VideoCon 6
12.5
23.0
2.0
8.9
17.0
0.8
VideoLlava 38
16.0
30.0
2.0
19.0
33.0
5.0
VideoScore 23
13.5
17.0
10.0
9.0
5.0
13.0
Gemini-2..0-PashhEEExp
18.5
26.0
11.0
21.0
31.0
11.0
20.5
31.0
10.0
VIDEOPHY-2-AUTOEVAL
42.0
47.0
37.0
41.0
45.0
37.0
Rel. to Best (%)
+47.4
+46.99
+48.0
+49.0
+36.4
+61.5
Rel. Gemini-2.0 (%)
+127.0
+80.8
+236.4
+107.1
+45.2
+281.8
Table 5: Auto-rater evaluation on joint score judgments. We present the joint accuracy and Fl score between the predicted scores and ground-truth scores (0-1) for our ViDEOPINY--AUTTEVAL and MateoCon-Physiee
Unseen prompts
Unseen models
Method
Avg.
Acc.
F1
Avg.
Acc.
Fl
VideoCon-Physics [8]
39.1
75,6
2.6
39.6
75
4.2
VIDEOPIY-2-AUTOEVAL
65.1
79.1
51.1
62.8
76.3
49.3
Rel. to VideoCCn-PPhsies (%)
+66.4
+49.1
Sora-generated video violates the physical rule "The frisbee must contact the hand before any upward movement occurs' (Appendix Figure (3).. We also provide several qualitative examples across diverse
physical law violations across different models in Appendix P For example, we highlight that the "golf ball does not move after being struck by the golf club' for Ray2 (Figure 29).
5.2 VIDEOPIY--2AUTORR
To enable scalable judgments, we supplement the dataset with an automatic evaluator VIDEOPHY- 2-AUTOEVAL. We consider two settings: (a) unseen prompts, where assess the videos from seen video models generated using unseen (testing) prompts, and (b) unseen video models, where assess the videos from unseen video models using unseen prompts. We compare the correlation performance of VIDEOPHY-2-AUTEVAA against several baselines in Table In particular, VIDEOPHY-2-ALL A-TTOEVVL achieves relative gains of 47.4% and 49% on un- seen prompts and unseen video models, respectively, compared to the best- best-performing baselines. Fur- ther, auto-rater outperforms Gemini-2.0-Flashh Exp. with relative gains 81% in semantic ad- herence and 236% in physical commonsense judg- ments. Further, we evaluate the accuracy and Fl performance of VIDEOPHYY-2-ATToES VideoCon-Physies for joint score judgments in Table [] Our results show that AODEOPHY---UTTOR maintains a strong balance between joint accuracy and F1 scores. Finally, we assess the physical rule classification accuracy of POREOPHY---AUTOEEEE
Table 6: Auto-rater evaluation physi- cal rule classification. We present the ac- curacy results for VIDEOPHY-2-AUTORE and other video-language models on the rule classification tasks.
Unseen prompts
Unseen
models
Random
34.5
31.2
against
VideoLlava (38
38.1
38.7
59.2
57.1
ViDEOPHY-2-AUTORNE
78.7
72.9
Rel. to Best (%)
+32,9
+27.7
against baselines in Table 6. Our model achieves relative gains of 32.9% on unseen prompts and 27.7% on unseen video models compared to 2emini-2200Flashh-- Further, find that VIDEOPHY- 2-AUTOEVAL performs much better than Semini-Flash-22 This suggests that recent improvements in sfate-of-the-arr VLMs cused on enhancing semantic adherence or physical commonsense judgments for synthetic videos. Therefore, using open and free AutoEval remains the more reliable choice for the future. Thus,
2.5-Faah consistent with our findings for
not fo-

---

Published as a conference paper at ICLR 2026
unified auto-rater reliably handle variety of tasks, providing a robust tool for testing VIDEOPHY-2.
RELATED WORK
Several works have introduced benchmarks assess the physical plausibility of generated videos.
VIDEOPHY was an early effort in this direction, evaluating semantic adherence and physical plausibility across 688 prompts. VIDEOPHY-2 extends this work by introducing more extensive dataset of 3940 prompts, shifting the focus from material interactions to real-world actions, and providing a 5-point Likert scale for human evaluation. Furthermore, VIDEOPHY-2 explicitly annotates physical rules in each video, allowing for a more fine-grained understanding of model behavior.
Physics-IQ 48
conditions models initial frames of real videos and measures the similarity between predicted and ground-trun truth video continuations. While this approach offers valuable in- sights, is primarily designed for video prediction rather than open- ppen-ended text-to-video generation. Additionally, extending it to longer or more complex multi-event scenarios presents challenges.
PhyGenBench [47] proposes a benchmark of 160 prompts and an automated evaluation framework, PhyGenEval. This work introduces structured evaluations of physical reasoning; however, its relatively small scale makes generalization difficult. Additionally, it adopts strict one-to-one mapping between prompts and physical laws, whereas real-world physics often involves multiple interacting principles. Its evaluation relies on sequential vision-language model queries, which can introduce inconsistencies and increase computational complexity. VIDEOPHY-2 builds on these efforts by expanding dataset size, allowing for prompts that reflect multiple physical laws, and providing more interpretable evaluation framework.
WorldSim Bench [52] assesses video models' ability to act as world simulators by aligning their outputs with numerical solvers. While valuable, this approach faces challenges in bridging the sim-to-real gap, as physical simulations may not fully capture the complexity and variability of real-world interactions. VIDEOPHY-2 addresses this by exclusively using real-world videos ensure that evaluations remain closely aligned with practical physical dynamics.
Impossible Videos 33 aims to evaluate the ability of the video generation models to create "anti or counterfactual real-world" videos (e.g.. A person pours milk into a glass cup half filled with milk, but the amount of milk in the glass cup does not change all). In contrast, our work focuses on testing the ability of the models to follow physical commonsense and semantic adherence for rreal-world" actions. this regard, the goals of these works are entirely different. Methodologicallly, IPV collects 260 prompts while we cover much larger set of 4000 prompts. While IPV just evaluates "visual quality" and "semantic adherence" as Yes/No judgment from human annotators, our work performs much more fine-gres grained and broader evaluation. For instance, assess the physical commonsense and semantic adherence on a scale of 1-5 which captures finer human preferences. Further, we annotate for the physical rules that are not followed in each video. In addition, we train automatic evaluator for scalable evaluation on our dataset while IPV just tests the ability of existing VideoLLMs to solve their task instead of training their own. As result, the practitioners will have to rely on expensive proprietary models (GPT-4o) to do automatic evaluation on IPV but our AutoEval is open and free. We present more related work in Appendix B
CONCLUSION
We introduce VIDEOPHY-2, a benchmark for evaluating physical commonsense in videos generated by modern models. We reveal large gap in their ability to align with prompts and generate videos that follow physical commonsense. Further, we provide physical law violations and an auto-rater for scalable evaluation. Overall, this dataset advances our understanding of the current state of the video generative models as general-purpose world simulators.
REFERENCES
[1] Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model
10

---

Published as a conference paper at ICLR 2026
platform for physical ai. arXiv preprint 2TXX::2501.03355,2222
[2] Eren Can Aybek and Cetin Toraman. How many response categories are sufficient for likert type scales? an empirical study based on the item response theory. International Journal of Assessment Tools in Education, 9(2):33-547, 2022.
[3] Zechen Bai, Hai Ci, and Mike Zheng Shou. Impossible videos. arXiv preprinl wrXiv:2503.13377 2025.
[4] Anton Bakhtin, Laurens van der Maaten, Justin Johnson, Laura Gustafson, and Ross Girshick. Phyre: A new benchmark for physical reasoning, 2019. URL 1908, 05656
http:/wwwwww......aaa/o. abs/
[5] Philip J. Ball, Jakob Bauer, Frank Belletti, Bethanie Brownfield, Ariel Ephrat, Shlomi Fruchter, Agrim Gupta, Kristian Holshcimer, Aleksander Holynski, Jiri Hron, Christos Kaplanis, Marjorie Limont, Matt McGill, Yanko Oliveira, Jack Parker-Holder, Frank Perbet, Guy Scully, Jeremy Shar, Stephen Spencer, Omer Tov, Ruben Villegas, Emma Wang, Jessica Yung, Cip Baetu, Jordi Berbel, David Bridson, Jake Bruce, Gavin Buttimore, Sarah Chakera, Bilva Chandra, Paul Collins, Alex Cullum, Bogdan Damoc, Vibha Dasagi, Maxime Gazeau, Charles Gbadamosi, Woohyun Han, Ed Hirst, Ashyana Kachra, Lucie Kerley, Kristian Kjems, Eva Knoepfel, Vika Koriakin, Jessica Lo, Cong Lu, Zeb Mehring, Alex Moufarek, Henna Nandwani, Valeria Oliveira, Fabio Pardo, Jane Park, Andrew Pierson, Ben Poole, Helen Ran, Tim Manuel Sanchez, Igor Saprykin, Amy Shen, Sailesh Sidhwani. Duncan Smith, Joe Stanton, Hamish Tomlinson, Dimple Vijaykumar, Luyu Wang, Piers Wingfield, Nat Wong, Keyang Xu, Christopher Yew, Nick Young, Vadim Zubov, Douglas Eck, Dumitru Erhan, Koray Kavukcuoglu, Demis Hassabis, Zoubin Gharamani, Raia Hadsell, Aäron van den Oord, Inbar Mosseri, Adrian Bolton, Satinder Singh, and Tim Rocktäschel. Genie 3: A new frontier for world models. 2025.
Salimans,
[6] Hritik Bansal, Yonatan Bitton, Idan Szpektor, Kai-Wei Chang, and Aditya Grover. Videocon: Robust video-language alignment via contrast captions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13927-13937, 2024.
[7] Hritik Bansal, Yonatan Bitton, Michal Yarom, Idan Szpektor, Aditya Grover, and Kai-Wei Chang. Talc: Time-aligned captions for multi-scene text-to-video generation. arXiv preprint wrXiv:2405.04682. 2024.
[8] Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang. and Aditya Grover. Videophy: Evaluating physical commonsense for video generation. In The Thirleenth International Conference on Learning Representations, 2025. URL,
[9] Peter W Battaglia, Jessica B Hamrick, and Joshua B Tenenbaum. Simulation engine of physical scene understanding. Proceedings of the National Academy of Sciences, 110(45): 18327-18332, 2013.
[10] Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piqa: Reasoning about physical commonsense in natural language, 2019. URL
1911.11641
[11] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint 15512 2023.
[12] Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models, 2023. /0ll
[13] Tim Brooks, Bill Peebles, Connor Holmes, Will DePue. Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman. Eric Luhman, Clarence Ng. Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024, URL httpps://ppeaaai wtt.ecco//ceeeeeeeeeee research/ ators
video- geneo-genrre -ontioon--mmeo oonmodeell s- wor
11

---

Published as a conference paper at ICLR 2026
[14] Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Bechtle, Feryal Behbahani, Stephanie Chan, Nicolas Heess, Lucy Gonzalez, Simon Osindero, Sherjil Ozair, Scott Reed, Jingwei Zhang, Konrad Zolna, Jeff Clune, Nando de Freitas, Satinder Singh, and Tim Rocktäschel. Genie: Generative interactive environments, 2024. URL https
wxzxiv.org/absa...... 2402. 15391
[15] Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Genie: Generative interactive environments. In Forty-first International Conference on Machine Learning, 2024.
[16] Joao Carreira, Eric Noland, Andras Banki-Hou Horvath, Chloe Hillier, and Andrew Zisserman. A short note about kinetics-600. arXiv preprint arXiv: 180111888110000 04340, 2018.
[17] Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models, 2024.
[18] Google DeepMind, Introducing Gemini 2.0: our new AI model for the agentic era - blog.google. https://blog. wtw.l/gggggoooleeeeeeeeeoo oogy/ howploloyygggllloooooooooooooole 2024/ 2025.
[19] Yilun Du, Sherry Yang. Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation. Advances in Neural Information Processing Systems, 36, 2024.
[20] Jiafei Duan, Arijit Dasgupta, Jason Fischer, and Cheston Tan. survey on machine learning approaches for modelling intuitive physics. arXiv preprint 22Xiv:2202.06481, 2022.
[21] GoogleDeepMind Genie2.
Genic 2: A large-scale foundation world model
deepmind.google
2024. [22] Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michalski, Joanna Materzynska, Susanne Westphal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag. al. The" something something" video database for learning and evaluating visual common sense. In Proceedings of the IEEE international conference on computer vision, pp. 5842-5850, 2017.
[23] Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj. et al. Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation. arXiv preprint arXiv:2406. /5252, 2024.
[24] Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David Fleet, and Tim Salimans. Imagen video: High definition video generation with diffusion models, 2022. URL (thhtps
7/arxiv. www./vrrrrreeeeeee wwtp://rgggaabaaeoeeoooo abs/
02303
[25] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusson models, 2022. URL /tattps://axxi..oom org/ abs 2bd//204..0 03455.0345 03458
[26] Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205. 15868, 2022.
[27] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu Yuanzhi Li. Shean Wang. Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1 (2):3, 2022.
[28] Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21807-21818. 2024.
12

---

Published as a conference paper at ICLR 2026
[29] Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan. Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. text2video-zero: imxge diffusion models zero-shot video generators, 2023. URL httphh:ts/:/aaaaaa /BL.attss///avvv......//////////
[30] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.69800 2014.
[31] Gergely Klár, Theodore Gast, Andre Pradhana, Chuyuan Fu, Craig Schroeder, Chenfanfu Jiang, and Joseph Teran. Drucker-prager clastoplasticity for sand animation. ACM Transactions on Graphics (TOG), 35(4):1-12, 2016.
[32] KlingAI KLING AI - klingai.com. https://www kaikkkinnnni com/ 2024.
[33] Dan Kondratyuk; Lijun Yu, Xiuye Gu, José Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vighnesh Birodkar, Jimmy Yan, Ming-Chang Chiu, Krishna Somandepalli, Hassan Akbari, Yair Alon, Yong Cheng, Josh Dillon, Agrim Gupta, Meera Hahn, Anja Hauth, David Hendon, Alonso Martinez, David Minnen, Mikhail Sirotenko, Kihyuk Sohn, Xuan Yang, Hsuan Yang, Irfan Essa, Huisheng Wang, David A. Ross, Bryan Seybold, and Lu Jiang. Videopoet: A large language model for zero-shot video generation, 2024. URL Sittps://aae /ttp://rxxiii wrw..orrg/aee.
Hartwig Adam, Ming-Hase
wttp///www.a..aaaaaooooaapp////t httpa//wwwww.rrr////////3/////// abt////ss//////// 14125
[34] Weijie Kong. Qi Tian, Zijian Zhang, Rox Min. Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03600; 2024.
[35] Laion. GitHub
A linear estimator top of clip to pre- dict the aesthetic quality of pictures - github.com. htttp:: com/ com/LAION LAION- AI/ aesthetic-predice.
[36] Junbang Liang, Ruoshi Liu, Ege Ozguroglu, Sruthi Sudhakar, Achal Dave, Pavel Tokmakov, Shuran Song, and Carl Vondrick. Dreamitate: Real-world visuomotor policy learning via video generation. arXiv preprint arXiv:2406. 16862, 2024.
[37] Rensis Likert. A technique for the measurement of attitudes. Archives of psychology, 1932.
[38] Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:231 10112,, 2023.
[39] Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to- visual generation with image-to-text generation, 2024.
URL htthtt:::/// pttttps://ariiv.. org
www.oorg///bb.o..ooooo..oe..eese abs/ 2404 01291
[40] Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shenlong Wang. Physgen: Rigid-body physics-grounde image-to-video generation. In European Conference Computer Vision, pp. 360-378. Springer, 2024.
[41] Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. Evalcrafter: Benchmarking and evaluating large video generation models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22139-22149, 2024.
[42] Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jianfeng Gao, et al. Sora: A review background, technology, limitations, and opportunities of large vision models. arXiv preprint 10X1 2024.
[43] Imarena. Chatbot Arena Leaderboard a Hugging Face Space by Imarena- ai huggingface.co.
chatbot-an sertbottrrena-ae Leaderboard
[44] LumaAI. Luma Ray2 aumalabs.a. https://lumg labs. ai ai/ray 2025.
13

---

Published as a conference paper at ICLR 2026
[45] Zhuoyan Luo, Fengyuan Shi, Yixiao Ge, Yujiu Yang, Limin Wang, and Ying Shan. Open- magvit2: An open- project toward democratizing auto-regressive visual generation, 2025.
URL /ttps::::///x org/ www.ooggg///.//oo////....e..eeae 2409. 0441C
[46] Michael McCloskey, Allyson Washburn, and Linda Felch. Intuitive physics: the straight-down belief and its origin. Journal of Experimental Psychology: Learning, Memory, and Cognition, (4):636, 1983.
[47] Fanqing Meng. Jiaqi Liao, Xinyu Tan, Wenqi Shao, Quanfeng Lu, Kaipeng Zhang, Yu Cheng, Dianqi Li, Yu Qiao, and Ping Luo. Towards world simulator: Crafting physical commonsense- based benchmark for video generation. arXiv preprint ar urXiv:2410003363, 2024.
[48] Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Rohert Geirhos. Do generative video models learn physical principles from watching videos? arXiv preprint arXiv:2501.0900 2025.
[49] James F O'brrien, Adam W Bargteil, and Jessica K Hodgins. Graphical modeling and animation of ductile fracture. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques, pp. 291-294, 2002.
[50] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
[51] Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Sean Shi, Michael Choi, Anish Agrawal, Arnv Chopra, et al. Humanity's last exam. arXiv preprint arXiv:2501.14249, 2025.
[52] Yiran Qin, Zhelun Shi, Jiwen Yu, Xijun Wang, Enshen Zhou, Lijun Li, Zhenfei Yin, Xihui Liu, Lu Sheng. Jing Shao, al. Worldsimbench: Towards video generation models as world simulators. arXiv preprint arXiv:2410. 18072, 2024.
[53] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pp. 8748-8763. PMLR, 2021.
[54] Jonathan Roberts, Mohammad Reza Taesiri, Ansh Sharma, Akash Gupta, Samuel Roberts, Ioana Croitoru, Simion- Vlad Bogolin, Jialu Tang, Florian Langer, Vyas Raina, et al. Zerobench: An impossible visual benchmark for contemporary large multimodal models. arXiv preprint arXiv:2502.09696 2025.
[55] Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0400, 2012.
[56) Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer EEcionEECCV 2020: 16th European Conference, Glasgow, UK. August 23-28 2020, Proceedings, Part 11 16, pp. 402-419 Springer, 2020.
[57] Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric and challenges, 2019. URL htRplpttts/:: org abs///12..1111111.../11/11.11...
[58]
GoogleDeepMind Vco2. Veo 2 deepmind google.
google/
Lechnoologg 2/| 2024.
[59] Ruben Villegas, Mohammad Babacizadeh. Pieter-Jan Kindermans, Hernan Moraldo, Han
Zhang, Mohammad Taghi Saffar, Santiago Castro, Julius Kunze, and Dumitru Erhan. Phenaki: Variable length video generation from open domain textual description, 2022. URL https
7/arxiv. f.axxlv..rr//bb//2111, 02399
[60] Wan. Wan-Al/Wan2.1-DV-144
Hugging Face
huggingface.co.
https://
huggingface. wtc.c////ann Man2. 1 T2V- 14B 2025.
14

---

Published as a conference paper at ICLR 2026
[61] Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-videe -iveeoo technical report, 2023. URL http::t//txtxt..coomm orq/ abs 2308
06571
[62] Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang. Yuwei Guo, Tianxing Wu, Chenyang Si, Yuming Jiang, Cunjian Chen, Chen Change Loy, Bo Dai, Dahua Lin, Yu Qiao, and Ziwei Liu. Lavie: High-quality video generation with cascaded latent diffusion models, 2023. URL https
//arxiv.org/abs//// /2099 15103
[63] Zhuoyi Yang. Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06077, 2024.
[64] Michal Yarom, Yonatan Bitton, Soravit Changpinyo, Roee Aharoni, Jonathan Herzig, Oran Lang, Eran Ofek, and Idan Szpektor. What you see is what you read? improving text-image alignment evaluation. Advances in Neural Information Processing Systems, 36:1601-1619, 2023.
15


Reviews:
[{"review_id": "qp7moIGRcf", "reviewer": "Reviewer_QRAy", "summary": "This paper introduces VIDEOPHY-2, a large-scale benchmark for evaluating physical commonsense in text-to-video generation. The dataset consists of 3,940 LLM-generated prompts covering 197 real-world actions, with detailed human annotations on semantic adherence, physical commonsense, and rule violations. It includes a hard subset emphasizing physics-rich and multi-event scenarios (e.g., backflip, throwing discus). The authors also develop VIDEOPHY-2-AUTOEVAL, a fine-tuned video-language model trained on 50K human annotations for scalable automatic evaluation. Extensive experiments on leading open and closed video models (e.g., Wan2.2-27B, Sora, CogVideoX) show that current models struggle with physical plausibility—particularly with conservation laws of mass and momentum—achieving only 47.7% joint adherence on the hard subset.", "questions": "1. Does training on VIDEOPHY-2 improve a model’s physical commonsense generation?\n2. Could the authors release example failure cases and annotation guidelines to help the community standardize future evaluation protocols?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed benchmark addresses an important gap in evaluating physical realism in video generation, a key step toward world-modeling AI, it extends the scale with more fine-grained physical law annotations compared to previous version VIDEOPHY.\n\n2. The proposed VIDEOPHY-2-AUTOEVAL is valuable and achieves large correlation gains over strong baselines (e.g., Gemini-2.0-Flash-Exp).\n\n3. Thorough experiments and insightful analysis: The fine-grained breakdown of violated physical laws (e.g., momentum, elasticity) provides deep diagnostic insight.", "weaknesses": "1. Missing related work discussion: The paper does not reference Impossible Videos, which also proposes a benchmark including evaluating violations of physical laws in video generation. A detailed comparison and discussion are needed to clarify the conceptual and methodological differences between VIDEOPHY-2 and that work.\n\n2. Prompt ambiguity in multi-event scenarios: The authors encourage LLMs to generate prompts depicting multiple events within a single description. While this increases task difficulty, it also introduces ambiguity when evaluating semantic adherence—for instance, when a video generator accurately follows only part of the prompt. How is semantic adherence rated in such cases? Is there a standardized or clearly defined evaluation guideline to handle partial adherence?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CIFrr8Aj6L", "reviewer": "Reviewer_VSb4", "summary": "The paper introduces VIDEOPHY-2, a large-scale benchmark for testing whether text-to-video generation models can follow basic physical commonsense. It expands the earlier VIDEOPHY dataset with 197 actions and 3,940 detailed prompts, focusing on action-centric, physics-rich scenarios. Authors also train VideoPhy-2-AutoEval, an automatic evaluator for fast assessment on their proposed dataset.", "questions": "1.See Weaknesses.\n2. What is the inter-annotator agreement among human evaluators? The paper should include an analysis of annotation consistency to verify the reliability of human evaluation.\n3. The amount of work in this paper is impressive, and I appreciate the effort. However, the overall writing quality is poor. The authors should seriously reconsider how to present and organize their work more clearly and coherently.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. The paper proposes a large-scale, carefully annotated benchmark for assessing the physical understanding ability of video generation models.\n2. An automatic evaluator is trained to approximate human judgments, which is useful in principle, though not always effective in practice.", "weaknesses": "1. I really dislike the presentation. Figure 2 does not clearly correspond to Section 2 (for example, the dense captioning component is not reflected in the figure). In Section 3.1, the distinction between PC and PR is unclear. Several parts are verbose and poorly organized; for instance, around line 275, experimental descriptions appear within the evaluation methodology section. Similar issues occur throughout the paper.\n2. The experimental setup is unfair, as some models use upsampled captions while others do not. The authors should generate and evaluate videos separately for fair comparison.\n3. The experimental result and tables should be more detailed. The main results table (Table 2) should explicitly report all metrics such as SA and PC.\n4. According to Table 4, the pearson's correlation between VIDEOPHY-2-AUTOEVAL and human ratings is below 0.5, indicating that the proposed automatic evaluator does not generalize well to unseen scenarios. Therefore, I have concerns about the benchmark’s reliability for automatic evaluation, especially since human evaluation of video generation models is labor-intensive and inefficient.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1tRZWHAO30", "reviewer": "Reviewer_74jg", "summary": "VIDEOPHY-2 is a large-scale benchmark for evaluating the physical common sense of video generation, centered on real-world \"actions\". It includes 3940 detailed prompts, multi-human evaluation, and fine-grained physics rule annotations, and provides an automated evaluator. Experiments show that even the strongest model achieves only about 47.7% of the joint semantic and physical metrics on hard sets, particularly weak in conservation laws (mass, momentum), highlighting the significant shortcomings of current video generation in terms of physical consistency.", "questions": "see the weaknesses", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The benchmark is based on 197 real-world actions and 3940 long text prompts, covering diverse scenarios that closely resemble everyday physical situations.\n\nIt includes human-annotated semantic and physical scores, along with compliance and violation annotations at the level of physical rules/laws, supporting detailed diagnostics.\n\nIt provides an AUTOEVAL model to improve evaluation efficiency and consistency with human judgment, facilitating large-scale, rapid model comparisons.", "weaknesses": "This paper has limited noverty, and the only difference between it and VideoPhy seems to be the addition of more prompts; it is not suitable as a standalone conference paper.\n\nTable 5 tests the OOD generalization ability of the eval model, but lacks comparisons using some cutting-edge VLMs, such as Qwen and GPT.\n\nThe paper lacks insight into how to enhance the ability of T2V models to generate physically consistent videos. Several necessary experiments could provide further insights, such as SFT, using an Eval model as a reward model for RL training, or RFT.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KVk54l7ix4", "reviewer": "Reviewer_tmU3", "summary": "Video generation models can produce realistic videos, making them ideal candidates for simulating the physical world. However, evaluating how well these models adhere to physical commonsense in real-world actions remains unclear. Existing models face issues such as limited size and lack of human evaluation, making it difficult to assess their ability to align with physical laws. In this work, the authors propose the Videophy-2 benchmark, which includes a dataset of 4000 diverse and detailed prompts, along with an automatic evaluator. Experimental results show that, under the Videophy-2 standard, even the best model achieves only 47.7% joint performance, exposing critical gaps in video generative models and providing direction for future research in physically-grounded video generation.", "questions": "- Is it possible to prove that Videophy-2-autoeval does not suffer from evaluation bias?\n\nIn this paper, the authors collect labeled data sampled from HunyuanVideo-13B, Cosmos-Diffusion-7B, and CogVideoX-5B as the training set for Videophy-2-autoeval and evaluate its performance on unseen prompts and unseen video models. However, the videos generated by HunyuanVideo-13B, Cosmos-Diffusion-7B, and CogVideoX-5B may contain biases in terms of adhering to or violating physical laws. Given that the annotator is trained on data generated by these models, can it be considered a fair standard for evaluating future video models?\n\n- Would incorporating real videos into the training set of Videophy-2-autoeval improve its evaluation performance?\n\nThe training set of Videophy-2-autoeval only includes synthetic data, whereas real videos inherently possess the best physical commonsense. It is a question that adding real videos to the training data and constructing a diverse training set by providing different prompt-video and rule-video pairs could help make Videophy-2-autoeval closer to human annotators.\n\n- Can I2V models be evaluated using VideoPhy-2?\n\nIn this paper, the authors mainly compare the performance of text-to-video (T2V) models in terms of physical alignment. For image-to-video (I2V) models, the initial image should significantly influence the evaluation score. The authors evaluate the performance of the SVD-I2V model by first generating an image using Stable Diffusion and then guiding the SVD-I2V model to generate a video based on that image. Is there a better method to assess the physical alignment performance of I2V models?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- It provides a challenging benchmark for physically-grounded video generation.\n\nIn the hard subset of Videophy-2, even the highest-performing model, Wan2.2-27B-A14B, only achieves a score of 47.7%. This means that Videophy-2 is a stringent benchmark, setting a standard for future evaluations of video generation models in terms of physical alignment.\n\n- Detailed evaluation dimensions.\n\n Unlike existing work that combines semantic and physical evaluations, leading to potential biases, Videophy-2 uses three distinct dimensions: Semantic Adherence (SA), Physical Commonsense (PC), and Physical Rules (PR). These dimensions evaluate whether the video aligns with the prompt, whether the video is physically aligned with the real world, and whether the video follows a specific physical law (e.g., gravity). This approach minimizes evaluation biases as much as possible.\n\n- An effective automated evaluation approach.\n\nAlthough human judgments can serve as the gold standard, the cost of human annotation limits the widespread application of evaluations. Therefore, Videophy-2 introduces an automatic evaluation method, using the VideoCon-Physics fine-tuned Videophy-2-autoeval (7B parameters) as the annotator. Experiments show that Pearson’s correlation of Videophy-2-autoeval exceeds that of several VLMs, including Gemini-2.0-Flash-Exp, and surpasses VideoCon-Physics in F1 score.", "weaknesses": "- The lack of comparison with more advanced VLMs as annotators.\n\nThe most advanced closed-source VLM used in this paper is Gemini-2.0-Flash-Exp, but there is no comparison with stronger models like Gemini-2.5. Given the relatively small evaluation dataset, using stronger VLMs as annotators would increase the credibility of the conclusions drawn in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)