$PhyWorldBench$: A Comprehensive Evaluation of Physical Realism in Text-to-Video Models
Authors: Jing Gu
Xian Liu
Yu Zeng
Ashwin Nagarajan
Fangrui Zhu
Daniel Hong
Yue Fan
Qianqi Yan
Kaiwen Zhou
Ming-Yu Liu
Xin Eric Wang
Venue: ICLR 2026 Oral (2026)
Keywords: Video Generation
Video Evaluation
Primary area: datasets and benchmarks
TL;DR: Large-scale, multidimensional video generation for physics

Abstract: Video generation models have achieved remarkable progress in creating high-quality, photorealistic content. However, their ability to accurately simulate physical phenomena remains a critical and unresolved challenge. This paper presents $PhyWorldBench$
, a comprehensive benchmark designed to evaluate video generation models based on their adherence to the laws of physics. The benchmark covers multiple levels of physical phenomena, ranging from fundamental principles like object motion and energy conservation to more complex scenarios involving rigid body interactions and human or animal motion. Additionally, we introduce a novel "Anti-Physics" category, where prompts intentionally violate real-world physics, enabling the assessment of whether models can follow such instructions while maintaining logical consistency. Besides large-scale human evaluation, we also design a simple yet effective method that could utilize current MLLM to evaluate the physics realism in a zero-shot fashion. We evaluate 10 state-of-the-art text-to-video generation models, including five open-source and five proprietary models, with a detailed comparison and analysis. we identify pivotal challenges models face in adhering to real-world physics. Through systematic testing of their outputs across 1,050 curated prompts—spanning fundamental, composite, and anti-physics scenarios—we identify pivotal challenges these models face in adhering to real-world physics. We then rigorously examine their performance on diverse physical phenomena with varying prompt types, deriving targeted recommendations for crafting prompts that enhance fidelity to physical principles.

Published as a conference paper at ICLR 2026
PHYWORLDBENCH: A COMPREHENSIVE EVALUATION OF PHYSICAL REALISM IN TEXT-TO-VIDEE MODELS
Jing Gul, Xian Liul, Yu Zeng', Ashwin Nagarajan', Fangrui Zhu Daniel Hong', Yue Fan Qianqi Yant, Kaiwen Zhou', Ming-Yu Yu Liut>" Xin Eric Wang 4,+
University of California, Santa Cruz NVIDIA Research 8 Northeastern University; 9 University of California, Santa Barbara; Co-advisors
Structured Benchmark Design
Fundamental Physics Categories
Object Mation Kinemeties
Intersction Dynanies
Deformations Elasticity
LevES
Liner Metiee
Nateraa Flicliitty
Circalor Marion
Orawtty and Free Bactrones sift
Partic Daformticr
Man
Boistinn Kaat
Brame Frachue
Horst Matee
lintic and Comperation
Contruction
Sabcotagorisa
from difeerei sspacti)
Energy Conservation
Flddd and Particle Dynomics
Lighting and Shadown
Variosa Scenarios under each subsotegory
Soeertel to sirats Onarge
Witee Mtation
Comvnnecy Ilummintonn
evegg Thorsher and Phose
Onergy Los
Cartere un Mucharical
Pericle Renoo
buppancy and Rlathin
Chramdhs Averrtt
Prompt Variations
Shadow Softnens
Composite Physics Categories
Anti-Physics
Animal Mation
Rigid Body Dynanics
Scals and Proportiona
Irfenctiooss
Time Reveroll Witertor
Fnergy Oiethor ferriisel Malion
Pholing Sarmell
Depliction
Leoer Accuss Tarriii
Canndralicc und Rise Metee
feaubiily
Inaluntary
Car fertenatt Mara
Rutatton and Tarppe
pirpat Defommatio
Scdas
Stracuuaal Eineerly Aciil
Socae oifirer
Buating Mushaatul
Figure I: Overview of PhyWorldBench. The benchmark follows a structured design, starting with 10 main physics categories, derived from physics literature and expert consultations. Each category is divided into 5 subcategories, capturing different aspects. Under each subcategory. scenarios are created, with 3 prompt variations per scenario to provide varying levels of detail and complexity. The figure presents the benchmark structure, showcasing the 10 main categories and their corresponding 5 subcategories.
ABSTRACT
Video generation models have achieved remarkable progress in creating high- quality, photorealistic content. However, their ability to accurately simulate physi- cal phenomena remains a critical and unresolved challenge. This paper presents comprehensive benchmark designed to evaluate video gen- eration models based on their adherence to the laws of physics. The benchmark covers multiple levels of physical phenomena, ranging from fundamental principles like object motion and energy conservation to more complex scenarios involving rigid body interactions and human or animal motion. Additionally, we introduce a novel "Anti-Physics" category, where prompts intentionally violate real-world physics, enabling the assessment of whether models can follow such instructions while maintaining logical consistency. Besides large-scale human evaluation, we also design a simple yet effective method that could utilize current MILM evaluate the physics realism in zero-shot fashion. We evaluate 12 state-of-the-an text-to-video generation models, including five open-source and five proprictary models, with detailed comparison and analysis. we identify pivotal challenges models face in adhering to real-world physics. Through systematic testing of their outputs across 1,050 curated cacnmnsssspamnne fundamental, composite, and anti- physics scenarios-we we identify pivotal challenges these models face in adhering to real-world physics. We then rigorously examine their performance diverse phys-

---

Published as a conference paper at ICLR 2026
Proprietary Madels
Open-Sourced Models
0.300
Physics Success Percentage
0.225
0,150
0.075
0,000
Pika
Sora
Kling
Luma
Gen-3
Wank
Hunyuan
Step-Video-T2V
Open-Sara
CogVideo
Open-Soro-Plan
LTX-Vidoo
Figure 2: Success rates of video generation models PhyWorldBench. Among open-source models, Wanx demonstrated the highest performance, while Pika achieved the best results among proprictary models with success rate of 0.262. Despite these advancements, substantial progress remains necessary to refine the capability of these models to accurately simulate the intricate dynamics of the real world.
ical phenomena with varying prompt types, deriving targeted recommendations for crafting prompts that enhance fidelity to physical principles.
INTRODUCTION
The field of video generation has made remarkable progress, with models producing visually com- pelling and often photorealistic outputs. These advances have enabled transformative applications across industries such as entertainment, education, and scientific visualization. However, despite their visual fidelity, do video generation models truly understand the laws of physics in the real world? To answer this question, we introduce how well video generation models can simulate real-world physics. As illustrated in Figure [ AhyWorldBend systematically tests models across multiple levels of physical phenomena, from fundamental concepts like object motion to complex dynamics, including rigid body interactions and human/animal motion. Additionally, deliberately violate real-world physics. On one hand, this design verifies whether models genuinely understand physical laws-rather than merely reproducing patterns from real-world training data. On the other hand, anti-physics content itself holds practical value in creative applications, where imaginative otherwise impossible scenarios are beneficial.
dBBnch, rigorous benchmark designed to evaluate
propose novel Anti-Physics category. where prompts
We meticulously designed and annotated 1,050 prompts and the standard set for each prompt individually to cover broad range of physical scenarios. This substantial annotation work ensures that our benchmark is both comprehensive and precise, allowing for more thorough assessment of video generation models' capabilities. Furthermore, we present context-aware aware-prompt metric using MILM (OpenAl Team 2024. Gemini Team 2024), which directly assesses if the video satisfies the physics standards or not. Such evaluation not only provided an unbiased metrie but also significantly reduced the evaluation cost.
To examine the current status of video generation models and provide a detailed analysis, we selected five proprietary Moodll-SSora-Turie
(OpenAl 2024), Gen-3 (Runway Team 2024), Kling
1.6 [KlingAl 2024), Pika 2.0 (Pika Labs Team 2024)), and Luma (Luma Al Team 2024) along with seven open-souree models: Hunyuan 720p (Kong al. 2024), Open-Sora 2.0 Peng al. 2025). Open-Sora-Plie 1.3 (Lin et al. 2024). CogVideoX-133 (Hong et al. 2002). Step-video T2V (Ma et al. 2025), Wanx-2.1 (Wan Team et al. 2025). and LTX Video (HaCohen al. 2024). We generated
2

---

Published as a conference paper at ICLR 2026
12,600 videos to evaluate SOTA models and analyze their ability to simulate real-world physics. We identify challenges, difficult scenarios, and key physics categories while proposing a structured approach to improve physical realism through prompt design.
The insights obtained from this benchmark will contribute to the development of more robust and phys- ically accurate video generation models, addressing both fundamental scientific questions and practi- cal challenges in the field. Figure 2 presents the overall benchmark results on PhyWor lBenchh. Despite recent advancements, models continue struggle with temporal consistency, realistic mo- tion, and physical plausibility, emphasizing the need for further research to enhance their physics correctness for real-world simulation.
Our work provides a benchmark for video generation models with a focus on physical realism. The key contributions are:
We propose PhyWor dBBnch, a large-scale, multi-dimennional physics benchmark for evaluating the physics ability of the video generation model.
We conduct an extensive evaluation of twelve state-of-the-arr video generation models (five pro- prietary and seven open-source) with 12,600 generated videos and identified key challenges in simulating real-world physics.
We study the effect of prompt variation on the performance of the video generation model and videos.
provided prompt guidelines for generating
2 RELATED WORK
Benchmarks for Text-to-Vide Generation. Proprictary video generators (OpenAl 2024 KlingAI
2024;
Pika Labs Team 2024; Runway Team 2024; HailuoA[ 2024 Luma Al Team 2024) achieve
striking quality and temporal coherence but remain opaque. Open- source counterparts (Peng et al.
2025 Lin et al. 2024 Hong et al.] 2022 Wang et al. 2023 HaCohen et al. 2024 Kong et al.
2024 Agarwal et al.] 2025) enable reproducible research and flexible benchmarking The develop-
ment of Text-to-Video (T2V) Generation models has been accelerated by benchmarks that evaluate performance across diverse aspects like quality, realism, and compositionality, VBench (Huang et al. [024] and EvalCrafter (Liu et al.] 2024) provide comprehensive frameworks for assessing video generation models, focusing on metrics such as diversity, temporal coherence, and scalability. Besides, T2V-CompBench (Sun et al. 2024) emphasizes compositionality, ensuring generated videos align with intricate textual prompts, a key challenge in real-world applications. Recent works em- phasize the importance of physical plausibility in video generation. Morpheus et al. (2025a) focuses on experimental physics. VideoPhy (Bansal et al.) 2024) and PhyGenBench (Meng et al. 2024) assess models' adherence to physical commonsense, while DEVIL (Liao et al. 2024) and VideoPhy-2 [Bansal et al. 2025) evaluates dynamic realism, including motion and temporal consis- tency. Physics-IQ (Motameda al. (2255) further tests understanding of fundamental principles like fluid dynamics and thermodynamics, highlighting the demand for physics-aware video generation.
Many physics-focused video generation models are also built (Zhang et al.) 2025b Yuan et al. 2025
Wang et al. 2025).
We introduce novel and substantially expanded dataset for evaluating video generation models across a broader range of physical laws. Unlike prior benchmarks, it includes diverse, carefully curated scenarios with both real and deliberately unphysical dynamics. Models perform significantly worse on our benchmark under identical settings, revealing its greater difficulty and realism.
Evaluation Metrics for Text-to- Video Generation. Traditional text-to-video evaluation metrics like Fréchet Video Distance (FVD) (Unterthiner et al.) (018)) and Inception Score (IS) (Salimans et al. 2016) focus on visual quality but fail assess motion plausibility and adherence physical laws, especially without reference videos. Recent methods address this gap: VideoScore (He et al. 2024) integrates human feedback, T2V-CompBench (Sun et al. 2024) uses VLMs for spatial and temporal evaluation, and PhyGenBench (Meng et al.) 2024) explicitly tests physical commonsense. However, these methods can be computationally intensive or subjective. We introduce a Yes/No answering metric for efficient, scalable, and objective evaluation of physical commonsense in video generation. Additionally, we propose a simple yet effective strategy leveraging MLLMs like GPT-ol to assess video quality.
3

---

Published as a conference paper at ICLR 2026
Create Physies Categories
Prompt Crafting
Evaluation Standards
Physics Textbooks
GPT-40
Gemini
Human
Expert Consultations
Human
Clear
Clear
Same standard
Exclusive
Diverse
Correct
Comprehensive
Correct
Ambiguous
thee--o Review
Figure 3: Creation Process of PhyWorldBench. The dataset is built through three-stage pipeline for clarity, consistency, and completeness. First, physics categories and prompts are defined using literature and expert input. Next, GPT-4o, Gemini-1.55 1.5-Pro together with human refine prompts for diversity and accuracy, Finally, curation phase standardizes all prompts, with suman-in-the-looy reviews ensuring clarity and eliminating ambiguities.
(Main category) Object Mation and Kinematics (Subcategory) Linear Mation Scenario Sample
(1) prompt: rocket lounches info sky.
Time
Physict-erhanced prompt: "A rockel launches verl info the sky, troveling shraight info the air'"
Time
lowering
Rey landscups engines rearing
indigo groand upwards. momertorily elongated patn, capturing every
glow,
brightness,
strofosphere' Time
Figure 4: Example of Three Prompt Types for Scenario. This figure illustrates scenario from the subcategory Linear Motion under the main category Object Motion and Kinematics. The scenario is presented through three levels of prompts: (1) Event Prompt, providing concise and straightforward event description; (2) by incorporating telstcs-related phenomena while avoiding explicit physical laws; and (3) Detailed Narrative Prompt, enriching the Evnet Prompt with vivid details and contextual elements.
Prompt, which builds on the general prompt
PHYWORLDBENCE
To evaluate the ability of text-to-video generation models to simulate physical reality, we introduce PhyWorl dBench, comprehensive benchmark spanning wide range of physical principles and scenarios. Its design is rooted in fundamental physics concepts such as motion, energy conser- vation, and interaction dynamics, drawing from established literature, including Fundamentals of Physics (Halliday et al. 2013) and Classical Mechanics (Goldstein et al. 2002) Developed with input from experts in physics and video generation.
3.1 DATASET CREATION PIPELINE
As shown in Figure the creation pipeline consists of three steps, with human-in mennannii-the-lloge the--ooppppppp reviews integrated at each step to ensure the benchmark's 's accuracy and diversity.
Step 1: Initial Physies Categories Definition. collaborative process between physies experts
and the authors to define the key (main) physics categories to be included the benchmark. Our benchmark categorizes three levels:
Fundamental Physics: Covers basic laws such as object motion, energy transfer, and optics. Composite Physics: Involves roaldworld phenomena that emerge from multiple interacting princi- ples, such as human motion.
4

---

Published as a conference paper at ICLR 2026
Table 1: Statistical Comparison with Existing Physics-Focused T2V Benchmarks. Compared to prior benchmarks, PhyWorl dBench offers a significantly larger and more diverse testbed for evaluating the physical commonsense capabilities of T2V models. It provides broader coverage of physics categories and a greater number of prompts, ensuring a comprehensive assessment of T2V performance.
Statistic
VideoPhy
PhyGenBench
Physics-IQ
T2VPhysBench
PhyWorldBene
Physics Categories
5
27
5
50
Prompts
688
160
396
84
1050
Anti-Physies: Scenarios intentionally designed to violate real-world physics.
The inclusion of anti- anti-physics cases is crucial, as discrepancies in model performance between physically accurate and unphysical scenarios reveal whether a model truly understands physics or merely replicates patterns from its training data, which predominantly follows real-world laws.
We identify broad categories such as kinematics, rigid body dynamics, fluid behavior, optics, and thermal dynamics as depicted in the main physics categorics in Figure [T] Each category is systematically divided into five subcategories ensure comprchensive coverage of physics principles from multiple perspectives.
Step 2: Prompt Creation Each subcategory is further expanded into 7 distinct scenarios, with each scenario incorporating three prompt variations: Event Prompt, Physics-enhanced Prompt, and Detailed Narrative Prompt, as illustrated in Figure 4 Specifically, we define these three types of prompts as follows:
Event Prompt: A concise and straightforward description of an event. These prompts are designed to assess the model's ability to generate based on minimal input, focusing high-level scene comprehension.
Physics-Enhanced Prompt: This type incorporates physics-related phenomena for Event Prompt to enrich the description naturally.
Detailed Prompt: Inspired by how current T2V models generate videos, these prompts are enriched with vivid details and contextual elements, aiming to create more immersive and visually rich outputs. This level of prompt allows us to assess whether providing richer context leads improved physical accuracy in video generation.
We first use statc-of-the-art large language models (LLMs), specifically GPT-4o (OpenAl Team 2024) and Gemini-1.5-Pro (Gemini Team, 2024) generate 20 Event Prompts under the predefined category and subcategory. Then we ask humans to select out of them based on diversity and physies representation. Then human experts meticulously review and validate the generated prompts, providing feedback to iteratively enhance their quality and relevance to the corresponding physics subcategory, ensuring balanced and representative assessment across various aspects of physical understanding. Furthermore, ethical review is conducted to ensure that all prompts fair, unbiased, and free from potentially harmful content. Based on the Event Prompt, human annotators create Physics-Enhancee Prompt by enriching it with physical consequences. Detailed Narrative Prompt generated by LLM based on Event Prompt using query in Tab. [2] This rigorous verification process results in a refined set of 1,050 high-quality prompts, systematically organized into main and subcategories.
Step 3: Standard Creation. We adopt Yes/No evaluation metric assess whether model can generate videos that accurately align with a given prompt. This metric is chosen for its simplicity and effectiveness and for allowing for an objective assessment while minimizing subjectivity in evaluation. As shown in Figure(3) we define two types of evaluation standards: Basic Standards and Key Standards. The Basic Standards specify the essential object(s) that should appear in the video and ensure that the main action or event is depicted. Meanwhile, the Key Standards describe the key physical phenomena that would occur if the event took place in the real world. The annotators are instructed to focus the natural physical consequences of the action and to be explicit and concise in their annotations. By following these guidelines, we ensure that the evaluation standards remain clear and objective. Following (Bansal et al. 2024 Meng et al. 2024). we use Semantic Adherence (SA) and Physical Commonsense (PC) to evaluate video performance. SA checks if both "objects" and "Action/Event" aligo with video, while PC assesses adherence to real-world physics and it checks
5

---

Published as a conference paper at ICLR 2026
Prompt: Two football players tackle each other.
Time
Standards
Yes/No Key standards: Yes/No
Objects: two football players Action/Event: tockle each other
contact point players should naturally, with realistically reacting and deforming upon impact.
direction changes both players
smooth and believable, respecting the laws of physics.
Prompt: apple falls from a tree branch.
Basic Standards
Yes/No
Key Yes/No
Objects: an apple, tree branch
V/ X
apple initially slowly from branch, speed visibly increasing as approaches the ground.
Action/Evant: falls
trajectory the apple straight path, showing horizontal deviation or external disturbances.
Figure 5: Illustration of Our Evaluation Metric and Human Annotations. We demonstrate our evaluation process for assessing the quality of generated videos based on evaluation criteria: Basic Standards and Key Standards. For Basic Standards, we verify whether the generated video contains the correct number of objects and accurately represents the intended action or event. For Key Standards, we define specific physical phenomena ground truth and measure if all of these phenomena generated video satisfies. Both lead to either score of "O" or "1" for a generated video. Red circles and yellow lines in the figure highlight instances where the generated videos fail to meet the Key Standards.
Time
(n) Aloace desccie tt's sioto, wlso Haz paseriee by gt Hi0 model 20 therz myy unioou Spats The cvont lods adherence peeneee to real world physks, with fluting and seltinok mowiin tools, incomistent megdoiing unmellttic www.eletiiiieeeeeeeeee intercons betweer tnmmer and tedge genenttes lluion setvtt but fals conwincingy simu lntt physical behavine (b) Plrsee dhi thi siiee, The video Rocyy serves demonstaation girden maintonance pronctional showcase for the efficincy the hodge trimmer
Time
generated, the motion insoastent siihtlyy from lypical neal realoworld buhesiio Nonetheless, central wheel being denorstrating retational dynamics. intertenal priecppie.
descche
Overali, switcing
obsorving
demonstrating
Figure 6: Context-aware prompt improves the assessment quality for GPT-4o. In the two examples, prompt (a) explicitly tells the MILMM that the video might contain potential issues, while prompt b does not. We found that the quality assessment is usually more accurate when the context is used. The text is irrelevant to quality is ignored for readability. This phenomenon applied to all our tested models including GPT-4o, GPT-ol, Gemini-2 2.0-flash, and Qwen-VL-2.0
when all "Key Standards" are satisfied. Annotated as SA, PC E 0, (1 indicating proper grounding), report the fraction of videos satisfying success when SA 1, PC = 1, and both jointly.
3.2 STATISTICS OF THE DATASET
We compare our benchmark with existing
focused T2V benchmarks (Bansal et al. 2024
Meng et al. 2024 Guo et al. 2025 Motameda et al.) 2025), with detailed statistical comparison presented in Tab. PhyMorldBench 1Benhh offers the most extensive coverage in terms the number of physics categories, curated prompts, human annotations, and evaluated T2V videos.

---

Published as a conference paper at ICLR 2026
Table 2: ROC-AUC for automatic evaluation methods and ablation on CAP, Rows now list the two metrics- SA (semantic adherence) and PC (physical commonsense) -wilee columns compare all baselines and ablations.
Metric
Qwen-VL-2200
2emini-2.0-Flash
GPT-4o
GPT-ol
CAP (w/o CoT)
CAP (w/o Context)
CAP
SA
72.4
74.6
72.1
75.4
76.3
77.3
80.3
PC
59.8
60.9
60.1
61.6
73.6
65,6
75.1
3.3
MLLM AS ZERO SHOT PHYSICS EVALUATOR FOR GENERATED VIDEOS
Human evaluation, while flexible, is costly at scale, Existing metrics often lack generalizability and struggle with the diversity in Phywor 1 dench (See Appx. D MILMs like GPT-o1 (OpenAl
Team 2024) and Gemini (Gemini Team 2024) show promise for video evaluation but fall short in reliably assessing physical realism (Bansal al. 2024 He et al. 2024 Meng al.) 2024). We find
that these models tend to rationalize unrealistic content, but their accuracy improves when informed that the video is Al-generated, as illustrated in Figure 6 We hypothesize that this improvement stems from MLLMs being primarily trained on real-world videos, leading them to inherently justify observed phenomena unless prompted otherwise. Inspired by this, we propose a simple yet effective method, Context-Aware Prompt (CAP), which explicitly informs the MILM that the video is generated rather than real. This approach enables the MLLM to provide a more accurate assessment of the video's physical realism. Critically, CAP does not assume videos incorrect by default but instead uses structured chain-of-thought prompt-first asking the model to describe the video and then to reason through any potential physics issues-and only labels video as incorrect if it clearly violates the category-speciff standards. We validate CAP neutrality by measuring false positive rates in physics correct videos and observe only a 0.001 absolute change in semantic adhesion (0.188 to 0. 187 and 0.014 change in physical commonsense 10.172 to 0816 showing that it does not significantly increase misclassification of correct videos.
Besides, inspired by Chain-of- Thought (Wei et al. 2022), we design two-step prompting strategy, where we first ask the MLLM to give a thorough description of the video including Object, Event, and Observations, then continue to prompt the MLLM for the final "Yes/No" answer for video quality. Please refer to Appx. K for prompts used in the two steps. We found this leads to consistent improvement in various MLLMs. In Tab. [5] we present the ROC AUC calculations comparing CAP with human results on PhyWorldBenee random guess would achieve a CAP is powered by GPT-ol. CAP significantly outperforms other methods, particularly in Physics Commonsense, where it achieves an absolute performance boost of 13.5 compared the standard GPT-ol. We find CAP also achieves great performance when equipping other proprietary or open- sourced MLLM. The bottom of Tab. [S] shows the ablation study of various components in CAP. "without CoT" means we directly ask the MLLM to give the answer of the question, and "without Context" means we do not inform MILM that the video could be low quality. Please refer Appx. D for performance of our method on other MILLM and analysis.
with videos generated by models in Figure 2 where of 50. We evenly sample 8 frames. In this evaluation,
4 VIDEO GENERATION RESULTS AND ANALYSIS
We evaluate both closed-soures source models including Tura-Turbo (OpenAl 2024). Gen-3 (Runway Team
2024), Kling 1.6 Alingg 2024), Pika 2.0 Pika Labs Team 2024). Luma (Luma Al Team (024), (024) and
open-source models including Hunyuan 720p (Kong al. 2024). Open-Sora 2.0 (Peng et al.) 2025)_
Video (HaCohen ct al. 2024). Step-video- T2V (Ma et al. 2025), Wanx-2.1 (WanTeam al., 2025). Please check Appx. for the implementation details. We generate 1050 videos for each model. We conduct human evaluation of generated videos using Amazon Mechanical Turk. We first introduce model performance and model- wise analysis and then break down the challenge for the video generation model to achieve real- world physics.
Open-Sora-Plan 1.3 (Lin et al., (2224). V1 (Hong et al. 2022).
4.1 MODEL PERFORMANCE
Tab. presents a comprehensive evaluation of twelve video generation models. We report the percentage of videos that satisfy both semantic adherence and physical commonsense criteria (Both).
7

---

Published as a conference paper at ICLR 2026
Table 3: We present the human evaluation results of 10 video generation models across 3 physies types. SA denotes semantic adherence, while PC represents physical commonsense. Both indicates
satisfaction of both criteria.
Fundamental Physics
Composite Physics
Anti Physics
Overall Performance
Model
SA
PC
Both
SA
PC
Both
SA
PC
Both
SA
PC
Both
Proprietory Models
Sora-Turbo (OpenAl] 2024
0.438
0,315
0.246
0.361
0.215
0.188
0.136
0.078
0.039
0.384
0.261
0.208
Gen-3 Runday Tehil PUR4
0.320
0.228
0.161
0.258
0.142
0.103
0.108
0.029
0.020
0.280
0,183
0.130
Kling [X0EAA
0.299
0.312
0.125
0.042
0.357
0.188
Pika 2.0
Pika Labs Teur 2024
0.587
0,375
0.312
0.479
0.274
0.239
0.228
0.043
0.011
0.521
0.314
0.262
Tuma
[.
0.464
0.259
0.220
0.328
0.199
0.160
0.056
0.000
0.000
0.385
0.218
0.184
Open-sourced
Models
Hunyuan 720p
Kong al. 2024
0.385
0.278
0.205
0.342
0.264
0.199
0.082
0.021
0,010
0.344
0.250
0.185
Open-Sora 2.0
(Png 2l 20235
0.434
0,268
0.205
0.279
0.202
0.154
0.056
0.021
0.010
0.348
0.223
0.170
Open-Sora-Plat
1,3 Lineer 2024
0.175
0.136
0.096
0.159
0.057
0.044
0.069
0.059
0.039
0.158
0.104
0.075
CogVideoX-1155
HHnyy er
0.419
0.266
0.308
0.207
0.154
0.062
0.000
0.351
Siep-video-72 Ma el 2023
0.353
0,245
0.189
0.333
0.251
0.191
0.085
0.021
0.020
0.320
0.224
0.173
Wanx-2.1 Wan ISa
2012
0.389
0.271
0.220
0.323
0.235
0.187
0.082
0.017
0.011
0.339
0.235
0.189
LTX Video [1 onen ct all 2024
0.221
0.102
0.080
0.177
0.071
0.042
0.076
0.011
0.000
0.194
0.085
0.062
The Overall Performance column provides an aggregate score for each model, averaged across 10 physics categories. Pika 2.0 achieves the best overall performance across all models. In open-sourced models, Hunyuan 720p achieves the best performance on PC, Wanx-2.1 achieves the best performance on Both, and Cog VideoX 1.5 achieves the best SA performance. It is worth noting that they even achieve a better performance compared with some of the proprictary. Please check Appx. P for Leaderhoard.
4.2 MODEL SPECIFIC ANALYSTS
Our evaluation reveals a trade-off between realism, stylization, and consistency in tcxt-to-video models. Wanx, Step- Ving-VidooT2V Gen-3, Hunyuan, Kling, and Sora produce the most realistic videos, while Pika and Luma add stylized lighting and colors. Open-Sora and LTX-Video struggle with realism, often generating low-fidelity distorted outputs. Object stability varies Cog Video Open-Sora, and UTX-Video frequently produce warped shapes, while Gen-3, Hunyuan, and Kling maintain more stable forms. Motion realism also differs: Kling and Gen-3 generate smooth motion, Open-Sora-Plan and Luma introduce slow-motion effects, and CogVideoX 1.5 and Open-Sora exhibit jittery, erratic movement. For a detailed analysis, see Appx.
4.3 EFFECT OF PHYSICS AND PROMPT ENHANCEMENT
Understanding real-world physics is essential for text-to-video models. Event-based prompts describe actions, while physics-enhances prompts include natural consequences. A model's ability to generate realistic videos based on these prompts reflects its real-worid understanding. As proprictary models conceal their generation processes, we cannot verify prompt refinement. Instead, we analyze prompt types using open-source models, where prompt integrity can be manually ensured. In Tab. 4 we compare model performance across different prompt types to gauge the effects of physics-aware prompting and refinement. While refinement improves clarity, object detail, and aesthetics, it does not notably enhance physics accuracy. The persistent inconsistencies suggest that these refinements primarily address surface-level quality, leaving deeper physics limitations unresolved. On the other hand, physics-cnhanced prompts often yield better results, reflecting the models' strong prompt- following ability- even though they still struggle with true physical comprehension. Consequently, we propose straightforward receipt: explicitly integrate physical phenomena into the prompt to nudge the model toward more realistic outcomes.
4.4 ANTI-PHYSICS ANALYSIS
As shown in Tab. 3 all models show performance decline from Fundamental Composite to Anti-Physics categories, reflecting the challenge of generating complex physical phenomena. While they capture basic physies, simulating intricate interactions and "unphysical" scenes remains difficult, highlighting a gap in physics adherence. Our analysis found that failures often stem from models interpreting prompts in more "reasonable" way rather than attempting and failing. For example, when prompted with "A glass sits on the table, and the wine in the glass has reversed gravity," the model generates a still image, adhering to real-worke physics instead of the prompt. This suggests that models prioritize mimicking training data over true physics understanding.
8

---

Published as a conference paper at ICLR 2026
4.5 FAILURE TO HANDLE ERRATIC VISUAL CHANGES
Certain physical events inherently result in erratic visual changes, such as collisions, breaking, or sudden state transitions. However, current video generation models struggle to accurately depict these scenarios. In the third example of Figure 10 a glass cup falls from table, an event that should cause it to shatter upon impact. Instead, the model rationalizes the action, generating a video where the glass remains intact. Rather than simulating realistic disruptions in structure, motion, or object state, the models tend to default to continuous, smoothed animations, avoiding abrupt transformations that are critical for representing real-world physics. We created 20 paired prompts for each test, applied them to 12 video generation models (240 videos total), and only evaluate the visual change correctness through human evaluation. We also highlight Pika, the strongest model overall. Tab. [6 shows models struggle with abrupt visual changes only 22.1% success overall vs. 42.1% for softened prompts; Pika 30% vs. 75%. When prompted with erratic visual changes, the model tends to simplify the task and generate the easier version.
4.6 BREAKDOWN IN PHYSICS AT HIGHER COMPLEXITY
Models' performance deteriorates under complex interactions involving multiple forces constraints. As demonstrated in Appx. R prompts involving multiple objects introduce significantly higher requirements for semantic adherence. For instance, when with the prompt "A feather and stone are dropped in the air, where both fall, and the stone drops much faster", ", all models fail to present the phenomenon correctly. The current models lack robust understanding of how multiple physical forces interact in complex environments. We similarly built 20 prompt pairs focused on interactions of multiple forces or collisions, applied them the 12 models, and highlighting Pika as the best model as in Tab. 17 Physics further breaks down at higher complexity - 12% success overall vs. 18% for simpler cases; Pika 15% vs. 40%. This suggest that the current video models do not really understand the physics thoroughly and are more likely to mimic the pattern in the training set.
Table 4: foylwingfollowing percentage on different types of prompt. Explicitly adding physics usually increases the physics-followin ability of video generation model, while prompt refinement process does not necessary leads improvement.
Model
Event Prompt
Physics-enhanced Prompt
Detailed Prompt
CogVideuX-1.5 Hong al.
2022
0.123
0.177
0.168
Hunyuan 720p (Kong et al.
2024
0.159
0.198
0.155
LTX-Video (Ha
al.
2024
0.056
0.065
0.066
Open-Sora-Plan 1.3
al. 2024
0.062
0.067
0.063
Open-Sora 2.0 (Peng
2025
0.167
0.177
0.173
Wanx-2.1 Wan a
2025
0.175
0.202
0.190
Step- Video-idd-- (Ma al. 2025
0.158
0.182
0.179
4.7 CINEMATIC EFFECTS VS. PHYSICAL PLAUSIBILITY
A major challenge in video generation is the tendency for models prioritize cinematic aesthetics over strict physical realism. Well-performed models like Sora, Gen-3, and Pika often introduce exaggerated motion dynamics, such as objects moving with heightened fluidity unnatural acceleration, making scenes appear choreographed rather than physically grounded. For example, an object that should fall naturally under gravity might instead descend too smoothly or float unrealistically. This issue is particularly evident when momentum conservation and force dynamics are distorted for dramatic effect. The first example in Figure I an apple floating in the air. then it suddenly dropped as the camera moved; such stylized rendering, while beneficial for artistic storytelling, poses significant challenges for applications requiring strict physical fidelity.
5 CONTRIBUTIONS AND FUTURE WORK
We propose PhyWorl BBench, large-scale, thorough, and multi-dimensiin benchmark for evaluating text-to-vider generation models. It provides insights into which physical phenomena hardest to simulate and what capabilities current models lack. Additionally, we propose an automated evaluator to measure physical accuracy and straightforward receipt for designing prompt. Future work will involve continuously evaluating emerging video generation models, updating our benchmark to reflect advancements in the field, and refining our leaderboard and analysis to track progress in physical accuracy and model capabilities. LLM usage is described in Appx. A

---

Published as a conference paper at ICLR 2026
Ethics statement The authors confirm that we have carefully reviewed and adhered to the ICLR Code of Ethics in this work.
Reproducibility statement We open-sourced our codebase to ensure reproducibility. We first released all prompts, evaluation standards, and we released the full evaluation code of our benchmark. We also released all the generated videos and their corresponding prompts.
REFERENCES
Niket Agarwal, Arslan Ali. Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chat- topadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical al. arXiv preprint arXiv:2551.0333 2025.
Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kei Chang, and Aditya Grover. Videophy: Evaluating physical commonsense for video generation. arXiv preprint arXiv:2406.0335 2024.
Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Goldenberg, Aditya Grover, and Kai-Wei Chang. Videophy-2: A challenging action-centric physical commonsense evaluation video generation. arXiv preprint arXiv:2503.06800 2025.
Gemini Team. Gemini: family of highly capable multimodal models, 2024.
Herbert Goldstein, Charles Poole, John Safko, and Stephen R Addison. Classical mechanics, 2002.
Xuyang Guo, Jiayan Huo, Zhenmei Shi, Zhao Song, Jiahao Zhang, and Jiale Zhao. T2vphysbench: A first-principles benchmark for physical consistency in text-to-video generation. arXiv preprint arXiv:2500.000337, 2025.
Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem. Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, Poriya Panet, Sapir Weissbuch, Victor Kulikov, Yaki Bitterman, Zeev Melumian, and Ofir Bibi. Ltx-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.001000 2024.
HailuoAI. Hailuo.
v deo/ 2024.
David Halliday, Robert Resnick, and Jearl Walker. Fundamentals of physics. John Wiley Sons, 2013.
Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, et al. Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation. arXiv preprint arXiv:2406. 15252, 2024.
Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205. 15868, 2022.
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, al. Vbench: Comprehensive benchmark suite for video generative models. 2024.
KlingAI. KLING. https://wwwt
k1 www.::wwingg..cc.m com/ 2024.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, Kathrina Wu, Qin Lin, Aladdin Wang, Andong Wang, Changlin Li, Duojun Huang, Fang Yang, Hao Tan, Hongmei Wang, Jacob Song, Jiawang Bai, Jianbing Wu, Jinbao Xue, Joey Wang. Junkun Yuan, Kai Wang. Mengyang Liu, Pengyu Li, Shuai Li, Weiyan Wang, Wenqing Yu, Xinchi Deng, Yang Li. Yanxin Long, Yi Chen, Yutao Cui, Yuanbo Peng. Zhentao Yu, Zhiyu He, Zhiyong Xu, Zixiang Zhou, Zunnan Xu, Yangyu Tao, Qinglin Lu, Songtao Liu, Dax Zhou, Hongfa Wang, Yong Yang, Di Wang, Yuhong Liu, Jie Jiang, and Caesar Zhong. Hunyuanvideo: A systematic framework for large video generative models, 2024. URL https://arxxt www.:rxwwwvvv.aaaooo//
03603
10

---

Published as a conference paper at ICLR 2026
Mingxiang Liao, Qixiang Ye, Wangmeng Zuo, Fang Wan, Tianyu Wang, Yuzhong Zhao, Jingdong Wang, Xinyu Zhang, et al. Evaluation of text-to-video generation models: A dynamics perspective. Advances in Neural Information Processing Systems, 37:109790-109816. 2024.
Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, al. Open-sora plan: Open-source large video generation model. arXiv preprint arXiv:2412.0013/ 2024.
Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang. Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. Evalcrafter: Benchmarking and evaluating large video generation models. 2024.
Luma Al Team. Luma AI. https: / luma abs. ai 2024. Generative Al platform specializing in 3D content and photorealistic modeling.
Guoqing Ma, Haoyang Huang. Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, Yu Zhou, Deshan Sun, Deyu Zhou, Jian Zhou, Kaijun Tan, Kang An, Mei Chen, Wei Ji, Qiling Wu, Wen Sun, Xin Han, Yanan Wei, Zheng Ge, Aojie Li, Bin Wang. Bizhu Huang, Bo Wang, Brian Li, Changxing Miao, Chen Xu, Chenfei Wu, Chenguang Yu, Dapeng Shi, Dingyuan Hu, Enle Liu, Gang Yu, Ge Yang. Guanzhe Huang, Gulin Yan. Haiyang Feng, Hao Nie, Haonan Jia, Hanpeng Hu, Hanqi Chen, Haolong Yan, Heng Wang. Hongcheng Guo, Huilin Xiong. Huixin Xiong. Jiahao Gong, Jianchang Wu, Jiaoren Wu, Jie Wu, Jie Yang. Jiashuai Liu, Jiashuo Li. Jingyang Zhang, Junjing Guo. Junzhe Lin, Kaixiang Li, Lei Liu, Lei Xia. Liang Zhao, Liguo Tan, Liwen Huang, Liying Shi, Ming Li, Mingliang Li, Muhua Cheng, Na Wang, Qiaohui Chen, Qinglin He, Qiuyan Liang. Quan Sun, Ran Sun, Rui Wang. Shaoliang Pang. Shiliang Yang. Sitong Liu, Siqi Liu, Shuli Gao, Tiancheng Cao, Tianyu Wang. Weipeng Ming, Wenqing He, Xu Zhao, Xuelin Zhang, Xianfang Zeng, Xiaojia Liu, Xuan Yang, Yaqi Dai, Yanbo Yu, Yang Li, Yineng Deng. Yingming Wang. Yilei Wang, Yuanwei Lu, Yu Chen, Yu Luo, Yuchu Luo, Yuhe Yin, Yuheng Feng. Yuxiang Yang. Zecheng Tang, Zekai Zhang, Zidong Yang. Binxing Jiao, Jiansheng Chen, Jing Li, Shuchang Zhou, Xiangyu Zhang. Xinhao Zhang, Yibo Zhu, Heung- Yeung Shum, and Daxin Jiang. Step-video-12v technical report: The practice, challenges, and future of video foundation model, 2025. URI.
10248
Fanqing Meng, Jiaqi Liao, Xinyu Tan, Wenqi Shao, Quanfeng Lu, Kaipeng Zhang, Yu Cheng, Dianqi Li, Yu Qiao, and Ping Luo. Towards world simulator: Crafting physical commonsense-baee benchmark for video generation. arXiv preprint arXiv:2410.05363, 2024.
Saman Motameda, Laura Culp, Kevin Swersky, Priyank Jain, and Robert Geirhos. Physics-iq: Evaluating physical understanding in generative video models. arXiv preprint 2iXV:255011000366
2025.
OpenAl. Sora: OpenAl's Multimodal http:://qeeni..cm//neee//ooorrr 2024. Accessed: 2024-11-24.
OpenAl Team. httpp://peeaa...cmm/ndeeeeelllla
2024.
Xiangyu Peng. Zangwei Zheng, Chenhui Shen, Tom Young. Xinying Guo, Binluo Wang, Hang Xu, Hongxin Liu, Mingyan Jiang, Wenjun Li, Yuhui Wang, Anbang Ye, Gang Ren, Qianran Ma, Wanying Liang, Xiang Lian, Xiwen Wu, Yuting Zhong, Zhuangyan Li, Chaoyu Gong, Guojun Lei, Leijun Cheng, Limin Zhang, Minghao Li, Ruijie Zhang, Silan Hu. Shijie Huang, Xiaokang Wang, Yuanheng Zhao, Yuqi Wang. Ziang Wei, and Yang You. Open-sora 2.0: Training a commercial-level 2025.
video generation model in $200k. arXiv preprint arXiv:2503.0960
Pika Labs Team. Pika Labs. creating video and visual content.
com 2024. Generative AI platform for
Runway Team. Runway, 2024. Platform for Al-powered video editing and generative media creation.
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
11

---

Published as a conference paper at ICLR 2026
Kaiyue Sun, Kaiyi Huang, Xian Liu, Yue Wu, Zihan Xu, Zhenguo Li, and Xihui Liu. T2v- compbench: A comprehensive benchmark for compositional text-to-video generation. arXiv preprint arXiv:2407.14505, 2024.
Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:/8/2.01711 2018.
Jing Wang. Ao Ma, Ke Cao, Jun Zheng, Zhanjie Zhang, Jiasong Feng, Shanyuan Liu, Yuhang Ma, Bo Cheng, Dawei Leng, al. Wisa: World simulator assistant for physics-aware text-to-Video generation. arXiv preprint arXiv:2503. 2025.
Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video generation with cascaded latent diffusion models. arXiv preprint
2023.
WanTeam, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen- Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang. Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang. Siyang Sun, Tao Fang. Tianxing Wang, Tianyi Gui, Tingyu Weng, Tong Shen, Wei Lin, Wei Wang, Wei Wang, Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang. Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503 20314, 2025.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
Yu Yuan, Xijun Wang, Tharindu Wickremasinghe, Zeeshan Nadir, Bole Ma, and Stanley H Chan. Newtongen: Physics-consistent and controllable text-to-video generation via neural newtonian dynamics. arXiv preprint arXiv:2509.21 2025.
Chenyu Zhang, Daniil Cherniavskii, Andrii Zadaianchuk, Antonios Tragoudaras, Antonios Vozikis, Thijmen Nijdam, Derck WE Prinzhorn, Mark Bodracska, Nicu Sebe, and Efstratios Gavves. Morpheus: Benchmarking physical reasoning of video generative models with real physical experiments. arXiv preprint arXiv:2504.02918 2025a.
Xiangdong Zhang, Jiaqi Liao, Shaofeng Zhang, Fanqing Meng, Xiangpeng Wan, Junchi Yan, and Yu Cheng. Videorepa: Learning physics for video generation through relational alignment with foundation models. arXiv preprint arXiv:2505.23666. 2025b.
A LLM USAGE STATEMENT
The large language model is used during the manual script proofreading to detect grammatical errors and rephrase some overly long sentences. Mcanwhile, as mentioned in the paper, our designed evaluator is based on LLM, the prompt draft creation process also involved L.M.
B BENCHMARK DETAILS
The benchmark revolves around main physics categories, each divided into subcategories that cover specific physics phenomena, such object occlusion, center of mass, and circular motion. Each subcategory includes 7 distinct scenarios, with 3 variations of prompts: general prompt, a ehysice-e prompt, and a detailed narrative prompt. These variations are designed to provide differing levels of complexity and context for the text-to-video models. The dataset was curated through three-stage process: first, defining key categories based fundamental physics literature;
12

---

Published as a conference paper at ICLR 2026
second, refining prompts through iterative reviews with large language models and human experts to ensure clarity and alignment with physics principles; and third, conducting detailed curation to create final set of 1,050 high-quality prompts for diverse set of physics phenomena. The evaluation utilizes a yes/no framework to assess model adherence to physical laws, categorized into basic standards and key standards, enabling comprehensive performance evaluation. This benchmark establishes foundation for evaluating models' ability to generate videos that align with both physical principles and real-world scenarios.
Following Bansal et al. 2024)) and Meng et al. (2024), we use Semantic Adherence (SA) and Physical Commonsense (PC) as metric to evaluate the video performance. SA measures if caption aligns with video frames, while PC assesses if actions obey real-world physics. Annotated as SA, PC E (0, 1}, where indicates proper grounding. We report the fraction of videos satisfying SA 1, PC = 1, and both jointly. We create SA and PC based on thorough review of all prompts and extensive feedback from human annotators, we establish a set of evaluation criteria to systematically assess the generated videos. The evaluation is structured around two key standards, each encompassing specific rules:
Semantic Adherence:
These criteria evaluate fundamental aspects of video generation to ensure prompt fidelity. The assessment includes:
1. Whether the video contains the correct number of objects specified in the prompt. (Yes/No)
2. Whether the depicted event or action accurately represented. (Yes/No)
Physical Commonsense: This aspect evaluates the model's capability to capture and represent underlying physical phenomena within the video. The assessment measures the number of distinct physical phenomena accurately showcased in the generated video.
By integrating these two standards, which consist of three core evaluation rules, we define a compre- hensive evaluation metric that ensures a balanced and thorough assessment of T2V model performance across multiple dimensions.
C ANNOTATION DETAILS
Prompt Annotation After designing the physics categories, we further prepared prompts within each category. To achieve this, gathered undergraduate and graduate students majoring in Computer Science for an annotation session. They were instructed to provide high-quality annotations by creating both an Event Prompt and corresponding Physics-enhanced Prompt that aligned with the primary physics category and sub-category. The Detailed Prompt was generated using GPT-4o with the MLLM prompt (as described in Tab. [2] and subsequently verified by human annotators. Finally, each prompt underwent a final verification by the authors.
Standard Annotation Undergraduate and graduate students in Computer Science were also tasked with annotating the objects and events described in the prompts. These annotations were used for SA (Semantic Adherence) evaluation. Additionally, they identified key physies phenomena that should be visually observable in the generated videos, which were used for PC (Physics Commonsense) evaluation. All annotations were reviewed and verified by the authors.
Video Evaluation For large-scale video evaluation, we utilized Amazon Mechanical Turk. The interface presented 7 Annotators were asked assess videos based on three criteria: the existence of objects, the presence of the described event, and the visibility of key physics phenomena, selecting either "Yes" or "No" for each. Each video was evaluated by three independent annotators. with the final decision determined by majority vote. All annotators were compensated at rate of S18 USD per hour.
D DETAILED ANALYSIS ON EVALUATOR
While some existing physics metrics have been proposed, none fit PhyWorldBenche needs due to domain structural mismatches (Guo al. 2025: Zhang al., 2025a). For example, PhyGenEval (Meng et al. 2024) assumes a fixed event order, which many of 1BenchhhdBBehhh scenarios do not have, and VideoCon-Physie (Bansal et al. 2024) is trained specifically its
13

---

Published as a conference paper at ICLR 2026
Task Instructions
chrietyy tollowing provided standards rather than roting
subjective fostor liee aesthetios.
Check axatetonns andeoati"" the video swiett "Yen" "Non "Yee" when number the
"a sksteboaed, a sicworlk".
Delemnie whether tetelebcard down antemale" visually depicteed video "Yea" or the alanddrrte wolkuation.
Imagees
aignilicann probiienas,
Video
Objectjij Event: akolaboard rolla down a flai adoowalk
Select an option
No
Select option
No
Phanomenon skateboard moves alrery spared visitle secodlaation decalenation.
Select an option
Yeu
Figure 7: Amazon Mechanical Turk interface.
14

---

about any Published as a conference paper at ICLR 2026
Table Precision and Recall for automatic evaluation methods and ablation on CAP, together with ROC-AUC from the submission.
Method
Precision SA
Recall of
PC
Precision PC
Recall PC
WL-2--
72.4
66.9
72.5
59.8
59.7
62.5
Gemini-2.0-Flash
74.6
68.0
73.1
60.9
61.1
61.8
GPT-4o
72.1
69.3
71.0
60.1
59.4
63.1
GPT-ol
75.4
71.1
73.2
61.6
63.1
62.4
CAP (without CoT)
76.3
71.8
79.3
73.6
70.3
74.6
CAP (without Context)
77.3
72.1
78.8
65.6
70.1
71.6
CAP
80.3
75.8
83.1
75.1
74.8
79.3
VideoPhy benchmark, yielding sub-optimal performance PhyWor 1BBench- so exclude both to avoid unfair comparisons. Besides, VideoPhy-2 (Bansal et al., (0255) focused on action and dynamic instead of general physics, and its evaluator is also designed and finetuned the associated benchmark annotation, therefore for the same reason, we also did not include this paper.
In Tab. 6 therefore evaluate CAP across multiple MLLMs, including proprietary GPT-4o, GPT-ol, Gemini-Flash-2200, and open-source Qwen-V1-2.0. Crucially, CAP uses a two-step chain-of-thought prompt-first first asking the model describe the video, then physics issues-so it does not assume videos are incorrect by default and yields only minimal false-positive increases (0.001 Semantic Adherence. 0.014 in Physical Commonsense), confirming its neutrality. Moreover, because it operates solely on the final video and its physics- based stan- dards, CAP agnostic to the underlying generative pipeline- whether text-to-video, image-to- video, video-to-video, or hybrid.
do observe two common failure modes: 1) Aesthetic bias, where visually compelling videos receive high scores despite only moderate physical accuracy (visual appeal and physical plausibility should idcally be assessed independently); and 2) Overconfidence, with the MLLM giving firm judgments in ambiguous edge-case scenarios where human annotators often express uncertainty disagreement.
To further validate design, we also measure Precision and Recall for both Semantic Adherence (SA) and Physical Commonsense (PC), which is shown in Table
Table 6: ROC-AUC Improvement of various MLLM for with our method automatic evaluation methods. Our method consistently achieves great performance on both open-sourced and proprictary models.
Method
SA
PC
Qwen-VL-20
VL-2.0
72.4-478.6
59.8-74.1
Gemini-2.0-Flash
74.6-79.9
60.9-74.0
GPT-4o
72.1-78.5
60.1-73.9
GPT-ol
75.4-+80.3
61.6-75.1
Moreover, even without any fine-tuning, our CAP evaluator achieves 80.3 Semantic Adher- ence and 75.1 Physical Commonsense on PhyWorldBench, and when this is compared to VideoCon-Physics" 82.0 SA and 73.0 PC on the VideoPhy benchmark after fine-tuning it highlights CAP's strong out-of-domain performance. Additionally, we conducted fine-tuning on an 80/20 train/test split of our annotated videos and found that these numbers can be boosted substantially: Qwen- Owen-VL-220 improves from 77.5 - 82.3 SA and 74.5 79.1 PC, and GPT-4o from 78.5 84.7 SA and 73.9 - 80.7 PC.
E MODEL. APPLICATION DETAILS
Open-source generators (code + weights).
Hunyuan (Tencent Hunyuan Community License Agreement): based on the official 720p repo pettps://githie htt.h//w.ccnneeeeeeeeeetarraraaa
15

---

Published as a conference paper at ICLR 2026
Table 7: Automatic Evaluation results from CAP on proprictary and open-source video models.
Model
SA
PC
Both
Proprictary Models
Pika 2.0
0.354
0.250
0.205
Luma
0.275
0.177
0.135
Kling-1.6
0.343
0.237
0.177
Sora-Turbo
0.357
0.225
0.176
Gen-3
0.380
0.164
0.133
Open-sourced Models
Hunyuan 720p
0.323
0.230
0.183
Vep-Video-T2V
0.335
0.187
0.153
Open-Sora-Plan 1.3
0.143
0.110
0.065
Open-Sora 2.0
0.330
0.196
0.143
Cog VideoX-1.5-5B
0.303
0.187
0.129
Wanx-2.1
0.365
0.243
0.191
ViXeo 0.197
0.075
0.052
ViXe (RAIL-M License): Hugging Face Diffusers pipeline pipeline
https://mugiingface.. /t/.c///o//// /ttp/////////.aeeee/////////aa//
CogVideo v1.5 (Apache-2.0) License)
Open-Sora v2.0
(Apache-2.0
License):
https://gittmdb.commpppaatll
Open Sora
Wanx v2.1 (Apache-2.0
Step-Video-T22
(MIT
License):
https://github..com/tttppu
Step-Video- T2V
Open-Sora-Plan v1.3 (MIT License): https///giiiib..cmm/PPP//hhhfhep
Opan
/ommercial/ proppieerr generators.
Sora-Turbo, Pika 2.0, Gen-3, Kling 1.6: accessed via the vendors' public web interfaces under their standard Terms of Service (no OSS license).
Luma: accessed through the official API.
All third-party code or data are used under the above licenses/Toß but are not redistributed in our release. Our own benchmark assets are released under the MIT License.
F
CAP RESULTS
Here further include the automatic evaluation results of CAP the video generation models Tab.
G INSIGHTS INTO PERFORMANCE BETWEEN MODELS
Our analysis reveals that both model size and sampling budget significantly impact physics perfor- mance, Larger models consistently outperform smaller ones on foundational physics tasks, as shown in Table g For instance, the fact that Cosmos 14B outperforms Cosmos 7B across all SA/PC metrics confirming that scaling yields clear gains in semantic adherence and physical commonsense.
We also observe that generation time correlates with physics accuracy: models with longer sampling schedules (c.g., Hunyuan at 40 min/video or Wanx-2.1 at hr/video) achieve stronger performance, and simply increasing Open-Sora's sampling steps from 50 to 200 yields significant boost. These
16

---

Published as a conference paper at ICLR 2026
Table Influence of model scale on physies performance. SA denotes semantic adherence, PC denotes physical commonsense, and Both indicates satisfaction of both criteria. To isolate the effect of size, we compare variants of the same base architecture.
Fundumental Physics
Compusite Physics
Physics
Overall Performance
Model
SA
PC
Both
SA
PC
Both
SA
PC
Both
SA
PC
Both
0.374
0.182
0.069
Cosmos 7B Agarwar or 2025
0.354
0.243
0.201
0.339
0.221
0.185
0.088
0.054
0.020
0.323
0.218
0.178
CogVideo
0.154
0.225
Cog 28 Hong crai
0.382
0.247
0.175
0,246
0.199
0.148
0.073
0.021
0.010
0.310
0.210
0.150
Table 9: We present the evaluation results of 10 video generation models across 10 physics categories. SA denotes semantic adherence, while PC represents physical commonsense. SA, PC indicates satisfaction of both criteria.
MaM
Coapera
876
1.87
8.50
84th
6391
635
1431
1.10
1351
2114
6140
1100
1371
6126
035
(31)
8.15
0215
6:22
K11
MAR
KJ5
013
L1O
183
5-48
MIC
KUM
1233
036
080
1.50
3.75
615
130
8.31
EX0
K15
(X) 0300
findings suggest that, alongside scaling, dedicating more compute to finer- tner-grained sampling can materially improve adherence to physical laws.
H DETAILED PERFORMANCE OVER ALL PHYSICS CATEGORIES
In the main paper, we presented results across different levels of physics. Here, we provide a detailed breakdown of results for each specific physics category in Tab,
I LIMITATIONS
While PhyWor dBench spans broad set of physical scenarios with its 1,050 prompts across ten categories, may not fully capture very specialized or highly intricate interactions that occur in some domains advanced applications. Similarly, CAP evaluator generally provides balanced feedback, yet it can display a slight preference for vidcos with particularly polished visual presentation, such smooth lighting or dynamic camera movement. Future work will focus expanding the prompt collection to include additional niche phenomena and on enhancing the evaluation workflow more clearly distinguish visual style from physical correctness.
J CODE AND DATA AVAILABILITY
To ensure full reproducibility, we provide both the generated video assets and experiment code:
Generated videos and prompts: All videos used in our experiments, along with the corre- sponding prompt JSON files, are hosted on HuggingFace at pttpttt::://. h/pps:://hgh// ng fa ce co/ coodaataeess// wdttaatts/llwwoolbebrrttt whwworldbenchh. httpp//www..cccphhooiioeooeo.oee wwwywwoldbenneeoo videos are not part of SeeyWorldBende they were instead utilized for our paper ex- periments. Also, the HuggingFace does not contain the evaluation standards, those part of the GitHub listed below.
Note that these generated
Experimentation code and instructions: The complete implementation of the experiments code as well as detailed setup instructions are available at https:: //gpthub..//////////// com/ ashwi neeiinn3333333 333/3hy w
17

---

Published as a conference paper at ICLR 2026
K
QUERY USED
Here we list the three MLLM queries used in this work, corresponding to the first-Stage description prompt (Tab. TOp, the second-stage structured-answer prompt (Tab. and the detailed- promppt refinement query (Tab, [2]. Note that for proprietary MLLMs such as GPT-ol1, GPT-4o and Gemini- 2.0-Flash, the model may refuse evaluate small fraction of generated videos; the refusal rate is below 3
Table 10: First prompt for MLLM to generate a detailed description.
Suppose you expert in judging and evaluating the quality of Al-generated videos. These frames evenly sampled from a generated video from the heginning to the end. This generated video from a video model rather than captured from real world, so the video could be low quality, such as fuzzy, inconsistency, especially not following real world physics. Please tell me what is in this video, including what happened and any physics phenomena you observe. Please be main event. any physies phenomena you observe.
include objects in the video, the
Table 11: Second prompt for MLLM to generate the final answer
Suppose you are an expert in summarization and finding answers. Here is the text description from another large language model about Al generated video: "[previous compare the objects and quantities present the video with the specified object(s): "(object)" Answer "Yes" if the object(s) could be found in the vidco, otherwise answer "No". Also, plcase check if "f" [cvent)" visually depicted in the video, and answer "Yos" or "No". Lastly, please check if vidco satisfies the standards in list: <physies.phenomenon www.nohhnnnn....mmmmmm list "", and answer "Yes" or "No" for cach standard in the list. Return your evaluation in the following JSON format: "Objects": "Yes/No".
Based on this description.
"Event": "Yes/No",
"Standard 1": "Yes/No".
"Standard 2": "Yes/No",
"Yes/No"
Table 12: MLLM prompt used to get detailed video generation prompts.
Refine sentence: [prommpt)" to contain subject description, action, scene description. (Optional: camera language, light and shadow, atmosphere) and conceive some additional actions to make the sentence more dynamic. Make sure it is fluent sentence, not nonsense.
L QUALITATIVE ANALYSIS ON INDIVIDUAL MODEL.
We present here qualitative examples from various models, as shown Figure Figure 9 Figure 10 Figure [1] Figure 12 Figure 13 Figure 14, Figure 15] Figure 16 Figure 17 Figure 18 Figure 19
that demonstrate violations of fundamental physics laws, highlighting their limitations. Below, we summarize key observations for each model:
Cog L.:: Tends to produce distorted objects, such non-spherical balls that ap- pear dented. Movements are often excessively fast, blurred, erratic, or jittery, sometimes appearing unnaturally sped up.
Generates videos with cinematic quality, featuring rich lighting and a high level of realism, akin to footage captured with a high-quality camera. Lighting is particularly smooth and well-balanced.
Gen-3:
Hunyuan: Produces highly realistic videos, with quality resembling footage taken by high-end camera.
Kling: Generates realistic videos, often incorporating dynamic, moving shots that simulate footage captured with moving camera.
18

---

Published as a conference paper at ICLR 2026
Table 13: Rationalization rate among failure cases. Values represent the percentage of failed videos
exhibiting static-scene rationalization.
Model
Pika
Sora
Kling
Luma
Wanx
Gen-3
Hunyuan
Step
CogVideo
Open-S
Open-S-P
LTX
Rate
34%
24%
24%
22%
28%
14%
20%
18%
16%
4%
6%
4%
LTX- Produces grainy footage with distorted objects that are difficult to identify, Moving objects often shift shape and struggle to maintain form.
Luma: Tends to generate realistic footage but with overly vibrant or bright colorization.
Moving objects sometimes experience distortion or loss of form.
Open-Sora-Plann Generates relatively realistic videos but with low fidelity, often lack- ing detail and featuring still objects. Movement can appear overly smoothed or surreal, sometimes resembling dlw-mootoon. The output often has a VHS-like quality.
Open-Sora: Generates videos where objects frequently distort and warp, failing to hold their shape. The outputs often appear less realistic, resembling animation or 3D graphics.
Pika: Produces realistic yet highly stylized videos. Lighting and objects have an unrealistic softness, contributing to distinctive stylized aesthetic.
Sora: Generates highly realistic videos with cinematic or stylized lighting. Movement is often exaggerated, enhancing the dramatic quality the footage.
Wanx: Produces visually crisp footage, yet physical interactions are frequently unreliable; rigid bodies may pass through one another, gravity-driven motions appear "floaty," and impacts are often muted or entirely missing.
Step- Seep-Video-TTae T2V: Creates sharp, aesthetically pleasing clips, yet physical reasoning is routinely oversimplified; light fails to refract, flexible objects stay rigid, mass differences not affect motion, and phenomena such as duplication or buoyancy are either absent or behave contrary to real-world expectations.
M STATIC-SCENE RATIONALIZATION AS AN EMERGING FAILURE MODE
Beyond explicit physical violations, we observe that several text-to-video models exhibit a distinct failure mode in which the model generates a partially or fully static scene avoid producing incorrect physical motion. Instead of attempting the intended physical interaction, the model suppresses motion entirely while still producing visually coherent video. This behavior effectively "rationalizes" the violation by minimizing dynamics.
To systematically investigate this phenomenon, we analyzed 600 failure cases from our evaluation set, sampling 50 failed videos from each of the 12 models. Each video was annotated for whether it exhibits static-seene rationalization, defined as replacing the expected motion with a static or minimally animated scene.
models with higher overall generation quality tend to exhibit substantially higher rationalization rates. Lower-performing models generally fail due to more fundamental issues-such as incorrect object identities, distorted geometry, or inconsistent metion little opportunity to "rationalize." In contrast, stronger models, once capable of producing semantically aligned and visually coherent scenes, often avoid explicit physics violations by freezing or minimizing motion, thus prioritizing aesthetic quality physical correctness.
clear trend emerges:
This pattern highlights a key challenge for future text-to-vide systems. As models advance in visual fidelity, they may increasingly adopt conservative strategies that circumvent difficult physics rather than attempting them. Quantifying this emerging failure mode is essential for developing models that do not merely avoid errors but actively engage in physically grounded reasoning.
19

---

Published as a conference paper at ICLR 2026
(a) A ceiling fan spins under a bright light.
(b) A bird flaps its wings to take off by generating lift through strong downward wing beats.
A person reacts after inhaling pepper.
(d) A person sleps on a bicycle's pedal.
(e) A child slides down from the top of slide, speeding up as they descend.
Figure 8: Physics violations observed in videos generated by the Sora-Turhe model. Ceiling fan's blades phase into the ceiling during rotation. b. Bird's supporting branch phases into nearby leaves, disconnected from the bird's movement. Bottle enters the person's 's mouth without spatial adjustment as they react to pepper. Person pedals through a bicycle instead of interacting correctly with the pedals. Child's face deforms unnaturally while sliding down a slide.
20

---

Published as a conference paper at ICLR 2026
(a) A buoy is in the ocean.
(b) A child swing experiences air resistance.
(c) A cyclist rounds curve track.
(d) A rubber ball bounces off wall.
(c) A child jumps onto a trampoline.
Figure 9: Physics Violations Observed in Videos Generated by the Gen-3 Model. a. Buoy remains unnaturally stationary despite visible water movement. b. Child morphs into different positions and exhibits abnormal motion. c. Cyclist pedaling forward moves backward while rounding curve track. d. Rubber ball bounces off the air instead of the wall. e. Child displays abnormal motion, with knees bending unnaturally.
21

---

Published as a conference paper at ICLR 2026
(a) A glass sits on the table, and the wine in the glass has reversed gravity.
(b) small toy airplane and a sull-sized airplane are pushed along the ground by two kids.
(c) A glass cup falls from table.
(d) A feather and a stone are dropped in the
(e) A hammer strikes aluminum can.
Figure 10: Physics Violations Observed in Videos Generated by the Kling-1.6 Model. a. Glass with wine remains unaffected by reversed gravity, showing changes in the liquid's behavior. b. Small toy airplane and full-sized airplane are depicted as same size, violating realistic proportions. Glass cup glides down instead of falling naturally and slides abnormally after hitting the ground. d. Feather and stone fall at the same speed, inaccurately disregarding differences in air resistance. e. Aluminum remains intact after being struck by hammer, failing collapse as expected.
22

---

Published as a conference paper at ICLR 2026
(a) An apple falls from tree branch.
(b) A person lifts heavy box from the ground.
(c) A dog shakes its body to dry off.
(d) A plastic bottle is stepped on.
(e) piece of tin foil is crumpled.
Figure 11: Physies Violations Observed in Videos Generated by the Pika 2.0 Model. a. Apple floats in the air, then suddenly falls unnaturally as the camera moves. b. Box moves up without any contact from hands. c. Dog's face deforms unnaturally along with water motion as shakes to dry off. d. Plastic bottle resists fully crushing when stepped on. Tin foil folds automatically into person's 's hand without proper crumpling motion.
23

---

Published as a conference paper at ICLR 2026
(a) A soccer ball rolls toward the goal directly after being kicked.
(b) A diver stands on platform and jumps into pool.
(c) A marble released the top of a half-pipe.
(d) A lump of dough is stretched out.
(e) A book slides off table.
Figure 12: Physics Violations Observed in Videos Generated by the Luma Model. a. Soccer ball rolls away from the goal in a straight line, showing no natural curve or arc its path after being kicked. b. Diver exhibits abnormal motion and phases into the water upon jumping from platform. Marble floats unnaturally upward, deviating from expected motion. d. Hands phase into a lump of dough while stretching it out. Book exhibits abnormal phasing during its motion.
24

---

Published as a conference paper at ICLR 2026
(a) A glass cup table-phasing, and it passes through solid table instead of resting on it.
(b) Two balloons with like charge are placed close together.
(c) A pendulum swings between two peaks.
(d) steel ball drops onto soft clay block.
E) Scissors cut through paper.
Figure 13: Physies Violations Observed in Videos Generated by the Hunyuan 720p Model. Glass cup rests on the table instead of phasing through it, failing to follow the anti-physics prompt. Two balloons with like charges remain stationary instead of repelling each other, disobeying electromaanetic interaction physics. c. Pendulum swinging between two peaks exhibits irregular and unrealistic motion. d. Steel ball dropping onto soft clay block does not deform the clay, defying expected material interaction physies. Scissors cutting through paper fail to visibly separate the paper, disobeying shear and cutting physics.
25

---

Published as a conference paper at ICLR 2026
(a) A train travels upside down, suspended from the tracks above.
(b) flashlight moves across dark room.
(c) Two billiard balls collide.
(d) A buckpuck is loaded off center.
(c) A bowling ball rolls toward the pins.
Figure 14: Physies Violations Observed in Videos Generated by the Open Sora Model. Train traveling upside down on suspended tracks appears unti-physics scenario. Flashlight moving across dark room does illuminate its surroundings. c. Two billiard balls collide but awkwardly phase into each other instead of bouncing off. d. Backpack loaded off-center remains unnaturally straight, with no tipping or imbalance. e. Bowling ball rolling toward pins exhibits abnormal phasing and unnatural motion.
failing to depict the intended
26

---

Published as a conference paper at ICLR 2026
(a) A lighthouse beam rotates across the ocean.
(b) person walks in front of projector screen.
(c) A rolling tire moves down the street.
(d) A rock dropped from a cliff.
(c) A tightrope walker moves across wire.
Figure 15: Physics Violations Observed in Videos Generated by the Open-Sora-Plan 1.3 Model. a. Lighthouse beam fails to visibly rotate across the ocean, undermining its expected motion. Person walking in front of projector screen casts shadow on the screen. Rolling tire phases into another tire while moving down the street. d. Rock dropped from cliff glides unnaturally and exhibits abnormal motion and phasing. Tightrope walker disobeys gravity with abnormal movement while crossing the wire.
27

---

Published as a conference paper at ICLR 2026
(a) Raindrops return to the clouds after hitting the ground.
(b) toy car and real car roll down identical ramps.
(c) A car drives on an icy road.
(d) A basketball is dropped onto the ground.
(c) person stands on one leg during yoga.
Figure 16: Physics Violations Observed in Videos Generated by the Cog 5b Model. Raindrops do not return to the clouds after hitting the ground; instead, they exhibit normal rainfall, failing demonstrate anti-physics. b. Toy car and real car rolling down ramps are depicted as the same size, violating proportional realism. c. Car driving on an icy road shows no slippage, unrealistically maintaining normal traction. d. Basketball dropped onto the ground displays unnatural bouncing motion. e. Person standing on one leg during yoga exhibits unrealistic human anatomy.
28

---

Published as a conference paper at ICLR 2026
(a) A stationary bicycle starts pedaling itself without a rider.
(b) A chair is placed on an uneven floor.
(c) A stone is thrown across a river.
(d) A rock sits at the edge of a hill, and begins to move.
(c) Wind blows over sand dunnes.
Figure 17: Physics Violations Observed in Videos Generated by the LTX-Video Model. a. Stationary bicyele appears with rider, failing follow the anti-physics prompt of pedaling without one. b. Chair placed on an uneven floor remains stable, inaccurately disregarding the effect of the uneven surface. c. Stone thrown across river skips the parabolic motion and instead appears directly in the water. d. Rock sitting on the edge of hill does not fall, disobeying natural gravitational physics and the prompt. e. Wind blowing over sand dunes fails to pick up sand particles, violating expected particle behavior physics.
29

---

Published as a conference paper at ICLR 2026
(a) baskethall hits the ground after being dropped.
pamu
(a) An Olympic athlete throws a javelin.
(a) Ice cubes are placed in glass of hot water.
(a) stone is dropped into still pond.
(a) hammer strikes an aluminum can.
Figure 18: Physics Violations Observed in Videos Generated by the Wanx-2.1 Model. a. Bas- ketball dropped from shoulder height rebounds to only fraction of its release height, indicating excessive inelasticity. b. During the javelin throw, the athlete's 's fore-arm visually interpenetrates the javelin shaft, breaking rigid-body constraints. Multiple ice cubes clip through one another instead of exhibiting realistic rigid-body collisions and buoyancy. d. The stone ricochets repeatedly on calm pond surface and never sinks, contrary to expected gravity submersion. The aluminum can remains undeformed when struck by the hammer, ignoring momentum transfer and material plasticity.
30

---

Published as a conference paper at ICLR 2026
(a) beam of light hits the surface of a calm pond, reflecting partially while refracting into the water.
(a) tree branch bends wind, flexing under pressure returning upright when wind subsides.
(a) child and an adult jump on a trampoline.
(a) A single coin continuously splits into two, never stopping.
(a) A balloon has small hole, causing air to escape from the hole, deflating the balloon and causing to collapse.
Figure 19: Physics Violations Observed in Videos Generated by the Step- Video-TT2 T2V Model. An object strikes a calm pond, but there is no visible beam of light or refraction as enters the water. A tree branch remains rigid in the wind, never bending springing back. jump on a trampoline yet rebound to identical heights, ignoring mass-dependent dynamics. A single coin expected duplicate continuously, but no new coins appear. e. A balloon with hole should deflate downward: instead, it inexplicably rises, contradicting physics.
child and adult
31

---

Published as a conference paper at ICLR 2026
Table 14: Video Length by Model
CogVikoo
Gen-1
Humyuam
Klngg
Vilo
Lama
Upen-Sara- Pam
Open Sorn
Pika
Somi
Step- Vide--
Duration (5)
N VIDEO LENGTH FROM MODELS
O FURTHER ANALYSIS ON GENERATIVE MODELS STRUGGLING WITH CERTAIN PHYSICAL SCENARIOS
We identify three key factors behind current models' failures on physics-intensifn scenarios: (1) Training Data Distribution: large web-scale corpora such as Web Vid,id, Panda70M, or other private web-crawled datasets are dominated by everyday scenes-our analysis shows this yields strong performance on CI (Object Motion and Kinematics), C4 (Fluid and Particle Dynamics), C5 (Lighting and Shadows), C8 (Rigid Body Dynamies), and C9 (Human and Animal Motion) but underrepresents abstract or multi-object interactions; (2) Video Captioning Process: MLLM-basee based prompt extraction captures main objects and events but routinely omits fine-grained physical dynamics; and (3) Model Architecture Limitations: even with detailed prompts, current architectures emphasize short-term frame coherence without physics priors, leading to physically inconsistent outputs-by contrast, models like TORA explicitly model trajectories for better motion fidelity.
P LEADERBOARD
Tab. 15 presents the leaderboard performance on both human evaluation and our designed CAP. The performance shows highly consistency and only difference is that for proprietary models, Sora ranked number 2 while Kling ranked number 2 in CAP. Through further analysis, we found that Kling's 's more cinematic style can bias CAP toward higher scores despite only moderate physi- cal correctness, whereas our human annotators who were instructed to focus strictly on physics plausibility penalize these artifacts, causing Sora rank higher under human evaluation.
Table 15: Leaderboard of open-sourced models and proprietary models. We report leaderboard on both human evaluation and CAP.
Open models
Proprietary models
Human
CAP
Human
CAP
Wanx
Wanx
Pika 2.0
Pika 2.0
Hunyuan
Hunyuan
Sora-Turbo
Kling
Step-Vide Viveoiide
Step- Video-Ti
Kling
Sora-Turbo
Open-Sora
Open-Sora
Luma
Luma
Cog -15
Cog 115
Gen-3
Gen-3
Open-Sora-Plan
Open-Sora-Plan
LTX-video
LTX-video
Q VIDEO QUANTITATIVE ANALYSIS
Here present the quantitative analysis of the generated video.
32

---

Published as a conference paper at ICLR 2026
Table 16: Quantitative analysis on Erratic Vi- sual Changes.
Table 17: Quantitative analysis on scene com- plexity.
Group
Style
Pass / Total
Rate
Group
Complexity
Pass / Total
Rate
All models
Abrupt
53 / 240
22.1
All models
Complex
28 / 240
11.7%
Non-abrupt
101 240
42. 42.1%
Less-complex
44 /40
18.3%
Pika
Abrupt
6 20
30.0%
Pika
Complex
3/20
15.0%
Non-abrupt
15 20
75.0%
Less-complex
8 8/
40.0%
Prompt: A feather and a stone are dropped the air, where both fall, and the stone drop much Fasher.
(b) Promot: large bost ocean weves, with minimal rocking.
rising and falling dramatically larger boat steadily
Figure 20: Sora failed examples on multi-object. We found that generation models tend to fail when the prompt is relatively complex, such when containing multiple objects.
R INFLUENCE OF MULTIPLE OBJECTS
We noticed that generation models often fail when the prompt is more complex, such when containing multiple objects. We show examples of Sora failing when given prompts with multiple objects in Figure 20
DATASET STRUCTURE
The dataset is organized into three main categories of physics phenomena: Fundamental Physics, Composite Physics, and Anti-Physics. Each category divided into subcategories, which are further broken down into five sub-subcategories, summarized below:
FUNDAMENTAL PHYSICC
(a) Object Motion and Kinematics: Linear Motion, Parabolic Motion, Circular Motion, Rotational Motion, Combined Translational and Rotational Motion
(b) Interaction Dynamics:
Collisions, Friction and Sliding, Gravity and Free Fall, Electromag- netic Interactions, Tension and Compression
(c) Energy Conservation: Potential to Kinetic Energy, Elastic Energy, Energy Loss (Damping), Conservation of Mechanical Energy, Heat Transfer and Phase Changes.
(d) Fluid and Particle Dynamies: Water Motion, Smoke and Gas Dynamics, Particle Behavior, Buoyancy and Floating. Viscosity and Flow
(e) Lighting and Shadows:
Consistency of Illumination, Object Occlusion, Chromatic Aberra- tion and Dispersion, Reflection and Refraction, Shadow Softness and Penumbra
(f) Deformations and Elasticity: Material Flexibility, Elastic Rebound, Plastic Deformation, Brittle Fracture, Thermal Expansion and Contraction
COMPOSITE PHYSICS
(a) Scale and Proportions:
Consistency of Forces Across Scales, Environmental Effects on Different Scales, Proportional Energy Transfer, Structural Integrity Across Scales, Scaling of Mechanical Properties
(b) Rigid Body Dynamics:
Rotation and Torque, Balance and Stability, Center Mass, Impact and Deformation, Shear and Cutting
33

---

Published as a conference paper at ICLR 2026
Fundamental Physics
2 Ooject Motion and Kinemalics
Interaction Dynamics
Energy Consersation
chid experences croutrr mation mary go round, traveling perfect
louncing
Ae a pendatan relesaed raxed poation, snenge accelnating
continung oppoute
decelenst
Fluld and Particle Dynamics
Lighting and Shadows
ind Deformations and Elasticity
bost moves ocrois lake, lmodng V-anaped wake dipieses dong path.
culing spira light iihht creating
sightly ipringing erignal
Complex Physics
Scale and Proportions
Rigid Body Dynomics
Humon and Animal Motion omechanica
guil
their
Anti--hyssis
y Physi-hhysis
tell rolls uphil denl goniig energy i5 ROME
Figure 21: Dataset Overview. PhyMorldBench contains ten physics categories under three different physics levels.
(c) Human and Animal Motion:
Anatomical Feasibility, Balance and Posture, Locomotion Across Terrains, Reflexive and Involuntary Motion, Coordination and Fine Motor Skills
ANTI-PHYSICS
(a) Anti-Physics: Defying Gravity, Energy Creation or Perpetual Motion, Object Phasing and Surreal Interactions, Time Reversal and Alteration, Infinite Duplication and Division
This hierarchical structure ensures coverage of realistic and non-realistic physical scenarios for diverse amalysis. Figure 21 visualises how these ten categories map onto the three physics levels and highlights the numbers of prompts in each branch.
IMPACT STATEMENT
PhyWor dBench establishes new standard for physics-focused text-to-video generation, signifi- cantly elevating the field's expectations of realism and accuracy. By rigorously assessing models' capabilities across a comprehensive spectrum of physical caanomena-including motion, energy transfer, and complex interactions this benchmark not only highlights critical gaps in current systems but also guides researchers toward developing more robust and physics-awas aware solutions. Moreover, PhyWor dBench acknowledges the ethical implications of physics-driven video synthe- sis, emphasizing the need for transparency, fairness, and responsible deployment. By mitigating risks such misinformation, biased simulations, and unintended real-world consequences, it ensures that advancements in this field are aligned with scientific integrity and societal well-being. Ultimately, PhyWor dBench paves the way for advanced applications in education, scientific visualization, and industry, demanding both photorealism and fidelity to real-world physics while upholding ethical standards.
34

---

Published as a conference paper at ICLR 2026
T DATA RELEASE
We publicly released a comprehensive dataset that includes the core assets of Phywor 1 dBench, the 1,050 example json file, the evaluation standards, and the physics categories and subcategories. used to assess physical realism in texi-to-video models.
35


Reviews:
[{"review_id": "w9QOvqF58L", "reviewer": "Reviewer_oKee", "summary": "Summary:   \nThis paper addresses the critical limitation of existing text-to-video (T2V) generation models—their inability to adhere to physical laws despite producing photorealistic content—by proposingPhyWorldBench, a comprehensive benchmark for evaluating physical realism in T2V models.  \n\nContributions:  \n（1）Development of a comprehensive physics benchmark: PhyWorldBench fills the gap of lacking holistic tools to evaluate physical realism in text-to-video models. Its hierarchical structure (three levels, 10 main categories, 50 subcategories) covers diverse physical scenarios—from basic motion and energy conservation to anti-physics scenarios—and includes 1,050 well-curated prompts with variations, enabling systematic testing of models’ physical reasoning capabilities.  \n（2）Creation of a zero-shot automatic evaluator: The proposed Context-Aware Prompt method resolves the limitations of traditional evaluation. By guiding large language models to explicitly assess AI-generated videos and use chain-of-thought reasoning, it achieves high accuracy in evaluating physical realism, providing a scalable and objective alternative to costly human evaluation.  \n（3）Extensive evaluation of state-of-the-art models: The paper conducts large-scale tests on 12 leading text-to-video models, generating 12,600 videos. This evaluation identifies key challenges models face—such as struggling with complex interactions and prioritizing cinematic aesthetics over physics—and quantifies performance gaps across different physics categories, offering clear directions for future model improvement.", "questions": "Questions:   \n（1）Questions About CAP Evaluator’s Aesthetic Bias  \nYou note the Context-Aware Prompt evaluator prefers visually polished videos but provide no quantitative data on this bias. How often does this preference lead to misclassifying physically incorrect yet visually polished videos as physically plausible? Does a visually vivid video with physical violations score higher than a plain but physically correct one? Have you tested if revising CAP prompts to explicitly ignore aesthetics reduces this bias? Clarifying these points will confirm if CAP’s objectivity is compromised and if adjustments can fix it.  \n（2）The comparison with the methods of predecessors is not sufficient.  \nA comparison between the dataset and some previous related datasets, such as VideoREPA[1], WISA[2], NewtonGen[3], etc.?  \n（3）Niche Fundamental Physics Performance\nPhyWorldBench covers 10 main physics categories, but your experiments focus on common phenomena and lack breakdowns for niche subcategories like pressure-dependent phase changes or non-uniform heat transfer. Do you have data on model performance in these niche areas? If not, why were they excluded from detailed analysis? Understanding these gaps will help researchers target specific physical principles models struggle with. If the author's reply is strong and reasonable, I will consider increasing my score.  \n\n[1] VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Models  \n[2] Wisa: World simulator assistant for physics-aware text-to-video generation  \n[3] NEWTONGEN: PHYSICS-CONSISTENT AND CONTROL-LABLE TEXT-TO-VIDEO GENERATION VIA NEURAL NEWTONIAN DYNAMICS", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "Strengths:   \n（1）Originality  \nIt proposes a three-tier (Fundamental, Composite, Anti-Physics) hierarchical structure with 10 main physics categories, going beyond prior benchmarks by using Anti-Physics scenarios to test true physical understanding. The CAP evaluator creatively combines LLMs with domain constraints, separating aesthetics from physics via two-step reasoning to fix traditional evaluators’ inaccuracies. It also offers three prompt variants to test model adaptability, filling gaps of static-prompt benchmarks.  \n（2）Quality  \nIts benchmark is built via a rigorous three-stage process (literature/expert category definition, LLM-human prompt generation, expert validation) to ensure 1,050 prompts are diverse and accurate. CAP aligns with human evaluation (ROC-AUC: 80.3 for SA, 75.1 for PC) and outperforms baseline LLMs. Experiments cover 12 models (5 proprietary, 7 open-source) with 12,600 videos and systematic analyses to avoid cherry-picking.  \n（3）Clarity  \nIt follows a clear \"problem-solution-validation\" flow, with the introduction outlining existing benchmark limits and methodology linking components to specific issues. Technical details (CAP’s reasoning, \"Yes/No\" criteria) are explained in plain language without jargon. Consistent terminology and condensed key findings make it readable for physics and AI researchers.  \n（4）Significance  \nAs a standardized tool, its open benchmark and CAP provide universal metrics for tracking text-to-video physical realism. Prompt design insights (e.g., Physics-Enhanced Prompts boost PC) guide real-world uses like scientific visualization. It shifts evaluation focus to physical correctness, identifies model weaknesses, and reduces educational misinformation risks.", "weaknesses": "Weaknesses:   \n（1）CAP Evaluator’s Aesthetic Bias  \nThe Context-Aware Prompt (CAP) evaluator favors visually polished videos (e.g., smooth lighting, dynamic camera movement) over physically correct ones, which conflicts with the benchmark’s goal of prioritizing physical plausibility. However, the paper doesn’t quantify this bias or fix it. For example, a visually vivid but gravity-violating floating apple video might get a higher score than a plain yet physically correct one.  \n（2）Inadequate Niche Fundamental Physics Analysis  \nPhyWorldBench focuses on common physics (e.g., free fall) but ignores niche subcategories of fundamental physics, like pressure-dependent phase changes (water boiling at high altitude) or non-uniform heat transfer (a metal rod heating unevenly). It only reports broad category performance, hiding which specific principles models struggle with.  \n（3）No Long-Duration Video Validation  \nThe paper doesn’t specify video duration or test how performance scales with length. Physical inconsistencies (e.g., trajectory drift) worsen in longer videos, which are needed for real uses (e.g., education). Without this data, the benchmark’s real-world value is limited.  \n（4）Lack of Specialized Benchmark Comparisons  \nThe paper only compares to general physics benchmarks (e.g., VideoPhy) but not specialized ones like Morpheus (real-world experiment focus) or T2VPhysBench (first-principles physics). This hides PhyWorldBench’s unique value.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "F7GOi12r5T", "reviewer": "Reviewer_hYMe", "summary": "This paper introduces PhyWorldBench, a comprehensive, large-scale benchmark for evaluating physical realism in Text-to-Video (T2V) models. It features 1050 prompts systematically categorized across 10 physics domains and uniquely includes an \"Anti-Physics\" category to test genuine physical understanding versus mere pattern imitation from training data. The study employed extensive human evaluation (via Amazon Mechanical Turk) on 12 leading models and proposed a novel automated MLLM evaluation method called CAP (Context-Aware Prompting). Findings show that all current T2V models have very low physical success rates (Pika at 26.2% being the highest), and notably, they tend to revert to realistic physical phenomena when instructed to perform anti-physical actions.", "questions": "1. The CAP automated evaluator, while highly accurate in classification, is noted to possess an \"aesthetic bias\" that causes ranking discrepancies with human judgment; given the non-scalability of human evaluation, what specific plans do the authors have to mitigate or quantify this aesthetic bias in the MLLM, ensuring the CAP tool reliably focuses on physical correctness rather than visual style?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The benchmark offers the most comprehensive and systematic coverage of physical phenomena\n2. he proposed CAP automated MLLM evaluator significantly improves the accuracy of machine-based physical assessment by using a context-aware prompt, offering a cost-effective and reproducible alternative to expensive human studies.", "weaknesses": "1. The automated CAP evaluator suffers from an \"aesthetic bias,\" leading to ranking discrepancies with the human gold standard for high-quality, closed-source models. This limits its reliability as a trustworthy, scalable proxy for future research.\n\n2. Many of the investigated physics phenomena (e.g., fluid viscosity, electromagnetic forces) are challenging to assess accurately by eye in a video. This visual complexity makes human scoring error-prone and may also challenge the perceptual accuracy of the MLLM evaluator.\n\n3. The evaluation primarily focuses on whether the final physical state is correct, but it lacks a detailed metric to measure if the physical process is smooth, coherent, and temporally consistent. This overlooks a critical challenge in dynamic T2V modeling.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tgrWH9jead", "reviewer": "Reviewer_FCTE", "summary": "This paper introduces PhyWorldBench, a comprehensive benchmark for evaluating physical realism in text-to-video generation models. The benchmark contains 1,050 prompts spanning 10 main physics categories (fundamental, composite, and anti-physics), with each category divided into 5 subcategories and 7 scenarios. The authors evaluate 12 state-of-the-art models (5 proprietary, 7 open-source) and propose CAP (Context-Aware Prompt), a method using MLLMs for automated evaluation. Results show that even the best models (Pika 2.0 achieving 26.2% success rate) struggle significantly with physical realism, particularly with complex interactions and anti-physics scenarios.", "questions": "- The paper mentions models sometimes \"rationalize\" physics violations by generating static scenes. Could you quantify how often this happens across models? This seems like an important failure mode worth analyzing systematically.\n- Have you considered releasing a \"difficulty rating\" for each prompt based on model performance? This could help researchers identify particularly challenging scenarios to focus on.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Important and timely problem. The paper tackles a crucial gap in bridging video generation and physical reasoning. The motivation is clear and aligns well with current research trends.\n- Comprehensive experimental design. The evaluation is thorough - 12 models tested, 12,600 videos generated, large-scale human annotation via MTurk. They evaluate across a diverse set of tasks, object falling, rolling, fluid dynamics, etc. \n- Practical automated evaluator. The CAP method achieves 80.3% ROC-AUC for semantic adherence and 75.1% for physical commonsense (Table 2), which is pretty solid for zero-shot evaluation. \n- Clear organization and presentation. The hierarchical organization is logical and makes the benchmark easy to understand and extend. The tables and plots are easy to follow.", "weaknesses": "- No end-to-end pipeline for new models. The paper doesn’t provide a unified automatic evaluation framework for new video generation models. While CAP is proposed as an automatic evaluator, there's no clear standalone pipeline or released code/API for researchers to evaluate their own models. The paper mentions \"we will open-source our codebase\" (reproducibility statement) but it's unclear if this includes an easy-to-use evaluation script. For a benchmark paper, providing a simple evaluate interface would greatly increase adoption. The current workflow seems to require manual video generation then evaluation, which is cumbersome. For practical use, an automated submission or leaderboard system would make the benchmark much more usable.\n- Metric definitions design lack justification. A few metrics (like the physics consistency score) are described in high-level terms but not fully formalized. It’s hard to know how reproducible they are from the text alone, e.g., whether they use learned physical estimators or ground-truth simulations. The Yes/No binary evaluation (Section 3.1) might be too coarse-grained - a partial physics violation might deserve a score between 0 and 1. The paper acknowledges models often \"rationalize\" actions rather than fail outright (Section 4.4), suggesting a more nuanced scoring could capture important phenomena.\n- Limited real-world coverage. Most of the benchmark focuses on synthetic data (e.g., MuJoCo or Unity scenes). It would be great to see more real videos or robotic interactions to test generalization.\n- The scope of the benchmark is somehow overlapped with previous works, e.g., the inclusion of an \"anti-physics\" category is similar as “counterfactual prompts” in [1]. \n\n[1] T2VPhysBench: A First-Principles Benchmark for Physical Consistency in Text-to-Video Generation. 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dmzcoJKqg1", "reviewer": "Reviewer_aZkX", "summary": "This paper proposes a framework for evaluating video generation models in terms of their understanding and generation of physical concepts. The authors establish a comprehensive and fine-grained categorization of physical dimensions, and employ GPT-o1 as the evaluation model to assess ten different models from the perspectives of Semantic Adherence and Physical Commonsense. The evaluation results are shown to be consistent with human judgments.", "questions": "See Weakness.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "1. Establishing a comprehensive and fine-grained categorization of physical dimensions which is larger than existing benchmarks.\n2. The evaluation results are shown to be consistent with human judgments.", "weaknesses": "1. Apart from introducing a more fine-grained implementation in the dimension categorization, I do not see any genuine innovation in this paper. The authors’ claim regarding Anti-Physics has already been presented and evaluated in paper: Impossible Videos. Another point emphasized by the authors—using prompts of different granularities for the same scenario and concluding that finer-grained prompts lead to better results—has also been thoroughly discussed in paper VBench-2.0.\n2. In addition to the lack of novel and valuable conclusions, the evaluation methodology used in this work is much less rigorous compared to previous approaches such as VBench-2.0 and PhyGenBench. Notably, the authors do not consider the impact of camera motion on physical phenomena. For example, in the case of an apple falling, if the camera moves synchronously with the apple, how does the model determine whether the apple is truly falling? The paper merely utilizes a stronger MLLM, GPT-o1, which superficially appears to yield more accurate results. However, the use of a closed-source model introduces a critical flaw: as the model is updated, previous results may become irreproducible, which is a fatal issue for any benchmark.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)