WISA: World simulator assistant for physics-aware text-to-video generation
Authors: Jing Wang
Ao Ma
Ke Cao
Jun Zheng
Jiasong Feng
Zhanjie Zhang
Wanyuan Pang
Xiaodan Liang
Venue: NeurIPS 2025 spotlight ()
Keywords: Video Generation
Diffusion Model
Video Dataset
Primary area: deep_learning

Abstract: Recent advances in text-to-video (T2V) generation, exemplified by models such as Sora and Kling, have demonstrated strong potential for constructing world simulators. However, existing T2V models still struggle to understand abstract physical principles and to generate videos that faithfully obey physical laws. This limitation stems primarily from the lack of explicit physical guidance, caused by a significant gap between high-level physical concepts and the generative capabilities of current models. To address this challenge, we propose the **W**orld **S**imulator **A**ssistant (**WISA**), a novel framework designed to systematically decompose and integrate physical principles into T2V models. Specifically, WISA decomposes physical knowledge into three hierarchical levels: textual physical descriptions, qualitative physical categories, and quantitative physical properties. It then incorporates several carefully designed modules—such as Mixture-of-Physical-Experts Attention (MoPA) and a Physical Classifier—to effectively encode these attributes and enhance the model’s adherence to physical laws during generation. In addition, most existing video datasets feature only weak or implicit representations of physical phenomena, limiting their utility for learning explicit physical principles. To bridge this gap, we present **WISA-80K**, a new dataset comprising 80,000 human-curated videos that depict 17 fundamental physical laws across three core domains of physics: dynamics, thermodynamics, and optics. Experimental results show that WISA substantially improves the alignment of T2V models (such as CogVideoX and Wan2.1) with real-world physical laws, achieving notable gains on the VideoPhy benchmark. Our data, code, and models are available in the [Project Page](https://wisav1.github.io/WISA/).

WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generation
Jing Wang!.2i; Ao Ma Ke Cao Jun Zheng', .Jiasong Feng", Zhanjie Zhang', Wanyuan Pang", Xiaodan Liang".4.5; Shenzhen Campus of Sun Yat-Sen University, 2360 Al Research, University of Science and Technology Beijing. *Peng Cheng Laboratory,
Guangdong Key Laboratory of Big Data Analysis and Processing,
wangj977 @mai 12. sysu. edu.
6g
Dynamic
Thermodyynamic
Optic
WISA-80K
12%
Ogtic
1.Rigid body motion
7.Meltng
Tolal:80000
13.Reflaction
Thecoudy munsls
13%
Datasel Frame Distribution
2.Collision
S.Solidilcation
14, 14.Refmactine
L.iqqidd motion
Vaporization
Captions:
...A lange orplodion sexcing
ttick upwonds obscurtig the poth. explosion creates bright light
15.tttiiinng
Physical A anolation
T Textual physical description
motion
10 Liguraaattoon
She exploston should realistcally demonstiaee how the shot propogates through ahr,ir, coasing neadly cleett to next (eg. tree seeyring debiss being digglaced, and how the smake rises and ongunds due to the got preducod...
16labeffernoe difiraction,
Q Quaiitative physis categories
Quantilative physical properties
Flastio modion
11. Combustion
Dynamirs: col ision gao motion
Dansity: amoiee 0.001 0.01 gicns ddarts to 2.5 pimm?
Thermodynamic explosion
Optics stateeiig anzatural light
Time wittnn O swandss
Camera Movementt yos
Temperature 500 to 1000 degree,
Change of subjects gas objects
O 20
cebdus
6.Defommaion
12 12.gxton
17.Unnstuurl light source
Figure 1: Overview of physical dataset WISA-80K. (Left) Examples of 17 physical phenomena three physics categories WISA-8OK. (Top right) WISA-80K consists approximately 80.000 video clips. with 35% related to Dynamics, 23% to Thermodynamics, and 42% to Opticx. (TTp right) Distribution of frame counts across all vidcos in WISA-80K. (Bottom right) An example of physical annotation in WISA-80K.
Abstract
Recent advances in text-to-video (T2V) generation, exemplified by models such Sora and Kling, have demonstrated strong potential for constructing world simulators. However, existing T2V models still struggle to understand abstract physical principles and to generate videos that faithfully obey physical laws. This limitation stems primarily from the lack of explicit physical guidance, caused by a significant gap between high-level physical concepts and the generative capabilities of current models. address this challenge, we propose the World Simulator Assistant (WISA), novel framework designed to systematically decompose and integrate physical principles into T2V models. Specifically, WISA decomposes physical knowledge into three hierarchical levels: textual physical descriptions. qualitative physical categories, and quantitative physical properties. then incor-
Equul Contribution. *Corresponding Authors. Project Leader: Conducted during internship
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

porates several carefully designed modules- as -ia--offco-------iiiie Attention (MoPA) and a Physical Classifier-to effectively encode these attributes and enhance the model's adherence to physical laws during generation. In addi- tion, most existing video datasets feature only weak or implicit representations of physical phenomena, limiting their urility for learning explicit physical princi- ples. To bridge this gap, we present WISA-80K, a new dataset comprising 80,000 human-curated videos that depict 17 fundamental physical laws across three core domains of physics: dynamics, thermodynamies, and optics. Experimental results show that WISA substantially improves the alignment of T2V models (such Cog VideoX and Wan2.1) with real-world physical laws, achieving notable gains the VideoPhy benchmark. Our data, code, and models are available in the http:///sawaliituuu./i///W/AA
1
Introduction
21 Sora [26). and Cog VogVieex have endeavored to develop robust -ox-too-video (T2V) models for building world simulators While these models are capable of generating highly realistic and text-consistent videos, leveraging the scale of their data and architectures, they still face challenges in understanding abstract physical principles and producing videos that fully align with real-world physical laws S 23
Many recent studies (e.g., Cosmos 11 Kling 14].
41
39
43
The substantial gap between abstract physical laws and their visual manifestations presents signifi- cant challenge for injecting physical guidance into T2V models. Physical principles or laws are often conveyed through abstract natural language. reflecting the underlying operational logic of the real world. In contrast, generative models map textual descriptions directly to the visual appearance of ob- jects, including their color and shape. There is complex logical reasoning process between physical principles and the visual physical phenomena they give rise to. However, generative models, which are trained to map learned data distributions, struggle to extract appropriate physical information from a single textual instruction and translate it into a physically consistent visual representation for specific scenario. This challenge becomes even more pronounced in video generation, where the strict temporal order of physical events must be preserved.
To this end, we propose the World Simulator Assistant (WISA), which decomposes abstract physical principles into multiple categories of physical information and integrates them into T2V models to enable physics-aware generation. Specifically, it decomposes physical principles into three levels: textual physics descriptions, qualitative physics categories, and quantitative physical properties, and designs appropriate tailored injection methods for each type of information. The Textual Physical Description outlines the physical principles relevant to the scene, the resulting physical phenomena, and their specific visual manifestations. WISA incorporates this information by concatenating it with the caption before feeding it into the text encoder. The Qualitative Physics Categories indicate the types of physical phenomena that may be present in a scene. Following the focus of existing physical T2V benchmarks (c.g., VideoPhy and PhyGenBench). WISA targets 17 representative phenomena commonly encountered in video generation tasks. These span three major branches of physics (i.c.. dynamics, thermodynamice and optics) and include examples such as collision (dynamics), refraction (optics), and melting (thermodynamics)) Recognizing that different physical phenomena require distinct physical features, WISA proposes Mixture-off Physical-Expere
Attention
(MoPA), inspired by MoE 330 and MoH 12. MoPA assigns expert attention heads to individual physics categories, activating only the relevant experts during generation to specialize in modeling the associated phenomena. When scene involves multiple physical phenomena, MoPA dynamically activates multiple expert heads, allowing the model to effectively capture and synthesize complex physical interactions. Quantitative Physics Properties refer to numerical physical attributes that directly influence the physical process, such as density, duration, and temperature. WISA encodes these properties as physical embeddings and injects them into the model via AdaLN 27). In addition, WISA employs a Physical Classifier, which is designed to recognize qualitative physics categories and assist in perceiving physical properties.
However, extracting the above physical information and subsequently understanding physical prin- ciples from general scene video existing datasets 25 36 is a suboptimal approach for T2V models. Firstly, general scene videos often feature the interweaving of multiple physical phenomena. Individual physical phenomena are not prominently visualized, which makes it difficult to accurately extract physical information and establish precise connection between the physical data and its
2

---

corresponding visual manifestation. Secondly, in these datasets, only few videos distinctly highlight specific physical phenomena as representative examples, while most videos treat physical phenomena as secondary elements. For instance, in the Figure the flow of water is a secondary element. Despite having physical information guidance, the T2V models are unable to perceive the physical principles of fluid motion from this type of data.
To address these challenges, we col- lect and construct WISA-80K, a dataset containing 80,000 videos that represent 17 physical phenom- ena across three major branches of physics as shown in Figure LI de- signed as a data assistant for world simulators. Specifically, based on the previously defined physics cate- gories, we manually collect videos that clearly exhibit obvious physical phenomena corresponding to each category (e.g., as shown in the lower part of Figure 2). We then apply shot boundary detection, aesthetic quality filtering, and video captioning to the raw vidcos. Subsequently, leverage GPT-4o mini to extract and decompose the physical information from the video captions into textual physics descriptions, qualitative physics categories, and quantitative physics properties for WISA
Liquid motion general scenarios Physical phenomena are obvious
Liquid motion in IMSA-80K: Physical phenoment are obvious
Figure 2: Comparison between general videos in Koala-36M and videos with distinct physical phenomena in WISA-80K.
Our contributions can be summarized as follows:
We propose a physical principle decoupling method, bridging the gap between physical laws and generative modeling. In this method, physical principles are represented as structured physical information, encompassing textual physical descriptions, qualitative physics categories, and quantitative physical properties.
We present the World Simulator Assistant (WISA), which guides T2V models efficiently learn specific physical phenomena based on structured physical information, through special- Attention (MoPA) and Physical Classifier.
ized designs such Mixture-of ffiture-of-PPyysicll-
We manually collect 80,000 video clips that clearly showcase physical phenomena, creating the first large-scale physics video dataset, WISA-80K It broadly covers common physical phenomena observed in the real world, encompassing 17 types of physical events (e.g.. Collision, Melting, and Reflection) across three major branches of physics.
Quantitative and qualitative experimental results demonstrate WISA and WISA-80K can greatly assist basic T2V models in producing videos that better align with real-world physical laws, while introducing only 3.5% increase in parameter count and 5 % inference time.
2 Related Work
Text-to-Videe Generation
Early text-to-video (T2V) generation research [00 9
19 22 with temporal capabilities to enable video generation. These methods often suffered from limited realism and restricted motion dynamics. The powerful 3D spatio-temporal modeling and scalability of Diffusion Transformers 27
primarily extend image generation models [5] [8
[7
28
15 have greatly advanced the development of visual generation models. Enabled by Diffusion Transformers, series of recent T2V works (including OpenSora [42) Cosmos [.] Sora 26]. CogVideo: [4I). Hunyuan Vi]eo[x], Kling [14), Wan2. [31], and Step- Step-Video- Video-T2V (21)) significantly improve the realism and motion quality of video generation by scaling up model parameters and training data. These works are widely considered as promising pathway towards building World Simulator. However, they still struggle to generate videos that fully comply with real-world physical laws as they essentially fit the data distribution [3] from general-scene datasets such as Koala-36M (36) and OpenVid Vid (25), where physical laws are not explicitly reflected and physical phenomena are not prominently presented (e.g., in the upper part of Figure 2 In contrast, our carefully curated WISA-80K dataset prioritizes the explicit presentation of typical physical phenomena as the primary criterion for video collection presented in Figure [ And it provides detailed and structured
3

---

physical information annotations, making it valuable data assistant for enhancing the physical consistency of video generation.
Physical-aware Video Generation Recently, researchers 24
3 23
16
2.
20
40
37 have in-
creasingly focused improving and evaluating the physical consistency of generated videos. On the one hand, Videophy [3] and PhyGenBench (23) build test samples that reflect various physical laws, and they evaluate how well generated videos follow real-world physical laws by either training physics classification models with manual annotations or using question-answeriin methods based on
U'sion-Languag models [38] Physics-IQ[24] establishes a high-quality image-to-vides benchmark designed to evaluate the ability of I2V models to generate physically consistent video sequences based on an initial state and textual instructions. On the other hand, DANO [T6] MotionCraft (2). and PhysGen 20 parse objects from images and estimate their rigid motion in a differentiable manner by considering physical properties such as mass, inertia, friction, and rotation. Based on these estimations, they animate the images into videos. However, these methods are restricted to fixed physical categories (e.g., rigid motion) and static scenarios that involve only object motion, which hinders their generalizability. PhyT2V [40] leverages large language models and vision-language models to extract physical inconsistency information from generated videos. Based on the extracted physical feedback, it iteratively refines the textual description over multiple rounds, improving video generation quality. Although this approach offers generality, introduces significant inference overhead and fails to enhance the generative model's ability encode physical knowledge. In this paper, WISA incorporates structured physical information into the generative model, enhancing its physical perception and enabling it handle various physical phenomena more effectively.
3 WISA-80K
3.1 Data Collection and Annotation
Physical Laws Definition: In previous physics evaluation benchmarks 23). the physical phenom- ena emphasized in video generation tasks have primarily focused on three fundamental branches Dynamics, Thermodynamics, and Optics. Therefore, in this paper, we select 17 repre- sentative physical phenomena from these three core domains, excluding specialized cases such as electromagnetic phenomena.
of physics:
Collision, Rigid Body Motion, Elastic Motion, Liquid Motion, Gas Motion, and Deformation. For instance, the swinging of pendulum serves as an example of Rigid Body Motion.
Dynamics: We consider six common dynamic phenomena:
Thermodynamics: We select six common thermodynamic phenomena: Melting, Solidification, Vaporization, Liquefaction, Explosion, and Combustion. For example, a time-lapse of melting cream illustrates the Melting phenomenon.
Optics: We define five common optical phenomena: Reflection, Refraction, Scattering, Interference and Diffraction, and Unnatural Light Sources.
We did not include certain physical phenomena (e.g., sublimation, condensation) due to their in- frequent occurrence in real-world scenarios and the associated difficulties in collecting sufficient high-quality data. Subsequently, for each selected physical phenomenon, we manually collected vidcos from the Internet that clearly demonstrate the corresponding behavior, without relying on any existing video datasets for filtering selection. During the collection process, we exclude videos with overlaid text or significant visual blur to ensure clarity and quality. As a result, we curate a dataset comprising approximately 40,000 videos.
split the raw videos into individual scene clips, followed by filtering based on aesthetic scores. This process yields approximately 80,000 high-quality clips. Then, we utilize Qwen2.5-VL. 335 to generate video captions using the following prompt: (Please describe the content of this video in as much detail as possible, including the objects, scenery, animals, and camera movements within the video.) } The caption length is limited to 256 tokens.
Pre-processing and Caption: We use PySceneDetect

---

Monually Search
Scone detection and striping
Oysanvic:
Video Data Collection
Phyvial Definition
(2) bigy neriee
X
Leschettss screening
2f makiyg
Quantitative physkal pesperties
Qudka sopoction
Physical Descriation with GPT-4o min!
Demiity
Description with Qwen2
Time 2 3o33
pentar::
Qualitative phypics catagories
A8
Tectual phylical descrittion
Text Frompt
Ohange of sulpeas:
Drpankoo cullaia Eeformttonn lqpal
Thorsafroas replodin
cusverddas of cxcmities nls de buloor
wast inperttt
penople
vate bakkee Eai land. Tax satl to Subdenn of deeciad tonuth the belbbee caiding it bund woder conting spick and bund of water drglies
The vinn depies
boddig
Camera Mosomoet 220
splisk clact
Manually
Change addects lngar upperspoo.
The plearmstio of cearniss crics, rarest with hhe same anging andraniio
Evaluation
#dtça ctee dbjec lung pracicee
Figure 3: Pipeline of WISA-8OK, We first define 17 common physical phenomena and, based on this, manually collect 80,000 video samples that clearly illustrate these phenomena. Then, perform shot detection and aesthetic filtering on the raw videos. Text descriptions are extracted using Qwen2.5-VI. and detailed physical annotations are generated with GPT-4o mini.
3.2 Physical Information Decompose
We believe that simple video captions and related physical phenomena in a video. As shown in the Figure 3 we further constructed structured physical annotations analyze the physical information from multiple dimensions. Specif- ically, we decompose the physical information into: textual physical descriptions, qualitative physics categories, and quantitative physical properties.
sufficient to clearly represent the physical information
Textual physical descriptions: Provide a detailed explanation of the physical principles to be considered and the resulting intuitive physical phenomena, while supplementing the missing physical information in the prompt. For instance, the prompt "an antique clock swings" corresponds to the textual physical description: "... the amplitude of the swing gradually decreases..."
These indicate the types of physical phenomena involved in a video. Although each video is collected based on a specific physical phenomenon, it may still encompass multiple types. Therefore, for each video, we identify the presence of dynamics-related, optics-related. and thermodynamics-relle
Qualitative physics categories:
phenomena to effectively handle cases where multiple physical effects are coupled. Additionally, three categories of anomalies (i.e., No obvious dynamic phenomenon, No obvious thermodynanic phenomenon, and No obvious optical phenomenon) for scenarios that do not involve dynamics, thermodynamics, or optical phenomena. Furthermore, nine categories of visual phenomena are introduced, two of which pertain to whether the shot exhibits motion, while the remaining seven correspond to changes in the state of moving entities (i.e., Object decomposition and splitting, Mixing of multiple objects ... For detailed explanations, please refer to Supplementary Material A.5 In total, there are 29 qualitative physics categories.
introduced to account
Quantitative physical properties: Three physical attributes related to multiple physical phenomena are annotated, namely the density of primary motion physics, the time range during which the physical phenomenon occurs, and the temperature range during which the physical phenomenon occurs.
Due to the significant computational overhead and cost associated with video multi-modal models, the annotation of the above physical information is carried out using GPT-4o mini based on the caption. Specifically, we conduct five rounds of annotation to label qualitative physical phenomenon categories (i.e., dynamics, thermodynanics, optics, motion, the state of objects), and three rounds to annotate quantitative physical attributes (i.c., Density, Time and Temperature). Detailed annotation prompts and examples are provided in the Supplementary Material A.7 and A.8
To verify the reliability of the automatic annotations, we conducted manual evaluation on a randomly sampled subset of 500 videos from our dataset. Our evaluation focused on the three types of Al- generated annotations: 1) Textual Physical Descriptions: We measured human rater satisfaction (i.e.. does the description accurately reflect the video's 's physics?). Result: 95% satisfaction rate. 2) Qualitative Physies Categories: We compared the Al-generated labels against the ground-truth labels assigned during initial video collection. Result: 76% accuracy. 3) Quantitative Physical Properties: We again measured human rater satisfaction (i.e., are the estimated densityliime/eeperauue values plausible?). Result: 86% satisfaction rate. While some label noise is inherent in any large-scale,
5

---

Vastal natoed tolen
Pryscca class viene
World Simulator Assistant (WTSA)
Mesiccl tolven
Prongt taken
Plysical Chessifie
Poyare Class Guldiance
Tediaal plasiial dencription
The read iepoetont plyyical procgle
cosatee consertttion eneeyy
DIT Block
Prynical Module
Quipui
dueppeng mnnnge
T5
Projection
fittiont the erpiltads of fhe oving
gradially decresss over time while the Inequeay
Physical Module
Teat Proic
The peodien of an untigua eainge
cideg and
Ougar
Gate
Head,
Paydics-Gudded Bandom Gate
Heady
Head,
Head,
Pertorbobbon
Kandom
Plasks Class Gutdaoee
mytheeic rastos
Mistum of Phystcal
Exports
ttrntion
Duanilative
Proparses
Plystca
PO antoodding
Sclerzfle
Linorr
SIMLL
Heed
Linwe
Head,
Fead,
Liver
Head
Dreansks 12) Baad body Ewrsio
Gralitaterr plases ettegortes
Vetabor
Expu4
Thamsdrraaaa (-41 obasss plenoconse
(g/owl))
Convendonn
Oytic: (20 dieces photaneaza
Time 50
Terperomm-P (N)
Timele
Addition
Concatinate
Change of sibjett (20)
Figure 4: Overview of the proposed WISA. WISA introduces the Physical Module and Physical Classifier, which leverage structured physical annotations guide and assist T2V models in generating physics-aware videos.
automatically annotated dataset, these results demonstrate that the overall quality of WISA-DATA- 80K's annotations is high. The data is sufficiently reliable to provide strong learning signal, as evidenced by the performance gains in experiments. More analysis of WISA-80K please refer to the Supplementary Material A.6 A.9
4 Method
4.1 Overview
Given textual physical descriptions, qualitative physical categories, and quantitative physical proper- ties, we design the WISA framework to efficiently incorporate these conditions into existing T2V models (i.e., CogVideo2 VideoX [41 or Wan2. I (3I). facilitate the learning of physical knowledge while preserving the model's original capabilities with limited video data, design three distinct condition injection methods tailored to each of the three categories of physical information, as concatenate them with illustrated in Figure 4 Specifically, for the textual physical descriptions, the video caption and leverage the generative model's 's inherent semantic understanding to generate visual phenomena described in text (Such "amplitude of the swing gradually decreases over time" in Figure 4.. For qualitative and quantitative physical conditions, WISA introduces the Physical -frire-o--hyysccll-eee
Module. In this module, we propose expert heads to each physics category to model category-specific features. Quantitative physical quantities are encoded as physical embeddings and then integrated into the denoising feature within the module using AdaLN. Additionally, we introduce a qualitative Physical Classifier to help the model understand the physical conditions. Due the sigmificant computational and parameter cost introduced by MoPA, only one physical module is inserted after all the Diffusion transformer blocks to accelerate training and reduce the overall burden. Detailed explanations and elaborations of the Physical Module and Physical Classifier are provided in Sec. 4.2 and Sec. 4.3
Attention (MoPA), which assigns
4.2 Physical Module
Most videos from real-world scenes involve the coupling of multiple physical phenomena. Even when decomposed into distinct physical categories in WISA-80K, it remains challenging for T2V models to comprehend the abstract qualitative physical categories and accurately model specific types of physical phenomena. To address this challenge, we propose a Mixture-of -oaisse-----hssca---Eee -xxperts Attention within the Physical Module. Inspired by MoH [12], this mechanism assigns each head in the multi-head self-attention to specific class of physical phenomena and activates the output of the relevant head only when the corresponding phenomenon present. This approach treats each head as expert in its domain, enabling it to independently model the properties of a particular physical phenomenon. In the presence of coupled physical phenomena, multiple corresponding expert heads activated to effectively model the interactions among them.

---

Specifically, qualitative physical categories are encoded as E.RC where C' denotes the number of defined physical phenomena (i.e., 29). Here, P indicates that the corresponding category is not activated, and P: = 1 indicates that the corresponding category is activated, with i being the category index. Physical categories cannot be absolutely correct and may contain noise, such as incorrect activations or suppressions. To mitigate the impact of these noises on training, we employ a random perturbation operation, where the positions with P. are set to O, and the positions with P. = 0 are set to 1.0 with a certain probability (i.e., 0.2), resulting P.. After the multi-head self-attention operation, the denoising feature Fh E Rxxxx (where presents the number of head and h = C, and denotes head dimension) will interact with P. to activate and suppress the experts corresponding different physical phenomena. The feature dimension is then restored through concatenation and a linear layer. The mathematical representation of this process as follows:
Pc Random(PP)) Fa MHSA(F),
F. Linear(Reshape(E) + Pc))
where Random denotes random perturbations operation, MHSA represents multi-head self-attention, and @ denotes element-wise multiplication.
Due to the large variations in the time and temperature spans of different physical phenomena, we first represent the temperature and time in quantitative information using scientific notation, with coefficients and exponents. These values Pp RR are mapped through a linear layer, concatenated with the timestep embedding Te E TR,,, and injected by AdaLN. The mathematical representation of this process as follows:
ar, 1,8, 7 Chunk ChineeL (Conaaa Ta)), dim 1)
F a) + 8, Fo Fo+
Generative models often consist of multiple transformer blocks with large feature dimensions, inserting the Physical Module after every block would lead to an explosion in both parameters and computational complexity. Therefore, insert the Physical Module only after the final transformer block, achieving efficient physical information guidance while mitigating the aforementioned issues.
4.3 Physical Classifier
To guide the generative model in understanding abstract physical categories and modeling physical properties, we introduce Physical Classifier after the Physical Module to predict qualitative physical categories. We introduce learnable embedding vector, which call the [PHYSICS_TOKEN] This token concatenated with the noisy visual tokens and text prompt tokens and is processed by the entire MM-DiT and MoPA architecture. The final hidden state of the [ F. E RC, is fed into a simple MLP classification head. This head outputs logits corresponding to the 29 qualitative physical categories, performing a multi-label classification task over the categories defined in our work. This output is used only to compute the multi-label binary cross-entropy loss for training.
Lpp = Σ (P)) + (1 Pogl1 f()),
where is the number of physical categories, and fa E RC represents the predicted probabilities, which are obtained by passing F. through the sigmoid function. For each video, the model predicts which of these phenomena are present. During inference, the output from the classifier head is entirely discarded and has influence the video generation process. Its sole purpose is to serve as an auxiliary training signal, guiding the model to better learn and represent physical concepts.
To balance the introduced classification loss Lye and the diffusion loss Lusff Jsiion. we adopt the following loss function optimize the physics-awes aware generative model.
L = Fuifiosion ALpe (1 + detach),,
(4)
where À is balance coefficient.
7

---

Table 1: Quantitative evaluation using VideoCon Phyysics conduct on the Videophy and PhyGenBench prompt lists. The best performing metrics are highlighted in bold.
Inference|
Prompts from VideoPhy
Prompts from PhyGenBench 2
Method
Time (s) IsS Ct)CLIP SA) (1) PC (t)IS
CLIPSIM (A) SA (t) (1)
VideoCrafter2
0.47
0.36
OpenSora
42
28.72
0.2638
0.21
0.35
Hunyuan Video
17
0.46
0.28
Cosmos- Cosmos-Diffusionn Diffiusion-B
11
600
25.58
0.2444
0.52
0.27
200117
0.1956
0,41
0.24
Cog Vod 5B
[4]
210
130.17
0.2714
0.57
0.41
26.49
0.2590
0.34
0.42
CogVideoX-5B- Phy
40
1800
0.59
0.42
0.38
0.42
CogWideoX-5BB-WIS.
220
34.62
0.2822
0.62
0.45
27.31
0.2813
0.39
0.45
Wan2.1-14B (3]
900
33..22
0.2686
0.54
0.31
[33.41
0.2488
0.39
0.28
Wan2.1-14B-WISA
960
38.18
0,2813
0.60
0.36
37.62
0.2725
0.42
0.33
5
Experiments
We select the current representative open-source T2V model, CogVideoX- and Wan2.1-14B. as the base T2V models to validate the effectiveness of WISA. More training detail, please refer to Supplementary Material
Training Setting:
Evaluation: We select VideoCon-Physic. from Videophy 3 evaluate the physical law con- sistency (PC) and semantic coherence (SA) of the generated videos. We use 160 carefully crafted prompts from PhyGenBench (23) and 344 prompts from Videophy, designed to reflect various physi- cal principles, for testing. Following VideoCon-Physice we compute SA and PC by averaging the predicted results. Additionally, we adopt the Inception Score (IS) to evaluate the perceptual quality of generated videos and employ CLIP similarity (CLIPSIM) [ to measure text-video alignment. More evaluation detail, please refer to Supplementary Material
5.1 Quantitative comparison
We select five general text-to-video generation models (i.e., VideoCrafter2, OpenSora, Hunyuan Video, CogVideoX-5B and Cosmos-Diiffsiion71D and PhyT2V, method specifically designed to enhance physical properties, for quantitative comparison, as shown in Table [
VideoPhy:
WISA achieves state-of-the performance on both SA and PC metrics while maintaining high efficiency. Compared to the Cog VideoX-SB. VogVido SBWWISA improves SA and PC scores by 0.05 and 0.04, respectively, demonstrating that our proposed method significantly enhances the realism of generated videos. PhyT2V improves its performance by iteratively analyzing physical errors in generated video captions and adjusting the input prompts based on feedback from VideoCon- Physics scores. However, its cumbersome pipeline, which involves multiple rounds of Tarsier- 34BB B22 inference for video generation, introduces extremely long inference time-approximately times longer than the original generation model. Cosmos exhibits poor performance due to the disordered physical processes and inconsistent temporal sequences. Furthermore, 1-
also achieves performance improvements on 1an2.1, showing advantages in metrics such as IS and CLIPSIM. However, since 1 is not among the nine generative models used to construct the training data for VideoCon-Physie (whereas Cog VideoX is included), it shows certain disadvantages in SA and PC. Despite this, Wan2.1 demonstrates superior video quality, achieving better performance in the IS.
PhyGenBench: We also evaluate WISA using prompts from PhyGenBench, observing significant improvements on both Cog Vidox and 1an2.1, which demonstrates the generalizability of WISA.
5.2 Qualitative comparison
We further provide a qualitative comparison with existing methods demonstrate the advantages of AISA As shown in the Figure 5) for the example of the rope supports a wooden swing, Cog VideoX-
8

---

Theppegguitt woodmnsing
Anuplee into cider sanding uig agmny
Figure 5: Qualitative comparison between CogVideoX-5575 BB-IIAA and existing T2V methods. VogViooXX-BB WISA exhibits better alignment with real-world physical laws.
Rigid Body Motion: A antigun dlock's pendulum hangs inside its omaat wonden casing, with the camera focusing rhythmic motion. The mital remaine
Rigid Body Motion: solitary lantern swings gently in night, casting flickering shadows the cobolestone creating dancing shapes that morph and twist
llsstc metion:A brigitt green rubber band stretches as flagers pall taut, reveeling thin demonstrating the clasticity of maternals through clear deformation.
Elastic motion: vibract red rubber ball bounoes steadily on smooth, subht sidewaik. each impart compersees the rubbee then releases, sending il souting back into the alt:
calm lake single gliding gracefilly surfice disterting its reflection... that ochoes the metion with ench gentle stroke of wings.
Reflection:
Interforeace diffraction: small soap bubble floats
The surfice of buibble canses light to interiere, creating henntffl rainbow
Melling:
... causes tiny droplets Explosion: firework bumts mid-air, mllassing briliant flash of lighht Pragments of colored paper scather wildly, spiraling away like confetti
farm on is surface, these dropies gradually morge into small stramms siding down the rdges.
Figure 6: More samples generated by Videod SB- WISA, covering additional physical phenomena.
5B-WISA generates video where the rope suspending the wooden seat swinging back and forth in accordance with physical laws. In contrast, Cog VideoX-5B produces unstable swing motion; Phy T2V fails generate the swinging behavior of the wooden seat; and Cosmos generates physically inconsistent scene where the rope breaks while the wooden seat remains horizontally suspended. In the example on the right, WISA successfully simulates the process of an apple falling into water: the water surface remains calm before the apple enters, splashes form as the apple impacts the water, and the apple experiences buoyant force after submersion. However, CogVi VideoX-5B generates chaotic water and apple movements, PhyT2V omits the falling process, and Cosmos mistakenly generates two apples at the end. Additional videos generated by CogVideoX SBSA demonstrating various physical phenomena, also presented in the Figure 6 All aforementioned videos, along with comparisons on Wan2.1, 1, are provided in the Project Page
5.3 Ablation Study
We conduct ablation studies on VideoPhy using VideoCon-Physice to verify the effectiveness of key components in our method, as shown in the Table The baseline is CogVideoX-5B. As expected, removing MoPA results
performance drop due to the absence of qualitative physical information as guidance. Similarly, the inclusion of the Physical Classifier aids the generative model in perceiving and modeling physical properties, thereby enhancing both semantic relevance and consistency with physical laws. Notably, the evaluation model CeeeoCoo-Physiic
3 is trained samples generated

---

by nine different T2V models, leading to distribution shift when compared to the real-world videos in WISA-80K. Consequently, relying solely on LoRA yields only limited improvement.
To further investigate the impact of clearly-defined physical phenomena data versus general scene data on physical perception, we fine-tune LoRA on 80,000 videos from open-source video dataset. This results in only slight performance decline, indicating that the physically grounded videos WISA-80K provide sub- stantial value for modeling physical properties.
Table 2: Ablation study the key components of WISA. "PC" denotes the Physical Classifier.
Textual
QualitaiiveQuaniittvve
Setting Data
Physical
Physies
physical proporties
SA (t)PC (1)
Castgontionsaarggoee
Baseline
0.57
0.41
only LoRA General Data
0.57
0.40
only LoRA
WISA-80 80K
0.58
0.43
w/o MoPA
WISA-80K
V
V
0.59
0.43
w/o MoPA
WISA-80K
0.57
0.42
w/o PC
WISA-8OK
V
0.60 0.44
w/o PC
WISA-80K
0.61
0.44
WISA
WISA-800 -OK
0.62 0.45
5.4
User Preference
The physical consistency of generated videos is abstract and difficult to quantify directly. Therefore, we conduct human evaluation to assess the effectiveness of WISA. Specifically, we selected three representative models for comparison. The evaluation considered two aspects: se- mantic consistency and physical aligmment. Each candi- date model ranked both aspects, receiving score based on its ranking: points for first place, 2 points for second, and points for last. We collected preference re- sults from 100 participants. As shown in the Figure Figure demonstrates that WISA achieves significant advantage physical alignment, while also maintaining strong semantic consistency.
User
CogodderE-Shh
400
200
Sementi: Cussislenty
Physial Aliganert
Figure 7: User Preference on VideoPhy prompts.
5.5 Attention Map Analysis
We further conduct visual analysis of the Mixture-of of-lurrc---phsyccallll Pilysicl-Expprtt attention maps, aiming to investigate whether different physical experts focus on the regions corresponding to distinct physical phenomena. As shown in the Figure the rigid body motion expert perfectly focuses the swing region, while the non-dynamics expert attends to the static background with no apparent motion. This demonstrates that the MoPA effectively models and captures the corresponding physical attributes.
Conclusion
In this paper, present WISA framework, which decomposes physical principles into structured physical information, includ- ing textual physical descriptions, qualita- tive physical categories, and quantitative physical properties. To help T2V mod- els learn these physical aspects effectively, WISA incorporates two key components: the Paiiee-f--Psscca---Fee and the Physical Classifier. Building on this, we construct WISA-80K. dataset containing 80,000 video clips that cover 17 physical phenomena across three funda- mental categories of physics, providing a high-qualit data foundation. Experimen- tal results show that WISA and WISA-80K can effectively help produce videos that better align with real-world physical laws, while the additional computational overhead is under 5%. We hope that WISA can provide valuable insights into the research on building powerful world simulators. We further discuss the limitation of this paper in the Supplementary Material A.2
Geverated video
Attention
Attention map of "rigid body motion" expert
Atention of "ne obvious dynamic phenomenon"
Figure 8: Attention maps of different physical experts.
10

---

ACKNOWLEDGEMENTS
This work is supported by Scientific Research Innovation Capability Support Project for Young Faculty (N.2ZYCXNNISYYCKK.ZZXX..22 (NSFC) under Grants No. No.62276293 and General Embodied Al Center of Sun Yat-sen University.
National Natural Science Foundation of China
References
[1] Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical ai. arXiv preprint arXiv:2501.00 20025500..375,,, 2025.
[2] Luca Savant Aira, Antonio Montanaro, Emanuele Aiello, Diego Valsesia, and Enrico Magli. Motioncraft: Physics-basee based zero-shot video generation. arXiv preprint 1.XXx::240..355555 2024.
[3] Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang, and Aditya Grover. Videophy: Evaluating physical commonsense for video generation. arXiv preprint arXivv2406.035200 2024.
[4] Andreas Blattmann, Tim Dockhor, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint S..i::23/1./5/222 2023.
[5] Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22563-22575, 2023.
[6] Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman. Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024.
[7] Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 7310-7320, 2024.
[8] Jiasong Feng, Ao Ma, Jing Wang. Bo Cheng, Xiaodan Liang, Dawei Leng, and Yuhui Yin. Fancyvideo: Towards dynamic and consistent video generation via frame textual guidance. arXiv preprini arXiv:2408.081/ 2024.
[9] Rohit Girdhar, Mannat Singh, Andrew Brown, Quentin Duval, Samaneh Azadi, Sai Saketh Rambhatla, Akbar Shah, Xi Yin, Devi Parikh, and Ishan Misra. Factorizing text-to-video generation by explicit image conditioning, In European Conference on Computer Vision, pages 205-2244 224. Springer, 2024.
[10] Yuwei Guo, Ceyuan Yang, Anyi Rao, Yaohui Wang, Yu Qiao, Dahua Lin, and Bo Dai. Ani- matediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04722 2023.
[11] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A ceference-free evaluation metric for image captioning, arXiv preprint arXiv:2104.08718 2021.
[12] Peng Jin, Bo Zhu, Li Yuan, and Shuicheng Yan. Moh: Multi-head attention mixture-of-head attention. arXiv preprini arXiv:2410. 1842, 2024.
[13] Bingyi Kang, Yang Yue, Rui Lu, Zhijie Lin, Yang Zhao, Kaixin Wang, Gao Huang, and Jiashi Feng. How far is video generation from world model: physical law perspective. arXiv preprint 2rXiv:2411.02385, 2024.
[14] Kuaishou. Kling. https: hww.hiinoiiikikssuu...u. com/ 2024. Accessed: 2024-09-03.
[15] Black Forest Labs. Flux. http:://ggithbb.com/mlaettttee -absresstt-a 2024.

---

[16] Simon Le Cleac"h, Long-Xiig Yu, Michelle Guo, Taylor Howell, Ruohan Gao, Jiajun Wu, Zachary Manchester, and Mac Schwager. Differentiable physies simulation of dynamics- augmented neural objects. IEEE Robotics and Automation Letters, 8(5):2780-27877 2023.
[17] Zhimin Li, Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Ying- fang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, et al. Hunyuan-dit: A powerful multi-resolution diffusion transformer with fine- gra-gra chinese understanding. arXiv preprint arXiv:2405.08744 2024.
[18] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02774, 2022.
[19] Shanyuan Liu, Dawei Leng, and Yuhui Yin. Bridge diffusion model: bridge non-english language-native text-to-image diffusion model with english communities. arXiv preprint arXiv:2399000552, 2023.
[20] Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shenlong Wang. Physgen: Rigid-body physies-grouss grounded mmage-to-vides generation. In European Conference on Computer Vision, pages 360-378. Springer, 2024.
[21] Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, al. Step-video-12v technical report: The practice, challenges, and future of video foundation model. arXiv preprint arXiv:2502.10248, 2025.
[22] Yuhang Ma, Shanyuan Liu, Ao Ma, Xiaoyu Wu, Dawei Leng, and Yuhui Yin. Hico: Hierarchical controllable diffusson model for layout-to-imas generation. Advances in Neural Information Processing Systems, 37:1 1 18886-118889)
2024.
[23] Fanqing Meng. Jiaqi Liao, Xinyu Tan, Wenqi Shao, Quanfeng Lu, Kaipeng Zhang, Yu Cheng, Dianqi Li, Yu Qiao, and Ping Luo. Towards world simulator: Crafting physical commonsense- based benchmark for video generation. arXiv preprint arXiv:2410.05363, 2024.
[24] Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do generative video models understand physical principles? arXiv preprint arXiv:2501.09035 2025.
[25] Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhenheng Yang. Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. Openvid m: A large-scale high-quality dataset for text-to-video generation. arXiv preprint arXiv:2407.02377, 2024.
[26] OpenAl. Sora.
com/ 2024. Accessed: 2024-09-03.
[27] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195 4205, 2023.
[28] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952. 2023. http://wwww.ceeeeettttttt com/
[29] PySceneDetect Developers. Pyscenedetect.
[30] Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, and Neil Houlsby. Scaling vision with sparse mixture of experts. Advances in Neural Information Processing Systems, 34:8583-8595, 2021.
[31] Wan Team. Wan: Open and advanced large-scale video generative models. 2025.
[32] Jiawei Wang, Liping Yuan, Yuchen Zhang, and Haomiao Sun. Tarsier: Recipes for training and evaluating large video description models. arXiv preprint arXiv:24077000633 2024.
[33] Jing Wang, Ao Ma, Jiasong Feng, Dawei Leng, Yuhui Yin, and Xiaodan Liang. Pt-12i/v: An efficient proxy-tokenized diffusion transformer for rext-to-image//iddeottak The Thirteenth International Conference Learning Representations.
12

---

[34] Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang. and Shiwei Zhang. Modelscope text-to-video technical report. arXiv preprint arXiv:2308.06: 06577, 2023.
[35] Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024.
[36] Qiuheng Wang, Yukai Shi, Jiarong Ou, Rui Chen, Ke Lin, Jiahao Wang. Boyuan Jiang, Haotian Yang, Mingwu Zheng, Xin Tao, al. Koala-36m: A large-scale video dataset improving con- sistency between fine-grained conditions and video content. arXiv preprint arXiv:2410.08260, 2024.
[37] Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video generation with cascaded latent diffusion models. International Journal of Computer Vision, pages 1-20, 2024.
[38] Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng. Zun Wang, Yansong Shi, al. Internvideo2: Scaling foundation models for multimodal video understanding. In European Conference on Computer Vision, pages 396-416. Springer, 2024.
[39] Jiannan Xiang, Guangyi Liu, Yi Gu, Qiyue Gao, Yuting Ning, Yuheng Zha, Zeyu Feng, Tianhua Tao, Shibo Hao, Yemin Shi, et al. Pandora: Towards general world model with natural language actions and video states. arXiv preprint arXiv:2406. 09455, 2024.
[40] Qiyao Xue. Xiangyu Yin, Boyuan Yang, and Wei Gao. Phyt2v: LIm-guided iterative self- refinement for physics-grounded text-to-video generation. arXiv preprint arXiv:2412.0059., 2024.
[41] Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408. 06072, 2024.
[42] Zangwei Zheng, Xiangyu Peng. Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all, March 2024.
[43] Zheng Zhu, Xiaofeng Wang, Wangbo Zhao, Chen Min, Nianchen Deng, Min Dou, Yuqi Wang, Botian Shi, Kai Wang, Chi Zhang, al. Is sora world simulator? comprehensive survey on general world models and beyond. arXiv preprint arXiv:2405.03520, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly articulate the main contributions of the paper. including the proposed WISA and WISA-80K (3), the design of the Physical Module (4.2). and the overall performance of WISA in experiments (5). These claims are well supported by the methods and experimental results presented in the main paper.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attainable by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of the proposed method have been clearly discussed in the
paper A.2
Guidelines:
The answer NA means that the paper has no limitation, while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
14

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the suppiemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper provides all necessary implementation details, including model architecture, training settings, hyperparameters, and dataset preprocessing steps. We will release the full code and trained models upon publication.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
15

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: To preserve anonymity during the review process, have not released the code and data yet. However, we commit to open-sourcing both the code and data, along with detailed instructions for reproducing the experiments, upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
httips.cc/public//ee
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide all relevant experimental details in Supplementary Material A.3 These details ensure the clarity and reproducibility of our experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: While we do not report explicit error bars or confidence intervals, we follow the standard evaluation protocols from prior related work ensure fair comparisons.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide detailed information about the compute resources used for our experiments, including the type of GPUs and other relevant training settings (A.3). These details are included to support reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httups:/ cttpiiipp.ccpppuieea hctps///cc...ttttttcccccl......d
Answer: [Yes]
Justification: The research conducted in this paper fully adheres to the NeurlPS Code of Ethics. We have ensured that all ethical guidelines regarding data usage, privacy, and research integrity have been strictly followed.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
17

---

Justification: The paper does not explicitly discuss societal impacts. However, since the work focuses on text-to-video generation, we acknowledge potential misuse, such generating misleading or harmful content. We plan to include safeguards and responsible usage guidelines when releasing models to mitigate such risks.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [Yes]
Justification: Since our work focuses on text-to-vides generation, we recognize its potential for misuse, such in generating misleading fake images. To mitigate this, intend to release our model under a research-only license with clear usage restrictions. The dataset used is public and curated avoid harmful content.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
18

---

Justification: We use several publicly available datasets and pretrained models (e.g.. XogVideoX, Wan2.1) in our experiments. All datasets and models are properly cited in the main text and/or appendix. We ensured that their licenses are respected, and no data under restrictive or ambiguous licenses was used.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the assct's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification: While we plan to release the code and model after the review process, at the time of submission no new assets are publicly released. Upon release, we will ensure that comprehensive documentation, licensing terms, and usage guidelines provided in accordance with the NeurlPS guidelines.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification; The paper does not involve any crowdsoureing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
19

---

15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing or research with human subjects, so IRB approval is not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration not required.
Answer: [NA]
Justification: LLMs were only used for writing and editing assistance. They were not involved in any part of the core methodology, experimental design, or scientific contribution.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy /ettps://neurjini htt.iipp.....oooteeeeeee cC/oofferencess22255 /LLM) for what should or should be described.
20

---

A Technical Appendices and Supplementary Material
A.1 Broader Impacts and Ethical Considerations
The development of powerful generative models, particularly those capable of creating realistic video content, carries a significant responsibility. The work presented in this paper, while aimed at advancing scientific understanding and technical capability physics-aware video generation, not exempt from potential societal impacts and ethical challenges. This section transparently discusses these issues and outlines the steps we are taking to mitigate potential harms.
We recognize the dual-use nature of our work. The same technology that can be used for creative applications, scientific simulation, or special effects could also be exploited enhanced physical realism of videos generated by a model like WISA could amplify the verisimilitude of synthetic media, increasing its potential be used for misinformation, disinformation (e.g.. "deepfakes"), or propaganda. Such content could erode public trust, be used as false evidence, or create realistic depictions of accidents or violence to incite fear.
Potential for Misuse and Societal Impact:
create compelling but fabricated content for malicious purposes. The
Dataset Ethics and Release Strategy: The creation and distribution of any large-scale dataset require careful consideration of privacy, copyright, and consent. The WISA-80K, dataset is constructed using video clips from publicly available channels on YouTube.
Copyright and Terms of Service: To respect the rights content creators and to comply with platform Terms of Service, we have adopted metadata-only release strategy. We will not host or distribute any video files, clips, or raw pixel data. The released dataset will contain only the public YouTube video IDs, the relevant start and end timestamps of the physical phenomena, and our corresponding physical annotations. This is a standard and widely accepted practice in the research community (e.g., AudioSet) that enables reproducible research while avoiding copyright infringement.
Responsible Data Access: To further ensure responsible use, the WISA-80K metadata will not be available for direct public download. Instead, we will implement a gated access mechanism. Researchers wishing to use the dataset must submit a request outlining their institutional affiliation and research purpose. They will be required to agree to a Data Usage Agreement (DUA), which will stipulate that: (1) the dataset is to be used for non-commercial research purposes only; (2) users are responsible for their own adherence to YouTube's s Terms of Service when accessing the original videos; and (3) the metadata and any derived video content may not be redistributed.
Content Creator Rights: We will provide clear and accessible opt-out mechanism on our future project page, Any content creator can request the removal of their video's ID from our dataset at any time, and we are committed to promptly honoring all such requests.
The physical annotations in WISA-8OK were generated using a large language model (GPT-40-mini). We acknowledge that this automated process may introduce biases or factual inaccuracies ("hallucinations").
Annotation Process: Quality and Bias:
Human Validation: To quantify the quality and reliability of these annotations, we conducted human validation study on randomly sampled subset of 500 videos. The results (detailed in Appendix X) indicate a high degree of quality, with 95% satisfaction for textual descrip- tions and 86% satisfaction for the plausibility of quantitative estimates. The accuracy for qualitative category classification was 76% against human-assigned labels.
Dataset Positioning: These results confirm that while some label noise is inherent, the annotations provide a strong and reliable learning signal. Nonetheless, WISA-80K should be understood as a large-scale, weakly-superrine dataset rather than a gold-standard, error-frec resource. We encourage future work to further refine and build upon these anmotations.
Mitigation Strategies for the Generative Model: To address the risks of misuse associated with the generative model itself, we commit to the following safeguards for any future public release:
21

---

Visible Watermarking: All video outputs generated by our released model will be pro- grammatically embedded with a clear and persistent visible watermark identify them as synthetic.
Responsible AI License: We plan to release the model under a Responsible Al License (e.g.. a CreativeML Open RAIL-M license). Such licenses contractually prohibit users from employing the model for malicious, deceptive, illegal, unethical purposes, including the generation of harmful misinformation.
A.2 Limitation
Although our approach significantly improves the ability of existing T2V models to generate videos that align with real-world physical laws, it still has the following limitations: 1) Limited physical categories: We collect 80,000 videos in WISA-80K, covering 17 types of physical phenomena. How- ever, due to constraints in time and manpower, the dataset does not include all physical phenomena encountered in the real world, such as corrosion or vacuum environments. 2) Limited physical information guidance: WISA primarily provides high-level semantic guidance and lacks detailed constraints at the physical mechanism level (e.g., energy conservation, Newton's 's laws). However, introducing more detailed physical principle constraints currently requires modeling object motion based on image or 3D information, which suffers from poor generalization and can only handle limited categories and scenarios. How to incorporate physical principle constraints into text-to-video generation while maintaining generalization remains an area worth further research. 3) Expanding WISA Unseen Classes: We conducted additional qualitative evaluations on unseen physical categories such as corrosion and electromagnetism, as shown in Figure ( As expected, both our WISA-enhanced model and the base model struggle to generate physically plausible videos for these categories. This limitation primarily stems from the absence of relevant concepts and visual examples in the WISA-80K training set, underscoring the current challenge generalizing to entirely out-of-distribution physical phenomena. However, argue that WISA's modular design, particularly the Mixture-of aff-Physical Expeerts Attention (MoPA) mechanism, makes it inherently more scalable than monolithic architectures. When introducing new physical category, traditional fine-tuning methods (e.g., applying LoRA on the base model) typically require retraining large portions of the network to assimilate the new knowledge. In contrast, MoPA allows for more targeted and efficient expansion: we can simply add and train a new expert head dedicated to the novel phenomenon while keeping the existing experts largely frozen, thus preserving prior knowledge and facilitating incremental learning.
Apple riredy nuildtive and corradie the
Touck she door Aunllr winter dea clatie electricity
Bookee
Weaks NISS
Figure Visual comparison on unseen categories.
Further work: In future research, we plan to explore physically consistent generation for the image-to-video (12V) task. principle, WISA can be applied to 12V as well. The main distinction between 12V and text-to-video (T2V) lies in the input: 12V models receive clean latent code for the first frame, whereas the rest of the imput text and model architecture remain the same. With explicit initial frame information, the 12V model primarily learns to infer subsequent states, generating intermediate and final frames that follow plausible physical processes.
Compared with T2V, 12V is particularly relevant in embodied intelligence and robotics, where agents often observe a concrete initial state and must predict or plan the outcomes of actions in physically consistent way. While T2V remains more challenging task reskirigg the model to imagine the entire physical process from scratch based solely on abstract text text-the I2V task provides complementary and practically important setting to study and improve physical consistency in video generation. We aim to extend WISA to I2V future work, leveraging its modular design to generate videos that maintain accurate physical dynamics starting from known initial state.
22

---

A.3 Training and Evaluation Detail
Training: We choose two representative open-source T2V models CogVideoX-5B and Wan2.1 14B-as the base models to validate the effectiveness of the proposed WISA. WISA is trained on our constructed WISA-80K dataset for 8,000 steps, using a learning rate of 2e-5 and batch size of 16. For CogVideoX-5hBB the video resolution is set to 480x720 with 49 frames per video, while for Wan2.1-14B, the resolution is 480x832 with 81 frames. We adopt LoRA with a rank of 128 and an alpha of 16. During training, only the physical module, physical classifier, and LoRA parameters updated, resulting in a total of 187 million learnable parameters for Cog VideoX and 587 million for Wan2.1-14B. All experiments are conducted on 8 A100 GPUs, each equipped with 80 GB of memory.
Evaluation: VideoCon-Physies was trained by collecting videos generated from nine different models, which were manually annotated for adherence to real-world physical laws and strong semantic consistency. Using this data, a vision-language model (VLM) was fine-tuned to serve as reward model. During inference, the generated video and corresponding text prompt are fed into SileoCon-Physicee which outputs scores ranging from 0 to I for both metrics.
A.4 Inference without Annotation
For any given user prompt, we use large language model (GPT-40) with a set of predefined instructions to generate the required physical annotations (textual description, qualitative categories, quantitative properties). Crucially, this process uses only the input text prompt, with access to any visual information, thus preventing any unfair information leakage. The cost is also minimal (approx. 2000 tokens per prompt). The detailed instructions are provided in the Figure [2] Figure 13 and Figure
14]
A.5
The Definition of Physical Categories
We define total of 29 qualitative physical categories, organized into 5 major classes. The physical categories within each class, along with their corresponding category IDs, are listed as follows:
Dynamics: I. Collision, 2. Rigid Body Motion, 3. Elastic Motion, 4. Liquid Motion, 5. Gas Motion, Deformation, and No obvious dynamic phenomenon
Thermodynamies: 8. Melting, 9. Solidification. 10. Vaporization, 11. Liquefaction, 12. Explosion, 13. Combustion and 14. No obvious thermodynamic phenomenon
Optics: 15. Reflection, 16. Refraction, 17. Scattering, 18. Interference and Diffraction, 19. Unnatural Light Sources, and 20. No obvious optical phenomenon
Camera motion: 21, Yes, 22. No
The state of object: 23. Liquids Objects Appearance, 24. Solid Objects Appearance, 25. Gas Objects Appearance, 26. Object decomposition and splitting, 27. Mixing of Multiple Objects, 28. Object Disappearance and 29, No Change
Specifically, Liquids objects appearance: to external forces, such as water squeezed out of a towel. Solid objects appearance: new solids appear from the camera over time and due external forces, such Chemical reaction that produces precipitates, or cars drive in from outside the camera. Gas objects appearance: new gas appears from the camera over time and due external forces. Object decomposition and splitting: Over time and under the action of external forces, an object is broken into multiple sub-parts: such as fruits and vegetables being cut in half. Mixing of multiple objects: Over time and with the action of external forces, two objects of the same state mix together, such as two solutions mixing. Object disappearance: As time passes and external forces act, objects disappear from the camera. No change: No change in the state of the object
liquids appear from the camera time and due
A.6 Dataset Property Analysis
We visualize the distribution of different physics categories and video frame counts in WISA- 80K, as shown in the paper Figure Dynamics frequently occur in daily life, accounting for the
23

---

Consistency
Inconsistency
Optic
77%
23%
Thermodnnamic
64%
36%
Dynamic
82%
18%
20 Figure 10: Accuracy of qualitative physical category annotations.
60
80
100
largest proportion at 47%. Optics and thermodynamics, which typically require specific temperature or environmental conditions, account for 29% and 24%, respectively. The proportions of each subcategory are shown in the outer ring of the Figure Based on the labels of the manually collected videos, we evaluate the accuracy of the qualitative physical category annotations. The results are shown in the Figure [0] where the accuracy for dynamics, optics, and thermodynamies reaches 84%, 71%. and 64%, respectively, with overall accuracy of 75%.
A.7 More Examples and Annotation
Following the proposed physical information annotation pipeline, we construct the WISA-80K dataset. Several example videos and their corresponding annotations are shown in the Figure IT This pipeline enables accurate and detailed annotation of physical information, ensuring that each video comprehensively labeled with its relevant physical properties and phenomena.
A.8
Annotation Prompts
The detailed prompt used for physical information annotation is illustrated in the Figure 12 Figure and Figure [4]
13
A.9
Word Cloud
We conducted word frequency analysis the textual physical description in the dataset and generated the word cloud shown in Figure [5]] To filter out irrelevant words, we retained only nouns and selected them based their frequency, from highest to lowest. Notably, physical terms such as "motion," "phenomenon," and 'light' appear more frequently, highlighting the strong physical relevance of the dataset.
A.10 Discussion of Quantitative Evaluation
During the quantitative evaluation, we observe several misjudgments VideoCon- VideoCon-Physics, as shown in the Figure 16 Specifically, WISA generates physically plausible process where the object enters the water first, Followed by the splash, aligning well with real-world physical laws. However, this sample only receives low score of 0.08 from SaleoCon-Physice We further conduct simple test using Qwen2.5-Vi for evaluation, and the model also struggles to distinguish the correct or incorrect sequence of physical events. These findings show the limitations of existing video-based physics evaluation metrics, indicating that future research into more reliable physical property assessments for videos is necessary.
24

---

Captions The video depicts close-up view mechaniccl operation. The device appears to be a type puess or hydradic machine, characterieed by its motaliic components and the presence of yollow and wamiig label, indicating
forest showing path winding through a durse area of trees. The trees are tall and green, indicating healthyy fortst environment I The path is flankod by the trunks of these traes, the gromnd covered with layer of fallen sky is not vistbble suggesting that the camera is focused on the ground Jevel. As the vidou progressas, them a sudden and dramatic change in the scume, large explossion occurs, sending billows upwartss obscuring the forest and the path. The explosion creates a hight flash of light which is visible through the smake. The force of the ex explosion camen, cousing vihmte slightly. explosion the focal point the video, and It dominates the soens. The smoke and debris the visible elements.
Captlions: The video begias with
top. The cylinder caution danget which is typical for industrial machinery. The machine is in the process comperssing or forming motal component, as widenced by of devios, which decosnding everting pressomm the lower part. lower part machine consists a direalar base and a cylindiccal component: that being compressed The base has handle atlached to it which likey used for manual operation adjustment.
Nack striped cylincer
small plants, The
the movement of the upper
cloud smoke and debris into
air. smokn
intersee that
components show signs wear use, visible scratches and marks, suggesting that this is a well-used plece of equipment The backgro and blured, locusing allentin the machine, but appears industrial sotting possibly workshup
appears
factory 'Retuual physical descrigtion: when generating a video of the descrihed mechamical devios, it crucial consider the priccple of force and pressum particularly hydraulic systems operate
Textual physkcll description: when guncrating
of explusion forest scene, It's cnucial to consider the princppees of conservation of momentum and enenyy well as the behavlorof gases and smoke in response ropid changes pressure the explosion should realistically demonstrate how the shockwaae propagates thoough the siz causing nearby objocts to noset (eg. trass swaying or dehris being dispiaced), and how the smoke and expands due the hot geses produced, following the laws fuid cynamics.
on pascal's princpple which states that pressure applied conflzed fuid is transmilted undiministed in all directions, addidonally, athention should be paid to the physical phenomenon deformation, the component being compressed will ex expecience changes in shape and possibly yleld stress, reflecting the material's under load.
Qualliative phgiiatieeegsissscceeriees
Qualltativ physies categaries
Dynamics: collision. gas mobion deformation. "Thrmadysamics explesion,
Dynomirs: collision, diformation,
Themmodynaiics no obvious themodysaaic phenomenoe,
Optics: scattering, untotural light source,
Optics: no obvious optical plecomesos,
Camera metion yes,
Camera
no,
The state of objects gas objects appeamace object decompusition
The state of olject: 1. no change 2. object decomposttin and splitting (if the metal component is being out brokes) 3. sclid objects appearazce (Lf
splitting 3. object chsapearranee
now motal componenis are formed during the process),
Qusnntitative plysical progertiess
Denenty: smoken 0.001 to 0.01 gicm' debris: 1 to 2.5 gimml
Quamiiiaatie quantitatihgggpsssla propreties
Time pecurnpids after the explosion. the main physical phenomens, of smoke and debrts, would typically take place within very short time trame, tstabased on destription, the explocion itself and the immediato effects would likely occur within: to 5 seconds.
Denstty: mechanical device: 8.0 giom* grindhtcal component: 7.5
inclading the explosion and the subsequent
to 8.0 gicm?
Time: tu 5 sccords,
Temperature: degrees celstus,
Tempecature 500 1000 degrees celsius,
Figure 11: The video data and its detailed annotations in WISA-80K.
25

---

"model": "GPT-4o-mim"
"role": "system"
"role": "user"
Textual physical description
content: "You are physics expert. Now you want to help the generative model generate videos based on descriptions. You need to provide some physics knowledge to make generated videos more. consistent with the lows of physics. What is the most importont physical principle and the special physical phenomena to be aware of when generating video based on the following sentence? Please explain briefly with one or two sentence."
content: caption
content: "You are a physicist with expertise in material properties and fluid mechanics. Your task is to assist generative model in creating videos that align with the laws of physics based on text descriptions. Carefully analyze the text object described, Express the density value range for each object the unit g/em' Ensure that each object is listed only once, with its correspending density value or ronge. If no density information can be estimated for an object, omit it from the output. Present the results in the format: Object: XXX to XXX (for range) or Object: XXX g/cm* (for single value), separated by newlines for each object."
provide single estimated density value or range
moving
Quantitative Physical Properties
g/ cm
content: caption
content: "You are physicist with expertise dynamics and time-related physical processes. Your task to ossist generative model creating videos that align with the lows of physics bosed text descriptions. Carefully analyze the text and estimate the time range during which the main physical phenomena occur. Only output the time range in the format XXX to XXX seconds."
content: caption
content: "You physicist with expertise thermodynamics and remleee phenomeno. Your task is to assist generative model in creating videos that align with the laws of physics based text 0O
descriptions. Carefully analyze the text and estimate the temperature range during which the main physical phenomena occur, expressed degrees Gelsius (e.g.. '100 200'). Only output the temperature range the format XXX to XXX degrees Gelsius."
content: caption
Figure 12: Prompts for annotating textual physical descriptions and quantitative physical properties
26

---

"model": GPT--o-miin mini"
0o
"role": "system"
"role": "user"
content: "You are a physicist with expertise classical and modern dynamics. Carefully analyze the following text and determine which of the following dynamic phenomena are most likely represented described scene. Please
the principles of motion, force interactions, and material options from the list below and provide your answer by outputting only the corresponding names, seporated by commas (if multiple apply):) Collision (e.g., objects impacting and exchanging momentum)
your judgment
behavior. Select one
Rigid body motion (e.g.. rotation translation without deformation)
Elastic motion (e.g., ascillations, vibrations, stretching and compressing of elastic materials) Liquid motion flow flow of liquids or interactions with liquids)
Gas motion (e.g., expansion, compression, or flow of gases)
Deformation (e.g., structural deformation under applied pressure)
obvious dynamic phenomenon."
content: caption
Qualitative physics categories
content: "You are a physicist with expertise thermodynamics. Your task to carefully analyze the following text
determine which thermodynamic phenomena are most likely represented described scene. Use the principles of energy transfer, phose transitions, and heat-related processes guide your judgment. Below are detailed explanations of the thermodyramic phenomena to help you make an accurate assessment:)
Melting: solid turns into liquid due to heat being absorbed. Exomples include ice melting into water or metal melting furnace. Solidification: A liquid turns into solid heat is removed. Exomples include water freezing into ice molten metal solidifying when cooled.
Vaporization: A liquid transforms into gos through boiling or evaporation. Examples include water boiling steam alcohol evaporating into vapor. Liquefaction: A gas transforms vapor condensing into liquid or liquefied natural gas formation.
liquid due cooling pressure increase. Exomples include water
Explosion: A sudden and rapid release of energy causes violent expansion. Examples include chemical explosions, gas detonations, or bursting pressurized containers.
Combustion: An exothermic chemical reaction, typically involving fuel and exygen, resulting heat light. Exomples include burning wood, gasoline, natural gas.
No obvious thermodynomic phenomenon: The described scene does not exhibit distinct thermodynamic process.
Carefully evaluate the text and select one or more phenomena from the above list based on the scene's description. Provide your answer by outputting only the corresponding names (e.g. "Melting, Combustion")
separated by commas if multiple phenomena apply. If none apply. output "No obvious thermodynamic phenomenon."
content: caption
Figure Prompts for annotating qualitative physics categories
27

---

"model": '6PT-44-mmin
"role": "system"
"role": "user"
You are physicist with expertise in optics. Your task is to analyze the following text carefully and determine which of the listed optical phenomena most likely represented based solely the clear and explicit descriptions provided in the text. Base your judgment strictly on principles of light behavior, wave properties, and interactions with matter. Avoid making assumptions about phenomena not clearly implied or described. Select one or more options from the list below and provide your answer by outputting only corresponding names, seporated by commas (if multiple apply):
content:
Reflection (e.g., light bouncing off surface)
Refraction (e.g., light bending as it passes through medium)
Scattering (e.g., light dispersed in various directions, such as in fag smoke)
Interference and diffraction (e.g., light waves overlapping, forming patterns or bending around obstacles)
Unnatural light source (e.g., artificial or unexpected light emissions, such as lasers or LEDs)
obvious optical phenomenon.
content: caption
cinematographer and physics expert. Your task is analyze the provided text description and determine whether the camera is in motion within the scene. Consider factors such panning, tilting, tracking, or any other type of comera movement. If the text implies camera motion, output "Yes' If the text indicates stationary camera or lacks information about camera motion, output 'No" Only output 'Yes' or 'No'"
content:
"You
content: caption
"You are ordinary residents. Please provide the possible phenomena's 's name in the following 7 types (i.e., liquids objects oppearance: solid objects appearance: object decomposition and splitting: mixing of multiple objects, object disappearance, no change) without any explanation.
content:
The types of phenomena:>
liquids objects appearance: new liquids appear from the camera over time due to external forces, such as squeezed out towel.
solid objects appearance:
solids oppear from the comera as Chemical reaction produces precipitate car drives in from outside the camera.
time and due external forces, such
Gas objects appearance: new gas appear from the comera over time and due to external forces.
Object decomposition and splitting: Over time and under the action of external forces, an object is broken into multiple sub uub-ports: such as fruits and vegetables being cut in half.
Mixing multiple objects: Over time and
the action external forces, two objects of the some solutions mixing.
state mix together, such
Object disappearance: As time passes and external forces act, objects disappear from the comera. No change: No change in the state of the object."
content: Which of the above phenomena are most likely to occur in the text description: (caption)
Figure 14: Prompts for annotating qualitative physics categories
28

---

mostior
sell
poessot
aree ane
erbiion
datorters
came sols
coen vieue
domittun
prtertte doccret
dinciies
forus
sececce de
aftereen
lanas offe
dlartyy
trie
lighting
appearance
resistance
reaction
tension
ormation
hands
impact
bind
sunset
material
visibiity
effect
liquid
dtraction
ilumination
inertia
propagation
materi
riats
manties
are
changes
pecception
troang
in
alscaity
won amn foom
sase
Intersiyy
physics
transfer
clous
reflectio
re
friction
eflection
in
surfaces
Surface
mirror
forces
sky
of
scenario
scce
angle
air
refrace
e
eno
lew
Colo
flames
com
cope
glass
bustion
begatnes
cnne
mutipaatton
coritnny of witlinn saae wature
bollng Nomba
mass porspactive
irppic shape
grutueen mued conuiee
priton suils momerra
pressure
aoauity
scattérine Battering
convection internctions
patterns
uP tack Shy &
waves
calt
gases
reflections
currents
rotas
movement
comorvation dancer dimersio,
on
concept
metil
angies
eavetngttis
P
phase
h
sunlight
ce
shadows
mist
dec
force
aee
momentum
heat
depth
fire
object
highlights
light
buoyancy
effects
som
gravity
h
sccoleration
color
cou
C
realism
subbles
demuy
incidence
from
interaction
direction
position
diffusion
speee
objects
smoke
bubble
ripples
relane
flidd
strutture
ture
conditions
flow
inteylay
bebages
Soatter
anoppien
Necegmend
stlare
expain
disutinnes
was
mett unstnd
reeciin
intodialo
pary
mpteig
gekaring
Lake
pant
transition
sundis interforance
on gast, greatonns meclum
netures
reenzs lolage
scation
height preductios
ccouisnce
tecrctes bowt coeeosion indescent
compasents
parformuce strom
alllwn explasion
condersation %
Figure 15: Word cloud generated from textual physical description, where larger words indicate higher frequencies in the dataset text
Viokating the laws phyyaes
0.05 -0% Violating the lines phasios
Consistan physiol
11778 1- Conststart plysical
Oagh
phyeial lawe
0.00 Viclatnn plysesd
RK
physiml
Violeting plasies
WIS4
Figure 16: Human and machine evaluation results do not fully align.
29


Reviews:
[{"review_id": "XMqJs5XfRg", "reviewer": "Reviewer_jiPQ", "summary": "This paper proposes WISA, a framework that injects explicit physics knowledge into text-to-video diffusion models. WISA encodes physics as structured text, qualitative categories, and quantitative attributes, and is trained on WISA-80K, an 80 K-video dataset rich in physical phenomena. Two key modules drive the approach:\n1. Mixture-of-Physical-Experts Attention (MoPA), where each attention head specializes in a distinct physical phenomenon;\n2. A multi-label physical classifier that supervises the model with BCE loss to focus on relevant attributes.\nOn the VideoPhy and PhyGenBench benchmarks, WISA achieves high scores for both physical and semantic consistency, incurring only ~5 % extra inference latency. Ablations confirm each component’s contribution.", "questions": "1. Does performance drop noticeably on unseen physical categories?\n2. Should provide representative failure cases.\n3. What is the model’s classification accuracy on physical categories during evaluation?\n4. Is there an easy way to extend the pretrained WISA model when new physical categories are introduced?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. Clear problem focus & motivation, and physics representation.  Tackles the well-known physics-violation issue in T2V models with an intuitive three-level representation (textual, categorical, numerical cues).\n2. Propose an open-source new dataset. Releases a well-annotated dataset that will benefit the community.\n3. Achieves strong scores on VideoPhy and PhyGenBench; ablation studies and human evaluation reinforce the gains.\n4. Provides visualization results. Qualitative examples and attention maps make the improvements easy to see and interpret.\n\nWeaknesses:\n1. Scalability: Adding a new physical category requires (i) manually defining the label, (ii) collecting extra data, and (iii) re-training the MoPA heads. This pipeline limits WISA’s ability to scale to larger, more diverse physics datasets.\n2. How does WISA behave on unseen physical categories? Should provide some failure cases. They would be beneficial for a more comprehensive analysis.\n3. MoPA is inserted only in the final layer; does this leave early representations under-constrained? If MoPA modules are sparsely inserted into earlier layers, does performance improve?\n4. What is the model’s classification accuracy on physical categories during evaluation?\n5. Missing some important references [1][2][3][4].\n\nDatasets:\n\n[1] ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation, NeurIPS 2024.\n\nMethods:\n\n[2]  PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generation, ECCV 2024\n\n[3] MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators, TPAMI 2025\n\n[4] Synthetic Video Enhances Physical Fidelity in Video Synthesis, Arxiv 2025", "was_revised": true, "final_justification": "The rebuttal addressed most my concerns, and I will maintain my score. \n\nThe authors further clarified WISA’s modular MoPA design and its ability to extend to new physical categories, provided experiments showing that earlier MoPA insertion yields minimal gains with high cost, and reported classification accuracy. They also committed to presenting additional failure cases on unseen categories and a discussion of the scalable extension process. While acquiring sufficient video data remains a key challenge, I consider this work a good starting point.", "rebuttal": "Dear Reviewer jiPQ,\n\nThank you very much for your valuable and insightful comments. We sincerely appreciate your recognition of our paper’s **clear motivation**, **the value of our carefully annotated dataset**, **the suprior performance of our approach**, and **the strength of our visual demonstrations**.\nBelow, we address each of your comments (including identified weaknesses and questions) in detail.\n\n---\n\n**W1 and Q4:** **Introduing New Physical Categories** \n> W1: Scalability: Adding a new physical category requires (i) manually defining the label, (ii) collecting extra data, and (iii) re-training the MoPA heads. This pipeline limits WISA’s ability to scale to larger, more diverse physics datasets. Q4: Is there an easy way to extend the pretrained WISA model when new physical categories are introduced?)\n\n\n**A1:** Thank you for raising this important point about scalability. This is a crucial aspect for any foundational framework.\n\nWe respectfully argue that WISA's modular design, particularly the Mixture-of-Physical-Experts Attention (MoPA), makes it more scalable than monolithic models. When introducing a new physical category, traditional fine-tuning (e.g., via LoRA on a base model) would require retraining the entire network to incorporate the new knowledge. In contrast, our MoPA architecture allows us to simply add and train a new expert head dedicated to the new phenomenon, while keeping the existing expert heads largely frozen. This is conceptually more efficient and targeted.\n\nFurthermore, as you correctly noted, the cost of generating annotations for new categories is low (approx. 2000 tokens per instance) thanks to our LLM-based pipeline. This combination of targeted modular retraining and low-cost data annotation provides a practical and scalable path forward.\n\nWe will add a discussion in the revised manuscript (e.g., in the Limitations or Future Work section) to explicitly detail this scalable extension process and highlight the advantages of our modular design.\n\n---\n\n\n**W2, Q1 and Q2:** **Failure Cases in Unseen Physical Categories** \n> W2: How does WISA behave on unseen physical categories? Should provide some failure cases. They would be beneficial for a more comprehensive analysis. Q1: Does performance drop noticeably on unseen physical categories? Q2: Should provide representative failure cases.\n\n**A2:** This is an excellent suggestion. A comprehensive analysis must include failure cases. We performed additional qualitative evaluations on unseen physical categories like corrosion and electromagnetism.\n\nAs expected, both our WISA-enhanced model and the base model struggle to generate physically plausible videos for these categories. This is primarily due to the lack of relevant concepts and visual examples in the training data (WISA-80K), highlighting the current limitations of generalization to entirely out-of-distribution physical phenomena.\n\nTo provide a more comprehensive analysis, we will add a new subsection in the appendix discussing these limitations and, critically, we will include representative visual failure cases for these unseen categories. This will offer readers a clearer understanding of our method's current boundaries.\n\n---\n\n\n**W3:** **Sparsely Inserted MoPA** \n> W3: MoPA is inserted only in the final layer; does this leave early representations under-constrained? If MoPA modules are sparsely inserted into earlier layers, does performance improve?\n\n**A3:** Thank you for this insightful question regarding the placement of MoPA. We did in fact explore this alternative design during our research.\n\nWe experimented with a configuration where six physical modules were sparsely inserted throughout the model (every 7 layers). As shown in the table below, this design yielded only marginal gains in performance (SA: 0.623 vs. 0.621; PC: 0.452 vs. 0.450) while incurring a significant computational cost: an additional 155M parameters and a ~40s increase in inference time.\n\n| Insert Manner | SA | PC | Params | Inference time|\n| ------- | ------- |------- |------- |------- |\n|   WISA (Sparsely gap = 7)     |    0.623     |    0.452    | 342M  | ~260s|\n|    WISA (Final Layer)     |     0.621    |     0.450   |  187M | ~220s |\n\nAfter carefully weighing this performance-efficiency trade-off, we concluded that inserting a single MoPA module at the final layer offers the best balance, achieving substantial improvements over the baseline with minimal computational overhead.\n\n\n---\n\n**W4:** *What is the model’s classification accuracy on physical categories during evaluation?*\n\n**A4:** We evaluated CogVideoX on the VideoPhy prompt set by computing the average of the logits over 50 sampling steps. The resulting mean accuracy across the 29 physical categories is 93.70%.\n\n---\n\n**W5:** *Missing some important references [1][2][3][4]*\n\n\n**A5:** Thank you for your suggestion. We will add a discussion of these references in the revised manuscript.\n\n---"}, {"review_id": "W1NB1EQwMA", "reviewer": "Reviewer_7j5D", "summary": "This paper addresses a critical and intriguing topic: physics-aware text-to-video (T2V) generation. The contributions are twofold. First, the authors propose a dataset named WISA-80K, which covers 17 types of physical phenomena. Second, they introduce a new framework designed to better integrate physical laws into T2V models. Experimental results validate the effectiveness of their approach.", "questions": "As stated in the Strengths and Weaknesses, my main concerns are as follows:\n\n1. The need for more rigorous evaluation specifically targeting physical plausibility (related to Weakness 1).\n\n2. A more detailed ablation study comparing the contributions of MoPA and the physics classifier (related to Weakness 3).\n\n3. A clearer and more thorough description of the physics classifier, including implementation details (related to Weakness 3).\n\n4. A comparison or discussion of the proposed physics classifier with DPO-based methods, particularly regarding their effectiveness in guiding models to learn physical reasoning (related to Weakness 3).", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. I appreciate the exploration of physics-aware T2V generation, which is both challenging and important. The analysis of how to integrate physical principles into T2V models, along with the proposed benchmark dataset WISA-80K, is valuable to the community.\n\n2. The proposed attention mechanism, MoPA, is novel. It also appears sound, as I have seen similar (not identical) designs in other works.\n\n3. The effort put into data annotation is commendable and appreciated.\n\n4. The paper is well-written and easy to follow.\n\n\nWeaknesses:\n\n1. The evaluation metrics used focus primarily on text-video consistency and visual quality, but they largely overlook how well the generated videos adhere to physical principles (beyond user studies).\n - Could the authors design specific metrics to assess physical plausibility?\n - A feasible approach might be to evaluate physical correctness in toy scenarios. For example, generate a video where a ball undergoes free fall motion, and measure how well it follows expected physical behavior.\n\n2. It would be beneficial to include video comparison results in the supplementary materials. This would help reviewers and area chairs better assess the effectiveness of the proposed method.\n\n3. How physical plausibility is ensured remains unclear. I have the following concerns and questions:\n - Between MoPA and the physics classifier, which contributes more to enhancing physics awareness? The paper lacks an ablation study on this aspect.\n - The core component—the physics classifier—is intended to guide the T2V model to understand physical information. However, the implementation details are vague, making it difficult for other researchers to reproduce the work.\n - Have you compared the proposed physics classifier with DPO-related methods? Which approach is more effective in guiding the model to understand and follow physical principles?", "was_revised": true, "final_justification": "The clarification on the evaluation metrics (Q1) does not convince me. Therefore, I maintain the score to 'Borderline accept'.", "rebuttal": "**Dear Reviewer 7j5D**,\n\nThank you very much for your valuable and insightful comments, as well as for recognizing **the novelty of our approach**, **the value of our carefully annotated dataset**, and **the quality and reproducibility of our manuscript**.\nBelow, we address and clarify each of your comments (including identified weaknesses and questions) in detail.\n\n**W1 and Q1:** **Evaluation Metrics**\n> W1: The evaluation metrics used focus primarily on text-video consistency and visual quality, but they largely overlook how well the generated videos adhere to physical principles (beyond user studies). \n> Could the authors design specific metrics to assess physical plausibility?\n> A feasible approach might be to evaluate physical correctness in toy scenarios. For example, generate a video where a ball undergoes free fall motion, and measure how well it follows expected physical behavior.                   \n> Q1: The need for more rigorous evaluation specifically targeting physical plausibility (related to Weakness 1).\n  \n**A1:** Thank you for this crucial point on evaluation. We agree that robustly and quantitatively measuring physical plausibility is a major challenge for the field.\n\nAs you noted, we used the established Physical Consistency (PC) metric from VideoPhy, which is a VLM-based evaluator specifically fine-tuned to assess adherence to physical laws. Our method shows significant gains on this metric (Table 1).\n\nWe find your suggestion of using toy scenarios (e.g., a free-falling ball) for precise dynamic analysis very insightful. This is a standard approach in physics-based animation. However, applying it to general-purpose text-to-video models presents significant challenges. The generated videos often have unknown or variable factors such as camera distance/angle, frame rate, and 2D projection effects, which make it difficult to reliably extract and verify kinematic properties (e.g., acceleration g). Moreover, many physical categories we study, such as melting, combustion, or optical phenomena (interference), cannot be easily evaluated with rigid-body dynamics models.\n\nGiven these challenges, the current state-of-the-art for evaluating physical commonsense in general T2V relies on VLM-based evaluators (like VideoPhy, PhyGenBench) or large-scale human studies. While we acknowledge their limitations, our work adheres to the current best practices in the field. Developing more robust, general-purpose physics evaluation metrics is an important open research problem.\n\n---\n\n**W2:** **Video Comparison**\n> It would be beneficial to include video comparison results in the supplementary materials. This would help reviewers and area chairs better assess the effectiveness of the proposed method.\n\n**A2:** Thank you for your suggestion. In the PDF abstract, we have provided a link to an anonymous repository that includes comparisons with other methods, dataset video examples, and more. We are also willing to further supplement these materials in the supplementary section.\n\n\n---\n\nThank you for these detailed questions about our core components. Let us clarify the roles of and relationship between MoPA and the Physical Classifier.\n\n**W3.1 and Q2:** **MoPA and the Physics Classifier**\n> W3.1: Between MoPA and the physics classifier, which contributes more to enhancing physics awareness? The paper lacks an ablation study on this aspect.  \n> Q2: A more detailed ablation study comparing the contributions of MoPA and the physics classifier (related to Weakness 3).\n\n**A3.1** The two components have a synergistic and progressive relationship. The Physical Classifier's role is to provide a direct supervisory signal during training. This signal guides the model to learn abstract physical categories. The MoPA module then leverages these learned categories to activate specific 'expert' attention heads, enabling it to model the distinct features of each phenomenon. The classifier enables MoPA's specialization. As our ablation study shows (Table 2, w/o PC), removing the classifier's supervision leads to a noticeable performance drop, confirming its crucial guiding role.\n\n---\n\n**W3.2 and Q3:** **Expanded Description of the Physics Classifier**\n> W3.2: The core component—the physics classifier—is intended to guide the T2V model to understand physical information. However, the implementation details are vague, making it difficult for other researchers to reproduce the work.  \n> Q3: A clearer and more thorough description of the physics classifier, including implementation details (related to Weakness 3).\n\n**A3.2:** We apologize for any confusion caused by the unclear description of the Physical Consistency (PC) component. We provide a more detailed explanation as follows:\n\nWe initialize a learnable token, referred to as the physical token, which is concatenated with the noise tokens and text tokens as input to the MM-DiT and MoPA modules. The physical token serves as a query to aggregate physical features. Just before the output projection layer, we extract the physical token and feed it into a multi-layer perceptron (MLP) classifier that outputs logits corresponding to the 29 quantitative physical categories. This classifier is supervised with a multi-label binary cross-entropy loss.\n\nWe will revise the manuscript accordingly to clarify the description of PC and improve reproducibility.\n\n---\n\n**W3.3 and Q4:** **Physics Classifier and DPO**\n> W3.3: Have you compared the proposed physics classifier with DPO-related methods? Which approach is more effective in guiding the model to understand and follow physical principles?  \n> Q4: A comparison or discussion of the proposed physics classifier with DPO-based methods, particularly regarding their effectiveness in guiding models to learn physical reasoning (related to Weakness 3).\n\n**A3.3:** This is a great question. Our Physical Classifier and DPO are complementary, not conflicting, methodologies. Our approach operates during the Supervised Fine-Tuning (SFT) stage, using explicit labels from WISA-80K. DPO, on the other hand, is a post-training alignment technique that relies on preference data. A promising future direction would be to first train WISA with our SFT approach and then use DPO with human preference data on physical plausibility to further refine the model.\n\nWe will add a discussion in the paper to clarify this distinction and position DPO as a potential future work.\n\n---"}, {"review_id": "2FL5KUUVDZ", "reviewer": "Reviewer_kzVn", "summary": "Current text-to-video models cannot produce videos that follow physical laws faithfully, degrading the realism significantly. Therefore, the paper proposes to curate high-quality video dataset, WISA-80k, which collects 80k video clips ranging from 17 different physical phenomena. After captioning with Qwen2.5-VL, three aspects of physical knowledge are decomposed with GPT-4o, including textual physical description, qualitative physical categories, and quantitative physical properties. To incorporate the physical knowledge into text-to-video generation process, the paper proposes a physical module, leveraging mixture-of-experts self-attention mechanism to handle different physical laws. Textual physical description and quantitative physical properties are encoded and integrated in the model input. Qualitative physical classifier predicts the physical category and is jointly trained with diffusion loss. The experiments show that WISA outperforms baseline methods (Tab. 1, Fig. 7), and ablation study is performed in Tab. 2.", "questions": "- How does the proposed method handle input prompts without detailed annotation?\n- How could the model and curated dataset be used for image-to-video generation?\n- How to compute physical consistency, semantic coherence?\n- Please include and discuss \"Physics-IQ: Benchmarking physical understanding in generative video models (2025)\"", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- The paper curates a high-quality video dataset, WISA-80k, covering 17 different physical phenomena across three major categories: dynamic, thermaldynamic, and optic. The dataset provides detailed annotations, including textual physical description, qualitative physical categories, and quantitative physical properties. This dataset could facilitate future research in video generation models and physical simulation. \n- The physical phenomena are decomposed into three aspects and integrated into the video generation pipeline. Specifically, a mixture-of-experts multi-head self-attention mechanism is used in the physical module, attending to different physical effects with a specific head, enhancing the physical plausibility of generated videos. The ablation study in Tab. 2 validates the design choices. \n\n## Weaknesses \n- It is unclear how WISA model handle input prompts without detailed annotation, such as textual physical description, qualitative physical categories, and quantitative physical properties. According to the training pipeline, this information is required and integrated into the framework, but in-the-wild prompt data do not have such detailed annotation. The paper should provide more details about how the model obtains such information for inference. Otherwise, the comparison with baseline methods might be unfair. \n- The proposed method is only used for text-to-video generation, and the paper does not discuss or evaluate the model's capability for image-to-video generation, which is possible to evaluate with the curated dataset.\n- Details are missing. For example, the details of computing physical consistency, semantic coherence are not included.\n- Missing discussion with important prior works. \"Physics-IQ: Benchmarking physical understanding in generative video models (2025)\" is another benchmark for analyzing physical plausibility of video generation models, but it is not discussed in the paper.", "was_revised": true, "final_justification": "The authors' response appropriately addresses my concerns in the initial review. As a result, I would like to raise my final score to \"Accept\". I also encourage the authors to integrate the response into the final version, including:\n\n- The inference strategy with input prompt without additional information or annotation\n- Discussion of image-to-video generation in the limitation and future work section\n- Details of metric computation, including physical consistency and semantic coherence\n- Discussion of Physics-IQ in the related works section", "rebuttal": "**Dear Reviewer  kzVn**,\n\nThank you very much for your valuable and insightful comments, as well as for recognizing **the novelty of our approach** and **the value of our carefully annotated dataset**.\nBelow, we address and clarify each of your comments (including identified weaknesses and questions) in detail.\n\n---\n\n**W1 and Q1: Handling Prompts Without Annotations**\n> W1: It is unclear how WISA model handle input prompts without detailed annotation, such as textual physical description, qualitative physical categories, and quantitative physical properties. According to the training pipeline, this information is required and integrated into the framework, but in-the-wild prompt data do not have such detailed annotation. The paper should provide more details about how the model obtains such information for inference. Otherwise, the comparison with baseline methods might be unfair.       \n> Q1: How does the proposed method handle input prompts without detailed annotation?\n\n\n\n**A1:** Thank you for this critical question regarding the handling of in-the-wild prompts. This is essential for the practical application and fair evaluation of our method.\n\n**Inference-time Annotation**: For any given user prompt, we use a large language model (GPT-4o) with a set of predefined instructions to generate the required physical annotations (textual description, qualitative categories, quantitative properties). Crucially, this process uses only the input text prompt, with no access to any visual information, thus preventing any unfair information leakage. The cost is also minimal (approx. 2000 tokens per prompt). The detailed instructions are provided in the appendix (Figures 11-13).\n\n**Fairness and Effectiveness of WISA**: To prove that WISA's performance gain comes from our structured information injection framework rather than just having more textual information, we conducted a new experiment. We concatenated all the generated physical annotations directly into the text prompt and fed it to the original CogVideoX-5B base model (without our WISA modules). The results on VideoPhy are shown below:\n\n| Setting| SA | PC |\n| ------- | ------- |------- |\n|  CogVideoX-5B + All annotations as text  |    0.56     |    0.41    |\n|  CogVideoX-5B (baseline)   |    0.57     |    0.41    |\n\nAs the results show, simply adding the physical information as text did not improve performance; it slightly hurt semantic consistency. This strongly suggests that the base model cannot effectively utilize this information without the specialized modules and training provided by WISA. Therefore, WISA's superiority stems from its core design, not from an unfair advantage.\n\n\n---\n\n**W2 and Q2:** **T2V or I2V Task**\n> W2: The proposed method is only used for text-to-video generation, and the paper does not discuss or evaluate the model's capability for image-to-video generation, which is possible to evaluate with the curated dataset.   \nQ2: How could the model and curated dataset be used for image-to-video generation?\n\n**A2:** In principle, WISA can also be applied to the I2V task. The main difference between image-to-video (I2V) and text-to-video (T2V) tasks lies in the input: I2V models receive a clean latent code for the first frame, while the rest of the input text and model architecture remain the same. The I2V task has explicit initial frame information and state, the model primarily learns to infer subsequent possible processes, generating intermediate and final states of physical phenomena. The T2V task is more challenging and closer to the vision of a true 'world simulator', as it requires the model to imagine the entire physical process from scratch—including the initial, intermediate, and final states—based solely on an abstract description. Consequently, we focus on building WISA for the T2V task, which poses a greater challenge to the model’s physical perception capabilities.\n\n---\n\n**W3 and Q3** **Physical Consistency and Semantic Coherence**\n> W3: Details are missing. For example, the details of computing physical consistency, semantic coherence are not included.     \nQ3: How to compute physical consistency, semantic coherence?\n\n**A3:** We adopt the automatic evaluation approach VideoCon-Physics proposed in VideoPhy to compute the Physical Consistency (PC) and Semantic Accuracy (SA) metrics. VideoCon-Physics was trained by collecting videos generated from nine different models, which were manually annotated for adherence to real-world physical laws and strong semantic consistency. Using this data, a vision-language model (VLM) was fine-tuned to serve as a reward model. During inference, the generated video and corresponding text prompt are fed into VideoCon-Physics, which outputs scores  ranging from 0 to 1 for both metrics. We compute the average scores over the entire test set to obtain the PC and SA results reported in the Tables 1 and 2 of manuscript.\n\n---\n\n**W4 and Q4** **Missing Discussion with Physics-IQ**\n>  W4: Missing discussion with important prior works. \"Physics-IQ: Benchmarking physical understanding in generative video models (2025)\" is another benchmark for analyzing physical plausibility of video generation models, but it is not discussed in the paper.  \n> Q4: Please include and discuss \"Physics-IQ: Benchmarking physical understanding in generative video models (2025)\"\n\n**A4:**  We will include a discussion of this benchmark in the Related Works section."}, {"review_id": "UjVctFteHz", "reviewer": "Reviewer_1tcW", "summary": "This paper considers the problem of physics-obeying t2v generation. Three main contributions:\n1. *(Modeling improvement)* the paper proposes \"WISA\", which is a collection of modeling changes (e.g. trainable adapters) that bake physics-obeying inductive biases into pretrained video generative models through fine-tuning\n2. *(Dataset improvement)* the paper proposes a video dataset, \"WISA-80k\", with videos of various physical phenomena with detailed annotations\n3. *(Combining modeling improvement with dataset improvement)* the authors find that training CogVideoX and Wan2.1 on the WISA-80k dataset, using the WISA framework, improves physics-obeying video generation performance", "questions": "1. The paper says (approx line 126) *\"we manually collected videos from the Internet that clearly demonstrate the corresponding behavior, without relying on any existing video datasets for filtering or selection\"*. Where on the internet were the videos gathered from? Can you please provide more details about this data gathering procedure.\n2. As I understand it, WISA includes three different ways of baking physical inductive biases into the model: 1) appending \"textual physical descriptions\" to the end of the text prompt; 2) using a Mixture of Physical Experts Attention head to encode qualitative physical attributes; 3) using AdaLN to encode numerical physical attributes into the model. My first question is: is that understanding correct? And my second question is: to what extent can each of the three of these be used without the other two, and still improve performance? E.g. perhaps you could append the textual physical descriptions, but not do either of 2 or 3, and that would give the same performance. Did you conduct any ablations on these sub-parts? How do we know that we can't achieve the same (or better) performance than the complicated WISA framework just with the test-time LLM-guided prompt-engineering? See line 138: \"we believe that simple video captions are not sufficient.\" This is a major hypothesis, and doesn't appear to be tested. A clear answer to this could increase my evaluation score.\n3. How accurate were the automatically generated annotations (see lines 161-167) for the WISA-80k dataset? Was there any effort to evaluate how accurate they are? (For example, I've seen prior works do things like, randomly sample a small subset of videos, have a user study to measure how accurate each aspect of the labels are, and use that as a proxy for the reliability of the overall dataset). Or is the hypothesis that these are likely noisy, but it doesn't matter, because it still improves performance? A clear answer to this could increase my evaluation score. For example, Appendix A.11 talks about limitations of AI-generated evaluation metrics (which is true). But the labels from WISA-80k are themselves potentially very noisy / AI generated. \n4. It is well-established that Wan 2.1 has better video generation quality than CogVideoX. Does this WISA technique have diminishing returns when applied to a base model which already has better physics? Is this reflected somewhere in the metrics? Additionally, do the authors have any explanation/intuitions for why the Wan2.1 scores aren't substantially higher than the CogVideoX scores?\n5. Why are there missing entries in Table 1?\n6. Why is the PhyT2V in the same section in Table 1 as CogVideoX? Is it built using the same base model? Line 251 implies it's built on top of Tarsier-34B, a different video generation model.\n7. To what extent did cherry picking of results happen before the evaluation pipeline / comparison study? E.g. did you choose the first seed, or the best seed for both models, or the best seed only for the WISA model, etc. Please provide more info about this.\n8. What is the \"open source video dataset\" mentioned in lines 289-290?\n9. Can you please provide more info about the User Preference study (section 5.4)? E.g. how many videos, which videos. \n10. In Section A.5 the paper mentions how many learnable params there are for CogVideo, but not for Wan2.1, how many are there?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The teaser figure is very informative and clear. It makes it very clear what the dataset contribution is.\n2. The motivation in the introduction is written+explained very well and is easy to follow.\n3. In Section 3.2, the paper makes good use of examples to make clear what is being constructed.\n4. The paper considers a very important problem (making video generative models better obey physical laws) and it proposes a new labeled dataset that will likely be useful for the community for building better such models.\n5. The description of section 4.2, the physical model, was very easy to understand.\n6. The WISA framework improves performance for CogVideoX and Wan2.1. \n7. The attention map analysis for the swing is very strong, I love that section and that figure.\n\nWeaknesses:\n1. The naming of things is confusing and unintuitive at times. For example, referring to the proposed framework of modules as a \"Simulator Assistant\" doesn't sound quite right to me, because the proposal involves network modules that must be integrated into the base video model and trained from scratch. Additionally, referring to the dataset as \"WISA-80k\", next to the framework of modules which is also titled \"WISA\", seems misleading; these are two completely different things--one is a new dataset, and other is a neural network. It seems like a non-standard/confusing way of naming things.\n2. I think it's very important to have a discussion of related works in the body of the paper, but the current paper has left all of the related works discussion to the appendix. I might suggest moving some other content to the appendix, moving some related works discussion to the body of the paper... related work is crucial to have in the part of the paper that most people will read\n3. I had a hard time understanding Section 4.3, the Physical Classifier. Can you please explain in more detail A) what is the input to this model at train/inference, B) what is the output to this model at train/inference, and C) what it is \"classifying\". \n4. It is unclear from the experiment design which of the three constituent parts of the WISA model adapters/additions are responsible for the improved performance (see Questions for more details)\n5. In Section 5.2, the qualitative comparison, the paper discusses how the WISA-based model does a better job erasing pencil marks. After watching the video, I disagree; it looks like the pencil marks disappear before the eraser even touches them, and they also disappear all at once. I might suggest highlighting this as a failure case, and removing the screenshots from the paper, because the screenshots and descriptions misleadingly imply that this is a success case. (Also a minor typo--line 265, it says \"CogVideoX-5B fails to generate any pencil marks\", when I think it should say \"fails to erase any pencil marks\"). In contrast, the apple example is excellent, that one should be front and center.\n6. I might suggest referring to the models as \"CogVideoX-5B-WISA\" and \"Wan2.1-WISA\" in the paper. (This is already done in Table 1, which is great but should be done in the text as well, in my opinion). It it very important to know which is the base model. (If the paper only had one base model, it wouldn't matter as much... )\n7. Overall, the paper is vague about some important experimental details (see Questions below)", "was_revised": true, "final_justification": "After engaging with the authors, I have decided to increase my scores: increasing \"quality\" rating from 2 to 3, \"clarity\" rating from 2 to 3, and my overall rating from 3 to 4. This is based mainly on the authors' committing to the following paper additions in the main body of the paper:\n\n* The ablation study on the major components\n* Adding that clarification / rewrite which explains what the classifier is classifying, and how it works at inference / training\n* Evaluation of the quality of the 3 types of AI-generated annotations\n* A discussion on related works\n\nIn particular, that ablation study (which evaluates to what extent each component is critical to the overall success of the method) was the main reason I did not recommend the paper for acceptance at my first rating. But the addition of that study, combined with additional clarifications and experiment details, make this paper's contribution more clear.", "rebuttal": "**Dear Reviewer  1tcW**,\n\nThank you very much for your valuable and insightful comments, and for your praise of our paper’s **clear figures**, **straightforward motivation**, **valuable finely annotated dataset**, **excellent writing and explanations**, as well as **significant performance improvements**.\nBelow, we address and clarify each of your comments (including identified weaknesses and questions) in detail.\n\n---\n**W1:** **Name**\n> W1: The naming of things is confusing and unintuitive at times ....\n\n**A1:** Thank you for this valuable feedback on the naming conventions. We agree that the dual use of \"WISA\" could cause confusion.\n\nOur original intent for the name \"World Simulator Assistant\" was to encapsulate its dual role: the dataset acts as a data assistant, while the framework acts as a model assistant. However, we recognize your point about the potential for ambiguity.\n\nTo improve clarity, we will rename the dataset to WISA-DATA-80K throughout the revised manuscript. This clearly distinguishes the dataset from the WISA framework and resolves the ambiguity.\n\n\n\n---\n**W2:** **Related Works**\n> W2: I think it's very important to have a discussion of related works in the body of the paper ... \n\n**A2:** Thank you for your suggestion. We will adjust the paper format and include the Related Work section in the main manuscript as much as possible.\n\n---\n**W3:** **Physical Classifier**\n> W3: I had a hard time understanding Section 4.3, the Physical Classifier. ...\n\n**A3:** We apologize that our description of the Physical Classifier was unclear. Thank you for giving us the opportunity to clarify. Here is a detailed breakdown following your A/B/C structure:\n\nA) Input:\n- At Training: The input is a learnable embedding vector, which we call the [PHYSICS_TOKEN]. This token is concatenated with the noisy visual tokens and text prompt tokens and is processed by the entire MM-DiT and MoPA architecture.\n- At Inference: The input is identical to training—the same trained [PHYSICS_TOKEN] is used.\n\nB) Output:\n- At Training: After being processed by the network, the final hidden state of the [PHYSICS_TOKEN] is fed into a simple MLP classification head. This head outputs logits for the 29 qualitative physical categories. This output is used only to compute the multi-label binary cross-entropy loss for training.\n- At Inference: The output from the classifier head is completely discarded. It does not influence the video generation process in any way. Its sole purpose is to provide an auxiliary training signal to help the model learn to represent physical concepts.\n\nC) What it is \"classifying\":\n- It is performing a multi-label classification task on the 29 qualitative physical categories that we define (e.g., \"Collision\", \"Melting\", \"No obvious dynamic phenomenon\", etc., as detailed in Appendix A.6). For each video, it predicts which of these 29 phenomena are present.\n\nWe will replace the current text in Section 4.3 with this detailed, structured explanation to ensure clarity and reproducibility.\n\n---\n**W4, W7, and Q2:** **Ablations on Major Components**\n> W4: ...\n> W7: ...\n> Q2: As I understand it, WISA includes three different ways of baking  .... \n\n\n**A4:** Thank you for this critical question. This gets to the heart of our contribution, and we are happy to provide a detailed ablation study to clarify the impact of each component.\n\nFirst, your understanding is perfectly correct. WISA integrates three types of physical information. To test your hypothesis and isolate the contribution of each, we conducted a new set of ablation experiments on CogVideoX-5B using our WISA-DATA-80K.\nThe results are summarized below:\n\n| Setting| SA | PC |\n| ------- | ------- |------- |\n|   Baseline |    0.57     |    0.41    |\n|   only  Textual Physical Descriptions    |    0.58     |    0.43    |\n|   only Qualitative Physics Categories   |    0.60     |    0.44    |\n| only Quantitative physical properties|     0.57    |     0.42   |\n| WISA|     0.62    |     0.45   |\n\nAnalysis: These results clearly demonstrate that:\n1. The Qualitative Physics Categories, guided by our MoPA and Physical Classifier, provide the most significant performance boost in isolation.\n2. Appending Textual Physical Descriptions offers a moderate but clear improvement.\n3. The full WISA framework, integrating all three components, achieves the best performance, indicating a synergistic effect.\n\n\nRegarding your final point on \"test-time LLM-guided prompt-engineering,\" this approach was explored by PhyT2V. As shown in Table 1, while it provides some benefit, its performance is limited and it incurs extremely high inference costs. Our results show that directly integrating these concepts into the model architecture via our proposed modules is a much more effective and efficient approach than relying on iterative prompting.\n\n---\n\n**W5 and W6:** **Minor Typo**\n\n**A5:** Thank you for your suggestion. We will revise the example images accordingly and correct the errors, including the model names in the paper entries such as Table 1.\n\n\n---\n**Q1: Data Sources**\n> The paper says (approx line 126) \"we manually collected videos from the Internet ... \n\n**A6:**\nFirst, we defined 17 categories of physical phenomena. Based on this categorization, annotators were tasked with searching for relevant videos of these physical phenomena from publicly available YouTube channels, without involving any exclusive or private data sources. They also annotated the time segments during which the physical phenomena occur.\n\nFinally, our data collection and release strictly comply with YouTube’s Data Privacy Policy and Fair Use Policy. The released dataset is intended solely for research purposes.\n\n\n---\n\n**Q3:** **Noise in WISA-80K**\n> How accurate were the automatically generated annotations (see lines 161-167) ... \n\n**A7:** This is an excellent and crucial question regarding the reliability of our dataset annotations. To quantify this, we performed a manual evaluation on a randomly sampled subset of 500 videos from our dataset.\n\nOur evaluation focused on the three types of AI-generated annotations:\n1. Textual Physical Descriptions: We measured human rater satisfaction (i.e., does the description accurately reflect the video's physics?). Result: 95% satisfaction rate.\n2. Qualitative Physics Categories: We compared the AI-generated labels against the ground-truth labels assigned during initial video collection. Result: 76% accuracy.\n3. Quantitative Physical Properties: We again measured human rater satisfaction (i.e., are the estimated density/time/temperature values plausible?). Result: 86% satisfaction rate.\n\nWhile some label noise is inherent in any large-scale, automatically annotated dataset, these results demonstrate that the overall quality of WISA-DATA-80K's annotations is high. The data is sufficiently reliable to provide a strong learning signal, as evidenced by the performance gains in our main experiments.\n\n---\n**Q4: Wan2.1 vs. CogVideoX**\n> It is well-established that Wan 2.1 has better video generation ... \n\n**A8:** We agree that if the base model already possesses strong physical understanding capabilities, the gains introduced by WISA may be less pronounced. The reason Wan2.1's scores are not substantially higher than CogVideoX's is that the VideoCon-Physics [1] evaluation model was trained on videos generated by CogVideoX, but not on those generated by Wan2.1. A detailed explanation please refer to the manuscript(lines 255–258).\n\n[1] VideoPhy: Evaluating Physical Commonsense for Video Generation\n\n---\n\n**Q5:** *Why are there missing entries in Table 1?*\n\n**A9:** Thank you for your question. The results for VideoCrafter2, OpenSora, and HunyuanVideo are sourced from the VideoPhy paper. Since these models were evaluated exclusively on the VideoPhy prompt set, inference times and performance on our PhyGenBench benchmark are not available. Additionally, as the VideoPhy paper provides output videos for OpenSora, we were able to evaluate its IS and CLIP-SIM scores accordingly.\n\nThe results for PhyT2V are taken directly from its original paper. As PhyT2V is open-sourced, we measured its inference time ourselves. However, since PhyT2V focuses specifically on physical reasoning tasks, we only compare our method with its SA and PC components.\n\nIf you have further concerns or suggestions regarding this comparison, we would be happy to discuss and incorporate them.\n\n---\n\n**Q6:** *Why is the PhyT2V in the same section in Table 1 as CogVideoX? Is it built using the same base model? Line 251 implies it's built on top of Tarsier-34B, a different video generation model.*\n\n**A10:** PhyT2V is a comprehensive test-time prompt-engineering framework that integrates a diffusion-based video generation model (CogVideoX-5B), a vision-language model for video captioning, and a large language model (Tarsier-34B) for prompt refinement.\nAs all methods in Table 1 adopt CogVideoX-5B as the underlying video generation model, the comparison is fair and consistent.\n\n---\n\n**Q7: Best Seed only for WISA?** \n> To what extent did cherry picking of results ... \n\n**A11:** The test results of WISA do not involve cherry-picking. We used the same random seed (42) as the baseline during evaluation to ensure a fair comparison.\n\n---\n\n**Q8:** *What is the \"open source video dataset\" mentioned in lines 289-290?*\n\n**A12:** The \"open source video dataset\" mentioned in lines 289–290 refers to Koala36M. We will add a clarification about this in the manuscript.\n\n---\n\n**Q9:** *Can you please provide more info about the User Preference study (section 5.4)? E.g. how many videos, which videos.*\n\n**A13:** The manual evaluation was conducted on 343 test samples from the VideoPhy dataset.\n\n---\n\n**Q10:** *In Section A.5 ... but not for Wan2.1, how many are there?*\n\n**A14:** The number of trainable parameters for Wan2.1 is 587 million. We will include this information in the revised manuscript."}]

Author rebuttal:

Meta-review: This paper aims to improve the physical faithfulness of text-to-video generation. To achieve this, the paper collects a new dataset including video clips with textual physical descriptions and qualitative categories. The videos are manually sourced, but the annotations are from LLMs (text descriptions from using Qwen, and physical annotations generated with GPT-4o). Based on the collected dataset, the paper trains a modified text-to-video model by adding physical classifier guidance and a MoE attention module with physics category information. Experiments are conducted on VideoPhy and PhyGenBench benchmarks.

Reviewers commented positively on the meaningful dataset, the method design, and the writing quality. Reviewers mainly queried about the ablation study, the applicability to image-to-video generation, and the evaluation of physical plausibility, and inference time annotation using LLMs, which are partially answered in the rebuttal, and the unaddressed issues are mainly due to the fact that they are still open questions for the whole field.

The main reason for acceptance is the contribution of the new dataset. It will benefit the community on this emerging topic if the data and all the related details and code can be released as promised. 

One potential issue in this paper is that the enhanced physical awareness in videos all relies on the physical knowledge in LLMs, including the annotation in dataset collection and also the test-time annotation. Whether LLMs are faithful to the real physical world is still questionable (without seeing the real physical world). The authors should discuss this to facilitate community discussion. Generalisation to image-to-video is also worth discussing, since it is more usable as a "world simulator" in some robotic cases (contrary to the authors' opinion).
Decision: Accept (spotlight)