DiffWind: Physics-Informed Differentiable Modeling of Wind-Driven Object Dynamics
Authors: Yuanhang Lei
Boming Zhao
Zesong Yang
Xingxuan Li
Tao Cheng
Haocheng Peng
Ru Zhang
yang yang
Siyuan Huang
Yujun Shen
Ruizhen Hu
Hujun Bao
Zhaopeng Cui
Venue: ICLR 2026 Poster (2026)
Keywords: Physics-based Modeling
3D Dynamics
System Identification
Differentiable Physics
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Modeling wind-driven object dynamics from video observations is highly challenging due to the invisibility and spatio–temporal variability of wind, as well as the complex deformations of objects. We present DiffWind, a physics-informed differentiable framework that unifies wind–object interaction modeling, video-based reconstruction, and forward simulation. Specifically, we represent wind as a grid-based physical field and objects as particle systems derived from 3D Gaussian Splatting, with their interaction modeled by the Material Point Method (MPM). To recover wind-driven object dynamics, we introduce a reconstruction framework that jointly optimizes the spatio–temporal wind force field and object motion through differentiable rendering and simulation. To ensure physical validity, we incorporate the Lattice Boltzmann Method (LBM) as a physics-informed constraint, enforcing compliance with fluid dynamics laws. Beyond reconstruction, our method naturally supports forward simulation under novel wind conditions and enable new applications such as wind retargeting. We further introduce WD-Objects, a dataset of synthetic and real-world wind-driven scenes. Extensive experiments demonstrate that our method significantly outperforms prior dynamic scene modeling approaches in both reconstruction accuracy and simulation fidelity, opening a new avenue for video-based wind–object interaction modeling. The project page is available at: [https://zju3dv.github.io/DiffWind/](https://zju3dv.github.io/DiffWind/).

Published as a conference paper at ICLR 2026
MODELING OF PHYSICS-INFORMEN DIFFERENTIABLE
OBJECT DYNAMICS
Yuanhang Lei"", Boming Zhao'", Zesong Yang",, Xingxuan Li', Tao Cheng',
Haocheng Peng'. Ru Zhang', Yang Yang Siyuan Huang?, Yujun Shen',
Ruizhen Hu', Hujun Bao', Zhaopeng Cui'
Equal contribution. i Corresponding author.
State Key Laboratory CAD & CG, Zhejiang University,
2 State Key Laboratory of General Artificial Intelligence, BIGAI
3 Ant Group, "Shenzhen University
ABSTRACT
Modeling wind-driven object dynamics from video observations is highly challeng- ing due to the invisibility and spatio-temporal variability of wind, as well as the complex deformations of objects. We present DiffWind, a physics-informed dif- ferentiable framework that unifies wind-object interaction modeling, video-bate based reconstruction, and forward simulation. Specifically, we represent wind as grid- based physical field and objects as particle systems derived from 3D Gaussian Splatting, with their interaction modeled by the Material Point Method (MPM). To recover wind-driven object dynamics. introduce reconstruction framework that jointly optimizes the spatio- ppatiottempora wind force field and object motion through differentiable rendering and simulation. To ensure physical validity, we incorporate the Lattice Boltzmann Method (LBM) as a physics-informed constraint, enforcing compliance with fluid dynamics laws. Beyond reconstruction, our method natu- rally supports forward simulation under novel wind conditions and enable new applications such as wind retargeting. We further introduce WD-Objects, dataset of synthetic and real-world wind-driven scenes. Extensive experiments demon- strate that our method significantly outperforms prior dynamic scene modeling approaches in both reconstruction accuracy and simulation fidelity, opening new avenue for video-based wind-object interaction modeling. The project page available at: http:///ju.d..gituubboo/Dii
INTRODUCTION
The motion of objects under wind, such as leaves swaying, flags fluttering, or fabrics billowing, is visually distinctive yet physically complex phenomenon, arising from the interplay between external fluid forces and internal material properties. From visual observations alone, humans intuit the presence of invisible wind and anticipate how objects would respond under similar conditions. Enabling computational systems to replicate this ability, which involves recovering both the underlying wind field and the resulting object dynamics from visual input, would have broad impact on applications in augmented and virtual reality, visual effects, scientific analysis, and iimulation-basel editing.
However, this task presents significant challenges: wind is invisible, dynamic, and spatially non- uniform, while object deformations depend on unknown physical parameters such as mass, elasticity, and geometry. Existing methods face significant limitations: Dynamic neural representations, such as Deformable NeRF Du al. (2021); Chu et al. (2022) and 3D Gaussian Splatting Yang et al. (2024b); Wu et al. (2024), model object appearance and motion over time but capture only visible dynamics, ignoring underlying physical causes like wind fields. Differentiable physics simulators Zhong et al. (2024); Li et al. (2023b); Zhang et al. (2024) can optimize motion parameters, but are restricted to simple, predefined motion patterns such as constant-form projectile motion and cannot handle complex object interactions. Video-based wind inference methods Zhang al. (2022); Runia et al. (2020) estimate coarse wind speed or target specific scenarios such as cloth deformation, lacking generality and physical consistency. These limitations motivate the following question: Can

---

Published as a conference paper at ICLR 2026
Wind Ratorgeting
Tn
Sparse View Videos
Objeet Interaction Modeling
Application
DiffWind is physics-informed differentiable wind-object interaction framework that models wind and object dynamics separately. This design enables the reconstruction of wind and object motion from sparse- -iew videos, physically consistent simulation under new wind conditions, and retargeting to novel objects.
Figure I:
jointly recover visible object dynamics and invisible wind fields from videos input, while ensuring physical consistency and generalization to arbitrary wind conditions?
To address this challenge, we propose DiffWind, a physics-informed differentiable framework for wind-object interaction. In DifTWind, the invisible wind is modeled as grid-based physical field, while objects are modeled as deformable particle systems derived from 3D Gaussian Splatting, with their interaction modeled by the Material Point Method (MPM). This design leverages physical intuition: fluids are naturally defined on Eulerian grids, where quantities such as velocity and pressure evolve over space and time, whereas solid semi-solid objects undergo localized deformation, making Lagrangian particles a more effective representation. By coupling these domains, our formulation supports accurate wind-driven object dynamics reconstruction, physically plausible wind-object interaction simulation, and novel applications such as wind retargeting. (Fig. 1)
A key strength of DiffWind is its differentiable formulation, which enables joint reconstruction of the spatio-temporal wind force field and dynamic object motion from sparse-view RGB videos. Reconstruction is achieved by minimizing the image-space reconstruction error via gradient-based optimization. However, photometric loss alone cannot guarantee physical validity, as wind forces follow complex fluid interactions that often entangle physical properties with scene-specifis dynamics, limiting transferability. To address this, we introduce enforces compliance with fluid dynamies. Specifically, we use the Lattice Boltzmann Method (LBM) Li et al. (2023a) to provide directional guidance for the wind force field at each time step, constraining its reconstruction yield physically plausible results.
optimization loss that
Our contributions can be summarized as follows:
We propose a novel differentiable modeling framework DiffWind for wind-object interaction, where the wind is represented as a grid-based physical field and the object is modeled with particle-based deformable geometry. This particle-grit coupling enables physically plausible, 3D-consistent simulations of wind-induced object motion.
Based the proposed interaction model, develop differentiable inverse reconstruction framework that simultaneously recovers the dynamic motion of visible objects and the invisible wind force field from sparse-view videos.
We employ the Lattice Boltzmann Method (LBM) as physics-informed constraint to ensure the wind force field adheres fluid dynamics laws, and demonstrate applications in forward simulation under novel wind conditions and wind retargeting.
We construct WD-Objects, a dataset covering both synthetic and real-world wind-driven object scenes. Extensive experiments show that Win substantially outperforms existing methods in reconstruction accuracy, simulation fidelity, and generalization ability, advancing the frontier of rideo-based wind objecttject interaction modeling.
RELATED WORK
4D Dynamic Scene Reconstruction. Dynamic scene reconstruction and free-viewpoint rendering
have been widely explored by extending Neural Radiance Field (NeRF) Mildenhall et al. (2020) and
2

---

Published as a conference paper at ICLR 2026
3D Gaussian Splatting (3DGS) Kerbl et al. (2023). NeRF- approaches model dynamics either by adding temporal inputs Du et al. (2021); Chu et al. (2022) or by learning canonical spaces with deformation fields Pumarola et al. (2021); Park et al. (2021). 3DGS offers fast reconstruction and high-quality rendering, and has been extended to dynamic scenes via Gaussian tracking Luiten et al. (2024); Zeng et al. (2025), canonical-space deformation Yang al. (2024b); Wu et al. (2024); Zhao et al. (2024), or 4D Gaussian primitives Duan et al. (2024); Wang et al. (2025). But they mainly capture visible object dynamics and struggle model complex, invisible effects such as wind fields, limiting their applicability to wind-driven object dynamics modeling.
Wind-Driven Object Dynamics Modeling. Wind-driven object dynamics modeling involves: (1) simulating wind-induced dynamics, and (2) reconstructing physical parameters from observed motion. Existing simulations use grid-based or particle-based based fluid methods to animate specific objects (e.g. grass, trees, cloth) Pirk al. (2014); Wilson al. (2014); Lo et al. (2016), but are often category-specifer We instead employ the Material Point Method (MPM) Hu et al. (2018) 8) to model complex deformations of various categories of objects under wind fields. Reconstruction methods estimate wind speed material properties from monocular videos Yang et al. (2017); Cardona et al. (2019); Rumia et al. (2020), but cannot recover full 4D dynamics. In contrast, our method jointly reconstructs temporally varying wind fields and dynamic scenes via differentiable optimization.
Differentiable Physics- Based System Identification. System identification is methodology for building mathematical models of dynamic systems using measurements of the imput and output signals of the system. Recent advances in differentiable physical simulation, enabled by frameworks like Taichi Hu et al. (2019; 2020) and Warp Macklin (2022), have facilitated such tasks. Prior works optimize material or geometry parameters for cloth Li et al. (2022), solids Jin et al. (2024), and fluids Li et al. (2024b) using differentiable solvers. Other studies combine differentiable simulation with NeRF Mildenhall et al. (2020) 3DGS Kerbl et al. (2023) to jointly recover object geometry and physical properties from multi-view observations Li et al. (2023b); Zhong et al. (2024): Cai al. (2024); Zhang et al. (2024): Cao et (2024); Liu et al. (2025), but are restricted to simple, predefined motion settings (e.g., gravity, dragging) and optimize initial velocity at the first frame. Moreover, these approaches focus solely on object motion and do not model the influence of the external environment such as wind fields on the dynamics. In contrast, our method reconstructs the wind force field through differentiable optimization process, while enabling the modeling of complex and dynamic wind-object interactions.
METHOD
Given the sparse-view observed videos as input (17) .... 1i N.....N)) of the wind-object interaction, our goal is to reconstruct the dynamic process in 3D. Our key idea is to optimize the wind force field to match the observed sequences through differentiable simulation and differentiable rendering.
Problem Specification:
As shown in Fig. 2, we model the wind as a grid field, where each node can contain various physical quantities. The object is represented material, and motion. We employ a differentiable Material Point Method (MPM) to model the interaction between the wind and the object. The foree property of the wind grid field is applied to the background grid of MPM, which drives the motion of object particles and results in object deformation. More details about the interaction modeling are provided in Sec. 3.2. Then, given observed sequences of object motion under wind influence, we perform differentiable optimization of the force property in the wind grid field that enables the reconstruction of the dynamic process (Sec. 3.3). Furthermore, we propose novel thysics-informed optimization method that leverages LBM to enforce the force field's compliance with the fundamental laws of fluid dynamics (Sec. 3.4).
set of particles, each carrying attributes such appearance,
3.1 PRELIMINARY: PHYSICAL MODELS
Fluid Mechanics. Our wind field dynamics are governed by incompressible Navier-Stoker Equa- tions (NSE). We build the fluid simulator by leveraging the Lattice Boltzmann Method (LBM), which solves a discretized form of the Boltzmann equation and recovers the NSE at the macroscopic level. Owing to its local update rules and explicit time integration, LBM is highly amenable to GPU-parallel
3

---

Published as a conference paper at ICLR 2026
Reconstruction Framework
Contream
NET POIEE RIN
Obsanved Sequences
Optimize Wind force
Wind Returgring
Deraity and Velecity on Grid
Object Partides
[eanne))
Wind Grid Field
Wind Reld Reslation
Mettac (s-991
Differentiable MPM
Foncs Dessit, Valecty
Wind Direction
Guided Loss
Wind-Object Interaction Modeling
Physics Optimization
Figure 2: Overview of DiffWind. We propose novel wind-object interaction modeling approach, where the wind represented as a grid field and the object is modeled as a set of particles. Based on this modeling approach, we introduce reconstruction framework for wind-object interaction by optimizing the wind force field. In addition, we employ the Lattice Boltzmann Method (LBM) to generate wind force field direction guidance enforce compliance with fluid dynamics laws.
computation, offering comparable numerical accuracy traditional NSE solvers Jasak (2009) while achieving significantly higher computational efficiency. With t discretized into fixed intervals, X discretized on regular grid and u,, discretized into fixed unit directions at grid points. Under this discretization, one can solve the NSE by computing the corresponding Lattice Boltzmann Equation (LBE) SHAN et (2006): fx t,, 1) fx(x.. Ω2 Foi.
where c; the discrete lattice velocity in the i-th direction, Ω2 is the Bhatnagar- Baannagar-Gross--e Gross-Grookokk Krook(BGK)) collision model term SHAN al. (2006) and Fo, is the external force. The macroscopic physical quantities of the wind field, such as density Pw. linear momentum Pn Mar and the stress tensor S.. can be calculated as:
q-1
g-1 Pw uw = E 1=0
1 + 2
Pw
Σ fnr 1=0
Foi p
c
Sas
fcr
where a, far, 2. refer to the coordinates of the stress tensor S., and is Kronecker delta. In particular, we use HOME-LBI Li et al. (2023a) as our simulator, which solves the LBE through High-Order Moment Encoding with Hermite polynomial expansion SHAN et al. (2006) achieve more efficient and accurate fluid simulation.
Continuum Mechanics. Our object deformations are governed by continuum mechanics which describes motions by a time-dependent deformation map C, = Φ (Xx., t) that relates rest material position and deformed material position and the deformation gradient F(X,, () Vx, o X t). The evolution of Φ is governed by:
Op. 0t
Ou. uo Up + PoV U. =0, Pol
+ u. Vuo) .. Po
0t
where Po is the mass density, u, the local material velocity of the object, (PF)) the Cauchy stress. These two equations respectively describe the conservation of mass and momentum. In particular, we use the differentiable Material Point Method Hu et al. (2020) to evolve Eq. (3).
3.2 WIND-OBIECT INTERACTION MODELING
We represent the object with 3D Gaussians, particle-based approach suited for high-quality recon- struction and rendering. The wind, being invisible and difficult to represent with discrete particles, is modeled as continuous grid-based field, where each grid node stores physical quantities such as density, velocity, and force, and its evolution simulated using the Lattice Boltzmann Method (LBM). To model the interaction between the representations, adopt the Material Point
4

---

Published as a conference paper at ICLR 2026
Method (MPM), which exchanges information between particle-based objects and the grid- rriddbased wind: objects are treated as moving Lagrangian partieles, while wind acts as a force field on an Eulerian grid.
Object Modeling. To integrate 3D Gaussians into physical simulations, the representation should minimize rendering artifacts under deformation, accurately capture surfaces for boundary conditions, preserve volume integrity, and support region-wise physical properties. Vanilla 3DGS, however, may produce needle-like artifacts and floaters under large deformations, lacks internal structure, and creates holes in weakly textured regions. We mitigate these issues by adding regularization terms during optimization and densifying invisible points in geometry-deficient areas.
In partieular, define the following loss to optimize the 3D Gaussians:
Staii
Loolor Ca + Xoho.
(4)
where Cootor is the color loss between render and training image in vanilla 3DGS; Caniso is designed constrain the anisotropy of the Gaussian kernels:
max s mmodian (157, s;]))
Laniso
Σ SEP
max
|P|
where e 2.0 hyperparameter that controls the level of anisotropy; and Co is designed to encourage the 3D Gaussian points to stay close to the object surface by encouraging each Gaussian's opacity to be either near zero or near one:
Co exp (-3, (o, 0.5)
(6)
where B, - 20 is a hyperparameter. Additionally, we prune Gaussian points whose opacity falls
below a threshold To 0.05.
Based on the optimized Gaussian kernels, we employ the octree-based voxel filling algorithm from Kaolin Fuji Tsang al. (2024) to densify additional invisible internal points for improved geometric support. Since 3DGS directly models the entire scene, extracting objects for simulation becomes necessary. Moreover, physical properties related object deformation cannot be directly obtained from static image observations. Therefore, we train 3D-consistent feature field by contrastive learning Ying et al. (2024); Yang et al. (2025) to extract 2D segmentation priors for 3D regions segmentation in the scene, and employ multimodal large language model (MLLM) to construct physical agent that infers their corresponding physical properties, including material name, density, Poisson's 's ratio up, and Young's modulus E. For more details about 3D regions segmentation and the physical agent, please refer to the Appendix.
Interaction Modeling. We employ Material Point Method (MPM) to model the interaction between the object and wind, by binding each Gaussian kernel and densified internal point to corresponding particle in the MPM simulation. During the P2G and G2P process in MPM. wind is applied as foree field to the MPM grid to simulate the deformation of objects. Similar to PhysGaussian Xie et al. (2024), we update the covariance matrix of 3DGS as (t() = F(t F2(t) is the deformation gradient of each particle p at time step f.
3.3 RECONSTRUCTION FRAMEWORK FOR WIND-OBIECT INTERACTION
We formulate the simulation process of wind-object interaction for single step
(xc
C,, pt+1 I = S(x', v', "',FF",C fp,,CC, O, Fw, At),
(7)
where (x',v','' )") denote the position, velocity, deformation gradient, and affine momentum of all object particles at time t, respectively. Note that the particles include both Gaussian kernels and densified internal points. denotes the collection of the physical properties of all particles, which are obtained during the object modeling step, and F denotes the wind force field.
We can then render the object at each time and each viewpoint by 3D Gaussian Splatting:
i, = Rrender ( (D) 0, gs: Σ, W),
(8)
where sr. and Ft denote the subset of xt and 5 corresponding to the Gaussian kemels.
5

---

Published as a conference paper at ICLR 2026
Owing to the differentiable nature of Eq. (7) and Eq. (8), we can optimize the invisible wind force field from the visible motion of the object by minimizing the photometric loss between the rendered image and imput image
No (1 - A) + ALD- SSIM,
Lrender
where No denotes the number of viewpoints, Li and - represent the Lı and structural similarity index measure losses viewpoint i, respectively. We set À = 0.1 in our experiments.
Although 0 also receives gradients during backpropagation, joinil optimizing both the wind force field F w and the material parameters leads an ill-posed problem, because different combinations of material stiffness and external forces can produce nearly identical motions. For instance, a small Young's s modulus paired with a small force can generate the same motion as a large Young's 's modulus with large force. Therefore, we use MLLM-based reasoning to obtain reasonable material priors as initialization (see Sec. 3.2), and treat only w. as the optimization target, initializing it from zero for each time-step optimization. We use the results from object modeling the initial state and optimize the wind force field between two consecutive states. The optimization proceeds sequentially, with each time step starting only after the previous one has been completed. The optimized object state from the previous step serves as the initial condition for the next which mitigates gradient instability that occur when performing multi-step simulation followed by single backward pass. By optimizing the wind force field at each time step, we simultaneously achieve the reconstruction of wind-object interaction dynamics.
Wind Retargeting. By accurately reconstruction of wind-object interaction, the proposed frame- work enables a novel task: wind retargeting. This involves applying the estimated wind force fields to other objects, as illustrated in Fig. 1, which is infeasible for the existing object dynamics modeling methods like Deformable-GS Yang et al. (2024b), which only focus on dynamic scene reconstruction.
3.4 POYSICS-INFORMER OPTIMIZATION OF WIND FORCE FIELD
Although we can optimize the wind force field each timestep using visual observations, it does not guarantee adherence to physical laws wind forces are fluid and exhibit complex interactions with the target objects. Therefore, we further exploit a physics-informe force field optimization process. Specifically, we use the Lattice Boltzmann Method (LBM) to solve Eq. (1), thereby generating the guiding direction of the force field.
optimization loss to guide the
Wind Source Inference. We start from the posed RGB video and estimate temporally-conisttee monocular depth maps using Vidco Depth Anything Chen et al. (2025). The RGB-depth pairs provided to a multimodal large language model (MLLM), which infers the wind source direction. Then, the inferred direction serves inlet velocity boundary condition for the LBM simulator.
Wind Field Evolution. We use HOME-LBM Li et al. (2023a) to simulate wind field evolution, where the positions of partieles serve as the boundary condition for LBM, thereby influencing the wind field dynamics. In particular, we first distinguish solid and fluid nodes in the LBM lattice based on the positions of particles at the current state by converting object particles into occupied voxel grids:
1, if ar, s.t. Ad E {x, y, 2), 5 F.. X < A.
Solid(x..)) = 0, otherwise
(10)
where x,, is the position of wind grid node, xp is the position of particle, and Ar is the grid spacing. This mapping enables the LBM simulator to capture the effect of the object's instantaneous position on the wind field. At each simulation step, macroscopic wind field quantities are computed from the distribution functions using Eq. (2). Additional details of the algorithm are provided in the Appendix.
Physics-Informed Loss. The wind velocity field from the LBM simulator guiide is used to guide the direction of the reconstructed wind force field D recon:
Cphys Dreeon
(Drecon D Dguide) Dguide 11?
(11)
where Ephys is the physics-informet optimization loss that measures the directional difference between the reconstructed wind force field and the one obtained from the LBM simulator. Here, the wind

---

Published as a conference paper at ICLR 2026
Dress: 41.76
Flag: 41.8
Pants: 58.38
Sweater: 41.72
ShapeNet: 52.54
Ficus: 48.82
Alocasia: 44.94
Figure 3: Our reconstruction dataset from one camera view, with PSNR values of our dynamic reconstruction shown above. Please use Adobe Reader/KDE Okular to see animations. Deformabl-GGS Efficient-GS 4D-GS
Ours
GausianPrediction
Figure 4: Qualitative results of novel view synthesis for selected synthetic wind-object interaction scene. We compare our method with Deformable- Yang et al. (2024b), Efficient- GS Katsumata al. (2024), 4D-GS Wu et (2024) and GaussianPrediction Zhao et (2024). Please use Adobe Reader/KDE Okular to see animations.
force is governed by the aerodynamic drag equation F., - Pm. C -.- where Cn is the drag coefficient and v the wind velocity, this formulation implies that an accurate estimation the wind velocity field direction can effectively guide the reconstruction of the wind force field. Through both physics-informed constraints and visual supervision, we ensure the accuracy and continuity of the reconstructed wind force field.
EXPERIMENTS
In this section, we first introduce the implementation details of our method (Sec. 4.1), followed by evaluations of the capability in reconstructing wind-driven object dynamics (Sec. 4.2). We then conduct ablation studies (Sec. 4.3) to validate key design choices. Finally, we demonstrate the application of our method to forward simulation of wind-driven object dynamics (Sec. 4.4).
4.1 IMPLEMENTATION DETAILS
For real-world captured data, we first use Detector-Free SFM He et al. (2024) compute the pose of each viewpoint. Subsequently, we use EntitySeg Lu et al. (2023) to generate instance-level 2D segmentation masks and Grounded-SAMM Ren al. (2024) to generate part-level 2D segmentation masks. Afterward, we train 3D Gaussians for 60k iterations. For 3D regions segmentation, we train 20k iterations build 3D consistent feature fields Ying al. (2024); Yang al. (2025), and then GPT-5 for physical property reasoning. In our experiments, we divide the simulation space into 1283 grid, and for real-world scenes, only the segmented foreground objects are used input the simulator. All components of our simulator are implemented using Taichi Hu et al. (2019; 2020), and all our experiments are evaluated on single NVIDIA GeForce RTX 4090 GPU.
4.2 EVALUATION ON RECONSTRUCTION
In this section, we first introduce our datasets, then compare our method with SOTA dynamic scene reconstruction approaches for novel view synthesis, and finally show our wind retargeting results.
Datasets. As there are no publicly available datasets for modeling wind-driven object dynamics, introduce both synthetic and real-world datasets, named WD-Objects, in this paper for evaluation.

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative comparison on the synthetic dataset for the novel view synthesis. Best results are highlighted as first second
third
Dross
Flig
Pants
Sweeter
Mathed
PSNR(()
SSIM;1)
1JMPS (3)
PSNR(1)
550M(1)
1PP08())
PSNR(1)
SSIM)1)
1/PPP5())
PSNR(1)
SSIM(()
Deformoll-GS
27.32
9915
0253
40.
09.30
0057
42.36
0924
.0149
33.10
9557
.0594
22.79
26.72
99522
26.06
9241
4D GS
27.64
.9729
0239
36.96
.968
0144
41.33
9911
J184
30.14
9556
.0702
Prrnsitanrooiicioon
29,00
39.89
9917
J168
28.64
Ours
41.76
41.86
58.38
3002
41.72
9951
Ficus
ShapeNetPlant
Abocasia
Average
Method
PSNRI")
SSIM
LPIPS())
PSNRI()
SSIM(1)
LPIPS443
PSNR/1)
SSIM(1)
LPIPS:4)
PSNR(1)
SSUM(")
LPIPS/4)
Defocmable- CSS
41.72
.9067
10130
38.58
.9145
0047
32.79
9681
.1364
36.68
9811
J0233
Elicien--18
22.67
.8951
24.63
.0533
26.57
4D-C8
28,
99508
0372
32.91
9865
0115
28,96
94991
.086
32.19
9704
0306
CoussizaPreeiccion
38,81
37.46
9738
.0992
35.79
Ours
48,82
.9955
1003
52.54
9998
4001
44,94
9980
.000
47.15
9983
.0088
Table 2: Quantitative comparison on the real-world dataset for the novel view synthesis. Best results are highlighted as first second
third
POTHOS
HAT
POMPON
TULIP
Method
PSNR(1)
SSIMi")
1PPPS(L)
PSNR(1)
5SIM(1)
(PSPS())
PSNR(T)
5SIM:1)
(PIPS())
PSNR(
SSIMLT)
LPPPS()))
Defommall-CIS
30.55
9575
0297
20.49
.536
Efficient GS
22:25
9368
.5733
29.53
9862
.042
24.02
.4477
0427
19.34
.432
.0707
4D-GS
24.37
30.32
9556
0347
2041
9527
24
9532
8401
90.36
0564
6307
25.57
.5688
0348
19.96
9511
.0463
Ours
26.18
9692
J0256
31.37
2583
.021
26.14
.9600
J027
23.33
9719
.038
For the synthetic dataset, we first reconstruct seven object-level 3D Gaussians from various sources, including NeRF Mildenhall al. (2020), PhysDreamer Zhang al. (2024), ShapeNet Chang et al. (2015); Ma et al. (2024), and LoopGaussian Li et al. (2024a). We then simulate dynamic scenes of the objects moving with the wind, further details are provided in Sec. 4.4. Next, we select eight orthogonal views to render reference videos, where four viewpoints are used for training and the remaining four for evaluating the quality of novel view synthesis. The evaluation is conducted over 50 simulation time steps. We show the object states from one camera view in Fig. 3. For more complete visualization of the motion process, please refer to our video supplementary. For the real- world dataset, we record 360-degree surround videos of real-world static scenes by GoPro cameras, each scene includes an object and a background. The objects include a pothos plant, a beanie hat, pompon flower, and tulip. Additionally, capture synchronized videos of these plants being affected by a hairdryer.
Baselines. We compare our approach against the current state-of-hee of-the-art dynamic scene reconstruc- tion methods: Deformable-Gr Yang et al. (2024b), Efficient-GS Katsumata et al. (2024), 4D-GS Wu al. (2024) and GaussianPredicction Zhao al. (2024). For fair comparison, we run their opti- mization process using reconstructed static 3D Gaussians as initialization, and apply the same regularization terms as described in Sec. 3.2.
Comparison on Novel View Synthesis. The quantitative results are shown in Table and Table 2. We present the PSNR/SSIM//LIPP (VGG) values for the novel view rendering results to validate the accuracy of our wind force field reconstruction results. We can see that the proposed method significantly outperforms existing methods. A set of qualitative results is shown in Fig. 4, with full results provided in the Appendix, demonstrating that our method produces more realistic novel views than existing approaches, owing to its superior modeling of wind-driven object dynamics.
4.3 ABLATION STUDIES
Effectiveness of Physics-Informed Loss. We evaluate the effectiveness physics-informed loss during dynamic reconstruction. The quantitative results in Table 3, show that our full model, by embedding physical directional constraints during training, further significantly enhances rendering quality, resulting in reconstructions that are not only more precise but also more physically plausible.
Table 3: Ablation of effectiveness of physics- informed loss on the synthetic dataset.
Config.
PSNR
SSIM
I.PIPS x 10
w/o Catys Model
51.31
9995
.0000
.997
.0211
8

---

Published as a conference paper at ICLR 2026
Ours
DynamiCrafee Crafter
Cog CogVideoX
Figure 5: Wind synthesis results the same viewpoint across different timesteps. DynamiCrafter Xing et al. (2023) fails to maintain temporal coherence, Cog VideoX Yang et al. (2024a) produces unrealistic jittering. contrast, DiffWind generates realistic time-evolving wind-object interactions. Please use Adobe Reader/KDE Okular to see animations.
Wind Condition
Retargeting
Wind Condition
Wind Retargeinn
Figure 6: Wind retargeting on synthetic data (left) and real-world data (right).
Robustness against Physical Property. We also inspect the robustness of method to variations in material pa- rameters, To this end, we conduct experiments on the synthetic dataset analyze how changes Young's mod- ulus affect reconstruction performance. We predefine three categories of Young's 's modulus: soft. medium, and hard, with values of E 4 x 10 4 x 10 and 4 x 10 re- spectively, and also include values inferred using MI..M. We perform dynamic reconstruction for each case without the physics- innfoes-infforme loss, in order to better assess the influence of physical properties on the reconstruction results. Table 4 shows that using different predefined physical parameters still achieves comparable reconstruction performance, while the values inferred by MLLM yield slightly better results. Hence, we adopt MLLM for more convenient and accurate physical property estimation.
Table 4: Ablation study of robustness against physical property the synthetic dataset.
Config.
PSNR
SSIM
LPIPS x 10
Soft
49.78
9993
.0040
Mediam
49.83
.9922
.0044
Hard
48.64
.0688
MLLM 51.31
9995
.0030
Necessity of Invisible Internal Points Densification. We also evaluate the impact of invisible internal points densi- fication, which provides geometric support for the simu- lation process. The quantitative results Table indicate that training with densified internal points improves the reconstruction results, justifying its necessity for accurate geometric modeling.
Table 5: Ablation study of necessity of in- visible internal points densification on the real-world dataset.
Config.
SSIM
x100"
w/o Densificatio
27.31
9614
.024
Influence of Grid Resolution. To evaluate how discretiza- tion resolution affects reconstruction quality and computa- tional efficiency, we compare grid sizes of 643, 1283, and 2563, As shown in Table 6 the quantitative results show that reconstruction quality remains stable across different grid resolutions, while 1 provides the best trade-off between fidelity and runtime. Under our default 128 set- ting. each optimization iteration, including the LBM update, the MPM simulation, the differentiable rendering, and the gradient backpropagation, takes approximately 1.17 seconds on a single NVIDIA RTX 4090 GPU, compared 1.02 seconds for 643 and 2.02 seconds for 2563.
Table Ablation study of influence of grid resolution on the synthetic datasct.
Config.
PSNR
SSIM
LPIPS 100
64
49.0%
.991
0146
$1.31
.995
0030
256
$1.90
9997
.024

---

Published as a conference paper at ICLR 2026
4.4 FORWARD SIMULATION OF WIND-DRIVEN OBJECT DYNAMICS
Simulation under Specified Wind Conditions. Benefiting from our wind bojd-object interaction mod- eling approach, we enable forward simulation of wind-driven object dynamics, where object de- formations are simulated by MPM and wind field evolution is simulated by LBM. To evaluate the visual quality of our wind-object simulations, we compare our method against SOTA and publicly available video generation models, including SVD Blattmann al. (2023), Cog VideoX Yang et al. (2024a), VideoCrafter Chen et al. (2024), and DynamiCrafter Xing al. (2023). The comparison is performed on four test scenes. The multi-view images of the static test scenes, Vase and Garden, are sourced from NeRF Mildenhall et al. (2020) and Feature- SapautrnSp Qiu et al. (2024), respectively. Additionally, we captured two static scenes ourselves, Cloth and Sock.
We conduct 20 randomly ordered videos generated by different meth-
study with 32 participants, who rate
Table 7: Human evaluation on visual quality.
ods. Both motion realism and the physical realism of wind-object interactions are evaluated on a 5-point Lik- ert scale, from (strongly disagree) Results in Table 7 demonstrate the superior visual quality of our simulations, with comparative examples shown in Fig. 5. Additional results are provided in the Appendix and supplementary video.
Method
Visual Quality
Cluth
Ganten
Seck
Viec
Aug
(strongly agree).
SVD
2.19
CogVideoX
2.31
1.72
2.69
2.63
2.34
VideoCralter
2.56
2.44
2.38
1.94
2.35
DymneiCraller
1.97
3.33
2.69
3.44
2.84
Ours
4.47
4.16
4.59
4.13
4.34
Wind Retargeting. By using the reconstructed wind force field as input for forward simulation, our method enables a new application, i.e., wind retargeting- We present several representative results of wind retargeting in Fig. I, Fig. 2 and Fig. 6, with additional examples provided in our supplementary video. can be observed that our method effectively decouples the invisible wind force field from the visible object dynamics, while enabling accurate transfer to previously unseen scenes.
CONCLUSION
In this paper, we present a novel framework for modeling wind-object interactions. The proposed framework can reconstruct wind-driven object dynamics, simulate in new wind conditions and perform wind retargeting. To build this framework, we represent wind as a grid-based physical field and model objects using prrticle-bbsed deformable geometry. By leveraging differentiable physical simulation and rendering, our system achieves backward reconstruction of wind-object interactions. In addition, we employ the LBM as a physics-informed constraint enforce compliance with fluid dynamics laws. We also present a new dataset for comprehensive evaluation. The experiments demonstrate the superiority of the proposed framework in modeling realistic wind-object dynamics.
Limitation and Future Work. The proposed framework currently focuses on modeling object-level dynamics under wind conditions, without accounting for interactions between objects. Robustly handling multi-object collisions is challenging due to discontinuous contact dynamics and complex foree interactions, which complicate brased optimization. In addition, MPM is primarily used for simulating continuum objects, however, our framework is simulator-agnostic and can be extended to non-continuum objects by replacing MPM with an appropriate differentiable simulator without changing the overall formulation, other types of simulators such as the spring-mi method or the Finite Element Method (FEM) could be explored to model a wider range of object motion behaviors. We leave these extensions future work.
ACKNOWLEDGMENTS
This work was supported by the National Key R&D Program of China (Grant No. 2024YFB4505500
& 2024YFB4505501), the NSFC (No. 62441222, No. 62572425 and No. 624B2132), Ant Group, Information Technology Center, and State Key Lab of CAD&CG, Zhejiang University. We also express our gratitude to all the anonymous reviewers for their professional and insightful comments.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Our study focuses on modeling wind-driven object dynamics using physics-informed differentiable simulations. All experimental evaluations are conducted using synthetic, publicly available, or self-captured datasets, curated to avoid sensitive or private content. We assert that this research has been carried out in accordance with the code of ethics.
REPRODUCIBILITY STATEMENT
In order to support reproducibility and verification, we present implementation and evaluation details in our paper, and will release the associated source code upon acceptance.
REFERENCES
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15121 2023. 10, 19, 20
Javier Bonet and Richard D Wood. Nonlinear continuum mechanics for finite element analysis. Cambridge university press, 1997. 16
Junhao Cai, Yuji Yang, Weihao Yuan, Yisheng He, Zilong Dong, Liefeng Bo, Hui Cheng, and Qifeng Chen. Gaussian-ifformed continuum for physical property identification and simulation. In Proc. of NeurlPS, 2024. 3
Junyi Cao, Shanyan Guan, Yanhao Ge, Wei Li, Xiaokang Yang, and Chao Ma. Neuma: Neural material adaptor for visual grounding of intrinsic dynamics. In In Proc. of NeurlPS, 2024. 3
Jennifer Cardona, Michael Howland, and John Dabiri. Seeing the wind: Visual wind speed prediction with a coupled convolutional and recurrent neural network. In Proc. of NeurlPS, 32, 2019. 3
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015. 8
Haoxin Chen, Yong Zhang. Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models, 2024. 10, 19,
20
Sili Chen, Hengkai Guo, Shengnan Zhu, Feihu Zhang, Zilong Huang, Jiashi Feng, and Bingyi Kang. Video depth anything: Consistent depth estimation for super-long videos. Proc. of CVPR, 2025. 6, 24
Mengyu Chu, Lingjie Liu, Quan Zheng, Erik Franz, Hans-Peter Seidel, Christian Theobalt, and Rhaleb Zayer. Physics informed neural fields for smoke reconstruction with sparse data. ACM TOG, 4144)11-14 2022. 1,3
Yilun Du, Yinan Zhang. Hong-Xing Yu, Joshua B Tenenbaum, and Jiajun Wu. Neural radiance flow for 4d view synthesis and video processing. In Proc. of ICCV, pp. 14304-14314 IEEE Computer Society, 2021. 1,3
Yuanxing Duan, Fangyin Wei, Qiyu Dai, Yuhang He, Wenzheng Chen, and Baoquan Chen. 4d- rotor gaussian splatting: Towards efficient novel-view synthesis for dynamic scenes. In Proc. of SIGGRAPH, 2024. 3
Clement Fuji Tsang. Maria Shugrina, Jean Francois Lafleche, Or Perel, Charles Loop, Towaki Takikawa, Vismay Modi, Alexander Zook, Jiehan Wang, Wenzheng Chen, Tianchang Shen. Jun Gao. Krishna Murthy Jatavallabhula. Edward Smith, Artem Rozantsev, Sanja Fidler, Gavriel State, Jason Gorski, Tommy Xiang, Jianing Li, Michael Li, and Rev Lebaredian. Kaolin: A pytorch library for accelerating 3d deep learning research. https: /gi thub. com www/NOID//.. kaolin, 2024. 5
Xingyi He, Jiaming Sun, Yifan Wang, Sida Peng, Qixing Huang, Hujun Bao, and Xiaowei Zhou. Detector-free structure from motion. Proc. of CVPR, 2024.7
11

---

Published as a conference paper at ICLR 2026
Yuanming Hu, Yu Fang, Ziheng Ge, Ziyin Qu, Yixin Zhu, Andre Pradhana, and Chenfanfu Jiang. A moving least squares material point method with displacement discontinuity and two-way rigid body coupling. ACM TOG, 37(4):1-14 2018. 3, 16
Yuanming Hu, Tzu-Mao Li, Luke Anderson, Jonathan Ragan-Kelley, and Frédo Durand. Taichi: language for nigh-performance computation spatially sparse data structures. ACM TOG, 38(6); 201, 2019. 3, 7
Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan-Kelley, and Frédo Durand. Difftaichi: Differentiable programming for physical simulation. Proc. of ICLR, 2020. 4,7
Hrvoje Jasak. Openfoam: Open source efd in research and industry. International journal of naval architecture and ocean engineering, 1(2):89-94, 2009. 4
Chenfanfu Jiang, Craig Schroeder, Andrew Selle, Joseph Teran, and Alexey Stomakhin. The affine particle-in-cell method. ACM TOG, 34(4):1-10, 2015. 16
Xutong Jin, Chenxi Xu, Ruohan Gao, Jiajun Wu, Guoping Wang, and Sheng Li. Diffsound: Dif- ferentiable modal sound rendering and inverse rendering for diverse inference tasks. In Proc. of SIGGRAPHH, pp. 1-12, 2024.3 3
Kai Katsumata, Duc Minh Vo, and Hideki Nakayama. A compact dynamic 3d gaussian representation for real-time dynamic view synthesis. In Proc. of ECCV, 2024. 7, 8, 19, 21, 22
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühllrr and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM TOG, 42(4):133-1) 2023. 3, 15
Jiyang Li, Lechao Cheng, Zhangye Wang, Tingting Mu, and Jingxuan He. Loopgaussian: creating 3d cinemagraph with multi-view images via eulerian motion field. In Proc. of ACM MM, pp. 476-485, 2024a. 8
Wei Li, Tongtong Wang. Zherong Pan, Xifeng Gao, Kui Wu, and Mathieu Desbrun. High-order moment-encoded kinetic simulation of turbulent flows. ACM TOG, 42(6))1-13, 2023a. 2, 4, 6, 18
Xuan Li, Yi-Ling Qiao, Peter Yichen Chen, Krishna Murthy Jatavallabhula, Ming Lin, Chenfanfu Jiang, and Chuang Gan. Pac-nerf: Physics augmented continuum neural radiance fields for geometry-agnont system identification. Proc. of ICLR, 2023b. 1,3
Yifei Li, Tao Du, Kui Wu, Xu, and Wojciech Matusik. Diffeloth: Differentiable cloth simulation with dry frictional contact. ACM TOG, 42(1):1-20, 2022.
Yifei Li, Yuchen Sun, Pingchuan Ma, Eftychios Sifakis, Tao Du Bo Zhu, and Wojeiech Matusik. Neuralfluid: Neural fluidic system design and control with differentiable simulation. In Proc. of NeurlPS, 2024b. 3
Zhuoman Liu, Weicai Ye, Yan Luximon, Pengfei Wan, and Di Zhang. Unleashing the potential of multi-modal foundation models and video diffusion for 4d dynamic physical scene simulation. Proc. of CVPR, 2025. 23
Yi Lo. Hung-Kur Kuo Chu, Ruen-Rone Lee, and Chun-Fa Chang. A simulation on grass swaying with dynamic wind force. In Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, pp. 181-181. 2016.
Qi Lu, Jason Kuen, Shen Tiancheng, Gu Jiuxiang, Guo Weidong, Jia Jiaya, Lin Zhe, and Yang Ming-Hsuan. High-quality entity segmentation. In Proc. of ICCV, 2023. 7, 16
Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In Proc. of 3DV. 2024.
Qi Ma, Yue Li, Bin Ren, Nicu Sebe, Ender Konukoglu, Theo Gevers, Luc Van Gool, and Danda Pani Paudel. Shapesplat: A large-scale dataset of gaussian splats and their self-supervised pretraining, 2024. URL https:/ arxiv. org abs 240 10906.
12

---

Published as a conference paper at ICLR 2026
Miles Macklin. Warp: A high-performance python framework for gpu simulation and graphics. https:: /tt::/gitthle ence. 3
March 2022. NVIDIA GPU Technology Confer-
Leland McInnes, John Healy, Steve Astels, et al. hdbscan: Hierarchical density based clustering. J. Open Source Softw, 2(11))205, 2017. 17
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In Proc. of ECCV, 2020. 2, 3, 8, 10
Keunhong Park. Utkarsh Sinha, Peter Hedman, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Bruad Brualla, and Steven M. Seitz. Hypernerf: A higher-dimensional representation for topologically varying neural radiance fields. ACM TOG, 40(6), dec 2021. 3
Sören Pirk, Till Niese, Torsten Hädrich, Bedrich Benes, and Oliver Deussen. Windy trees: Computing stress response for developmental tree models. ACM TOG, 33(6):1-11, 2014.
Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer, D-nerf: Neural radiance fields for dynamic scenes. In Proc. of CVPR, pp.
2021.
Ri-Zhao Qiu, Ge Yang. Weijia Zeng, and Xiaolong Wang. Language-driven synthesis and editing via feature splatting. In Pruc. of ECCV, 2024. 10
moysics--bsedd scene
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14155 2024. 7, 17
Tom FH Runia, Kirill Gavrilyuk, Cees GM Snoek, and Arnold WM Smeulders. Cloth in the wind: A case study of physical measurement through simulation. In Proc. of CVPR, pp. 10498-10507, 2020. 3,
XIAOWEN SHAN, XUE-FENG YUAN, and HUDONG CHEN. Kinetic theory representation of hydrodynamics: a way beyond the navier-stokes equation. Journal of Fluid Mechanics, 550: 413-441, 2006. doi: 10.0017/5002211155500001
4, 15
Alexey Stomakhin, Craig Schroeder, Lawrence Chai, Joseph Teran, and Andrew Selle. A material point method for snow simulation. ACM TOG, 32(4):1-10, 2013. 16
Yifan Wang, Peishan Yang, Zhen Xu, Jiaming Sun, Zhanhua Zhang, Yong Chen, Hujun Bao, Sida Peng, and Xiaowei Zhou. Freetimegs: Free gaussian primitives at anytime anywhere for dynamic scene reconstruction. In Proc. of CVPR, pp. 21750-21760, 2025.
Keith Wilson, Aleka McAdams, Hubert Leo, and Maryann Simmons. Simulating wind effects on cloth and hair in disney's 's frozen. In ACM SIGGRAPH 2014 Talks, pp. 1-1. 2014. 3
Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. Pruc. of CVPR, pp.
June 2024. 3, 7, 8, 19, 22
Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. Physgaussian: Physics-integrant 3d gaussians for generative dynamics. In Proc. of CVPR, pp. 4389-4398, 2024. 5
Jinbo Xing, Menghan Xia. Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Xintao Wang, Tien-Tsin Wong, and Ying Shan. Dynamicrafter: Animating open-domain images with video diffusion priors. arXiv preprint arXiv:2310.122 12190, 2023. 9, 10, 19, 20
Shan Yang, Junbang Liang, and Ming Lin. Learning- cloth material recovery from video. In Proc. of ICCV, pp. 4383-4393, 2017.
Zesong Yang, Bangbang Yang, Wenqi Dong, Chenxuan Cao, Liyuan Cui, Yuewen Ma, Zhaopeng Cui, and Hujun Bao. Instascene: Towards complete 3d instance decomposition and reconstruction from cluttered scenes. In Proc. ICCV, 2025. 5,7
13

---

Published as a conference paper at ICLR 2026
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang. Guanyu Feng, et al. Cogvideox: fext-to-video diffusion models with expert transformer. arXiv preprint arXhv:2408.06077. 2024a. 9, 10, 19, 20
Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. 2024b. 1, 3, 6, 7, 8, 19, 21,
22
Haiyang Ying. Yixuan Yin, Jinzhi Zhang, Fan Wang. Tao Yu, Ruqi Huang, and Lu Fang. Omniseg3d: Omniversal 3d segmentation via hierarchical contrastive learning. In Proc. of CVPR, pp. 20612- 20622, 2024. 5, 7, 16
Lin Zeng, Boming Zhao, Jiarui Hu, Xujie Shen, Ziqiang Dang, Hujun Bao, and Zhaopeng Cui. Gaussianupdate: Continual 3d gaussian splatting update for changing environments. Proc. of ICCV, 2025. 3
Qin Zhang, Jialang Xu, Matthew Crane, and Chunbo Luo. See the wind: Wind scale estimation with optical flow and visualwind dataset. Science of The Total Environment, 846:157204, 2022. 1
Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Feng, Changxi Zheng, Noah Snavely, Jiajun Wu, and William T. Freeman. PhysDreamer: Physics-based interaction with 3d objects via video generation. Proc. of ECCV. Springer, 2024. 1, 3, 8, 23
Boming Zhao, Yuan Li, Ziyu Sun, Lin Zeng, Yujun Shen, Rui Ma, Yinda Zhang, Hujun Bao, and Zhaopeng Cui. Gaussianprediction: Dynamic 3d gaussian prediction for motion extrapolation and free view synthesis. In Proc. of SIGGRAPH, pp. 1-12, 2024. 3, 7, 8, 19, 21, 22
Licheng Zhong, Hong- Xing Yu, Jiajun Wu, and Yunzhu Li. Reconstruction and simulation of elastic objects with spring-mane mass 3d gaussians. Proc. of ECCV, 2024. 1, 3
14

---

Published as a conference paper at ICLR 2026
APPENDIX
We highly encourage watching the supplementary video where our results are best illustrated.
A BACKGROUND
A.1 3D GAUSSIAN REPRESENTATION
3D Gaussian Splatting (3DGS) Kerbl et al. (2023) employs a substantial number of explicit 3D Gaussians represent static 3D scene. Each 3D Gaussian G is defined by a full covariance matrix Σ and center location
G(x) = {(=-n) x-
(12)
For differentiable rendering optimization, 3DGS decomposes into scaling matrix S and rotation matrix R: Σ RSS R", where S and R are stored 3D vector and quaternion q respectively. To project these 3D Gaussians to 2D image, given a viewing transformation W, we obtain the 2D covariance matrix E' and 2D center location p'
Σ' JWEW Jr. J JWW,
(13)
where J is the Jacobian of the affine approximation of the projective transformation. Then we can use the neural point-basee based a-blending to render the color C' of each pixel with ordered 3D Gaussians:
C Σ
Ticaa
(14)
where T;, ag are calculated as:
T, = az); 0r oge - ===- (x-x())
(15)
Here, o, the opacity of the 3D Gaussian. Therefore, the 3D scene can be represented by the parameter set P of 3D Gaussians, where P {G, p4 qs S4, C,2 0.
A.2 FLUID MECHANICS
Navier-Stokes Equations. describe fluid mechanics:
Navier-Stokes Stokes equations are well-known fundamental equations that
Ou
+ Uw Vu
Vp + vV Vu + F..
Pw
(16)
Uup = =0,
where uw is the wind velocity field, Pw is the density, is the pressure, v is the kinematic viscosity and F, the external force, which represents the influence of objects interacting with the wind field in our work. The first equation describes the momentum conservation, while the second equation enforces the incompressibility constraint.
Continuous Boltzmann Equation. The Boltzmann equation was proven to recover Navier-Stokes equation macroscopically SHAN et al. (2006). It describes fluid dynamics through the time evolution of mesoscopic distribution function f(uw: Xw: t):
0f
VS ((f) F.. V...
(17)
a1
where F. represents external forces, represents the probability of a particle being at position time with velocity u and f2 is the collision term to relax distribution function towards the equilibrium state, typically using the Bhatnagar- Saoaaggg--Grss-----
model SHAN al. (2006). By integrating the microscopic properties, the macroscopic physical quantities of the wind field, such as density Pies linear momentum Pr uw and the stress tensor S., can be calculated:
Pw -f
f duw: uw = Uwe duw:
(18)
(u2 [n
1 _0 f duw
Sw. 10
15

---

Published as a conference paper at ICLR 2026
where a; B E {a. y, refer to the coordinates of the stress tensor S.. and is Kronecker delta.
We note that all numerical solvers for fluid dynamics, including the Lattice Boltzmann Method (LBM) adopted in our framework, face inherent limitations under highly turbulent flows when the local Reynolds number exceeds the stability range of the scheme. In such regimes, solvers may fail to fully resolve small-scale vortical structures highly chaotic flow patterns. In our reconstruction pipeline, LBM is used as a plausible behaviors. Consequently, our method can reliably reconstruct moderately complex and unsteady wind fields, but consistent with the intrinsic limitations of the underlying solver, it does not capture the full spectrum of fine aee-grrandd turbulent structures.
constraint to regularize wind estimation toward physically
A.3 CONTINUUM MECHANICS
The object deformations are governed by continuum mechanics. In this section, we introduce the relevant background.
Constitutive models describe the material responses to different mechanical loading conditions, which provide the stress-strain relations to formulate the governing equations. We use the Neo- Koo-Ho constitutive model to capture the non-linearity of the object's response to the wind force field:
Constitutive Models.
11 + (FF)
[F
F) 1] (l) + log²(J).
(19)
Örb
P(F) OF ((F F²) + Alog(J)F
where v is the hyperelastic energy density function, P is the first Fola-KKrchhott stress, F is the deformation gradient that encodes local transformations including stretch, rotation, and shearBonet Wood (1997) and J = Fet(FF). The Lame parameters μ and X are related to Young's 's modulus E and Poisson's 's ratio Vp:
E ((++up) X
Eup
(20)
(1 + 111 2vp)
Material Point Method. Material Point Method (MPM)Stomakhin et al. (2013); Hu et al. (2018) solves the governing equations by transferring physical properties between Lagrangian particles and Eulerian grid. In particular, we use MLS-MPMHu et al. (2018) as our simulator. Each particle's state is described by five-tuple (Irp. my: up Cps Fp): :: Tp the particle position, mp the particle mass, Up is the particle velocity, Cp is the affine momentumJiant et al. (2015) particle, F, is the clastic deformation gradient tracked on the particle. MPM operates in a particle-to- (P2G) and cerrttl-a (G2P) transfer loop simulate the object dynamics. In the P2G stage, we transfer mass and momentum from particles to the grid as:
m
whn mps
m/u! Σ mb (u', + C (z; - xx,))
(21)
In the G2P stage, transfer velocities back to particles and update particle states x + A/u,t)
1-1
Σ
x 1+1
ip:
(22)
where represents the property on the Eulerian grid, represents the property at time t,, and wi, is the B-spline kernel weight at 2; The deformation gradient is updated by MLS approximation as:
F'+1 (I+ AtC)) P
(23)
For more details about the simulation algorithm of MLS-MPMHu et al. (2018), we refer the reader to its original publication.
B MORE DETAILS ON 3D REGIONS SEGMENTATION
A number of existing methods support 3D Gaussians segmentation. Specifically, we employ contrastive learning strategy to lift 2D segmentation results into 3D Gaussians, following Óm- niSeg3DYing et al. (2024). In particular, we use EntitySegl.u et al. (2023) to generate instance-level
16

---

Published as a conference paper at ICLR 2026
2D segmentation masks and Grounded-SAN Ren et al. (2024) to generate part-level 2D segmentation masks, where we design prompts for GPT-4o to identify the names of different parts in the scene as input to Grounded image, and concatenate it with the reference image as a prompt image, then we guide the physical agent identify the region in the marked area and explain all possible material types with physical properties, including Poisson's 's ratio V4 and Young's 's modulus E. Next, we associate each Gaussian kernel with an optimizable feature vector hg E IR 16 for the features with region- wise labels L = {2} in 2D level, we maximize the similarity for features with the same region and low similarity between different regions. We design the following loss for L as:
For each 2D region, we blend its mask with a canonical view reference
expll; P/())
LLc(L)
log
(24)
En' kml expll) 18 16))))
f=1
where N, is the number of regions involved in L, n2 = 11 13 is the render feature with point Σm اا index j and region id i, is the mean value for 12, and Φb ni ing(n++a) is the region temperature, where a = is a smoothing parameter. After training the feature field, we apply HDBSCAN clustering algoribhmMeInnes et al. (2017) to automatically classify different regions without requiring a predefined number of clusters, we then compute the convex hull of each region to categorize the densified invisible points and Gaussian points that may not have been classified by HDBSCAN and we employ KNN to assign any remaining unclassified points based on their proximity to existing clusters. Finally, we assign physical properties to each 3D region based on the correspondence between 2D and 3D segmentations.
C MORE DETAILS ON DENSIFY INVISIBLE POINTS
Although PhysGaussian proposes using a 3D opacity field from 3D Gaussians for internal filling, in practice, this method often fails achieve satisfactory results. Moreover, it requires the 3D Gaussian point cloud to be densely distributed and is not suitable for hole completion. Therefore, we adopt the octree- sctree-based voxel filling algorithm provided by the Kaolin library to densify invisible points for improved geometric support. The algorithm mainly consists of the following three steps:
(1) Voxelization of 3D Gaussians via Octree Construction. The set of 3D Gaussians is first converted into voxel representation using a hierarchical algorithm built upon Kaolin's Structured Point Cloud (SPC), which functions as an octree. The axis-aligned bounding box (AABB) of all Gaussians is enclosed within a cubical root node, which is recursively subdivided in an 8-way manner. For each sub-node, a list of overlapping Gaussian IDs is maintained. Only the nodes that contain relevant Gaussian density are further subdivided and checked for overlaps. This process continues until a specified octree resolution level is reached. The frontier nodes of the octree represent a voxelized shell of the 3D Gaussians. Optionally, nodes whose accumulated opacity values fall below a predefined opacity threshold are culled to remove low-density regions. As result, this step yields a voxel shell approximation of the Gaussian surface, without including inner volume content.
(2) Volume Filling via Multi- Depth Fusion. To convert the voxelized shell into a volumetric solid, the algorithm performs space carving based on rendered depth maps. Specifically, the SPC is ray-traced from an icosahedral set of camera viewpoints to generate depth maps. These depth maps are then fused into second sparse SPC, where each node of the octree maintains an occupancy state: empty, occupied, or unseen. The occupancy status of each voxel in regular 3D grid is determined by querying this SPC. Finally, the union of all occupied and unseen voxels yields a volumetric representation that better captures the object's solid geometry.
(3) Dense Point Sampling. After volume filling, the 3D Gaussians are now represented as dense voxels that include the object's 's interior volume. A point is sampled at the center of each occupied voxel to construct a dense point cloud. Optionally, a small random perturbation is added to each point. This perturbation is constrained to be small enough to ensure the point remains within the bounds of its voxel. By the end of this step, each voxel contains at most one sampled point, resulting in a uniformly distributed and volume-complete point cloud.
17

---

Published as a conference paper at ICLR 2026
D MORE DETAILS ON HOME-LBM ALGORITHM
The post-collision distribution function in HOME-LBM is given by:
(e)::
fi = pw)
1+ c
2c²
1
+
Hany (c:) + 2Sayus 2u
2c
+
H 09y (Si) Us + 2Szy 2uz
|4]
H 1 (es)(Sax (Sax + 2S2x 2u (uuU
(c:)( + 2S, Uz))
(25)
H (c.)( My 2Sya U2 22)
H uya (5)) 2 2uy uz)
+ [|] ( (c:) + SyzMs + Say 21
where H is the Hermite polynomials, S is the stress tensor and u is the velocity. This post-collision distribution function enables the computation of the microscopic distribution function in the LBM, which can then be integrated into the standard LBM simulation pipeline.
To address the low accuracy problem caused by the BGK collision model in the classical LBM algorithm. HOME-LBM introduces a high-order collision model based on Hermite expansion after the streaming step the update process is given by:
plz, t + 1) p²,
Ho (a t + 1) u.a +
Fa,
2p
Say t 1) =(1 (1- 1
2-
Sty uu +
(Faul + Fyghi))
2Tp"
7 --1 1
1 +2 thy + ut?
Sea (z, t + 1)
(2S
Syy Stx)) +
37
+
1
Fau +
-
(2 Fyr - F2ul)..
(26)
p
3rp"
Sy (2, 1)
t
(2S) Shh St.) Us
.2 uny us²
Fg + 3rp"
(2F FFu F.ul).
Pr
S2x(w, 1 1)
1
(2S - S (yy) + 3
22 uny u.
3r
1
+ Flu +
(2F. - F.V., F.ue))
3rp"
Here, the superscript denotes the values obtained after the streaming step and F denotes the force term derived after the streaming step. For a more detailed mathematical derivation, please refer to the original HOME-LBM paper Li al. (2023a).
E MORE EXPERIMENTAL RESULTS
E.1 MORE SIMULATION RESULTTS
18

---

Published as a conference paper at ICLR 2026
Table 8: Detailed time statistics for forward simulation. #OBG denotes the number of object particles, and #BG indicates the number of background Gaussian particles. "LBM Time", "MPM Time", and "Render Time" represent the average computation time per timestep for LBM simulation, MPM simulation, and 3D Gaussian Splatting rendering, respectively.
Scene.
#OBG
BBG
LBM Time(x10-3)
MPM Time(s)
Render Time(x10
Cloth
125223
527159
0.702
0.919
0.598
Gurden
52505
4484605
0.710
Sock
131794
630068
0.709
0.669
Vase
145868
1944064
0.720
viewO
view1
view2
view3
view4
Original
SVD
CogVideoX
VideoCrattar
Dynamicratter
Figure 7: Wind synthesis results the same timestep from multiple views. SVD Blattmann al. (2023) exhibits uncontrolled camera movements, VideoCrafter Chen et al. (2024) and Dynami- Crafter Xing al. (2023) fail maintain 3D consistency object geometry and generate only subtle motion, while Cog VideoX Yang et al. (2024a) fails to produce geometrically consistent object motion across views. contrast, DiffWind maintains 3D consistency and produces realistic wind-object interaction.
Fig. 7 and Fig. compare our results against various diffusion models, demonstrating that our method produces more physically plausible and 3D-consistent motion. The time performance of our forward simulation is reported Table 8. Please refer to our video supplementary material for more results.
E.2 MORE RECONSTRUCTION RESULTS
Baselines.
We compare approach against the current state-of-ther oo-the--rttttttt dynamic scene reconstruc- tion methods: Deformable-GS Yang et al. (2024b), Efficient-GS Katsumata et al. (2024), 4D-GS Wu al. (2024) and GaussianPredicttion Zhao et al. (2024).
19

---

Published as a conference paper at ICLR 2026
Original
Windy
Original
Windy
Figure 8: DiffWind is capable of simulating realistic wind-object interactions in static scenes reconstructed from real-world captured multi-view images.
t0
t1
12
SVD
CogVideoX
VideoCrafter
DynamiCrafter
Ours
Figure 9: Wind synthesis results at the same viewpoint across different timesteps. SVD Blattmann et al. (2023) suffers from noticeable camera shake, CogVideoX Yang al. (2024a) produces unrealistic jittering, VideoCrafter Chen et al. (2024) distorts the original object structure and DynamiCrafter Xing al. (2023) fails maintain temporal coherence, resulting in implausible motion artifacts. In contrast, DiffWind generates realistic time-evolving wind-object interactions.
Deformable-GS reconstructs dynamic scenes by introducing additional MLP-based deformation field, while Efficient-GG reconstructs dynamic scenes by learning trajectory functions that govern the motion of Gaussian kernels over time. Similar to Deformable- 4D-GS models the deformation field via HexPlane representations, and CquusianPredicttion associates each Gaussian point with an additional motion feature extra input to the deformation field. For fair comparison, run their optimization process using our reconstructed static 3D Gaussians as initialization, and apply the same regularization terms as described in Sec. 3.2.
Data. We also capture real-world datasets for evaluation. Specifically, we record 360-degree surround videos of woal-world static scenes by GoPro cameras. Each scene includes an object and background. The objects include pothos plant, beanie hat, pompon flower, and tulip. Additionally, we capture synchronized vidcos of these plants being affected by hairdryer. We report the quantitative comparison results in Table. and show the qualitative results in Fig. 11.
20

---

Published as a conference paper at ICLR 2026
Deformable-G
GaussianPrediciion
Figure 10: Qualitative results of novel view synthesis synthetic wind-object interaction scenes. We compare our methods with Deformable-GSYang et al. (2024b), atticinntCGSKassumaataa al. (2024), 4D-GSWu et al. (2024) and GaussianPPreiictionZZaaa et al. (2024).
21

---

Published as a conference paper at ICLR 2026
Deformable-GS
Efficient-GS
4D-GS
Predsianno
Ours
GT
Figure 11: Qualitative results of novel view synthesis on real-world wind-object interaction scenes. We compare our method with Deformable-Gas Yang et al. (2024b), (2024), 4D-GSWu et (2024) and GaussianPredictionZZan
et al.
al. (2024).
Table 9: Evaluations of reconstructed wind force fields on the synthetic dataset.
Metrics
Flag
Pants
Sweater
Dress
Fieus
ShapeNet
Alocasia
Average
CosSim
0.9489
0.9585
0.9474
0.8390
0.8695
0.9883 0.0078
0.9229
0.9249
NMSE
0.0341
0.0276
0.0350
0.1074
0.0870
0.0514
0.0500
Ablation on Invisible Point Densification. We inspect the effectiveness of densifying invisible points and conduct the experiments on the real-world dataset. In Fig. 12, the left side shows the original locations of the 3D Gaussians, while the right side shows the complete point set after densification.
W/O Densification
Full Model
Please generate a caption for given image explicitty mentions the different part categories present the image. The caption should only indude the names of the parts (egg, "lower pot' 'Boowrrs" without any detailed descriptions. Ensure the capsion is concise and clearly incicates the presence of dietinet parts.
Example cutput:
inA potted plant with flower pot and flowers." inA car with wheels, windows, and noof." inA chair with seat, backrest, and
Next, Extract and list
from provide poti 'flwwrss) parts,
simple part period nuspuc
singl
spaces
Example cutput:
parts: "flwwrr pot. flowers""
Figure 12: Visualization of object particle posi- tions before and after invisible points densifica- tion.
Figure 13: Prompt for 3D Segmentation.
Evaluations Reconstructed Wind Force Fields. We assess the fidelity the reconstructed wind force field F,, under controlled synthetic settings, as shown in Table 9. Let F and Fre denote the ground-truth and reconstructed wind force fields. We normalize the force vector at each grid location to unit length, and denote the normalized vectors as Fs and Fave, For directional consistency, report the average cosine similarity over all time steps CosSim N EN RS pase
22

---

Published as a conference paper at ICLR 2026
with range [-1, 1), where higher values indicate stronger alignment between reconstructed and ground truth directions. Additionally, we report the normalized mean squared error over all time steps NMSE = + Σ |
1 with range [0, 4], where lower values indicate better reconstruction. These results show that the recovered wind force fields are directionally coherent and maintain consistent relative magnitudes with respect to the ground truth on the synthetic dataset.
Evaluations of Other Baselines. Note that there are existing methods that also employ differen- tiable physical simulation for physics parameter es- timation from video observation, including Phys- Dreamer Zhang et al. (2024) and PhysFlow Liu et al. (2025). However, both of them only optimize the initial motion velocities at first frame and Young's modulus for simple and predefined motion patterns, which cannot fully characterize the complex dynam- ics provided in the input video, especially for windy object reconstruction, shown in Fig. 14. There- fore, the results obtained by these methods are gen- erally worse than those of the dynamic scene recon- struction methods we listed above for novel view synthesis. For example, for the Ficus example in our dataset, the PSNR/SSIM/L PIPSS metrics for Phys- Dreamer and PhysFlow are 22200.854600.000 18. 11/0.859/0.000 respectively, which are consistently worse than other methods. Hence, we focus on the comparison of our method those SOTA dynamic scene reconstruction methods for the novel view synthesis task.
PhysDreamer
PhysFlow
Ours
Figure 14: PhysDreamer and PhysFlow fail to reconstruct dynamics for objects with com- plex wind-induced motion, with the GT motion (red) shown underneath for reference, while method faithfully captures the motion.
and
E.3
COMPUTATIONAL COST AND SCALABILITY
All experiments our paper can be executed on single NVIDIA RTX 4090 GPU (24GB). Both the LBM and MPM simulators are implemented using the Taichi GPU framework, enabling high- performance GPU execution of the entire physics pipeline. For forward simulation, DiffWind requires less than 1 second per frame for the complete LBM wind update, MPM object simulation, and differentiable rendering. demonstrating higher efficiency than existing video generation baselines, for example, VideoCrafter requires ~ 81.68s generate 16 frames, DynamiCrafter 151.47s for 16 frames, and Cog VideoX ~ 292.37s for 50 frames. For reconstruction tasks, each optimization iteration includes the LBM update, MPM simulation, differentiable rendering, and gradient computation, and runs in ~ 1.17 seconds. The overall reconstruction process takes roughly hours per scene. This makes the optimization computationally practical while maintaining high reconstruction fidelity and physical consistency. For comparison, 4DGS typically optimizes for ~ I hour per scene. While DiffWind slower than 4DGS-style baselines, this is expected because each iteration involves full differentiable physics simulation, tackling a substantially more complex problem. Improving the efficiency of differentiable physical simulators is a broader challenge for the community and a method.
direction for future research, not limitation unique
We also evaluated scalability with respect to grid resolution. GPU memory usage during training increases moderately with grid size: 64 uses ~ 12.3GB, 128 uses 13.0GB, 256 uses ~ 14.8GB, and 5123 uses ~ 21.0GB. This indicates that memory usage remains practical for moderately sized scenes, thanks to Taichi's efficient memory handling. For larger-scale scenarios, multi-GPU parallelization and other scalability strategies can be employed.
F MORE DETAILS ON PROMPT DESIGN
We present the prompt template for precise 3D segmentation, as shown in Fig, 13. This template is designed to generate a brief yet accurate segmentation caption when provided with given image. Next, design a physical agent based on MLLMs for opca-vocabulary semantic reasoning about materials and their physical properties. For each segmented object part, we combine its mask with a canonical reference image and concatenate it with the reference image form prompt image. The
23

---

Published as a conference paper at ICLR 2026
garden: A bouquet composed mainly of eucalyptus leaves with sprigs of small pink belge dark weathered
sock: A large, festive Christmas stocking, primanlyy red and olive green, featuring white "Memy Christmas" lettering and an attached Santa Claus figurine with fluffy beard, clipped several white clothespins brown wooden stand. The scene set indoors against plain off-white wall and light-colored tiled floor. The camera remains perfectly static, maintaining consistent medium gentle breeze blows from the right, causing the stocking to sway softly and naturally to the left. The fabric of the stocking should show slight ripples, should exhilbit realistic, subtle movements in response to the airflow.
and white flowers, contained ceramic pitcher, sits wooden table. Next to It rests small vintage-style gramophone accent piece. The camera remains still, zoorning, closing in, no direction -hanniinnn just remains still. A strong wind gusts from the right, causing sway and dance dramatically and naturally to the left with significant motion.
dark
zoom, pan, or tilt. steady,
and tollage
vase: vibrant bouquet white lilies, purple alstroemeria and green chrysanthemums in clear glass vase sits on weathered wooden planks. The camera remains as strong wind gusts from right, causing the flowers and follage sway and dance dramatically to the left with significant motion.
cloth: A light beige jacket hangs from white plastic hanger. This hanger is hooked onto the side of small, light-colored wooden shelf-ike structure that rests atop stack of features dark, plush anmchairs and additional cardboard boxes, indicating an indoor setting. The camera maintains completely statkc medium shot, with no zoom, pan, tilt. strong and steady breeze emanates from the right, causing the jacket sway violently and naturally towards the left. The fabric of the jacket, especially its sleeves and lower portions, should exhibit realistic ripples and movements due to the wind.
brown cardboard
background
Figure 15: Prompt for video diffusion models.
agent is then guided to identify the object part within the marked area and reason about all possible material types and their associated physical properties. The prompt template is shown in Fig. 17. Note that the MLLM we used here is GPT5. For wind source inference, given the observed RGB video, we first generate the monocular depth video through Video Depth AnythingChen et al. (2025), and then infer the wind source direction using the prompt illustrated in Fig. 16.
In addition, we utilize the prompt design shown in Fig. 15 to guide video generation models in synthesizing scenes of wind-induced object motion. These synthesized videos are qualitatively compared with our physically-basse forward simulation results to evaluate visual realism and motion plausibility, as presented in the main paper.
24

---

Published as a conference paper at ICLR 2026
You Al model specialized in analyzing RGB video and depth video. Focus exclusively on the following task and do not generate unrelated content.
Wait until provide the video input before producing any result.
Task Description:
You will be given a monocular RGB video and its corresponding depth video that together capture an object being blown by the wind.
Your goal is to estimate the direction of the wind source in 3D space.
A It is essential that you use both the RGB video and the depth video together. Do not rely only on RGB appearance or 2D motion you must incorporate depth information to reconstruct the object's **3D motion trajectory** before inferring the wind source direction.
Coordinate System:
x-axxis: horizontal direction (positive to the right)
- y-aaxis: vertical direction (positive upward)
z-axis: perpendicular to the screen, pointing outward toward the viewer The origin is at the bottom-left corner of the image.
Instructions:
1. When the videos are provided, jointly analyze the object's motion using RGB appearance Depth structure.
2. Explicitly use the depth data to lift the motion from 2D image space into 3D motion space.
Infer the most likely wind source direction that causes this motion.
Output the result strictly in the following format: ""sson
"wind Londd_surre___e
welssssuuuureettiooooe direction": [x, y, z).
"explanation": "concise explanation showing how both RGB and Depth information together indicate this direction"
Figure 16: GPT Prompt for Wind Source Inference.
25

---

Published as a conference paper at ICLR 2026
You are ChatGPT, large language model trained by OpenAl, based on the GPT-5 architecture.
You need to complete task related materials in computer graphics. The task here is object recognition and reasoning about the possible material types and material properties (such Young's modulus and Poisson's ratio) of the object based on the object information you recognized. will provide you with image of cobject_name]) name]. This image is divided into two sub-images. The left one is the original image, and the right one uses blue color mask the object. need you to: Identify what object the blue masked area corresponds to. Use your identification results to infer the material type of the object. For the identified material, provide the corresponding Young's modulus and Poisson's ratio. # For instance, give you an image of a potted plant where the area masked in blue corresponds to the pot. You should identify it as the pot, and then infer the material of the pot, such ceramic, plastic, metal, based on the characteristics of the pot. Then, provide the Young's modulus and Poisson's ratio for the inferred material (e.g,, ceramic: Young's modulus 70 GPa, Poisson's ratio 0.25).
[object
The right sub-image indicates the mask area, and then you could judge based on the left sub-image which is the original image. You must zoom in the corresponding area in the left original image for better recognition. VERY IMPORTANT: For object recognition, you must make your judgment based on this principle: The mask area all of the object and the area outside the mask does not belong to the object. Let's think step by step for object recognition.
IMPORTANT: Please also combine image information when reasoning about materials, and sort the results according to likelihood. Let's think step by step for material reasoning.
To provide an answer, please provide short analysis for object recognition and material reasoning. Analysis: Object recognition: The object in the masked area is [object name], because [reasoning for identification]. Material reasoning: The possible material types are [material 1], [material 2]. ..., because [reasoning for material types]. Material properties: Young's modulus: [value] GPa (for identified material) Poisson's ratio: [value] (for identified material) Final answer: [material 1]
[material 2] [material 3] (sorted by likelihood) Example Output:
Analysis:
Object recognition: The object in the masked area is the pot, because its shape and appearance match typical pot structures.
Material reasoning: The possible material types are ceramic, plastic, metal, because pots are commonly made from these materials, based on the visual texture and color.
Material properties:
-Young's modulus: 70 GPa (for ceramic)
Poosssn's ratio: 0.25 (for ceramic)
Final answer: ceramic plastic metal
Figure 17: GPT Prompt for Physical Properties Reasoning.
26


Reviews:
[{"review_id": "wpEKvDQOlE", "reviewer": "Reviewer_r5yh", "summary": "The paper proposes DiffWind, a physics-informed, differentiable framework that jointly reconstructs an invisible, time-varying wind field and the visible, deformable object motion from sparse-view RGB videos. The authors also introduce WD-Objects (synthetic + real scenes) and report improved novel-view rendering and physically plausible simulations, including wind retargeting and forward simulation under new wind conditions.", "questions": "1. What are the failure modes when the wind is highly turbulent or when Reynolds numbers push LBM discretization limits at the chosen grid resolution?\n2. How does performance scale with grid size (e.g., $128^3$) vs. runtime, and is there an adaptive meshing strategy planned?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The motivation is quite clear. The paper is well written and easy to follow.\n2. The creative combination of LBM + MPM + 3DGS removes limitations of prior work that either modeled only visible dynamics or only simple forces.\n3. The wind retargeting demonstrates a strong capability to generalize the wind to other objects.\n4. On both synthetic and real data, DiffWind outperforms state-of-the-art dynamic 3DGS baselines on novel view synthesis.", "weaknesses": "1. The method explicitly optimizes only the wind force field while keeping material parameters fixed after MLLM “physical agent” reasoning; this makes it hard to disentangle whether observed motion comes from wind magnitude or material stiffness/damping.\n2. Evaluation on real data relies on image metrics (PSNR/SSIM/LPIPS) and a user study, but no direct wind-field measurements are reported. Given that there are no public datasets for wind-driven dynamics, it would strengthen claims to instrument selected scenes with anemometers or PIV.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "scxEABjihT", "reviewer": "Reviewer_uA1M", "summary": "This paper proposes a physics-informed, differentiable modeling framework designed to simulate and reconstruct wind-driven object dynamics from videos. \nIt models wind as a grid-based physical field and objects as particle systems, and uses the MPM for object-wind interaction. DiffWind optimizes both wind forces and object dynamics via differentiable rendering and simulation with LBM, ensuring physical consistency.\nThe objective experimental results are quite impressive.", "questions": "1. I wonder whether it becomes computationally intensive for large-scale or complex simulations with LBM and MPM?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The framework allows joint optimization of wind forces and object dynamics and leverages differentiable physics simulation for accurate reconstruction. The use of LBM ensures the wind dynamics adhere to fluid mechanics laws. I think this should be effective and novel.\n2. The method outperforms state-of-the-art dynamic scene modeling approaches in both reconstruction accuracy and simulation fidelity.\n3. It introduces wind retargeting to enable wind dynamics to be applied to novel objects. This expands the range of its use in simulations and visual effects.\n4. It introduces a dataset WD-Objects for modeling wind-driven object dynamics.", "weaknesses": "1. As the authors stated, the current implementation focuses on modeling object-level dynamics without accounting for interactions between multiple objects.\n2. This method requires accurate segmentation for optimal performance. This may limit its application in less controlled environments and practical scenarios.\n3. This paper focuses on continuum objects. What will happen when the method is extended to simulate behaviors in other types of objects?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IFXphT2ygs", "reviewer": "Reviewer_QPR1", "summary": "This work proposes a method for wind-driven object dynamics reconstruction. It leverages a combination of physics-based simulations and machine learning techniques to accurately model the behavior of objects subjected to wind forces. Given observational data, the method learns to infer the underlying physical parameters used in the Material Point Method (MPM) simulations, enabling realistic reconstructions of object dynamics in windy environments. A physics-aware optimization strategy is employed to improve the fidelity of the reconstructions.", "questions": "1. What's the computational cost and runtime of the proposed method compared to baselines?\n2. How does the method initialize the physical parameters for optimization? A comparison between the initialization and the final learned parameters would be insightful.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The coupling of differentiable physics (LBM + MPM) with 3DGS for realistic wind–object interaction modeling is novel. \n2. Experiments on synthetic and real-world datasets demonstrate clear performance gains over state-of-the-art methods. \n3. The introduction of WD-Objects and the novel “wind retargeting” task broaden research potential.", "weaknesses": "1. This work can be viewed as exploration in generative simulation. However, the experimental results in the paper are still toy examples. The author should justify the practicality of the proposed method in real-world applications. What can this method be used for in practice? \n2. The novelty of the proposed method should be further emphasized. How does it compare to existing approaches in the literature? Based on PhysGaussian, the authors should discuss more about the differences and improvements over prior optimization-based methods for physical parameter estimation, such as PhysDreamer, Physics3D, DreamPhysics, OmniPhysGS, etc. \n3. In the supplementary material, the authors only provide backward reconstruction results for clean-background cases. It seems that the method can only handle simple scenarios, which limits its applicability. The authors should provide more analysis and discussion on the limitations of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KdS5oLAo1m", "reviewer": "Reviewer_8z3H", "summary": "This paper introduces **DiffWind**, a physics-informed generative framework that reconstructs and simulates **hidden wind fields** from **multi-view videos** of wind-driven objects.  \nThe core idea is to jointly optimize the **latent wind field** and the **object motion** under physical constraints.  \nDiffWind represents wind using an **Eulerian grid** and objects using **Lagrangian particles**, coupled via the **Material Point Method (MPM)** for differentiable wind–object interaction.  \nThe method further enforces **fluid dynamics consistency** through a loss derived from the **Lattice Boltzmann Method (LBM)**, ensuring physically plausible flow.  \n\nThe authors also construct a new dataset (**WD-Objects**) containing both synthetic and real scenes of deformable objects driven by wind.  \nExperiments show that DiffWind achieves high-quality 3D reconstruction, realistic forward simulation, and plausible “wind relocation” (transferring estimated wind fields to new objects or scenes).  \n\nOverall, this is a technically strong paper with detailed derivations — good job by the authors.", "questions": "**Q1.** How robust is DiffWind to **incomplete or single-view** input? Could it generalize if only a subset of views is available?  \n\n**Q2.** I am curious about the **training data setting**. The quantitative metrics in Table 1 are extremely high — for instance, a PSNR of 52.5 dB suggests a very dense camera setup. How many camera views were used during training?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- **Novel problem formulation:** One of the first attempts to jointly reconstruct *hidden wind fields* and *object dynamics* from visual observations, bridging 3D reconstruction, differentiable physics, and generative modeling.  \n- **Strong technical depth:** The coupling of the Eulerian grid (wind) and Lagrangian particles (objects) via MPM is elegant and physically grounded. The inclusion of an LBM-based regularization further enforces physical realism.  \n- **Thorough experiments:** Evaluated on both **synthetic and real** WD-Objects datasets with multiple categories (cloth, flags, plants, etc.), demonstrating reconstruction quality (PSNR, SSIM, LPIPS) and perceptual realism via user studies.  \n- **Demonstrated versatility:** The model supports *forward simulation* and *wind relocation* tasks, showing potential for cross-scene generalization.", "weaknesses": "**W1. Dependence on ideal inputs:** The method requires multi-view, calibrated video and accurate segmentation, which limits its applicability to real-world, in-the-wild scenarios.  \n\n**W2. Computational cost:** The framework integrates MPM, LBM, and differentiable rendering, but training time and memory requirements are not reported. Practical efficiency and scalability remain unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)