Multi-Object System Identification from Videos
Authors: Chunjiang Liu
Xiaoyuan Wang
Qingran Lin
Albert Xiao
Haoyu Chen
Shizheng Wen
Hao Zhang
Lu Qi
Ming-Hsuan Yang
Laszlo A. Jeni
Min Xu
Yizhou Zhao
Venue: ICLR 2026 Poster (2026)
Keywords: Object Property Identification
Physics-based Modeling
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: We introduce the challenging problem of multi-object system identification from videos, for which prior methods are ill-suited due to their focus on single-object scenes or discrete material classification with a fixed set of material prototypes. To address this, we propose MOSIV, a new framework that directly optimizes for continuous, per-object material parameters using a differentiable simulator guided by geometric objectives derived from video. We also present a new synthetic benchmark with contact-rich, multi-object interactions to facilitate evaluation. On this benchmark, MOSIV substantially improves grounding accuracy and long-horizon simulation fidelity over adapted baselines, establishing it as a strong baseline for this new task. Our analysis shows that object-level fine-grained supervision and geometry-aligned objectives are critical for stable optimization in these complex, multi-object settings. The source code and dataset will be released.

Published as a conference paper at ICLR 2026
MOSIV: MULTI-OBJECT SYSTEM IDENTIFICATION FROM VIDEOS
Chunjiang Liu Xiaoyuan Wang** Qingran Lin Albert Xiao
Haoyu Chen
Shizheng Wen"
Hao Zhang" Lu Qi" Ming-Hsuan Yang"
Laszlo A. Jeni Min Xu Yizhou Zhao
CMU
Georgia Tech
Harvard
ETH Zurich
A UIUC
IIsta360
UC Merced
Dool
Predicted Categorical Distribution
Fixed Expert Constitutive Models
Existing Work
Goometric Reconstracionn
System Identification
Simulation & Rendering
Per-Object Optimized Constitutive Models Ours
Observations
Predictions
Figure 1: From multi-view observations of multi-object scenes (left), prior approaches select from a fixed library of expert constitutive models via categorical prediction, leading to visually implau- sible and weakly calibrated physics dynamics. MOSIV instead performs geometric reconstruction, per-object system identification of continuous constitutive parameters, enabling both faithful repro- duction of observed interactions and accurate prediction of future behaviors (right).
ABSTRACT
We introduce the challenging problem of multi-object system identification from videos, for which prior methods are ill-suited due to their focus on single-object scenes discrete material classification with fixed set of material prototypes. To address this, we propose MOSIV, new framework that directly optimizes for con- tinuous, per-object material parameters using differentiable simulator guided by geometric objectives derived from video. We also present a new synthetic bench- mark with contact-rich, multi-object interactions facilitate evaluation. On this benchmark, MOSIV substantially improves grounding accuracy and long-horizon simulation fidelity over adapted baselines, establishing it as a strong baseline for this new task. Our analysis shows that object-level fine-grained supervision and gcometry-alignen objectives are critical for stable optimization in these complex, multi-object settings. The source code and dataset will be released.
"quual contribution. 'Equal advisorship.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Real-world scenes are dynamic and often chaotic; multiple objects collide, slide, and reconfigure themselves through a constant dance of contact. Most methods (Cai al., 2024; Zhao et al., 2025; Liang et al., 2019; Raissi et al., 2019 Li et al., 2023a; 2022) that try to understand physics from video are designed for simple, controlled settings ttppicllyy a single object moving in isolation. These approaches fail in complex, everyday environments where objects bump into one another, block each other from view, and have their motions intricately linked. To enable advanced applications like robotic manipulation in cluttered spaces (Yin et al., 2021; Shi et al., 2023; 2024a) or physically plausible scene editing, we need a method that can learn the physical properties of all objects and their interactions simultaneously, just by watching videos of them.
object's
We introduce and formalize this challenge as multi- mujti- system identification from videos. Our goal straightforward: given multi-view videos of interacting objects, we aim to reconstruct their chang- ing 4D geometry (3D shape over time) and identify the physical properties of each object-such as its stiffness, plasticity, and friction. A successful result is a "digital twin" of the scene. where physics simulator can reproduce the observed motion, accurately predict future interactions, and generalize to novel scenarios, such as different initial conditions or force fields.
Object interactions are double-edged sword: they provide rich signals that make hidden physical properties observable, but they also create challenges like occlusions and abrupt, complex motions. Solving this multi-object problem requires an accurate 4D reconstruction (Kratimenos et al., 2024), a simulator that can handle contact and friction between different materials (De Vaucorbeil et al., 2020), and a learning process focused on identifying specific parameters rather than just selecting a material category. Ambiguities like distinguishing stiffness from friction can't be resolved by appearance alone; the system must analyze geometry and motion over time (Cai et al., 2024).
We compare our method to OMNIPHYSGS (Lin et al., 2025), a baseline chosen for its ability to model scenes with varying materials. However, its core design is ill-suited for task. OM- NIPHYSGS performs model selection it classifies materials by picking from a small, fixed li- brary-rather than identifying the continuous parameters (e.g.. E, M, (1) needed for accurate physies. In contrast, our method learns continuous parameter maps for each object and couples this represen- tation with differentiable MPM simulator (Jiang et al., 2016; Hu et al., 2018; Geilinger al., 2020; Du et al., 2021; Qiao et al., 2021) that accurately models how different materials interact, yielding identifiable parameters and realistic dynamics. We also compare COUPNERF (Li al., 2024a), which tackles multi-object system identification with an implicit NeRF representation under free- fall regime. While effective in that setting. onteooptmiized NeRF fields are computationally heavy and prone to temporal inconsistency in contact-rich, highly deformable scenes.
Our solution built on three synergistic components. First, object-aware dynamic Gaussians track each object's unique material properties in 4D with pre-defined 2D material masks. Second, differentiable Material Point Method (MPM) simulator accurately models complex inter-material physics, including contact and friction. Third, joint multi-object fitting learns continuous parameters by aligning simulated surfaces and silhouettes with visual evidence from the video. To validate this approach, we formalize the task, release a new synthetic dataset, and adapt the OMNIPHYSGS and COUPNERF baseline to use direct visual supervision for a fair comparison.
On contact-rich scenes, our method significantly reduces parameter error and improves simulation accuracy over time compared to the adapted baseline. Our simulated trajectories remain stable and aligned with the observed video, whereas the baseline's discrete material choices cause drift. Ablation studies confirm that all three components are essential for achieving stability and accuracy.
To sum up, the main contributions of this work are threefold:
We formalize the task of multi-objecc system identification from videos and release challenging synthetic dataset with ground-truth physical parameters to drive future research.
We propose a new framework that combines object-aware dynamic Gaussians with joint multi- supervision to directly identify the continuous,
object fitting. This approach uses geometry-dre object-specific physical properties from video.
We validate method the new dataset, demonstrating tfate-of-the-art performance. Our approach surpasses the OMNIPHYSGS- based baseline in identifying material parameters and achieves significantly higher physical accuracy and visual fidelity in simulations.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Dynamic Reconstruction. Dynamic 4D reconstruction aims recover temporally varying, high- fidelity geometry and appearance from single or multi-view video. Implicit models, such as Neural Radiance Fields (NeRF) (Mildenhall et al., 2021), have been a foundational approach for novel-view synthesis. To handle motion, techniques extend NeRF with explicit deformation fields (Pumarola et al., 2021) or regularize them with priors on volume and topology (Park al., 2021a;b). While these implicit models excel at novel-view synthesis, they often yield noisy or poorly conditioned geometry, limiting downstream physical analysis (Li et al., 2023b). Later, the advent of 3D Gaus- sian Splatting (3DGS) (Kerbl et al., 2023) introduced a fast, explicit representation and catalyzed a wave of dynamic methods that either reconstruct each frame independently (Luiten et al., 2024; Wu et al., 2024a) or learn a canonical set of Gaussians that deform over time (Yang et al., 2024; Krati- menos et al., 2024; Wang et al., 2025). These approaches improve real-time rendering and geometric stability compared to purely implicit fields, but they typically do not encode physical laws.
Dynamic Simulation. The intersection of perception and physics has led to methods that infuse physical structure into generative pipelines. Approaches often couple text and video generative models with 3D representations to synthesize dynamic scenes (Bahmani et al., 2024; Ling et al., 2024: Ren et al., 2023; Singer et al., 2023). Other work treats Gaussian kernels as both visual and physical primitives, embedding Newtonian dynamics or constitutive behavior to enable constrained rendering and simulation (Xie et al., 2024; Liu et al., 2024a; Lin et al., 2025; Li et al., 2023b; Qiu et al., 2024; Borycki et al., 2024; Zhong et al., 2024; Fu et al., 2024). For instance, Gaussian Splashing (Feng et al., 2025) integrates position-based dynamics within 3DGS to handle solids, object fluids, and deformables. A complementary trend-motion conditione simulation trajectories using learned priors rather than explicit solvers (Li al.. 2024b; Geng et al., 2025; Wang et al., 2024: Wu et al., 2024b; Shi et al., 2024b). More recent "neural physics" approaches infer dynamics from video or generative priors to synthesize physically plausible motion without dedicated simulators (Zhang et al., 2024; Liu et al., 2024b; Huang et al., 2025; Feng et al., 2024; Tan et al., 2024). Despite promising results, many methods specialize to limited material families or rely on priors trained for visual fidelity rather than faithful mechanics, which hinders generalization.
System Identification From Videos. System identification in vision and robotics seeks to infer la- tent physical laws and material properties directly from visual observations (Li et al., 2023a: Liang et al., 2019; Raissi et al., 2019; Sundaresan et al., 2022; Li et al., 2022; Zhong et al., 2024; Zheng et al., 2024b), capability that underpins realistic simulation and effective robot interaction with de- formable and elasto-plastic objects (Shi et al., 2023; 2024a; Liang et al., 2024; Zheng et al., 2024a; Qiao et al., 2022). Classical approaches leverage explicit simulators such as FEM or mass cass-n systems (Takahashi & Lin, 2019; Wang et al., 2015), but they typically assume known geometry and limited material families. Data-driven alternatives learn dynamics or parameters from expe- rience (Sanchez-Gonzalez et al., 2020; Li et al., 2018; Xu et al., 2019), improving flexibility yet often struggling to generalize across unseen materials and conditions. Differentiable physics has narrowed this gap by enabling end-to-end gradient-based based estimation through simulators (Hu et al., 2019; Huang et al., 2021: Chen et al., 2022; Du et al., 2021: Geilinger et al., 2020; Heiden et al., 2021; Jatavallabhula et al., 2021; Ma et al., 2022; Qiao et al., 2021; Kaneko, 2024), though accu- rate modeling and priors remain crucial. Recent directions fuse neural representations with phy sics, either learning neural constitutive laws that augment expert simulators (Ma et al., 2023; Cao et al., 2024) or adopting hybrid formulations (e.g., MPM//pring construct geometry and identify material properties from video (Li et al., 2023b; Cai et al., 2024; Zhong et al., 2024; Shao et al., 2024). CoupNeRF(Li et al., 2024a) uses a hybrid approach that com- bines implicit NeRF representation with differentiable MPM to perform multi-object systemlI.
with 3D Gaussian splatting) to re-
3 METHOD
3.1 PROBLEM STATEMENT
We consider a scene with K deformable objects observed as multi-view RGB videos over T times- tamps. Each object may contain one material class drawn from a material set M. The goal is to recover (i) simulation-readdy continuum for all objects across all time and (ii) collection of per- material parameters θ = {O... mme such that a forward simulator reproduces the observed motion and predicts future motions over long horizons. The estimator uses only videos, camera calibra-
3

---

Published as a conference paper at ICLR 2026
Multi-Vin View Video
4DGS
Continuums
Novel Inferactions
AD Recon
Object-Specifit
Geometric Clues
Continuums
Material Point Method (MPM)
Silhouettes
Figure 2: Geometric reconstruction. From multi-view RGB videos, we reconstruct object geometry and disentangle material-specifie motion via optimizing 4D Gaussian Splatting (4DGS) with object masks. (2) Continuum simulation. The reconstructed Gaussians are lifted into object- specific continuums, which serve as the initial states for differentiable MPM. Geometry-aligner losses on surfaces and silhouettes drive physics parameter optimization under inter-material contact and friction. (3) Applications. The calibrated model generalizes to novel interaction scenarios, enabling physically faithful rollouts and long- Long-horizon predictions of complex multi-object dynamics.
tion, and instance masks extracted from the videos. Unlike single-object settings, our formulation estimates parameters independently for each object instance. That is, every object in the scene is equipped with its own set material parameters, which are optimized directly from its geometry and motion cues. This per-instance treatment avoids the need for pre-defined material sharing across objects, and enables our pipeline to handle multiple objects even under complex contact.
3.2 OVERVIEW
The pipeline Fig. proceeds in three stages. First, we reconstruct an object-aware dynamic Gaussian field from the multi-view video, with additional supervision from labeled 2D material masks that indicate the material type of each object. Second, following GIC (Cai et al., 2024), compact
lifting converts each object's reconstruction into simulation particle set, where particles carry positions. material-famil label, and shared material parameters. Third, starting from this particle state, the differenniable MPM is rolled out over the observed frames. Our geometry-aligned objectives then compare these simulated surfaces and silhouettes to those extracted from the reconstructed Gaussians (via per-object 3D Chamfer and 2D alpha-mask losses), and back-propagate through the MPM to jointly optimize the unknown physical parameters e.
3.2.1 PRELIMINARIES: MATERIAL POINT METHOD
We evolve a set of Q material points with a differentiable time-stepping map Zn+1 = T(z,;; (). n N.....N- - where 2,, (i), (1). F; ( contains position, ve- locity, and the elastic part of the deformation gradient step n, and N TT/r with simulation step size chosen so that N > T. Each step transfers particle states background grid, evaluates stresses an elastic law E to obtain the first Piola-Kirchhoff stress P., updates momenta and velocities the grid, and transfers back to particles. A plastic projection P maps the trial clastic tensor to an admissible F Grid-level contact and Coulomb friction resolve interactions between objects and materials. The resulting map T is fully differentiable with respect both state and parameters, thereby enabling efficient gradient- based identification and optimization.
3.2.2 PRELIMINARIES: DYNAMIC GAUSSIAN RECONSTRUCTION
We represent the scene with canonical Gaussian kernels that are warped in time by a low-rank deformation. Let Go {(14, r; )) denote kemel centers, isotropic scales, colors, and opacities.

---

Published as a conference paper at ICLR 2026
A pair of networks produces temporal bases and spatially varying gates, yielding for each time
Ne = p + Σ oi (8)
(t).
Vt +
Σ olb (1( (E()),
(1)
with 25/" E R2 and 2ig E R. We optimize photometric agreement across views,
min
II, + ASSIM (I, 12) + A,1lrell1s
(2)
ne.
where i, are rendered frames. Instance masks partition kernels by object; material masks, when available or synthesized, partition kernels by material. These labels are passed to the simulator.
3.3 GAUSSIAN-TO-CONTINNNN
LIFTING THE MULTI-ORJECT REGIME
Dynamic Gaussians optimized for rendering and are spatially nonuniform. We therefore derive simulation particles from thin occupancy field per object. For each object k. We generate a rough internal shape by randomly sampling particles within the bounding box of Gaussian points and retaining only those that align with the object's depth as rendered from multiple camera views. Then we construct density field that progressively increases resolution. In each iteration, we upsample the grid, smooths the field (mean filtering) to blur boundaries, and reassign high density voxels containing actual particles prevent the smoothing process from eroding the object's true shape. Finally, the specific object surface is isolated by applying a threshold to this high-resolution, refined density field.
Compared to single-object lifting, the multi-object setting requires two additional constraints. First, we enforce disjoint supports between objects at initialization by assigning overlapping voxels the nearest object surface and removing residual interpenetrations. Second, we maintain material labels on particles and ensure that per-object grids use a compatible resolution so that interfaces align at contact. The output is a set of particles (0) with per-pan per-particle object and material tags; we use these particles only for shape rendering and as the initial state for simulation.
3.4 MULTI-MATERIAL PARAMETERIZATION AND CONTACT
Each material m MM is associated with parameter vector 6n controlling its elastic, plastic, and viscous response. To reduce degrees of freedom while capturing inter inter-materin material behavior, we model Coulomb friction as an interface between materials m and m using a symmetric composition Mm,,m P(pmm Mmme with g(a, b) {a + b). although fully pairwise parameterization is also supported when annotations allow. We assign parameters on a per-object basis: each object instance k carries its own parameter vector 0k. which governs its clastic, plastic, and frictional response. Even if two objects correspond to the same real-world material, we do not impose parameter sharing; identifiability emerges from object- wise geometry and silhouette constraints under interaction. This per- per-instance treatment ensures flexibility when objects deform or respond differently under contact.
3.5 GEOMETRY-ALIGNEE OBJECTIVES FOR MULTI-OBJECT IDENTIFIATION
For each camera at observed times (t, [t we render silhouettes (t,)(ta) per object & and compare to target silhouettes (t,) We also compare simulated and extracted surfaces (t,) and (t,) via symmetric Chamfer distance. The overall objective is
m
K
LDD
(s.(())))...t
c (A)
(3)
m
j=
3.6 OPTIMIZATION IN THE MULTI-ORIECT SETTING
Training occurs in three stages. Stage reconstructs dynamic Gaussians and assigns instance par- titions using object masks. Stage converts each object's reconstruction into a simulation-ready continuum via Caussian-to-continuun lifting. Stage III optimizes the per-object parameter vectors K k=1 by minimizing equation 3 through MPM. To stabilize training, we adopt a horizon currieu- lum that gradually increases the rollout length alignment improves, and an alternating update strategy that interleaves parameter optimization with occasional re-synchronization of particle state reduce drift. All experiments follow this fixed schedule unless otherwise noted.
{0
5

---

Published as a conference paper at ICLR 2026
MOSIV MOSIV_Origual OOiignl
MOSIV_Now
MOSIV_ Orignal
MOSIV -ovel
MOSIV MOSIV_Orignil OOrggnal
MOSIV_Novel
T-O
T-12
T-17
T-26
Figure 3: Novel Interaction. Left-MOSIV original GT video sequence. Right-rollout after swapping object physics parameters while keeping initial conditions unchanged. Rows show time.
3.7 NOVEL INTERACTIONS
We enable multi-object novel interactions by varying initial conditions and material assignments. Novel interactions arise from changes in velocities, object placements, physical properties. Since existing datasets already span diverse velocity and impact settings, we focus on the material dimension. Specifically, for each sequence keep geometry, poses, and velocities fixed, and per- mute the identified per-object constitutive parameters create new material assignments. We then roll out the differentiable MPM predict the resulting dynamics. As shown Fig. these material swaps yield distinct yet physically plausible outcomes consistent with the reassigned stiffness, yield, and friction, demonstrating MOSIV's capacity to predict behaviors beyond observed interactions.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETTING
Datasets and evaluation protocol To evaulate our system identification methods, we generate a new multi-object dataset using the Genesis physics platform (Xian et al., 2024), an engine that supports simulation via the Material Point Method. This dataset is composed of 45 multi-view videos two-object interactions. Specifically, we use 10 unique geometries (egg, pawn, apple, bread, cream, barrel, potato, cushion, banana, mushroom), and materials (elastic, elastoplastic, liquid, sand, snow), and we assign each material to two objects. For each pair of objects, we intialize them to random position and rotation, then set the initial velocities such that the two objects collide at a specified time. The objects are then set in motion, collide, free-fall due to gravity, and eventually land flat table surface. We capture 30 frames of this interaction with 11 camera views evenly spaced around the hemisphere above the table. For enhanced photorealism, we use 10 different background environments, 12 different table textures, and realistic color for each object. We provide example views from some selected sequences in the Appendix.
Baselines We adapt OmniPhysGS-RGB the video-driven SysID setting as follows: (1) ini- tialize the reconstruction using the fused input point cloud at the first frame; (2) keep the decoder and expert-library architecture unchanged; (3) replace the original SDS objective with an image- space photometric loss equation 1. We further construct oracle variant of OmniPhysGS-RGD named with OmmiPhysGS-RGB w/ Oracle, to isolate the impact of material model selection. In- stead of requiring OmniPhysGS to infer the correct constitutive models, we directly provide it with the ground-trun -ruth per-object models while keeping the rest of the architecture and training setup un- changed. This gives upper-bound reference by removing the challenge of material identification.

---

Published as a conference paper at ICLR 2026
Inter- Muterial
Interaction
Intru-Muterial
Interuction F-F
Method
Averuge
E-P
E-F
E-S
P-F
F-S
E-E
P-P
S-S
OPGS (Lin et al., 2025)
27.63
27.01
24.46
26.24
26.80
24.89
26.84
29.79
23.59
24.72
25.93
PSNR
OPGS w/ Oracle (Lin et al., 2025)
25.37
24.62
23.26
23.81
25,98
23,36
25.06
25.86
22.53
24.52
24.39
MOSIV (Qurs)
30.89
30.29
26.57
32.21
29.07
29.88
27.96
36.16
35.16
26.87
30.51
OPGS (Lin et ul., 2025)
0.968
0.966
0.892
0.971
0.945
0.951
0.951
0.980
0.953
0.931
0.945
SSIM
OPGS w/ Oracle (Lin et al., 2025)
0,952
0.941
0.877
0.949
0.938
0.933
0.948
0.955
0.936
0.931
0.930
MOSIV (Ours)
0.983
0.982
0.945
0.986
0.973
0.977
0.970
0.992
0.987
0.971
0.977
OPGS (Lin et al., 2025)
11.10
3,931
27.97
2.692
10.16
8.165
23.82
1,030
7.281
13.85
11.79
CDJ
OPGS w/ Oracle (Lin et al., 2025)
33.24
81.98
46.09
91.28
17.33
43.18
10.35
77.54
43.82
3.01
43.50
MOSIV (Ours)
1,095
0.358
2,022
0,183
0.839
0.593
4,876
0.129
0.166
2.301
1.256
OPGS (Lin et ul., 2025)
0.085
0.078
0.135
0.069
0.085
0.092
0.134
0.052
0.105
0.104
0.095
EMD
OPGS w/ Oracle (Lin et al., 2025)
0.157
0.227
0.188
0.243
0.112
0.186
0.113
0.228
0.199
0.063
0.168
MOSIV (Ours)
0.043
0.041
0.069
0.028
0.047
0.052
0.064
0.012
0.033
0.103
0.049
Table 1: Observable state simulation on MOSIV Synthetic dataset. Columns are grouped by material-pair types. Material abbreviations: (elastic), P (plastic), F (fluid), S (sand).
Metries We report standard geometric and photometric measurements. Discrepancies between reconstructed and ground-truth point sets are measured by Chamfer Distance (CD) (Ma et al., 2020) (reported in 10a mm²) and Earth Mover's Distance (EMD). On sequences with reference renderings (e.g.. Spring-Gaus), we assess frame fidelity using PSNR (Hore & Ziou, 2010) and SSIM (Wang al., 2004) to quantify how well predicted states match future observations.
Implementation details Our dynamic Gaussian module follows the design in (Kratimenos et al., 2024) (motion backbone: 8 fully connected layers; 10 lightweight heads output per-basis residuals for centers and scales; coefficient network: 4 fully connected layers). Training uses the photomet- ric objective in Eq. equation 2. We employ compact object-wise occupancy refinement to obtain simulation particles; overlapping voxels at initialization are assigned to the nearest object surface to ensure disjoint supports, and per-ohject grids are aligned in resolution so that contact interfaces match. We adopt differentiable MPM simulator with time step T 1/4800 (200 substeps per 24 fis frame) and a grid resolution of 40963. Model parameters are optimized with the Adam opti- mizer, performing 80 iterations for initial velocity estimation followed by 200 iterations for physical property refinement. All experiments are conducted on an NVIDIA RTX A6000 GPU (48 GB).
4.2 QUANTITATIVE RESULTS COMPARISON
Observable state simulation. In Tab. 1, we present quantitative comparison between MOSIV, OmniPhysGS RGB w/ Oracle, and OmniPhysGS-RROn RGB on the task of observable state simulation. As shown in the table, MOSIV consistently and substantially outperforms both OmniPhysGS-RRGi and OmniPhysGS-RGGI w/ Oracle across all reported metrics. This highlights MOSIV 's ability to accurately reconstruct and predict object dynamics even in scenes that feature a wide variety of material properties and complex physical interactions. These results underscore the robustness and capacity of MOSIV for challenging real-world dynamics.
Future state simulation. In Tab. 2, we evaluate MOSIV challenging tasks of future state sim- ulation, where the objective is to forecast long-term scene evolution beyond the observed frames. MOSIV clearly surpasses OmniPhysGS- RGB and OmniPhysGS-RRG. RGB w/ Oracle across all reported metries, highlighting its capability to anticipate object trajectories under complex physical interac- tions and diverse material compositions. This improvement reflects the method's accurate system identification, effectively inferring each object's geometry, dynamic behavior, and underlying physi- cal properties. Such understanding of both scene structure and physics enables MOSIV to generalize well, delivering reliable predictions in previously unseen and physically intricate scenarios. 4.3 QUALITATIVE RESULTS COMPARISON
Observable and Future state simulation. Fig. 4 compares Ground Truth, MOSIV, OmniPhysGS- RGB w/ Oracle, and OmniPhysGS GGGGGGGG-GBB---- on two representative scenes plastcine ffuid (P-F) and sand-sand (S (S-S). Across the observed frames, MOSIV better preserves object geometry and con- tact boundaries: fluids do not over-spread, sand clusters remain compact, and plastic bodies retain plausible deformation. Baselines show blur, shape erosion, and contact leakage. In the predicted
7

---

Published as a conference paper at ICLR 2026
Mater-aatrrial Interaction
Intra Material Interaction
Method
Average
E-P
E-F
E-S
P-F
P-5
F-S
E-E
P-P
F-F
S-S
SSIM PSNR
OPGS (Lin et ol,, 2025)
20.65
20.68
16.40
20.26
19.97
18.03
18:40
25.19
21.48
16.31
19.00
OPGS w/ Orucle (Lin et al., 2025)
19.58
18.91
15.82
17.83
19.12
16.43
17.89
21.14
18.91
18.37
17.97
(Qurs)
OPGS (Lin et al., 2025)
0.947
0.941
0.741
0.954
0.895
0.914
0.902
0,970
0.942
0.847
0.888
Oracle (L.in at2,,
0.916
0.721
0.927
0.939
0.895
MOSIV (Ours)
0.970
0.975
0.891
0.980
0.966
0.971
0.942
0.994
0.984
0.956
0.963
OPGS (Lin 2025)
5.620
2.181
CDJ
OmniPhysOS w/Oraele (Lin al., 2025)
151.76
455.23
235.67
448.43
53.49
200.27
38.33
461.80
279.00
12.09
215.83
MOSIV (Ours)
5,939
0.532
9.31
0.255
1,151
1.141
16.59
0.132
0.183
1.867
3.710
OPGS (Lin et al,, 2025)
0.258
0.161
0.346
0.101
0.144
0.155
0.396
0.082
0.123
0.207
0.199
BMD
OPGS w/ Orucle (Lin et., 2025)
0.361
0.600
0.497
0.600
0.211
0.456
0.260
0.600
0.500
0.123
0.408
MOSIV (Ouns)
0.081
0.048
0.135
0.029
0.062
0.061
0.140
0.019
0.035
0.102
0.071
Table 2: Future state simulation on MOSIV Synthetic dataset. Columns are grouped by material- pair types. Material abbreviations: (elastic), (plastic), F (fluid), (sand).
Ground Truth
MOSIV
OPGS Ornele
OPGS
Ground Truth
MOSIV
OPGS w/ Onclc
Oburved Framiss
Prodiced Faaees
Figure 4: Qualitative comparison of multi-object interactions. The first four columns shows plasticine-flidd (P-F) example; the last four columns shows sand-sand (S-S) example.
frames of Fig. MOSIV sustains stable long-horizon rollouts: collision timing and post-impact trajectories remain consistent with Ground Truth. OmmiPhysGS-RGG variants drift over time: Flu- ids overshoot and sands disperse unrealistically. indicating weaker identification of the system and contact handling. Fig. 5 compares Ground Truth, CoupNeRF" and MOSIV on two highly dynamic scenes with significant scene, CeUPN does not produce the correct physics: both plasticine and sand behave like viscous fluid, losing the expected distinction between granular flow for sand and plastic deformation for plasticine. In the E-P scene, CoupNeRF F* also deviates from the correct behavior and the ap- pearance is also distorted. By contrast, MOSIV preserves the expected material -speciffc dynamics and maintains consistent appearance across frames.
cefformationspplattee plastcine-sante sand (P-S) and plastic-plastic (E-P). In the P-S
Trajectory comparison. Fig. visualizes particle trajectories. MOSIV streamlines align tightly with Ground Truth, showing coherent paths through contact events and minimal accumulation enror. In contrast, OmniPhysGS-Ree BBGBB and OmniPhysGS-RGii w/ Oracle produce fragmented or biased paths and increasing drift. Overall, the qualitative results mirror the quantitative trends: MOSIV more faithfully captures contact-rich, multi-material dynamics over long horizons.
4.4 ABLATION: OBJECT-AWARE SUPERVISION VS. SCENE- ICENE-WISE SUPERVISION
central challenge the multi multi-object regime association ambiguity at contact: nearest- neighbo geometry and silhouette losses computed the union of objects spuriously explain simulated
8

---

Published as a conference paper at ICLR 2026
CoupNeRF
CoupNeit
Olberee Famnes
Prediced Frances
:
Figure 5: Qualitative comparison between MOSIV and baselines.
Ground
OPGS Groud
OPGS
Figure 6: Qualitative comparison of trajectory. The trajectories illustrate how well each method captures long-term dynamics.
point object & by ground-truth point object k when the two bodies touch interpenetrate projection. This cross-object borrowing hides parameter miscalibration (e.g., an overly soft deforming into k') and produces optimistic rollouts.
Scene-wise losses (naive). Let (t) and (t) denote the union of simulated and ground- truth surface samples at time t. The global Chamfer loss, celphal (t) d(psim(t), (t)) + d (t), (t)) with d(.,-) -) the one-sided nearest-neighbor distance, admits cross-object matches at contact. Similarly, a single alpha-mask loss per view, Cglobal (t, (() (t) is blind to which object explains which pixels.
Object-wise losses (ours). We enforce supervision at the object level to preserve identities through contact. Let paim (L) and (() be the simulated and target samples for object k; and (1). (t) the per-object silhouettes at view Our geometry loss sums disjoint Chamfer distances: K ceb (t) k= d(prim (t), P (t)) + d(Pp (t), Prim (t)) and the silhouette loss aligns each ob- (1))(()) ||- This prevents the optimizer from trading deformation on one object against another to satisfy global loss, yiclding sharper gradients for contact mechanics and materially correct parameter updates. In practice, find that object-aware supervision is crucial during impact and stick-slip transitions, where scene wise losses can be min- imized by swapping
ject separately: co [ (1,
stiffness across bodies in projection.
Ablation Results. We conduct the experiment on subset of MOSIV, selecting six scenes total, from each inter-material interaction type. Table shows that replacing scene-wise supervision

---

Published as a conference paper at ICLR 2026
Supervision Granularity
LCD
La
PSNR t
SSIM f
CD 4
EMD
x
26.59
0.964
53.21
0.132
Scene-wise wise losses (naive)
/
27.59
0.959
40.29
0.119
/
27.89
0.968
22.13
0.091
X
V
30.18
0.975
0.985
0.045
Object-aware losses (ours)
29.86
0.975
1.17
0.043
30,24
0.977
0.696
0.041
Table 3: Supervision granularity ablation. Comparison of scene-wise vs. object-wise supervision while toggling the Chamfer term LCD and silhouette term La.
with object-wing wise losses leads to improvements across all metrics. Visual fidelity improves, with re- constructions better aligned to the ground truth, while geometric distances decrease substantially. In particular, the large Chamfer Distance observed with scene-wise losses reflects unstable simulation training and inaccurate contact handling. whereas object-wise supervision corrects this, yielding robust optimization and physically meaningful rollouts. In addition, the results demonstrate that single-source supervision is insufficient for robust physical property training.
DISCUSSION
Despite promising results, our approach has several limitations. It relies on predefined constitutive models and could benefit from directly learning physical models (e.g., via neural networks) han- materials with unknown properties (Ma et al., 2020; Cao et al., 2024; Zhao et al., 2025). The optimization computationally intensive and sensitive initial geometry, motivating more efficient strategies and more robust 3D reconstruction, particularly in cluttered scenes with occlusions. Ex- tending the framework from controlled settings real-world videos with complex lighting and noise also remains challenging, requiring further efforts to bridge the sim-to-real gap.
CONCLUSION
We introduce the challenging problem of multi-object system identification from videos and present MOSIV, which serves both as framework for reconstructing objects' geometry, dynamics, and physical properties and as a comprehensive synthetic dataset for rigorous evaluation. MOSIV re- constructs object-aware Gaussians from multi-view video observations and integrates geometry- driven supervision with a differentiable simulator to recover object geometry, dynamics, and physi- cal properties. This formulation moves beyond prior single-object discrete material classification methods, enabling physically grounded scene reconstruction and prediction. Extensive experiments synthetic benchmark show that MOSIV achieving accurate observable dynamics and strong generalization to future state over complex and diverse scenes.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENT
This work was supported in part by U.S. NSF DBI-2238093. YZ was supported in part by the SoftBank Group-ARM, Fellowship.
REFERENCES
Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park. Andrea Tagliasacchi, and David B Lindell. 4d-fy: Text-to-4d generation using hybrid score distillation sampling. In CVPR, 2024.
Piotr Borycki, Weronika Smolak, Joanna Waczyáska. Marcin Mazur, Stawomir Tadeja, and Przemyslaw Spurek. Gasp: Gaussian splatting for physic-based simulations. arXiv preprint DxXiv:2409.058// 2024.
Junhao Cai, Yuji Yang, Weihao Yuan, Yisheng He, Zilong Dong, Liefeng Bo, Hui Cheng, and Qifeng Chen. Gic: simulation. Advances in Neural Information Processing Systems, 37:75035-75063, 2024. 2, 3,
continuum for physical property identification and
Junyi Cao, Shanyan Guan, Yanhao Ge, Wei Li, Xiaokang Yang, and Chao Ma. Neuma: Neural material adaptor for visual grounding of intrinsic dynamics. NeurIPS, 2024. 3, 10
Hsiao-yu Chen, Edith Tretschk, Tuur Stuyck, Petr Kadlecek, Ladislav Kavan, Etienne Vouga, and Christoph Lassner. Virtual elastic objects. In CVPR, 2022. 3
Alban De Vaucorbeil, Vinh Phu Nguyen, Sina Sinaie, and Jian Ying Wu. Material point method after 25 years: Theory, implementation, and applications. Advances in applied mechanics, 2020.
Tao Du, Kui Wu, Pingchuan Ma, Sebastien Wah, Andrew Spielberg, Daniela Rus, and Wojeiech Matusik. Diffpd: Differentiable projective dynamics. ACM TOG, 2021. 2,
Yutao Feng. Yintong Shang, Xuan Li, Tiajjia Shao, Chenfanfu Jiang, and Yin Yang. Pie-nerf: Physics-based interactive elastodynamics with nerf. In CVPR, 2024. 3
Yutao Feng, Xiang Feng, Yintong Shang, Ying Jiang, Chang Yu, Zeshun Zong, Tianjia Shao, Hongzhi Wu, Kun Zhou, Chenfanfu Jiang, al. Gaussian splashing: Unified particles for versatile motion synthesis and rendering. In Proceedings of the computer vision and pattern recognition conference, pp. 518-529, 2025.
Zhoujie Fu, Jiacheng Wei, Wenhao Shen, Chaoyue Song, Xiaofeng Yang. Fayao Liu, Xulei Yang, and Guosheng Lin. Sync4d: Video guided controllable dynamics for physics-based 4d generation. arXiv preprint arXiv:2405. 16849, 2024.
Moritz Geilinger, David Hahn, Jonas Zehnder, Moritz. Bächer, Bernhard Thomaszewski, and Stelian Coros. Add: Analytically differentiable dynamics for multi-body systems with frictional contact. ACM TOG, 2020. 2,
Daniel Geng, Charles Herrmann, Junhwa Hur, Forrester Cole, Serena Zhang, Tobias Pfaff, Tatiana Lopez-Guevara, Yusuf Aytar, Michael Rubinstein, Chen Sun, et al. Motion prompting: Controlling video generation with motion trajectories. In Proceedings of the Computer Vision
and Pattern Recognition Conference, pp. 1-12, 2025. 3
Eric Heiden, Miles Macklin, Yashraj Narang, Dieter Fox, Animesh Garg, and Fabio Ramos. Disect: A differentiable simulation engine for autonomous robotic cutting, arXiv preprint vrXiv:2105.12244 2021.
Alain Hore and Djemel Ziou. Image quality metries: Psnr vs. ssim. 2010 20th international conference on pattern recognition, 2010. 7
11

---

Published as a conference paper at ICLR 2026
Yuanming Hu, Yu Fang, Ziheng Ge, Ziyin Qu, Yixin Zhu, Andre Pradhana, and Chenfanfu Jiang. A moving least squares material point method with displacement discontinuity and two-way rigid body coupling. ACM TOG, 2018. 2
Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan- Kelley, and Frédo Durand. Difftaichi: Differentiable programming for physical simulation. arXiv preprin! 1eXiv:191000099: 2019. 3
Tianyu Huang, Haoze Zhang, Yihan Zeng, Zhilu Zhang, Hui Li, Wangmeng Zuo, and Rynson WH Lau. Dreamphysics: Learning physies-based 3d dynamics with video diffusion priors. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 3733-3741, 2025.
Zhiao Huang, Yuamming Hu, Tao Du, Siyuan Zhou, Hao Su, Joshua Tenenbaum, and Chuang Gan. Plasticinelab: soft-body manipulation benchmark with differentiable physics. arXivv preprint arXiv:2104.033// 2021. 3
Krishna Murthy Jatavallabhula, Miles Macklin, Florian Golemo, Vikram Voleti, Linda Petrini, Martin Weiss, Breandan Considine, Jérôme Parent-Lévesque, Kevin Xie, Kenny Erleben, et al. gradsim: Differentiable simulation for system identification and visuomotor control. arXiv preprint Xiiv:2104.0264: 2021. 3
Chenfanfu Jiang, Craig Schroeder, Joseph Teran, Alexey Stomakhin, and Andrew Selle. The material point method for simulating continuum materials. ACM SIGGRAPH 2016 Courses, 2016. 2
Takuhiro Kaneko. Improving fllysics-aummente continuum neural radiance field-based crometry-agnostie system identification with lagrangian particle optimization. In CVPR, 2024.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM TOG, 2023.
Gergely Klár, Theodore Gast, Andre Pradhana, Chuyuan Fu, Craig Schroeder, Chenfanfu Jiang, and Joseph Teran. pragerr-prager elastoplasticity for sand animation. ACM Transactions on Graphics (TOG), 35(4): 1-12, 2016. 2
Agelos Kratimenos, Jiahui Lei, and Kostas Daniilidis. Dynmf: Neural motion factorization for real-time dynamic view synthesis with 3d gaussian splatting. ECCV, 2024. 3,
Jin Li, Yang Gao, Song Wenfeng, Yacong Li, Shuai li, Aimin Hao, and Hong Qin. Coupnerf: Property-aware neural radiance fields for multi-material coupled scenario reconstruction. Computer Graphics Forum, 43, 10 2024a. doi: 10. w..11.//cg..... 15208. 2, 3,
Jinxi Li, Ziyang Song, and Bo Yang. Nvfi: Neural velocity fields for 3d physics learning from dynamic videos. Advances in Neural Information Processing Systems, 36:34723-3477 2023a. 2,
Xuan Li, Yi-Ling Qiao, Peter Yichen Chen, Krishna Murthy Jatavallabhula, Ming Lin, Chenfanfu Jiang, and Chuang Gan. Pac-nerf: Physics augmented continuum neural radiance fields for geometry-agnostic system identification. arXiv preprint arXiv:2303.05512 2023b. 3
Yifei Li, Tao Du, Kui Wu, Jie Xu, and Wojciech Matusik. Diffeloth: Differenniablee cloth
simulation with dry frictional contact. ACM TOG, 2022. 2,
Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B Tenenbaum, and Antonio Torralba. Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids. arXiv preprint arXiv: 1800000600 2018. 3
Zhengqi Li, Richard Tucker, Noah Snavely, and Aleksander Holynski. Generative image dynamics. In CVPR, 2024b.
Junbang Liang, Ming Lin, and Vladlen Koltun. Differentiable cloth simulation for inverse problems. NeurIPS, 2019. 2,3
12

---

Published as a conference paper at ICLR 2026
Xiao Liang, Fei Liu, Yutong Zhang, Yuelei Li, Shan Lin, and Michael Yip. Real-to-sim deformable object manipulation: Optimizing physics models with residual mappings for robotic surgery. In ICRA, 2024. 3
Yuchen Lin, Chenguo Lin, Jianjin Xu, and Yadong Mu. Omniphysgs: 3d constitutive gaussians for general physics-based dynamics generation. arXiv preprini arXiv:2501. 189822 2025. 2, 3, 7, 8, 4,
Huan Ling, Seung Wook Kim, Antonio Torralba, Sanja Fidler, and Karsten Kreis, Align your gaussians: Text-to-4d with dynamic 3d gaussians and composed diffusion models, In CVPR, 2024. 3
Fangfu Liu, Hanyang Wang, Shunyu Yao, Shengjun Zhang. Jie Zhou, and Yueqi Duan. Physics3d: Learning physical properties of 3d gaussians via video diffusion. arXiv preprint 04f1:224060033388 2024a.
Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shenlong Wang. Physgen: Rigid-body physics-grounded image-to-video generation. In ECCV, 2024b.
Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In 2024 International Conference on 3D Vision
(3DV), pp. 800-809, IEEE, 2024.
Baorui Ma, Zhizhong Han, Yu-Shen Liu, and Matthias Zwicker. Neural-pull: Learning signed distance functions from point clouds by learning to pull space onto surfaces. arXiv preprint 1xXi::2011.13495, 2020. 7, 10
Pingchuan Ma, Tao Du, Joshua B Tenenbaum, Wojeiech Matusik, and Chuang Gan. Risp: Rendering-invant invariant state predictor with differentiable simulation and rendering for cross-domain parameter estimation. arXiv preprint arXiv:2205.0560 2022. 3
Pingchuan Ma, Peter Yichen Chen, Bolei Deng, Joshua Tenenbaum, Tao Du, Chuang Gan, and Wojciech Matusik. Learning neural constitutive laws from motion observations for generalizable pde dynamics. In ICML, 2023.
Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 2021. 3
Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz, and Ricardo Martin- Bauain-Brualla. Nerfies: Deformable neural radiance fields. ICCV, 2021a.
Keunhong Park. Utkarsh Sinha, Peter Hedman, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualle and Steven Seitz. Hypernerf: A higher-dimensional representation for topologically varying neural radiance fields. arXiv preprint arXiv:2106. 13228, 2021b.
Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francese Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. In CVPR, 2021. 3
Yi-Ling Qiao, Alexander Gao, and Ming Lin. Neuphysics: Editable neural geometry and physics from monocular vidcos. NeurIPS, 2022.
Yiling Qiao, Junbang Liang, Vladlen Koltun, and Ming Lin. Differentiable simulation soft multi-body systems. NeurIPS, 2021. 2,
Ri-Zhao Qiu, Ge Yang, Weijia Zeng, and Xiaolong Wang. Language-driven physics-based based scene
synthesis and editing via feature splatting. In ECCV, 2024.
Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physies-informet neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 2019. 2,
13

---

Published as a conference paper at ICLR 2026
Jiawei Ren, Liang Pan, Jiaxiang Tang, Chi Zhang, Ang Cao, Gang Zeng, and Ziwei Liu. Dreamgaussian4d: Generative 4d gaussian splatting. arXiv preprint arXiv:2312.17142. 2023. 3
Alvaro Sanchez- Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter
Battaglia. Learning to simulate complex physics with graph networks. In JCML, 2020. 3
Yidi Shao, Mu Huang, Chen Change Loy, and Bo Dai. Gausim: Registering elastic objects into digital world by gaussian simulator. arXiv preprint arXiv:2412. 17804, 2024. 3
Haochen Shi, Huazhe Xu, Samuel Clarke, Yunzhu Li, and Jiajun Wu. Robocook: Long-horizon clasto-plastic object manipulation with diverse tools, arXiv preprint arXiv:2306. 2023.
Haochen Shi, Huazhe Xu, Zhiao Huang, Yunzhu Li, and Jiajun Wu. Robocraft: Learning to see, simulate, and shape elasto- -lasttc objects in 3d with graph networks. The International Journal of Robotics Reseurch, 2024a. 2,
Xiaoyu Shi, Zhaoyang Huang, Fu-Yun Wang, Weikang Bian, Dasong Li, Yi Zhang, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, et al. Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling, In ACM SIGGRAPH 2024 Conference Papers, 2024b.
Uriel Singer, Shelly Sheynin, Adam Polyak, Oron Ashual, lurii Makarov, Filippos Kokkinos, Naman Goyal, Andrea Vedaldi, Devi Parikh, Justin Johnson, al. Text-to-4d dynamic scene generation. arXiv preprint arXiv:2301. /280, 2023. 3
Priya Sundaresan, Rika Antonova, and Jeannette Bohgl. Diffcloud: Real-to-sim from point clouds with differentiable simulation and rendering of deformable objects. In IROS, 2022.
Tetsuya Takahashi and Ming C Lin. Video-guided real-to-virtual parameter transfer for viscous fluids. ACM TOG, 2019.
Xiyang Tan, Ying Jiang, Xuan Li, Zeshun Zong, Tianyi Xie, Yin Yang, and Chenfanfu Jiang. grounded dynamics from single image. arXiv preprint 1rXiv:2411.171899 2024. 3
Physmotion:
Bin Wang, Longhua Wu, KangKang Yin, Uri M Ascher, Libin Liu, and Hui Huang. Deformation capture and modeling of soft objects. ACM TOG, 2015. 3
Xiaoyuan Wang, Yizhou Zhao, Botao Ye, Xiaojun Shan, Weijie Lyu, Lu Qi, Kelvin CK Chan, Yinxiao Li, and Ming-Hsuan Yang. Holigs: Holistic gaussian splatting for embodied view synthesis. In NeurlPS, 2025. 3
Zhou Wang. Alan C Bovik, Hamid R Sheikh, and Ecro P Simoncelli. Image quality assessment: from error visibility structural similarity. IEEE TIP, 2004. 7
Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Papers, 2024. 3
Guanjun Wu, Taoran Yi, Jiemin Fang. Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In CVPR. 2024a.
Weijia Wu, Zhuang Li, Yuchao Gu, Rui Zhao, Yefei He, David Junhao Zhang, Mike Zheng Shou, Yan Li, Tingting Gao, and Di Zhang. Draganything: Motion control for anything using entity representation. In ECCV, 2024b. 3
Zhou Xian, Yiling Qiao, Zhenjia Xu, Tsun-Hsuan Wang, Zhehuan Chen, Juntian Zheng, Ziyan Xiong, Yian Wang, Mingrui Zhang, Pingchuan Ma. Yufei Wang, Zhiyang Dou, Byungchul Kim, Yunsheng Tian, Yipu Chen, Xiaowen Qiu, Chumru Lin, Tairan He, Zilin Si, Yunchu Zhang, Zhanlue Yang, Tiantian liu, Tianyu Li, Kashu Yamazaki, Hongxin Zhang, Huy Ha, Yu Zhang, Michael Liu, Shaokun Zheng, Zipeng Fu, Qi Wu, Yiran Geng, Feng Chen, Milky Yuanming Hu, Guanya Shi, Lingjie I. Liu, Taku Komura, Zackory Erickson, David Held, Minchen Li, Linxi Jim
14

---

Published as a conference paper at ICLR 2026
Fan, Yuke Zhu, Wojeiech Matusik, Dan Gutfreund, Shuran Song, Daniela Rus, Ming Lin, Bo Zhu, Katerina Fragkiadaki, and Chuang Gan. Genesis: A generative and universal physics engine for robotics and beyond, 2024. URL
https: / / thub. com/ Stom//enee Genest s Embodiedd /
Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. Physgaussian: Physics-integrated 3d gaussians for generative dynamics. In CVPR. 2024.
Zhenjia Xu, Jiajun Wu, Andy Zeng, Joshua B Tenenbaum, and Shuran Song. Densephysnet: Learning dense physical object representations via multi-step dynamic interactions. arXiv preprint arXiv: 03853, 2019. 3
Ziyi Yang. Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. In CVPR, 2024. 3
Hang Yin, Anastasia Varava, and Danica Kragic. Modeling, learning, perception, and control
methods for deformable object manipulation. Science Robotics, 2021. 2
Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Y Feng, Changxi Zheng, Noah Snavely, Jiajun Wu, and William T Freeman. Physdreamer: Physics-basee baysic interaction with objects via video generation. In European Conference Computer Vision, pp. 388-406. Springer, 2024. 3
Yizhou Zhao, Haoyu Chen, Chunjiang Liu, Zhenyang Li, Charles Herrmann, Junhwa Hur, Yinxiao Li, Ming-Hsuan Yang, Bhiksha Raj, and Min Xu. Masiv: Toward material-agnostic system identification from videos. arXiv preprint arXiv:2508.04 /100000/11000 /2, 2025. 2, 10
Dongzhe Zheng, Siqiong Yao, Wenqiang Xu, and Cewu Lu. Differentiable cloth parameter identification and state estimation in manipulation. IEEE Robotics and Automation Letters, 2024a.
Yang Zheng, Qingqing Zhao, Guandao Yang, Wang Yifan, Donglai Xiang, Florian Dubost, Dmitry Lagun, Thabo Beeler, Federico Tombari, Leonidas Guibas, and Gordon Wetzstein. Physavatar: Learning the physics of dressed 3d avatars from visual observations. In European Conference on Computer Vision (ECCV), 2024b. 3
Licheng Zhong, Hong-Xing Yu, Jiajun Wu, and Yunzhu Li. Reconstruction and simulation of elastic objects with spring-mass 3d gaussians. In European Conference on Computer Vision, pp. 407-423. Springer, 2024.
15

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 PHYSICAL PARAMETERS.
Our differentiable MPM implementation supports five standard material classes: clastic solids, plasticine, granular media (e.g., sand), Newtonian fluids, and non-Newtonian fluids. For each class, we optimize a small set of physically interpretable parameters:
Elasticity: Young's modulus (E) controlling material stiffness, and Poisson's 's ratio (и)
controlling volume preservation under deformation.
Plasticine: Young's modulus (E), Poisson's ratio (2)) and yield stress (Ty), which
determines the stress level at which permanent (plastic) deformation occurs.
Newtonian fluid: fluid viscosity (μ)) governing resistance to velocity changes, and bulk modulus (k), governing volume preservation.
Non-Newtonian fluid: shear modulus (μ), bulk modulus (s), yield stress (7y) ), and plastic viscosity (17), which encodes the decayed, time-dependent resistance to yielding. Sand: friction angle (ftric), which determines the stable slope of a sand pile and controls shear resistance granular flow.
A.2 PHYSICAL MODELS
MPM can be paired with a broad family of constitutive and plasticity models. the continuum formulation, internal forces are expressed via the Cauchy stress T, a tensor field defined as a function of the deformation gradient The deformation gradient is tracked on MPM particles measure their distortion relative to the rest configuration. For plasticity, F is constrained to an elastic region, and return mapping Z projects F back to the yield surface when this region violated.
Elasticity.
We use a neo-Hookean model for elastic solids. The Cauchy stress is given by
(F(F) μ (FFT) (Alog(J) (1) I,
(4)
where = (F), and μ, X are Lamé parameters related Young's modulus E and Poisson's ratio y via
E A= # 2(1+
vE
(5)
(1+ 2v)
We model Newtonian fluids using J-based volumetric term combined with Newtonian fluid. viscous term:
Vv + (
TT(F) = 1/m (vv Vv
(6)
where v is the velocity field, μ is viscosity. and x is the bulk modulus.
Plasticine. Plasticine is modeled using a St. Venant-Kirchhoff (StVK) elastic model combined with von Mises plastic return mapping. The elastic stress is
JT(F) U (2ue xTr(E)) UT,
where F T is the SVD of F and e = log Σ) is the Hencky strain.
The von Mises yield condition is
TY
87 ||||
0,
(8)
2μ
where é is the deviatoric Hencky strain and ry is the yield stress. When d7 > 0, the deformation exceeds the clastic region, and the deformation gradient is projected back to the yield surface via the return mapping
87 < O, UUexpe - 8T VT otherwise.
z(F)

---

Published as a conference paper at ICLR 2026
Sand. Granular materials (sand) are modeled using Drucker Prager yield criterion (Klár et al.,
Brucker-Praggr 2016) with an underlying StVK elastic model. The yielding conditions are
tr(e) > or 87 = |ēllæ + (d2 + 2/)) tr(c) > 0,
(10)
2/
where
2 sin Ofric
(11)
sin Pteic
and Ofiic is the friction angle. The corresponding return mapping is
UV 2(F) = F
tr(e) > 0, by 0, tr(c) - O, otherwise.
(12)
U Uexp(E- - ( ) V
A.3 GENESIS MULTI-OBJECT DATASET PHYSICAL PARAMETERS
Figs. and shows a sample from the Genesis Multi-Object dataset. We parameterize each material by set of physical attributes, and assign every object the parameter values associated with its material label. Some of these parameters are selected uniformly at random from a range to increase variation in the dataset, and not all parameters used by each material. The numerical details are shown below.
Young's s Modulus (E)t
Poisson's 's Ratio (v)
Density (p)
Elastic
[4.75 x 10 x 10*]
200 30]
[800, 1200)
Elastoplastic
[4.75 x 104,5.25 5.25 x 10a]
[20, 30]
[800, 1200]
Snow
[4.75 104, 5.25 x 104]
[20, 30]
[800, 1200]
Liquid
4.75 x 10 5.25 x 10a]
[20, 30]
[800, 1200]
Shear Modulus (p))
Yield Stress Range (TY)
Friction Angle (0)
Elastic
Elastoplastic
2.5 10 10-2,4.5 10-2
Snow
2.5 x 10 10-2, 4.5 10 10-21
Liquid
[2.4 x 10,,3.6 3.6 x 109)
Table 4: Physical parameter values for objects in the Genesis Multi-Object Dataset (10 Geometry Shapes)
f indicates that the value is selected uniformly at random from the given range.
A.4 MATERIAL POINT METHOD
The Material Point Method (MPM) (Jiang et al., 2016) hybrid Sulerian-Lagraneee method for simulation the behavior of continuum materials, based on its physical parameters. MPM represents continuum as particles, each with its own physical parameters. interacting with background grid from which governing physics laws are solved to obtain an update, and then the updates at grid locations propagate back the particles. Although (Jiang et al., 2016) covers the mathematical derivations in detail, we provide brief overview of the method and the most relevant equations.
A.4.1 INTTIALIZATION
The continuum is first represented by a set of discrete particles. We enumerate the particles as
= (1, 2,..., P} where P is the total number of particles. Each particle E P, initialized with starting position zi.. velocity of. mass my. volume 1,- deformation gradient F30 and material parameters. Each particle also stores an affine matrix By The grid is also initialized with grid points g 21,2,.. ..., G} where G is the total number of grid locations.
2

---

Published as a conference paper at ICLR 2026
Figure 7: MOSIV Dataset (2 Objects) Example.
Figure 8: MOSIV Dataset (3 Objects) Example.
A.4.2 PARTICLE TO GRID TRANSFER
At this stage, particle properties are transferred to grid locations to perform physics calculations. Each grid point stores a mass and momentum. At timestep grid point has mass:
mf
Σ
w m
p
PEP
and momentum
mfof = Σ
wty my (o/ + ()( -'(f)))"(((0) - (zf1=z;))
pep
where
D
Σ
w (x - 2 =x,)"
NEG
3

---

Published as a conference paper at ICLR 2026
Grid velocities are caleulated by dividing momentum by mass. In the case that the mass is zero, the velocity is instead set to O.
A.4.3 COMPUTING GRID FORCES AND VELOCITY UPDATE
Now, the force acting upon each grid point result of elastic stresses from nearby particles is calculated for grid point i at timestep as:
OV OF (F.) (2)
Σ PEP
V P
(F2) Vic
Here, Ft is the deformation gradient time for particle p and stress tensor that particle. Now, we update the velocity at each grid point
0V
3F represents the first
Piola-Kine Kirchhoff Piola-Kirchhoff with:
oftz u,t At (x))mmm
In this step, boundary conditions and collisions are also taken into account and resolved.
A.4.4 GRID TO PARTICLE TRANSFER
Now that the relevant values have been computed at the grid points, we can propagate these changes back the particles by updating their deformation gradients as follows for particle p timestep t:
F++1
I
At
(Vwi FØ
iEg
At this stage, updates to each particle's velocity and B affine matrix are also calculated. The velocity calculated as:
1+1
w
Egg
The affine matrix is updated as:
B++1
Σ
w w of - x
ieg
A.4.5 PARTICLE UPDATE
Finally, with the updated velocities, the particle locations can now also be updated as follows for particle p timestep t:
x = orf + Ato
Finally, with the updated velocities, the particle locations can now also be updated as follows for particle p timestep l:
++1 = 2 + Ato
A.5 MORE QUANTITATIVE RESULTS ON MOSIV DATASET
In Tab. report an average performance comparison of MOSIV and all baselines on the MOSIV dataset. Across all reported metrics, MOSIV consistently outperforms both OPGS and CoupNeRF" indicating sharper, more structurally faithful reconstructions and more accurate geometric distributions. These gains persist in the both observable and future regime, underscoring robust system identification that generalizes beyond observed frames.
Ohservable Simulation
Future Simulation
Method
PSNR
SSIM
CD 4
EMD
PSNR
SSIM t
CD
EMD |
OPGS(Lin et al., 2025)
25.93
0.945
11.79
0.095
19.00
0.888
51.92
0.199
CoupNeRF" (Li et al., 2024a)
25.88
0.962
0.927
0.047
20.62
0.942
1.859
0.062
MOSTV (Ours)
31.17
0.975
0.389
0.033
28.92
0.964
0.894
0.050
Table 5: Avg. Performance Comparison on MOSIV dataset. for reproduced implementation.
4

---

Published as a conference paper at ICLR 2026
A.6 COMPUTATION OVERHEAD COMPARISON
For single 30-frame sequence (same resolution and views as in our main experiments), we measure the average training time and peak GPU memory. Specifically, MOSIV and CoupNeRF are both trained on a single NVIDIA A6000 (48 GB). contrast, OPGS cannot trained on an A6000 due to out-of out-of-memoot (OOM) and therefore requires NVIDIA H100 (80 GB). As shown in Tab. 6, despite running on a less powerful GPU, MOSIV consitently achieves the lowest runtime and peak memory.
Metric Method
CoupNeRF*
OPGS
MOSIV
Training Time (s) +
9591.09
5263.70 61.08
5021.46
Peak GPU Memory (GB) +
31.14
29.79
Table 6: Runtime and memory cost
A.7 EXPERIMENT RESULTS ON MOSIV DATASET EXTENSION (3 OBJECTS)
We augment MOSIV with three-object interaction benchmark to further stress complex, contact-rich dynamics. As shown Tab. 7, across this more challenging setting, our approach demonstrates consistently strong quantitative results.
Observable Simulation
Future Simulation
Method
PSNR f
SSIM t
CD 4 EMD
PSNR
SSIM
CD
EMD J
MOSIV
23.98
0.941
2.401
0.043
19.81
0.904
5.800
0.095
Table 7: Quantitative Evaluation of MOSIV Dataset Extension
A.8 SENSITIVITY ANALYSIS
We robustness reconstruction inaccuracy by perturbing Gaussians trained from fixed ground-truth point clouds before system identification. For each object, we add Gaussian noise with standard deviation Tab. and Tab. 9. This scales the perturbation relative to each object's 's spatial extent. For every frame, we consider two variants of corruption: (1) i.i.d. noise, where each point receives an independent sample, and (2) shared noise, where all points of an object share the same offset. We then apply MOSIV for system identification and long-horizon prediction. As shown the tables, increasing the noise level from 0.005 (about 1-2 dB) and a corresponding increase in CD/EMD, while SSIM remains high (2 0.95), At the largest perturbation level (a 0.05), PSNR and SSIM remain in the 23-24 dB and 0.93 0.94 ranges, respectively, and CD/EMD increase further but stay within the same order of magnitude. Overall, the metrics degrade smoothly rather than abruptly, indicating that MOSIV can tolerate substantial reconstruction noise.
(xyzmax syimmi)) where a is the noise magnitude shown in
0.02 produces moderate decrease in PSNR
Observable Simulation
Future Simulation
Noise level(a)
PSNR
SSIM t
CD +
EMD
PSNR 1
SSIM t
CD 4
EMD +
0.005
27.86
0.96
0.22
0.03
27.47
0.96
0.29
0.03
0.020
26.06
0.95
0.41
0.03
25.43
0.95
0.52
0.05
0.050
23.73
0.94
1.27
0.05
23.39
0.93
1.72
0.07
Table 8: Sensivity analysis of i.i.d. noise
5

---

Published as a conference paper at ICLR 2026
Observable Simulation
Future Simulation
Noise level (a)
PSNR
SSIM f
CD 4
EMD 4
PSNR
SSIM
CD 4
EMD
0.005
27.82
0.96
0.24
0.03
27.43
0.96
0.35
0.03
0.020
26.17
0.95
0.38
0.03
25.57
0.95
0.49
0.04
0.050
23.92
0.93
1.14
0.04
23.52
0.93
1.50
0.06
Table 9: Sensitivity analysis of shared noise
A.9 MORE QUALITATIVE RESULTS
We conduct extra qualitative comparison of multi-material interaction, showing the results of OmniphysGS (Lin et al., 2025), OmniphysGS w/ Oracle (Lin et al., 2025), and MOSIV in Figs. to 18. Each figure compares the results of both observed and predicted frames and MOSIV shows finer deformations with greater accuracy, demonstrating superior adaptability diverse materials interactions.

---

Published as a conference paper at ICLR 2026
Ground Truth
MOSIV
OPGS w/ Oracle
OPGS
Observed Frames
T=0
T-6
T-12
T-17
Predicted Frames
T=22
T=26
Figure 9: Qualitative Comparison with material type elastic/fluid (E/F).
7

---

Published as a conference paper at ICLR 2026
Ground Truth
MOSIV
OPGS w/ Oracle
OPGS
Observed Frames
T=0
T=6
T-12
T-17
Predicted Frames
T=22
T=26
Figure 10: Qualitative Comparison with material type elastic/plasticine (E/P).
8

---

Published as a conference paper at ICLR 2026
Ground Truth
MOSIV
OPGS w/ Oracle
OPGS
Observed Frames
T=0
T-6
T-12
T-17
Predicted Frames
T=22
T=26
Figure 11: Qualitative Comparison with material type elastic/sand (E/S).

---

Published as a conference paper at ICLR 2026
Ground Truth
MOSIV
OPGS w/ Oracle
OPGS
Observed Frames
T=0
T=6
a
T-12
T-17
Predicted Frames
T=22
T=26
Figure 12: Qualitative Comparison with material type fluid/sand (F/S).
10

---

Published as a conference paper at ICLR 2026
Ground Truth
MOSIV
OPGS w/ Oracle
OPGS
Observed Frames
T=0
T-6
T-12
T-17
Predicted Frames
T=22
T=26
Figure 13: Qualitative Comparison with material type plasticine/fluid (P/F).
11

---

Published as a conference paper at ICLR 2026
Ground Truth
MOSIV
OPGS w/ Oracle
OPGS
Observed Frames
T=0
T=6
T-12
T-17
Predicted Frames
T=22
T=26
Figure 14: Qualitative Comparison with material type sand/plasticine (S/P).
12

---

Published as a conference paper at ICLR 2026
Ground Truth
MOSIV
OPGS w/ Oracle
OPGS
Observed Frames
T=0
T-6
T-12
T-17
Predicted Frames
T=22
T=26
Figure 15: Qualitative Comparison with material type fluid/fluid (F/F).
13

---

Published as a conference paper at ICLR 2026
Ground Truth
MOSIV
OPGS w/ Oracle
OPGS
Observed Frames
T=0
T-6
T-12
T-17
Predicted Frames
T=22
T=26
Figure 16: Qualitative Comparison with material type plasticine/plastiinnee (P/P).
14

---

Published as a conference paper at ICLR 2026
Ground Truth
MOSIV
OPGS w/ Oracle
OPGS
Observed Frames
T=0
T-6
T-12
T-17
Predicted Frames
T=22
T=26
Figure 17: Qualitative Comparison with material type elastic/elastic (E/E).
15

---

Published as a conference paper at ICLR 2026
Ground Truth
MOSIV
OPGS w/ Oracle
OPGS
Observed Frames
T=0
T-6
T-12
T-17
Predicted Frames
T=22
T=26
Figure 18: Qualitative Comparison with material type sand/sand (S/S).
16


Reviews:
[{"review_id": "vyNXlFrdVM", "reviewer": "Reviewer_dfDn", "summary": "The paper proposes a framework to recover material properties from multi-view videos of multi-object scenes. The task setting is similar to PAC-NeRF except that multiple objects exist in the same scene. In additional to pixel supervision, the method also reconstructs particle trajectories using a 4D Gaussian framework, so that 3D surface loss can also be used as supervision.", "questions": "Key questions that needs justifications or clarifications are mentioned above.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "- The experiments are comprehensive and promising.", "weaknesses": "- The contribution is limited. The following two works are not cited. Justification of contributions needs revision.\n    - The task of multi-object system identification from videos is not novel: [1] also tackles multi-object system ID based on NeRF.\n    - Geometry-driven supervision is not novel: [2] uses 4D Gaussian to reconstruction mesh sequence from multiview videos and use 3D loss to tune physical parameters of cloth.\n\n[1] Li, J., Gao, Y., Song, W., Li, Y., Li, S., Hao, A. and Qin, H., 2024, October. CoupNeRF: Property‐aware Neural Radiance Fields for Multi‐Material Coupled Scenario Reconstruction. In Computer Graphics Forum (Vol. 43, No. 7, p. e15208).\n\n[2] Zheng, Y., Zhao, Q., Yang, G., Yifan, W., Xiang, D., Dubost, F., Lagun, D., Beeler, T., Tombari, F., Guibas, L. and Wetzstein, G., 2024, September. Physavatar: Learning the physics of dressed 3d avatars from visual observations. In European Conference on Computer Vision (pp. 262-284). Cham: Springer Nature Switzerland.\n\n- Some key descriptions of the proposed method are missing: \n    - How discrete constitutive models of each objects are set? If the types are given, \"Predicted Categorical Distribution\" in Fig 1 is very misleading. And [1] already handled this case. If the types are predicted, the paper does not mention the procedure at all in method section.\n    - How silhouettes are rendered? Are they rasterized directly from MPM point clouds?\n    - How are \"extracted surfaces\" (Line 256) extracted in detail?\n\n- No real-world examples are provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BQSUJtbjqD", "reviewer": "Reviewer_yKeE", "summary": "This paper introduces MOSIV, a new framework created to solve the problem of identifying the physical properties of multiple interacting objects simultaneously from a video. MOSIV works by using a differentiable simulator. It directly optimizes the specific, continuous material parameters for each object by trying to match the geometry observed in the video. MOSIV also presents a new synthetic benchmark with contact-rich, multi-object interactions.", "questions": "- The method appears to be computationally expensive? It involves (1) optimizing a 4D Gaussian scene and assigning instance partitions, (2) converting object's reconstruction into simulation-ready continuum (3) running a differentiable MPM simulation to optimize per-object parameter vectors, and (4) optimizing this entire unrolled simulation. What are the typical run time and memory footprint? how does it compare with baseline methods?\n- what are the novel interactions in section 3.7? if my understanding is correctly, the physical motions are kept the same, you only change the materials?\n- what consitutive models have been used? how are they chosen, can you provide more details on that?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- the vision of learning physical properties and its interactions directly from videos seems appealing\n- The overall approach is logical and builds upon several state-of-the-art components, including the dynamic Gaussian Splatting for reconstruction and a differentiable MPM for physics-based parameter identification\n- the author introduces a new multi-object dataset with diverse geometry, materials properties and physical motions, that could be used from the community", "weaknesses": "- the paper studies multi-object system interaction, but the proposed dataset only contains two-object interactions, also 30 frames of interactions seems quite short for evaluation, given the authors claim that the calibrated models generalizes to \"long-horizon predictions of complex multi-object dynamics\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MzunPHGh7s", "reviewer": "Reviewer_yotJ", "summary": "The paper presents MOSIV (Multi-Object System Identification from Videos), a novel framework designed to infer continuous physical constitutive parameters for multiple interacting objects directly from multi-view video observations. By employing per-object geometric reconstruction followed by the system identification of continuous parameters, MOSIV moves beyond prior work that relies solely on selecting from a fixed, categorical library of expert constitutive models. This advancement significantly enhances the fidelity, precision, and physical plausibility of subsequent dynamic simulations.", "questions": "See weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 4, "strengths": "+ Pioneering Continuous Parameter Identification: The primary strength is the innovative shift from categorical material classification to the estimation of continuous constitutive parameters. This allows for highly granular, object-specific physical calibration, overcoming a major limitation of current video-to-physics pipelines.\n+ Effective Multi-Object Handling: The framework successfully tackles the complexity of simultaneous system identification across multiple objects that are undergoing complex interactions (e.g., contact, collision) within the same observed scene, which is a critical capability for real-world robotics and simulation.\n+ Robust Framework Integration: The successful integration of geometric reconstruction and a differentiable physics pipeline suggests a robust, end-to-end optimization strategy capable of generating strongly calibrated and faithful physical models from raw visual data.", "weaknesses": "- Overall, this paper presents a clear goal and a coherent overall narrative. However, the main issue lies in the lack of clear motivation behind many of the detailed design choices. In several instances, important implementation details are missing, which can lead to reader confusion and make the paper difficult to follow.\n- According to the description in subsection 3.3, objects are first represented as Gaussians, then converted into voxels, and finally transformed into particles. However, the whole process is quite unclear given the current simple explanation. Moreover, what parameters define these particles? Are they simply points? If so, why not use point clouds as the initial representation instead?\n- In subsection 3.5, why are silhouettes and surfaces used as the objective? Is this a common approach for such tasks? If so, please provide appropriate references; if not, please clarify the rationale behind this choice.\n- In Figure 4, different methods appear to have different observed frames. Does this imply that they were simulated using different parameters or settings? Shouldn’t the observed frames be consistent across methods to ensure a fair comparison?\n- Given that the method involves per-object geometric reconstruction and the iterative optimization of continuous parameters, the computational expense is likely very high. Could the authors provide a rigorous analysis of the computational complexity (e.g., model size, training time, and inference time) and a comparison with other methods?\n- How robust is MOSIV to different levels of reconstruction uncertainty? Did the authors conduct a sensitivity analysis showing how small errors in the initial reconstruction propagate into the estimated values of the constitutive parameters?\n- While the parameters are continuous, the underlying physics model must still be chosen. Does MOSIV include a mechanism for a priori selecting the correct base constitutive model type, or is that externally provided?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dGvZ6wnwJM", "reviewer": "Reviewer_SAcW", "summary": "The paper introduces a synthetic video dataset that presents contact and interaction between different materials. A baseline framework that identifies the continuous, object-specific physical properties is also proposed, with geometric-driven supervision and object-aware dynamic component.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- I found this paper reused and properly directed to external sources for development that helps the readers from diverse backgrounds.\n- A few features I see interesting to have together in this framework: simulation-ready continuum for multiple objects across all time, permaterial parameters to predict future motions.", "weaknesses": "My background is not in physics simulation, so I look at the paper from the perspective of a computer vision researcher.\n- It is unclear how long the predicted horizon is. The example videos appear quite short—only a few seconds in duration, and seem not to show motion prediction (which I find different from trajectory prediction. Trajectory prediction is accumulating past locations, while motion prediction is to predict future locations). Please clarify the temporal length of the predictions.\n- How many object categories are supported in the dataset? A summary table presenting dataset statistics would be helpful for readers. Also, I find that only two object interactions are limited compared to existing datasets such as CLEVRER.\n- Additionally, please discuss or evaluate the model's ability to generalize to out-of-distribution in both object categories and interactions. This would also help to strengthen section subsection 3.7.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)