Title: Phys-Bench: A Physics-aware Benchmark with Multi-Body Interactions for 3D Dynamic Scene Understanding
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: 4D Gaussian Splatting, Physics, Dynamic Novel View Synthesis

Abstract: We introduce Phys-Bench, a novel physics-aware benchmark for 3D dynamic scene understanding. This benchmark is designed to evaluate methods for reconstructing 4D scenes and understanding underlying physics from given videos, with a main
focus on the Dynamic Novel View Synthesis (DyNVS) task. 
While existing algorithms and benchmarks primarily focus on photorealistic reconstruction, they largely overlook physics understanding. This neglect is a critical limitation, as a true understanding of dynamic scenes requires models to reason about physical interactions, not just appearance. 
Our benchmark provides complex dynamic scenarios with rich multi-object interactions, featuring realistic collisions and force exchanges that are faithfully generated to strictly adhere to physical laws. 
Furthermore, it contains a diverse range of physical materials, such as liquid, gas, rheological substances, and textiles, which move beyond the rigid bodies prevalent in existing benchmarks. 
To enable quantitative evaluation, we provide essential ground-truth information such as 3D particle trajectories and physics parameters and propose two novel metrics tailored to assessing physical realism. 
We further evaluate existing Dynamic Novel View synthesis and physics parameter estimation method on our benchmark and reveal their overlooked limitations in physics understanding and multi-body dynamics handling. 
We believe Phys-Bench will serve as a crucial foundation for advancing research in dynamic view synthesis, physics-based scene understanding, and the integration of deep learning with physical simulation, ultimately enabling more faithful reconstruction and interpretation of complex 3D dynamic scenes.