Physics3D: Learning Physical Properties of 3D Gaussians via Video
  Diffusion
Authors: name: Fangfu Liu; name: Hanyang Wang; name: Shunyu Yao; name: Shengjun Zhang; name: Jie Zhou; name: Yueqi Duan
Published: 2024-06-06 17:59:47

Abstract: In recent years, there has been rapid development in 3D generation models,
opening up new possibilities for applications such as simulating the dynamic
movements of 3D objects and customizing their behaviors. However, current 3D
generative models tend to focus only on surface features such as color and
shape, neglecting the inherent physical properties that govern the behavior of
objects in the real world. To accurately simulate physics-aligned dynamics, it
is essential to predict the physical properties of materials and incorporate
them into the behavior prediction process. Nonetheless, predicting the diverse
materials of real-world objects is still challenging due to the complex nature
of their physical attributes. In this paper, we propose Physics3D, a
novel method for learning various physical properties of 3D objects through a
video diffusion model. Our approach involves designing a highly generalizable
physical simulation system based on a viscoelastic material model, which
enables us to simulate a wide range of materials with high-fidelity
capabilities. Moreover, we distill the physical priors from a video diffusion
model that contains more understanding of realistic object materials. Extensive
experiments demonstrate the effectiveness of our method with both elastic and
plastic materials. Physics3D shows great potential for bridging the gap between
the physical world and virtual neural space, providing a better integration and
application of realistic physical principles in virtual environments. Project
page: https://liuff19.github.io/Physics3D.