GaussianCube: Structuring Gaussian Splatting using Optimal Transport for
  3D Generative Modeling
Authors: name: Bowen Zhang; name: Yiji Cheng; name: Jiaolong Yang; name: Chunyu Wang; name: Feng Zhao; name: Yansong Tang; name: Dong Chen; name: Baining Guo
Published: 2024-03-28 17:59:50

Abstract: 3D Gaussian Splatting (GS) have achieved considerable improvement over Neural
Radiance Fields in terms of 3D fitting fidelity and rendering speed. However,
this unstructured representation with scattered Gaussians poses a significant
challenge for generative modeling. To address the problem, we introduce
GaussianCube, a structured GS representation that is both powerful and
efficient for generative modeling. We achieve this by first proposing a
modified densification-constrained GS fitting algorithm which can yield
high-quality fitting results using a fixed number of free Gaussians, and then
re-arranging the Gaussians into a predefined voxel grid via Optimal Transport.
The structured grid representation allows us to use standard 3D U-Net as our
backbone in diffusion generative modeling without elaborate designs. Extensive
experiments conducted on ShapeNet and OmniObject3D show that our model achieves
state-of-the-art generation results both qualitatively and quantitatively,
underscoring the potential of GaussianCube as a powerful and versatile 3D
representation.