GaussianDreamerPro: Text to Manipulable 3D Gaussians with Highly
  Enhanced Quality
Authors: name: Taoran Yi; name: Jiemin Fang; name: Zanwei Zhou; name: Junjie Wang; name: Guanjun Wu; name: Lingxi Xie; name: Xiaopeng Zhang; name: Wenyu Liu; name: Xinggang Wang; name: Qi Tian
Published: 2024-06-26 16:12:09

Abstract: Recently, 3D Gaussian splatting (3D-GS) has achieved great success in
reconstructing and rendering real-world scenes. To transfer the high rendering
quality to generation tasks, a series of research works attempt to generate
3D-Gaussian assets from text. However, the generated assets have not achieved
the same quality as those in reconstruction tasks. We observe that Gaussians
tend to grow without control as the generation process may cause indeterminacy.
Aiming at highly enhancing the generation quality, we propose a novel framework
named GaussianDreamerPro. The main idea is to bind Gaussians to reasonable
geometry, which evolves over the whole generation process. Along different
stages of our framework, both the geometry and appearance can be enriched
progressively. The final output asset is constructed with 3D Gaussians bound to
mesh, which shows significantly enhanced details and quality compared with
previous methods. Notably, the generated asset can also be seamlessly
integrated into downstream manipulation pipelines, e.g. animation, composition,
and simulation etc., greatly promoting its potential in wide applications.
Demos are available at https://taoranyi.com/gaussiandreamerpro/.