Compress3D: a Compressed Latent Space for 3D Generation from a Single
  Image
Authors: name: Bowen Zhang; name: Tianyu Yang; name: Yu Li; name: Lei Zhang; name: Xi Zhao
Published: 2024-03-20 11:51:04

Abstract: 3D generation has witnessed significant advancements, yet efficiently
producing high-quality 3D assets from a single image remains challenging. In
this paper, we present a triplane autoencoder, which encodes 3D models into a
compact triplane latent space to effectively compress both the 3D geometry and
texture information. Within the autoencoder framework, we introduce a 3D-aware
cross-attention mechanism, which utilizes low-resolution latent representations
to query features from a high-resolution 3D feature volume, thereby enhancing
the representation capacity of the latent space. Subsequently, we train a
diffusion model on this refined latent space. In contrast to solely relying on
image embedding for 3D generation, our proposed method advocates for the
simultaneous utilization of both image embedding and shape embedding as
conditions. Specifically, the shape embedding is estimated via a diffusion
prior model conditioned on the image embedding. Through comprehensive
experiments, we demonstrate that our method outperforms state-of-the-art
algorithms, achieving superior performance while requiring less training data
and time. Our approach enables the generation of high-quality 3D assets in
merely 7 seconds on a single A100 GPU.