LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesis
Authors: name: Kevin Xie; name: Jonathan Lorraine; name: Tianshi Cao; name: Jun Gao; name: James Lucas; name: Antonio Torralba; name: Sanja Fidler; name: Xiaohui Zeng
Published: 2024-03-22 17:59:37

Abstract: Recent text-to-3D generation approaches produce impressive 3D results but
require time-consuming optimization that can take up to an hour per prompt.
Amortized methods like ATT3D optimize multiple prompts simultaneously to
improve efficiency, enabling fast text-to-3D synthesis. However, they cannot
capture high-frequency geometry and texture details and struggle to scale to
large prompt sets, so they generalize poorly. We introduce LATTE3D, addressing
these limitations to achieve fast, high-quality generation on a significantly
larger prompt set. Key to our method is 1) building a scalable architecture and
2) leveraging 3D data during optimization through 3D-aware diffusion priors,
shape regularization, and model initialization to achieve robustness to diverse
and complex training prompts. LATTE3D amortizes both neural field and textured
surface generation to produce highly detailed textured meshes in a single
forward pass. LATTE3D generates 3D objects in 400ms, and can be further
enhanced with fast test-time optimization.