AToM: Amortized Text-to-Mesh using 2D Diffusion
Authors: name: Guocheng Qian, name: Junli Cao, name: Aliaksandr Siarohin, name: Yash Kant, name: Chaoyang Wang, name: Michael Vasilkovsky, name: Hsin-Ying Lee, name: Yuwei Fang, name: Ivan Skorokhodov, name: Peiye Zhuang, name: Igor Gilitschenski, name: Jian Ren, name: Bernard Ghanem, name: Kfir Aberman, name: Sergey Tulyakov

We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh
framework optimized across multiple text prompts simultaneously. In contrast to
existing text-to-3D methods that often entail time-consuming per-prompt
optimization and commonly output representations other than polygonal meshes,
AToM directly generates high-quality textured meshes in less than 1 second with
around 10 times reduction in the training cost, and generalizes to unseen
prompts. Our key idea is a novel triplane-based text-to-mesh architecture with
a two-stage amortized optimization strategy that ensures stable training and
enables scalability. Through extensive experiments on various prompt
benchmarks, AToM significantly outperforms state-of-the-art amortized
approaches with over 4 times higher accuracy (in DF415 dataset) and produces
more distinguishable and higher-quality 3D outputs. AToM demonstrates strong
generalizability, offering finegrained 3D assets for unseen interpolated
prompts without further optimization during inference, unlike per-prompt
solutions.