TexGen: Text-Guided 3D Texture Generation with Multi-view Sampling and
  Resampling
Authors: name: Dong Huo; name: Zixin Guo; name: Xinxin Zuo; name: Zhihao Shi; name: Juwei Lu; name: Peng Dai; name: Songcen Xu; name: Li Cheng; name: Yee-Hong Yang
Published: 2024-08-02 14:24:40

Abstract: Given a 3D mesh, we aim to synthesize 3D textures that correspond to
arbitrary textual descriptions. Current methods for generating and assembling
textures from sampled views often result in prominent seams or excessive
smoothing. To tackle these issues, we present TexGen, a novel multi-view
sampling and resampling framework for texture generation leveraging a
pre-trained text-to-image diffusion model. For view consistent sampling, first
of all we maintain a texture map in RGB space that is parameterized by the
denoising step and updated after each sampling step of the diffusion model to
progressively reduce the view discrepancy. An attention-guided multi-view
sampling strategy is exploited to broadcast the appearance information across
views. To preserve texture details, we develop a noise resampling technique
that aids in the estimation of noise, generating inputs for subsequent
denoising steps, as directed by the text prompt and current texture map.
Through an extensive amount of qualitative and quantitative evaluations, we
demonstrate that our proposed method produces significantly better texture
quality for diverse 3D objects with a high degree of view consistency and rich
appearance details, outperforming current state-of-the-art methods.
Furthermore, our proposed texture generation technique can also be applied to
texture editing while preserving the original identity. More experimental
results are available at https://dong-huo.github.io/TexGen/