MVPaint: 3D Texture Generation with Multi-View Consistency

Authors: Wei Cheng, Juncheng Mu, Xianfang Zeng, Xin Chen, Anqi Pang, Chi Zhang, Zhibin Wang, Bin Fu, Gang Yu, Ziwei Liu, Liang Pan
Primary area: generative models
Keywords: texture generation

Abstract:
Texturing is a crucial step in the 3D asset production workflow, which enhances
the visual appeal and diversity of 3D assets. Despite recent advancements in
generation-based texturing, existing methods often yield subpar results, primar-
ily due to local discontinuities, inconsistencies across multiple views, and their
heavy dependence on UV unwrapping outcomes. To tackle these challenges,
we propose a novel generation-refinement 3D texturing framework called MV-
Paint, which can generate high-resolution, seamless textures while emphasizing
multi-view consistency. Given a 3D mesh model, MVPaint first simultaneously
generates multi-view images by employing a Synchronized Multi-view Genera-
tion (SMG) module, which leads to coarse texturing results with unpainted parts
due to missing observations. To ensure complete 3D texturing, we introduce the
Spatial-aware 3D Inpainting (S3I) method, specifically designed to texture pre-
viously unobserved areas effectively. Furthermore, MVPaint employs a UV Re-
finement (UVR) module for improving the texture quality in the UV space. UVR
first performs a UV-space Super-Resolution, followed by a Spatial-aware Seam-
Smoothing algorithm for revising spatial texturing discontinuities caused by UV
unwrapping. Moreover, we perform meticulous manual annotations to filter the
Objaverse dataset, resulting in around 100,000 high-quality 3D data for texturing
generation. Extensive experimental results demonstrate that MVPaint surpasses
existing state-of-the-art methods. Notably, MVPaint could generate high-fidelity
textures with fewer multi-face issues and better cross-view consistency

Decision: reject

Reviews and discussion:

Metareview: