Material Anything: Generating Materials for Any 3D Object via Diffusion
Authors: name: Xin Huang; name: Tengfei Wang; name: Ziwei Liu; name: Qing Wang
Published: 2024-11-22 18:59:39

Abstract: We present Material Anything, a fully-automated, unified diffusion framework
designed to generate physically-based materials for 3D objects. Unlike existing
methods that rely on complex pipelines or case-specific optimizations, Material
Anything offers a robust, end-to-end solution adaptable to objects under
diverse lighting conditions. Our approach leverages a pre-trained image
diffusion model, enhanced with a triple-head architecture and rendering loss to
improve stability and material quality. Additionally, we introduce confidence
masks as a dynamic switcher within the diffusion model, enabling it to
effectively handle both textured and texture-less objects across varying
lighting conditions. By employing a progressive material generation strategy
guided by these confidence masks, along with a UV-space material refiner, our
method ensures consistent, UV-ready material outputs. Extensive experiments
demonstrate our approach outperforms existing methods across a wide range of
object categories and lighting conditions.