DiMeR: Disentangled Mesh Reconstruction Model
Authors: name: Lutao Jiang, name: Jiantao Lin, name: Kanghao Chen, name: Wenhang Ge, name: Xin Yang, name: Yifan Jiang, name: Yuanhuiyi Lyu, name: Xu Zheng, name: Yingcong Chen

With the advent of large-scale 3D datasets, feed-forward 3D generative
models, such as the Large Reconstruction Model (LRM), have gained significant
attention and achieved remarkable success. However, we observe that RGB images
often lead to conflicting training objectives and lack the necessary clarity
for geometry reconstruction. In this paper, we revisit the inductive biases
associated with mesh reconstruction and introduce DiMeR, a novel disentangled
dual-stream feed-forward model for sparse-view mesh reconstruction. The key
idea is to disentangle both the input and framework into geometry and texture
parts, thereby reducing the training difficulty for each part according to the
Principle of Occam's Razor. Given that normal maps are strictly consistent with
geometry and accurately capture surface variations, we utilize normal maps as
exclusive input for the geometry branch to reduce the complexity between the
network's input and output. Moreover, we improve the mesh extraction algorithm
to introduce 3D ground truth supervision. As for texture branch, we use RGB
images as input to obtain the textured mesh. Overall, DiMeR demonstrates robust
capabilities across various tasks, including sparse-view reconstruction,
single-image-to-3D, and text-to-3D. Numerous experiments show that DiMeR
significantly outperforms previous methods, achieving over 30% improvement in
Chamfer Distance on the GSO and OmniObject3D dataset.