DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement
  Learning
Authors: name: Ruowen Zhao, name: Junliang Ye, name: Zhengyi Wang, name: Guangce Liu, name: Yiwen Chen, name: Yikai Wang, name: Jun Zhu

Triangle meshes play a crucial role in 3D applications for efficient
manipulation and rendering. While auto-regressive methods generate structured
meshes by predicting discrete vertex tokens, they are often constrained by
limited face counts and mesh incompleteness. To address these challenges, we
propose DeepMesh, a framework that optimizes mesh generation through two key
innovations: (1) an efficient pre-training strategy incorporating a novel
tokenization algorithm, along with improvements in data curation and
processing, and (2) the introduction of Reinforcement Learning (RL) into 3D
mesh generation to achieve human preference alignment via Direct Preference
Optimization (DPO). We design a scoring standard that combines human evaluation
with 3D metrics to collect preference pairs for DPO, ensuring both visual
appeal and geometric accuracy. Conditioned on point clouds and images, DeepMesh
generates meshes with intricate details and precise topology, outperforming
state-of-the-art methods in both precision and quality. Project page:
https://zhaorw02.github.io/DeepMesh/