TreeMeshGPT: Artistic Mesh Generation with Autoregressive Tree
  Sequencing
Authors: name: Stefan Lionar, name: Jiabin Liang, name: Gim Hee Lee
Organization: Sea AI Lab

We introduce TreeMeshGPT, an autoregressive Transformer designed to generate
high-quality artistic meshes aligned with input point clouds. Instead of the
conventional next-token prediction in autoregressive Transformer, we propose a
novel Autoregressive Tree Sequencing where the next input token is retrieved
from a dynamically growing tree structure that is built upon the triangle
adjacency of faces within the mesh. Our sequencing enables the mesh to extend
locally from the last generated triangular face at each step, and therefore
reduces training difficulty and improves mesh quality. Our approach represents
each triangular face with two tokens, achieving a compression rate of
approximately 22% compared to the naive face tokenization. This efficient
tokenization enables our model to generate highly detailed artistic meshes with
strong point cloud conditioning, surpassing previous methods in both capacity
and fidelity. Furthermore, our method generates mesh with strong normal
orientation constraints, minimizing flipped normals commonly encountered in
previous methods. Our experiments show that TreeMeshGPT enhances the mesh
generation quality with refined details and normal orientation consistency.