Title: ARMesh: Autoregressive Mesh Generation via Next-Level-of-Detail Prediction
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3d mesh generation

autoregressive modeling

coarse-to-fine

level of detail
Abstract: Directly generating 3D meshes, the default representation for 3D shapes in the graphics industry, using auto-regressive (AR) models has become popular these days, thanks to their sharpness, compactness in the generated results, and ability to represent various types of surfaces. However, AR mesh generative models typically construct meshes face by face in lexicographic order, which does not effectively capture the underlying geometry in a manner consistent with human perception. Inspired by 2D models that progressively refine images, such as the prevailing next-scale prediction AR models, we propose generating meshes auto-regressively in a progressive coarse-to-fine manner. Specifically, we view mesh simplification algorithms, which gradually merge mesh faces to build simpler meshes, as a natural fine-to-coarse process. Therefore, we generalize meshes to simplicial complexes and develop a transformer-based AR model to approximate the reverse process of simplification in the order of level of detail, constructing meshes initially from a single point and gradually adding geometric details through local remeshing, where the topology is not predefined and is alterable. Our experiments show that this novel progressive mesh generation approach not only provides intuitive control over generation quality and time consumption by early stopping the auto-regressive process but also enables applications such as mesh refinement and editing.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes ARMesh, a novel autoregressive framework for direct 3D mesh generation that synthesizes mesh progressively from coarse to fine levels of detail (LOD), departing from conventional lexicographic face-by-face generation. Its core contributions are:\n1. Insight:\nLimitations in current AR mesh generators (e.g., MeshGPT, PolyGen) lies in constructing meshes in fixed orders unrelated to geometric hierarchy. ARMesh models the generation porcess by next level-of-detail(LOD) prediction compure to next scale generation in 2D image generation problem, which able to model the unorder 3D space.\n2. Generalized Simplification (GSlim):\nIntroduces a topology-aware mesh simplification algorithm operating on simplicial complexes which generalizing meshes to include points/edges. It reduces any input mesh to a single point in n-1 steps, producing a fine-to-coarse LOD sequence.\n3. Progressive Simplicial Complex (PSC):\nReverses GSlim into a generative coarse-to-fine process which enables the following AR mesh generation process.\n4. Autoregressive Modeling:\n  - Tokenizes PSC sequences into discrete vocabularies.\n  - Trains a transformer decoder to predict next refinement tokens autoregressively.\n  - Employs constrained decoding (DFS + hardcoded rules) ensuring topological validity.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. Originality & Significance:\n  - Novel Next-LOD Prediction Paradigm: Proposes a fundamentally new autoregressive approach (Next-LOD prediction) for mesh generation. This departs significantly from prior lexicographic/traversal-order face-by-face generation, offering a more geometrically reasonable (coarse-to-fine) generation process. This is a conceptually original contribution to the field.\n  - GSlim & PSC Framework: Introduces the generalized simplification algorithm (GSlim) enabling topology changes and reduction to a single point, and its reversible counterpart (PSC). This technical innovation is crucial for constructing the LOD sequences and is a significant advancement over traditional simplification (e.g., QSlim) for generative modeling.\n2. Technically Sound Foundation:\n  - The core methodology (GSlim simplification → PSC reversal → tokenization → AR transformer training) is well-motivated, rigorously described (esp. topological constraints and tokenization), and demonstrates clear technical competence.\n3. Clarity:\n  - Well-Structured and Readable: The paper is exceptionally well-written. The problem motivation (limitations of prior AR orderings), inspiration (2D next-scale prediction), proposed solution (LOD reversal), and technical details (GSlim, PSC, tokenization) are presented logically and clearly. Figures effectively illustrate concepts (e.g., vertex split, topology cases).\n\n\nWeaknesses\n1. Limited Scalability Demonstration:\nTraining and evaluation are conducted primarily on small-scale dataset. Crucially, recent state-of-the-art methods (e.g., EdgeRunner, MeshXL) leverage large-scale datasets like Objaverse. This omission raises concerns about the method’s scalability and generalization capability to complex, diverse shape distributions.\n2. Incomplete State-of-the-Art Comparison:\nWhile comparisons against PolyGen and MeshGPT are provided, Table 3 lacks critical benchmarks:\n  - GET3D results for the Bench and Lamp categories\n  - Contemporary baselines (e.g., EdgeRunner, MeshXL, MeshAnything) that report strong unconditional generation performance.\nValidation against these methods on larger datasets (e.g., Objaverse) would significantly strengthen empirical claims.\n3. Insufficient Ablation Analysis:\nWhile ablation studies explore LOD ratios and penalty factors, key design choices lack rigorous validation:\n  - Impact of transformer architecture/configurations (depth, width, attention mechanisms)\n  - Tokenization alternatives beyond the proposed scheme\n  - Quantitative and qualitative ablation results would better elucidate the contribution of individual components.\n\n4. Narrow Scope in Conditional Generation:\nThe work focuses exclusively on unconditional generation. But it will be easy to adopt this method into condition mesh generation. Demonstrating efficacy in conditional settings (e.g., text-to-mesh, point cloud completion) would enhance the practical impact and versatility of the approach. Comparisons with conditional mesh generators are notably absent.\n\n**If the authors solve my concerns during the rebuttal, I will increase my score to 5.**\n\nQUESTIONS:\n1. Could the authors evaluate ARMesh on a large-scale 3D dataset (e.g., Objaverse, ShapeNet-XL) to assess scalability and generalization? I fully understand that full training might be infeasible during rebuttal period, minimal experiments (e.g., fine-tuning on a subset) or qualitative results would suffice.\n2. Could the authors compare ARMesh with SOTA autoregressive methods? I understand they are trained on some large-scale dataset like the objverse. Could author re-train them on identical dataset which paper use?\n3. Could authors provide quantitative results on condition generation result? For example: coarse mesh/sparse point cloud conditioned.\n\nLIMITATIONS:\n1. Scalability to Real-World Complexity： The training and evaluation of ARMesh is limited to small-scale datasets. Its performance remains unverified on large-scale, diverse 3D collections (e.g., Objaverse. Without validation on high-complexity shapes or diverse topologies , claims of generalizability remain speculative. This restricts confidence in real-world applicability.\n2. Unexplored Conditional Generation: While ARMesh intrinsically supports progressive refinement (Fig 13–14), its potential for conditional generation—e.g., text-guided LOD synthesis or point cloud completion—remains unexplored. The method’s coarse-to-fine paradigm is theoretically well-suited for such tasks (e.g., refining sketch inputs to detailed meshes).", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel method to generate meshes in a coarse-to-fine manner. To construct a compatible representation for arbitrary topology, especially the starting point, the authors generalize 3D meshes to simplicial complexes and adapt the commonly used mesh simplification algorithm to GSlim. By tokenizing the vertex split operation, the paper parses each mesh as a 1D refining sequence that can be further modeled with a transformer-based AR model.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The method to tokenize the simplification operation is non-trivial and well-designed.\n- The overall writing is good, making the technical details clear.\n\nWeaknesses:\n- For a mesh with N vertices, the length of mesh sequences is roughly larger than $10N$ or even longer, which is longer than per-face per-coordinate tokenization. This may hinder the model ability to generate complex meshes.\n- Compared to direct mesh generation, it is not entirely clear how we can use the intermediate results for some potential applications. Although there are some mentions, no detailed algorithms or results are given in the article.\n- The dataset and evaluation are limited. At least, edgerunner, meshanything, BPT are quite relevant but they are not shown in the quantitative results. Shapenet is not diverse enough for mesh generation.\n\nQUESTIONS:\nI am generally positive about the paper. The paper proposes a good way to model meshes in a coarse-to-fine way. But the tokenization process seems too complicated, making the sequence quite long. I think it's a big limitation on how far we can go. What do we get out of making such sacrifices, in other words, are there some potential solutions, or how can we utilize the intermediate results? What is the difference between generating the whole mesh and them simplifying it, and the proposed method?\n\nOther comments:\n- The title of section 3.1: simplical -> simplicial\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel autoregressive (AR) framework for 3D mesh generation that constructs meshes in a coarse-to-fine manner, departing from the conventional face-by-face generation paradigm. Inspired by next-scale prediction in 2D image generation, the authors model the mesh generation process as the reverse of a mesh simplification procedure. They leverage a transformer-based AR model to reconstruct the mesh, starting from a single vertex and gradually increasing geometric detail without a fixed mesh topology. The method improves mesh quality and supports downstream tasks such as mesh refinement and editing. The method is novel in the field of mesh generation as it's the first next-scale prediction mesh generation method.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n1. **Novelty.**  \n   The paper introduces a new perspective on mesh generation by reversing the mesh simplification process and formulating it as a progressive coarse-to-fine autoregressive task. To the best of our knowledge, this is the first work to adopt a next-scale prediction approach for meshes, analogous to recent advances in 2D generation.\n\n2. **Application.**  \n   The proposed coarse-to-fine generation process naturally enables applications such as mesh refinement and editing, which are challenging to achieve with conventional face-by-face generation schemes.\n\n3. **Clarity.**  \n   The paper is well-written and provides a thorough description of the proposed method.\n\n## Weaknesses\n\n1. **Limited comparison.**  \n   The paper lacks comprehensive comparison with other recent mesh generation methods such as MeshXL, MeshAnything, and DeepMesh, all of which are publicly available.\n\n2. **Lack of conditional generation.**  \n   The experiments are limited to unconditional generation. Given that applications like mesh refinement and editing naturally arise in conditional settings, it is important to validate the proposed method under such scenarios (e.g., conditioned on an image or point cloud).\n\nQUESTIONS:\n1. **Training and inference speed.**  \n   How does the training and inference speed of the proposed method compare with face-by-face generation methods such as MeshGPT?\n\n2. **Extension to conditional generation.**  \n   Can the proposed framework support conditional refinement or editing tasks, such as conditioning on an image or partial point cloud?\n\n3. **Failure rate.**  \n   What is the failure rate of the method during generation? Prior face-by-face methods often exhibit limited success when generating high-face-count meshes—how does this method compare?\n\n4.  Lack of comparison experiments with recent strong baselines and the absence of experiments on conditional generation.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper formulates the triangle mesh generation process as an autoregressive (AR) prediction on the next level-of-detail. The key idea is to ask the AR model to learn the mesh “densification” process (reverse of simplification) while the simplified results are provided by mesh simplification method geometry processing. The produced sequence naturally forms a sequence of increasing levels of detail. The challenge is to “translate” the full sequence of mesh densification to tokens that are compatible with the AR model. For that purpose, the paper proposes a generalized mesh simplification method and designs a method to serialize the vertex split operation as a token. With the train AR model, the resulting system is able to produce a realistic sequence of generation. The authors also show results of mesh simplification (with the proposed modified simplification method), mesh generation, fitting and application such as editing and guided generation given user input.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n- Very interesting formulation to convert the progressive generation process of triangle mesh into autoregressive sequence prediction. \n- The key element of the paper is how to convert progressive mesh generation (vertex split operation) into an AR model compatible token.\n\n\nWeakness:\n- I’m concerned about how robust the generated results are. In particular, will the mesh have topology issues since the model allows flexible change of topology.\n- The authors put quite some effort on explaining the tokenization process. But there are still quite some parts that are not very clear. This would be key to this paper, and it would be great to address my questions, as those might be the questions for future audiences.\n\nQUESTIONS:\n1. “Virtual Edges” in (a bit after than) Line 136: I like the idea to include topological change.\n- (a) How are these virtual edges represented in the later tokenization process? The paragraph in Line 159-171 is great, walking the audience through the newly defined concepts. It would be great to describe other examples (in particular, corner case scenario) with some examples (maybe in the supp. mat.).\n- (b) While flexible, allowing topology changes can also open a box of worms, enabling the model to create a mesh with unacceptable topologies. Will the proposed system suffer from this? How to prevent it?\n\n\n2. I appreciate Section 4.2 as it tries to define the vertex split operation and formally “serialize” the operation as a symbol to be compatible with later AR model learning processes. The paper also explores properties of the topological constraints. Are the four rules in Line 180 complete? How to show if the rules here are complete.\n\n\n3. The example in Line 154 is not fully explored. The “topo” list: I see that those integers refer to the case ID in Table 1 (and Line 166). So those values in that example are not corresponding to the example in Fig 4? This part is a bit confusing. Currently I assume the content in Line 154 is just arbitrary and not connected to later paragraphs.\n\n\n4. It’s a missed opportunity to use the newly defined rules/topo case in Line 167-184 to explain the example in Fig. 4. The “Topological List” in Fig 4 is still not easy to understand.\n\n\n5. The fitting experiment in Fig 11: can the authors provide a more detailed description of how this experiment is done? What loss function/process to guide the generation towards the target bunny mesh? What starting point (single vertex, or a rough shape of the bunny mesh) for the generation? \n\n\n6. Any failure case examples? The limitation section in Line 292 mentioned the limited generalization issue. Can we see some examples? Will it have geometry artifacts and/or topology issues?\n\nLIMITATIONS:\nI’m concerned about the robustness of the method. Can it always produce nice geometry and topology? See my comment in Question section item 1 and 6.\n\n\nThe authors mentioned the generalization issue in the limitation section. Can we see some examples and do some analysis?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}