PivotMesh: Generic 3D Mesh Generation via Pivot Vertices Guidance
Authors: Haohan Weng
Yikai Wang
Tong Zhang
C. L. Philip Chen
Jun Zhu
Venue: ICLR 2025 Poster (2025)
Keywords: mesh generation
auto-regressive generation
3D generation
Primary area: generative models
TL;DR: We extend the native mesh generation to large-scale dataset with the proposed generic and scalable mesh generation framework, namely PivotMesh.

Abstract: Generating compact and sharply detailed 3D meshes poses a significant challenge for current 3D generative models. Different from extracting dense meshes from neural representation, some recent works try to model the native mesh distribution (i.e., a set of triangles), which generates more compact results as humans crafted. However, due to the complexity and variety of mesh topology, most of these methods are typically limited to generating meshes with simple geometry. In this paper, we introduce a generic and scalable mesh generation framework PivotMesh, which makes an initial attempt to extend the native mesh generation to large-scale datasets. We employ a transformer-based autoencoder to encode meshes into discrete tokens and decode them from face level to vertex level hierarchically. Subsequently, to model the complex typology, our model first learns to generate pivot vertices as coarse mesh representation and then generate the complete mesh tokens with the same auto-regressive Transformer. This reduces the difficulty compared with directly modeling the mesh distribution and further improves the model controllability. PivotMesh demonstrates its versatility by effectively learning from both small datasets like Shapenet, and large-scale datasets like Objaverse and Objaverse-xl. Extensive experiments indicate that PivotMesh can generate compact and sharp 3D meshes across various categories, highlighting its great potential for native mesh modeling.

Published as a conference paper at ICLR 2025
PIVOTMESH: GENERIC 3D MESH GENERATION VIA PIVOT VERTICES GUIDANCE
Haohan Weng Yikai Wang?t Tong Zhang'
C. Philip Chen' Jun Zhu
South China University of Technology
Tsinnguua University $ShengShu
ABSTRACT
Generating compact and sharply detailed 3D meshes poses significant challenge for current 3D generative models. Different from extracting dense meshes from neural representation, some recent works try to model the native mesh distribution (i.e., a set of triangles), which generates more compact results as humans crafted. However, due to the complexity and variety of mesh topology, most of these methods are typically limited to generating meshes with simple geometry. In this paper, introduce generic and scalable mesh generation framework PivotMesh, which makes an initial attempt to extend the native mesh generation to large-scale datasets. We employ transformer-bas based autoencoder to encode meshes into discrete tokens and decode them from face level to vertex level hierarchically. Subsequently, to model the complex typology, our model first learns generate pivot vertices as coarse mesh representation and then generate the complete mesh tokens with the same ruto-regressive Transformer. This reduces the difficulty compared with directly modeling the mesh distribution and further improves the model controllability. PivotMesh demonstrates its versatility by effectively learning from both small datasets like Shapenet, and large-scale datasets like Objaverse and Objaverse-xl. xl. Extensive experiments indicate that PivotMesh can generate compact and sharp 3D meshes across various categories, highlighting its great potential for native mesh modeling. Project Page: https: htt////aohaat gi wig.titttttt thub. / pivotmesh
INTRODUCTION
The field of 3D generation has witnessed remarkable advancements in recent years (Poole et al., 2023; Hong et al., 2023; Xu et al., 2024a). Meshes, the predominant representation for 3D geometry, are widely adopted across various applications from video games and movies to architectural modeling. Despite the promising performance of current methods, they mostly rely neural 3D representation like triplanes (Hong et al., 2023; Li et al., 2023) and FlexiCubes (Xu et al., 2024a). Post- cost-processed meshes extracted from these representations tend to be dense and over-smoothed, which are unfriendly for modern rendering pipelines as shown in Figure (bottom). In contrast, meshes crafted by humans are typically more compact with fewer faces, reusing geometric primitives to efficiently represent real-world objects.
To avoid extracting dense meshes through post-processing another promising direction is emerging that focuses on explicitly modeling the mesh distribution (i.e., native mesh generation). This line of works (Nash al., 2020; Siddiqui et al., 2023; Alliegro al., 2023) generates meshes by predicting the 3D coordinates of faces, thus producing compact meshes as humans. However, due to the complexity and variety of topological structures in meshes, most of these methods are typically confined to generating meshes with simple topology, hindering the generalizability across complex objects. Therefore, still remains a challenge to establish a generic generative model for native mesh generation at scale.
In this paper, we propose PivotMesh, a generic and scalable framework to extend mesh generation to large-scale datasets across various categories. PivotMesh consists of two parts: mesh autoencoder and a pivot-guided mesh generator. First, the autoencoder is based on the Transformer to encode
Work done during the internship at ShengShu
1 Corresponding author

---

Published as a conference paper at ICLR 2025
Generate by PivotMesh
InstantMesh
InstantMesh (Decimated)
PivotMesh
Different from 3D generation methods based on neural representations like InstantMesh (Xu et al., 2024a), our methods can generate compact and sharp meshes with much fewer faces when producing similar shapes.
Figure 1:
meshes into discrete tokens. We also adopt a two-stage decoding strategy to decode mesh tokens from face level vertex level hierarchically, which further improves the reconstruction performance and mesh surface continuity. Second, we employ auto-regressive Transformer to learn the joint distribution pivot vertices and complete mesh tokens, where the pivot vertices serve as the coarse representation guide the following mesh generation. Specifically, pivot vertices selected based on vertex degree and dropped randomly to prevent overfitting. As shown in Figure (top), once the model is trained, it can produce meshes from scratch, starting with the generation of pivot vertices followed by the complete mesh token sequence. Furthermore, can perform conditional generation given the pivot vertices from the reference mesh and supports downstream applications.
PivotMesh is designed to be scalable and extensible. We initially evaluate its effectiveness on small dataset ShapeNet (Chang al., 2015) as previous settings (Siddiqui et al., 2023). Next, carefully curate and train our model on the existing largest 3D datasets Objavese (Deitke et al., 2023) and Objaverse-xt xl (Deitke al., 2024). By leveraging large datasets, our model can generate generic meshes across various categories to accelerate the mesh creation process. Both the qualitative and quantitative experiments show that the proposed PivotMesh beats previous mesh generation methods like PolyGen (Nash et al., 2020) and MeshGPT (Siddiqui et al., 2023) by a large margin.
The contributions of this paper can be summarized as follows:
We propose generic and scalable mesh generation framework PivotMesh, which makes initial attempt to extend the native mesh generation to large-scale datasets.
We present Transformer-based autoencoder preserve the geometry details and surface continuity in meshes by efficiently decoding from face level to vertex level hierarchically.
We introduce pivot vertices guidance for complex mesh geometry modeling, which serves as coarse representation guide the complete mesh generation in coarse-to-fine manner.
PivotMesh achieves promising performance in various applications like mesh generation, variation, and refinement, accelerating the mesh creation process.
RELATED WORKS
Neural 3D Shape Generation. Most previous attempts learn 3D shape with various representations,
e.g., SDF grids (Cheng et al., 2023; Chou et al., 2023; Shim et al., 2023; Zheng et al., 2023) and neural fields (Gupta et al., 2023; Jun & Nichol, 2023; Müller et al., 2023; Wang et al., 2023a; Zhang
2

---

Published as a conference paper at ICLR 2025
Table 1: Difference between MeshGPT and two concurrent works with PivotMesh. Our main contribution mainly falls on two aspects. First, our autoencoder yields great reconstruction perfor- mance with shorter sequence length. Second, our model can produce more complex typology with the pivot guidance. n is the number of faces and is the number of vertices.
Difference
MeshGPT
MeshXI.
MeshAngthing
PivotMesh
AE Architecture AE Decoding
GNN-CNN
N/A
Transformer
Transformer
Face-level
N/A
Face-level
Feve-level & Vertex-level
Sequence Type
Latent
Coordinates
Latent
Pivot woot-guided Latent
Generation Formulation
Direct
Direct
Direct
Coarse-to-fine 0.1e 6n
Sequence Length
6n
9n
9n
Compression Ratio (L)
66.7%
100%
100%
68.9%
et al., 2023; Liu al., 2023b; Lyu al., 2023). To improve the generalization ability, researchers start to leverage pretrained 2D diffusion models (Rombach et al., 2022; Saharia et al., 2022; Liu et al., 2023a) with score distillation loss (Poole et al., 2023; Lin et al., 2023; Wang al., 2023b) in a per-shape optimization manner. Multi-view diffusion models (Shi et al., 2023b; Weng et al., 2023; Zheng & Vedaldi, 2023; Shi al., 2023a; Chen et al., 2024d; Voleti et al., 2024) used to further enhance the quality and alleviate the Janus problem. Recently, Large Reconstruction Models (LRM) (Hong al., 2023; Li et al., 2023; Xu et al., 2023; Wang et al., 2024; Xu al., 2024b; Tang et al., 2024a; Xu et al., 2024a) train the Transformer backbone on large scale dataset (Deitke et al., 2023) to effectively generates generic neural 3D representation and shows the great performance of scaling. However, these neural 3D shape generation methods require post conversion (Lorensen & Cline, 1998; Shen et al., 2021) for downstream applications, which is non-trivial and easy to produce dense and over-smooth meshes.
Native Mesh Generation. Compared with the well-developed generative models of neural shape representations, the generation of the mesh remains under-explored. Some pioneering works try to tackle this problem by formulating the mesh representation as surface patches (Groueix al., 2018), deformed ellipsoid (Wang et al., 2018), mesh graph (Dai & Nießner, 2019) and binary space partitioning (Chen et al., 2020). PolyGen (Nash et al., 2020) uses two separated auto-regressive Transformers to learn vertex and face distribution respectively. Polydiff (Alliegro et al., 2023) learns the triangle soups of mesh with a diffusion model. MeshGPT (Siddiqui al., 2023) is most relevant to our work, which first tokenizes the mesh representation with GNN-based encoder and learns the mesh tokens with a GPT-style Transformer. Despite its promising results on small datasets, is non-trivial to extend MeshGPT to the large-scet scale dataset. Our research, along with several concurrent works (Chen et al., 2024a;b;e; Tang et al., 2024b) as shown in Table I, is designed to build generic generative model for native mesh generation within large-scale datasets.
3 METHOD
In this section, we will introduce the details of the proposed PivotMesh shown in Figure 2. The challenges to scale up native mesh generation are analyzed Section 3. Meshes formulated as triangle face sequences are first encoded into discrete tokens by the proposed mesh autoencoder (Section 3.2). Then, we use an regresseee Transformer to learn the joint distribution of pivot vertices and mesh tokens (Section 3.3).
3.1 CHALLENGES FOR NATIVE MESH GENERATION ON LARGE DATASETS
There are two main challenges for scaling up native mesh generation to large datasets.
Mesh Reconstruction. It is challenging tokenize meshes due to its high requirement on recon- struction accuracy to preserve the mesh surface continuity. Previous works like MeshGPT (Siddiqui et al., 2023) formulate meshes face graphs for reconstruction. which only focuses on face-level relationships and neglects the connection and interaction among vertices. Furthermore, the limited network capability of autoencoder (i.e., GNN and CNN) also hinders its scalability large-scale
3

---

Published as a conference paper at ICLR 2025
Vector Quantization Module
Transformer Face Encoder
Transformer Face Decoder
Face
Face Embedding
Vert.
Vert.
Vert.
Pivot Vertices (P)
Full Mesh Tokens (T)
Transformer Vertex Decoder
Reconstruct
Auto-regressive Transformer
(a) Hierarchical Mesh Auto-enoder
(b) Pivot-guided Mesh Transformer
Figure 2:
The overall method of PivotMesh. (a) Triangle mesh sequences are tokenized into mesh tokens and hierarchically decoded from face level to vertex level via our mesh autoencoder. (b) The nuto-regressive Transformer first learns to generate pivot vertices coarse mesh representation and then generates the complete mesh tokens in a coarse-to-fine manner.
datasets. To this end, we propose Transformer-based autoencoder to preserve the geometry details and surface continuity by decoding from face level to vertex level hierarchically.
Complex Typology Modeling. Due to the complexity and variety of mesh topology, directly modeling the mesh sequence on large-scale datasets makes it casy to produce trivial meshes with simple geometry (e.g., cubes). To model complex mesh typology, a natural solution is to first generate coarse representation and then the full mesh sequence. For this purpose, (the sequence of high-degree vertices) as the coarse representation of meshes. With the guidance of pivot vertices, our model is capable of generating complex mesh geometry in coarse-to-fine manner.
define pivot vertices
3.2 ENCODE MESIES INTO DISCRETE TOKENS
triangle mesh M with n faces can be formulated as the following sequence:
(I) where each face Si consists of 3 vertices and each vertex contains 3D coordinates discretized with a 7-bit uniform quantization. To effectively learn the mesh distribution, we first tokenize the sequence into discrete tokens with the proposed transformer-basee autoencoder.
M:= (f1, f2, fn) (V11, V12, V13, U21, U22, U23: Un1, Un2, Un3),
Attention-bann Tokenizer. Different from MeshGPT equipped with a GNN-CNN-based autoen- coder, we employ a Transformer-based architecture the backbone for the encoder, capturing the long-range relationship between faces. Furthermore, we replace the vanilla positional encoding in the Transformer with single-layer GNN to capture the local topology of meshes. This preserves the permutation invariance of faces with higher scalability, yielding more effective and robust token representation for meshes.
Hierarchical Decoding. To further improve the reconstruction performance and mesh surface contiguity, design hierarchical decoder from face level to vertex level. The face embedding F? from the vector quantization module is first passed to a face-level decoder. Then, the decoded face embedding is converted to vertex embedding V by simple MLP. The vertex embedding is then decoded by a vertex-level decoder, whose architecture is similar to the face decoder except that its input sequence 3 times longer.
F FaceDec(FF),
V VertexDDec(LLP..-sCl (F.).
The final decoded vertex embedding V, then converted to the quantized 3D coordinate logits (1,2,..,,7) for each axis (x, y and 2) and computes the cross entropy with the input mesh
4

---

Published as a conference paper at ICLR 2025
Simple Geometry
A (deg: 1)
Face Sequence:
F12 F13; F2,, F22, F23)
F1
Vertex Sequence (MeshGPT):
B
(deg:2)
2)
(A,
Complex Geometry
Pivot guided Sequence (PivotMesh):
D (deg: 1)
(B,C A, A,B,C,
toy mesh faces
Vertices: A, B, C,
Foces: F1, F2
Coarse Rep.
Full Rep.
MeshGPT
PivotMesh
Figure 3: Illustration for givodguiiedd mesh sequence. is hard to directly generate vertex sequence with geometry details (MeshGPT formulation) on large scale datasets. In our paper, we found that some vertices (with high degrees) repeatedly vertices for coarse mesh representation. By first generating pivot vertices and then the full meshes, our model capable of producing more complex geometry with higher quality.
in vertex sequence thus define them pivot
sequence. Such hierarchical architecture allows the connection and interaction among both face and vertex level, thus improving the reconstruction accuracy and surface continuity.
3.3 GUIDE MESH GENERATION WITH PIVOT VERTICES
To model complex mesh typology, it is natural to first generate coarse representation and then the full mesh sequences. However, non-trivial find such a coarse representation preserve typology information with short length. As shown in Figure 3, we found that some vertices repeatedly occur the mesh sequence (since such vertices connect multiple faces), therefore they are highly informative. Furthermore, these vertices are with high degrees thus preserving more geometry details. To this end, we define these vertices as pivot vertices, and propose to first generate them the coarse representation for full mesh sequence.
Degree-based Pivot Vertices Selection. First, we need to select the most frequently occurring vertices the pivot vertices. Specifically, a mesh can be regarded as a graph, where each vertex U; represents a node and the connection between vertices represents the edges. Then, we compute the vertex degree dvg(v;) in mesh graphs and select the pivot vertices set P with the top-degree vertices. the number of vertices with fixed ratio Neelect-
The size of the pivot vertices set is proportional Furthermore, to prevent overfitting in pivot-to-men to-mesh modeling, we randomly drop some pivot vertices with the ratio Adrpp of all vertices for each training iteration. In experiments, the select ratio Tselect = 15% and the dropping ratio ']rop = 5%. yielding the final pivot vertex ratio = 10%. The benefits of our pivot selecting strategy fall into two aspects. First, it leverages frequently occurring vertices (with higher degree), enabling the Transformer to utilize these as conditional tokens mesh sequence generation efficiently. Second, it tends preserve intricate mesh details, as regions with finer geometry typically necessitate more faces and thus larger vertex degrees.
Coarse-to-fine Mesh Modeling. As shown in Figure 3, we employ a standard auto-regressive Transformer decoder learn the joint distribution of the pivot vertex tokens p; P and the complete mesh tokens 4 E T. A learnable start and end token are used to identify the beginning and end the token sequence, while pad token used separate the pivot vertex tokens and mesh tokens. The order of both pivot vertices tokens and full mesh tokens is sorted by z-y-z coordinates from lowest to highest. Different from the Transformer in Section 3.2, we add absolute positional encoding here to indicate the position in the token sequence. The token sequences are modeled with Transformer with parameter by maximizing the log probability:
I pttlfti P:
II p(p (pll-pr))))))
5

---

Published as a conference paper at ICLR 2025
PolyGen
MeshGPT
PivotMesh
Figure 4: Qualitative comparison of unconditional generation on ShapeNet. Each line represents subset of ShapeNet (bench, chair, lamp, table).
With such formulation, the regressiee Transformer first learns generate pivot vertex tokens P as coarse mesh representation and then generates the complete mesh tokens 7' in a coarse-to-fine manner. In such coarse-to-fnee manner, our model can effectively learn the complex mesh typology, which can be easily extended to large-scale datasets. Once the model is trained, it can produce meshes from scratch, starting with the generation of pivot vertices followed by the full mesh sequence. Furthermore, different from previous mesh generation methods (Nash et al., 2020; Siddiqui et al., 2023), our model can perform conditional generation given the pivot vertices from the reference mesh, and also supports downstream applications like mesh variation and refinement.
4 EXPERIMENT
4.1 EXPERIMENT SETTINGS
Datasets. Our model is trained on various classes and scales datasets, including ShapeNetV2 (Chang al., 2015), Objaverse (Deitke al., 2023), Objaverse-xl (Deitke et al., 2024). For ShapeNet, follow previous settings (Nash et al., 2020; Siddiqui et al., 2023), we use 4 subsets (chair, table, bench, lamp) and filter the faces less than 800 after plannar decimation (with a fixed angle tolerance apply the data curation and filter the objects whose
== 109). For Objaverse and Objaverse-xl, faces are less than 500 without decimation to preserve the mesh quality. The final datasct size after filtering of Shapenet, Objaverse, and Objaverse-xl x is around 10k, 40k, and 400k respectively. For each dataset, we split 1 samples for testing, and leave the rest as the training data. For data augmentation, we use random scaling on each axis and random shifts (+0001) enhance the data diversity.
Baselines. We benchmark our approach against leading mesh generation methods: PolyGen (Nash et al., 2020), which generates polygonal meshes by first generating vertices followed by faces conditioned on the vertices with two separate Transformer; MeshGPT (Siddiqui et al., 2023), which tokenizes the mesh sequence into mesh tokens with a GeN-ResNet based autoencoder and learns the mesh tokens directly with Auto-regressive Transformer. We only visually compare with MeshXL (Chen et al., 2024a) since we use different training data and can not directly compute the metrics.
Metrics. Following the evaluation settings previous mesh generation methods (Siddiqui et al., 2023; Alliegro et al., 2023), we evaluate the reconstruction quality by two metrics, triangle accuracy and 12 distance. We use the following metrics for mesh quality assessment: Minimum Matching Distance (MMD), Coverage (COV), and -Nearest-Neighhor Accuracy (1-NNA). For MMD, lower better; for COV, higher is better; for 1-NNA, 50% is the optimal. We use the Chamfer Distance (CD) to compute these metrics 1024-dim point clouds uniformly sampled from meshes.

---

Published as a conference paper at ICLR 2025
PevotMeshh
MesXXL
MeshGPT
PolyGen
Figure 5: Qualitative comparison of unconditional generation Objaverse.
Table 2: The unconditional generation results on Shapenet dataset. The proposed PivotMesh
significantly improves the mesh quality compared with baselines by large margin.
Model
PolyGen
MeshGPT
PivotMesh
Subset
COV(%J
MMDr10 Р)
I-NNA(%%)
COV(S6J
MMD(10 KN
I-NNA(%)
COV(S))
MMD(1 'N I-NNA(%6)
Chair
29.47
13:34
81.45
41.31
10.30
61.84
52,89
9.77
56.71
Tahle
38.67
15.84
66.27
43,00
9.77
62.83
51.68
9.28
56.55
Bench
37.50
10.8
79.69
Lamp
31.76
33.87
81.76
45,88
23.43
57.06
50.58
22.65
54.71
Mixed
33.91
13.47
74.63
43,89
11.48
63.24
50.42
11.03
60.89
Implementation Details.
autoencoder, the face encoder has 12 layers with a hidden size of 512, the face decoder has 6 layers with a hidden size of 512, and the vertex decoder has 6 layers with hidden size of 256. For vector quantization, the number of residual quantizers r 2, and the codebook is dynamically updated by exponential moving averaging with codebook size 16384 and codebook dimension 256. It is trained on an 8xA100 100-800B machine for around day with a batch size of 64 for each GPU. For regressive Transformer, it has 24 layers with hidden size of 1024. is trained on an 8xA 100000880BB machine for around 3 days with batch size 12 for each GPU. The temperature used for sampling is to 0.5 to balance the quality and diversity. We flash attention for all Transformer architecture and fp16 mixed precision to speed up the training process. We use AdamW Loshchilov Hutter (2017) as the optimizer with 0.9 and Bg 0.99 with learning rate of 10 for all the experiments.
4.2 MESH GENERATION FROM SCRATCH
Comparison with baselines on various-scale benchmarks. We first evaluate the proposed Piv- otMesh on the commonly used benchmark Shapenet with four selected categories, chair, table, bench,
Table 3: The unconditional generation results on the Objaverse and Objaverse-xl dataset.
Model
Objaverse
Objaverse-xl
Dataset
COV(S))
MMD(10-3),
I-NNA(5F)
COV(S))
MMD(10-31)
I-NNA(%%
PolyGen
23.86
24.01
84.07
21.79
22.68
83.40
Me
35.03
17.30
63.86
41.50
14.76
64.25
PivotMesh
46.48
16.66
58.55
45.30
14.33
57.75

---

Published as a conference paper at ICLR 2025
Input Point Cloud
Input Point Cloud
PivotMesh
MeshAnything
PivatMesh
MeshAnything
Figure 6: Further conditioning point clouds. It shows that our model produce more complex typology with the pivot vertex guidance.
Table 4: Ablation study of autoencoder and auto-regressive Transformer on Objaverse. (a) The proposed autoencoder reconstructs meshes with much higher accuracy compared with MeshGPT, showing the effectiveness of Transformer architecture and hicrarchical decoding strategy. (b) Without the degree-based selection strategy (i.e., random selecting pivot vertices), the generation performance will greatly reduce and even worse than that without pivot vertices, showing the importance of degree-based selection strategy.
Method
Accuracy($)" L2 Distancel
Method
COV($E) MMD(10 3) I-NNA($)
w/o Transformer (MeshGPT)
86.89
10.34
w/o pivot guidance
42.76
16.83
61.38
w/o hierarchical
92.30
degree selection
42.48
17.62
62,90
Ours
y7.89
1.05
Ours
46.48
16.66
58.55
(a) Ablation study autoencoder.
(b) Ablation study on aulo-regressive transformer.
and lamp. Following the previous setting (Siddiqui et al., 2023; Alliegro et al., 2023), we first pretrain our model on the mixture dataset of four selected categories and then finetune each category separately. We report the generation results both on the mixed dataset and each subset in Table 2. Furthermore, train model the larger scale datasets Objaverse and Objaverse-xl xl and report the performance Table 3. For all these experiments, our method can achieve state-of-the-an performance all evaluation metrics. As shown in Figure 4 and Figure 5, model generate meshes with the best visual quality and geometry complexity. MeshGPT can produce complete meshes but it is trapped in simple geometry due its network capability and the complex mesh sequence. With the hierarchical autoencoder and pivot vertices guidance, our model can produce compact meshes with sharp details and complex geometry. Besides unconditional generation, we also compare with MeshAnything in point cloud conditioning as shown in Figure 6. Our model shows the advantages of modeling complex mesh geometry.
Shape Novelty Analysis. To show that our model can create novel shapes instead of memorizing the training set, conduct shape novelty analysis similar to (Hui et al., 2022; Erkoç al., 2023; Siddiqui et al., 2023). We generate 500 shapes and search the 3 closest neighbors from the training set measured in Chamfer Distance (CD), shown in Figure 7 (left). We also show top-1 CD distribution between the generated shapes and the training set in Figure 7 (right). The CD results show that our method not only covers shapes the training set (low CD values) but also creates novel and realistic shapes (high CD values).
4.3 ABLATION STUDY
The effectiveness of Transformer-based hierarchical autoencoder. For mesh autoencoder, pre- vious works like MeshGPT (Siddiqui et al., 2023) employ GNN the encoder and CNN as the
8

---

Published as a conference paper at ICLR 2025
50t Percenole
MgR Porceztile
0.00 0.02
0.03
0.04
0.06
Generated Shapes
Most similar shupes from training set
Reequency
Figure 7: Shape novelty analysis on Objaverse dataset. We show the 3 nearest neighbors measured in Chamfer Distance (CD) for generated shapes (left). We plot the distribution of 500 generated shapes from our method and their minimum CD to the training set (right). Shapes at the 50th percentile look different from the closest train shape. It shows that our method not only covers shapes in the training set (low CD values) but also creates novel and realistic shapes (high CD values).
Figure 8: Conditional generation results with pivot vertices guidance from Objaverse test set. The generated meshes marked in blue and the ground truth meshes marked in gray. The diversity of generation results also shown on the bottom line.
decoder. Table (w/o Transformer) shows the reconstruction performance of such architecture, which significantly lower than our Transformer-basee autoencoder. The performance gain is mainly from the high capability and scalability of the Transformer compared with the GNNs and CNNs. Table 4 (w/o hier. decode) shows the performance of the autoencoder when removing the hierarchical decoding mechanism. It shows that such hierarchical network design endows the decoder with better alignment both face and vertex levels, thus improving the final reconstruction results.
The effectiveness of pivot vertices guidance and their selection. Table 4 (w/o pivot guidance) shows the results of directly employing an auto-regressive Transformer to model the mesh tokens from our autoencoder. Without the pivot vertex guidance, the model fails to produce meshes with complex geometry. instead of the proposed degree-based strategy. Some metrics from its results are even worse than that without using pivot vertices guidance. The strong performance degradation because the randomly selected vertices make it hard summarize the whole geometry of the meshes and the Transformer incapable of learning such mesh joint distribution, showing the importance of pivot vertices selection strategy.
Table 4 (w/o degree selection), we employ random pivot vertex selection

---

Published as a conference paper at ICLR 2025
(a) Mesh Variation
(b) Coarse Mesh Refinement
Figure 9: PivotMesh can support various downstream applications (Reference meshes marked in gray). (a) Mesh variation: PivotMesh generates diverse meshes similar to reference meshes but with different details. (b) Coarse Mesh Refinement: PivotMesh refine the details for coarse meshes to accelerate the mesh creation.
4.4 PIVOT VERTICES GUIDANCE ANALYSIS
To trade-off between the generalization and the visual quality for pivot-guided mesh generation, we first pretrain model in the mixture of objaverse and objaverse-xl datasets. Then, finetune the pretrained model on the well-curated objaverse to further improve the generated mesh quality.
Pivot-guided Mesh Generation. Given reference mesh, we first encode it into mesh tokens and then select the pivot vertices. Our model can generate the corresponding meshes as shown in Figure 8. shows that our model can generate diverse and high- meshes while maintaining the high-level structure corresponding to the pivot vertices.
Downstream Applications. PivotMesh can serve as generic mesh generative model support various applications in Figure 9, For mesh variation, model can generate diverse variants with the user-given meshes. For mesh refinement, can be regarded as special case mesh variation, but with coarse meshes as input. Our model can refine the coarse meshes to fine meshes with detailed geometry to accelerate the mesh creation process.
4.5 LIMITATIONS AND FUTURE WORK
Although PivotMesh can produce compact meshes with high quality, it still has some limitations. First, the controlling ability of PivotMesh is still not enough. Since the same pivot vertices may indicate diverse meshes, our method may sometimes produce undesired geometry. This can alleviated by adding more control conditions like images and texts and we leave it for our future work. Second, the scale of data and the number of model parameters is still limited. Due to the constrained computation resources, only train PivotMesh meshes less than 500 for both Objaverse and Objaverse-xl. and the number of model parameters is much less than recent advances large language models (Touvron et al., 2023).
5 CONCLUSION
In this paper, we introduce PivotMesh, scalable framework to generate generic meshes with compact and sharp geometry. By employing pivot vertices as coarse representation to guide the mesh generation process and leveraging a Transformer-based hierarchical autoencoder, PivotMesh demonstrates its capability to generate high-quality meshes on both small and large-scale datasets. The proposed model significantly outperforms existing methods and sets new benchmark for native mesh generation tasks, showcasing its potential for creating novel shapes and supporting downstream applications.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENT
This work was funded in part by the National Natural Science Foundation of China grant under number 62222603, in part by the STI2030-Major Projects grant from the Ministry of Science and Technology of the People's Republic of China under number 20212D0200700, in part by Youth Fund of the National Natural Science Foundation of China (No. 62306163), and in part by China Postdoctoral Science Foundation (No. 2023M741951), in part by the Key-Area Research and Development Program of Guangdong Province under number 2023B0303030001, in part by the Program for Guangdong Introducing Innovative and Entrepreneurial Teams (20192T08X214) in part by the Science and Technology Program of Guangzhou under number 2024A04J6310.
REFERENCES
Antonio Alliegro, Yawar Siddiqui, Tatiana Tommasi, and Matthias Nießner. Polydiff: Generating 3d polygonal meshes with diffusion models. arXiv preprint arXiv:2312.11417, 2023. 1,3, 6, 8
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, al. Shapenet: An information-rin 3d model repository. arXiv preprint 1rXiv:1512.0300
2015. 2,
Sijin Chen, Xin Chen, Anqi Pang, Xianfang Zeng, Wei Cheng, Yijun Fu, Fukun Yin, Yanru Wang, Zhibin Wang, Chi Zhang, et al. Meshxl: Neural coordinate field for generative 3d foundation models. arXiv preprint 24Xx:2405.20853, 2024a. 3, 6
Yiwen Chen, Tong Hc, Di Huang, Weicai Ye, Sijin Chen, Jiaxiang Tang, Xin Chen, Zhongang Cai, Lei Yang, Gang Yu, et al. Meshanything: Artist-created mesh generation with autoregressive transformers. Xrviv preprint arXiv:2406. 10163, 2024b. 3
Yiwen Chen, Yikai Wang, Yihao Luo, Zhengyi Wang, Zilong Chen, Jun Zhu, Chi Zhang, and Guosheng Lin. Meshanything v2: Artist-created mesh generation with adjacent mesh tokenization. arXiv preprint 2tXiv24488.02555 2024c.
Zhiqin Chen, Andrea Tagliasacchi, and Hao Zhang. Bsp-net: Generating compact meshes via binary space partitioning. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 45-54, 2020.
Zilong Chen, Yikai Wang, Feng Wang, Zhengyi Wang, and Huaping Liu. V3d: Video diffusion models are effective 3d generators. arXiv preprint WXXv:2403.006738 2024d. 3
Yen-Chi Cheng, Hsin- Ying Lee, Sergey Tulyakov, Alexander G Schwing, and Liang- Yan Gui.
Sdfusion: Multimodal 3d shape completion, reconstruction, and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4456-4466 2023. 2
Gene Chou, Yuval Bahat, and Felix Heide. Diffusion-sdff Conditional generative modeling of signed distance functions. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2262-2272, 2023. 2
Angela Dai and Matthias Nießner. Scan2mesh: From unstructured range scans to 3d meshes. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp.
5574-5583, 2019. 3
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of anno- tated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13142-13153, 2023. 2, 3,
Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram Voleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances Neural Information Processing Systems, 36, 2024. 2, 6
Ziya Erkoç, Fangchang Ma, Qi Shan, Matthias Nießner, and Angela Dai. Hyperdiffusion: Generating implicit neural fields with weight-space diffusion. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 14300-14310, 2023. 8
11

---

Published as a conference paper at ICLR 2025
Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. A papier-máché approach to learning 3d surface generation. In Proceedings of the IEEE conference computer vision and pattern recognition, pp.
2018. 3
Anchit Gupta, Wenhan Xiong. Yixin Nie, Ian Jones, and Barlas Oguz. 3DGen: Triplane Latent Diffusion for Textured Mesh Generation, March 2023. 2
Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint Siii:2331.004000 2023. 1,
Ka-Hei Hui, Ruibui Li, Jingyu Hu, and Chi-Wing Fu. Neural wavelet-domain diffusion for 3d shape generation. In SIGGRAPH Asia 2022 Conference Papers, pp. 1-9, 2022. 8
Heewoo Jun and Alex Nichol. Shap-e: Generating conditional 3d implicit functions. arXiv preprint HrXiv:2305.02466 2023. 2
Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. arXiv preprint arXiv:2311.06214 2023. 1,3
Chen-Hsuan Lin, Jun Gao, Luming Tang. Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yo Yu Liu, and Tsung- Yi Lin. Magic3D: High-Resolution Text-to-3D Content Cre- ation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 3
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. 1-to-1 Zero-shot One Image to 3D Object, March 2023a. 3
Zhen Liu, Yao Feng. Michael J. Black, Derek Nowrouzezahrai, Liam Paull, and Weiyang Liu. MeshD- iffusion: Score-based Generative 3D Mesh Modeling. In The Eleventh International Conference on Learning Representations, February 2023b.
William Lorensen and Harvey E Cline. Marching cubes: A high resolution 3d surface construction algorithm. In Seminal graphics: pioneering efforts that shaped the field, pp. 347-353. 1998. 3
Iya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 1rXiv:111..01001 2017.
Zhaoyang Lyu, Jinyi Wang, Yuwei An, Ya Zhang, Dahua Lin, and Bo Dai. Controllable mesh genera- tion through sparse latent point diffusion models. Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pp. 271-280, 2023.
Norman Müller, Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Bulo, Peter Kontschieder, and Matthias Nießner. Diffrf: Rendering-guiee guided 3d radiance field diffusion. In Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition, pp. 4328 44388,33.2-- 2023. 2
Charlie Nash, Yaroslav Ganin, SM Ali Eslami, and Peter Battaglia. Polygen: An autoregressive generative model of 3d meshes. In International conference machine learning, pp. 7220-7229.
PM 2020. 2, 3, 3,6
Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. DreamFusion: Text-to-3D using 2D Diffusion. In ICLR, arXiv, 2023. 1,3
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-Resolution Image Synthesis With Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. 2022.
12

---

Published as a conference paper at ICLR 2025
Tianchang Shen, Jun Gao, Kangxue Yin, Ming- Yu Liu, and Sanja Fidler. Deep marching tetrahedra: a hybrid representation for high-resolution 3d shape synthesis. Advances in Neural Information Processing Systems, 34:6087-6101, 2021. 3
Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero single image to consistent multi-view diffusion base model. arXiv preprint arXiv:2310.15111 2023a.
Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d generation. arXiv preprint arXiv:2308. 16512, 2023b. 3
Jachyeok Shim, Changwoo Kang, and Kyungdon Joo. Diffusion-based signed distance fields for 3d shape generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 20887-20897, 2023.
Yawar Siddiqui, Antonio Alliegro, Alexey Artemov, Tatiana Tommasi, Daniele Sirigatti, Vladislay Rosov, Angela Dai, and Matthias Nießner. Meshgpt: Generating triangle meshes with decoder-only transformers. arXiv preprint w1..v:2311.1555.. 15475, 2023, 1,2, 3, 6,
Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation. arXiv preprint arXiv:2402.05054, 2024a.
Jaxxian Tang. Zhaoshuo Li, Zekun Hao, Xian Liu, Gang Zeng, Ming-Yu Liu, and Qinsheng Zhang. Edgerunner: ruto-segressive auto-encoder for artistic mesh generation. arXiv preprint arXiv:2409.18114, 2024b.
Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov. Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cristian Cantón Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel M. Kloumann, A. V. Korenev, Punit Singh Koura, Maric-Anne Lachaux. Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, R. Subramanian, Xia Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zhengxu Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. ArXiv, aabs/2307.09285 09288,, 2023. URL https:/ aapi. semant i ceschola aar. wwr.or//oooooo prrg/CorpusDD 259950998. 10
Vikram Voleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view synthesis and 3d generation from single image using latent video diffusion. arXiv preprint arXiv:2403. 12008, 2024.
Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. Pixel2mesh: Generating 3d mesh models from single rgb images. In Proceedings of the European conference on computer vision (ECCV), pp. 62- 2018. 3
Tengfei Wang, Bo Zhang, Ting Zhang, Shuyang Gu, Jianmin Bao, Tadas Baltrusaitis, Jingjing Shen, Dong Chen, Fang Wen, Qifeng Chen, et al. Rodin: A generative model for sculpting 3d digital avatars using diffusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4563-4573, 2023a.
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolific- Dreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation, May 2023b. 3
Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xiang, Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, and Jun Zhu. Crm: Single image to 3d textured mesh with convolutional reconstruction model, arXiv preprint 45X0v:2403.05034, 2024. 3
13

---

Published as a conference paper at ICLR 2025
Haohan Weng, Tianyu Yang, Jianan Wang, Yu Li, Tong Zhang, CL Chen, and Lei Zhang. Consistent123: Improve consistency for one image to 3d object synthesis. arXiv preprint arXiv:2310.08092, 2023. 3
Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint arXiv:2404.07191, 2024a. 2.
Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang, Jiahao Li, Zifan Shi, Kalyan Sunkavalli, Gordon Wetzstein, Zexiang Xu, et al. Dmv3d: Denoising multi-view diffusion using 3d large reconstruction model. arXiv preprint arXiv:231 1.09217, 2023. 3
Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang. Sida Peng, Yujun Shen, and Gordon Wetzstein. Grm: Large gaussian reconstruction model for efficient 3d reconstruction and generation. arXiv preprint 14X1v:2403.16621, 2024b. 3
Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neural fields and generative diffusion models. ACM Transactions on Graphics (TOG), 42(4):1-16, 2023. 2
Chuanxia Zheng and Andrea Vedaldi. Free3d: Consistent novel view synthesis without 3d represen- tation. arXiv preprint arXiv:2312.04551, 2023. 3
Xin-Yang Zheng, Hao Pan, Peng-Shuai Wang, Xin Tong, Yang Liu, and Heung- Yeung Shum. Locally attentional sdf diffusion for controllable 3d shape generation. ACM Transactions Graphics (TOG), 42(4):1-13, 2023. 2
14

---

Published as a conference paper at ICLR 2025
A ADDITIONAL RESULTS
A.I
FAILURE CASES
Pivot Vertices
Generated
Ground Truth
Pivol Vertices
Generated
Ground Truth
Figure 10: Some failure cases for the pivot-guided mesh generation. Since the same pivot vertices may indicate diverse meshes, our method may sometimes produce undesired geometry. This can be alleviated by adding more control conditions like images and texts.
A.2
USER STUDY
As shown in Table 5, conduct the user study two dimensions Aesthetics and Complexity scores, which range from 0-5 (higher is better).
Table 5: User study on the generated meshes. The users are asked to score on two dimensions mesh aesthetics and complexity, which range from
(higher is better).
Method
Aestheticst
Complexity!
PolyGen
2.49
2.35
MeshGPT
3.20
2.50
PivotMesh
3.60
3.38
A.3 COMPARISON WITH INSTANTMESII
To produce similar shapes for a more straightforward visual comparison, we use the rendering images of our generated mesh as the image condition for InstantMesh to generate the following instances in Figure and Figure 11.
InstantMesh (Decimated)
InstantMesh (Decimated)
PivetMesh
InstantMesh
Figure 11: Additional results for comparison with InstantMesh.
15

---

Published as a conference paper at ICLR 2025
A.4 UNCONDITIONAL GENERATION ON SHAPENET
Figure 12: Additional results for unconditional generation on Shapenet.
16

---

Published as a conference paper at ICLR 2025
A.5 UNCONDITIONAL GENERATION ON OBJAVERSE
LUILTU
Figure 13: Additional results for unconditional generation on Objaverse.
17

---

Published as a conference paper at ICLR 2025
A.6 PIVOT-GUIDED MESH GENERATION
Pivot Vertices
Generated
Ground Truth
Pivot Vertices
Generated
Ground Truth
Figure 14: Additional results for pivot-guided mesh generation on Objaverse.
A.7
SHAPE NOVELTY ANALYSIS
Generated
Most similar shupes from training
Generated
Most similar shapes from training set
Figure 15: Additional results for shape novelty analysis on Objaverse.
18

---

Published as a conference paper at ICLR 2025
A.8 PIVOTMESH APPLICATIONS
(a) Mesh Variation
(b) Coarse Mesh Refinement
Figure 16: Additional results for the applications of PivotMesh.
A.9 MESHES WITH COMPLEX TOPOLOGIES
Meshes with thin structures
Meshes with holes
Figure 17: Generated Meshes with more complex (opologies(espccially for meshes with holes and thin structures.)
A. COMPARISON WITH MORE METHODS
Input Point Cloud
MeshArything
MeshArythinguz
Input Point Cloud
PivottVlesh
MeshAnything
MechArythinge2
Figure 18: Comparison with concurrent works Meshanything and Meshanythingy2 for point-cloud conditioning.
19

---

Published as a conference paper at ICLR 2025
A.11 11
DATA DISTRIBUTION
4Face Distribution Training Dataset
#Fac Distribution Dataset
1750
1250
750
250
100
200
300
4000
500
100
200
300
400
300
The Number Tace
The Number of Face
#Veettces Distribution Training Dataset
#Vertices Distribution Dataset
100
0000
Regerrd
Requercy
80
4000
2003
20-
200
400
600
800
Figure 19: The distribution for the number of vertices and faces in the training and testing data.
20


Reviews:
[{"review_id": "yQ7aCT1vGg", "reviewer": "Reviewer_2uyx", "summary": "The paper introduces PivotMesh, a framework for generating compact and detailed 3D meshes at scale. The approach leverages a transformer-based auto-encoder to encode meshes into discrete tokens, which are then decoded hierarchically from face to vertex level. A key contribution is the use of pivot vertices as a coarse representation to guide the generation of complete mesh tokens, allowing for complex topology modeling. The method is evaluated on diverse datasets, including ShapeNet, Objaverse, and Objaverse-xl, demonstrating its versatility and effectiveness in generating high-quality 3D meshes.", "questions": "The paper is well-written and presents a contribution to the field of 3D mesh generation. I am impressed with the quality of the generated results, but there are some minor issues: the comparison with other similar baselines and more diverse generated shapes. With the above suggestions addressed, the paper would be a strong candidate for publication. I recommend accepting this paper with minor revisions.\nDetailed questions please refer to weaknesses.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. PivotMesh addresses a significant challenge in 3D mesh generation by proposing a scalable framework that can handle large-scale datasets with simplified triangles. The use of pivot vertices as a coarse representation for guiding mesh generation is innovative and effectively handles complex topologies.\n2. The paper provides a thorough evaluation of PivotMesh across various datasets and applications, including mesh generation, variation, and refinement. The generated meshes are of high quality, with sharp details and complex geometries, outperforming existing methods.", "weaknesses": "1. Diversity of Generated Meshes: While the paper shows diverse mesh generation, a more systematic analysis or quantification of diversity could strengthen the results. The evaluations on more diverse shapes with complex topologies are encouraged to be conducted, such as some shapes with lots of holes, the thin structures (ficus in nerf-synthetic data), and so on.\n2. More direct comparisons with the current state-of-the-art methods, especially in terms of computational efficiency and mesh quality, could be beneficial. There is more relative work, such as meshanything(2), EdgeRunner, the comparison is very essential to demonstrate the superiority of the proposed method.\n3. The limitations and failure cases should be discussed comprehensively. And the paper acknowledges that the controlling ability of PivotMesh is not sufficient, and sometimes undesired geometries are produced. Enhancing the control mechanisms could be a valuable addition.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Zw1rYsM2lY", "reviewer": "Reviewer_szcv", "summary": "In this work, the authors propose a novel framework for explicit mesh generation that can be trained on large-scale datasets. They employ a transformer-based autoencoder to obtain discretized tokens representing the mesh, which are later decoded hierarchically. Additionally, the generative networks are designed to first produce a sequence of pivot mesh vertices as a coarse representation, then use these as conditions to generate the actual mesh tokens. Experimental results demonstrate that the proposed framework achieves comparable and often superior performance to concurrent works. The authors also showcase various downstream applications of their method.", "questions": "- Including MeshXL's quantitative results would be highly beneficial, despite its use of a different training dataset. While this is concurrent work and the results would be for reference only, they could serve as a valuable benchmark, helping readers better understand the effectiveness of the proposed method.\n- The paper claims that the proposed pivot mesh formulation can generate shorter sequences, providing estimated sequence lengths in Table 1. However, measuring these lengths on a real dataset would offer more clarity and insight into the method's efficiency.\n\nJustification of Rating:\nOverall, this work provides a novel direction for compact mesh generation and demonstrates its effectiveness in various aspects. The quantitative and qualitative comparisons show that it achieves good performance compared to existing and concurrent works. Additionally, various applications have outlined its potential beyond unconditional generative tasks. Despite some minor suggestions, I am currently leaning towards acceptance of this work.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The exploration of generating pivot vertices before faces appears to be a novel approach. Ablation studies validate its effectiveness, particularly without pivot guidance and degrees selection.\n- Extensive experiments compare unconditional generation quality across both small and large datasets, such as ShapeNet and ObjaverseXL. The proposed framework outperforms existing methods in all settings.\n- The framework demonstrates versatility through various applications, including point cloud-guided generation, mesh variations, and coarse mesh refinement.\n- A novel analysis ensures the diversity and originality of the generated objects.", "weaknesses": "Although concurrent works are mentioned, several studies have begun exploring the generation of compact meshes in large datasets, demonstrating some success to a certain extent. It is recommended to revise the introduction and abstract to avoid claiming that existing work is difficult to extend to large datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "px2yrontk8", "reviewer": "Reviewer_EqsP", "summary": "This paper focuses on conditional triangle mesh generation based on the pivot vertices of the provided mesh.  The authors proposed to tokenize the mesh with transformer, rather than the GCN architecture used by MeshGPT. The pivot-guided method is shown to outperform PolyGen and MeshGPT.", "questions": "- It is confusing how the discrete mesh tokens from auto-encoder and pivot vertices are combined for conditional mesh generation.  \n- In lines 303-306 of page 6, the authors mentioned about face filtering in the datasets. Is there a distribution map to summarize the number of vertices and faces in the training and test meshes?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- A transformer-based auto-encoder is proposed to encode the triangle meshes into discrete tokens.\n- An auto-regressive transformer is proposed to generate the complete mesh tokens from pivot vertices.\n- By conditioning on pivot vertices, the proposed method outperforms PolyGen and MeshGPT. \n- Downstream applications of the method are shown for mesh variation and coarse mesh refinement.", "weaknesses": "- Both the auto-encoder and auto-regressive transformer use 24 layers of transformers, resulting the method to be memory taken and less efficient.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)