GenUDC: High Quality 3D Mesh Generation With Unsigned Dual Contouring Representation

Authors: sharp  parts
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-12 07:47:47

Abstract: Generating high-quality meshes with complex structures and realistic surfaces is the primary goal of 3D generative models. Existing methods typically employ sequence data or deformable tetrahedral grids for mesh generation. However, sequence-based methods have difficulty producing complex structures with many faces due to memory limits. The deformable tetrahedral grid-based method MeshDiffusion fails to recover realistic surfaces due to the inherent ambiguity in deformable grids. We propose the novel GenUDC framework to address these challenges, leveraging the Unsigned Dual Contouring (UDC) as a better mesh representation. UDC discretizes a mesh in a regular grid and divides it into the face and vertex parts, recovering both complex structures and fine details. As a result, the one-to-one mapping between UDC and mesh resolves the ambiguity problem. In addition, GenUDC adopts a two-stage, coarse-to-fine generative process for 3D mesh generation. It first generates the face part as a rough shape and then the vertex part to craft a detailed shape. Extensive evaluations demonstrate the superiority of UDC as a mesh representation and the favorable performance of GenUDC in mesh generation. The code and trained models will be released upon publication.

Main paper: 
 introduction: 
 Mesh plays an important role in 3D content generation and reconstruction [21, 40, 41, 68] , AR/VR [22] , robotics [20, 65] , and autonomous driving [4, 9, 18] and other 3D tasks. It can flexibly represent various complex geometric shapes. High editability allows meshes to be modified and adjusted easily in computer-aided design (CAD). Additionally, it is effortless for users to convert meshes to other 3D representations, e.g., voxels, point clouds, and neural implicit functions. Besides, the rendering pipelines are designed for meshes, enabling high-quality 3D visualization effects. However, Employing deep neural networks on meshes is tricky because the numbers of vertices and faces are constantly changing, and modeling the complex topology structure of faces is also an obstacle. To navigate those challenges, a mesh representation compatible with deep learning and a capable generative framework adapted to this mesh representation are both highly desired.Most existing approaches focus on intermediate representations, e.g., voxels [57, 71] , point clouds [1, 45, 46] , neural implicit functions [12, 33, 68, 77, 78] and so on, which are highly compatible with deep learning. However, those methods require a post-processing step [10, 13, 15, 28, 43, 58] to extract meshes, resulting in over-smooth surfaces and lacking detailed geometry. PolyGen [50] first treats vertices and faces as sequences and uses transformer networks [66] to generate vertices and then faces. MeshGPT [63] and Poly-Diff [2] follow similar ideas but concentrate on faces. All three approaches cannot produce mesh with intricate geometry since the memory limits the number of faces to no more than 2800. MeshDiffusion [42] chooses to combine a deformable tetrahedral grid with Signed Distance Functions (SDF) to model meshes. However, its data preparation is especially slow (Tab. 6), and the generated meshes are crumpled due to the deformable nature of the grid and the inaccurate 2D supervision.To address these two challenges, we construct a novel framework dubbed GenUDC to combine the Unsigned Dual Contouring representation (UDC) with a two-stage, coarse-to-fine generative process, enabling high-quality mesh generation. As a mesh counterpart, UDC consists of a face part and a vertex part. Accordingly, we decompose the mesh generation into two subtasks: the face part generation and vertex part generation. To address these subtasks, we devise the two-stage, coarse-to-fine pipeline, which involves generating faces first and then vertices.Precisely, to find a proper mesh representation, we expand Dual Contouring [28] , which has long been regarded as an isosurface reconstruction method, to generation tasks. Thus, we obtain the UDC representation to model meshes as shown in Fig. 1 (a) and Fig. 2 . In UDC, we discretize a mesh in a regular grid. The faces part of UDC is a set of tiny faces represented by boolean values. The vertex part of UDC contains all the actual and potential vertices of those tiny faces. Since the values of the face part and vertex part are arranged in a regular grid, we can conveniently employ deep learning-based generative models to learn the distribution of UDC.Another pivotal component is the two-stage, coarse-to-fine generative process specially designed for UDC. Because the mesh is discretized in UDC, the face part draws the rough shape, and the vertex part describes the details. Consequently, we first employ a latent diffusion model to generate the face part, determining the mesh's rough shape and topological structure. Then, conditioning the rough shape, we take a vertex refiner to generate the vertex part. Such a two-stage, coarse-to-fine pipeline is a natural solution. Without this pipeline, the edges would be jagged due to the inaccurate vertex part. We will study the necessity of this two-stage, coarse-to-fine pipeline in Sec. 4.4.Finally, using GenUDC, we can produce high-quality meshes with complex structures and realistic details as shown in Fig. 1 (b ) and Fig. 3 . Comprehensive experiments demonstrate the superiority of our method over existing ones in mesh generation. In data fitting, compared with MeshDiffusion, our method runs at 3274% times their speed and consumes only about 13% of their total memory as shown in Tab. 6.To summarize, the contributions of this paper are:• We propose a novel generative framework, called GenUDC, utilizing UDC as the representation for high-quality mesh generation. • We design a two-stage, coarse-to-fine generative pipeline to UDC, which generates faces and then vertices, circumventing the jagged edges problem. • Extensive experiments demonstrate our method's superior performance in mesh generation and data fitting.  
related work: 
 In this section, we will outline some closely related topics to our study: 3D shape generation, isosurface reconstruction, and diffusion models.With the advent of deep learning, researchers have been exploring the generation of 3D voxels [34, 57, 67, 71, 73] and point clouds [1, 3, 17, 26, 29, 32, 36, 45, 46, 61, 72, 74] using neural networks. However, voxels suffer from memory limits, and point clouds lack topology of shapes. Until the dawn of neural implicit function [12, 49, 53] , the community finds it an excellent shape representation, which does not require a lot of memories and can be easily transformed into meshes. The neural implicit function is specially designed for advanced deep neural networks and inspires a lot of work [14, 16, 25, 27, 31, 33, 35, 37, 48, 51, 60, 62, 68, 70, [76] [77] [78] . It utilizes SDF values or occupancy values as the intermediate representation of 3D shapes. By using some isosurface reconstruction methods like Marching Cube [15, 43] , and Dual Contouring [28] , meshes can be reconstructed from those neural implicit functions. However, this also means those implicit function-based methods still require a post-processing step and cannot directly generate meshes. A collection of existing methods [6-8, 11, 39, 54, 64, 69, 75] adapt Neural Radiance Fields (NeRF) or 3D Gaussian Splatting (3DGS) as 3D representation. They utilize the powerful text-toimage generative model, Stable Diffusion [55] , as the guidance to optimize NeRF or 3DGS with a text prompt. After optimization, the final NeRF or 3DGS contains both 3D shape and texture information. The mesh can be extracted from it by some post-processing methods. However, they are time-consuming, taking hours of optimization for each text prompt. They also suffer from artifacts such as oversaturated colors and the multi-face problem.Moreover, some works are trying to find a proper mesh representation to generate meshes directly. PolyGen [50] , MeshGPT [63] , and PolyDiff [2] are inspired by natural language processing to process meshes as sequences. By leveraging the power of transformer network [66] , they can theoretically produce vertices and faces of any length. In practice, the limited memory constrains the complexity of synthetic mesh structures, making it difficult for them to generate curved surfaces. In MeshDiffusion [42] , a deformable tetrahedral grid and SDF values are utilized to recover meshes. It supposes all mesh vertices are on the edges of the deformable tetrahedral grid. It can use linear interpolation to compute mesh vertices with the coordinates of adjacent grid points and SDF values. After getting the mesh vertices, It produces faces by connecting mesh vertices in the same tetrahedrons. However, the deformable grid brings ambiguity to the fitting mesh. Two different tetrahedral grids with distinct SDF values may recover the same mesh. In addition, to fit a mesh, the deformable grid is trained in the supervision of rendered images, which are inaccurate due to various rendering settings. The ambiguity and 2D supervision tend to result in deficient surfaces shown in Fig. 7 . As for data preparation, it takes too much time and memory to fit a tetrahedral grid on a shape due to the 2D supervision, as shown in Tab. 6 .In contrast to sequence-based methods, e.g., PolyGen, MeshGPT, and PolyDiff, our method is capable of using limited memory to generate a diverse range of mesh structures, such as flat surfaces, thin parts, curved surfaces, sharp parts, and so on, as shown in Fig. 1 (b) . Compared with MeshDiffusion, we use a regular grid to fit meshes with more accurate results, less processing time, and less memory, as shown in Tab. 6 and Fig. 7 . We taxonomize methods that can directly generate meshes in Tab. 1. We present more details of the data fitting comparison between MeshDiffusion and ours in Sec. 4.5.Typically, isosurface reconstruction methods extract meshes from volume data (e.g. voxels and SDF volumes). As a pioneering work, the original Marching Cubes (MC) method is proposed by Lorensen and Cline [43] . It discretizes a mesh into a regular grid and creates approximative surfaces in each cube according to intersections between the mesh and grid. Its most well-known variant, MC33 [15] , can even model all possible topological cases in a cube. However, since vertices of approximative surfaces are on the edges of the grid, it is hard for the marching cubes method to model sharp parts. The Dual Contouring method (DC) [28] is thus proposed. Its vertices of approximative faces (also called dual faces) are in the cubes. So DC can recover sharp parts. With the rise of deep learning, Deep Marching Cubes [38] first applied deep learning to isosurface reconstruction. Neural Marching Cubes (NMC) [13] and Neural Dual Contouring (NDC) [10] focus on building a learnable version of MC and DC. Manifold Dual Contouring [58] and FlexiCubes [59] try to solve the non-manifold problem in DC. VoroMesh [47] introduces Voronoi diagrams to isosurface reconstruction.All isosurface reconstruction methods focus on developing a pipeline to transform a kind of 3D data into a counterpart of mesh. In contrast, we adopt UDC and expand it to shape generation by learning the distribution of the UDC representations. In other sections, with a little abuse of the abbreviation, we refer to UDC representation as UDC.Diffusion models are a class of deep generative models that play an important role in artificial intelligence generated content (AIGC). It achieves pleasant results in various applications, such as image generation [24, 55] , image super-resolution [55] , shape generation [42] , text-to-3D [16, 35] , etc. Diffusion models are designed to model the step-by-step transformations between a simple distribution (e.g. Gaussian distribution) and a complex distribution of data. Once trained, a diffusion model can map a sample of the simple distribution to the desired data distribution. As a milestone of diffusion models, the Denoising Diffusion Probabilistic Model (DDPM) [24] introduces variational inference into diffusion models and shows greater potential over generative adversarial networks [19] . But it still suffers from the huge memory requirement. Therefore, the latent diffusion model (LDM) [55] proposes to train diffusion models in a low-dimensional latent space instead of the high-dimensional data space. It has been demonstrated that this technique can speed up training and reduce memory footprints without degradation of generation quality. In this paper, we adopt the LDM in the face part generation (see Sec. 3.3) since the regular grid takes a lot of memory footprints. 
method 3.1 overview: 
 Aiming at mesh generation, how to represent meshes, and how to process meshes with neural networks are two critical issues. To address them, we propose GenUDC, a novel generative framework for mesh generation. In GenUDC, we discretize a mesh in a regular grid to get its corresponding Unsigned Dual Contouring representation (UDC). Thus, due to the regular grid structure of UDC, neural networks can easily be used on both watertight and non-watertight meshes. We further propose a two-stage, coarse-to-fine pipeline adapted to UDC, which generates faces and vertices successively. In summary, we offer a new and straightforward solution for mesh generation.In the following sections, we first elaborate on UDC in Sec. 3.2. Then, we illustrate our generative models for face generation in Sec. 3.3 and vertex generation in Sec. 3.4. Finally, the implementation details are presented in Sec. 3.5.We have briefly shown the main ideas of the Unsigned Dual Contouring representation (UDC) in Fig. 1 (a) and Fig. 2 . For more details, in a grid G with the size of (𝑋 + 1, 𝑌 + 1, 𝑍 + 1), UDC can be formalized as:EQUATIONwhere C are the cubes in the grid, V are the vertices, E are the edges inside the grid, and F are the faces (also called dual faces) denoted by the intersection flags of edges. The grid G contains (𝑋 + 1)(𝑌 + 1)(𝑍 + 1) nodes. There are |C| = 𝑋𝑌𝑍 cubes in the grid, and each cube contains a vertex 𝑣 ∈ V. Considering the edges along the x-axis, y-axis, and z-axis, there are |E | = 𝑋 (𝑌 -1) (𝑍 -1) + (𝑋 -1)𝑌 (𝑍 -1) + (𝑋 -1)(𝑌 -1)𝑍 inside edges. If the intersection flag of an edge is true, four adjacent vertices make up two triangle faces that are dual to the edge. In other words, the edge intersects with one of the two triangle faces when the flag is true. If not, there is no face intersecting with this edge. When translating a UDC to a correlative mesh, we craft faces by traversing all intersection flags in F and remove a subset of V which are not in these faces. By this means, faces and remaining vertices comprise the final mesh.Compared with the traditional SDF-based methods [42, 77] , which usually generate over-smooth shapes, UDC can easily model the sharp parts as shown in Fig. 7 . In addition, the rigid grid used in UDC is suited for deep neural networks and can produce more realistic surfaces than the deformable grid of MeshDiffusion [42] , which will be evaluated in Sec. 4.5. Moreover, UDC has the potential to model non-watertight shapes shown in Fig. 7 .In practice, V are the relative coordinates in each cube, which means 0 ≤ 𝑚𝑖𝑛(V) and 𝑚𝑎𝑥 (V) ≤ 1 and F are boolean values. When 𝑋 = 𝑌 = 𝑍 , we pad F with zeros to the same size as V. We call V as the vertex part and F as the face part.Data Fitting. We follow a similar procedure of DC [28] to fit a mesh with UDC. Given a mesh M = (V M , F M ) and a grid G = (C, E), we first find the crossing vertices V E of the mesh M on the edges E. Then, we compute the normals N E of M at those crossing vertices. With V E and N E , we can create UDC as:EQUATIONEQUATIONThe dual contouring vertices V should be on the surfaces of M. So we extrapolate neighboring normals N E to find a point of best fit in each cube:EQUATIONwhere 𝑣 𝑥 𝑦𝑧 is the vertex inside the cube C 𝑥 𝑦𝑧 which is indexed by (𝑥, 𝑦, 𝑧), and 𝑒 are 12 edges of C 𝑥 𝑦𝑧 . 0 ≤ 𝑥 < 𝑋 , 0 ≤ 𝑦 < 𝑌 , and 0 ≤ 𝑧 < 𝑍 . By default, if there is noV E 𝑒 or N E 𝑒 in a cube, 𝑣 𝑥 𝑦𝑧 is set to [0.5, 0.5, 0.5].Besides, we only craft faces F when M intersects with an edge𝑒 ∈ E at the crossing vertex 𝑣 ∈ V E :EQUATION)In UDC, we have devised a simple and intuitive method for generating faces by connecting them with intersection flags. If an edge's intersection flag is true, it crosses faces. If not, there is no face. By this means, we can denote all faces of the mesh as boolean values and arrange them into a regular grid as a face tensor F ∈ B | E | . We can easily employ typical neural networks to face part generation thanks to the regular grid.To reduce the memory footprint, we use a Latent Diffusion Model (LDM) [55] to learn the distribution of F . Our LDM consists of a Variational AutoEncoder (VAE) [30] and a diffusion model [24, 55] . VAE compresses a F to a latent representation 𝑧. Then, we take latent representations 𝑧 to train our diffusion model. Thus, by extracting the compression process from the generative learning phase, we can speed up the diffusion model training process and reduce the memory footprints. And since the latent space is perceptually equivalent to the input space, there is no quality reduction for the diffusion model. We provide detailed descriptions of VAE and the diffusion model below.A VAE comprises an encoder 𝐸 and a decoder 𝐷. Given a face tensor F ∈ B | E | , we first normalize F to [-1.0, 1.0] using min-max normalization. Then 𝐸 encodes F to a mean code 𝜇 ∈ R 𝑐 ×𝑑 ×ℎ×𝑤 and a standard deviation code 𝜎 ∈ R 𝑐 ×𝑑 ×ℎ×𝑤 . We use the mean code 𝜇 as the latent code 𝑧 ∈ R 𝑐 ×𝑑 ×ℎ×𝑤 without reparameterization, which differs from the typical VAE. Finally, 𝐷 decodes 𝑧 back to the face tensor F 𝑝𝑟𝑒𝑑 = 𝐷 (𝑧). We train our VAE with the mean squared error (MSE) loss and the Kullback-Leibler divergence (KL) loss:EQUATIONSince we do not use the reparameterization technique, our VAE is more like an autoencoder (AE) producing compact latent codes (close to zero).Diffusion Model. After encoding the face part F to the latent code 𝑧 with our VAE, we employ a diffusion model [24, 55] to the latent code distribution 𝑝 (𝑧). We first normalize 𝑧 0 ∈ 𝑝 (𝑧) to [-1.0, 1.0]. Then, through a series of diffusion steps, we introduce the controlled Gaussian noise 𝜖 to 𝑧 0 and transform it to𝑧 𝑡 = √ ᾱ𝑡 𝑧 0 + √ 1 -ᾱ𝑡 𝝐,where 𝑡 = 1 . . . 𝑇 and ᾱ𝑡 = 𝑡 𝑖=1 𝛼 𝑖 . 𝛼 𝑡 = 1 -𝛽 𝑡 and 𝛽 𝑡 is the predefined variance. The diffusion model 𝜃 is trained to predict the noise 𝜖, aiming at reversing the diffusion steps. The training objective isEQUATIONAfter training, to generate a face part F , a sampled Gaussian noise 𝜖 ∼ N (0, 1) is seen as 𝑧 𝑇 . Then, our trained diffusion model denoises 𝑧 𝑇 to 𝑧 0 . 𝑧 0 is further denormalized from [-1.0, 1.0] to the original The vertex part V is a set of relative vertex coordinates, and all vertices are arranged in a regular grid. Each vertex is in a cube of this grid. The vertex part contains all actual and potential vertices of a mesh. Since several vertices compose a face, there is a tight correlation between V and F . Therefore, learning this correlation is a crucial problem in the vertex part generation.In UDC, when the face part is determined, the rough shape is known, and the variance of the vertex part is limited. So we treat the vertex part generation as a regression task. We take a vertex refiner to generate V conditioned on F . Here, we use a 3D version of U-Net [56] as the vertex refiner. Note that F is padded to the same size as V described in Sec. 3.2. Firstly, we normalize the face part F and the vertex part V to [-1.0, 1.0]. Secondly, the vertex refiner takes the F as the conditional input and generates a vertex part V 𝑝𝑟𝑒𝑑 ∈ R 3× | C | as shown in Fig. 2 . We compare V 𝑝𝑟𝑒𝑑 with the ground truth V 𝑔𝑡 to train networks:EQUATIONEQUATIONwhere L 𝑚𝑠𝑒 is MSE loss. V 𝑔𝑡 is the ground truth vertices paired with F . In the inference phase,V 𝑝𝑟𝑒𝑑 is denormalized from [-1.0, 1.0] to [0.0, 1.0].This is a natural and efficient solution to learn the correlation between F and V for vertex part generation with reasonable training costs. If we eliminate the vertex refiner and generate F and V together, synthesized meshes will contain jagged edges due to inaccurate vertex coordinates. We will illustrate the necessity of the vertex refiner in Sec. 4. 4 .More details of the network are in the supplemental material.If In contrast, the diffusion model is trained in a category-specific manner. We use the AdamW optimizer [44] with 𝛽 1 , 𝛽 2 = [0.9, 0.999] for all networks. Empirically, large 𝛽 values can make our diffusion model produce realistic meshes. During the inference of diffusion models, we adopt the sampling method in Denoising Diffusion Probabilistic Models [24] and set the inference step as 1000. 
experiments 4.1 data: 
 Following the protocol of MeshDiffuision [42] , we use the ShapeNet Core (v1) dataset [5] to train and test our networks. Airplane, car, chair, refile, and table -five categories are used in our experiments.For each category, we split all data like [23] and [77] do: 70% as the training set, 20% as the test set, 10% as the validation set. To be clear, the validation set is not used. For a fair comparison, we remove the interior of shapes. We apply the data-fitting method in Sec. 3.2 to generate UDC for all mesh data.To evaluate the quality of shape generation, we compare our method with IM-GAN [12] , SDF-StyleGAN [77] , MeshDiffusion [42] and, LAS-Diffusion [78] . IM-GAN, SDF-StyleGAN, and LAS-Diffusion are neural implicit function-based shape generation methods. IM-GAN predicts the occupancy values. Similarly, SDF-StyleGAN and LAS-Diffusion predict the SDFs. We apply MC to create meshes from synthesized implicit representations, following their protocols.MeshDiffusion is also a mesh generation method that utilizes a deformable tetrahedral grid and SDF values to generate meshes directly. We do not compare ours with PolyGen, MeshGPT, and PolyDiff because it is unfair that their faces are limited to no more than 2800, and they cannot produce complex geometric shapes. Four metrics and three kinds of distances are used in the quantitative experiments as shown in Tab. 2. We take the test dataset as the reference set B and generate samples A of the same number, i.e. |A| = |B|. To calculate chamfer distance (COV) and earth mover's distance (EMD), we sample 2048 points for each mesh of A and B. Note that all point clouds are normalized to [-1.0, 1.0], and meshes are normalized to [-0.5, 0.5]. More details of metrics are elaborated in the supplementary materials.Quantitative evaluation. We present metric values in Tab. 2. Our method outperforms others in most cases, indicating that our approach is superior in terms of quality, diversity, and distribution. Particularly in the car and airplane category, our method demonstrates significantly better performance than others. It can be attributed to our excellent ability to generate details, considering the minimal intra-class variation within cars. We also achieve good performance in high resolution, shown in Tab. 3.Qualitative evaluation. We show rendered meshes of various methods in Fig. 3 . As seen, neural implicit function-based methods tend to produce over-smooth shapes and inaccurate parts, e.g., arms of chairs, wheels of cars, and legs of tables. MeshDiffusion usually produces pits on surfaces due to the ambiguity and inaccurate 2D supervision, which we have examined the reason in Sec. 2.1. The Laplacian smoothing used in MeshDiffusion reduces its generation quality by removing details and thin parts, such as chair arms and legs, rifle barrels, aero engines, and airplane propellers. In contrast, our GenUDC can generate high-quality meshes with realistic appearances, various structures, and rich details. We provide more visual samples in the supplementary materials.Since NDC [10] is an isosurface reconstruction method, we cannot directly compare GenUDC with NDC. Thus, we train NDC and UNDC networks with the default setting of their codes and the data from the airplane category of ShapeNetCore (v1) [5] for 2500 epochs. After training, we apply them to SDFs to create meshes for comparison. Qualitative evaluations are shown in Tab. 4. The performance of NDC and UDNC is poor due to the distribution gap between SDFs generated by SDF-StyleGAN and SDFs for training NDC, shown in Fig. 4 . Fig. 5 shows that NDC and UDNC cannot handle the generated SDFs, resulting in surface distortion and floating artifacts. Overall, integrating NDC (UNDC) into the SDF generation method introduces too many uncertainties, making it unsuitable for mesh generation. In contrast, our GenUDC directly generates high-quality meshes using UDC, demonstrating that our paradigm is more suitable for mesh generation.In this section, we compare the GenUDC to the one without the U-Net to demonstrate the necessity of the vertex refiner, i.e., U-Net.In the one without the U-Net, we concatenate the face part F and vertex part V as a mesh tensor and then use the LDM to learn the distribution of mesh tensors. Other settings are consistent with the vanilla GenUDC. More network details are in the supplementary materials. Then, we take mesh tensors to train the LDM, learning the joint distribution of F and V. At the inference, it simultaneously generates F and V. However, it is quite difficult for a single LDM to learn this joint distribution and build the correlation between F and V. To prove this, we present some similar samples produced by GenUDC with and without U-Net in Fig. 6 . As we can see, removing U-Net results in jagged edges and unsmooth surfaces. Only by modeling the vertex part generation conditioned on the face part, we can learn the correlation between F and V. The quantitative evaluation in Tab. 5 also proves our opinion.MeshDiffusion Car Airplane Rifle Table Ours ChairOur UDC is a discretized mesh counterpart, which requires a data fitting process. In this section, we demonstrate the superiority of UDC in the data fitting process compared with MeshDiffusion, which uses a deformable tetrahedral grid to discretize a mesh. For quantitative evaluation, we randomly select one hundred meshes and record the average processing time and memory footprints in Tab. 6. As shown, UDC outperforms MeshDiffusion in both speed and memory footprint. The reason is that MeshDiffusion uses the rendered 2D images as the supervision of data fitting. Rendering 2D images requires a lot of GPU and CPU resources, and it takes a long time to fit data. In contrast, we only use the CPU to directly calculate the fitting vertices and faces of UDC as we elaborate in Sec. 3.2, which is resource-efficient and fast.To visually illustrate UDC's superiority, we present some samples in Fig. 7 . As seen, MeshDiffusion is unavoidable to produce pits on the mesh surfaces and lack of details, such as the line and crack on the car. The reason is the ambiguity and inaccurate 2D supervision talked about in Sec. 2.1. Laplacian smoothing used by MeshDiffusion even removes details and sharp parts instead of the pits. In comparison, UDC can fit flat surfaces, sharp parts, and curved surfaces with details. 
limitation: 
 Firstly, the main limitation of our method is the non-manifold issue. Since we adopt UDC as the mesh representation, our method inherits the non-manifold issue from DC. However, such an issue rarely occurs. It can be resolved by "tunneling" through vertices/edges or dividing them with the approaches introduced by [52, 58] . Secondly, the memory footprint constrains the resolution of our results. Table 6 : Quantitative evaluation of data fitting in terms of mean processing time, GPU memory footprints, and CPU memory footprints of one hundred samples. All the programs are executed in a single thread, using an NVIDIA RTX 3090 GPU, an Intel i7-10700 CPU, and 64GB of memory. Thirdly, since the face part is a set of boolean values, our models may predict wrong boolean values, resulting in pits on the surface. We can solve this pit problem with the post-processing method in [10] . 
conclusion & future works: 
 In conclusion, we propose a novel 3D generative framework, GenUDC, using the Unsigned Dual Contouring representation (UDC) for highquality mesh generation. Our method can directly generate highquality meshes without using isosurface reconstruction methods. Specifically, following the discretization idea, we fit a mesh in a regular grid to get its UDC representation. Since UDC is composed of the face and vertex parts, we use a two-stage, coarse-to-fine pipeline to learn its distribution. Firstly, we use a latent diffusion model to generate the face part. Secondly, we take a U-Net as a vertex refiner to synthesize the vertex part conditioned on the face part. Experiments demonstrate our superiority over baselines in shape generation and data fitting. The ablation study proves the validity of network design. We believe that our method offers a new paradigm for further work in mesh generation.In the future, we plan to apply GenUDC to various applications, such as text-to-3D, joint generation of texture and shape, single view 3D reconstruction, shape editing, etc.  
 
 Appendix:  


Decision: The paper proposes a generative framework, which aims to generate high-quality 3D meshes with complex structures and realistic surfaces using UDC as the mesh representation. The proposed method addresses challenges in mesh generation via UDC.

This paper accept two final borderline accepts and one initial weak reject. The reviewer who rates as weak reject didn't update the final recommendation. I read the author's rebuttal, and found that the rebuttal well address the second reviewer's concern. Therefore the paper can be accepted for publication at ACM MM.

Reviews: review: paper_summary: paper_summary: The paper introduces the GenUDC framework for high-quality 3D mesh generation, GenUDC uses Unsigned Dual Contouring (UDC) to represent meshes, discretizing them into face and vertex components to accurately recover complex structures and fine details. This approach resolves ambiguity issues through a one-to-one mapping between UDC and the mesh. GenUDC employs a two-stage generative process: first generating a coarse shape via the face part, then refining it with detailed vertices.
strength_weakness: strength_weakness: 1. The paper is clearly written and figures are useful.
2. Qualitative and quantitative evaluations and comparisons with baselines show that the method outperforms the current methods on mesh generation.
3. The first work that utilizing UDC as the representation for high-quality mesh generation.
limitations: limitations: 1. The results in Table 2 are not good for the Chair and Table categories, but there are better results for other categories. Why is this? Is it because the UDC representation is not suitable for this type of geometric generation?
2. Why was there no comparison with Get3D[1]? 


[1] Gao J, Shen T, Wang Z, et al. Get3d: A generative model of high quality 3d textured shapes learned from images[J]. Advances In Neural Information Processing Systems, 2022, 35: 31841-31854.
paper_summary: paper_summary: This article introduces a novel generative framework called GenUDC, which aims to generate high-quality 3D meshes with complex structures and realistic surfaces using Unsigned Dual Contouring (UDC) as the mesh representation. Compared to existing methods based on sequence data or deformable tetrahedral meshes, GenUDC addresses challenges in mesh generation through UDC, achieving precise reconstruction of both face and vertex parts and thus solving the ambiguity problem of deformable meshes. The framework employs a two-stage, coarse-to-fine generation process: first generating the face parts of the rough shape, followed by generating the detailed vertex parts. Experimental results demonstrate that GenUDC excels in mesh generation, being fast and memory-efficient while producing high-quality meshes with intricate structures and realistic details.
strength_weakness: strength_weakness: The GenUDC framework proposed in this article uses the UDC representation to address the challenges of generating complex structures and realistic surfaces, ensuring the creation of high-quality meshes. It employs a two-stage generation process, starting with a coarse stage followed by a fine stage, effectively preserving the accuracy and details of the mesh. GenUDC excels in speed and memory efficiency, capable of efficiently generating meshes with intricate structures and realistic details.
limitations: limitations: 1.The article lacks an in-depth discussion and comparative analysis of the GenUDC framework. It is recommended to highlight its innovations in generating complex structures and realistic surfaces.
2.There are some deficiencies in the experimental design and result presentation. The article lacks a comprehensive comparison and in-depth analysis with other related methods, making it difficult for readers to accurately assess the practical application and advantages of the GenUDC framework in the field of mesh generation.
3.The article uses only one dataset. It is suggested to add two more datasets for comparative analysis. Besides analyzing the advantages of your model, please also analyze its disadvantages. The dataset used in the article may have biases or be insufficient to comprehensively evaluate the performance of the GenUDC framework.
4. There is a lack of discussion on the advantages and limitations of the method as a mesh representation.
5. It is recommended that the authors enhance the discussion on techniques such as neural rendering in future research, improve the experimental design and result presentation, and thereby enhance the academic quality and credibility of the article.
paper_summary: paper_summary: This paper proposes to employ Unsigned Dual Contouring (UDC) as the 3D representation for 3D mesh generation, which exhibits high-quality results. UDC representation converts mesh to a grid-based representation, which can be divided into two parts: vertex parts for expressing high-quality details and face parts for describing the overall shape. Based on this dividing, they propose a method using two-stage, coarse-to-fine generative process to synthesize high-quality results.
strength_weakness: strength_weakness: This paper leverages a novel representation of 3D mesh and produces high-quality results.
Exhaustive experiments validate the superiority of their methods.
limitations: limitations: Writing: 

1.	In the first paragraph:“However, Employing” -> “However, employing”

2.	It’s unclear what are the two challenges in line 126. (Slow preparation? Crumpled? Limits the number of faces?) 

3.	Carefully redraft Paragraph 2 to demonstrate the core challenge(s) of this task. 1) “No more than a specific number (2800)” without any explanation is nonsense. There must be a more fundamental reason. 2) Why does the deformable nature of the grid lead to crumpled mesh? Do you forget any citations or explanations there? 3) Why is 2D supervision there? It’s different from your input. Besides, it can be solved by providing accurate 2D supervision and hence it does not highlight the need of a new 3D representation.

4.	Do not repeat two-stage, coarse-to-fine twice in the same paragraph.


Overall I like the idea and the results exhibited in this paper. However, I have some concerns. 
1.	If F are Boolean values, why is it necessary to normalize it using min-max? 

2.	UDC representation is still limited by the grid resolution like other grid-based representations (e.g., SDF). What are the fundamental features that distinguish it from the SDF representation in terms of better quality? 

3. Can it generalize to objects with different orientations?

4.	The synthesized vertices might be incompatible with the faces. How do you handle this situation?

5.	This paper mentions that text-to-3D would be a future application. Are all the results in the paper synthesized by unconditional generation? How do you generate results in the same category from a noise?