Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a 3D textured mesh generator trained using 2D images, known camera distributions and object silhouettes. The generator is composed of a geometry generator, which generates the SDF value and vertice deformations to a tetrahedron field. The mesh is then extracted using differentiable marching tets. The texture generator generates a texture field, which predicts the reflectance of a point in space using its coordinates and the latent codes. By differentialbly rendering them into images and sillhouettes, image-based discriminator is used to provide the signal for training. 
In addition, the authors also presented two variants, first is generating the material maps instead of reflectance, where the generated objects can be relit. Second, using a clip discriminator, the generator can be tailored to perform the task of text-to-textured-mesh. 
strengths_and_weaknesses: Strengths:
Originality:

Even though the core compnent of the system is based on prior works(nvdffrast, tet marching), the system itself is novel and nontrivial to setup. As far as I know, the proposed system is the first of its kind to generate high-quality textured mesh. 

Significance:

The proposed system tackels multiple drawbacks of previous results; it uses mesh as the final geometric representation, but also not constrained by topology nor utilizes a template mesh. The quality of the result is also appealing, making it a promising candidate for generating assets for graphics applications. 

Quality:

This paper is of high quality. In addition to technical details, this paper presents multiple experiments with two variants of the method to support the validity of their design choices and the potential of the method. 

Clarity:

This paper is well written and easy to follow.
questions: In general I find this paper above the acceptance bar. The following questions are intended to elicit more insights and details of the proposed methods:

1. How robust is the proposed method to distribution shifts of camera poses? Since it samples from the distribution of camera poses, does it really matter to have access to the exact camera pose, which seems to be a limitation of the method to apply to real images?

2. In addition to camera pose, there's also the matter of camera intrinsics. For real images, those are usually unknow and varying as well. Will the method robust enough to handle that?

3. How delicate is the system to train successfully? Though GANs are hard to tune overall, it would be nice to have some insights on how easy it is for others to re-train the system on different categories.
limitations: The authors provided adequate discussions on the limiation of the method.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a generative method to produce textured 3D mesh that can be directly used in general 3D rendering engines. The overall pipeline contains a geometry generator, a texture generator, a differentiable rendering module, and two discriminators. The geometry generator exploits DMTet as surface representation, and a tri-plane texture field is used for the texture generator.  Besides, the differentiable module is used to predict the 2D silhouette and 3D on-surface coordinates that are used to query the corresponding RGB color in the texture field. The two discriminators are inspired by StyleGAN for silhouette and rendered RGB images to achieve RGB-supervised training, respectively.  In general, this paper is a combination of DMTet, texture field, NVdiffrec, and StyleGAN. While the performance excell SOTA method, the novelty is limited. 
strengths_and_weaknesses: The strength of this work is the generated texture mesh can be directly used in general 3D rendering engines, which is not suitable for previous works. Meanwhile, the supervision is 2D images that are more widely available than 3D geometry data. 
The weakness of this work contains:
1. Although this work only requires 2D images as supervision, all experiments are trained with rendered RGBs with known camera parameters and light variation. The application for in-the-wild RGB images needs more discussion. Hence the application scenario of this method is limited. 
2. The author claims their method can generate high-quality 3D textures meshes, but local details (e.g. face) of the human character are missed (as in Fig.1, Fig.5). A qualitative comparison of local detail with previous SOTA methods (e.g. EG3D) will make this claim more valid. 
3. The lack of quantitative comparison of human character. 

Typo: 
P7, L222, "Tbl. 2". 
Supp. P3, L101, "Fret"

questions: Please provide some discussion about the weakness 
limitations: Yes
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposed GET3D, a generative model that synthesizes high-quality 3D meshes with textures.
The authors put the recent 3D shape reconstruction work [40] into a styleGAN architecture and demonstrated that this framework is capable to generate diverse 3D meshes with various textures. The experiments on multiple datasets show the proposed method is superior to the prior arts.
strengths_and_weaknesses: The writing is clear and the idea is easy to follow.

The authors closely followed the most recent 3D mesh reconstruction work nvdiffrec [40] and developed further to incorporate the 3D reconstruction pipeline into a GAN. I am glad to see that without many bells and whistles, the proposed GET3D works pretty well on multiple datasets.

The experiments are sufficient overall, though I do have some minor comments.

Weakness:
1. It is great that the generated 3D textured meshes can be directly used in the production tools like Blender. I wonder how the texture is represented in the final output as I don't think these tools can utilize texture fields? Is it easy to get the UV map from the proposed model? Or does "texture" only mean "colored vertices"?
2. Based on Table 1, [46] actually is closer to what the authors proposed in this paper (without the texture part though). Why [46] is excluded from the comparison?
3. What is the illumination model in the differentiable rendering part? If I understand correctly, it is a Phone model with only 1.0 ambient lighting.
4. Though in L255, the authors claimed that "hence the subdivision cannot provide further improvements", in Table 2, subdiv actually hurts the performance seriously. Any idea?
5. Shape interpolation is impressive. It would be great to see the texture interpolation as well (though the title of Fig 6 is shape interpolation, the texture has also changed...why...).
questions: I do not have major concerns. Just some minor questions and comments as listed in the weakness section.

Overall I think this is a high-quality paper. The idea is straightforward and the results are convincing. Though I have some minor concerns, I lean into accepting this paper.
limitations: The authors listed two major limitations in their work: 1. the evaluation was done on the synthetic data only and 2. per category training. The authors also discussed the potential social impact of their work applying on privacy or biased data. All these sound sufficient to me.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: - A generative model of 3D models for 3D content creation should output a mesh of detailed geometry and any topology and must be trained using 2D images without 3D supervision. The proposed method is the first that has these properties. The proposed realizes it by combining a 3D representation of meshes with any topology (DMTet) [51] (S.3.1.1), texture fields [43] (S.3.1.2), differentiable rendering [31], and StyleGAN [29] (S.3.2).
- The experiments conducted on ShapeNet, Turbosquid, and Renderpeople demonstrate that the variety and quality of generated 3D models by the proposed method are better than existing 3D mesh generation models and 3D-aware image generation models. Also, the proposed method is applied to (1) unsupervised decomposition of geometry, light, and material, and (2) text-guided 3D model generation.
strengths_and_weaknesses: #### Strength

- The proposed method can handle topology changes as it uses DMTet as a 3D representation, different from The existing generative model of meshes [46]. It is a significant step toward realistic 3D content generation models.
- Comparison with existing 3D generation methods [46,11] are conducted on multiple datasets, and quantitative improvement is certain in the geometry diversity and visual quality metrics. Also, it is a bit surprising that the proposed method generates better images than recent 3D-aware image generation models inspired by NeRF.
- Qualitative results are impressive and clearly outperform the baselines. Improvement of texture is explained by the image resolution in training (Table 3), which is also an interesting finding.
- Though no quantitative comparison is provided, the quality of text-guided 3D model generation is significantly better than Text2Mesh [37].

#### Weakness

- The proposed method is a straightforward combination of existing methods [51,43,31,29], and each component is technically not novel, although what is realized by the combination is novel.
- Training the proposed method requires the distribution of camera parameters and silhouette images, so it practically requires a synthetic dataset of 3D models, as noted in l.300. Therefore, the cost of training datasets is still not very lower than existing methods that require 3D supervision.
- The generated shape and texture look a bit coarse when zoomed up, though they are finer than [46,11].
- DMTet is not very good at representing high-resolution scenes as it stores information in 3D grids. Therefore, it is difficult by the proposed method to generate a large-scale scene containing multiple objects, such as bedrooms, unlike 2D GANs.
- The result of unsupervised geometry, lighting, and material decomposition look appealing, but I am not very sure the proposed method is actually good because quantitative comparison is not provided.

Although the technical contribution is not very strong, I recommend acceptance because of the impressive results shown in the experiment.
questions: It is written that it is difficult to extract meshed textures from neural fields (l.45, l.85), but what exactly is the difficulty? That is understandable when volume rendering is used, but when SDF is used, I guess the difficulty is the same as the proposed method using texture fields.
limitations: It is described that it is difficult to apply the proposed method to natural datasets because the distribution of camera poses and silhouettes images are assumed to be known. No other serious limitations were found.
review_title: null

Meta-review: metareview: 
The paper proposes a generative model for synthesizing textured 3D meshes given only a collection of 2D images. The paper has received overwhelmingly positive reviews. Many reviewers find the idea interesting, the paper well-written, the results compelling, and the experiments comprehensive. The rebuttal further addressed the concerns such as camera poses and missing comparisons. The AC agreed with the reviewers’ consensus and recommended accepting the paper. 
confidence: Certain
Decision: Accept