Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This work proposes GenSDF, a method for object surface estimation, parameterized by SDF, from 3D point cloud inputs. The main distinctions of GenSDF to prior work is its ability for high quality generalization to novel classes not seen during training, and the ability to take advantage of "unlabeled" point cloud data (without know SDFs) as supervision during training. GenSDF achieves this by two well designed training phases, first a meta learning procedure to train a general shape prior, by sampling labelled and unlabeled subsets of the labelled training set, and a second semi-supervised step where the fully labelled training set is used in addition to an additional entirely unlabeled set. GenSDF also proposes a well designed loss for using unlabeled point clouds as supervision for an SDF prediction model.

Evaluation is performed on Acronym (a subset of ShapeNet), split into 20 labeled training classes, 70 unlabeled classes for semi-supervised training, and 166 classes for testing. GenSDF significantly outperforms SDF-supervised and unsupervised . GenSDF can also directly generalize to point clouds from scans of the YCB dataset. Ablation studies directly demonstrate the benefit of the proposed two-stage training procedure.
strengths_and_weaknesses: ## Originality
### Strengths
Direct generalization to unseen categories in the domain of surface estimation from point clouds has not been investigated before, and is an important application (see Significance). This method proposes a novel and well designed two stage training procedure, along with a loss function for using unlabeled data samples. The model's design is supported by high quality ablation studies and significantly higher performance than baselines in terms of chamfer distance.

## Quality
### Strengths
1. _Simple, elegant and effective implementation_: The high level idea of learning a general shape prior that can be used for semi supervised training with unlabeled data makes sense, and paper addresses this using a simple but effective approach of repeatedly partitioning the training set into labeled and unlabeled classes. Further, in order to train models on point clouds only, where only the distance to the nearest point in the point set is available, it's necessary to use a loss that is a proxy for SDFs. The proposed loss is a straightforward and improved proxy between nearest point unsigned distance and SDF values relative to prior methods, and includes an additional term to encourage precision closer to the 0-level set.
2. _High performance relative to baselines_: The performance of GenSDF vs. the closest baseline in terms of chamfer distance is 0.351 vs 1.348 for seen and 0.407 vs 2.333 for unseen. This further highlights the poor generalization ability of prior methods.

### Weaknesses
1. _Evaluation to unseen classes_: The Acronym dataset is based on ShapeNetSem. While ShapeNetSem does contain 270 different categories, a lot of these are distinct but semantically similar. For example, Chair, OfficeChair, Stool, SideChair, Barstool, AccentChair are all similar chair-like objects that are distinct categories in ShapeNetSem, and Table, RoundTable, EndTable, DiningTable, CoffeeTable, AccentTable are also all similar table-like objects that are distinct categories in ShapeNetSem. If the dataset contains categories that are distinct, but very similar according to their shape, this may potentially affect to what extent claims can be made about generalization.
2. _Further evaluation on YCB_: The paper shows two qualitative examples on YCB. It would be very helpful to see multiple instances of real world generalization capability, especially of different types of objects. The data is not re-scaled to match the distribution of the original training data. While this makes sense to showcase generalization ability, not doing this pre-processing and showing the results removes a valuable evaluation of the model.

## Clarity
The paper is very well written and easy to follow. The visualizations of the reconstructions and the diagrams are high quality. The tables are clearly formatted.

## Significance
### Strengths
It is useful to develop object surface estimation methods that generalize, so that they can be used in cases where lots of ground truth meshes (necessary to derive SDF for supervision) are not available.

### Weaknesses
Lack of provided context for real world applicability. There is a multitude of ways in which 3D point clouds can be obtained for which we would like to get a detailed and high quality mesh estimate. However, the characteristics, such as the amount and kinds of noise,  or number of points of the point clouds depend on the method they were obtained (e.g. multiple depth sensor readings vs. multi-view stereo). It would be very useful if the paper included some text about how the tasks designed and used for evaluation (e.g. 5K points with a specific kind of Gaussian noise) in the paper are a good proxy for the real world utility of models like GenSDF.

questions: 1. It is clear that GenSDF is better at generalization to unseen categories than prior methods. However, the shape similarity of what are supposed to be different categories in ShapeNetSem affects these claims to some extent. Can the authors please comment on how the fact that there are semantically very similar categories that may end up being split in the seen and unseen sets affects the claims that can be made about generalization ability?
2. Regarding the point under weaknesses in significance, can the authors please provide a bit more context about how they design the evaluation task relative to potential real world applications of GenSDF? Including this in the introduction of the paper would help the reader understand the significance of what the model can do.
limitations: The authors have discussed some limitations of their work throughout the draft. The paper itself doesn't have high potential for negative societal impact.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper considers the task of learning to reconstruct shapes (via the SDF) from "unlabelled" point clouds (no mesh and no normals) using a single network that must generalize for unseen shapes (and shape classes). The network is allowed to train on both labelled data (meshes) and unlabelled data (unoriented point clouds). The proposed method trains with a standard supervised loss on labelled data, and a self supervised loss that builds upon Neural Pull [19] (but corrects it to properly penalization incorrect signs) for unlabelled data. They further train the supervised loss with the self-supervised loss on psudo-unlabelled data, claiming it greatly helps generalize, with ablations studies demonstrating this.
strengths_and_weaknesses: Strengths
- A data driven approach to shape reconstruction with a strong focus on generalization
- Specifically allows to learn to generalize from a large corpus of unlabelled shapes.
- Fixes an important weakness in the NeuralPull training objective
- Proposes introducing unsupervised training into the the initial supervised training step that shows significant improvement in generalization
- Shows large improvement over baseline methods for both unsupervised and supervised methods, without further test time refinement

Weaknesses
- The baselines for this task could be better. 
	- Section 4.4 says that "No split" is essentially ConvOccNet but has lower CD due to a different choice of decoder, but the difference from this choice as decoder is really large. As a result one of the baselines for Section 4.1 should be ConvOccNet with the same decoder as you use.
	- The method is technically given access to 76 more classes via unsupervised training for it to learn to generalize on (since the other methods do not have an unsupervised learning method), while supervised methods only get 20. It would be better to try supervised training on more diverse classes (but a few in each class such that the overall number of supervised meshes to train on is the same) to better emulate a supervised training approach that is trying to generalize to other shape classes. 
	- It would be interesting to have a baseline that does the same setup as your approach (Phase 1 and 2) but change the self-supervised loss to be the same as Neural Pull, which would show how important your change is.
	- For unsupervised methods (Section 4.2), it would be good to compare to IGR without normals (the autodecoder setting), which has a better unsupervised loss (Eikonal term) than SAL and doesn't have the defect that NeuralPull does. For an even stronger baseline, PHASE+FF [* 1] or DiGS [* 2] would be even better (again with the autodecoder setting).
- No quantitative comparison to single network per shape methods (which you call Single-object methods in the supp material), e.g. IGR, SIREN. Although, as you argue in the supp material, this is a different setting, such quantitative results (which you have given a qualitative result for) would serve as a good upper bound for seeing how close your method can generalize.

[* 1] https://arxiv.org/abs/2106.07689
[* 2] https://arxiv.org/abs/2106.10811
questions: Why does adding the semi-supervised training decrease the Chamfer distance on seen classes so much (0.503 to 0.351)? Shouldn't the network have been trained enough to memorise/overfit those shapes/shape classes? What do you believe the semi-supervised training is adding here (we don't need object level generalisation to perform well on seen shapes)?
limitations: A short discussion of limitations is given in the conclusion. No negative societal impacts are discussed, but I agree that there are no direct negative impacts.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Given multiple-category labeled (with GT SDF) and unlabeled point cloud data,  this paper introduces a simple yet effective meta-learning inspired pretraining stage before the conventional semi-supervised stage in order to train a generalizable SDF function model.  Their strategy frequently randomly splits the labeled data into a self-supervsied and GT-supervised group with different categories, which creates diversity of different tasks and encourage a shared embedding network for better model initialization, and better leverage labeled multi-category data as guidance, assisted by self-supervision loss on unlabeled data.


strengths_and_weaknesses: **Strengths**\
[originality]
It is novel to apply episodic scheme on randomly split hybrid training for SDF learning as pretraining;
They further introduce a self-supervised loss regularization item[Section 3.2], which is well considered for the target task;

[quality & clarity] 
The paper is written with a smooth logic flow, and clear explanation on the methods and experiments, with a lot of high-quality visualization and comparison, like Figure 5 in main paper, and figures in the supps. 

[significance]
This simple yet effective training paradigm is helpful and inspiring for a lot of 3D learning tasks, the achieved results[Figure 3, Table 1] are 
much better than compared baselines.

**Weakness**
1. This paper gives me an illusion of as long as you can keep randomly splitting your labeled multi-category dataset
and train with two different losses(GT-supervised and Self-supervised), you can get a well-generalized model with good pretraining weights;
I am surprised at this conclusion, in which the author might want to give more details like training curves, failure cases, to really show the full potential and limitation of this proposed framework, while also explain better on the intuition behind;

2.  No ablation study on the discussed 'Self-supervised Loss Component' in line 163-182, we want experiments to support the authors' argument on this alternative formulation to avoid incorrect sign predictions; 

3. As mentioned by the author in the supp., it seems current work only supports complete point cloud, it doesn't generalize to real-world partial scans, which is the major and crucial application scenerios for semi-supervised learning/Meta-learning to demonstrate its effectiveness. Even though current work is not designed for partial scans, the author should at least show some qualitative results.
questions: 1. What if in stage 1, we simply first train on all the labeled data directly with GT/SSL loss, what will be the results on the next stage of semi-supervised training? The no split in Table 2 seem to be the desired one, but I am not sure whether it is followed by semi-supervised training, and have the exactly same training setting except for the splitting;

2. How to decide the split ratio between labeled and unlabeled dataset, would smaller number of labeled data (like only 5 classes instead of 20/1) lead to big degrade to the methods? How is the split frequency being decided, will it be a big influence factor to the pretraining performance?

3. A recent paper 'Shape As Points: A Differentiable Poisson Solver' can also reconstruct surface from complete point cloud by learning a general geometry equation solver, is it possible to also compare with this paper? Although quantitative results would be better, comparison in text is valid if the experiments become too complex.
limitations: The main limitations are two-fold as mentioned by the authors:
1. It requires multi-category data with ground truth for training;
2. It does not consider partial scans, and may fail on partial and noisy data;
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The main contribution of this paper is to improve the generalization capabilities of implicit representations learning. This paper introduce an episodic training scheme where we split data into subsets with disjoint classes to learn a generalized shape representation at the first stage. The second stage allows the model to ingest large amounts of unlabeled data to diversify the shape model learned in the first stage.
strengths_and_weaknesses: Strengths：

(1) This paper proposes a training scheme that can learn implicit representations from both labeled and unlabeled point cloud data, and the method produces effective experimental results.

(2) Enhancing the implicit representation learning capability of the network using both labeled and unlabeled point cloud data simultaneously is a meaningful research problem, and from the experimental results, the paper also proposes a framework that can combine both kinds of data effectively.

Weaknesses：

Even though I am knowledgeable about works related to implicit representation learning, I have difficulty understanding the reasons why the framework proposed in the paper works outperforms existing neural SDF methods. No convincing explanation is given in the paper for this proposed framework to achieve such a large effect of enhancement.
questions: (1) In the first stage, every f epochs, this paper splits data X into two subsets with disjoint categories, and then train both data with self-supervised loss and supervised loss respectively. According to my understanding, training the network with ground-truth signed distance values supervision through the supervised loss will give more accurate shape priors. I do not understand the role played by the self-supervised loss here. Even though the paper gives a vague explanation: “episodic training on non-overlapping categories forces the model to develop generalized representations as it cannot overfit to the data.” If the self-supervised loss simply prevents overfitting to the data, there are many common training strategies in the field of machine learning  that can prevent overfitting. A more convincing proof of principle for the self-supervised loss is needed here.

(2) I am puzzled by the results of Table 1, whether such a large improvement can be achieved using only unlabeled point cloud data, and what is the reason for the improvement? How is the prediction of /phi(x) guaranteed to be correct if it does not depend on the labeled point cloud. The loss function of Eq. 7 can be equally reduced if it is predicted /phi(x) to all positive or negative signs in Eq. 8. Both SAL and NeuralPull give proofs that they can predict the correct sign, and this paper is unconvincing in this part.
limitations: I don't foresee any potential negative societal impacts.
review_title: null

Meta-review: metareview: This paper studies the generalization ability of neural signed distance functions by proposing a two-stage semi-supervised meta-learning framework. The method has been tested on both synthetic data and real point clouds. The paper received a total of 4 reviews.  After the rebuttal, Reviewers 84Fy (accept), 8obm (week accept), GFH2 (week accept) voted for accepting the paper because they reached an agreement that the paper proposes a novel, simple yet effective method to learn generalizable signed distance functions. Reviewer 39TN voted for a “Borderline reject” due to his/her concern about the lack of theoretical understanding of the proposed method, but in the rebuttal, the authors actively replied to Reviewer 39TN’s concern and provided additional intuition and experiments to illustrate the effectiveness of our approach. After an internal discussion, AC recommends accepting the paper because it presents a very useful tool for 3D representation and all the major concerns raised by the reviewers have been addressed during the rebuttal. AC urges the authors to improve their paper by taking into account all the suggestions from reviewers.


confidence: Certain
Decision: Accept