Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a method to cast fine-grained visual parsing into analogical inference instead of previous methods that mapping input scenes to part labels. The main idea is to retrieve the new input point cloud feature embedding from memories of labelled feature embeddings. Then the several results with the highest similarity are retrieved are used as prior knowledge for subsequent tasks. Empirical results show that this approach achieves state-of-the-art results the PartNet dataset.
strength_and_weaknesses: - The main contributions are clearly stated.
- Appendices provide useful, supplementary information.

clarity,_quality,_novelty_and_reproducibility: - This work lacks a detailed description of the retrieval section. I'm not sure if the retrieval task is only applicable to datasets with very different objects like PartNet dataset.

- Could the authors provide more experiments on different datasets, if more different datasets are easy to access. Could you please show the retrieval ability of your approach as a table? Besides, I am not sure that the inner product is the most correct way. Could you compare it with several common distance formulas?


summary_of_the_review: I recommend this paper towards acceptance. Using retrieval from memory is a simple and effective method. And it can inspire future related work.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes Analogical Networks, a method for segmenting 3D parts of objects which excel in the few-shot setting. Specifically, the method retrieves examples from a memory bank closest to the query object, then assigns correspondences between parts of the query object and those of the examples. The approach outperforms existing 3D object segmentation and meta-learning methods for few-shot 3D part segmentation. Additionally, the paper shows that part correspondences for novel classes emerge from training without part specific labels.  
strength_and_weaknesses: Strengths:
1. Overall the proposed approach is interesting and, to my knowledge, novel. The problem of few-shot segmentation is decently motivated given that dense annotations are time consuming to obtain at scale. 

2. The ablations are well thought out and provide interesting empirical insights such as how parametric methods compare to a memory-based approach for 3D object parsing. 

3. The within-instance pretraining using point cloud augmentation is a clever solution to the non-differentiable aspect of the retrieval mechanism. 

4. The method outperforms 3D-DETR and Prototypical Networks for 3D part segmentation. 

Weaknesses:

1. The baselines compared against seem relatively weak. 3D-DETR is designed for detecting whole objects and Prototypical Networks are not designed for 3D object detection. It would be informative to compare against baselines meant for 3D part segmentation such as PartNet[1] and PriorNet[2] or explain why these methods are not suitable for comparison.

2. Qualitative results are only shown for the base classes. Given that the goal is few-shot transfer, it would be better to see qualitative results for the few-shot classes (washer, fridge, bed, table). 

3. Discussion of 3D object detection in the related works would be helpful to situate the work in context. The paper states that works thus far have used separate networks to parse each semantic category, but does not explain or show experimental results to demonstrate why this approach is worse. 

4. The motivation for part segmentation given in the abstract is dubious. The paper states, "Fine-grain visual parsing is necessary for ... action recognition". However, most state-of-the-art action recognition methods work directly from pixels and do not parse objects into fine-grain parts. 

[1] Yu, Fenggen, et al. "Partnet: A recursive part decomposition network for fine-grained and hierarchical shape segmentation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.

[2] Wang, Xiaogang, et al. "Learning fine-grained segmentation of 3d shapes without part labels." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.


clarity,_quality,_novelty_and_reproducibility: Clarity:
1. For the emergent part-correspondences experiment it appears that 3D-DETR is comparable in performance to Analogical Networks at 1-shot and 5-shot, but significantly lower at many-shot. The trend is the opposite for the part segmentation experiment. I’m wondering why this is the case, considering the claim that Analogical Networks are better in the few-shot data regime. 

2. It is unclear which experiments address which questions posed at the beginning of the experiments section. For example, the question “How much the performance of Analogical Networks varies with different retrieval mechanisms?” is only answered by looking through the appendix. 

3. In the section for within-instance pretraining, how the encoder is trained is not well explained. The text states the encoder is pretrained using the within-instance correspondences, but figure 2 (which is meant to illustrate the pretraining) only shows the modulator. As I understand it, the encoder is not directly optimized for the retrieval objective, and gradients are propagated through the modulator to the encoder from the part correspondence objective. However, this is not clear from the figure or text. 

Reproducibility:
Code snippets are provided in the appendix.

For novelty see strengths and weaknesses.

summary_of_the_review: My initial recommendation is marginally below the acceptance threshold. Overall casting the problem of parsing 3D objects as analogical correspondence is a novel and interesting idea. However, the paper does not compare with other 3D part segmentation methods or clearly place their approach in the context of part segmentation papers. Additionally, the insights in the experiments section are difficult to follow. The empirical questions posed at the beginning of the section are interesting, however it is not clear where each one is explicitly answered. With improved clarity in the experiments section and more complete comparison to existing work I would be happy to improve my score.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This work devised a novel framework called *Analogical Networks* for *few-shot* 3D object part parsing. The framework works as follows: (1) given an input point cloud (of novel object type), the top k closest labeled object instances are retrieved from memory set (2) these objects and their labels are used as context information to modulate the 3D parsing of the novel object. Experiments show this framework can work better than previous meta-learning approaches in *few-shot* learning settings. Many ablation studies are also provided for better understanding of the strength and the limitation of this proposed framework.
strength_and_weaknesses: Strength:
  + The proposed Analogical Networks look novel to me. It opens new directions to pursue and poses interesting questions to answer.
  + The performance gain in few-shot learning settings is significant.

Weaknesses:
 + Although the author has documented some limitations in their work already, it is worth reiterating here (1) the retriever is not end-to-end differentiable (2) the retriever is not scalable due to operating over the whole object memories.
clarity,_quality,_novelty_and_reproducibility: Strength:
  + Good presentation in general on the motivation, the proposed method, and the experiments.

Weaknesses:
 + The description of the pretraining phase and the regular training phase is lacking clarity and details. As mentioned before, it would be difficult for audience to reimplement this work just following the text.
 + It could be difficult for audience to reimplement this work, as it is comprised of many components and requires many phases of training. Therefore to really facilitate the adoption of this framework, I strongly recommend the author to make their code for training and evaluation both available to the public, as promised in the paper as well.
summary_of_the_review: I agree with the author on their motivation and the proposed approach. This framework looks novel to me and presents many interesting future directions. Therefore I would recommend for accepting this paper, and hope the authors release their code as promised to the facilitate the adoption and the adaption of this framework.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper reformulates few shot learning about 3D parsing from  analogy-driven prediction view. The authors design an  Analogical Networks that are comprised of an encoder, retriever and modulator sub-modules. Experiment are performed on benchmark for 3D object segmentation.
strength_and_weaknesses: Strength:
1. All figures and tables are in good illustration
2. The idea is simple and easy to be implemented.

Weakness:
1. The abstract should be revised. The main content is not close to the title, such as 3D parsing is missing in the abstract.
2. Overclaim issues. In the introduction section, the authors claims that  This allows us to train one model across all tasks. However, only one task is evaluated meanwhile this 3d parsing is not a representative few shot learning task.
3. Besides, the idea of Memory-augmented neural networks for few shot learning has been studied in the previous works[1]. From this point, the novelty of the whole method is limited.
[1] Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., Lillicrap, T.: Meta-learning with memory-augmented neural networks. In: ICML (2016)
4. In experimental section, the setting of many shots are nonsense. This paper focuses on few-shot learning.
5. The experimental results are not sufficient. The compared methods are missing.

clarity,_quality,_novelty_and_reproducibility: The novelty of the whole paper is limited. The clarity is good and quality it not satisfying.  Reproducibility seems good.
summary_of_the_review: This paper presents a new view for few shot learning. However, the idea is incremental from the memory based few shot learning solution. The experimental results and analysis are not sufficient. Overall, the quality of the whole paper can not reach the level of the top AI conference.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper presents a method to segment 3D object parts in a few shot manner. It uses retrieval and correspondence assignment to label object parts given just a few examples. The paper presents results that outperform past works for few shot 3D segmentation. Four reviewers provided reviews for this paper. They found the approach to be novel and interesting, the ablations to be thorough and the gains of the proposed method to be significant over prior work. Some concerns included the need for stronger and recent baselines as comparisons, a need for more qualitative results and a rewrite of some sections of the paper such as the abstract. The authors have done a commendable job in their rebuttal and have addressed most concerns thoroughly including providing new quantitative comparisons. For the most part, reviewers were happy with the rebuttal and edits to the paper and two reviewers increased their scores as a result. The paper now has three accept scores (with two scores of 8) and one borderline score. Given the reviews, revisions and feedback from the reviewers, I am recommending an accept.
justification_for_why_not_higher_score: I am recommending an accept given the reviews, rebuttal and comments by the reviewers. Although the work has clearly been well received by the reviewers, they are not overly excited with the contributions. The presented method has limitations such as the retriever not being differentiable and not scalable. I think the reviewers would have been a lot more excited if the work even partially addressed at least some of these limitations.

I am not an expert in this area and to some extent am leaning on the reviewers expertise, and as a result, I am recommending a poster accept.
Decision: Accept: poster