Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper presents a method for unsupervised generative part decomposition based on part kinematics. By employing a common encoder, operating on the input point cloud, the method learns a non-primitive-based implicit field for each part using a set of category and instance specific shape decoders, and the joint parameters of each part using a set of category and instance specific pose decoders. 
main_review: # Strengths:
The proposed method attacks the very challenging problem of capturing the shape of an articulated object, together with its variations within the object category, its decomposition into parts in a parsimonious way, and the estimation of each part's pose in terms of a joint state and, in the case of revolute joints, a pivot point. 

Although the overall architecture is somewhat complicated with numerous loss terms involved, the proposed method achieves improved performance with respect to other unsupervised generative part decomposition methods on standard metrics considered for the task. In terms of part pose estimation, it achieves comparable performance to the NPCS which assumes part segmentation supervision, even though the proposed method is unsupervised. Additionally, the part parsing produced is more parsimonious and intuitive qualitatively, in terms of interpretability. 

The experimental evaluation, although compact, is quite comprehensive covering many aspects of the method. It considers also a different input modality (depth maps) as well as the application of the method on real data. Also, the authors have made significant effort to assure that comparison with other baselines is fair. 

Regarding reproducibility, all the implementation details are provided, making it possible to reproduce the results. It would also be important though to make the source code available.

# Weaknesses:
There are few weaknesses I can identify. One is that the notation is quite heavy, involving numerous symbols and many superscipts/subscripts. Admittedly, given the overall complexity of the task and the method proposed, it might be difficult to make the notation lighter. One thing that might help, would be to include full words in the subscripts of the loss terms, as their number is high and it is not easy to remember what each of them tries to achieve. Nevertheless, given the compactness of the Methods section, the paper manages to provide a quite clear explanation of the method.

A minor issue, in my view, is that the fact that a discriminator is used is stated quite late in the paper (Section 3.2, pg. 6). I would prefer to know earlier (possible from the introduction), that the method is based on an adversarial learning setting.

Another issue of the overall presentation regards the hyperparameters. Some of them, like e_i and y_i, provide crucial information to make learning possible and seem to depend to some extent on the structure of the particular objects, in general, and the way they are represented in the dataset, specifically. I would prefer if these hyperparameters were presented and discussed in a more straightforward way, rather than been less evident within the text.

## Minor Comments
- Introduction: "have mainly focus"
- Figure 1: Left and Right labels should be swapped
- Notation: \mathcal{X} is not properly defined, this makes difficult to interpret the domain of \hat{O}_i (R^3\times \mathcal{X})
- The phrase: "Instead of using a single constant vector, ... biases" (before eq.4 on pg.5) is not very clear
- Tables: please indicate which metrics are higher-better and which are lower-better
- Appendix B: Wrong citation for NPCS 
summary_of_the_review: Overall, I think that the paper is quite strong and the method proposed contributes some quite interesting ideas for effectively addressing the challenging problem of unsupervised pose-aware generative part decomposition. The experimental evaluation is comprehensive and follows correct practices.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The proposed paper introduces a pose-aware part-based method for articulated objects. The proposed model takes as input a pointcloud and predicts a set of part poses as implicit functions. The network architecture comprises an encoder that maps the input pointcloud to three latent vectors, a category-common decoder and an instance pose decoder that take the two latent vectors and regress the part pose parameters and a set of shape decoders that are used for generating the part-wise implicit fields. To the best of my knowledge, this architecture is novel and the authors demonstrate that their model yields better segmentation results than various baselines on the Motion dataset. Moreover, the authors evaluate their model wrt the reconstruction accuracy of the final prediction. From the provided analysis it seems to perform worse than existing baselines. Finally, while the authors claim that their model can be used on various generative tasks, they only provide some interpolation results. To summarize, I think this paper is an important step towards pose-aware primitive-based representations, however some parts of the text require further clarifications.
main_review: ### Strengths:
------------
1. I really like the direction of having generative part-based models. I think this topic has not been adequately explored and while this paper does not provide enough experimental evidence on generative tasks, I consider it an important step towards developing generative models for part-based methods.
2. The paper seems technically sound and the proposed network architecture is novel.
3. I appreciate the authors's efforts in providing extensive experiments both in the main paper and in the appendix.

### Weaknesses:
-------------

1. My major concern for the paper is related to its clarity. While I overall like the proposed model, I found it quite hard to follow the paper. First of all, the goal of the proposed method is not crystal clear. In some parts of the text it is mentioned that the main focus of this work is developing a generative part-based model (e.g. 3rd paragraph in the Related Work Section), whereas elsewhere it is stated that the focus of this work is on learning shape abstractions (e.g. first sentence of Method Section). I do not think that the authors sufficiently demonstrate that the proposed method can be used for generative tasks, since they only show some interpolation results in Figure 5. However, this doesn't diminish the contributions of this work. Therefore, for future version of the paper, I would recommend that the authors try to ease their claims regarding their model being able to operate on generative tasks. Secondly, I think that the Method Section should be restructured in order to improve the paper's readability. Please see my comments below for more details. Finally, closely related to my first point, also the experiment section does not properly reflect the contributions of this work. Namely, just by looking Table 4, a reader would probably think that the proposed method is not the best choice if we are interested in accurate reconstructions, since it is consistently worse than all baselines wrt Chamfer loss for all object categories and also performs worse wrt F-score in the 4 out of 5 object categories.
2. My second concern is related to the dataset that the authors decided to use. I think the current dataset is quite simple. Since the authors are interested in articulated objects, it might be useful to also consider the D-FAUST dataset that contains humans performing various actions. I believe that the claims of the paper would be significantly stronger if the authors were showcasing that their model can also operate in more challenging setups.
3. While I appreciate the author's efforts in providing extensive experiments both in the main paper and in the supplementary. For the final version, I would like to see additional qualitative results on more object instances for the various categories. From the current experiments, it looks to me that the same objects instances are repeatedly used. Showing additional examples, would only make the claims of the paper stronger.

### Questions / Detailed Comments:
-------------------------------

1. For the experiment in Table 2, what is the number of ground-truth parts for each object category (on average)? Are these numbers reported in Table 7 in the Appendix? I think it would be better to fix the number of parts for all baselines (similar to Fig 3.) so that the results are comparable because with the current comparison it is unclear whether the superiority of the proposed model stems from the fact that it is using fewer components, which result in more coarse abstractions.
2. For the experiment in Table 4, can the authors clarify how is the F-score computed? In addition, how many points are used for computing the Chamfer-L1 loss? Is there any reason why the authors are not reporting the mean-IoU which is also commonly used for evaluating the reconstruction accuracy of primitive-based representations? I believe that including an additional comparison wrt the IoU would make this experiment more complete. Finally, it might also be useful to compare the proposed method with the more recent work of [2] since the authors demonstrate that they can capture complex geometries with few components.
3. For the experiment in Table 4, can the authors please clarify how the authors compute the number of learnable parameters for each method? Instead, I think it would be better, to report the number of primitives used by each method, as in Table 2.
4. Can the authors describe their intuition for the second term of Eq. 2. Is the second term really necessary? Unless I am misunderstanding something, I would expect that only using the first term of Eq. 2, which is the standard reconstruction loss for the majority of part-based models that rely on implicit surfaces, should yield good results? Can the authors elaborate on this and maybe also add a small quantitative comparison with and without the second term of Eq 2?
5. I am wondering whether the use of the VQ-VAE is really necessary. While I understand the authors's argumentation regarding its use allowing to capture discrete, multi-modal category common biases, I believe it would be beneficial for the paper to also ablate its use. Therefore, I recommend that the authors include this experiment for the final version of their paper.
6. In page 4, in Section 3.1, the authors state that their model first tries to reconstruct the target shape with a single part and as the training progresses it focuses on the remaining parts. This is also nicely shown in Figure 11 in the Appendix. Can the authors provide some intuition regarding why this is the case? For the majority of part-based methods, the models tend to distribute all parts on the target shape and then as the training progresses, each part focuses on a specific region of the target object. Why is this different for the proposed method? Moreover, if the model manages to capture the geometry of the target object using a single part, what is the incentive for utilizing multiple components in later training stages? I personally, find it quite hard to explain why this is the case.
7. Since the proposed method performs worse than all baselines wrt Chamfer loss and F-score (see Table 4), I believe the authors should try to describe why this is the case. Intuitively, I would have expected the proposed method to perform comparable to BSP-Net, however BSP-Net seems to be significantly better. My intuition is that for BSP-Net the authors are using significantly more primitives, thus the discrepancy. However, it would be nice to also elaborate more on this.

### Suggestions:
--------------

1. In page 3, in Section 3, in the second paragraph, where the authors briefly describe the components of their architecture, I think it would be helpful to add a sentence briefly summarizing the purpose of each component intuitively and at a higher level. Currently, from reading this paragraph it is hard to say what is the purpose of the having a category-common shape decoder and an instance-dependent shape decoder. Simply adding a sentence elaborating on the purpose of each component would significantly improve the paper's clarity. Similarly, I would really appreciate it if the caption of Figure 2, was more detailed. Specifically, it would be nice to provide the higher level concept of each component in order to allow the reader to better comprehend how the model works.
2. I think it would be great to move Figure 12 from the appendix to Section 3.2 in the main paper. The notation in this chapter is quite heavy and adding a figure that shows each variable would definitely be useful for the reader.
3. In the caption of Table 2, it would be better to mention what is metric that is reported in the corresponding table. In addition, in the table caption, instead of mentioning generated parts, I think it is more accurate to mention recovered parts.
4. In Section 4.1, the authors state "We can also see that our part shapes are more semantic and interpretable than the previous works.". I am not sure whether this is 100% true, since the final decomposition in particular for the case of NSD seems to be quite comparable with the proposed method.
5. I think that the Method Section can be restructured. I recommend to first describe the part pose parametrization (Section 3.2), the introduce the part shape representation through the implicit functions (Section 3.1) and finally add a separate section that presents and describes the loss terms of Eq. 2,3,4,5,6,7,8,9.
6. Throughout the text, the authors say that they have "single-frame shape supervision". Maybe instead say that the supervision comes in the form of a watertight mesh parametrized as a set of occupancy pairs indicating whether a point lies inside or outside the target object.

[1] Dynamic {FAUST}: Registering Human Bodies in Motion, Bogo et al. CVPR 2017

[2] Neural parts: Learning expressive 3d shape abstractions with invertible neural networks., Paschalidou et. al. CVPR 2021

### Minor Comments / Typos:
-------------------------
- In page 2, "To address the problems associated with (1) kinematically learning ... can effectively addresses these problems.": the authors should state how the above concerns are addressed with the current model.
- In page 3, "In contrast, our approach focuses on the man-made articulated object of the general category with various kinematic structures.": instead I would rephrase to "In contrast, our approach focuses on man-made articulated objects of a category with various kinematic structures."
- In page 3, "PPD employs an autoencoder architecture, and trained ...": instead it should be "is trained"
- In page 3, "Given a single-frame point cloud I ...": maybe remove "single-frame", I personally find it a bit confusing
- In page 5, "Through the observation, we assume ...": It is not clear to which observation the authors refer to.
summary_of_the_review: While I like the paper, some aspects are unclear to me and I expect the authors to clarify them during rebuttal. Regarding the paper's clarity, I am confident that the paper can be significantly improved if the authors spend some time restructuring and rewriting parts of the text. Finally, I believe that showing additional qualitative results and ideally also providing results on a more challenging dataset would make the claims of the paper significantly stronger. Therefore, I think that the paper in its current state is below the threshold of acceptance. However, if the authors address my concerns I am happy to increase my initial score.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This submission proposes an unsupervised method to segment point clouds of objects that are articulated. In contrast to previous work with similar tasks, the learning is done in an unsupervised manner, outputting a segmented point cloud represented as a combination of implicit fields that can be rigidly articulated. The paper is well presented, technically sound, and correctly evaluated and compared to works with similar goals.
main_review: STRENGTHS
- Well-written paper, detailed introduction, and motivation of the work.  
- Good description of the method, with sufficient details to allow re-implementation and deep understanding of the implementation.
- Good evaluation, where the different tasks enabled by the method are evaluated: semantic segmentation, par pos estimation, and reconstruction. Also ablation studies are provided.
- First method to do unsupervised decomposition of articulated objects.

WEAKNESSES
- I think comparing to BAE and BSP can be a little bit misleading since these methods do not assume articulated objects. Working under the 'articulation' assumption allows the authors to frame their loss, and it is understandable that testing on articulated objects works better. A more fair (or additional) comparison would be to use this method on non-articulated objects and compare it to BAE and BSP.
- The symbols used for encoders and decoders are too cryptic and hard to parse while reading the paper (e.g. $F^{p,z}$, $F^{p,c}$, $F^{s,z}$, etc.). I'd encourage the authors to rename those to more meaningful names to ease reading.
summary_of_the_review: I think this is a nice written paper with a good technical contribution. The paper is well written and the results look convincing. Due to the specific scope of the paper (i.e., articulated objects) and the lack of works with this specific goal, comparisons to SOTA methods are somewhat difficult to parse. Nonetheless, seems to propose a step forward in the scenario that is designed for.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper addresses the problem of unsupervised parts decomposition for articulated objects with kinematic structures. In particular, the proposed approach disentangles the shape and pose of each part separately. Instead of adopting the primitive-based shape representation, each part is represented as an implicit field and disentangled part poses of three kinematic types. A category-specific shape/pose prior and instance-dependent shape/pose details are learned separately. The proposed approach is compared with existing works on benchmark datasets.
main_review: Pros:

+ It is an unsupervised approach for parts decomposition which does not require parts labelling. 
+ It can handle objects with varying number of parts.
+ It models the part poses of different kinematic types which can better model the articulated objects for unsupervised part decomposition.

Cons:
1)	It is not clear how to avoid the cases where multiple implicit fields are used to model the same part. Will the spread of the pivot points discourage the overlapping of the parts?
2)	It would be good to provide details about how to obtain the vector c^p.
3)	Please define the semantic meaning. It is not clear how each part is guaranteed to have semantic meaning.
4)	Is the parameterisation of the part pose correlated to the part semantics? In the implementation section, the number of different joint types are predefined. Will this implicitly indicate the parts structure? What is the influence of this predefined numbers on the segmentation results?
5)	What are the overlappings between different parts?
6)	It is not clear why not compare with Neural parts (Despoina et al, CVPR 2021) which is the most recent unsupervised parts segmentation method.
7)	The parts visualisation in Figure 4 is impressive. However, it is not clear how the drawers are segmented including the interior parts that are not visible from the surface defined by the point cloud.
8)	Please highlight the best performance in the Table 4. Based on the listed numbers, the proposed method does not achieve the best performance in terms F-score and Chamfer distance.
9)	It is not clear what the redundant parts learn if the parts number of the objects is smaller than the predefined maximum number.
10)	It would be interesting to visualise the category-specific shapes.
11)	Minors: The writing of the paper should be further polished. …poses can effectively addresses -> address; 
Please explain semantic capability when it firstly appears in the text (above related works)
Please be consistent about the definition of {F_i^{s,c}}_{i=1}^N and {F_i^{s, z}} within the method section (see confusing definitions in the paragraphs above and below Sec. 3.1) Is {F_i^{s,c}}_{i=1}^N the category-common decoder or instance-dependent shape decoder?

summary_of_the_review: Overall, the proposed unsupervised parts segmentation approach from disentangled shape and pose is novel. However, the paper is not well-written. Details are not clear to the reviewer. The results are not quite promising as demonstrated in Table 4.
review_title: null

Meta-review: The submission received split reviews: two reviewers recommended accepts, and the other two rejects. The AC went through the reviews, responses, and discussions carefully. The AC appreciates the authors' effort during the response period and agreed that the revision has addressed some of the concerns of the reviewers. However, a few key issues are not fully addressed. This includes results on additional, more complex object categories; discussion on why the performance of the proposed method is not even as comparable as BSP-Net (Table 4); and others.  Further, while the authors have significantly refactored the paper to address the concern on presentation and clarity, the changes are too major for the reviewers to review during the response period (the reviewers are expected to check minor updates, but not review a new paper during the response period).  

Considering all pros and cons, the AC recommends rejection. The authors are encouraged to revise the paper for the next venue.
Decision: Reject