Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: This paper studies the problem of part segmentation in objects represented as a point cloud. The main novelty is in the fact that the proposed method uses a bottom-up iterative merging framework inspired by perceptual grouping and finds that it transfers better to unseen categories. In zero-shot transfer experiments, the proposed method performs better than all four other baselines compared; but is worse than Mo et al. (2019) in known categories.

The paper hypothesizes that top-down approaches do not generalizes well to new categories because they end up overfitting to the global context. While this is reasonable, I find that the experiments are not sufficient to validate this claim (please see questions below). Evaluation on unseen object categories is an underexplored topic, and the paper is generally well written. I think the submission can be an above-threshold paper if the questions are addressed.

- I’d like to see some evidence for the claim that classic segmentation methods "can perform much better for unseen object classes" (last paragraph of page 1), and see how the proposed method compares to those baselines.

- If my understanding of Table 3 is correct, "PartNet-InsSeg" (Mo et al. 2019) is a top-down approach yet it performs better than SGPN which is a bottom-up grouping method (as summarized on page 7) in novel categories. If so, can it be explained in a way that is consistent with the paper's findings?

- Table 4 shows some ablation study in an attempt to justify the proposed design, but I think it should be more thorough. e.g. it is not immediately obvious why the authors did not included a baseline that consists only of the rectification module with a termination threshold (seems like the most basic design that doesn't have the large-part bias or explicitly require a termination module).



Typos:

psilon-greedy   (page 6 paragraph 2)
backpropogation  (page 6 under training losses)
In consequences (page 5 under termination network)
epilson  (page 5, under network training)
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper proposes a method for part segmentation in object pointclouds. The method is to (1) break the object into superpixel-like subparts (without semantic meaning yet), then (2) score pairs of parts on their mergeability, (3) greedily merge the best pair, and repeat. The scoring has a unary component (called a "purity" module), and a pairwise component (called a "rectification" module); the unary component determines if the joined pointcloud of two sub-parts appears part-like, and the pairwise component determines if the features of the two sub-parts appear compatible. These components are implemented as pointnets/MLPs. Finally there is a termination module, which sigmoid-scores part pairs on whether they should actually merge (and the algorithm continue), or not (and we stop). The purity and termination modules are trained supervised, to mimic intersection-like and mergeability scores, and the rectification module with a "reward" which is another mergeability score (coming from GT and the purity module).

The method is interesting for being (1) iterative, and (2) driven by purely local cues. The iterative approach, with small networks doing the work, is a nice relief from the giant-network baselines (such as PartNet-InsSeg) that take the entire pointcloud as input and produce all instance segmentations directly. Also, whereas most works try to maximize the amount of contextual input to the learning modules, this work makes the (almost certainly correct) observation that the smaller the contextual input, the smaller the risk for overfitting. This is a bit like making the approach "convolutional", in the sense that the same few parameters are used repeatedly over space (and in this case, also repeated over scale). The design of the local modules makes sense, although I would prefer they be called unary/pairwise instead of purity/rectification, and the RL training procedure looks reasonable also.

I am not totally clear on how the termination module actually comes into play. From the name, it sounds like this network would output 1 when the algorithm should terminate, but in its usage, it seems to output 1 when the best-scored pair should be merged. So then, does the algorithm terminate when this module decides to NOT merge the best-scored pair? This sounds like it bears great risk of early stopping. I would appreciate some clarification on this.

The abstract says that locality "guarantees the generalizability to novel categories". This is an overstatement, since "guarantees" implies some theoretical proof, and also since the paper's own results (in Table 1 and 3) indicate that cross-category generalization is far from addressed, and depends partly on the categories used in training (shown in Table 2). 

I assume that this method has (or at least can have) far fewer parameters than the baselines, since the components never need to learn broad contextual priors. Can the authors clarify and elaborate on this please? If you can show that your method has far fewer parameters than the baselines, it would improve the paper I think.

Can the authors please provide some statistics on the earliest stage of the method, where superpixel-like parts are proposed? How many proposals, and how many pairs does this make, and how slowly do the main modules proceed through these pairs? 

Is there a missing step that makes the part selection non-random? It seems like many of the pairs can be rejected outright early on, such as ones whose centroids exceed some distance threshold in 3D.
review_title: Official Blind Review #1

final_score: rating: 8: Accept
review_content: This paper describes a method for segmenting 3D point clouds of objects into component parts, with a focus on generalizing part groupings to novel object categories unseen during training.  In order to improve generalization, the paper argues for limiting the influence of global context, and therefore seeks to build compact parts in a bottom-up fashion by iterative merging of superpixel-like point subsets.  This is achieved by defining a RL merge policy, using merge and termination scores formed by a combination of explicitly trained part purity (each part should comprise one true part), and policy-trained pair comparison network.  The system is evaluated using PartNet, using three categories for training and the rest for testing, showing strong performance relative to baselines.

The system is described well, and shows good performance on a nicely motivated task.  A few more ablations would have been nice to see (in questions below), as might more qualitative results.  Overall, the method is presented and evaluated convincingly.


Questions:

*  What is the effect of the purity score regression?  Since the policy network is trained using a pair-comparison module anyway, what happens if the explicit purity score supervision is removed?

* What if the "rectifier" module is made larger (with or without purity module), e.g. the same size as the termination network?  Does this improve or overfit to the training categories?

* Sec 5.3 mentions "segmentation levels for different categories may not share consistent part granularity ....  Thus, ... we train three networks corresponding to three levels of segmentation for training categories".  While it makes sense to have three networks for the three levels (each have different termination points, and perhaps even merge paths), I don't see how this follows from the levels being inconsistent between categories.  In fact, it seems just the opposite, that if the levels are inconsistent, this could pose a problem when a part at one level for one category is "missing" from the other category, due to level numbers not coinciding.  Or, is this actually not a problem because on the three training categories selected, the levels are in fact consistent?

* Can termination be integrated into the policy network or policy itself?
 

A couple typos I noticed:

p.5 "In consequences," --> "As a consequence,"
p.11 "in-balanced" --> "unbalanced"

review_title: Official Blind Review #3

Meta-review: This paper presents and evaluates a technique for unsupervised object part discovery in 3d -- i.e. grouping points of a point cloud into coherent parts for an object that has not been seen before. The paper received 3 reviews from experts working in this area. R1 recommended Weak Accept, and identified some specific technical questions for the authors to address in the response (which the authors provided and R1 seemed satisfied). R2 recommends Weak Reject, and indicates an overall positive view of the paper but felt the experimental results were somewhat weak and posed several specific questions to the reviewers. The authors' response convincingly addressed these questions. R3 recommends Accept, but suggests some additional qualitative examples and ablation studies. The author response again addresses these. Overall, the reviews indicate that this is a good paper with some specific questions and concerns that can be addressed; the AC thus recommends a (Weak) Accept based on the reviews and author responses.
Decision: Accept (Poster)