Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper introduces a self-supervised category-level 6D pose estimation to minimize ideally canonical reconstructed point cloud and input point cloud based on rotation-equivariant features. Experiment results can verify the effectiveness on both complete and partial point-based observations.
main_review: The paper is difficult to digest due to the writing due to a large number of typos. For example, on lines 65-68, features for 6D pose estimation can be equivariant rather than the problem itself. 

Technically, the design of canonical reconstruction and using residual pose predictions for progressive alignment to canonical space is not novel in the field. In the context of category-level 6D pose estimation, the proposed method can be considered to combine two branches of canonical reconstruction and pose estimation in a self-supervised manner. However, due to lack of canonical reference, ideally canonical reconstruction can be affected by intra-class shape variations and inaccurate pose predictions. The authors are suggested to explain its effects on pose estimation and do some ablation studies, which are missing in the paper.


limitations_and_societal_impact: The following items need to be addressed:
1.  clarify main contribution of the paper;
2. ablation studies on effects of quality of reconstructed shape on pose estimation
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a self-supervised approach to 6 DoF pose estimation. Usually such methods are either instance-level, assuming known CAD models for all object instances; or category-level, assuming a reference frame that can be generalized to all instances of a certain category. The second approach is more generalizable, however it requires a substantial amount of annotated training data, often from synthetic datasets. So far, self-supervision in this setting has been limited to instance-level, with CAD models but without pose annotations. This paper proposed category-level self-supervision and claims to be the first to do so, eliminating the need of both CAD models and large-scale annotated data. 
main_review: 
I find this paper quite novel, and as far as the authors claim (I am not an expert in the field) this is the first method to achieve self-supervision on category-level 3D object pose estimation. The reasoning behind how the authors achieve this is also well-explained and motivated throughout the paper (although I found it difficult to read in some points, see below). This also makes the paper significant, as it opens up a lot of potential follow-up work in self-supervised learning for this another task. 

Overall, the paper should be proof-read and improved in terms of grammar, presentation and clarity. Particularly the Method section, probably due to lack of space a lot is introduced in long sentences and paragraphs, it is hard to parse all the information. Having sub-section and proper equations with numbers and details would have been helpful. Same thing with experiments, there are a lot of long sentences and paragraphs with combined information that makes it hard to parse. 

Other comments: 

---

The contributions of this paper are three-fold:

- A general self-supervised framework for category-level 6D object pose estimation without CAD models or annotations.
- Showing that the concept of category-level reference frame can be recovered naturally during self-supervised training
- High accuracy in synthetic and real-world category-level datasets 

I think contributions 1 and 2 can be combined into one, as the authors are proposing 1 by means of 2. About 3, "achieving high accuracy" is vague, it would be better to report relative improvements and other real comparisons with other methods. Is this method outperforming other self-supervised methods, or competitive with supervised methods?

---

About experiments, I would like to have seen ablations with different amounts of training data. This is the biggest benefit of using self-supervised learning, so improvement with data is an important aspect to show as a way to potentially boost results even further. 

-- 

On table 1, why does the proposed method outperforms EPN only for chair, and why does KPConv outperforms EPN only for bottle, but by a very large margin?

--- Minor comments

line 360: Unfinished sentence.
line 117: "only one network" or "the only network"?  I didn't understand this sentence.  EPN achieves equivariance without increasing in complexity?


limitations_and_societal_impact: The authors mention certain limitations of their approach, such as degradation when objects exhibit symmetries, or viewpoint distribution is highly biased, or under heavy occlusions. There are no quantitative ablations or examples of such cases from the datasets considered in experiments, that would have been helpful to further illustrate these points. As for societal impact, the authors mention negative impact to people who work on object pose annotations.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes a framework to learn 6D object pose estimation without GT pose annotations and CAD models (while GT object masks are available, and each object is already isolated). The key is to jointly learn canonical shapes and poses by a SE(3)-equivariance network. It shows promising performance on both synthetic and real depth data.
main_review: Strongness:
- Clear writing
- Experiments on multiple datasets, including both synthetic and real ones
- Detailed analysis of failure cases and limitations

---

Weakness:
1. The authors miss some important related works. Although relying on multi-view supervision, there is a line of previous works focusing on learning-shape-from-x without GT labels, e.g. [1], [2], [3]. Especially, [3] has proposed a quite similar framework, like disentangling canonical shapes and poses, self-supervised losses, as well as selecting the best hypothesis during inference. The major differences are the type of supervision (multi-view) and the network design (PointNet + regression). It also presents many observations mentioned in this paper. For example, symmetric objects like cars are difficult for such self-supervised frameworks. 
2. Although there are no existing approaches to compare directly, the authors could provide more ablation studies to verify the design choices, which is kind of missing now. Although Sec 2.4 in Supp has presented some results, can the authors analyze the impact of equivariant networks on the shape and pose separately? For example, KPConv (or EPN) for shapes and EPN (or KPConv) for poses. Compared to [3], one key improvement of this paper is to introduce SE(3)-equivariant networks to generate consistent canonical shapes. Another question is how important it is to predict multiple hypotheses for 6D poses. Can the authors study the impact of the number of hypotheses (predefined groups)? And can the authors directly regress poses rather than using any hypothesis?
3. The second car in Figure 2 looks strange. The blue point cloud (canonical shape) in the 3rd column looks different from that in the 2nd column.
4. How stable is the training process? It seems that the network can easily get stuck in local minima (imperfect shapes will lead to wrong poses, and imperfect poses can also lead to imperfect shapes. This phenomenon might be most obvious for cars). Can the authors report the result of multiple trials of experiments to show the variance of the method?

---

Typos:
- L152: there are two "j-th", and I think one should be "j'-th".
- L203-L204:  $g_x=\Delta g_{x_0} \cdot g_x$

---
References
- [1] Unsupervised Learning of Shape and Pose with Differentiable Point Clouds
- [2] Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Prediction
- [3] Weakly-supervised 3d shape completion in the wild
limitations_and_societal_impact: The authors have adequately addressed the limitation.
review_title: null

Meta-review: 
This paper proposes a self-supervised category-level 6D pose estimation method in which canonical shapes for the input objects are predicted without explicit supervision of orientation. It introduces an  equivariant network architecture and shows its importance for canonical shape estimation.  Reviewers point out missing citations of previous works that consider similar setup (yet different neural architectures), specifically:
[1] Unsupervised Learning of Shape and Pose with Differentiable Point Clouds
[2] Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Prediction
[3] Weakly-supervised 3d shape completion in the wild
and the authors are encouraged to reference these and discuss in detail the difference of their method against the above. Reviewers agree regarding the good empirical performance of the method and the detailed ablations contained in the paper, and the paper is suggested for publication.

Decision: Accept (Poster)