Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new 3D shape representation learning method using multi-scale wavelet decomposition. In particular, the authors introduce a neural network architecture that decomposed 3D shapes into sub-bands components at multiple scales. In particular, starting from a pointcloud the proposed model learns to decompose it into coarse (high frequency) and detail (low frequency) components using an adaptive lifting scheme, similar to the original lifting scheme introduced for defining second-generation wavelets. Subsequently, two transformer models are used to refine the coarse and approximate geometry of the 3D shape. The proposed model achieves state-of-the-art results on the shape classification task on the ModelNet40 and the ScanObjectNN dataset and on the part segmentation task on the ShapeNet Part dataset. The concept of using such an adaptive lifting scheme seems to facilitate learning and to the best of my knowledge is novel for the case of shape representation learning.
main_review: ### Strengths:
------------

1. The idea of decomposing 3D shapes into sub-bands components is really nice and to the best of my knowledge such representation has not been explored in prior work.
2. The paper is clear, nicely written and easy to follow.
3. The proposed model outperforms prior work on the object classification task on ModelNet40 dataset and on ScanObjectNN dataset and on the object part segmentation task on the ShapeNet Part dataset.

### Weaknesses:
-------------

1. While I acknowledge that the paper outperforms prior works on both the classification and the object part segmentation task, the proposed model is marginally better than previous methods. For example for the object classification task, the proposed model is only 0.1% better than GDANet on the ModelNet40 dataset, while it performs on par with it on the ScanObjectNN dataset. Therefore, I am wondering how statistically significant are these experiments?
2. GDA-Net by Xu et al. is performing comparably well  with the proposed method. However it is not thoroughly discussed in the Related Work section. Therefore I recommend that the authors describe the differences between their model and GDA-Net in detail so that the reader can easily understand what are the differences between the two models.
3. Various implementation details are missing. For example what are the weights $\lambda_1$ and $\lambda_2$ in Eq (2)? What optimizer did the authors use? For how many epochs was their model trained? What was the learning rate? Unfortunately, I wasn't able to find these details in the main paper. However, the authors should provide them in order for the results of the paper to be reproducible.

### Questions / Detailed Comments:
---------------------------------

1. From Table 1, it seems that the proposed method using the vanilla Transformer performs slightly worse than the variant that uses the Performer. Can the authors provide some intuition for this? I would have expected that using Perform would result in faster inference, however it is not clear to me why it also results in improved performance.
2. What is the number of scales used for the classification accuracy experiment. In Table 1, the proposed model achieves 93.9 accuracy, however from the ablation study in Table 7, the best performing model achieves 93.4 with 2 scales. So how under which configuration was the performance of Table 1 achieved?
3. In prior work, authors typically consider the shape classification task on the S3DIS dataset that contain indoor scenes. I think that the claims of the paper would be even stronger if the authors added an additional evaluation on the more challenging S3DIS dataset.
4. For the Object Segmentation experiment, in Table 3, the authors should also include the results from Point Transformer by Zhao et al. that achieves 83.7 class mean IoU and 86.6 instance mean IoU, as stated in the original paper.
5. For the Even-Odd Split operation, it is not clear to me, why it is necessary to have both a weighted and an unweighted adjacency matrix. Can the authors please comment on this?
6. Some additional references are missing in the Transformers in Vision Section:
- In addition to PCT that applies transformers for 3D pointcloud classification and segmentation, the authors should also mentions the more recent works of:
    * Point Transformer, Zhao et al. ICCV 2021
    * PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers, Yu et al. ICCV 2021
- In addition to the mentioned works on efficient transformers, the authors should also mention the work of "Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention" by Katharopoulos et al. ICML 2019, since it was among the first to address the quadratic complexity of transformers

### Minor Comments / Typos:
-------------------------

- In Page 2, in Section Introduction "state-of-the-arts": should be "state-of-the-art"


summary_of_the_review: Overall, I really like this paper. It was easy to ready and nicely written. Moreover, the proposed model outperforms previous methods on the scene classification task on multiple datasets and on the part segmentation task. While the proposed model is marginally better than previous models, I really like the concept of utilizing wavelets for 3D shape modelling thus I vote for accepting this paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a novel 3D point cloud representation learning framework. At the core of this method, is a lifting scheme inspired by wavelet decomposition. The proposed method roughly splits the input data in half at each stage, producing a down-sampled approximation C and detail d. Then C is further processed by the next layer, forming a multiscale pyramid. In summary, the contribution of this paper is:

1. Proposed to use the lifting scheme in point cloud processing, using graph convolution networks and transformers as backbone.

2. Evaluated the method against state-of-the-art baselines and showed that the proposed scheme performs well.

main_review: Strengths:

- The authors propose a novel framework for point cloud processing, which is inspired by the lifting scheme in wavelet transformation. This new approach, equipped with performer and GCN, makes it state-of-art in point cloud classification and segmentation. 

- Extensive experiments on the performance of the model, especially w.r.t number of scales, point sparsity and rotations. 

- This paper is well written, where readers can easily understand the design of the proposed method, and is clear enough for reproducing the method without any code references. 

Weakness:

- Major concern

   1. A lack of consistent motivation of using a lifting scheme.
    Though using the lifting scheme is novel, it is less motivated during the narrative of the paper. The only direct motivation is more or less the Fig.1, where the detail features sort of concentrates on 'junctions'. But it's still hard to interpret, and the number of qualitative results are too limited. I like the lifting scheme story, but I don't think it's being motivated enough through the paper. 
    
    What's the unique edge of using this lifting scheme, apart from the few point gains if you use a voting scheme + performer? Does it learn meaningfully samplings of the original point could? does it highlights consistent keypoints within a semantic class?
  
   2. Are the comparisons fair?
       I'm not really comfortable comparing the proposed method using the performer. Though it gives a few point in performance, I'm not sure if that's fair for evaluating the lifting scheme against other baselines. This relates closely with concern 1, since I believe the merit of having a lifting scheme should not just be the bold numbers, and it's okay if it's not: it just to be on par with state-of-the-art. 

   3. Why more scales hurt?
       I don't really understand why more scale hurts. Why does it introduce `redundant information`? I think it's introducing more and more redundant representation, but not necessarily redundant information. Is it because the more scales you have, the more over-parametrized you are and therefore more prune to overfitting? 




- Minor concern

1. Is the name of wavelet transform adequate?
    This may or may not be a huge issue, but it's less intuitive for me to see this as a wavelet transformer, but more or less a lifting transformer. The wavelet in the title sort of suggests there's wavelet filters involved, but it's more like putting neural networks(transformers)  in a lifting scheme. 



summary_of_the_review: In summary, this paper proposes a method for processing point cloud using a lifting scheme. The proposed method is novel, and with some bells and whistles, it achieves the state-of-art performance on point cloud segmentation and classification. However, there's some major issues blocking me from rating this paper above the acceptance bar. First of all, it is not super clear to the reader why one would like to incorporate the lifting scheme for point cloud processing. I don't think it is just for a few points gain in the performance. Secondly, there's some issues in the evaluation due to the choice of the performer vs plain transformer.

I'm willing to raise my score if the authors could address those concerns. 

------------------------

After reading the response from the authors, I would like to raise my score from 5 to 6. The authors have addressed my concerns, and I think this paper meets the bar of ICLR.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents a novel framework for 3D shape representation learning, which is based on multi-scale wavelet decomposition. This is very different from existing works. A novel transformer-based neural network, AWT-Net, is also proposed.
main_review: The strengths: 
1) The paper is well-organized, the experiments are sufficient demonstrating the effectiveness of the proposed framework design. 
2) Relying on wavelet decomposition for 3D shape learning is theoretically sound and interesting. 
3) The authors made many efforts on the framework design, like the proposed new adaptive lifting scheme, the transformer etc. 

The weaknesses:
My major concern is about the performance. It seems the performance is just comparable with existing methods. This brings a doubt to me: is it really necessary to use wavenet, a relatively new framework? 
summary_of_the_review: Although the proposed method is theoretically sound and the framework design makes sense, the performance gain is hard to convince me the superiority of such a different learning framework.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper presents a new deep neural network architecture for 3D point cloud representation learning, based on wavelet decomposition. In particular, the authors propose a data-driven adaptive lifting scheme that introduces non-linearity into wavelet. The original linear operators update(U) and predict(P) in wavelet decomposition are replaced by non-linear graph convolutional networks (GCN). Equipped with wavelet transform and Transformers, the proposed network aims to captures and refines the holistic and complementary geometry of 3D shapes to supplement neighboring local information. Experimental results on standard benchmarks (i.e., shape classification and part segmentation) show that it achieves state-of- the-arts or competitive performance.
main_review: Pros:
1. The paper is well written and provides sufficient background knowledge.
2. Applying wavelet transform for 3D representation learning seems quite new and is not much explored before.
3. For the experiment results, the numbers themselves are stronger and ablation study is well conducted. 

Cons & questions:
1. My biggest concern is about the results comparison, even though the final number itself is strong. Table 1 shows the proposed AWT-Net achieves the best results when using Performer (AWT-Net +P). It a little bit unfair since others transformer-based prior works (e.g. PointTransformer, PCT) use vanilla Transformer. Their results might also be improved with Performer. If we look at only results for 'AWT-Net +V' in Table 1 and Table 3, the numbers fall behind prior transformer-based works like PointTransformer and PCT. Is using Performer a must? In addition, why doesn't Table 2 also show the result for 'AWT-Net + V'? I think it's better to put it on if there are no particular reasons.
2. The module analysis of ablation study is sort of unclear to me. First, what exactly is the baseline (the first row in table 4)? It's unclear to me what's the network architecture after ablating all the modules. I feel it's better to have a supplementary document describing the details. Second, the improvement (from 92.3 to 93.4) by adding ‘approximation’ and ‘detail’ doesn't seem that much. It is because this task is already saturated? Or gives a convincing evidence that it is a considerable improvement.  
Is there any other way to demonstrate the effeteness of using wavelet decomposition? The visualization in Figure 1 is a good example to understand the learned features but could potentially be explored more (e.g. how is hierarchy/multi-resolution reflected here).
3. I'm also curious about the execution time. The proposed method is quite complex (with split scheme, GCN and Transformer), so I wonder if it will be much slower than previous works. A table showing the execution time and model parameters would be much helpful.
4. Will the randomness in even-odd node split (Algorithm 1) affect the results? Just want to confirm different node partition for the same 3D shape does not much affect the learned feature.

Minor point:
- The resolution of Figure 5 right is too low. Consider using a pdf version.
summary_of_the_review: In general, the proposed method is novel in that it combines wavelet transform into 3D representation learning (I'm not an expert in wavelet transform, so I'm not quite sure how challenging this is). Experiments and ablation study are comprehensive, and the final quantitative numbers are competitive. My biggest concern is that the improvement over prior works are quite limited (no more than 0.2%), given all the complex schemes of the method and its required usage of Performer. I understand these benchmark results are somewhat saturated and hard to improve a lot. It's ok, but the paper will be more convincing if the authors can analyze the features learned by their wavelet transform, and show how the learned features are better or different from prior works that do not use wavelet transform.
review_title: null

Meta-review: The submission initially received mixed reviews. The authors presented convincing answers during the author response period, after which all reviewers recommended weak accepts. The AC has carefully read the reviews, responses, and discussions, and agreed with the reviewers' recommendation. Despite the marginal performance gains, the submission has presented a useful and inspiring way of learning shape representations. The AC, therefore, recommends acceptance.

The authors are encouraged to further revise the paper based on the reviews. In addition, the authors should use $\citep$ for all citations that are not used as a pronoun, including all citations in the tables. Please find more information here: https://journals.aas.org/natbib/
Decision: Accept (Poster)