Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors introduce channel-wise convolutions, and an optimized degree mixing set in order to construct equivariant layers that exhibit improved scaling properties and parameter efficiency on some prototypical spherical CNN tasks. 

Section 2 is unnecessarily math heavy with representations and terminologies introduced which are not relevant to the central claims in the paper and not reused in latter sections. The authors should pick out the essentials bits and place the rest of the technical bits to the supplement. The gained space should be used to expand and better explain section 3 which is extremely hard to understand. 

Reading and re-reading Section 3 several times, I am still lost as to what figure 1 is trying to demonstrate. I do not understand what are the trade-offs involved with the constrained generalized convolution as opposed to generalized convolution. From the results it seems that it does not matter, which is counter intutitive. Group convolution has an adverse effect on performance on standard CNNs. I am not sure about the validity of the following statement: "restricted N_b in which only a subset of P_L is used for each degree ` still defines a strictly equivariant operator". Though it makes intutive sense, is there a proof of the same? 

The results section leaves a lot to be desired. In table 3, we see that it is not state of the art on several metrics. The authors do not compare with the improved Esteves et. al. paper from 2020. This result should be added. What is the implication of the CL logL complexity. This should translate to reduced flop count, but there is no discussion on flop or timing of this approach anywhere in the paper. This makes me skeptical whether the proposed approach improves efficiency in practice. There are many typos, incomplete sentences, long hard to read sentences etc. I would recommend the authors to compare on all benchmarks provided in Esteves et. al. (2020). 

Overall, the paper is a very hard read which no clear message on the key contributions. It seems that the MST approach is driving the efficiency (without proof?), coupled with group convolutions which is well known in leterature and cannot be credited as a contribution. This coupled with the marginal improvement on select datasets and incomplete evaluation does not inspire acceptance. 

Post rebuttal comment: Having read the reviews from other reviewers who are subject matter experts, and the authors rebuttal which helped clarify most of my concerns, I am increasing my rating for the paper.  I recommend acceptance as two of the reviewers are convinced about the positive impact of the paper. 

review_title: Hard to understand paper with marginal performance improvement. Insufficient experiments. 

final_score: rating: 6: Marginally above acceptance threshold
review_content: **Summarize what the paper claims to contribute.**
The authors claim to introduce an efficient alternative to previous Spherical CNN models

**Strengths:**
The authors consider the problem of spherical image processing using convolution
The authors present strong empirical results

**Weaknesses:**
Both the mathematical presentation and discussion are difficult to follow

**Clearly state your recommendation (accept or reject) and justification.**
Reject. It seems the authors have given considerable attention to the problem and produced compelling results; however, for me, the mathematical presentation and discussion are difficult to follow which I expect will make it difficult for readers to understand and build upon what has been done. My impression is that some of the difficulty could be resolved with more standard notational choices (e.g. nonlinearities are not often written \mathcal{N}_\otimes), and limiting the use of inline equations.

**Ask questions you would like answered by the authors to help you clarify your understanding of the paper and provide the additional evidence you need to be confident in your assessment.**
(first paragraph 2.1) Does the operator map spherical signals to signals on SO(3)?
The mathematical presentation is given with filters and functions in \mathbb{C}, is reflected in the implementation?
It is unclear to me what the authors mean (specifically) by a hybrid approach

**Provide additional feedback with the aim to improve the paper.**
Perhaps state that \mathcal{H} is the space of spherical signals and the superscript indicates the layer
The notation is a bit difficult to follow (and read since quite a bit is inline) and often is not explained, for example it could be helpful to say that L^2(S^2) are the square integrable functions on the sphere and show what that means.
I think the paper would be easier to read if the language was consistent, for example, in the introduction the language of real and harmonic space is used and in section 2 it seems to change to real and Fourier space. I wonder if spatial and spectral are good words to use in place of these.
(paragraph below eqn 3) remove (w.r.t)
In part because the authors attempt to describe [1,2] and [3] at the same time and because of the abundance of long inline equations, the mathematical presentation is difficult to follow
Moreover, the mathematics are not trivial and not particularly well known, perhaps providing intuition along with the equations would improve readability

**Possible typos:**
(Conclusion) powerful hybrid model → powerful hybrid models
(Introduction) Many fields involve → many fields use

**Post rebuttal**
With consideration of the improved readability of the new submission and comments of other reviewers, I have modified both my initial rating and confidence.

[1] Kondor, Risi, Zhen Lin, and Shubhendu Trivedi. "Clebsch–gordan nets: a fully fourier space spherical convolutional neural network." Advances in Neural Information Processing Systems. 2018.
[2] Taco Cohen,  Mario Geiger,  Jonas K ̈ohler,  and Max Welling.   Spherical CNNs.   InInternationalConference on Learning Representations, 2018
[3] Carlos  Esteves,  Christine  Allen-Blanchette,  Ameesh  Makadia,  and  Kostas  Daniilidis.   LearningSO(3) equivariant representations with spherical CNNs.   InProceedings of the European Con-ference on Computer Vision (ECCV), pp. 52–68, 2018

review_title: Difficult to follow

final_score: rating: 7: Good paper, accept
review_content: This paper introduces a generalized spherical convolution operation that is strictly equivariant to rotation. The authors show that the spherical convolution operations introduced in prior works can be encompassed by the proposed approach. Because spherical convolutions introduce significant computational overhead, the authors also introduce an array of methods that reduce the computational cost while maintaining the model accuracy. Experiment results on multiple benchmark datasets show that the proposed approach outperforms the alternative approaches while having less number of parameters.

This paper studies an important problem. In particular, it addresses an important issue in spherical convolution operation, i.e. the computational cost of the operation. The proposed operation has the desirable property of strict rotational invariance, and it is general enough to replace existing spherical convolution operators and may be used as the basic component for CNN on spherical signals. The experiment results also verify the benefit of the proposed method.

On the other hand, there are several aspects on which the paper may be improved. First of all, there are some designs in the proposed method that are not carefully discussed or tested:
1) While the authors use tensor-product to replace pointwise activation, it is unclear what's the relation between these two operations. Is tensor-product equivalent, more or less expressive than pointwise activation? Given that activation plays an important role in neural network, the authors should try to provide more information about the new activation function.
2) The authors propose channel wise activation and degree mixing to reduce the computational cost. However, they also reduce the expressiveness of the model. Therefore, it is worthwhile to provide some study on how they impact the performance of the model. For example, what will the model performance be if these methods are not applied?

Second, the experiments are somehow limited. The authors only test the proposed convolution operations on a single model, and the model size is different from the baselines except for the MNIST experiment. It is unclear why the model sizes are not tied in the experiments. A more informative experiment will be comparing different methods over different model sizes. Also, while the main contribution of this work is to reduce the time complexity of the convolution operation, the experiments do not show the comparison in run time. The authors should also try to evaluate the model efficiency as well as memory overhead, as these are also important factors that limit the usage of spherical convolution operation.

The rebuttal provides valuable information that was missing in the original paper and improves the readability. Therefore, I recommend accepting the paper.
review_title: This paper addresses an important problem in spherical convolution, i.e. the computational cost, and proposes a series of approach to reduce the time complexity.

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The paper introduces a framework for computationally efficient and exactly rotation-equivariant spherical CNNs. The work most closely resembles the Fourier space method of Kondor et al., but improves on it in a number of ways: firstly, a channel-wise structure is introduced for the tensor product nonlinearities, which avoids the degree blowup of this operation while still allowing mixing between different harmonic degrees. Secondly, computational complexity of linear layers is reduced by factorizing it into three operators, two of which operate similar to depthwise-separable convolutions and one of which acts uniformly across channels. Thirdly, an optimized sparse degree mixing set is proposed, based on a minimum spanning tree. Finally, a more efficient sampling theorem is used that reduces the Nyquist rate by a factor of two compared to the ones used in previous works on spherical CNNs.

The paper is very well written and the authors clearly have a thorough understanding of the noncommutative harmonic analysis involved. This does not mean the paper will be easy to understand for all readers, but for those familiar with the relevant mathematics, either from textbooks or earlier works in the spherical CNN literature, the paper is very readable. The proposed improvements make a lot of sense to me, and their computational complexity improvements are clearly stated. The performance of a network architecture that includes the new layers is tested and shown to yield competitive or state of the art performance on several benchmark problems that have been used in many previous works.

Overall I think this is a very nice paper, but I have a few minor concerns and points of improvement:

The degree mixing set (3.1.3) is a minimum spanning tree that minimizes a certain computational cost. This makes some sense, but it is not clear to me that this approach is optimal in any meaningful sense or necessary at all. I have personally experimented with sparse channel connectivity in planar CNNs, and found that it does not seem to matter much how exactly the channels are connected, with the main factor determining compute/accuracy being the number of connections. Full degree mixing does seem desirable, but this implies the need of a MST only if one wishes to use the same connectivity structure in multiple layers. An interesting baseline would be to do the degree mixing using a random pattern in each layer, with various sparsity levels. It may turn out that only the sparsity level but not the precise connectivity structure matters in practice. Such a finding would not diminish the paper's significance.

It should be clarified in the paper that full mixing happens only across several layers (as many as the maximum path length / tree width in the MST). The question then arises whether full mixing actually happens in the considered architecture, given that it is not very deep.

It would be interesting to see actual implementation details in some DL framework, as well as wallclock timings. Also, code would be much appreciated.

The appendix describes a method for enforcing spatial localization of the spectral filters, but it is not clear from the paper if/how this is actually used in the network architecture that is tested.

It would be nice to know why the initial convolution layers are necessary, instead of just using the generalized layers introduced in this paper in their full glory.

I may have missed it, but could not figure out what L_G^(psi) refers to in 2.6.

Post rebuttal update: I have read the author response and updated paper, as well as the other reviews, and have decided to maintain my rating.
review_title: Review

Meta-review: This paper proposes an efficient approach for computing equivariant spherical CNNs, significantly reducing the memory and computation costs. Experiments validate the effectiveness of the proposed approach.

Pros:
1. Speeding up equivariant spherical CNNs is a valuable topic in deep learning. 
2. The proposed approach is effective, in all parameter size, memory footprint and computation time.
3. The theory underpinning the speedup method is sound.

Cons:
1. The readability should be improved. Two of the reviewers complained that the paper is hard to read and only Reviewer #2 reflected that it is "easy" to read (but only under the condition that the readers are familiar with the relevant mathematics), and this situation is improved after rebuttal. Nonetheless, this should be further done.
2. The experiments are a bit limited. This may partially be due to limited benchmark datasets for spherical data, but for the existing datasets used for comparison, Esteves et al. (2020) is not compared on all of them. Esteves et al. (2020) is only reported on spherical MNIST, which has very close performance to the proposed one. This worries the AC, who is eager to see whether on QM7 and SHREC’17 the results would be similar. 

After rebuttal, three of the reviewers raised their scores. So the AC recommended acceptance.
Decision: Accept (Poster)