Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper propose the hybrid discrete-continuous group convolution on sphere that possess both equivariant property and computational scalability. The sparse tensor representation is used in implementation to further save computation cost and memory. Experiments are performed on spherical MNIST, Omni-SYNTHIA, 2D3DS and Pano3D, which shows some improvements over previous works. 


strength_and_weaknesses: Two main advantages of this paper are as follows:

1, The paper is well written and the key contribution is easy to understand. 

2, The analysis of equivariance error, computational cost and memory usage are provided to support the efficiency of the methods.

Two main disadvantages of this paper are as follows:

1, In experiments, the state-of-the-art claim seems unprecise. For example, in 2D3DS dataset, the HoHoNet (https://arxiv.org/pdf/2011.11498v3.pdf) even achieve mIoU of 52%. Besides, the improvement is minor without data augmentation compared to previous methods.

2, The sufficiency of the restriction from SO(3) to SO(3)/SO(2) equivariance need to be further developed. It seems that the method can’t capture the fully SO(3) equivariance as continuous approaches, which may lead to performance degradation on dataset defined on sphere with random rotation.

clarity,_quality,_novelty_and_reproducibility: $\textbf{Clarity}$:
This paper is clearly written. However, the description of the asymptotic equivariance of SO(3)/SO(2) is a little bit abstract, it is better to illustrate such equivariance with figure. Besides, it would be benefit to give the pseudo-code of the implementation of the custom sparse gradients. 

$\textbf{Quality}$:
This paper is technically sound, for example, the custom sparse gradients which are not supported by TensorFlow and Pytorch, are implemented by the authors.

$\textbf{Novelty and originality}$:
This paper is original and novel, proposing a new model to overcome the shortcoming that exist in continuous spherical CNNs and discrete spherical CNNs.


summary_of_the_review: In summary, this paper is well written, but it still needs to be refined carefully. I recommend marginally below the acceptance threshold for this paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a kind of scalable and equivariant spherical convolution by applying continuous filters to discrete spherical inputs. The authors claim that this method is more computation- and memory-efficient than previous works.
strength_and_weaknesses: Strength:
1. This method combines the advantages of continuous and discrete spherical convolutions, and is both rotation-equivariant and scalable. 

Weaknesses:
1. This paper contains many long sentences, reads rather awkwardly and even has many fault wordings, e.g., 
"the sphere is a homogeneous space with global symmetries on which act elements of the group of 3D rotations SO(3)"
"The DISCO approximation of Equation 3 is highly accurate for bandlimited signals, which real-world signals can be well approximately by for sufficient bandlimit."
"The filter is represented continuously so also does not introduce any error."
I think the writing should be significantly improved.

2. To my understanding, the main idea is in Equation (3), i.e., applying continuous filters \psi to the discrete inputs f, and then implementing group convolutions, which is rather trivial.

3.  Restricting rotations to SO(3)/SO(2) will degrade the equivariance over arbitrary 3D rotations to that over z-rotations. Actually, the equivariance w.r.t z-rotations can be easily achieved by employing oriented-aware spherical CNNs, such as HexUNet. 

4. The baselines of spheical MNIST and Pano3D are both planar CNNs, which are very weak. The result for 2D3DS is also not the state-of-the-art, as the performance is much weaker than [1].

5. The paper claims that this method is both memory- and computation-efficient. However, there is no practical experimental evaluation about that.

[1] Eder et al. Tangent Images for Mitigating Spherical Distortion. CVPR 2020.
clarity,_quality,_novelty_and_reproducibility: See the comments above.
summary_of_the_review: This idea is trivial and the writing should be significantly improved. Also, their experiments cannot support their claims well.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: There are now a wide crop of Spherical CNN models, which can be classified along several different axes e.g. fully Fourier/fully real/partly real and partly harmonic; or continuous models and discrete models. If we focus on the latter, existing continuous models guarantee rotational equivariance, while being computationally expensive (and this varies depending on how much equivariance error is tolerated). On the other hand, discrete approaches (such as those that represent spherical signals as a graph) are more computationally attractive, but don't handle equivariance in a satisfactory manner. Part of the expense of the continuous approaches comes from the prohibitive cost of the spherical/rotational Fourier transform. This paper proposes a framework that attempts to combine the advantages of both of these styles of models, leading to the proposed spherical CNN scaling up much better than existing models, and also becoming capable of doing dense prediction. 

The paper first describes the basic idea of the continuous approach (exemplified by Cohen et al, Kondor et al., Esteves et al. and Cobb et al.), while describing some of the issues concerning sampling and the computational load. It then briefly discusses some of the discrete models (which tend to be graph based and are inherently limited in the symmetries that they can capture). It is also mentioned that recent scattering-based methods still don't manage to work quite well with high-resolution images -- something that I am in full agreement with. The approach of the paper is described, at a high-level, in sectoin 3.1. The key equations are equations 3 and 4. The main innovation of this section is to replace the integral by a quadrature (possible source of error), which is evaluated at some sample positions. However, the fllter, and its transformation is kept continuous -- and consequently does not lead to errors. The quadrature error is minimized (or in principle can be avoided) by appeal to an interesting sampling theorem from a decade ago. After this the authors describe a few area where the computational load can be further chopped off (and permit linear scaling with the pixels). This includes reducing the space of rotations SO(3) to SO(3)/SO(2) (the authors keep using equivariance to SO(3)/SO(2) -- which I am not sure what it means since it is not a group, it is quotient space). Filter parameterization also uses simple linear interpolation after sample points are selected and reduces the load a bit further (although possibly being a source of error). The authors also detail how they leverage the sparse representations that are obtained for further computational (and memory) gains. The memory gains are also augmented by making use of a symmetry in the sampling procedure. Eventually, the authors also describe an extremely well-engineered system that is able to scale to levels unlike other spherical CNNs and performs wells on some experiments (including dense segmentation). 
strength_and_weaknesses: - The paper is very well motivated and solves an important problem -- that is of scaling spherical CNNs so that they become useful for larger images and for dense prediction. 
- The main conceptual contribution is quite simple, but still significant. 
- The system is implemented very efficiently, and the engineering effort and attention to detail is admirable. 
- The dense prediction results are good, and unlike other spherical CNNs. 
- Paper is also very well written. 

Weaknesses
- The paper seems to make no indication about code availability. I will definitely consider this to be a negative.
clarity,_quality,_novelty_and_reproducibility: Clarity and quality: The paper is presented impeccably -- I quite enjoyed reading it. The development of the main contribution of the paper is done in a way that makes it seem obvious. 

Novelty: The paper presents a heavily engineered spherical CNN that can scale linearly with number of pixels. It does a clever combination of continuous and discrete aspects in the convolution formula, using an interesting sampling theorem. The results on dense prediction tasks for spherical images are also interesting and new. 

Reproducibility: Given the writing and presentation of the paper, I am confident that it is easy to reproduce. But I am a little turned off by non-availability, and lack of indication thereof, of the code and experimental replication.

Minor comment:
- On page 2, f, \psi: G/N -> \mathbb{R} should be G/H
summary_of_the_review: See above.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes a novel form of group convolution called Discrete-Continuous (DISCO) group convolution, which is targeted at making group convolutions scalable and at the same time retain equivariance properties. DISCO discretizes the input signal space while the filters and output space are maintained to be continuous. If the input signals are band-limited and there are sufficient samples for reconstruction, then the group convolution can be performed exactly using DISCO convolution. This means that the computation is much faster and also, the operation is exactly equivariant to group action. The authors then discuss the special case of signals on the $S^2$ sphere with the natural $SO(3)$ action as well as $SO(3)/SO(2)$ transformation. They also discuss in detail how to build a computationally scalable DISCO convolution using sparse tensors and other memory optimization. Experiments are performed on Rotated-MNIST recognition and semantic segmentation and depth estimation for omnidirectional images. 
strength_and_weaknesses: Strengths:

1. The paper addresses an important question which is how to make group convolution scalable while maintaining equivariance. Earlier attempts at scalability required full discretization which sacrificed equivariance. In contrast, here, only the band-limited signal is discretized to a level where all the information is still maintained, and the filters and the outputs are continuous. The authors then show that this maintains equivariance while making the computation much cheaper as expensive Fourier transforms are no longer required. This is a novel direction.

2. These general ideas are made useful in practice for spherical images by presenting DISCO version of the spherical and $SO(3)$ convolution.

2. The theoretical speedup is huge and the computation is linear in the number of pixels when the filters have local support and are small. 

3. The authors describe the implementation in detail using sparse tensor which shows linear scaling with number of pixels. This is very good result, in my opinion. And as the authors say, for 4k images, this shows a theoretical speedup of $10^9$.

4. The authors also describe smart tricks for memory optimization by exploiting the patterns in the filter arrays and the convolution operation leading to a $10^4$ improvement. 

5. Experimental results on equivariance tests clearly show that the method maintains nearly perfect equivariance. The error mainly comes from having to eventually discretize the integral in order to compute the convolution on a computer. 

6. Although I am not familiar with the baselines, results on semantic segmentation and depth estimation datasets show very good results using DISCO. I am especially happy to see the results on depth estimation performing comparably with a U-Net that uses 40x the number of parameters.

Weaknesses:

1. There is a crucial detail that I am not following in the method which I hope the authors can explain. I understand that the sampling theorems on a compact group or its homogeneous space (e,g, sphere) show how many samples are needed to preserve all the information after discretization. But I don't see how this leads to Eq. (3) (and Eq(4)), Given a discrete signal, I would think we would still need to first reconstruct the underlying continuous signals by some low-pass filter (convolving with some form of a sinc function?) on the space for performing the convolution. I am not sure I understand how the result in Eq. (3) and Eq. (4) is possible. 

2. While the speedup and memory improvements are fantastic, they are still based on theoretical analysis. Can the authors show computation times observed empirically?

3. Experimental results should ideally have error bars and statistical significance results. 

4. I urge the authors to try and release their software implementation to make the experiments easily reproducible, but of course, this is not required.
clarity,_quality,_novelty_and_reproducibility: I think the paper has several novel ideas for making group convolutions scalable while retaining equivariance, describing how these ideas are applicable for spherical signals and showing large computation and memory improvements. The paper is generally clear on most of these aspects. I have difficulty in understanding some parts which I have addressed under weaknesses. In principle, it may be reproducible, but I think, practically, it would be quite difficult, especially, writing custom sparse gradients.
summary_of_the_review: I think this paper has several very good ideas, implementation and experimental results. There are some crucial details that I am not able to understand by reading the paper which I hope the authors can explain. For now, I am giving this paper a "marginally above threshold", but would be more than happy to increase my rating to an accept or a strong accept depending on how the authors address the weaknesses I have listed. 

UPDATE AFTER AUTHOR RESPONSE:

I thank the authors for their explanations and improvements to the paper. Based on this and the other reviews and response, I am recommending acceptance of the paper.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper originally got one 8 (accept, good paper), one 6 (marginally above threshold) and two 5s (marginally below threshold). The strength includes clear paper writing, detailed analysis, notable theoretical speedup and memory saving etc. The weakness includes not-so-strong novelty and experiments, limitation on SO(3)/SO(2) equivariance, no validation on the claim of memory-and-computation efficiency, etc. After rebuttal, one reviewer raised from 6 to 8. By the overall scores, the AC recommended acceptance.
justification_for_why_not_higher_score: Please see the weakness in part 1. The paper is actually marginally acceptable.
justification_for_why_not_lower_score: The average score, 6.5, of the paper makes the paper ranked the 2nd among the papers I handled. There are two strong supports from the reviewers (two 8's) and the negative score (two 5's) are not bad.
Decision: Accept: poster