Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper builds the connection between various steerable CNN structures based on group representation theory and filter transformations. Using the discrete rotation and reflection group as an example, the paper establishes ways to construct steerable convolutional filters that transform features in trivial, regular, and irreducible representations. In particular, recent works on steerable CNNs such as ORN, RotDCF, TI-pooling, and RotEqNet can all be explained under such framework.

The paper is generally well written and well organized. However, I reckon the material would be quite dense for readers interested in equivariant CNNs but not well-versed in group representation theory, so a bit more explanation either in text or in the appendix would be helpful. The main theoretical results in the paper on the considered discrete group transformation are interesting, and can potentially lead to other development in the community. The numerical experiments seem to be limited (though this phenomenon seems to be the issue for most of the papers in this area...)

** Pros
1. Well written and technically correct.
2. Interesting theoretical results that might inspire other works in the field.

** Cons (to be explained in more details in questions)
1. Some notation and abbreviation are used without proper explanation, which might be confusing.
2. Limited experimental results

** Questions and comment
1. Some of the notations and abbreviations are used without proper explanation. For example, "K" in equation (12), OCR on page 1, and missing reference on R2Conv. I would encourage the authors to make a greater effort to clarify their ideas and results.

2. The theory seems to be built only on discrete group. Does it generalize to continuous group transformation (such as SO(2) before discretization)?

3. Also, does the theory generalize to non-compact groups such as  scaling and shearing?

4. The experiments seem to be limited, even though this seems to be a common issue in papers in this field. However, I do recommend the authors to present the "equivariant loss" when using their proposed FILTRA, considering that discretization and interpolation might cause a problem in their setting unlike other means of steerable CNNs such as RotDCF and Harmonic Net
review_title: An interesting paper that builds the connection between steerable CNN based on group representation theory and filter transformation

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper studies the connection between steerable CNN and filter transformation. The authors show theoretically that filter transformation can be used to realize steerable filters over different group representations. The authors also empirically show that the filter transformation based steerable CNN performs on par with the implementation based on harmonic bases.

Overall, I think this paper is hard to follow. The presentation and notation depends on that of prior works and is hard to understand without being familiar with Steerable CNN. Compared with prior works, this paper focuses more on the mathematical derivation but falls short of explaining their implication. The result is therefore hard to understand for those that are not familiar with the theoretical basis. Also, the authors do not clearly show why the contribution of this work is significant. The theoretical connection doesn’t seem to provide a significant advantage nor solve the problems in existing methods. Similarly, the experiment doesn’t provide much information and only shows that the proposed implementation works on simple datasets. The authors should try to highlight how this work may benefit other research or applications and verify that the proposed method works on more complicated and realistic data.

Another aspect that can be improved is to discuss how the theory generalizes when multiple convolution layers are applied. In general, the theoretical properties of steerable convolution does not automatically generalize to multiple convolution layers. This is particularly important in real world data, because the transformation may happen at different scales and may need to be accounted for at different layers in the network. Therefore, the results derived from a single convolution operation may not be sufficient.


review_title: This paper provide the theoretical connection between steerable convolution and filter transform method.

final_score: rating: 5: Marginally below acceptance threshold
review_content: ***Summary***

The paper outlines the construction of filters in a group equivariant convolutional network equivariant to the groups CN (cyclic group of order N (N rotations)) and DN (dihedral group of order N (N rotations and flipping)). The authors achieve this by taking linear combinations of (harmonic) basis filters, as was shown in Weiler et al. (2018). They then proceed to show explicitly how one can build filters between particular representations of each group, namely, the trivial, irreducible, and regular representations. For experiments they demonstrate that the filters perform en par with those of Weiler and Cesa (2019), who had performed a near-exhaustive comparison among representations of SE(2, R).



***Pros***

Technically I believe the work to be sound. I did not see any mistakes jump out at me. I also think that it is laudable that the authors included a high amount of detail in their exposition, which lays bare the exact mechanisms by which one would effectively go about building an equivariant layer.

Experimentally it appears that the authors chose a sensible baseline and that they compared on dataset, which are relevant to the topic and commonly used in the equivariance literature. The authors also provide timing information on how fast each filter can be generated, which is something I have not seen before in the equivariance literature and which I appreciate seeing.




***Cons and constructive feedback***

Perhaps my largest criticism is that it is not obvious exactly what the contribution of the paper is meant to be. Perhaps this originates from my not knowing what the authors mean by “filter transform” and “steerable CNN”. My understanding is that the authors believe it is currently unknown how to construct a steerable CNN for the cyclic and dihedral groups. They provide explicit constructions in section 3, but to my knowledge these are already provided in the long appendix of Weiler and Cesa (2019), furthermore they can be found in Cohen and Welling (2016), Cohen and Welling (2017) and Bekkers et al. (2018) and most notably Weiler et al. (2018), who were to first to show how to construct equivariant filter layers some linear combinations of harmonic basis filters. As a result I am unsure of how to gauge the novelty of this contribution.

In the experiments of Table 2 (classification and regression) the authors compare their framework with an R2conv (representative harmonic based convolution) and a regular translationally equivariant network. I would like to know what exactly is a representative harmonic based convolution? Is that meant to refer to the work of Weiler and Cesa (2019)? 

In the results tables there are no error bars. If possible I would have liked to have seen them. Since they are not there is it very hard to judge the efficacy of the results, which differ from the baselines by very small amounts.

Given the proximity of the work to Weiler and Cesa (2019), I would like to know exactly what differentiates the two works.

Please include exactly what the functions roll, flipped, and circulant actually do mathematically. I found these difficult to parse.

The mathematical level of the paper is pretty heavy for the uninitiated. It may be advisable for the authors to include a glossary of terms, if not short descriptions, in an appendix. If this is too much, at least please point to other papers, which have easily readable background sections for those not already well-read in group theory.

The authors may wish to have the submission proofread for spelling and grammar.

***Post rebuttal review***

Having read through the rebuttal, the updated submission, and the reviews of the other reviewers I have upgraded my review from a reject to marginally below acceptance. This is for two main reasons. 1) the authors have vastly improved the presentation of the submission, which now looks a lot easier to read, 2) the authors have clarified for me, at least, what the main contribution of the work is. 

That said I am not entirely sure what this contribution adds to the equivariance community, hence why my recommendation still leans towards reject. As far as I am aware, solving the equivariance equations is not the large bottleneck to progress in our community. They are linear equations, and there is work back into the 80's solving them (check out people like Pietro Perona, Patrick Teo). I think more importantly we need to focus on pushing the boundaries in areas such as extension to non-Euclidean manifolds, convolution over non-compact groups, learning symmetries, etc. While this work is clearly mathematically sound and the authors have demonstrated deep knowledge of the area, it feels a little like retracing prior works. That said, if the other reviewers disagree then I don't mind this paper being accepted. Perhaps since I have worked in this area, what appears as obvious to me is not generally acknowledged and this paper may serve as a useful clarification for those wishing to dive into the literature.
review_title: Initial review for FILTRA

final_score: rating: 6: Marginally above acceptance threshold
review_content: Equivariant Steerable CNNs for 2D/3D rotation+reflection+translation groups have generally been implemented as a filter transform/expansion step followed by a standard convolution. The filter expansion step involves taking a linear combination of steerable basis filters. These basis filters are pre-computed before network training by solving a linear system or by sampling the continuous analytical solution (this can take a few minutes). Depending on the chosen group representation wrt which the network layer is equivariant, a different filter basis will emerge, but in general one can see that the basis filters come out as rotated and flipped copies of some basis filters, with the occasional sign flip (this has been stated in some earlier works). The precise way in which a basis filter is to be rotated and flipped to obtain the steerable filter basis for the 2D case had not been worked out before, to my knowledge, and this is one of the contributions of this paper. The analysis is done for each (input, output) representation type chosen from {trivial, irreducible, regular} representations. Having worked this out, the paper proposes to use this as a way of implementing the filter expansion step, starting from a basis filter and rotating/flipping it to obtain an expanded filter bank. 

The proposed method (FILTRA) does not require a precomputation step if I understand correctly, which is a significant practical advantage. Experiments further show that the method is similar or faster at filter expansion. Finally, the method is validated by training networks on benchmark tasks and shown to perform similarly to or better than the steerable CNN implementation of Weiler & Cesa, which is the best existing implementation.

The paper briefly mentions that filters cannot be rotated exactly on a discrete grid, but I didn't figure out how the authors propose to deal with this issue. How exactly are the filters rotated?

I think the method proposed in the paper is useful, as it is both faster and better than existing steerable CNN implementations. The paper itself is fairly well written and technically correct as far as I can tell, but may be challenging to read for those who are not yet knowledgeable about steerable CNNs. Those readers however are unlikely to be interested in learning about the implementation details of steerable CNNs anyway, so perhaps this is fine. The reason I am not giving a higher rating is that I think that although this is a useful contribution to the literature on steerable CNNs, which are being used in an increasing number of applications, the paper does not represent a major breakthrough and although the calculations are non-trivial, does not contain highly unexpected or deep theoretical results.

Typos:
Cadestrian -> Cartesian
irreduciable -> irreducible
equity -> equality

Edit:
Having read the reviews, rebuttal and updated paper, I have decided to maintain my score of 6.
review_title: Review

Meta-review: This paper introduces an approach based on filter transform for designing networks equivariant to different transformation groups.
Especially, the authors rely on the haramonic analysis view of steerable CNNs given in Weiler & Cesa (2019) to design an equivariant filter bank by computing simple transforms over base filters. 

The reviewers finds the paper technically solid but difficult to read and with a limited contribution. 
The AC carefully reads the paper and discussions. Although the connection between steerable CNNs and filter transform are interesting, the AC considers that the main contributions of the paper should be consolidated, especially the positioning with respect to Weiler & Cesa (2019). \
Therefore, the AC recommends rejection.
Decision: Reject