Venue: ICLR 2017 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper deals with convolution along Lie groups.

Pros:
- Good numerical results
- Interesting applications linked to convolutions on Lie groups

Cons:
- Sometimes, the writing is not clear/poor.
- The theorems 1 and 2 are in fact direct applications of Lie group theory, but this is not made explicit.

I do not detail more my review, as I believe my questions/answers complete it fairly. However, if the authors require it, I can be more specific. I think the writing should be improved a bit.
review_title: Review

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper shows how group convolutions (for two dimensional commutative groups) can be performed by standard CNNs if the input is warped using a fixed warp. The idea is practical and seems to work well. The paper is well written.

I agree with reviewer 1 that the ‘theorems’ do not deserve to be labelled as such. Theorem 1 is equivalent to the second equation from this section of the wikipedia page on convolution: https://en.wikipedia.org/wiki/Convolution#Convolutions_on_groups. The existence of Haar measure (from which commutation of Lg and convolution follows immediately) is a well known and elementary theorem in harmonic analysis, which would be treated in the first few pages of any textbook on the subject. It also immediately clear that any commutative group has an additive parameterization (theorem 2). The fact that the paper does not present new deep mathematical results is not a significant weakness in my opinion, but the derivations should not be camouflaged as such.

The claim that previous methods that use group convolutions are slow and that the presented approach has better computational complexity is not supported by empirical evidence, and the theoretical analysis is still a bit misleading. For example, the authors write “Unfortunately these approaches do not possess the same memory and speed benefits that CNNs enjoy. The reason is that, ultimately, they have to enumerate all possible transformations”. The presented method also has to enumerate all transformations (in a limited range, on a discretized grid), and this is feasible only because the group is only 2 dimensional (and indeed this is also true for standard CNNs which enumerate translations). 

As noted in my pre-review question, I believe the computational complexity analysis is not entirely correct, and assume the authors will correct this.

Equation 4 is presented as a new invention, but this has been used in previous works and is well known in mathematics, so a citation should be added.

The main advantage of the presented method over several earlier methods is that it is very simple to implement, and can re-use highly optimized convolution routines. As I understand it, Dieleman et al. and Cohen & Welling also use standard convolutions (after a fixed filter / feature map warp), but these papers only consider discrete groups. So it seems like this paper occupies a unique place in the space of equivariant convolutional networks: non-commutative (-), low-dimensional (-), continuous (+) groups, simple (+) and efficient (+) algorithm. Given the proximity though, a more thorough and balanced appraisal of the merits and demerits, as well as the novelty, relative to each of the previous works would be useful. 

Provided that these issues are cleared up, I would recommend the paper for publication.
review_title: Good paper, but several issues remain

final_score: rating: 7: Good paper, accept
review_content: The paper suggests method to make the convolution invariant to other type of spatial transformations besides translation. It is interesting that the convolution can be invariant to different variation types by simple warping of the input image, which is related to arbitrary data perturbation in image recognition.
Experiment section needs more description and technique implementation details.
The relationship of theorems to Lie group is not defined properly.
It need more detailed comparison of method with available methods in the literature. 

review_title: Efficient invariance to spatial transformation using input warping for Convolution

final_score: rating: null
review_content: How does the method deal with local transformations? It seems like the method would only be able to deal with a single global transformation. For example, the log-polar warp would turn rotations around x_0 into translations, but not rotations about arbitrary points. Is that correct?

Regarding computational complexity: to implement the generalized convolution we could do the following. For each of the n^2 output positions, we can take s^2 bilinear samples from the image (cost O(s^2)) and dot them with the filter (again, cost O(s^2)) for a total of O(n^2 s^2), just like normal convolutions. Is there any reason such an implementation is not preferred? (If so, that should be given as the reason for preferring the presented algorithm over previous work, instead of computational complexity.)
review_title: Locality & complexity

final_score: rating: null
review_content: From my understanding of the paper, you reparametrized an image via a 2 dimensional Lie group. I have few questions.

First I do not understand the difference between equivariance and invariance: in the introduction, you say you will introduce an invariant operator, and finally the networks are equivariant. Could you clarify?

You claim: "Projecting a signal onto such a basis still has an asymptotic complexity higher than that of a simple convolution, and the basis itself requires more storage than a convolutional filter."

I do not understand what it exactly means, for instance, in Cohen & Welling, it is tried "to keep the number of parameters approximatively fixed". Could you clarify? Also, in their case, they do not explicitly encode the p4/p4m group in the first layer, and they don't compute either the orbits of each images. They first apply 2D-convolutions(covariant with p4/p4m), and then, the convolution becomes 3 dimensional. So I do not understand why you write "projecting a signal", since they seem to project the filters, not the input.

Is additivity a restriction for the 2 dimensional parametrizations via Lie groups? I understand the condition of fixed point as the fact that the action of the group is transitive, am I correct? If a pivot point exists, then isn't any point a pivot(in the additive case)? Can you give examples of parametrisation where there is no pivot point?

Besides, you used only 2D structures, but could you use scales, pitches, .. all together? In your experiments, you removed the translation variabilities by choosing centered images. Do you have any ideas how to solve this problem if this was not the case? I guess the fact to stay in the 2D domain might be a limitation to keep the Theorem 2 true. Do you have any suggestions to overpass this property?

In section 5.3, you write that in order to maintain additivity, you did not consider the radius of a sphere. Do you think a mapping of the radius like exponential mapping could work or is there a limitation?

In the Google experiment, why did you downscale the images? Can you give more details about the training of the deep network: how did you avoid overfitting (maybe there wasn't?)? Do you have any intuitions on the nature of the learned features? Is the "object scale", the parameter "s" mentioned in the previous sections?

In the faces experiment, can you explain why you had to increase the support of the filters, from 5 to 9?

Thanks!

review_title: (Repost - mistakes in the console)

Meta-review: The reviewers found the idea interesting and practical but had concerns about the novelty of the approach and the claims and theory presented in the paper. In particular, it seems that the reviewers feel that the authors' claim to present novel theory is unjustified (i.e. the theorems presented are not novel). Also claims regarding the advantages of this approach relative to related literature (in particular Dieleman et al. and Cohen & Welling) seem to be unsubstantiated. This work would likely be more well received just through rewriting the manuscript with, as stated by AnonReviewer3, "a more thorough and balanced appraisal of the merits and demerits, as well as the novelty, relative to each of the previous works" and weakening the claims of novel theory.
Decision: Reject