Venue: ICLR 2017 conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: I sincerely apologize for the late review!

The first part has a strong emphasis on the technical part. It could benefit from some high level arguments on what the method aims to achieve, what limitation is there to overcome. I may have misunderstood the contribution (in which case please correct me) that the main novel part of the paper is the suggestion to learn the group parameterizations instead of pre-fixing them. So instead of applying it to common spatial filters as in De Brabandere et al., it is applied to Steerable Frames?

The first contribution suggests that "general frame bases are better suited to represent sensory input data than the commonly used pixel basis.". The experiments on Cifar10+ indicate that this is not true in general. Considering the basis as a hyper-parameter, expensive search has to be conducted to find that the Gauss-Frame gives better results. I assume this does not suggest that the Gauss-Frame is always better, at least there is weak evidence on a single network presented. Maybe the first contribution has to be re-stated. Further is the "Pixel" network representation corrected for the larger number of parameters. As someone who is interested in using this, what are the runtime considerations? 

I would strongly suggest to improve Fig.3. The Figure uses "w" several times in different notations and depictions. It mixes boxes, single symbols and illustrative figures. It took some time to decipher the Figure and its flow. 


Summary: The paper is sufficiently clear, technical at many places and readability can be improved. E.g., the introduction of frames in the beginning lacks motivation and is rather unclear to someone new to this concept. The work falls in the general category of methods that impose knowledge about filter transformations into the network architecture. For me that has always two sides, the algorithmic and technical part (there are several ways to do this) and the practical side (should I do it)? This is a possible approach to this problem but after the paper I was a bit wondering what I have learned, I am certainly not inspired based on the content of the paper to integrate or build on this work. I am lacking insights into transformational parameters that are relevant for a problem. While the spatial transformer network paper was weaker on the technical elegance side, it provided exactly this: an insight into the feature transformation learned by the algorithm. I am missing this here, e.g., from Table 2  I learn that among four choices one works empirically better. What is destroyed by the x^py^p and Hermite frames that the ResNet is *not* able to recover from? You can construct network architectures that are the superset of both, so that inferior performance could be avoided. 

The algorithm is clear but it is similar to the Dynamic Filter Networks paper. And I am unfortunately not convinced about the usefulness of this particular formulation. I'd expect a stronger paper with more insights into transformations and comparisons to standard techniques, a clear delineation of when this is advised. 
review_title: null

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper presents an improved formulation of CNN, aiming to separate geometric transformation from inherent features. The network can estimate the transformation of filters given the input images. 

This work is based on a solid technical foundation and is motivated by a plausible rationale. Yet, the value of this work in practice is subject to questions:

(1) It relies on the assumption that the input image is subject to a transformation on a certain Lie group (locally). Do such transformations constitute real challenges in practice? State-of-the-art CNNs, e.g. ResNet, are already quite resilient to such local deformations. What such components would add to the state of the art? Limited experiments on Cifar-10 does not seem to provide a very strong argument.

(2) The computational cost is not discussed.
review_title: review

final_score: rating: null
review_content: - The local steerability seems to be an useful tool, especially when invariance to the selected group of transformations is needed in the low level image features (e.g. for the edge detection). Do you think that this concept would work also for the higher network layers, e.g. for rotation invariance for object classification? Do you think that the higher layers would be steerable? And what would be the shortcomings of your method for this case? 
Please take this question more as an open question to clarify the details and the context of your method.
review_title: Pre-Review Questions

final_score: rating: 7: Good paper, accept
review_content: This works applies steerable frames for various tasks where convolutional neural networks with location invariant operators are traditionally applied. Authors provide a detailed overview of steerable frames followed with an experimental section which applies dynamic steerable network to small machine learning problems where the steerability is conceptually useful.

Even though the evaluation is performed only on few small tasks, the reason why more tasks were not evaluated is that piece-wise pose invariance is needed only for a subset of tasks. The fact, that simply using overcomplete bases as a sort of "feature pre-processing" improves the results for already highly optimized ResNet and DenseNet architectures is quite interesting achievement.

For the edge detection, a relatively hard baseline is selected - the Dynamic Filter Networks, which already attempts to achieve position invariant filters. The fact that DSFN improves the performance on this task verifies that regressing the parametrization of the steerable filters yields better results than regressing the filters directly.

In the last experiment authors apply the network to video classification using LSTMs and they show that the improved performance is not due to increased capacity of the network.

In general, it is quite interesting work. Even though it does not offer ground-breaking results (mainly in a sense of not performing experiments on larger tasks), it is theoretically interesting and shows promising results.

There are few minor issues and suggestions related to the paper:
* For the LSTM experiment, in order to be more exact, it would be useful to include information about total number of parameters, as the network which estimates the pose also increases the number of parameters.
* Would it be possible to provide more details about how the back-propagation is done through the steerable filters?
* For the Edge Detection experiment, it would be useful to provide results for some standard baseline - e.g. CNN with a similar number of parameters. Simply to see how useful it is to have location-variant filters for this task.
* The last sentence in second paragraph on page 1 is missing a verb. Also it is maybe unnecessary.
* The hyphenation for ConvNet is incorrect on multiple places (probably `\hyphenation{Conv-Net}` would fix it).

review_title: Interesting approach for adaptable convolutional filters

Meta-review: This paper studies how to incorporate local invariance to geometric transformations into a CNN pipeline. It proposes steerable filter banks as the ground-bed to measure and produce such local invariance, building on previous work from the same authors as well as the Spatial Transformer Networks. Preliminary experiments on several tasks requiring different levels of local invariance are presented. 
 
 The reviewers had varying opinions about this work; all acknowledged the potential benefits of the approach, while some of them raised questions about the significance and usefulness of the approach. The authors were very responsive during the rebuttal phase and took into account all the feedback. 
 
 Based on the technical content of the paper and the reviewers opinion, the AC recommends rejection. Since this decision is not consensual among all reviewers, please let me explain it in more detail.
 
 - The current manuscript does not provide a clear description of the new model in the context of the related works it builds upon (the so-called dynamic filter networks and the spatial transformer networks). The paper spends almost 4 pages with a technical exposition on steerable frames, covering basic material from signal processing. While this might indeed be a good introduction to readers not familiar with the concept of steerable filters, the fact is that it obfuscates the real contributions of the paper. which are not clearly stated. In fact, the model is presented between equations (10) and (11), but it is not clear from these equations what specifically differentiates the dsfn from the other two models -- the reader has to do some digging in order to uncover the differences (which are important). 
 
 Besides this clarity issue, the paper does not offer any insight as to how the 'Pose generating network' Psi is supposed to estimate the pose parameters. Which architecture? what is the underlying estimation problem it is trying to solve, and why do we expect this problem to be efficiently estimated with a neural network? when are the pose parameters uniquely determined? how does this network deal with the aperture effects (i.e. the situations where there is no unicity in determining a specific pose) ?
 Currently, the reader has no access to these questions, which are to some extent at the core of the proposed technique.
Decision: Reject