Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The manuscript describes a novel attentional mechanism applied to fine-grained recognition. 

On the positive side, the approach seems to consistently improve the recognition accuracy of the baseline  (a wide residual net). The approach is also consistently tested on the main fine-grained recognition datasets (the Adience age and gender recognition benchmark, Caltech-UCSD Birds-200-2011, Stanford Dogs, Stanford Cars, and UEC Food-100).

On the negative side, the paper could be better written and motivated.

First, some claimed are made about how the proposed approach "enhances most of the desirable properties from previous approaches” (see pp 1-2) but these claims are never backed up. More generally since the paper focuses on attention, other attentional approaches should be used as benchmarks beyond the WRN baseline. If the authors want to claim that the proposed approach is "more robust to deformation and clutter” then they should design an experiment that shows that this is the case. 

Beyond, the approach seems a little ad hoc. No real rationale is provided for the different mechanisms including the gating etc and certainly no experimental validation is provided to demonstrate the need for these mechanisms. More generally, it is not clear from reading the paper specifically what computational limitation of the CNN is being solved by the proposed attentional mechanism. 

Some of the masks shown in Fig 3 seem rather suspicious and prompt this referee to think that the networks are seriously overfitting to the data. For instance, why would attending to a right ear help in gender recognition? 

The proposed extension adds several hyperparameters (for instance the number K of attention heads). Apologies if I missed it but I am not clear how this was optimized for the experiments reported. In general, the paper could be clearer. For instance, it is not clear from either the text or Fig 2 how H goes from XxYxK for the attention head o XxYxN for the output head.

As a final point, I would say that while some of the criticisms could be addressed in a revision, the improvements seem relatively modest. Given that the focus of the paper is already limited to fine-grained recognition, it seems that the paper would be better suited for a computer vision conference.


Minor point: 

"we incorporate the advantages of visual and biological attention mechanisms” not sure this statement makes much sense. Seems like visual and biological are distinct attributes but visual attention can be biological (or not, I guess) and it is not clear how biological the proposed approach is. Certainly no attempt is made by the authors to connect to biology.

"top-down feed-forward attention mechanism” -> it should be just feed-forward attention. Not clear what "top-down feed-forward” attention could be...
review_title: A limited evaluation and a limited contribution

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes a feed-forward attention mechanism for fine-grained image classification. It is modular and can be added to any convolutional layer, the attention model uses CNN feature activations to find the most informative parts then combine with the original feature map for the final prediction. Experiments show that wide residual net together with this new attention mechanism achieve slightly better performance on several fine-grained image classification tasks.

Strength of this work:
1) It is end-to-end trainable and doesn't require multiple stages, prediction can be done in single feedforward pass.
2) Easy to train and doesn't increase the model size a lot.

Weakness:
1) Both attention depth and attention width are small. The choice of which layer to add this module is unclear to me. 
2) No analysis on using the extra regularization loss actually helps.
3) My main concern is the improvement gain is very small. In Table3, the gain of using the gate module is only 0.1%. It argues that this attention module can be added to any layer but experiments show only 1 layer and 1 attention map already achieve most of the improvement. From Table 4 to Table 7, WRNA compared to WRN only improve ~1% on average.  

review_title: Improvement gain is small

final_score: rating: 6: Marginally above acceptance threshold
review_content: Paper presents an interesting attention mechanism for fine-grained image classification. Introduction states that the method is simple and easy to understand. However, the presentation of the method is bit harder to follow. It is not clear to me if the attention modules are applied over all  pooling layers. How they are combined? 

Why use cross -correlation as the regulariser? Why not much stronger constraint such as orthogonality over elements of M in equation 1? What is the impact of this regularisation?

Why use soft-max in equation 1? One may use a Sigmoid as well? Is it better to use soft-max?

Equation 9 is not entirely clear to me. Undefined notations.

In Table 2, why stop from AD= 2 and AW=2?  What is the performance of AD=1, AW=1 with G? Why not perform this experiment over all 5 datasets? Is this performances, dataset specific?

The method is compared against 5 datasets. Obtained results are quite good.


review_title: Review for A Painless Attention Mechanism for Convolutional Neural Networks 

Meta-review: This paper received borderline reviews. Initially, all reviewers raised a number of concerns (clarity, small improvements, etc). Even after some back and forth discussion, concerns remain, and it's clear that while the idea has potential, another round of reviewing is needed before a decision can be reached. This would be a major revision in a journal. Unfortunately, that is not possible in a conference setting and we must recommend rejection. We recommend the authors to use the feedback to make the manuscript stronger and submit to a future venue. 
Decision: Reject