Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper frames hard visual attention as a Bayesian optimal experimental design (BOED) problem from which the optimal locations to attend to are those with greatest expected reduction in the entropy of classification. With the methodology from BOED literature, this paper approximate the optimal behavior to generate 'near optimal' sequences to partially supervise the learning of hard attention.
Overall, I think the paper is easy to read, the proposed idea is solid and well-motivated, although the actual implementation seems to be quite complicated which involves additional two different models (attentional variational posterior and image completion model). It is hard to tell how the learning of these modules will affect the final performance. 

Additionally I have some specific questions regarding to the paper:
(1) I am a bit confused with Figure 3 and Sec 4. It is mentioned that the mask is concatenated as an additional channel. However, since you already mask the unobserved pixels to zero, why it is still needed?
(2) Following Eq (7), is $g_{AVP}$ trained with randomly sampled glimpses? How can we make sure we can predict the label correctly from a random glimpse? Also, why it is called "attentional" posterior?
(3) How the quality of $r_{img}$ will impact the results? Do we need to train a different $r_{img}$ first every time before testing on a new dataset? Is it possible to only condition on the current image to create glimpse supervisions with $g_{AVP}$ instead of using image completion?
(4) How to inspect if the generated supervisions might hurt the performance? For instance, as the supervisions are generated to approximately maximize the entropy reduction, it is still possible that such attention is not optimal for neural network to predict the final label. 



review_title: Novel idea of learning hard visual attention using Bayesian Optimal Experimental Design

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper presents a learning framework for a hard attention mechanism. The glimpses captured by the attention mechanism are guided by the goal of minimizing output uncertainty for a downstream task such as classification. The authors pose this problem in a probabilistic framework which is based on Bayesian optimal experimental design (BOED). They devise a tractable approximation to the entropy over images and glimpse sequences and search for the glimpse sequences which minimize the output entropy of a recurrent classification model.

With the ability to incorporate supervised glimpse training into the attention mechanism, the authors are able to alleviate the cold start issue which many hard attention models over images suffer from.

While the probabilistic formulation described in the paper is fairly detailed, a figure showing how the attentional variational posterior ($g_{AVP}$) is integrated into the rest of the attention model in Figure 1. Is $g_{AVP}$ trained through some form of model distillation with the attention model?

What types of hand-crafted glimpse sequences are beneficial for the model? CUBs annotated features were not intended to be a glimpse sequence, so how was the order of the annotations chosen?

Can the authors provide a pseudo-code description of the attention algorithm? This would help clarify the pipeline of how $g_{AVP}$ is both trained and integrated into the larger model.


review_title: A well thought out procedure to learn a hard attention mechanism over real images

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper introduces a way to annotate a glimpse sequence for an image. It uses BAYESIAN OPTIMAL EXPERIMENTAL DESIGN to achieve this. Using the obtained annotations, hard attention can be trained with a partially supervised way.

My concerns mainly focus on the motivation and evaluation. Specifically, the authors claim using hard attention to do image classification may enable the use of modern approaches to computer vision in low-power settings such as mobile devices. But from the paper (besides the computation of glimpse sequence is computationally expensive.) at the end, the proposed method needs to run the glimpse network several times, and the structure of the glimpse network is similar to standard classification network, which needs only a single forward pass on the whole image. I'm doubt adopting the proposed approach can truly lead to a low power solution.  Image classification seems like an unsuitable task to show the benefits of hard attention.  Even if we would like to test on image classification, we need to include more baselines, such as standard convnets and compare them to the proposed method in terms of computational complexity and effectiveness. 

Other issues:
  Stochastic image completion involves creating an empirical image distribution with 1.5 million images. Will this lead to an unfair comparison or data leakage?
  It's better to include standard datasets for image classification such as ImageNet, CIFAR, stc. 
  It's interesting to see whether a hard attention method trained with REINFORCE will obtain a similar glimpse sequence and the proposed annotation method.
review_title: not well motivated

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper trains hard attention for image classification. The network is partially supervised by attention locations proposed to maximally reduce the entropy of the image label distribution. To propose these locations, the method needs an already trained image classifier conditioned on glimpses and their locations. Additionally, the method needs a generator of images, conditioned on the glimpses and their locations. This generator is approximated by searching a set of 1.5 million pre-generated images for close matches.

It is visible that the paper required a lot of work. Unfortunately, the method is not very practical and the method is not better than simpler alternatives.

Pros:
- The paper is well written.

Main cons:
1) The greedy minimization of the expected posterior entropy is not optimal, if additional glimpses can be done in the next steps.
2) The searching in the pre-generated dataset of images is not practical. It would work worse on more diverse images. The method was tested only on faces and birds.
3) If the aim is to speed-up training of new networks, a simpler alternative is to distill the attention locations and classification outputs from a pretrained Recurrent Attention Model (RAM+).
review_title: Complicated and not optimal

Meta-review: This paper proposes a new and unusual way of training hard attention mechanisms in vision models. Instead of training with reinforcement learning (as is typical), the authors develop a procedure for generating "glimpse sequences" that can be effectively used as supervision. Models trained in this way produce qualitatively "better" glimpse sequences, higher accuracy, and converge in fewer steps. There was some disagreement and discussion about the merits of the paper. Overall, there were some major concerns:
- The process for obtaining glimpse sequences is very computationally expensive. The authors argue that this cost can be amortized because the same glimpse sequences can be computed once for a given dataset and reused. However, there was limited real-world motivation for this setup, apart from mentioning neural architecture search (a niche method that is not widely used, and probably has never been used to develop hard-attention models).
- The method relies on a "convincing" generative model for a given dataset. This limits experiments to simple datasets with unrealistically-constrained visual structure. The authors point out that as generative models get better, their method could be applied to more realistic datasets, but as it stands the experimental validation is correspondingly weak.
- The improvement in performance is not huge - the "WSRAM" baseline outperforms the use of "near-optimal" glimpse sequences. While it is certainly true that the proposed method converges faster, the fact that the proposed method requires such an expensive preprocessing step downgrades this benefit significantly
- While the authors provided insightful distillation-based baselines in the rebuttal, it remains to be seen whether simple distillation from stronger RAM models (e.g. WSRAM which outperforms the proposed method) could be made to work better/more efficiently.
These factors lead to a reject decision overall.
Decision: Reject