Near-Optimal Glimpse Sequences for Training Hard Attention Neural Networks
Authors: William Harvey
Michael Teng
Frank Wood
Venue:  (2021)
Keywords: attention
hard attention
variational inference
bayesian optimal experimental design

Abstract: Hard visual attention is a promising approach to reduce the computational burden of modern computer vision methodologies. Hard attention mechanisms are typically non-differentiable. They can be trained with reinforcement learning but the high-variance training this entails hinders more widespread application. We show how hard attention for image classification can be framed as a Bayesian optimal experimental design (BOED) problem. From this perspective, the optimal locations to attend to are those which provide the greatest expected reduction in the entropy of the classification distribution. We introduce methodology from the BOED literature to approximate this optimal behaviour, and use it to generate `near-optimal' sequences of attention locations. We then show how to use such sequences to partially supervise, and therefore speed up, the training of a hard attention mechanism. Although generating these sequences is computationally expensive, they can be reused by any other networks later trained on the same task.

Under review as a conference paper at ICLR 2021
NEAR-OPTIMAL GLIMPSE SEQUENCES FOR IMPROVED HARD ATTENTION NEURAL NETWORK TRAINING
Anonymous authors Paper under double-blind review
ABSTRACT
Hard visual attention is promising approach to reduce the computational burden of modern computer vision methodologies. Hard attention mechanisms are typically non-differentiable They can trained with reinforcement learning but the high- variance training this entails hinders more widespread application. We show how hard attention for image classification can be framed as a Bayesian optimal experimental design (BOED) problem. From this perspective, the optimal locations to attend to are those which provide the greatest expected reduction in the entropy of the classification distribution. We introduce methodology from the BOED literature to approximate this optimal behaviour, and use it to generate 'near- optimal' sequences of attention locations. We then show how use such sequences to partially supervise, and therefore speed up, the training of hard attention mechanism. Although generating these sequences computationally expensive, they can be reused by any other networks later trained on the same task.
INTRODUCTION
Attention can be defined as the "allocation of limited cognitive processing resources" (Anderson, 2005). In humans the density of photoreceptors varies across the retina. It is much greater in the centre (Bear et al., 2007) and covers an approximately 210 degree field of view (Traquair, 1949). This means that the visual system is a limited resource with respect to observing the environment and that it must be allocated, or controlled, by some attention mechanism. We refer to this kind of controlled allocation of limited sensor resources "hard" attention. This is in contrast with "soft" attention, the controlled application of limited computational resources to full sensory input. Hard attention can solve certain tasks using orders of magnitude less sensor bandwidth and computation than the alternatives (Katharopoulos & Fleuret, 2019; Rensink, 2000). It therefore may enable the of modern approaches to computer vision in low-power settings such mobile devices.
This paper focuses on the application of hard attention in image classification. Our model of attention (shown in Fig. 1) is follows: recurrent neural network (RNN) is given 7 steps to classify some unchanging input image. Before each step, the RNN outputs the coordinates of a pixel the image. patch of the image centered around this pixel is then fed into the RNN. We call this image patch glimpse, and the coordinates glimpse location. As such, the RNN controls its input by selecting each glimpse location, and this decision can be based on previous glimpses. After T' steps, the RNN's hidden state is mapped to classification output. As with most artificial hard attention mechanisms (Mnih al., 2014: Ba et al., 2014). this output not differentiable with respect the sequence of glimpse locations selected. This makes training with standard gradient backpropagation impossible, and so high variance gradient estimators such as REINFORCE (Williams, 1992) are commonly used instead (Mnih et al., 2014; Ba et al., 2014). The resulting noisy gradient estimates make training difficult, especially for large T.
In order to improve hard attention training, we take inspiration from neuroscience literature which suggests that visual attention is directed so as to maximally reduce entropy in an agent's 's world model (Bruce& & Tsotsos, 2009; Itti & Baldi, 2009; Schwartenbeck et al., 2013; Feldman & Friston, 2010). There a corresponding mathematical formulation of such an objective, namely Bayesian optimal experimental design (BOED) (Chaloner & Verdinelli, 1995). BOED tackles the problem

---

Under review as a conference paper at ICLR 2021
ho
h
h2
hT
Is 1o(0y11h
go (h)
co(1lyy,,11 41)
Ls (1 (1
RNN
Glimpse emb.
Y2
YT
Location net.
I
I
Classifier net.
Figure I: The hard attention network architecture we consider, consisting of an RNN core (yellow), a location network (light blue), a glimpse embedder (dark blue), and classifier (red), hy is the RNN hidden state after steps. The network outputs distributions over where to attend (44) at each time step, and over the class label (f) after T steps.
of designing an experiment to maximally reduce uncertainty in some unknown variable. When classifying an image with hard visual attention, the "experiment" is the process of taking a glimpse; the "design" is the glimpse location; and the unknown variable is the class label. In general, BOED is applicable only when probabilistic model of the experiment exists. This could be, for exam- ple, a prior distribution over the class label and a generative model for the observed image patch conditioned the class label and glimpse location. We leverage generative adversarial networks (GANs) (Goodfellow et al., 2014) to provide such a model.
We methodology from BOED to introduce the following training procedure for hard attention networks, which we call partial supervision by near-optimal glimpse sequences (PS-NOGS).
1. We assume that we are given an image classification task and corresponding labelled dataset. Then, for subset of the training images, we determine an approximately optimal (in the BOED sense) glimpse location for hard attention network to attend at each time step. We refer to the resulting sequences of glimpse locations as near-optimal glimpse sequences. Section 4 describes our novel method to generate them.
2. We use these near-optimal glimpse sequences as an additional supervision signal for training hard attention network. Section 5 introduces novel training objective for this.
We empirically investigate the performance of PS-NOGS and find that it leads to faster training than our baselines, and qualitatively different behaviour with competitive accuracy. We validate the use of BOED to generate glimpse sequences through comparisons with supervision both by hand-crafted glimpse sequences, and by glimpse sequences sampled from trained hard attention network.
2 HARD ATTENTION
Given an image. I, we consider the task of inferring its label, 0. We use architecture based on that of Mnih et al. (2014), shown Fig. runs for a fixed number of steps, T. At each step I, the RNN samples glimpse location, 4, from distribution conditioned hidden state. A glimpse. in the form of a contiguous square of pixels, is extracted from the image at this location. We denote this y, = fiowes (I, lt). An embedding of y, and 1, is then input the RNN. After T glimpses, the network outputs a classification distribution qo (0) yI:r, !rr),, where & are the learnable network parameters. Mnih al. (2014) use glimpses consisting of three image patches at different resolutions, but the architectures are otherwise identical. As it directly processes only a fraction of an image, this architecture is suited to low-power scenarios such as use on mobile devices.
previous glimpses via the RNN's
During optimisation, gradients cannot be computed by simple backpropagation since Sroves is non- differentiable. An alternative, taken by Mnih et al. (2014) and others the literature (Ba et al., 2014; Sermanet et al., 2014). is to obtain high varianar gradient estimates using REINFORCE (Williams, 1992). Although these are unbiased, their high variance has made scaling beyond simple problems such as digit classification (Netzer et al., 201 1) challenging. Section 7 describes alternatives (Ba et al., 2015; Lawson et al., 2018) to training with REINFORCE, but similar problems with scalability exist. This has led many studies to focus on easing the learning task by altering the architecture: e.g.. to process a downsampled image before selecting glimpse locations (Ba et al., 2014; Sermanet et al., 2014; Katharopoulos & Fleuret, 2019). We summarise these innovations in Section 7 but they tend to be less suitable for low-pont low-power computation. We therefore believe that improved training of the architecture in Fig. is important research problem, and is the focus of this paper.
2

---

Under review as a conference paper at ICLR 2021
t 2
f == 3
f
Glimpses EPE vs 4
x
0.6
1 - (4,)
-044
0.2
0.0
Figure 2: A near-optimal glimpse sequence being generated for the task of inferring the attribute 'Male'. Top row: A heatmap of estimated expected posterior entropy for each possible next glimpse location 4. The red cross marks the minimum, which is chosen the next glimpse location. Bottom row: Observed parts of the image after taking each glimpse.
BAYESIAN OPTIMAL EXPERIMENTAL DESIGN
Designing an experiment to be maximally informative a fundamental problem that applies as much to tuning the parameters of political survey (Warwick & Lininger, 1975) as to deciding where to direct attention to answer a query. BOED (Chaloner & Verdinelli, 1995) provides unifying framework for this by allowing formal comparison of possible experiments under problem-specific prior knowledge. Consider selecting the design, I, of an experiment to infer some unknown parameter, 0. For example, may be the median lethal dose of a drug, and / the doses of this drug given to various groups of rats (Chaloner & Verdinelli, 1995). Alternatively, as we consider in this paper, is the class label of an image and determines which part of the image we observe. The experiment results in measurement of y p(y [1, 0). Following the previous examples, could be the number of rats which die each group or the observed pixel values. Given a prior distribution over 0 and knowledge of p(y(7, f), we can use the measurement to infer a posterior distribution over 0 using p(y(),0)(())
Bayes' rule: p(0ly. 1) I (.))((()))) The aim of our experiment is infer 6, and so a well designed experiment will reduce the uncertainty about 0 by as much as possible. The uncertainty after the experiment can be quantified by the Shannon entropy in the posterior,
H [p( y, 2)] = Eppell..) [- poly,
To maximally reduce the uncertainty, we wish to select to minimise this posterior entropy. However, the design of the experiment must be chosen before y measured and so cannot evaluate the posterior entropy exactly. Instead, we minimise an expectation of it over p(y[() = Epco) ((y)), the marginal distribution This is the expected posterior entropy, or EPE.
EPE(() Eptyi)) Hl [p[Oly,
Above, we considered the case of selecting a one-off design for an experiment, such as taking a single glimpse. For the case where sequence of glimpses be taken, we need sequential experimental design. In this scenario, the choice of design 4 can be informed by the designs and outcomes of previous experiments, 11.t-1 and Y1.t-1- The marginal distribution over outcomes is therefore (ieellaa...--11 rather than p()))()). Similarly, the posterior after observing Y: is p(0 V1.)) Therefore, in the sequential case which we consider throughout the rest of the paper, we greedily minimise the following form of the EPE on each iteration:
EPE (4) Epty 1dnd) [H (1((0ly1..
(3)
To summarise, sequential BOED involves, at each time t, selecting 4 = arg min, E (4)
and then performing the experiment with design 4 to observe y..
4 GENERATING NEAR-OPTIMAL. GLIMPSE SEQUENCES
Role of BOED pipeline To reiterate the outline of our method, first annotate portion of the training data with glimpse sequences, and then in the second stage use these to speed up the training of hard attention mechanism. This section details our BOED pipeline for the first stage.
3

---

Under review as a conference paper at ICLR 2021
I
mask(I1d)
CNN
gAVP (avr(0ly1a,41
cmbedding(Z 11:1)
mask (11:)
Figure 3: Attentional variational posterior CNN. An RGB image and 11 are processed to create an embedding the information gained from glimpses to This embedding is fed into an image classifier to obtain approximation of p(0)y1.,44..)
EPE estimator BOED requires probabilistic model of the measurements and parameters we wish to infer. That is, we need to define p(0, (i=af1e) for any 11: To do so in the visual attention setting, we first define p(0, Z] to be the intractable joint distribution over labels and images from which our training and test data originate. To be consistent with our definition in Section 2 of as a deterministic function of and we then define p(y;|Z, 4,) to be Dirac-delta distribution on ftoea (I,4). 4)) The joint distribution is then
p(0, y1al(1:) = p(0, II PIY.
(4)
Given this joint distribution, EPEy dan-1 1 is well defined but intractable in general. We therefore consider how approximate To simplify our method for doing so, first rearrange the expression given in Eq. (3) so that the expectation is over I rather than Yt. Taking advantage of the fact that y; deterministic function of and I, allows it to be rewritten as follows (proof in the appendix). Defining flovea (I, l1:t) = (frowe (I, 11,,..... flovea (I, 4)).
EPEy (Ie) = E (H [p(0) Stoea (I, h1)))
(5)
Given this form of the expected posterior entropy, we can approximate it if we can leverage the dataset to obtain:
learned attentional variational posterior, ( ha) and stochastic image completion distribution rimg ( 11:-1) yIayaa-1, 1a-1).
We expand on the form of each of these approximations later in this section. First, combining them with Eq. (5) and using Monte Carlo estimate of the expectation yields our estimator for the EPE:
N
EPByLe-AdE- (1))
RJ
H ([)Ap((0) Stovca (I(1), (1)) 41:)
(6)
N
n=1
with [(1), ,...,, ,(M) Timg (I
Overview of BOED pipeline We select with a grid search. That is, denoting the set of allowed values of L., compute our approximation of EPE EPEE 11 1.1. (4,) for all 4 L. We then select the value of le for which this is least. To do so, our full BOED pipeline is as follows.
1. Sample I()) .0....,IK" (NN ~ Timg 1 11-11)
2. For each 4 E L. approximate the expected posterior entropy with Eq. (6).
3. Select the value of I, for which this approximation is least.
1,.... T yields near-optimal glimpse sequence l for image I. Figure 2 shows an example of this process. We must do this for all images in some subset of a dataset to be able to partially supervise hard attention training as described in Section 5. We now describe the form of gavp (the attentional variational posterior) and rimg (stochastic image completion).
Repeating these steps for
Attentional variational posterior In this section we introduce our novel approach for efficiently approximating the intractable posterior p(O (1))))) We train convolutional neural network (CNN) to map from a sequence of glimpses N1:t, and their locations, 41s, to gavp (0 (11,, I1:t), an approximation of this posterior. We call this the attentional variational posterior CNN (AVP-CNI) To allow single CNN to cope with varying Yıa- his and even varying t, we embed its input as shown in Fig. 3. Essentially, h is used create image-sized mask which is I for observed
4

---

Under review as a conference paper at ICLR 2021
pixels and 0 for unobserved pixels. Elementwise multiplication of this mask with the imput image sets unobserved pixels to zero. The mask is then concatenated as an additional channel. This embedding naturally maintains spatial information while enforcing invariance to permutations of the glimpse sequence. We use a Densenet- 121 (Huang et al., 2017) CNN architecture (pretrained on ImageNet (Deng et al., 2009)) map from this embedding to vector of probabilities representing gAvP. We train the network to minimise the KL divergence between its output and p(Oly1:,, (1x). That (1:e)]lgave (0ly.:t Ltt (1:))) To ensure that gAvP is close for all t. 41 and Yia. the is, D (p(Oly 1:t, l.t,
loss used is an expectation of this KL divergence over p(y1:s M1 (u(t, 11:1). We factorise u(t, 11:0) as ((()) IT= v((:) where, so that all times and glimpse locations are weighted equally in the loss, u(!) is a uniform distribution over I.....? T' and u(4, is a uniform distribution over all image locations. Denoting the network parameters X, the gradient of this loss is
i x = Ep(P yiu (t. ax
[-
ax log (1.)
This gradient is the same as that of a cross-entropy loss on data sampled from p(0, Vi:ella)) 12x). and can be approximated by Monte Carlo estimate.
Our approximation of the EPE in Eq. (6) involves the entropy of JAvP. Since JAVP is categorical distribution, this is simply computed analytically. This amortised approximation of the posterior entropy is inspired by Foster et al. (2019), but has two important differences to their estimator:
Foster et al. learn mapping from y: to g( h1:), sharing information between "nearby" samples of Y: to reduce the computational cost of the experimental design. Our AVP- CNN takes this amortization further by learning single mapping from t, (1. and y1:t to gAvp(0) y 1ss, 1z), which yields significant further efficiency gains in our setting.
Whereas we approximate H[pp with H[awe] = Egwwel [ log gavel, Foster et al. use - 9]- This provides an upper bound on H(p but is not applicable in case as we cannot sample from p(0)y1:e: (1:)). Both approximations are exact when JAVP p.
Stochastic image completion We considered numerous ways to form Timg [Z]y):r- 11 (1==1) includ- ing inpainting (Pathak et al., 2016; Isola et al., 2017) and Markov chain Monte Carlo in a generative model. Future research in generative modelling may provide alternatives to this component of our method but, for now, we choose to represent Ting using a technique we developed based on image retrieval (Jégou et al., 2010). Of the methods we considered, this gave the best trade-off between speed and sample quality. It involves creating an empirical image distribution with 1.5 million images for each experiment using GANs with publicly available pre-trained weights (StyleGAN (Karras et al., 2018) for CelebA-HO and FineGAN (Singh et al., 2019) for Caltech-UCSD Birds). We note that the use of pre-trained models makes test leakage possible but verify in Appendix C.4 that this is unlikely to impact our results. During sampling, the database is searched for images that 'match' the previous glimpses (y1:t-1 and (1.0-1). )). How well these glimpses match a database image, I', is measured by the squared distance in pixel space at glimpse locations: Σ lly: - frowea (I', 44) 112. This distance detines a probability distribution over the images in the database. To reduce computation, we first compare approximations of the observed parts of each image using principal component analy- sis (Jolliffe, 2011), I), and compute exact distances only when these are close. The overall procedure to sample from Timg corresponds to importance sampling (Arulampalam et al., 2002) in model where p(y, [I, 1.) is relaxed from a Dirac-delta distribution to a Gaussian. See the appendix for details.
TRAINING WITH PARTIAL SUPERVISION
The previous section describes how to annotate an image with a near-optimal sequence of glimpse locations for particular image classification task. This section assumes that these, or other forms of glimpse sequence (e.g. the handcrafted glimpse sequences in Section 6), exist for all, or some, images in a dataset. These can then be used to partially supervise the training of hard attention mechanism on this dataset. We refer to glimpse sequences used in this way as supervision sequences. We use separate losses for supervised (i.e. annotated with both class label and sequence of glimpse locations) and unsupervised (i.e. annotated with a class label but not glimpse locations) examples. By minimising the sum of these losses, procedure can be viewed as maximising the joint log-likelihon likelihood of the class labels and supervision sequences. To be precise, let go (8', , (T' be a network's 's joint distribution over the chosen glimpse locations and predicted class label on image
5

---

Under review as a conference paper at ICLR 2021
RAM (baseline)
PS-NOGS (ours)
t
t -
2
t= 3
4
5
1
2
t
3
t
4
t
5
High
cheekbones
Male
Mouth
slightly open
0.5
20
100
% of runs observing each pixel
Figure 4: Comparison of glimpse locations chosen by RAM and PS-NOGS on the CelebA-HQ test set for three classification tasks. For each t (1. 3, 4, 5). we show an image where each pixel's colour corresponds to how often it was observed at this time step during testing. The outlines are produced by averaging outputs from a face detector across the dataset. For t = 1, each network learns single location which it attends to on every test image. This is expected behaviour the first location is chosen before taking any glimpses. and therefore before being able to condition on the image. RAM appears to then fail to learn to direct the later glimpses, attending almost uniformly across the image. In contrast, PS-NOGS distributes these glimpses broadly over the salient regions.
I'. Let (o( be the marginalisation of this distribution over li... We maximise a lower bound on
supervised objective
unsupervised objective
L
=
log (0', (i)+
log qo (0'Iz)
(8)
sEsu.
*Eunsup.
where 'sup' is the set of training indices with supervision sequences, and 'unsup' is the remainder. When running on unsupervised examples, we follow Mnih et al. (2014) 4) and train the location network with REINFORCE estimate of the gradient of the accuracy, using learned baseline to reduce the variance of this estimate. Mcanwhile, all other network components are trained to maximise the log-likelihood of the class labels (i.e. minimise a cross-entropy loss). Ba et al. (2014) noted that this maximises a lower bound on the unsupervised objective in Eq. (8). For examples with supervision sequences, the supervised objective in Eq. (8) is maximised by gradient backpropagation. The loss is computed by running the network with its glimpse locations fixed to those in the supervision sequence. The location network is updated to maximise the probability of outputting these locations while, as for unsupervised examples, the other network modules trained to maximise the likelihood of the class labels. Minibatches can contain both supervised and unsupervised examples, with gradients computed simultaneuosly. We emphasise that supervision sequences are used throughout training. Although is common to attenuate such supervision signals so that an end-to-end loss eventually dominates, preliminary experiments showed no improvements from this.
6 EXPERIMENTS AND RESULTS
Datasets and network architectures We test our approach on CelebA-HQ (Karras et al., 2017) and a cropped variant of Caltech-UCSD Birds (CUB) (Wah et al., 2011). For both, GANs exist which satisfy the requirement to have a convincing generative model (Karras et al., 2018; Singh et al., 2019). The RNN is GRU (Cho et al., 2014) and we use a simple classifier and location network architecture (see Appendix B for details). For both datasets, we use T = The -ptaset-speciffc details are as follows: (1) CelebA-HQ Our experiments tackle 40 different binary classification tasks, corresponding to the 40 labelled attributes. We resize the images to 224 x 224 and use training, validation, and test sets of 27 000, 500, and 2500 images respectively. We use 16 x 16 pixel glimpses, with 50 x 50 grid of allowed glimpse locations. The glimpse network has two convolutional layers followed by linear layer. (2) CUB We perform 200-way classification of bird species. We crop the images using the provided bounding boxes and resize them to 128 x 128. Cropping is necessary because good generative models do not exist for the uncropped dataset, but there is still considerable

---

Under review as a conference paper at ICLR 2021
Training steps (+ S.D.)
1 1x 10
RAM WSRAM
PS-RAM
PS-NOGS (ours)
x 104
Bhurry Bald
Gontee Open
Smiling No Beard Mustacka sestcche Shadkrw
Pale Skin
Busby Eys brows
E Eyeeglasses
High Chee meethooee
Bangs
Arched Eyel brroww
Gray Hair
Wearing Lipstick
Sidehurd nu
Bugs Under Eyres
Young
Blond Hair
Bis Nose
Black Hair
Wearing Hat
Heavy Makeuu
Receding Hairlin
Wavy Hair
Chubby Chubby
Double Chin
Wearing Barring
A Atrrcttive
Wearing Neckole
Rony Cheako
Male
Brown Hair
Straight Hair
Wearing Neodan Neddacss daee
Narrow Eves
Oval Face
Fointy Nowe
Big Lipe
Mouth Srighly
5 5 o'Olockk
Figure 5: Number of training iterations for each CelebA-HQ attribute before a validation cross- entropy loss within 0.01 of the best is achieved. On average, PS-NOGS trains almost 7x faster than RAM, the fastest method without supervision sequences. PS-NOGS also exhibits greatly reduced variance in the training time. Attributes are sorted by the mean training time.
variation in pose after cropping. We use 5120 training images, a validation set of 874 and a test set of 5751 (having removed 43 images also found in ImageNet). We use 32 x 32 pixel glimpses and a 12 x 12 grid of allowed glimpse locations so that adjacent locations are pixels apart. The glimpse network is the first 12 convolutional layers of a VGG pretrained ImageNet (Simonyan & Zisserman, 2014; Deng et al., 2009).
We create 600 oear-optimal glimpse sequences for each of the 40 CelebA-HQ classification tasks, and 1000 for CUB. This a one-off computation that need only be done once for a particular task. It took 20 GPU-hours for CUB, and 10 GPU days for each CelebA-HQ task. We will publicly release these sequences along with our code, allowing them to be re-used by anyone to speed up the training of hard attention networks these tasks.
BOED
Baselines All methods we compare use the same neural architecture; only the training algo- rithm is varied. For each experiment we compare against the RAM algorithm (Mnih et al., 2014), which is equivalent to the special case of our partially supervised objective with zero su- pervision sequences. We also compare against training with wake-sleep (WSRAM) (Ba al.. 2015), which we describe in Section 7. Furthermore, we consider training with supervision se- quences created using two baseline methods. One is PS-RAM, which involves training hard attention network with RAM and then sampling supervision sequences from the learned pol- icy. The second, which we use on CUB, is to create "hand-crafted glimpse sequences" (PS- HGS). These are designed, using CUB's hand-annotated features, to attend to the beak, eye, forehead, belly and feet (in that order). If any of these are obscured, they are replaced by randomly selected visible body part. For both partially supervised baselines, we use the same
Validation accuracy (% S.D.)
number of supervision sequences with PS- NOGS. We have not observed significant perfor- mance improvements from varying this number.
55
Partial-supervision for CelebA-HQ In Fig. 5, plot the number of iterations until conver- gence on each CelebA-HO classification task.
50
On almost all tasks, the methods which involve partial supervision (PS-RAM and PS-NOGS) are faster than those which do not. Table 1 cor- roborates this finding, showing that PS-NOGS trains an average of 6.6x faster on CelebA-HQ than RAM+, the fastest unsupervised method. Furthermore, the test accuracy for the partially supervised methods is competitive with the unsu- pervised baselines. Fig. 4 compares the learned attention policies for several tasks, with the re-
45
RAM
PS-HGS
RAM+
PS-RAM+
WSRAM
PS-NOGS
40
0
25
50
100
150 200 250
Epoch
Figure 6: CUB validation accuracy over training.
7

---

Under review as a conference paper at ICLR 2021
Table 1: Results summary
CelebA-HQ (avg.)
CUB
Method
Iterations
Accuracy (%)
Iterations
Accuracy (%)
RAM/ RAM+
25 100
87.1
5690
53.7
WSRAM
40700
88.1
800
55.7
PS-HGS
1790
50.5
PS-RAM / PS-RAM+
4750
87.0
1980
53.2
PS-NOGS (ours)
3820
87.0
3530
54.4
mainder in the appendix. Unlike RAM, PS-NOGS appears to learn a suitable policy at every time step. These intuitively reasonable policies may explain why PS-NOGS trains 20% faster than PS-RAM.
Surtial-superviiion for CUB On CUB, pretraining stage is necessary to achieve high accuracy. We therefore pretrain the classifier, RNN, and glimpse network with glimpse locations sampled independently at each time step from either a uniform distribution (in the case of RAM) or from heuristic which assigns higher probability to more salient locations, as estimated using the AVP-CNN (RAM+ and all others). See the appendix for details. Figure 6 shows the validation accuracy of each method throughout training. As seen on CelebA-HQ. all methods with partial supervision train faster than any without; Table I show the number of iterations on CUB before achieving validation accuracy within 1% of the highest. Of the methods with partial supervision, PS-HGS quickly saturates and achieves the lowest test accuracy of the methods accuracy than RAM++ whose policy it is trained to imitate. PS-NOGS achieves the highest accuracy of the partially supervised methods, indicating that near-optimal glimpse sequences more suitable for training than any of the supervision sequences used by our baselines. However, we note that after sufficiently many training epochs (approximately 100), WSRAM achieves higher validation accuracy than PS-NOGS, and correspondingly achieves higher test accuracy. This may be due to some bias introduced by approximations the generation of near-optimal glimpse sequences, which will not go away with further training.
compare. PS-RAM+ achieves slightly lower
RELATED WORK
Variational approaches A notable body of work (Ba et al., 2015; Lawson et al., 2018; Shankar
& Sarawagi, 2018) frames the glimpse locations as latent variables and trains an inference network to approximate qo (41.T (0, I). This allows for better estimates of the objective log q (0 which is marginalised over glimpse locations. Our comparison against WSRAM (Ba et al., 2015) indicates that using supervision sequences allows faster training than these variational techniques alone. Further, supervision sequences could be used in conjunction with these techniques (Teng et al., 2020).
Hard attention architectures Elsayed et al. (2019) recently demonstrated hard attention network which achieved accuracy on ImageNet (Deng al., 2009) close to that of CNNs which use the whole image. However, their approach neccesitates running convolutional network on the entire image select glimpse locations. As such, they advertise improvements in interpretability rather than computational efficiency. Sermanet et al. (2014) train a hard attention architecture with REINFORCE to achieve ofte-offthheart accuracy on the Stanford Dogs dataset. In addition accessing the full image in low resolution at the start, they use large glimpses (multiple 96 x 96 pixel patches at different resolutions) to effectively solve the task in a single step. This avoids problems resulting from learning long sequences with REINFORCE but also rules out the computational gains possible with smaller glimpses. Katharopoulos & Fleuret (2019) proposed form of hard attention where, after processing the downsampled image, multiple glimpses are sampled and processed simultancuosly. This is again incompatible with a low-power setting where we cannot afford to operate on the full image.
Supervised attention We are not alone in providing supervision targets for attention mechanism. This is common for soft attention in visual question answering, where targets have been created by human subjects, either with gaze- tracking (Yu et al., 2017) explicit annotation (Das et al., 2017). Either way is expensive and dataset-specifee (Das et al., 2017; Gan et al., 2017). Recent work has reduced this cost by e.g. extrapolating supervision signals from annotated datasets to other, related,
8

---

Under review as a conference paper at ICLR 2021
datasets (Qiao et al., 2018), using existing segmentations to speed up annotation (Gan et al., 2017). Even so, considerable human effort is required. PS-NOGS automates this for image classification.
DISCUSSION AND CONCLUSION
We have demonstrated a novel BOED pipeline for generating near-optimal sequences of glimpse locations. We also introduced a partially supervised training objective which uses such supervision signal to speed up the training of a hard attention mechanism. By investing up-front computation in creating near-optimal glimpse sequences for supervision, this speed up be achieved along with comparable final accuracy. Since we release the near-optimal glimpse sequences we generated, faster training and experimentation these tasks is available to the public without the cost of generating new sequences. Our work could also have applications in neural architecture search, where this cost be amortised over as many as 12 800 (Zoph & Le, 2016) training runs with different architectures. Finally, our framework could also be extended to attention tasks such as question answering where the latent variable of interest is richly structured, or scaled more complex images with structured image completion models (Dai et al., 2018).
REFERENCES
John R Anderson. Cognitive psychology and its implications. Macmillan, 2005.
M Sanjeev Arulampalam, Simon Maskell, Neil Gordon, and Tim Clapp. A tutorial on particle filters for online nonlinearhoon-Causiian Bayesian tracking. IEEE Transactions on signal processing, 50 (2):174 188, 2002.
Jimmy Ba, Volodymyr Mnih, and Koray Kavukcuoglu. Multiple object recognition with visual attention. arXiv preprint arXiv:1412.7733 1412.7755, 2014.
Jimmy Ba, Russ R Salakhutdinov, Roger B Grosse, and Brendan J Frey. Learning wake-sleep recurrent attention models. In Advances in Neural Information Processing Systems, pp. 2593-2601, 2015.
Mark Bear, Barry Connors, and Michael A Paradiso. Neuroscience, volume Lippincott Williams & Wilkins, 2007.
Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and Noah D. Goodman. Pyro: Deep universal probabilistic programming. arXiv preprint arXiv:/810.099: 180.09938, 2018.
Neil DB Bruce and John K Tsotsos. Saliency, attention, and visual search: An information theoretic approach. Journal of vision, 9():5-55, 2009.
Kathryn Chaloner and Isabella Verdinelli. Bayesian experimental design: review. Science, pp. 273-304, 1995.
Statistical
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406 1078, 2014.
Angela Dai, Daniel Ritchie, Martin Bokeloh, Scott Reed, Jürgen Sturm, and Matthias Nießner. Scan- complete: Large-scale scene completion and semantic segmentation for 3d scans. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 4578 4587, 2018.
Abhishek Das, Harsh Agrawal, Larry Zitnick, Devi Parikh, and Dhruv Batra. Human attention in visual question answering: Do humans and deep networks look at the same regions? Computer Vision and Image Understanding, 163:90-100, 2017.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPRO9, 2009.
Simon Duane, Anthony D Kennedy, Brian J Pendleton, and Duncan Roweth. Hybrid Monte Carlo. Physies letters B, 195(2):216-222, 1987.

---

Under review as a conference paper at ICLR 2021
Gamaleldin F Elsayed, Simon Kornblihh, and Quoe V Le. Saccader: Improving accuracy of hard attention models for vision. arXiv preprint arXiv:1908.0766 2019.
Harriet Feldman and Karl Friston. Attention, uncertainty, and free-energy. Frontiers in human neuroscience, 4:215, 2010.
Adam Foster, Martin Jankowiak, Eli Bingham, Paul Horsfall, Yee Whye Teh, Tom Rainforth, and Noah Goodman. Variational estimators for Bayesian optimal experimental design. arXiv preprint arXiv:1903.054800 2019.
Chuang Gan, Yandong Li, Haoxiang Li, Chen Sun, and Boqing Gong. VQS: Linking segmentations to questions and answers for supervised attention in VQA and inestion-focusedd semantic segmen- tation. In Proceedings of the IEEE International Conference on Computer Vision, pp. 181 1-100, 2017.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural informa- 2014.
tion processing systems. pp.
Matthew D Hoffman and Andrew Gelman. The No-U-Turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, 15(1)11933-1233 2014.
Gao Huang. Zhuang Liu, Laurens Van Der Maaten, and Kilian Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
2017.
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference computer vision and paltern recognition, pp. 112--1134, 2017.
Laurent Itti and Pierre Baldi. Bayesian surprise attracts human attention. Vision research, 49(10): 1295-1306, 2009.
Hervé Jégou, Matthijs Douze, and Cordelia Schmid. Improving bag-of-features for large scale image search. International journal of computer vision, 87(3):316-335 13636,6366666 2010.
Ian Jolliffe. Principal component analysis. Springer, 2011.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint 170. 10196, 2017.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. arXiv preprint arXiv:1812.04900 2018.
Angelos Katharopoulos and Francois Fleuret. Processing megapixel images with deep attention- sampling models. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 3282-3291, Long Beach, California, USA, 09-15 Jun 2019. PMLR. URL http:/ ppoceedings mlr press / /97 20007//aahaae katthaaopoollssooo 9a.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 141.::1412.6980, 2014.
Dieterich Lawson. Chung-Cherg Chiu, George Tucker, Colin Raffel, Kevin Swersky, and Navdeep Jaitly. Learning hard alignments with variational inference. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5799-5803. IEEF, 2018.
Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shufflenet v2: Practical guidelines for efficient enn architecture design. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 116-131. 2018.
Volodymyr Mnih, Nicolas Heess, Alex Graves, et al. Recurrent models of visual attention. In Advances in neural information processing systems, pp. 2204-2212, 2014.
10

---

Under review as a conference paper at ICLR 2021
Kamyar Nazeri, Eric Ng. Tony Joseph, Faisal Qureshi, and Mehran Ebrahimi. Edgeconnect: Genera- tive image impainting with adversarial edge learning. arXiv preprint arXiv:1901.00212 2019.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011, 2011.
Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2536-2544, 2016.
Tingting Qiao, Jianfeng Dong, and Duanqing Xu. Exploring human-like attention supervision in visual question answering. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
Ronald A Rensink. The dynamic representation of scenes. Visual cognition, 7(1-3):17-42, 2000.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234-241. Springer, 2015.
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang Liang-Chieh Chen. Mo- bilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510-4520, 2018.
Philipp Schwartenbeck, Thomas FitzGerald, Ray Dolan, and Karl Friston. Exploration, novelty, surprise, and free energy minimization. Frontiers in psychology, 4:710, 2013.
Pierre Sermanet, Andrea Frome, and Esteban Real. Attention for fine-grained categorization. arXiv preprint arXiv:1412.7054: 2014.
Shiv Shankar and Sunita Sarawagi. Posterior attention models for sequence to sequence learning. In International Conference Learning Representations, 2018.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint 1rXiv:1409.15535 2014.
Krishna Kumar Singh, Utkarsh Ojha, and Yong Jae Lee. Finegan: Unsupervised hierarchical disentanglement for fine-grained object generation and discovery. Proceedings of the IEEE Conference Computer Vision and Putlern Recognition, pp. 6490-6499, 2019.
Michael Teng, Tuan Anh Le, Adam Scibior, and Frank Wood. Semi-supervised sequential generative models. In Conference on Uncertainty in Artificial Intelligence, pp. 649-658- PMLR, 2020.
Harry Moss Traquair. An introduction to clinical perimetry. Mosby, 1949.
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Faltech-UCSD Birds-200-200 Dataset. Technical Report CNS-TR-2011 -020010000 California Institute of Technology, 2011.
Donald Warwick and Charles Lininger. The sample survey: Theory and practice. McGraw-Hill, 1975.
Ronald J Williams. Simple statistical folliig algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229-256, 1992.
Youngjae Yu, Jongwook Choi, Yeonhwa Kim, Kyung Yoo, Sang-Hun Lee, and Gunhee Kim. Super- vising neural attention models for video captioning by human gaze data. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 490-498. 2017.
Barret Zoph and Quoc V Le. Neural architecture search with reinforcement learning, arXiv preprint 17115 2016.
11


Reviews:
[{"review_id": "nk-Oyzg1PUR", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces a way to annotate a glimpse sequence for an image. It uses BAYESIAN OPTIMAL EXPERIMENTAL DESIGN to achieve this. Using the obtained annotations, hard attention can be trained with a partially supervised way.\n\nMy concerns mainly focus on the motivation and evaluation. Specifically, the authors claim using hard attention to do image classification may enable the use of modern approaches to computer vision in low-power settings such as mobile devices. But from the paper (besides the computation of glimpse sequence is computationally expensive.) at the end, the proposed method needs to run the glimpse network several times, and the structure of the glimpse network is similar to standard classification network, which needs only a single forward pass on the whole image. I'm doubt adopting the proposed approach can truly lead to a low power solution.  Image classification seems like an unsuitable task to show the benefits of hard attention.  Even if we would like to test on image classification, we need to include more baselines, such as standard convnets and compare them to the proposed method in terms of computational complexity and effectiveness. \n\nOther issues:\n  Stochastic image completion involves creating an empirical image distribution with 1.5 million images. Will this lead to an unfair comparison or data leakage?\n  It's better to include standard datasets for image classification such as ImageNet, CIFAR, stc. \n  It's interesting to see whether a hard attention method trained with REINFORCE will obtain a similar glimpse sequence and the proposed annotation method.", "extra_scores": {}, "extra_text": {}}, {"review_id": "gCjDe_ApEKg", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper frames hard visual attention as a Bayesian optimal experimental design (BOED) problem from which the optimal locations to attend to are those with greatest expected reduction in the entropy of classification. With the methodology from BOED literature, this paper approximate the optimal behavior to generate 'near optimal' sequences to partially supervise the learning of hard attention.\nOverall, I think the paper is easy to read, the proposed idea is solid and well-motivated, although the actual implementation seems to be quite complicated which involves additional two different models (attentional variational posterior and image completion model). It is hard to tell how the learning of these modules will affect the final performance. \n\nAdditionally I have some specific questions regarding to the paper:\n(1) I am a bit confused with Figure 3 and Sec 4. It is mentioned that the mask is concatenated as an additional channel. However, since you already mask the unobserved pixels to zero, why it is still needed?\n(2) Following Eq (7), is $g_{AVP}$ trained with randomly sampled glimpses? How can we make sure we can predict the label correctly from a random glimpse? Also, why it is called \"attentional\" posterior?\n(3) How the quality of $r_{img}$ will impact the results? Do we need to train a different $r_{img}$ first every time before testing on a new dataset? Is it possible to only condition on the current image to create glimpse supervisions with $g_{AVP}$ instead of using image completion?\n(4) How to inspect if the generated supervisions might hurt the performance? For instance, as the supervisions are generated to approximately maximize the entropy reduction, it is still possible that such attention is not optimal for neural network to predict the final label. \n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "cOB9tgKuVBr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a learning framework for a hard attention mechanism. The glimpses captured by the attention mechanism are guided by the goal of minimizing output uncertainty for a downstream task such as classification. The authors pose this problem in a probabilistic framework which is based on Bayesian optimal experimental design (BOED). They devise a tractable approximation to the entropy over images and glimpse sequences and search for the glimpse sequences which minimize the output entropy of a recurrent classification model.\n\nWith the ability to incorporate supervised glimpse training into the attention mechanism, the authors are able to alleviate the cold start issue which many hard attention models over images suffer from.\n\nWhile the probabilistic formulation described in the paper is fairly detailed, a figure showing how the attentional variational posterior ($g_{AVP}$) is integrated into the rest of the attention model in Figure 1. Is $g_{AVP}$ trained through some form of model distillation with the attention model?\n\nWhat types of hand-crafted glimpse sequences are beneficial for the model? CUBs annotated features were not intended to be a glimpse sequence, so how was the order of the annotations chosen?\n\nCan the authors provide a pseudo-code description of the attention algorithm? This would help clarify the pipeline of how $g_{AVP}$ is both trained and integrated into the larger model.\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "B2JvGXVCa6R", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper trains hard attention for image classification. The network is partially supervised by attention locations proposed to maximally reduce the entropy of the image label distribution. To propose these locations, the method needs an already trained image classifier conditioned on glimpses and their locations. Additionally, the method needs a generator of images, conditioned on the glimpses and their locations. This generator is approximated by searching a set of 1.5 million pre-generated images for close matches.\n\nIt is visible that the paper required a lot of work. Unfortunately, the method is not very practical and the method is not better than simpler alternatives.\n\nPros:\n- The paper is well written.\n\nMain cons:\n1) The greedy minimization of the expected posterior entropy is not optimal, if additional glimpses can be done in the next steps.\n2) The searching in the pre-generated dataset of images is not practical. It would work worse on more diverse images. The method was tested only on faces and birds.\n3) If the aim is to speed-up training of new networks, a simpler alternative is to distill the attention locations and classification outputs from a pretrained Recurrent Attention Model (RAM+).", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a new and unusual way of training hard attention mechanisms in vision models. Instead of training with reinforcement learning (as is typical), the authors develop a procedure for generating "glimpse sequences" that can be effectively used as supervision. Models trained in this way produce qualitatively "better" glimpse sequences, higher accuracy, and converge in fewer steps. There was some disagreement and discussion about the merits of the paper. Overall, there were some major concerns:
- The process for obtaining glimpse sequences is very computationally expensive. The authors argue that this cost can be amortized because the same glimpse sequences can be computed once for a given dataset and reused. However, there was limited real-world motivation for this setup, apart from mentioning neural architecture search (a niche method that is not widely used, and probably has never been used to develop hard-attention models).
- The method relies on a "convincing" generative model for a given dataset. This limits experiments to simple datasets with unrealistically-constrained visual structure. The authors point out that as generative models get better, their method could be applied to more realistic datasets, but as it stands the experimental validation is correspondingly weak.
- The improvement in performance is not huge - the "WSRAM" baseline outperforms the use of "near-optimal" glimpse sequences. While it is certainly true that the proposed method converges faster, the fact that the proposed method requires such an expensive preprocessing step downgrades this benefit significantly
- While the authors provided insightful distillation-based baselines in the rebuttal, it remains to be seen whether simple distillation from stronger RAM models (e.g. WSRAM which outperforms the proposed method) could be made to work better/more efficiently.
These factors lead to a reject decision overall.
Decision: Reject