Achieving Explainability in a Visual Hard Attention Model through Content Prediction

1 INTRODUCTION . Though deep convolution networks achieve state of the art performance on the image classification task , it is difficult to explain which input regions affected the output . A technique called visual hard attention provides this explanation by design . The hard attention model sequentially attends small but informative subregions of the input called glimpses to make predictions . While the attention mechanism explains the task-specific decisions , the attention policies learned by the model remain unexplainable . For example , one can not explain the attention policy of a caption generation model that correctly predicts the word ‘ frisbee ’ while looking at a region far from an actual frisbee ( Xu et al . ( 2015 ) ) . The majority of hard attention models first analyze a complete image to locate the task-relevant subregions and then attend to these locations to make predictions ( Ba et al . ( 2014 ) ; Elsayed et al . ( 2019 ) ) . However , in practice , we often do not have access to the entire scene , and we gradually attend to the important subregions to collect task-specific information . At each step in the process , we decide the next attention-worthy location based on the partial observations collected so far . The explainable attention policies are more desirable under such partial observability . Pioneering work by Mnih et al . ( 2014 ) presents a model that functions under partial observability but their attention policies are not explainable . They train their model with the REINFORCE algorithm ( Williams ( 1992 ) ) , which is challenging to optimize . Moreover , the model ’ s performance is affected adversely if the parameterization of the attention policy is not optimal . For example , an object classification model with unimodal Gaussian policy learns to attend the background region in the middle of the two objects ( Sermanet et al . ( 2014 ) ) . This paper develops a hard-attention model with an explainable attention policy for classifying images through a series of partial observations . We formulate the problem of hard attention as a Bayesian Optimal Experiment Design ( BOED ) . A recurrent model finds an optimal location that gains maximum expected information about the class label and attends to this location . To estimate expected information gain ( EIG ) under partial observability , the model predicts content of the un- seen regions based on the regions observed so far . Using the knowledge gained by attending various locations in an image , the model predicts the class label . To the best of our knowledge , ours is the first hard attention model that is entirely explainable under partial observability . Our main contributions are as follows . First , our attention policies are explainable by design . One can explain that the model attends a specific location because it expects the corresponding glimpse to maximize the expected information gain . Second , the model does not rely on the complete image to predict the attention locations and provides good performance under partial observability . Third , the training objective is differentiable and can be optimized using standard gradient backpropagation . We train the model using discriminative and generative objectives to predict the label and the image content , respectively . Fourth , our attention policy is non-parametric and can be implicitly multi-modal . 2 RELATED WORKS . A hard attention model prioritizes task-relevant regions to extract meaningful features from an input . Early attempts to model attention employed image saliency as a priority map . High priority regions were selected using methods such as winner-take-all ( Koch & Ullman ( 1987 ) ; Itti et al . ( 1998 ) ; Itti & Koch ( 2000 ) ) , searching by throwing out all features but the one with minimal activity ( Ahmad ( 1992 ) ) , and dynamic routing of information ( Olshausen et al . ( 1993 ) ) . Few used graphical models to model visual attention . Rimey & Brown ( 1991 ) used augmented hidden Markov models to model attention policy . Larochelle & Hinton ( 2010 ) used a Restricted Boltzmann Machine ( RBM ) with third-order connections between attention location , glimpse , and the representation of a scene . Motivated by this , Zheng et al . ( 2015 ) proposed an autoregressive model to compute exact gradients , unlike in an RBM . Tang et al . ( 2014 ) used an RBM as a generative model and searched for informative locations using the Hamiltonian Monte Carlo algorithm . Many used reinforcement learning to train attention models . Paletta et al . ( 2005 ) used Q-learning with the reward that measures the objectness of the attended region . Denil et al . ( 2012 ) estimated rewards using particle filters and employed a policy based on the Gaussian Process and the upper confidence bound . Butko & Movellan ( 2008 ) modeled attention as a partially observable Markov decision process and used a policy gradient algorithm for learning . Later , Butko & Movellan ( 2009 ) extended this approach to multiple objects . Recently , the machine learning community use the REINFORCE policy gradient algorithm to train hard attention models ( Mnih et al . ( 2014 ) ; Ba et al . ( 2014 ) ; Xu et al . ( 2015 ) ; Elsayed et al . ( 2019 ) ) . Among these , only Elsayed et al . ( 2019 ) claims explainability by design . Other works use EMstyle learning procedure ( Ranzato ( 2014 ) ) , wake-sleep algorithm ( Ba et al . ( 2015 ) ) , a voting based region selection ( Alexe et al . ( 2012 ) ) , and differentiable models ( Gregor et al . ( 2015 ) ; Jaderberg et al . ( 2015 ) ; Eslami et al . ( 2016 ) ) . Among the recent models , Ba et al . ( 2014 ) ; Ranzato ( 2014 ) ; Ba et al . ( 2015 ) look at the lowresolution gist of an input at the beginning , and Xu et al . ( 2015 ) ; Elsayed et al . ( 2019 ) ; Gregor et al . ( 2015 ) ; Jaderberg et al . ( 2015 ) ; Eslami et al . ( 2016 ) consume the whole image to predict the locations to attend . In contrast , our model does not look at the entire image at low resolution or otherwise . Moreover , our attention policies are explainable . We can apply our model in a wide range of scenarios where explainable predictions are desirable for the partially observable images . 3 MODEL . In this paper , we consider a recurrent attention model that sequentially captures glimpses from an image x and predicts a label y . The model runs for time t = 0 to T − 1 . It uses a recurrent net to maintain a hidden state ht−1 that summarizes glimpses observed until time t − 1 . At time t , it predicts coordinates lt based on the hidden state ht−1 and captures a square glimpse gt centered at lt in an image x , i.e . gt = g ( x , lt ) . It uses gt and lt to update the hidden state to ht and predicts the label y based on the updated state ht . 3.1 ARCHITECTURE . As shown in Figure 1 ( a ) , our model comprises the following three building blocks . A recurrent feature aggregator ( F and R ) maintains a hidden state ht . A classifier ( C ) predicts the class probabilities p ( y|ht ) . A normalizing flow-based variational autoencoder ( S and D ) predicts a complete image given the hidden state ht ; a flow-based encoder S predicts the posterior of a latent variable z from ht , and a decoderD predicts a complete image from z . The BOED , as discussed in section 3.2 , uses the predicted image to find an optimal location to attend at the next time-step . To distinguish the predicted image from the actual image , let us call the former x̃ . Henceforth , we crown any quantity derived from the predicted image x̃ with a ( ˜ ) . Next , we provide details about the three building blocks of the model , followed by a discussion of the BOED in the context of hard attention . 3.1.1 A RECURRENT FEATURE AGGREGATOR . Given a glimpse gt and its location lt , a feed-forward module extracts features ft = F ( gt , lt ) , and a recurrent network updates a hidden state to ht = R ( ht−1 , ft ) . Following Mnih et al . ( 2014 ) , we define F ( gt , lt ) = BN ( LeakyReLU ( Fg ( gt ) + Fl ( lt ) ) ) where Fg and Fl are deep networks , and R ( ht−1 , ft ) = LN ( LeakyReLU ( Linear ( ht−1 ) +Linear ( ft ) ) ) . Here , BN is a BatchNorm layer ( Ioffe & Szegedy ( 2015 ) ) and LN is a LayerNorm layer ( Ba et al . ( 2016 ) ) . 3.1.2 A CLASSIFIER . At each time-step t , a linear classifier predicts the distribution p ( y|ht ) = C ( ht ) from a hidden state ht . As the goal of the model is to predict a label y for an image x , we learn a distribution p ( y|ht ) by minimizing KL [ p ( y|x ) ||p ( y|ht ) ] . Optimization of this KL divergence is equivalent to the minimization of the following cross-entropy loss . LCE ( t ) = −p ( y|x ) log ( p ( y|ht ) ) ( 1 ) 3.1.3 A PARTIAL VARIATIONAL AUTOENCODER . We adapt a variational autoencoder ( VAE ) to predict the complete image x from the hidden state ht . A VAE learns a joint distribution between the image x and the latent variable z given ht , p ( x , z|ht ) = p ( x|z ) p ( z|ht ) . An amortized encoder infers the posterior q ( z|x , ht ) , which is an approximation of the true posterior p ( z|x , ht ) , and a decoder infers the likelihood p ( x|z ) . The training of VAE requires optimizing the Evidence Lower Bound ( ELBO ) , which involves calculating KL [ q ( z|x , ht ) ||p ( z|ht ) ] ( Kingma & Welling ( 2013 ) ) . As the hard attention model does not observe the complete image x , it can not estimate q ( z|x , ht ) . Hence , we can not incorporate the standard VAE directly into a hard attention framework . At the time t , we separate an image x into two parts , ot — the set of regions observed up to t , and ut — the set of regions as yet unobserved . Ma et al . ( 2018 ) observed that in a VAE , ot and ut are conditionally independent given z , i.e . p ( x|z ) = p ( ut|z ) p ( ot|z ) . They predict ut independently from the sample z ∼ q ( z|ot ) , while learning the approximate posterior q ( z|ot ) by optimizing the ELBO on log ( p ( ot ) ) . They refer to the resultant VAE as a Partial VAE . Recall that the hidden state ht calculated by our attention model is a summary of the glimpses observed up to t , which is equivalent to ot , the set of observed regions . Hence , we can write q ( z|ot ) as q ( z|ht ) in the ELBO of the Partial VAE . LPVAE ( t ) = Eq ( z|ot ) log ( p ( ot|z ) ) −KL [ q ( z|ot ) ||p ( z ) ] = Eq ( z|ht ) log ( p ( ot|z ) ) −KL [ q ( z|ht ) ||p ( z ) ] ( 2 ) In a Partial VAE , p ( x , z|ht ) = p ( ut|z ) p ( ot|z ) p ( z|ht ) . We implement a decoder D that predicts the complete image given the sample z ∼ q ( z|ht ) . Let mt be a binary mask with value 1 for the pixels observed by the model up to t and 0 otherwise ; hence , ot = mt x , where is an element-wise multiplication . We write the log-likelihood in equation 2 using the mask mt as follows . log ( p ( ot|z ) ) = −0.5 ∑ |mt D ( z ) −mt x|2 = −0.5 ∑ mt |D ( z ) − x|2 ( 3 ) In equation 2 , the prior p ( z ) is a Gaussian distribution with zero mean and unit variance . To obtain expressive posterior q ( z|ht ) , we use normalizing flows ( Kingma et al . ( 2016 ) ) . As an explicit inversion of the flows is not required , we use auto-regressive Neural Spline Flows ( NSF ) ( Durkan et al . ( 2019 ) ) and efficiently implement them using a single feed-forward network with masked weights as in De Cao et al . ( 2019 ) . Between the two flow layers , we flip the input ( Dinh et al . ( 2016 ) ) and normalize it using ActNorm ( Kingma & Dhariwal ( 2018 ) ) . In Figure 1 ( a ) , the flow-based encoder S infers the posterior q ( z|ht ) = S ( ht ) . As mentioned earlier , we refer to the prediction from the Partial VAE as x̃ . The BOED uses x̃ to find an optimal location to attend . 1

Summary: This paper follows a less explored strategy for achieving explainability via hard attention. They proposed a recurrent architecture which sequentially observe regions (glimpse) from an image. To decide where to look next, the model maintains a hidden state and use it to estimate the full image (or features of the image). This "content prediction" module allows the model to look ahead and make a decision based on the expected information gain (EIG) over different locations. The objective function (i.e. partial VAE loss and classification loss) in this system is differentiable thus the system can be trained with gradient descent. The authors validated the system on several benchmarks and show comparable performance with baselines.