Classifier-agnostic saliency map extraction

Extracting saliency maps , which indicate parts of the image important to classification , requires many tricks to achieve satisfactory performance when using classifier-dependent methods . Instead , we propose classifier-agnostic saliency map extraction , which finds all parts of the image that any classifier could use , not just one given in advance . We observe that the proposed approach extracts higher quality saliency maps and outperforms existing weakly-supervised localization techniques , setting the new state of the art result on the ImageNet dataset . 1 INTRODUCTION . The success of deep convolutional networks for large-scale object recognition Krizhevsky et al . ( 2012 ) ; Simonyan & Zisserman ( 2014 ) ; Szegedy et al . ( 2015 ) ; He et al . ( 2016 ) has spurred interest in utilizing them to automatically detect and localize objects in natural images . Pioneering this direction , Simonyan et al . ( 2013 ) and Springenberg et al . ( 2014 ) demonstrated that the gradient of the class-specific score of a given classifier could be used for extracting a saliency map of an image . Such classifier-dependent saliency maps can be utilized to analyze the inner workings of a specific network . However , as only the part of the image that is used by a given model is highlighted , these methods are not identifying all “ evidence ” in a given image . They also tend to be noisy , covering many irrelevant pixels and missing many relevant ones . Therefore , much of the recent work has focused on introducing regularization techniques of correcting such classifier-dependent saliency maps . For instance , Selvaraju et al . ( 2017 ) propose averaging multiple saliency maps created for perturbed images to obtain a smooth saliency map . We argue , however , that applying tricks and tweaks on top of methods that were designed to analyze inner workings of a given classifier is not a principled way to get saliency maps that focus on all useful evidence . In this work , we aim to find saliency maps indicating pixels which aid classification , i.e . we want to find pixels in the input image such that if they were masked , it would confuse an unknown classifier . Assuming we were given a classifier , a naive approximate solution would be to train a generative model to output a mask ( a saliency map ) confusing that classifier . That can be achieved using a simple GAN-like approach ( Goodfellow et al. , 2014 ) where the classifier acts as a fixed discriminator . Unfortunately , as we prove experimentally , this solution suffers from the same issues as prior approaches . We argue that the strong dependence on a given classifier lies at the center of the problem . To tackle this directly we propose to train a saliency mapping that is not strongly coupled with any specific classifier . Our approach , a class-agnostic saliency map extraction , can be formulated as a practical algorithm that realizes our goal . Our focus on classifier-agnostic saliency maps is not our objective per se , it is a remedy that resolves the core problem . The proposed approach results in a neural network based saliency mapping that only depends on an input image . We qualitatively find that it extracts higher quality saliency maps compared to classifier-dependent methods , as can be seen in Fig . 2 . Extracted saliency maps show all the evidence without using any symptom-masking methods : difficult to tune regularization penalties ( such as total variation ) , exotic activation functions , complex training procedures or image preprocessing tricks ( such as superpixels ) , etc . We also evaluate our method quantitatively by using the extracted saliency maps for object localization . We observe that the proposed approach outperforms the existing weakly-supervised techniques setting the new state of the art result on the ImageNet dataset and closely approaches the localization performance of a strongly supervised model . Furthermore , we experimentally validate that the proposed approach works reasonably well even for classes unseen during training . Our method has many potential applications , in which being classifier-agnostic is of primary importance . For instance , in medical image analysis , where we are interested not only in class prediction but also in indicating which part of the image is important to classification . Importantly , however , it is criticial to indicate all parts of the image , which can influence diagnosis , not just ones used by a specific classifier . 2 CLASSIFIER-AGNOSTIC SALIENCY MAP EXTRACTION . In this paper , we tackle a problem of extracting a salient region of an input image as a problem of extracting a mapping m : RW×H×3 → [ 0 , 1 ] W×H over an input image x ∈ RW×H×3 . Such a mapping should retain ( = 1 ) any pixel of the input image if it aids classification , while it should mask ( = 0 ) any other pixel . 2.1 CLASSIFIER-DEPENDENT SALIENCY MAP EXTRACTION . Earlier work has largely focused on a setting in which a classifier f was given ( Fong & Vedaldi , 2017 ; Dabkowski & Gal , 2017 ) . These approaches can be implemented as solving the following maximization problem : m = argmax m′ S ( m′ , f ) , ( 1 ) where S is a score function corresponding to a classification loss . That is , S ( m , f ) = 1 N N∑ n=1 [ l ( f ( ( 1−m ( xn ) ) xn ) , yn ) +R ( m ( xn ) ) ] , ( 2 ) where denotes elementwise multiplication ( masking ) , R ( m ) is a regularization term and l is a classification loss , such as cross-entropy . We are given a training set D = { ( x1 , y1 ) , . . . , ( xN , yN ) } . This optimization procedure could be interpreted as finding a mapping m that maximally confuses a given classifier f . We refer to it as a classifier-dependent saliency map extraction . A mapping m obtained with a classifier f may differ from a mapping m′ found using f ′ , even if both classifiers are equally good in respect to a classification loss for both original and masked images , i.e . L ( 0 , f ) = L ( 0 , f ′ ) and L ( m , f ) = L ( m′ , f ′ ) , where L ( m , f ) = 1 N N∑ n=1 l ( f ( ( 1−m ( xn ) ) xn ) , yn ) . ( 3 ) This property is against our definition of the mapping m above , which stated that any pixel which helps classification should be indicated by the mask ( a saliency map ) with 1 . The reason why this is possible is these two equally good , but distinct classifiers may use different subsets of input pixels to perform classification . An example This behaviour can be intuitively explained with a simple example , illustrating an extreme special case . Let us consider a data set in which all instances consist of two identical copies of images concatenated together , that is , for all xn , xn = [ x′n ; x ′ n ] , where x ′ n ∈ RW/2×H×3 . For such a data set , there exist at least two classifiers , f and f ′ , with the same classification loss . The classifier f uses only the left half of the image , while f ′ uses the other half . Each of the corresponding mappings , m and m′ , would then indicate a region of interest only on the corresponding half of the image . When the input image does not consist of two concatenated copies of the same image , it is unlikely that two equally good classifiers will use disjoint sets of input pixels . Our example is to show an extreme case when it is possible . 2.2 CLASSIFIER-AGNOSTIC SALIENCY MAP EXTRACTION . In order to address the issue of saliency mapping ’ s dependence on a single classifier , we propose to alter the objective function in Eq . ( 1 ) to consider not only a single fixed classifier but all possible classifiers weighted by their posterior probabilities . That is , m = argmax m′ Ef [ S ( m′ , f ) ] , ( 4 ) where the posterior probability , p ( f |D , m′ ) , is defined to be proportional to the exponentiated classification loss L , i.e. , p ( f |D , m′ ) ∝ p ( f ) exp ( −L ( m′ , f ) ) . Solving this optimization problem is equivalent to searching over the space of all possible classifiers , and finding a mapping m that works with all of them . As we parameterize f as a convolutional network ( with parameters denoted as θf ) , the space of all possible classifiers is isomorphic to the space of its parameters . The proposed approach considers all the classifiers and we call it a classifier-agnostic saliency map extraction . In the case of the simple example above , where each image contains two copies of a smaller image , both f and f ′ , which respectively look at one and the other half of an image , the posterior probabilities of these two classifiers would be the same1 . Solving Eq . ( 4 ) implies that a mapping m must minimize the loss S for both of these classifiers . 2.3 ALGORITHM Algorithm 1 : Classifier-agnostic saliency map extraction input : an initial classifier f ( 0 ) , an initial mapping m ( 0 ) , dataset D , number of iterations K output : the final mapping m ( K ) Initialize a sample set F ( 0 ) = { f ( 0 ) } . for k ← 1 to K do θf ( k ) ← θf ( k−1 ) − ηf∇θfL ( m ( k−1 ) , f ( k−1 ) ) F ( k ) ← F ( k−1 ) ∪ { f ( k ) } f ′ ← Sample ( F ( k ) ) θm ( k ) ← θm ( k−1 ) + ηm∇θmS ( m ( k−1 ) , f ′ ) F ( k ) ← Thin ( F ( k ) ) The optimization problem in Eq . ( 4 ) is , unfortunately , generally intractable . This arises from the intractable expectation over the posterior distribution . Furthermore , the expectation is inside the optimization loop for the mapping m , making it even harder to solve . Thus , we approximately solve this problem by simultaneously estimating the mapping m and the expected objective . First , we sample one f ( k ) with the posterior probability p ( f |D , m ( k−1 ) ) by taking a single step of stochastic gradient descent ( SGD ) on the classification loss with respect to θf ( classifier f parameters ) with a small step size : θf ( k ) ← θf ( k−1 ) − ηf∇θfL ( m ( k−1 ) , f ( k−1 ) ) . ( 5 ) This is motivated by earlier work ( Welling & Teh , 2011 ; Mandt et al. , 2017 ) which showed that SGD performs approximate Bayesian posterior inference . We have up to k + 1 samples2 from the posterior distribution F ( k ) = { f ( 0 ) , . . . , f ( k ) } . We sample3 f ′ ∈ F ( k ) to get a single-sample estimate of m in in Eq . ( 4 ) by computing S ( m ( k−1 ) , f ′ ) . Then , we use it to obtain an updated θm ( mapping m parameters ) by θm ( k ) ← θm ( k−1 ) + ηm∇θmS ( m ( k−1 ) , f ′ ) . ( 6 ) We alternate between these two steps until m ( k ) converges ( cf . Alg . 1 ) . Note , that our algorithm resembles the training procedure of GANs ( Goodfellow et al. , 2014 ) , where mapping m takes the role of a generator and the classifier f can be understood as a discriminator . Fan et al . ( 2017 ) also applied adversarial training in order to achieve better saliency maps . The relation to the work is discussed in details in Section 6 . Score function The score function S ( m , f ) estimates the quality of the saliency map extracted by m given a data set and a classifier f . The score function must be designed to balance the precision and recall . The precision refers to the fraction of relevant pixels among those marked by m as relevant , while the recall is the fraction of pixels correctly marked by m as relevant among all the relevant pixels . In order to balance these two , the score function often consists of two terms . The first term is aiming to ensure that all relevant pixels are included ( high recall ) . As in Eq . ( 2 ) , a popular choice has been the classification loss based on an input image x masked out by 1−m ( x ) . In our preliminary experiments , however , we noticed that this approach leads to obtaining masks with adversarial artifacts . We hence propose to use the entropyH ( f ( ( 1−m ( x ) ) x ) ) instead . This makes generated masks cover all salient pixels in the input , avoiding masks that may sway the class 1We assume a flat prior , i.e. , p ( f ) = c. 2A usual practice of “ thinning ” may be applied , leading to a fewer than k samples . 3We set the chance of selecting f ( k ) to 50 % and we spread the remaining 50 % uniformly over F ( k−1 ) . prediction to a different , but semantically close class . For example , from one dog species to another . The second term , R ( m ) , excludes a trivial solution , m simply outputting an all-ones saliency map , which would achieve maximal recall with low very precision . In order that , we must introduce a regularization term . Some of the popular choices include total variation ( Rudin et al. , 1992 ) and L1 norm . For simplicity , we use the latter only . In summary , we use the following score function for the class-agnostic saliency map extraction : S ( m , f ) = 1 N N∑ n=1 [ H ( f ( ( 1−m ( xn ) ) xn ) ) − λR‖m ( xn ) ‖1 ] , ( 7 ) where λR > 0 is a regularization coefficient . Thinning As the algorithm collects a set of classifiers , f ( k ) ’ s , from the posterior distribution , we need a strategy to keep a small subset of them . An obvious approach would be to keep all classifiers but this does not scale well with the number of iterations . We propose and empirically evaluate a few strategies . The first three of them assume a fixed size of F ( k ) . Namely , keeping the first classifier only , denoted by F ( F ( k ) = { f ( 0 ) } ) , the last only , denoted by L ( F ( k ) = { f ( k ) } ) and the first and last only , denoted by FL ( F ( k ) = { f ( 0 ) , f ( k ) } ) . As an alternative , we also considered a growing set of classifiers where we only keep one every 1000 iterations ( denoted by L1000 ) but whenever |F ( k ) | = 30 , we randomly remove one from the set . Analogously , we experimented with L100 . Classification loss Although we described our approach using the classification loss computed only on masked images , as in Eq . ( 3 ) , it is not necessary to define the classification loss exactly in this way . In the preliminary experiments , we noticed that the following alternative formulation , inspired by adversarial training ( Szegedy et al. , 2013 ) , works better : L ( m , f ) = 1 2N N∑ n=1 [ l ( f ( ( 1−m ( xn ) ) xn ) , yn ) + l ( f ( xn ) , yn ) ] . ( 8 ) We thus use the loss as defined above in the experiments . We conjecture that it is advantageous over the original one in Eq . ( 3 ) , as the additional term prevents the degradation of the classifier ’ s performance on the original , unmasked images while the first term encourages the classifier to collect new pieces of evidence from the images that are masked .

Summary: This paper proposes a classifier-agnostic method for saliency map extraction. In order to address the dependence of saliency map extraction on the classifier, the authors propose to learn a saliency mapping by considering all possible classifiers (i.e., a certain classifier structure w.r.t. the space of all its parameters). The goal is to find the relevant features in the data that work with all possible classifiers. The proposed framework is formulated as a min-max game between two players: a mask m corresponding to the saliency mapping, and a function f sampled from a set of classifiers with the same structure but different parameters. The mapping m is optimized to maximize the masked-out classification error (such that m captures all relevant features whose removal can maximally confuse the classifier), while f is optimized to minimize the mask-out classification error.