Why do These Match? Explaining the Behavior of Image Similarity Models

1 INTRODUCTION . Many problems in artificial intelligence that require reasoning about complex relationships can be solved by learning some feature embedding to measure similarity between images and/or other modalities such as text . Examples of these tasks include scoring fashion compatibility ( Han et al. , 2017b ; Hsiao & Grauman , 2018 ; Vasileva et al. , 2018 ) , image retrieval ( Kiapour et al. , 2015 ; Radenovi et al. , 2018 ; Yelamarthi et al. , 2018 ) , or zero-shot recognition ( Bansal et al. , 2018 ; Li et al. , 2018b ; Wang et al. , 2018 ) . Reasoning about the behavior of similarity models can aid researchers in identifying potential improvements , show where two images differ for anomaly detection , promote diversity in fashion recommendation by ensuring different traits are most prominent in the top results , or simply help users understand the model ’ s predictions which can build trust ( Teach & Shortliffe , 1981 ) . However , prior work on producing explanations for neural networks has primarily focused on explaining classification models ( e.g . ( Fong & Vedaldi , 2017 ; Nguyen et al. , 2016 ; Petsiuk et al. , 2018 ; Ribeiro et al. , 2016 ; Selvaraju et al. , 2017 ; Zeiler & Fergus , 2014 ) ) and does not directly apply to similarity models . Given a single input image , such methods produce a saliency map which identifies pixels that played a significant role towards a particular class prediction ( see Figure 1a for an example ) . On the other hand , a similarity model requires at least two images to produce a score . The interaction between both images defines which features are more important , so replacing just one of the images can result in identifying different salient traits . Another limitation of existing work is that the saliency alone may be insufficient as an explanation of ( dis ) similarity . For image pairs where similarity is determined by the presence or absence of an object , a saliency map may be enough to understand model behavior . However , when we consider the image pair in Figure 1b , highlighting the necklace as the region that contributes most to the similarity score is reasonable , but uninformative given that there are no other objects in the image . Instead , what is important is the fact that the necklace shares a similar color with the ring . Whether these attributes or salient parts are a better fit as an explanation is not determined by the image domain ( i.e . attributes for e-commerce imagery vs. saliency for natural imagery ) , but instead by the images themselves . For example , an image can be matched as formal-wear because of a shirt ’ s collar ( salient part ) , while two images of animals can match because both have stripes ( attribute ) . Guided by this intuition , we introduce Salient Attributes for Network Explanation ( SANE ) . Our approach generates a saliency map to explain a model ’ s similarity score , paired with an attribute explanation that identifies important image properties . SANE is a “ black box ” method , meaning it can explain any network architecture and only needs to measure changes to a similarity score when provided with different inputs . Unlike a standard classifier , which simply predicts the most likely attributes for a given image , our explanation method predicts which attributes are important for the similarity score predicted by a model . Predictions are made for each image in a pair , and allowed to be non-symmetric , e.g. , the explanation for why the ring in Figure 1b matches the necklace may be that it contains “ black ” , even though the explanation for why the necklace matches the ring could be that it is “ golden. ” A different similarity model may also result in different attributes being deemed important for the same pair of images . SANE combines three major components : an attribute predictor , a prior on the suitability of each attribute as an explanation , and a saliency map generator . Our underlying assumption is that at least one of the attributes present in each image should be able to explain the similarity score assigned to the pair . Given an input image , the attribute predictor outputs a confidence score and activation map for each attribute , while the saliency map generator produces regions important for the match . During training , SANE encourages overlap between the similarity saliency and attribute activation . At test time , we rank attributes as explanations for an image pair based on a weighted sum of this attribute-saliency map matching score , the explanation suitability prior of the attribute , and the likelihood that the attribute is present in the image . Although we evaluate only the top-ranked attribute in our experiments , in practice more than one attribute could be used to explain a similarity score . We find that using saliency maps as supervision for the attribute activation maps during training not only improves the attribute-saliency matching , resulting in better attribute explanations , but also boosts attribute recognition performance using standard metrics like average precision . We evaluate several candidate saliency map generation methods which are primarily adaptations of “ black box ” approaches that do not rely on a particular model architecture or require access to network parameters to produce a saliency map ( Fong & Vedaldi , 2017 ; Petsiuk et al. , 2018 ; Ribeiro et al. , 2016 ; Zeiler & Fergus , 2014 ) . These methods generally identify important regions by measuring a change in the output class score resulting from some perturbation of the input image . Similarity models , however , typically rely on a learned embedding space to reason about relationships between images , where proximity between points or the lack thereof indicates some degree of correspondence . An explanation system for embedding models must , therefore , consider how distances between embedded points , and thus their similarity , change based on perturbing one or both of the input images . We explore two strategies for adapting these approaches to our task . First , we manipulate just a single image ( the one we wish to produce an explanation for ) while keeping the other image fixed . Second , we manipulate both images to allow for more complex interactions between the pair . See Section 3.2 for additional details and discussion on the ramifications of this choice . Our paper makes the following contributions : 1 ) we provide the the first quantitative study of explaining the behavior of image similarity models ; 2 ) we propose a novel explanation approach that combines saliency maps and attributes ; 3 ) we validate our method with metrics designed to link our explanations to model performance , and find that it produces more informative explanations than adaptations of prior work to this task and also improves attribute recognition performance . 2 RELATED WORK . Saliency-based Explanations . Saliency methods can generally be split into “ white box ” and “ black box ” approaches . “ White box ” methods assume access to internal components of a neural network , either in the form of gradients or activations of specific layers ( e.g . ( Cao et al. , 2015 ; Chang et al. , 2019 ; Nguyen et al. , 2016 ; Selvaraju et al. , 2017 ; Simonyan et al. , 2014 ; Yosinski et al. , 2015 ; Zhang et al. , 2016 ; Zhou et al. , 2016 ) ) . Most of them produce a saliency map by using some version of backpropagation from class probability to an input image . In contrast , “ black box ” approaches require no knowledge of the internals ( e.g . weights , gradients ) of the models . These methods obtain saliency maps by perturbing the input in a predefined way and measuring the effect of it on the model output , such as class score . We adapt and compare three “ black box ” and one “ white box ” methods for our saliency map generator in Figure 2 . “ Black box ” approaches include a Sliding Window ( Zeiler & Fergus , 2014 ) , which masks image regions sequentially , and Randomized Input Sampling for Explanations ( RISE ) ( Petsiuk et al. , 2018 ) , which masks random sets of regions . Both measure the effect removing these regions have on the class score . LIME ( Ribeiro et al. , 2016 ) first obtains a super-pixel representation of an image . Super-pixel regions are randomly deleted , and their importance is estimated using Lasso . “ White box ” Mask ( Fong & Vedaldi , 2017 ) learns a saliency map directly by using different perturbation operators and propagating the error to a low resolution mask . Although there is some limited work which adapts some saliency methods to the image similarity setting ( e.g. , Gordo & Larlus ( 2017 ) ) , they present qualitative results only , i.e. , these methods are not evaluated quantitatively on their explanation accuracy as done in our work . Natural Language and Attribute-based Explanations . Instead of producing saliency maps , which can sometimes be difficult to interpret , researchers have explored methods of producing text-based explanations . These include methods which justify a model ’ s answer in the visual question answering task ( Huk Park et al. , 2018 ; Li et al. , 2018a ) , rationalize the behavior of a self-driving vehicle ( Kim et al. , 2018b ) , or describe why a category was selected in fine-grained object classification ( Hendricks et al. , 2016 ) . Lad & Parikh ( 2014 ) used human-generated attribute explanations describing why two images are similar or dissimilar as guidance for image clustering . Our approach could be used to automatically generate these explanations rather than relying on human feedback . Several works exist which learn attribute explanations either to identify important concepts ( Kim et al. , 2018a ; Bau et al. , 2017 ; Fong & Vedaldi , 2018 ) or to justify a model ’ s decision by pointing to evidence ( Anne Hendricks et al. , 2018 ) . Kim et al . ( 2018a ) learns a concept activation vector that separates examples with an attribute against examples without it , and then scores the sensitivity of attributes based on how often a directional derivative changes the inputs towards the concept . However , all these methods were designed to explain categorical predictions rather than similarity models . To the best of our knowledge , ours is the first work which uses attribute explanations in the image similarity setting . 3 SALIENT ATTRIBUTES FOR NETWORK EXPLANATIONS ( SANE ) . We are given a fixed model that predicts the similarity between two images , and must explain why a query image is similar to a reference image . While typical models for predicting similarity are learned from data , using an embedding method and a triplet loss , our approach is agnostic as to how the model being explained is built . Our method consists of three components : the attribute explanation model ( Section 3.1 ) , the saliency map generator ( Section 3.2 ) , and an attribute explanation suitability prior ( Section 3.3 ) . Although we train a CNN to produce attribute predictions , the image similarity model we wish to explain is kept fixed . At test time , one recovers a saliency map for the match from the query image in a pair , then uses the attribute explanation model and attribute suitability prior to rank each attribute ’ s ability to explain the image similarity model .

Summary: This paper proposes a novel method for image similarity models explanation, introducing Salient Attributes for Network Explanation (SANE). The method identifies attributes that contribute positively to the similarity score, thus explaining the important image properties, and pair them with a generated saliency map unveiling the important regions of the image. The method combines three major components: