Axiomatic Explanations for Visual Search, Retrieval, and Similarity Learning

1 INTRODUCTION . Search , recommendation , retrieval , and contrastive similarity learning powers many of today ’ s machine learning systems . These systems help us organize information at scales that no human could match . The recent surge in million and billion parameter contrastive learning architectures for vision and language underscore the growing need to understand these classes of systems ( Nayak , 2019 ; Chen et al. , 2020b ; a ; Radford et al. , 2021 ; Caron et al. , 2020 ) . At its core , these systems face the same challenges that classifiers and regressors face : richer models can often improve performance but hinder interpretability . For high-risk domains like medicine , incorrect search results can have serious consequences . In other domains , search engine bias can disproportionately hide certain voices and have systemic impact ( Mowshowitz & Kawaguchi , 2002 ; Diaz , 2008 ; Goldman , 2005 ) . Currently , there are several competing techniques practitioners can use to understand similarity model predictions ( Zhu et al. , 2019 ; Zheng et al. , 2020 ; Dong et al . ; Selvaraju et al. , 2017 ; Vaswani et al. , 2017 ) . However , it is often unclear when one approach should be chosen over another and there is not a formal theory describing an optimal search explanation method . To help solve this challenge we show that the theory of fair credit assignment provides a uniquely determined and axiomatic approach for “ explaining ” why a trained model judges two objects as similar . In many cases , existing approaches are special cases of this formalism . This allows us to design variants of these methods that better satisfy the axioms of fair credit assignment and can handle counterfactual or relative explanations . Though we explore this topic through the lens of visual search , we note that these techniques could also apply to text , tabular , or audio search systems . This work identifies two distinct classes of search engine explainability methods . “ First order ” approaches highlight the most important pixels that contribute to the similarity of objects and “ Second order ” explanations provide a full correspondence between the parts of query and retrieved image . We relate first order interpretations to existing theory on classifier explainability through a generic function transformation , as shown in the third column of Figure 1 . We find that second order explanations correspond to a uniquely specified generalization of the Shapley values ( Sundararajan et al. , 2020 ) and is equivalent to projecting Harsanyi Dividends onto low-order subsets ( Harsanyi , 1963 ) . Using this formalism , we present new second-order generalizations of Class Activation Maps ( Zhou et al. , 2016 ) , GradCAM ( Selvaraju et al. , 2017 ) , LIME ( Ribeiro et al. , 2016 ) , and SHAP ( Lundberg & Lee , 2017 ) . These contributions generalize several existing methods , illustrate a rich mathematical structure connecting model explainability and cooperative game theory , and allow practitioners to understand search engines with greater nuance and detail . We include a short video detailing the work at https : //tinyurl.com/2j3y2ffn . In summary we : • Present the first uniquely specified axiomatic framework for model-agnostic search , retrieval , and metric learning interpretability using the theory of Harsanyi dividends . • Show that our framework generalizes several existing model explanation methods ( Zhou et al. , 2016 ; Selvaraju et al. , 2017 ; Zhu et al. , 2019 ; Ribeiro et al. , 2016 ) and allows us to extend these methods to handle counterfactual explanations and extract dense pairwise correspondences between query and retrieved images . • Introduce a new kernel-based approximator for Shapley-Taylor indices that requires about 10× fewer function evaluations . • Show that our axiomatic approaches provide more faithful explanations of image similarity on the PascalVOC and MSCoCo datasets . 2 BACKGROUND . This work focuses on search , retrieval , metric learning , and recommendation architectures . Often , these systems use similarity between objects or learned features ( Bengio et al. , 2013 ) to rank , retrieve , or suggest content ( Bing , 2017 ; Su & Khoshgoftaar , 2009 ; Chopra et al. , 2005 ; Radford et al. , 2021 ) . More formally , we refer to systems that use a distance , relevance , or similarity function of the form : d : X × Y → R to quantify the relationship between items from sets X and Y . In search and retrieval , X represents the space of search queries and Y represents the space of results , the function d assigns a relevance to each query result pair . Without loss of generality , we consider d as a “ distance-like ” function where smaller values indicate more relevance . The expression arg miny∈Y d ( x , y ) yields the most relevant result for a query x ∈ X . Specializing this notion yields a variety of different kinds of ML systems . IfX = Y = Range ( N ( · ) ) whereN is an image featurization network such as ResNet50 ( He et al. , 2016 ) , the formalism yields a visual search engine or “ reverse image search ” . Though this work focuses on visual search , we note that ifX is the space of character sequences and Y is the space of webpages , this represents web search . If X are users and Y are the space of items , such as songs or news articles , the formalism represents a recommendation problem . In this work we aim to extract meaningful “ interpretations ” or “ explanations ” of the function d . 2.1 MODEL INTERPRETABILITY . The Bias-Variance trade-off ( Kohavi et al. , 1996 ) affects all machine learning systems and governs the relationship between a model ’ s expressiveness and generalization ability . In data-rich scenarios , a model ’ s bias dominates generalization error and increasing the size of the model class can improve performance . However , increasing model complexity can severely degrade its interpretability because added parameters can lose their connection to physically meaningful quantities . This is a challenge not just for classification and regression systems , but for search and recommendation architectures as well . For example , the best performing competitors on the Netflix prize dataset ( Bennett et al. , 2007 ) involve significantly more complex models than earlier baselines ( Rendle et al. , 2019 ) . Famously , the Netflix grand prize winner , the “ BellKor ” algorithm ( Koren , 2009 ) , boosts and ensembles several different underling methods making it difficult to interpret the model by inspecting its parameters . To tackle these challenges , some works introduce model classes that are naturally interpretable ( Nori et al. , 2019 ; Hastie & Tibshirani , 1990 ) . Alternatively , other works propose model-agnostic methods to explain the predictions of classifiers and regressors . Many of these approaches explain the local structure around a specific prediction . Lundberg & Lee ( 2017 ) show that the Shapley value ( Shapley , 1951 ) , a measure of fair credit assignment , provides a unique and axiomatically characterized solution to classifier interpretability ( SHAP ) . Furthermore , the authors show that Shapley values generalize LIME , DeepLIFT ( Shrikumar et al. , 2017 ) , Layer-Wise Relevance Propagation ( Bach et al. , 2015 ) , and several other methods ( Štrumbelj & Kononenko , 2014 ; Datta et al. , 2016 ; Lipovetsky & Conklin , 2001 ; Saabas , 2014 ) . Many works in computer vision use an alternative approach called Class Activation Maps ( CAMs ) . CAM projects the predicted class of a deep global average pooled ( GAP ) convolutional network onto the feature space to create a low resolution heatmap of class-specific network attention . GradCAM ( Selvaraju et al. , 2017 ) generalizes CAM to architectures other than GAP and yields robust interpretations with only a single network evaluation . In Section 4 we show that CAM , GradCAM , and their analogue for search engine interpretability , Zhu et al . ( 2019 ) , are also unified by the Shapley value and its second order generalization , the ShapleyTaylor index . 2.2 FAIR CREDIT ASSIGNMENT AND THE SHAPLEY VALUE . Shapley values provide a principled and axiomatic framework for classifier interpretation . We provide a brief overview of Shapley values but point readers to Molnar ( 2020 ) for more details . Shapley values originally arose in cooperative game theory as the only fair way to allocate the profit of a company to its employees based on their contributions . They have since become a standard method for explaining predictions of machine learning models . In particular , Shapley values measure of how much “ credit ” a model ’ s features receive for the prediction under a fair credit assignment . More formally , a “ coalition game ” is a set N of |N | players and a “ value ” function v : 2N → R. In cooperative game theory , this function v represents the expected payoff for a given cooperating coalition of players . Shapley ( 1951 ) show that the unique , fair credit assignment to each player , φv ( i ∈ N ) , can be calculated as : φv ( i ) : = ∑ S⊆N\ { i } |S| ! ( |N | − |S| − 1 ) ! |N | ! ( v ( S ∪ { i } ) − v ( S ) ) ( 1 ) Informally , this equation measures the average increase in value that a player i brings to a coalition S by weighting each increase , v ( S ∪ { i } ) − v ( S ) , by the number of ways this event could have happened during the formation of the “ grand coalition ” N . We note that this assignment , φv , is the unique assignment that satisfies four reasonable properties : symmetry under player re-labeling , no credit assignment to dummy players , linearity ( or it ’ s alternative monotonicity ) , and efficiency which states that Shapley values should sum to v ( N ) − v ( ∅ ) ( Young , 1985 ) . Intuitively , these axioms require that a fair explanation should treat every feature equally ( Symmetry ) , should not assign importance to features that are not used ( Dummy ) , should behave linearly when the value function is transformed ( Linear ) , and should sum to the function ’ s value ( Efficiency ) . To connect this work to model interpretability , we can identify the “ features ” used in a model as the “ players ” and interpret the value function , v ( S ) , as the expected prediction of the model when featuresN \S are replaced by values from a “ background ” distribution . This background distribution allows for “ counterfactual ” or relative explanations ( Goyal et al. , 2019 ) . 3 RELATED WORK . The field of model interpretability has a considerable body of literature , and we mention just a handful of the works that are particularly related . One of our baseline methods , Dong et al. , was one of the first to present a generic visual search engine explanation reminiscent of a Parzen-Window based estimator . Fong & Vedaldi ( 2017 ) introduce a method for explaining classifiers based on meaningful pertubation and Chefer et al . ( 2021 ) introduce a method for improving interpretation for transformer-based classifiers . Zhu et al . ( 2019 ) lifted CAM to search engines and we find that our Shapley-Taylor based method aligns with their approach for GAP architectures . Singh & Anand ( 2019 ) and Fernando et al . ( 2019 ) use LIME and DeepSHAP to provide first-order interpretations of text but do not apply their methods to images . Ancona et al . ( 2019 ) introduce a distribution propagation approach for improving the estimation of Shapley Values for deep models and can be combined with our approach . Many works implicitly use components that align with ShapleyTaylor indices for particular functions . Works such as Fischer et al . ( 2015a ) ; Sun et al . ( 2020 ) ; Wang et al . ( 2020a ) ; Chen et al . ( 2020c ) ; Hou et al . ( 2019 ) use feature correlation layers to estimate and utilize correspondences between images . We show these layers are equivalent to Shapley-Taylor indices on the GAP architecture , and this allows create a correlation layer that handles counterfactual backgrounds . Other recent works have used learned co-attention within transformer architectures to help pool and share information across multiple domain types ( Wei et al. , 2020 ) . Fu et al . ( 2020 ) attempt to learn a variant of GradCAM that better aligns with axioms similar to Shapley Values by adding efficiency regularizers . The method is not guaranteed to satisfy the axioms but is more “ efficient ” . We rely on several works to extend Shapley values to more complex interactions . Harsanyi ( 1963 ) generalized the Shapely value by introducing a “ dividend ” that , when split and distributed among players , yields the Shapley values . Owen ( 1972 ) introduces an equivalent way to extend Shapley values using a multi-linear extension of the game ’ s characteristic function . Sundararajan et al . ( 2020 ) introduce the Shapley-Taylor index and show is equivalent to the Lagrangian remainder of Owen ’ s multi-linear extension . Integrated Hessians ( Janizek et al. , 2020 ) enable estimation of a secondorder variant of the Aumann-Shapley values and we use this approach to create a more principled second-order interpretation method for differentiable search engines .

Summary: This paper proposes a unified framework to explain the current model explaining algorithms. Under this framework, the authors discovered the current algorithms are approximating second-order Shapley-Taylor indices, where they also proposed a fast-kernel-based estimation method. Empirically, the authors benchmarked different algorithms in two large segmentation datasets. They showed the proposed algorithms achieved better faithfulness and inefficiency.