Analyzing the Effects of Classifier Lipschitzness on Explainers

1 INTRODUCTION . Machine learning models have improved over time at prediction and classification , especially with the advances made in deep learning and availability of large amounts of data . These gains in predictive power have often been achieved using increasingly complex and black-box models . This has led to significant interest in , and a proliferation of , explanation models that provide explanations for the predictions made by these black-box models . We focus our work in analyzing the behaviour of these explanation models . In particular , we explore when are explainers reliable by analyzing the connection between the robustness of explainer methods and the smoothness of the black-box functions they are trying to explain . We propose and formally define explainer astuteness – a property of explainers which captures the probability that a given method provides similar explanations to similar data points . We then provide a theoretical way to connect this explainer astuteness to the probabilistic Lipschitzness of the black-box function that is being explained . Since probabilistic Lipschitzness is a measure of the probability that a function is smooth in a local neighborhood , our results demonstrate how the smoothness of the black-box function itself impacts the astuteness of the explainer . This implies that enforcing smoothness on black-box functions lends them to more robust explanations . Related Work . A wide variety of explainers have been proposed in the literature ( Guidotti et al. , 2018 ; Arrieta et al. , 2020 ) . Explainers can broadly be categorized as feature attribution or feature selection explainers . Feature attribution explainers provide continuous valued importance scores to each of the input features , while feature selection explainers provide binary decisions on whether a feature is important or not . Some popular feature attribution explainers can be viewed through the lens of shapley values such as SHAP ( Lundberg & Lee , 2017 ) , LIME ( Ribeiro et al. , 2016 ) and LIFT ( Shrikumar et al. , 2016 ) . Some models such as CXPlain ( Schwab & Karlen , 2019 ) , PredDiff ( Zintgraf et al. , 2017 ) and feature ablation explainers ( Lei et al. , 2018 ) calculate feature attributions by simulating individual feature removal , while other methods such as RISE ( Petsiuk et al. , 2018 ) calculate the mean effect of a feature ’ s presence to attribute importance to it . Feature selection methods , on the other hand include individual selector approaches such as L2X ( Chen et al. , 2018 ) and INVASE ( Yoon et al. , 2018 ) , and group-wise selection approaches such as gI ( Masoomi et al. , 2020 ) . These models , while seemingly diverse have been shown to have striking underlying simi- larities , for example Lundberg & Lee ( 2017 ) unify six different explanation models under a single umbrella . Recently , Covert et al . ( 2020 ) went a step ahead and combined 25 existing methods under the overall class of removal-based explanation models . Similarly , there has been a recent increase in research focused on analyzing the behaviour of these explanation models themselves in ways similar to how classification models have been analyzed . Recent work has focused on dissecting various properties of explainers . Yin et al . ( 2021 ) propose stability and sensitivity ( Yin et al. , 2021 ) as measures of faithfulness of explainers to the decisionmaking process of the black-box model . Li et al . ( 2020 ) explore connections between local explainability and model generalization . Ghorbani et al . ( 2019 ) poke at the robustness of explainers to systemic and adversarial perturbations . Alvarez-Melis & Jaakkola ( 2018 ) empirically show that robustness , in the sense that explainers should provide similar explanations for similar inputs , is a desirable property and how forcing this property yields better explanations . Recently , Agarwal et al . ( 2021 ) explore the robustness of LIME ( Ribeiro et al. , 2016 ) and SmoothGrad ( Smilkov et al. , 2017 ) , and prove that for these two methods their robustness is related to the maximum value of the gradient of the predictor function . Our work is closely related to Alvarez-Melis & Jaakkola ( 2018 ) and Agarwal et al . ( 2021 ) on explainer robustness . However , as opposed to only forcing explainers to be robust themselves ( Alvarez-Melis & Jaakkola , 2018 ) , our theoretical results suggest that ensuring robustness of explanations also depends on the smoothness of the black-box function that is being explained . Our results are complementary to the results obtained by Agarwal et al . ( 2021 ) in that our theorems cover a wider variety of explainers ( see contributions below ) . We further relate robustness to probabilistic Lipschitzness of blackbox functions which is a quantity that can be empirically estimated . Contribution : • We formalize and define explainer astuteness which captures the probability that a given explainer provides similar explanations to similar points . This formalism allows us to theoretically analyze robustness properties of explainers . • We provide theoretical results that connect astuteness of three classes of explainers ; shapley value based ( e.g . SHAP ) , explainers that simulate mean effect of features ( e.g . RISE ) , and explainers that simulate individual feature removal ( e.g . CXPlain ) , to the smoothness of the black-box function . Our results suggest that enforcing Lipschitzness on black-box functions can result in explainers providing more astute explanations . Formally our theorems establish a lower bound on explainer astuteness that depends on the Lipschitness of the black-box function and square root of data dimensionality . Figure 1 summarizes this main contribution of our work . • We demonstrate experimentally that this lower bound indeed holds in practice by comparing the astuteness predicted by our theorems to the observed astuteness on simulated and real datasets . The remainder of this paper is organized as follows : we first provide background on the type of explanation methods we restrict ourselves to , i.e . removal based feature explainers , and the summary of notation used in this paper in Section 2 . We then provide formal definitions and theorems with proofs in Section 3 . Finally , we conclude the paper by providing experimental results in Section 4 . 2 BACKGROUND AND NOTATIONS . 2.1 REMOVAL-BASED FEATURE EXPLAINERS . As mentioned in Section 1 a wide variety of explainer methods has been introduced . Owing to this diversity , in this work , we concern ourselves with removal-based feature attribution explainers as defined by Covert et al . ( 2020 ) . Removal based feature attribution explainers are methods that define a feature ’ s influence through the impact of removing it from a model and assign continuous valued scores to each feature signifying its importance . This includes popular approaches such as SHAP and SHAP variants including KernelSHAP , LIME , DeepLIFT ( Lundberg & Lee , 2017 ) , mean effect based methods such as RISE ( Petsiuk et al. , 2018 ) , and individual effects based methods such as CXPlain ( Schwab & Karlen , 2019 ) , PredDiff ( Zintgraf et al. , 2017 ) , permutation tests ( Strobl et al. , 2008 ) , and feature ablation explainers ( Lei et al. , 2018 ) . All of these methods simulate feature removal either explicitly or implicitly . For example , SHAP explicitly considers effect of using subsets that include a feature as compared to the effect of removing that feature from the subset . RISE removes subsets of features while always keeping the feature that is being evaluated , and estimates the average effect of keeping that feature when other features are randomly removed . CXPlain explicitly considers the impact of removing a feature on the loss function used in training the predictor function . 2.2 NOTATION . Before formalizing explainer astuteness and our theoretical results , we explain the notations used in the rest of the paper . We denote d-dimensional input data as x ∈ Rd , from a given dataset D. The black-box predictor function is denoted by f , where f ( x ) is the prediction given x , this function is assumed to have been trained on the given D. The explainer is represented by a function φ where φ ( x ) ∈ Rd is the feature attribution vector representing attributions for all features in x while φi ( x ) ∈ R is the attribution for the ith feature . To simulate the presence or absence of features in a given subset of features , we use an indicator vector z ∈ { 0 , 1 } d , where zi = 1 when the ith feature is present in the subset . To indicate we are only using subsets where feature zi = 1 , we use z+i ; and to indicate only using subsets where feature zi = 0 , we use z−i . Lastly , the p-norm induced distance between any two points x , x′ is denoted by dp ( x , x′ ) . 3 EXPLAINER ASTUTENESS . Our main interest is in defining a metric that can capture the difference in explanations provided by an explainer to points that are close to each other in the input space . The same question has been asked for classifiers and Bhattacharjee & Chaudhuri ( 2020 ) came up with the concept of Astuteness of classifiers , which captures the probability that similar points are assigned the same label by a classifier . Formally they provide the following definition : Definition 1 . Astuteness of Classifiers ( Bhattacharjee & Chaudhuri , 2020 ) : The astuteness of a classifier f over dataset D , denoted as Ar ( f , D ) is the probability that ∀x , x′ ∈ D such that d ( x , x′ ) ≤ r the classifier will predict the same label . Ar ( f , D ) = Px , x′∼D [ f ( x ) = f ( x′ ) |d ( x , x′ ) ≤ r ] ( 1 ) The obvious difference in trying to adapt this definition of astuteness to explainers is that explanations for nearby points do not have to be exactly the same . Keeping this in mind , we propose and formalize explainer astuteness , as the probability that the explainer method assigns similar explanations to similar points . The formal definition is as follows : Definition 2 . Explainer astuteness : The explainer astuteness of an explainer E over dataset D , denoted as Ar , λ ( E , D ) is the probability that ∀x , x′ ∈ D such that dp ( x , x′ ) ≤ r the explainer E will provide explanations that are at most λ ∗ dp ( x , x′ ) away from each other , where λ ≥ 0 Ar , λ ( E , D ) = Px , x′∼D [ dp ( φ ( x ) , φ ( x ) ) ≤ λ ∗ dp ( x , x′ ) ∣∣∣ dp ( x , x′ ) ≤ r ] ( 2 ) A critical observation about definition 2 is that it not only relates to the previously defined notion of classifier astuteness , but also connects to the concept of probabilistic Lipschitzness . Probabilistic Lipschitzness captures the probability of a function being locally smooth given a radius r. It is specially useful for capturing a notion of smoothness of complicated neural network functions for which enforcing global and deterministic Lipschitzness is difficult . Mangal et al . ( 2020 ) formally defined probabilistic Lipchitzness as follows : Definition 3 . Probabilistic Lipschitz ( Mangal et al. , 2020 ) : Given 0 ≤ α ≤ 1 , r ≥ 0 , a function f : X→ R is probabilistically Lipschitz with a constant L ≥ 0 if Px , x′∼D ( dp ( f ( x ) , f ( x′ ) ) ≤ L ∗ dp ( x , x′ ) ∣∣∣ dp ( x , x′ ) ≤ r ) ≥ 1− α ( 3 ) Notice the similarity between definitions 2 and 3 . This very correspondence forms the basis of our motivation to explore the connection between the Lipschitzness of the black-box prediction function , and the astuteness of the explainer , which we explore theoretically in this section and experimentally in 4 .

Summary: In this study, a metric called Astuteness is introduced as a criterion to evaluate the reliability of explanations. Intuitively, if the distance between explanations given for two samples that are within a certain distance is also within a certain distance, then the explanation has astuteness. This definition can be interpreted as defining the robustness of an explanation. More specifically, it relates the distance in the sample space to the distance in the explanation space by probabilistic Lipschitzness. In this study, the authors show that for some certain classes of explanatory methods, such as SHAP and CXplain, the probabilistic lower bound of the astuteness of an explanation depends on the square root of the number of feature dimensions D and the Lipschitz constant L. The reliability of this probabilistic lower bound is also evaluated experimentally with several datasets.