Analyzing the Effects of Classifier Lipschitzness on Explainers
Authors: Zulqarnain Khan
Aria Masoomi
Davin Hill
Jennifer Dy
Venue: Submitted to ICLR 2023 (2023)
Keywords: Explainers
Explanation
Robustness
Astuteness
Lipschitz
Blackbox
Classifiers

Abstract: Machine learning methods are getting increasingly better at making predictions, but at the same time they are also becoming more complicated and less transparent. As a result, explainers are often relied on to provide interpretability to these \textit{black-box} prediction models. As crucial diagnostics tools, it is important that these explainers themselves are reliable. In this paper we focus on one particular aspect of reliability, namely that an explainer should give similar explanations for similar data inputs. We formalize this notion by introducing and defining \textit{explainer astuteness}, analogous to astuteness of classifiers. Our formalism is inspired by the concept of \textit{probabilistic Lipschitzness}, which captures the probability of local smoothness of a function. For a variety of explainers (e.g., SHAP, RISE, CXPlain), we provide lower bound guarantees on the astuteness of these explainers given the Lipschitzness of the prediction function. These theoretical results imply that locally smooth prediction functions lend themselves to locally robust explanations. We evaluate these results empirically on simulated as well as real datasets.

Under review as a conference paper at ICLR 2023
ANALYZING THE EFFECTS OF CLASSIFIER LIPSCHITZ- NESS ON EXPLAINERS
Anonymous authors Paper under double-blind review
ABSTRACT
Machine learning methods are getting increasingly better at making predictions, but at the same time they are also becoming more complicated and less transparent. As result, explainers are often relied black-box prediction models. As crucial diagnostics tools, it is important that these explainers themselves are reliable. this paper we focus on one particular aspect of reliability, namely that an explainer should give similar explanations for similar data inputs. We formalize this notion by introducing and defining explainer astuteness, analogous to astuteness of classifiers. Our formalism is inspired by the concept of probabilistic Lipschitzness, which captures the probability of local smoothness of function. For variety of explainers (e.g., SHAP, RISE, CXPlain), we provide lower bound guarantees on the astuteness of these explainers given the Lipschitzness of the prediction function. These theoretical results imply that locally smooth prediction functions lend themselves to locally robust explanations. We evaluate these results empirically on simulated as well as real datasets.
these provide interpretability
INTRODUCTION
Machine learning models have improved over time at prediction and classification, especially with the advances made in deep learning and availability large amounts of data. These gains predictive power have often been achieved using increasingly complex and black-box models. This has led to significant interest in, and a proliferation of, explainers that provide explanations for the predictions made by these black-box models. Given the crucial importance of these explainers it is imperative to understand what makes them reliable.
In this paper we focus on explainer robustness. A robust explainer is one where similar inputs results in similar explanations (Alvarez-Melis & Jaakkola, 2018). For example, consider two patients given the same diagnosis in a medical setting. These two patients share identical symptoms and are demographically very similar, therefore a diagnostician would expect that factors influencing the model decision should be similar well. Prior work in explainer robustness suggests that this expectation does not always hold true (Alvarez-Melis & Jaakkola, 2018: 8: Ghorbani et al., 2019); small result in large shifts in explanation. For this reason we investigate
changes to the input samples the theoretical underpinning of explainer robustness. Specifically, we focus on investigating the connection between explainer robustness and smoothness of the black-box function being explained.
We propose and formally define explainer astuteness a property of explainers which captures the probability that given method provides similar explanations similar data points. This definition allows us to evaluate the robustness for given explainer over the entire dataset and helps tie explainer robustness to probabilistic Lipschitzness of classifiers. We then provide theoretical way connect this explainer astuteness to the probabilistic Lipschitzness of the black-box function that is being explained. Since probabilistic Lipschitzness a measure of the probability that a function is smooth in local neighborhood, our results demonstrate how the smoothness of the black-box function itself impacts the astuteness the explainer. This implies that enforcing smoothness on black-box functions lends them more robust explanations.
Related Work. A wide variety of explainers have been proposed in the literature (Guidotti et al., 2018; Arrieta et al., 2020). Explainers can broadly be categorized as feature attribution or feature selection explainers. Feature attribution explainers provide continuous-vas valuued importance scores to each of the input features, while feature selection explainers provide binary decisions whether

---

Under review as a conference paper at ICLR 2023
feature is important not. Some popular feature attribution explainers can be viewed through
the lens of Shapley values such as SHAP (Lundberg & Lee, 2017), LIME (Ribeiro et al., 2016) and LIFT (Shrikumar et al., 2016). Some models such as CXPlain (Schwab & Karlen, 2019), PredDiff (Zintgraf et al., 2017) and feature ablation explainers (Lei et al., 2018) calculate feature attributions by simulating individual feature removal, while other methods such RISE (Petsiuk et al., 2018) calculate the mean effect of a feature's 's presence to attribute importance to it. In contrast, feature selection methods include individual selector approaches such as L2X (Chen et al., 2018) and INVASE (Yoon et al., 2018), and group-wise selection approaches such as gl (Masoomi et al., 2020). While seemingly diverse, these models have been shown to have striking underlying similarities, for example, Lundberg & Lee (2017) unify six different explainers under a single framework. Recently, Covert et al. (2020) went a step further and combined 25 existing methods under the overall class of removal-based explainers.
Similarly, there has been a recent increase in research focused on analyzing the behaviour of these explainers themselves in ways similar to how classification models have been analyzed. Recent work has focused on dissecting various properties of explainers. Yin et al. (2021) propose stability and sensitivity as measures of faithfulness of explainers to the box model and empirically demonstrate the usefulness of these measures. Li et al. (2020) explore connections between local explainability and model generalization. Ghorbani al. (2019) test the robustness of explainers through systemic and adversarial perturbations. Agarwal et al. (2022) define and discuss theoretical guarantees around faithfulness and stability in the context of Graph Neural Networks, Our definition of astuteness is related to what they call stability, but defined as a probability over all available instances in such a way that connection to probabilistic Lipschitzness of the classifier becomes clear. Alvarez-Meli & Jaakkola (2018) empirically show that robustness, in the sense that explainers should provide similar explanations for similar inputs, is a desirable property and how forcing this property yields better explanations. Recently, Agarwal et al. (2021) explore the robustness of LIME (Ribeiro et al., 2016) and SmoothGrad (Smilkov et al., 2017), and prove that for these two methods their robustness is related to the maximum value of the gradient of the predictor function. Our work is closely related to Alvarez-Melis & Jaakkola (2018) and Agarwal et al. (2021) on explainer robustness. However, instead of enforcing explainers to be robust themselves (Alvarez-Melis & Jaakkola, 2018), our theoretical results suggest that ensuring robustness of explanations also depends on the smoothness of the black-box function that is being explained. Our results are complementary to the results obtained by Agarwal et al. (202 I) in that our theorems cover wider variety of explainers compared to only Continuous LIME and SmoothGrad (see contributions below). We further relate robustness to probabilistic Lipschitzness of black-box models, which is a quantity that can be empirically estimated.
process of the black-
Additionally, there has been recent work estimating upper-bounds of Lipschitz constant for neural networks (Virmaux & Scaman, 2018; Fazlyab et al., 2019; Gouk et al., 2021), and enforcing Lipschitz continuity during neural networks training, with an eye towards improving classifier robustness (Gouk et al., 2021; Aziznejad et al., 2020; Fawzi et al., 2017; Alemi et al., 2016). Fel et al. (2022) empirically demonstrated that 1 I-Lipschitz networks are better suited as predictors that are more explainable and trustworthy. Our work provides crucial additional motivation for that line of research; i.., it provides theoretical reasons to improve Lipschitzness of neural networks from the perspective of enabling more robust explanations.
Contributions:
We formalize and define explainer astuteness which captures the probability that a given explainer provides similar explanations to similar points. This formalism allows us to theoretically analyze robustness properties of explainers.
We provide theoretical results that connect astuteness of explainers to the smoothness of the black-box function they are providing explanations on. Our results suggest that smooth black-box functions result in explainers providing more astute explanations. While this statement is intuitive, proving it is non-trivial and requires additional assumptions for different explainers (See Section 3.2).
Specifically we prove this result for astuteness of three classes of explainers: (1) Shapley value based (e.g. SHAP), (2) explainers that simulate mean effect of features (e.g. RISE), and (3) explainers that simulate individual feature removal (e.g. CXPlain). Formally, our theorems establish a lower bound on explainer astuteness that depends on the Lipschitzness
2

---

Under review as a conference paper at ICLR 2023
Input space
Prediction space
Explanation space
Low Lipschitz Constant
160
(x,,0)
x' X (,00,)))
Higher Lipschitz Constant (()
660
600
M 1gd, Iddp(xxx))
In this figure we visualize the implication of our theoretical results. For a black-box prediction function that is locally Lipschitz with constant L1, the predictions for any two points x such that (2, x²) S are within L (x, distance from each other. Given such a prediction function, the explanation for the same data points are also expected be within (1,)(1, of each other where A = CL1 d where
Figure I:
a constant. we consider a second black-box function with L2 L1 that results in A2 λ1, indicating that the explanations for this black-box function actually end up being farther apart as compared to the first prediction function. This result implies that locally smooth black-box functions lend themselves to more astute (i.e., robust) explanations.
of the black-box function and square root of data dimensionalityy. Figure 1 summarizes this main contribution of our work.
We demonstrate experimentally that this lower bound indeed holds in practice by comparing the astuteness predicted by our theorems to the observed astuteness on simulated and real datasets. We also demonstrate experimentally that the same neural network when trained with Lipschitz constraints lends itself to more astute explanations compared to when is trained with no constraints.
2 BACKGROUND AND NOTATIONS
2.1 REMOVAL-BASED FEATURE EXPLAINERS
As mentioned in Scction 1, there exist a wide variety of explainers. Owing to this diversity, in this work, concern ourselves with removal-based feature attribution explainers defined by Covert et al. (2020) (which showed 25 existing methods under this umbrella). Removal based feature attribution explainers
methods that define feature's influence through the impact of removing from a model and assigning cidinuous-valuee scores to each feature signifying its importance. This includes popular approaches such SHAP and SHAP variants including KernelSHAP, LIME, DeepLIFT (Lundberg & Lee, 2017), mean effect based methods such as RISE (Petsiuk et al., 2018), and individual effects based methods such as CXPlain (Schwab Karlen, 2019), PredDiff (Zintgraf et al., 2017). permutation tests (Strobl et al., 2008), and feature ablation explainers (Lei et al., 2018). All of these methods simulate feature removal either explicitly or implicitly. For example, SHAP explicitly considers effect of using subsets that include feature as compared to the effect of removing that feature from the subset. RISE removes subsets of features while always keeping the feature that is being evaluated, and estimates the average effect of keeping that feature when other features are randomly removed. CXPlain explicitly considers the impact removing a feature on the loss function used in training the predictor function.
2.2 NOTATION
We denote d-dimensional input data as æ E Rd, from a data distribution D. The black-box predictor function is denoted by f. where f(x) is the prediction given æ, this function is assumed to have been trained on the training samples from D. The explainer is represented by a function where ((x) Rd is the feature attribution vector representing attributions for all features in ø while (x) E IR is the attribution for the ith feature. To simulate the presence or absence features in a indicator vector (0, 1)d, where z; when the tth feature is
given subset of features,
3

---

Under review as a conference paper at ICLR 2023
present in the subset. To indicate we only using subsets where feature 2; 1, 246 and to indicate only using subsets where feature 2 = 0, we use 2-5. Lastly, the p-norm induced distance between any two points 2, x' is denoted by dz 2') || z'Ipp. where I|-IIp is the p-norm.
3 EXPLAINER ASTUTENESS
Our main interest is in defining a metric that can capture the difference in explanations provided by an explainer to points that are close each other in the input space. The same question has been asked for classifiers. Bhattacharjee & Chaudhuri (2020) came up with the concept of Astuteness of classifiers, which captures the probability that similar points are assigned the same label by a classifier. Formally they provide the following definition:
Definition 1. Astuteness of classifiers (Bhattacharjee & Chaudhuri, 2020): The astuteness of a classifier over D, denoted as (f, D) is the probability that Vx, x' E 'D such that d(z, S r the classifier will predict the same label.
A. P...'=//((' = ((x)) s]
(1)
The obvious difference in trying to adapt this definition of astuteness to explainers is that explanations for nearby points do not have be exactly the same. Keeping this in mind, we propose and formalize explainer astuteness, as the probability that the explainer assigns similar explanations to similar points. The formal definition as follows:
Definition 2. Explainer astuteness: The explainer astuteness of explainer F. over D, denoted as A (F, D) the probability that Vxr x' E DD such that (xx x') S r the explainer will provide explanations ((x), 6(x") that most X d2(2,, x²) away from each other, where >
A,,(E, D) = Pa, pp.p' (d)))(()) ((x))) S X do(x,x)) dp(x, x') s|
(2)
A critical observation about definition 2 is that it not only relates to the previously defined notion of classifier astuteness, but also connects to the concept of probabilistic Lipschitzness. Probabilistic Lipschitzness captures the probability of function being locally smooth given a radius r. It is specially useful for capturing notion of smoothness of complicated neural network functions for which enforcing global and deterministic Lipschitzness is difficult. Mangal et al. (2020) formally defined probabilistic Lipschitzness as follows:
Definition 3. Probabilistic Lipschitzness (Mangal al., 2020): Given 0 S a S 1, 2 a function f: X + R is probabilistically Lipschitz with a constant 2 0f
I "=0")))4(((0))... f(x)) S L do(2,x') do((, Sr)] 1-a
(3)
3.1 THEORFTICAL BOUNDS OF ASTUTENESS
A cursory comparison between equation 2 and equation 3 hints at the two concepts being related to each other. In fact, explainer astuteness can be viewed as probabilistic Lipschitzness of the explainer when it is viewed question to explore is how the astuteness of explainers is connected to the Lipschitzness of the black-box model they are trying to explain. We introduce and prove the following theorems which provide theoretical bounds that connect the Lipschitz constant L of the black-box model to the astuteness of various explainers including SHAP (Lundberg Lee, 2017), RISE (Petsiuk et al., 2018), and methods that simulate individual feature removal such as CXPlain (Schwab & Karlen, 2019).
function with a Lipschitz constant x. However, much more interesting
3.1.1 ASTUTENESS OF SHAP
SHAP (Lundberg & Lee, 2017) one of the most popular feature attribution based explainers in use today. Lundberg & Lee (2017) unify 6 existing explanation approaches within the SHAP framework. Each of these explanation approaches (such as DeepLIFT and kernelSHAP) can be viewed as approximations of SHAP, since SHAP in its theoretical form is difficult to calculate. However, in this section we use the theoretical definition of SHAP to establish bounds on astuteness.
For given data point æ E X and a prediction function f, the feature attribution provided by SHAP for the th feature is given by:
(x(x)=
Σ
12 (!dd - |2_ 1M ((x -
(4)
d!
4

---

Under review as a conference paper at ICLR 2023
Before moving to the actual theorem, introduce and prove the following Lemma which is necessary for the proof of Theorem 3.1. Lemma 1. If.
Pa.
s L do(x,=²) do(a,x') 21/21
then for y x 2 24. x' o 244 i. v.u. E UN- tyly E Ilello k # 0} for
k d
P Pa.'' (l) f(/)) & L dv. dz(..() < 2 1-3
where 3 2 a assuming that the distribution D is defined for all x and and the equality is approached if the probability of sampling points from the set 14, fyly Rd llyla k, # 0} approaches zero for k = 2..... d relative to the probability of sampling points from N1.
Proof. (Sketch, full proof in Appendix A)
Assume Pk: is the probability input space and 7% is the probability of the set of points that violate Lipschitzness the set N₃- In finite case each set Nş can be mapped
occurrence of the set N₂ = {x|æ R =, * 0} in the
set N/ of cardinality 2d* after masking with all 2d-2pe possible 2+4. In probability terms, the probability of N₂ be written as p/e 2 'p, Let be the the proportion of points in all Nú that also violate Lipschitzness in their unmasked form then can be written as
Σボ= 2 PLTR
Σj=1 'p,
Considering worse case 33 requires solving the following equation,
Ei 2 PI Σ 2-
2
8
max 71 odd
pry = 0,0 l
(5)
The result of this maximization will be 8 2 a. In the specific case where pk for k. 2, (i.e., where the probability of sampling any with a 0 valued element is 0), 3 + a.
Theorem 3.1. (Astuteness of SHAP) Consider given 2 0 and O S S 1, and a trained predictive function f that is probabilistic Lipschitz with a constant radius r measured using dp(-;.) and with probability at least 1 Then for SHAP explainers have astuteness Ar.
for
X = 2/.L. Where B 2 a, and - under conditions specified in Lemma 1.
Proof. Given input and another input x' s.t. d(x,x') x') S
And letting
[= :iddd1 12 :1-12 12
O2,
Using equation we can write,
(,(0(()), = =11
fxx02 Σ Calf(e' (2))) oz.)]llp (6)
Combining the two sums and re-arranging the R.H.S.
d,(6)(z), (x)2)
Σ C= 24x) f(z' @ 24t) -=-
(7)
Using triangular inequality the R.H.S twice,
((((()) = c. - f(x' @ 2x)ll + Σ
Σ C.ll((roz))) - f(x' 2+)) C (ol(aao020 - f(x 2.)bbb
We can replace each value inside the sums in equation 8 with the maximum value across either sums. Doing would still preserve the inequality in equation 8, the sum of values is always less than
5

---

Under review as a conference paper at ICLR 2023
the maximum among those summed times. Without loss of generality let us assume this maximum [/(r - f( o æit) for some particular z", This gives us:
(2(01()). s 11
f( 2te)ll
I (9)
However, Σ._, C2 = E... b Bo = 1, which gives us, (()(())) & 21l o 21,) f(a' rI,)
o oz..)) (10)
Using the fact that f is probabilistic Lipschitz with a given constant L 0, d2 x3) s. dr 2400 o =),) s dp (2, (0,z) æ') and Lemma 1. We get:
(fz ii,) o zi.) - 2L 2L d,(e,,,2)))
Since equation 10 establishes that (o)( oz 2d o 24:), o =t,)), the below inequality can be now established:
d. & 2L dz x')l 2 3
(1I)
Note that equation 11 is true for each feature E {1, d}. To conclude our proof, we note that
(r v) m wil s max bz wipp Vdmax ( (2),
Utilizing this in equation 11, without loss of generality assuming dx(d)(x), (x())) corresponds to the maximum, gives us:
P ((l(((2),)('"))) $ 22 dx(x, 1-3
(12)
Since Pldp(o(x), ((x))) s 2 (x,,,²)) in equation 12 defines Ax, for A 207L, this concludes the proof.
Corollary 1. If the puuntcioonfunccionn f is locally deterministically Lipschhht (a = 0) at radius r then Shapley explainers are X-astule for radius r 2 0 for X = 247L
Proof. Note that definition reduces to the definition of deterministic Lipschitz if 0. Which means equation 12 will be true with probability 1. Which concludes the proof.
D
3.1.2 ASTUTENESS OF "REMOVE INDIVIDUAL" " EXPLAINERS
Within the framework of feature removal explainers, a sub-category is the explainers that work by removing single feature from the set of all features and calculating feature attributions based on change in prediction that result from removing that feature. This category includes Occlusion, CXPlain (Schwab & Karlen, 2019), PredDiff (Zintgraf al., 2017) Permutation tests (Strobl et al., 2008), and feature ablation explainers (Lei et al., 2018).
"Remove individual" explainers determine feature explanations for the 4th feature by calculating the difference in prediction with and without that feature included for given point x. Let 2 E {0,1) represent a binary vector with 2, = 0, then the explanation for feature can be written as:
((z.)= f(x) f(x 2-.)
(13)
Theorem 3.2. (Astuteness of Remove individual explainers) Consider given and 0 S 1 and a trained predictive function f that is locally probabilistic Lipschitz with a constant L, radius measured using dp(...) and probability at least a. Then for Remove individual explainers, we have the astuteness Ar, 2 a. for X 2 2/dL, where d the dimensionality of the data.
Proof. (Sketch, full proof in Appendix A) By considering another point x' such that dx(x, Sr and equation we get,
dr(((z:), o(x))) (f( f(x (-1), f(x' @ 2-)))
(14)
then following the exact same steps the proof for Theorem 1 i.e. writing the right hand side in terms of p-norm, utilizing triangular inequality, and the definition of probabilistic Lipschitzness leads us to the desired result.

---

Under review as a conference paper at ICLR 2023
If the prediction function f is localty L Lipschitz radius then remove individual explanations a-atue for radius and À = 2 VdL.
Corollary 2.
Proof. Same proof for Corollary 2.1.
3.1.3 ASTUTENESS OF RISE
RISE determines feature explanation for the 1th feature by sampling subsets of features and then calculating the mean value of the prediction function when feature included in the subset. RISE feature attribution for given point æ and feature i for prediction function f can be written as:
de (x) Bp 3 2.]
(15)
The following theorem establishes the bound on À for explainer astuteness of RISE in relation the Lipschitzness of black-box prediction function.
Theorem 3.3. (Astuteness of RISE) Consider a given r 2 and S a S 1, and a trained predictive function f that is locally deterministically Lipschitz with a constant L (a 0), radius measured using dp(..-) and probability at least 1 - a. Then for RISE explainer is -aastute for radius r and
X == VdL.
Proof. (Sketch, full proof in Appendix A)
Given input and another input x' s.t. d(x, S using equation 15 we can write
dp(d:(z), (x')) (p 2)] Ep(als.=)](((' 2 +))) = ||Bpt= @ =)] -pp @ =)][|r = [| O f(a' o z)]lk
(16)
Using Jensen's 's inequality on R.H.S followed by the fact that Elf] S max
dp(ó. (x), (x')) S max @ f(x' + 2))
(17)
Using the fact that f is is deterministically Lipschitz and dp(0(z).((z)) max; (z(()()). gives us,
Vd
P(d)( s VaL d,(,,
(18)
Since ((())) Vdl-d, xf) defines Ax, for A Vdl, this concludes the proof.
m
3.2 IMPLICATIONS
The above theoretical results all provide the same critical implication, that is, explainer astutness is lower bounded by the Lipschitzness the prediction function. This means that black-box classifiers that are locally smooth (have a small L at given radius r) lend themselves to probabilistically more robust explanations. This work provides the theoretical support on the importance of enforcing smoothness of classifiers to astuteness of explanations. Note that while this implication makes intuitive sense, proving it for specific explainers is non-trivial as demonstrated by the three theorems above. The statement holds true for all three explainers when the classifier can be assumed to be deterministically Lipschitz, the conditions under which it is still true for probabilistic Lipschitzness vary in each case. For Theorem 3.1 we have assume that distribution D is defined over masked data in addition to the input data and ideally the probability of sampling of masked data from is significantly smaller compared to probability of sampling points with value exactly equal O. For Theorem 3.2 the statement is true without additional assumptions. For Theorem 3.3 we can only prove the statement be true for the detereminsitic case.
EXPERIMENTS
To demonstrate the validity of our theoretical results, we perform series of experiments. We train four different classifiers on each of five datasets, and then explain the decisions of these classifiers using three explainers.
We utilize three simulated datasets introduced by (Chen et al., 2018) namely Orange Skin Nonlinear Additive(NA) and Switch, and two real world datasets from UCI Machine Learning
7

---

Under review as a conference paper at ICLR 2023
repository (Asuncion & Newman, 2007) namely Rice (Cinar& & Koklu, 2019) and Telescope (Ferenc et al., 2005). Details for these datasets can be found in Appendix B.
A two-layer MLP with ReLU activations. For simulated datasets each layer has 200 neurons, while for the 2 real datasets we use 32 neurons in each layer. 4layer: A four-layer MLP with ReLU activations, with the same number of neurons per layer as 2layer. linear: A linear classifier. svm: A support vector machine with Gaussian kernel. The idea here is that each of these classifiers will have different Lipschitz behavior, and that should lower bound the explainer astuteness when explaining each of these classifiers according to our theoretical results.
For each dataset train the following four classifiers; 2layer:
We evaluate 3 explainers here that are representative of our 3 theorems. Gradient based approximation and kernel shap approximation of SHAP(Lundberg & Lee, 2017) for the NN classifiers and SVM respectively serve as representative of Theorem 3.1. Both are included in the implementation provided by the authors. We modify the implementation of RISE(Petsiuk et al., 2018) provided by the authors for image datasets to work with tabular datasets, this serves as representative for Theorem 3.3. Implementation provided by authors for CXPlain (Schwab & Karlen, 2019) serves as representative for Theorem 3.2.
4.1 EFFECT OF LIPSCHITZ CONSTRAINTS ON EXPLAINER ASTUTENESS
Following Gouk et al. (2021)'s proposal we constrain the Lipschitz constant for each layer by adding projection step during training where after each update the weight matrices are projected feasible set if they violate the constraints on the Lipschitz constant, the constraints can be controlled via a hyperparameter. We use this method to train four layer MLP with high, low and no Lipschitz constraint. We then calculate astuteness of each of our explainers for all three versions of this neural network. Figure shows the results. The goal of this set of experiments to demonstrate the relationship between Lipschitz regularity of a NN and the astuteness of explainers. As the same NN is trained on the same data but with different levels of Lipschitz constraints enforced, the astuteness of explainers varies accordingly. In all cases we see astuteness reaching 1 for smaller values of X for the same NN when it is highly constrained (lower lipschitz constant L) vs less constrained or unconstrained. The results provide empirical evidence in support of the main conclusion that can be drawn from our work: Le., enforcing Lipschitzness on classifiers lends them to more astute post-hoc explanations.
4.2 ESTIMATING PROBABILISTIC LIPSCHITZNESS AND LOWER BOUND FOR ASTUTENESS
To demonstrate the connection between explainer astuteness and probabilistic Lipschitzness as alluded to by our theory we need to estimate probabilistic Lipschitzness for classifiers. our experiments we achieve this by by empirically estimating the PE.='~D (equation 3) for range of values of L E (0, 1) incremented at 0.1. We do this for each classifier and for each dataset D and set r' as median of pairwise distance for all training points. According to equation 3 this gives us an upperbound on 1 1- a i.e. we can say that for a given I., r the classifier is Lipschitz with probability at least 1 - a. We can use the estimates for probabilistic Lipschitzness to predict the lower bound of astuteness using our theorems. We do this by noting that our theorems imply that for X CLVd, explainer astuteness is 2 1 a. This means we can guarantee that for À > T.CV explainer astuteness should be lower bounded by 1 - a. For each cittastt-lassiffer-expaaiii One, that represents the predicted lower bound on explainer astuteness given a classifier, as described in the previous paragraph. Second, the actual estimations of explainer astuteness using Definition 2. According to our theoretical results, at given the estimated explainer astuteness should stay above the predicted astuteness based on the Lipschitzness of classifiers. We show these curves in Appendix Figure 3 but summarize them in tabular form in Table to conserve space. The table shows the difference between the AUC under the estimated astuteness curves (AUC) and the AUC under the predicted lower bound (AUC1)). This number captures the average difference of the lowerbound over range of À values. Note that the values are all positive supporting our result lower bound.
combination we can plot two curves.
!SHAP: btttpa://githhie cwwp:/tw..oooooooooooooeooeala.a www.ee/gg/nddd//////oommmo /shapp RISE: RISK, CXPLAIN: https: http:///gggthtcccc////////////// cxppl///cpppppppppp
que/
8

---

Under review as a conference paper at ICLR 2023
Repularized High
Regularized tow
Not Regularited
Figure 2: Regularizing the Lipschitness of neu- ral network during train- ing results in higher tuteness for the same value of X. Higher regu- larization results in lower Lipschitz constant (Gouk et al., 2021). Astuteness reaches for smaller val- ues of À with Lipschitz regularized training, as expected from our the- orems. The errorbars represent results across runs to account for randomness in explainer runs.
0.2
0.22
1.0
RICe
0.2
1.00
Telescope
A
A
Table I: AUC AUC The observed AUC is lower bounded by the predicted AUC. As
expected, The difference between the two is always 2 0.
2layer
4layer
linear
svm
Datasets
SHAP
RISE
CXPlain
SHAP
RISE
CXPlain
SHAP
RISE
CXPlain
SHAP
RISE
CXPlain
OS
.585
.477
.551
.489
.415
.426
.043
.017
.043
.761
.628
.732
NA
.359
.289
.318
.285
-216
.244
.452
.391
.474
.742
.653
.708
Switch
.053
.053
.003
.086
.083
.039
.043
.028
.034
.557
.472
.524
Rice
224
.142
.229
.292
.331
.252
258
.65
241
.426
347
.13
Telescope
CONCLUSION, LIMITATIONS AND BROADER IMPACT
In this paper we formally defined explainer astuteness which captures the probability that given explainer will assign similar explanations to similar points. We theoretically prove that this explainer astuteness proportional to the prohabilistic Lipschitzness of the black-box function that is being explained. As probabilistic Lipschitzness captures local smoothness properties of function, this result suggests that enforeing smoothness on black-box models can lend these models to more robust explanations. In terms of limitations, we observe that our empirical results suggest that our predicted lower bound can be tightened further. One possible conjecture here that the tightness of this bound depends how different explainers calculate attribution scores, e.g. empirically observe RISE and SHAP (that both depend on expectations over subsets) behave similarly to each other but different discretization
from CXPlain. Some explainers such as LIME for tabular data have the option step prior to calculating feature attributions. As a consequence, two observations with all features belonging to the same bins would receive exactly the same explanation, whereas two arbitrarily close inputs may receive completely different explanations (when the number of perturbed sample is large (Garreau von Luxburg, 2020)). In that sense, tabular LIME would not be astute by our formulation, regardless of classifier Lipschitzness. Robustness is also only one property of reliable explainer; there are other properties that are investigated recent literature. as we outline Section These other properties, e.g. faithfulness (Agarwal et al., 2022) may also be theoretically probed in very similar ways as we did for robustness. Additionally, robustness can sometimes be at odds with with correctness (See for example Zhou et al. (2022) and "Logic Trap 3" in Ju et al. (2022)) and is best viewed as one part explanation reliability and trustworthiness (Zhou et al., 2022).
From broader societal impact perspective, would like make it clear that just enforcing Lipschitzness on blackbox classifiers should not be considered as doing enough in terms making them transparent and interpretable. Our work intended to be a call action for the field to concentrate more on improving blackbox models for explainability purposes when they are conceptualized and trained and provides one of possibly many ways to achieve that goal.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Chirag Agarwal, Marinka Zitnik, and Himabindu Lakkaraju. Probing gnn explainers: A rigorous theoretical and empirical analysis of gnn explanation methods. In International Conference on Artificial Intelligence and Statistics, pp.
PMLR, 2022.
Sushant Agarwal, Shahin Jabbari, Chirag Agarwal, Sohini Upadhyay, Zhiwei Steven Wu, and Himabindu Lakkaraju. Towards the unification and robustness of perturbation and gradient based explanations. arXiv preprint arXiv:2 102. 202.000.01100 /0618, 2021.
Alexander A Alemi, lan Fischer, Joshua Dillon, and Kevin Murphy. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410, 2016.
David Alvarez-Melis and Tommi S Jaakkola. On the robustness of interpretability methods, arXiv preprint arXiv:1806.08044 2018
Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Gareía, Sergio Gil-López, Daniel Molina, Richard Benjamins, al. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information Fusion, 58:82- 11582211,,,. 2020.
Arthur Asuncion and David Newman. Uci machine learning repository, 2007.
Shayan Aziznejad, Harshit Gupta, Joaquim Campos, and Michael Unser. Deep neural networks with trainable activations and controlled lipschitz constant. IEEE Transactions on Signal Processing, 68:4688-4699, 2020.
Robi Bhattacharjee and Kamalika Chaudhuri. When are non-parametric methods robust? In International Conference on Machine Learning, pp. 832 841. PMLR, 2020.
Jianbo Chen, Le Song, Martin Wainwright, and Michael Jordan. Learning to explain: An information- theoretic perspective model interpretation. In International Conference Machine Learning, pp. 883-892. PMLR, 2018.
Ilkay Cinar and Murat Koklu. Classification of rice varieties using artificial intelligence methods. International Journal of Intelligent Systems and Applications Engineering, 188-144, 2019.
Ian Covert, Scott Lundberg, and Su-In Lee. Explaining by removing: A unified framework for model explanation. arXiv preprint arXiv:2011.14878 2020.
Alhussein Fawzi, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. The robustness of deep
networks: A geometrical perspective. IEEE Signal Processing Magazine, 34(6):50-62, 2017.
Mahyar Fazlyab, Alexander Robey, Hamed Hassani, Manfred Morari, and George Pappas. Efficient and accurate estimation of lipschitz constants for deep neural networks. Advances in Neural Information Processing Systems, 32, 2019.
Thomas Fel, David Vigouroux, Rémi Cadène, and Thomas Serre. How good is your explanation? algorithmic stability measures to assess the quality of explanations for deep neural networks. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 720-730, 2022.
Daniel Ferene, MAGIC collaboration, et al. The magic gamma-ray observatory. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, (13)(1)):2742222 2005.
Damien Garreau and Ulrike von Luxburg. Looking deeper into tabular lime. arXiv preprint arXiv:2008. 11092, 2020.
Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 3681-3688, 2019.
Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael J Cree. Regularisation of neural networks by enforcing lipschitz continuity. Machine Learning, 10(())393-416 2021.
10

---

Under review as a conference paper at ICLR 2023
Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. A survey of methods for explaining black box models. ACM computing surveys (CSUR), 51(5):1-42, 2018.
Yiming Ju, Yuanzhe Zhang, Zhao Yang. Zhongtao Jiang, Kang Liu, and Jun Zhao. Logic traps in evaluating attribution scores. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 5911-5922, 2022.
Jing Lei, Max G'Sell, Alessandro Rinaldo, Ryan Tibshirani, and Larry Wasserman. Distribution-frer free predictive inference for regression. Journal of the American Statistical Association, 113(523); 1094-1111, 2018.
Jeffrey Li, Vaishnavh Nagarajan, Gregory Plumb, and Ameet Talwalkar. learning theoretic perspective local explainability. arXiv preprint arXiv:2011.01205, 2020.
Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Proceed- ings of the 31st international conference on neural information processing systems, pp. 4768-4777, 2017.
Ravi Mangal, Kartik Sarangmath, Aditya V Nori, and Alessandro Orso. Probabilistic lipschitz analysis of neural networks. In International Static Analysis Symposium, pp. 274-309. Springer, 2020.
Aria Masoomi, Chieh Wu, Tingting Zhao, Zifeng Wang, Peter Castaldi, and Jennifer Dy. Instance- wise feature grouping. Advances in Neural Information Processing Systems, 33, 2020.
Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.0744: 2018.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Patrick Schwab and Walter Karlen. Cxplain: Causal explanations for model interpretation under uncertainty. arXiv preprint arXiv:1910.12336 2019.
Avanti Shrikumar, Peyton Greenside, Anna Shcherbina, and Anshul Kundaje. Not just black box: Learning important features through propagating activation differences. arXiv preprint arXiv:1605.0/7/5 2016.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint 1rXxiv1706.0380 2017.
Carolin Strobl, Anne-Laure Boulesteix, Thomas Kneib, Thomas Augustin, and Achim Zeileis. Conditional variable importance for random forests. BMC bioinformatics, 9(1):-11, 2008
Aladin Virmaux and Kevin Scaman. Lipschitz regularity of deep neural networks: analysis and efficient estimation. Advances in Neural Information Processing Systems, 31, 2018.
Fan Yin, Zhouxing Shi, Cho-Jui Hsich, and Kai-Wei Chang. On the faithfulness measurements for model interpretations. arXiv preprint S0Xiv:2104.08770 2021.
Jinsung Yoon, James Jordon, and Mihaela van der Schaar. Invase: Instance-wise variable selection
using neural networks. In International Conference Learning Representations, 2018.
Yilun Zhou, Marco Tulio Ribeiro, and Julie Shah. Exsum: From local explanations to model understanding, arXiv preprint arXiv:2205.00) 30130,00000000000000000000000000 2022.
Luisa M Zintgraf, Taco S Cohen, Tameem Adel, and Max Welling. Visualizing deep neural network decisions: Prediction difference analysis. arXiv preprint (Tt.:1700.045999 2017.
11

---

Under review as a conference paper at ICLR 2023
A
DETAILED PROOFS
We include the detailed proofs for Lemma 1, Theorems 3.3 and 3.2 here.
Proof. (Proof for Lemma 1)
Let us assume,
Ps = P.N.). fo R 11=1|| k. 24 + 0}
and let L be the set of points that violate Lipschitzness, then assume,
7
PIL N₂]
given that is the probability of the set of points that violate Lipschitzness across D, we can use Bayes' rule to write.
a Plil]
Σ
PETA
we consider the case where the sets N₂ are finite, each N₂ can be mapped to a set N₁. of cardinality,
d k b
N
N = 2
6=0
In more general terms, the probability of N/ can be written as,
2d 2d Ph
2 pp
Phe = PIN,]
Σ5=1 ip;
Σラ 2 2-
Let us define the proportion of points in all N₂ that also violate Lipschitzness in their unmasked form. This leads us the following equation for 3
Σl_ 2 2 PaTA
Σ 2-1p,
The worse B would then be obtained by considering maximization over %i.
2 PITT 2
2 E 2-
8
max
=mm=a.0sa
L...,,
(19)
This constrained optimization problem can be solved by assigning ? for the largest p& until the budget a is exhausted where only fractional value of can be assigned, and () for the remaining values of This 3r 2 in general. In the specific case where Pı - for i: 2.... d, when compared to pı (i.e. where the probability of sampling a point from D such that any of the values are exactly very small compared to the probability sampling points with non-zero values which would generally be the case for sampling real data), 8* + a
Proof. (For Theorem 3.2) By considering another point o' such that dx(x, S r and equation 13 we get,
(o(s)) d(a()) d,(s(=) (( f(e' 2.))
(20)
using the fact that dz v) = || - u|lp where ||-I|p p is the p-norm, the RHS gives us,
(z),)()), (.)))))))))))))))))))))))))))))) [[f()) 0(=0)=))= f(x') f(a 2-1)l
(21)
12

---

Under review as a conference paper at ICLR 2023
using triangular inequality.
(p(0.()).
- - + 11/0= 2-1)ll
(22)
w.l.o.g assuming the first term the right is higger than the second term
(2(0 s - (d,(f()), -11
(23)
using the fact that is probabilistic Lipschitz get us,
P(d.(()(()) s 2Ld,(z, x²)] 2 -a
(24)
conclude the proof note that (((()) Vi max, (1)(()),, which gives
Pidr(((2) o(z)) (6)'))))))'))1111-
(25)
Proof. (For Theorem 3.3)
Given input and another input æ! s.t. d(x,,²) S r, using equation 15 we can write
(=(d(()) df E, 1)(f(= e +))) = (f= [,(2/2 = 1)[f(²( +
(26)
|| f(x' o =)]IIP
Using Jensen's inequality on R.H.S.
d (((((())
f(d o =)]l
(27)
Using the fact that Fl. S max f.
((()(()) s max maxx|/(==0))) f(x' o =)
(28)
0f) @
Using the fact that f is deterministically Lipschitz with some constant L 0, and dp(x @ 2, o 2) x dp(x, x²), Vz. Then using the definition of probabilistic Lipschitz with a = we get,
(f f(d' 22)) s I d(),) 21
(29)
Using this in equation 28 gives us,
(ld)( & d(x,>'))
(30)
Note that equation 30 is true for each feature E (l...., d). To conclude the proof note that dp()(z), d(x')
<
Vd max, (x)( Utilizing this with equation leads
Pld,(o(z). s VaL dz(2, x 2
(31)
Since P(d,(6(z). Vdl. dz(a, defines Axr for A Vdl. this concludes the proof.
13

---

Under review as a conference paper at ICLR 2023
B DATASET DETAILS
Orange-skin: The input data is again generated from a 10-dimensional standard Gaussian X 4).
distribution. The ground truth class probabilities are proportional to In this case the first features are important globally for all data points.
Nonlinear-additive Similar to Orange skin dataset except the ground trugh class proba- bilities are proportional to e sin 2X1 + 2| + X3 + exp{-Xa] ). and therefore each of the important features for prediction
nonlinearly related the prediction itself.
Switch: This simulated dataset is specifically for instancewise feature explanations. For the input data feature X is generated by mixture of Gaussian distributions centered at +33. If X1 is generated from the Gaussian distribution centered +3, X2 Xs used to generate the prediction probabilities according to the Orange skin model. Otherwise X X9 are used to generate the prediction probabilities according to the Nonlinear-additii model.
Rice (Cinar & Koklu, 2019):This dataset consists of 3810 samples of rice grains of two different varieties (Cammeo and Osmancik). 7 morphological features provided for each sample.
Telescope(Ferens al., 2005): This dataset consists of 19000+ Monte-Carlo generated samples to simulate registration of high energy gamma particles in a ground-based atmo- spheric Cherenkov gamma telescope using the imaging technique. Each sample is labelled either background or gamma signal and consists of 10 features.
C TRAINING DETAILS
Training splits and hyperparameter choices have relatively little effect experiments. Regardless, the details used in results shown are provided here for completeness:
Train/Test Split: For all synthetic datasets we use 10th training points and 10 test points. The neural networks classifiers were trained with batch size of 1000 for 2 epochs. While SVM was trained with default parameters used in https: wttp:://tttttttte...orr. s stable/ aalle//odie htt///oolleeeeeeee http://gneeeeeeeeeeeeeeeeeeeeeee sww.arn...nnn..mm svm SVC html.
For Telescope and Rice datasets test set sizes 5% and 33% were used, with a batch size of 32 trained for 100 epochs. SVM was again trained with default parameters.
radius r: For all experiments we used radius equal to the median of pairwise distance. This is standard practice and also allows for big enough where we can sample enough points to provide empirical estimates.
D ADDITIONAL RESULTS
Table 2 shows the normalized AUC for the estimated explainer astuteness and the predicted AUC based on the predicted lower bound curve. As expected the predicted AUC lower bounds the estimated AUC.
Figure shows the same plots as shown in Figure ?? but includes all datasets.
14

---

Under review as a conference paper at ICLR 2023
2layer
4Layer
Linear
RISE
CXPLAIN
10
Nonlinear Additive
0.8
0.6
0.
02
00
10.
0.8
Frange Skin
0.6
0.4
02
10.
0.8
Astutenes
0.6
Switch
0.4
02
316
632
10
0.8
0.6
Rice
0.0
632
00
316
00
632
10
lelescppe
0.
632
00
À
316 0.
632
Figure 3: This figure experimentally shows the implication of our theoretical results. It corresponds to the AUC values shown in Table 1. Given each combination of dataset, classifier and explainer observe that the estimated explainer astuteness for SHAP, RISE and CXPLAIN lower bounded by the astuteness predicted by our theoretical results given value of X. The predicted lower bound is depicted by dashed lines, while solid lines depict the actual estimate explainer astuteness.
15

---

Under review as a conference paper at ICLR 2023
Table 2: Observed AUC and (Predicted AUC). The observed AUC lower bounded by the predicted
AUC and so the observed AUC should always be higher than the predicted AUC. The AUC values are normalized between and 1.
2layer
4layer
lincar
svm
Datasets
SHAP
RISE
CXP
(I.B)
SHHP
RISE
CXP
(I.B)
SHAP
RISE
CXP
(I.B)
SHAP
RISE
CXP
(I.B)
08
954
847
920
(3369)
.669
.866
.906
(.480)
.994
.967
.994
(9500)
.945
813
.917
(.184)
978
936
(618)
.981
.926
.940
(.696)
.972
.912
.994
(.520)
.971
.937
(.229)
Switch
998
.966
948
(945)
.996
9988
9488
(900)
.994
.978
.988
(.950)
-969
.885
.936
(.412)
Rice
962
.886
9744
(.803)
933
.244
932
(.793)
.968
901
.962
(.800)
.88
906
.970
(.715)
Telescope
962
.663
954
(6637)
955
863
944
(6610)
.90
9667
(,756)
966
992
(.467)
Regularized High SHAP
Regularized Low RISE
Not Regularized CXPLAIN
1.0
Nonllnear Additive
Astutene5s
06
0.01
1.06
0.01
1.06
0.1
1.06
Astuteness
Switch
0.1
1.00
001
100
001
106
Orange 5kin
Astuteness
0.8
0.4
0.0
1.0
Astutene55
Rice
0.1
1.0
0.1
100
001
1.06
Astuteness
lelscoppe
00
À
À
Figure 4: Regularizing the Lipschitness of neural network during training results in higher astuteness for the same value of Higher regularization results in lower Lipschitz constant (Gouk et al., 2021). Astuteness reaches for smaller values of À with Lipschitz regularized training, as expected from our theorems. The errorbars represent results across runs to account for randomness in training.
16


Reviews:
[{"review_id": "lX_06JwjYE", "reviewer": "Reviewer_bjVt", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes smoothness/robustness of explainers of blackbox classifiers as a desirable objective. It gives theoretical bounds for this measure of robustness, (called explainer astuteness) in terms of the Lipschitz constant of the learned blackbox model. Theorems relating the two are given for three types of explainers, and the results are demonstrated empirically on a few datasets.", "summary_of_the_review": "Summary:\n\n1. The significance of explainer astuteness to the end-user needs validation.\n2. The mathematical details are imprecise.\n3. A method for modifying existing explainers, with increased astuteness without sacrificing faithfulness to the black-box model is required  to bump this paper to the next level \n4. A result showing the theoretical maximum astuteness of any explainer for a given blackbox model (with appropriate faithfulness) would also be appreciated.\n\nComments:\n\nIf explainer astuteness is proposed to be of independent interest, a subjective study is required where test subjects rank different explainers based on repeated use of the blackbox model and the explainer.", "strength_and_weaknesses": "Strengths:\n\nThe paper has an easy-to-follow structure and repeatedly emphasizes its message relating the model Lipshitzness and explainer astuteness. \n\nWeakness:\n\nIt is not at all clear why the end user of a blackbox explainer cares about the robustness of the explainer. The blackbox model is typically not under the end-user's control, and the only thing expected of an explainer is its faithfulness to the blackbox model. These results would be interesting, if different explainers had drastically different robustness for the same blackbox model -- that way the paper can be viewed as an advisory on what types of explainers to use in case the user cared about smoothness of the explainer.\n\nThe technical results are hard to follow and not written mathematically precisely. e.g. in Lemma 1 what are these \"sampling points\"? What is \"i\" in the set N_k? The usage of \"approaching\" is confusing -- what is the limit or sequence here? It looks like the set N_k is the set of points with k non-zero co-ordinates. Does the statement mean that beta=alpha when the data points all contain only one non-zero entry? Elsewhere in the paper something else is mentioned that confuses me further. \n\nAs the authors themselves mention, it is not at all surprising that the Lipshitz constant of the learned model affects the Lipschitz constant of the explainer, as indeed the explainer is designed and required to mimic the black-box model. \n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "T35KtlNbS6", "reviewer": "Reviewer_tAqb", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper provides a theoretical analysis on the robustness of explainers.\nHere, the robustness is measured as the difference between the two point $x$ and $x'$ in a close neighborhood.\nIf explanations for $x$ and $x'$ largely differ, the explainer is deemed to be not robust.\nFor the analysis, the authors adopted the notion of *astuteness* which was originally introduced for measuring the robustness of classifiers.\nThe authors extended the idea of astuteness for explainers and defined the explainer astuteness (larger the better), which is the robustness measure considered in this paper.\nThe authors derived the lower bounds of the explainer astuteness for some popular explainers, such as SHAP, feature removal, and RISE.\nThe results indicate that if the target function to be explained is $L$-Lipschitz with respect to the $\\ell_p$ distance (more formally, locally probabilistic Lipschitz), the explainers are $O(\\sqrt[p]{d}L)$-Lipschitz with high probability.\n", "summary_of_the_review": "This is an interesting study connecting the smoothness of the function and the robustness of the explainers.\nAlthough the results are not very surprising, establishing a theoretical framework would be an important contribution to the field.\nThe current paper analyzed a few popular explainers and concluded that they are all $O(\\sqrt[p]{d}L)$-Lipschitz.\nThe results will be far more interesting if there are explainers that have different orders of Lipschitzness.", "strength_and_weaknesses": "### Strength\nThis will be the first study connecting the smoothness of the target function and the robustness of the explainers.\nAs the authors stated as \"the statement is intuitive\", the results look reasonable; for less smooth functions with sharp changes, the explainers also need to change drastically to reflect the change of the function, leading to less robust explanations.\nProviding rigorous proofs to intuitions will be an important step towards understanding the relationship between the models and the explainers.\n\n### Weakness\n(My comment here is not really a weakness, but a suggestion for a further improvement.)\nThe weakness of the paper is that all the explainers considered are found to be $O(\\sqrt[p]{d}L)$-Lipschitz.\nAlthough dedicated proof techniques are required for different explainers, the results are not very surprising.\nThe results will be far more interesting if there are explainers that have different orders of Lipschitzness.\nFor example, if an explainer with large Lipschitzness is found, the use of that explainer should be avoided.\nOr, if an explainer with small Lipschitzness is found, that explainer can be a recommended choice in practice.\nI wonder whether there are any explainers with these interesting properties.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "37MXNWM4WV", "reviewer": "Reviewer_Hif5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "With the increase in the complexity of machine learning models, it becomes crucial to explain model decisions and increase transparency between human-model interactions. To this end, several explanation methods have been proposed in recent literature for explaining model predictions, but few works discuss necessary desiderata (faithfulness, stability, and fairness) to evaluate the reliability of an output explanation. In this work, the authors discuss the stability property of an explainer, i.e., an explainer should give similar explanations for similar data inputs. The paper formalizes explainer astuteness by leveraging probabilistic Lipschitzness (the probability of local smoothness of a function). Further, it provides bounds for guarantees on the astuteness of these explainers given the Lipschitzness of the prediction function and empirically validates the behavior using real-world datasets. ", "summary_of_the_review": "The paper provides interesting theoretical and empirical stability analysis of diverse explanation methods. The proof for the Astuteness of explainers that remove individual features should scale to a plethora of feature attribution methods, but the paper does not show the results for any.", "strength_and_weaknesses": "**Strengths**\n\n1. The paper presents a theoretical analysis of the stability of explanations generated using a variety of diverse explanation methods, including SHAP, RISE, and CXPlain.\n2. Theoretical and empirical analysis show that locally smooth predictive models lead to more robust explanations.\n\n**Weaknesses and Open Questions**\n\n1. The notion of stability is limited to the prediction behavior of the model, defined in Alvarez-Melis & Jaakkola et al. However, recently Agarwal et al. highlighted that the existing stability definition assumes model $f$ has the same behavior for similar inputs $\\mathbf{x}$ and $\\mathbf{x}'$, i.e., the model uses the same logic path (e.g., activating same neurons in a deep neural network) to predict the same label for the original and perturbed instance. It would be great if the authors can comment on this property of relative stability.\n2. Previous works like Alvarez-Melis & Jaakkola et al., Agarwal et al., and Fel et al. have used the predictive model's Lipschitz constant to quantify the stability of explanations. This limits the novelty of the proposed work.\n3. Unlike RISE and SHAP, we do not observe higher astuteness for the CXplain explanation method. It would be beneficial for the readers if the authors can give any insights into this behavior.\n\n\n**References**\n1. Fel, T., Vigouroux, D., Cadène, R. and Serre, T. How good is your explanation? algorithmic stability measures to assess the quality of explanations for deep neural networks. In WACV, 2022.\n2. Agarwal, C., Zitnik, M. and Lakkaraju, H. Towards a rigorous theoretical analysis and evaluation of gnn explanations. In AISTATS, 2022.\n3. Alvarez-Melis, D. and Jaakkola, T.S. On the robustness of interpretability methods. In arXiv, 2018.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject