Axiomatic Explainer Locality With Optimal Transport
Authors: Joshua Bone
Aria Masoomi
Jennifer Dy
Venue: Submitted to ICLR 2023 (2023)
Keywords: Interpretability
Explainability
Optimal Transport
Wasserstein

Abstract: Explainability methods have been notoriously difficult to evaluate and compare. Because of this, practitioners are often left guessing as to which explainer they should use for their task. Locality is one critical property of explainers which grants insight into the diversity of produced explanations. In this paper, we define a set of axioms which align with natural intuition regarding globalness, the inverse of locality. We then introduce a novel measure of globalness, Wasserstein Globalness, which uses optimal transport to quantify how local or global a given explainer is. Finally, we provide theoretical results describing the sample complexity of Wasserstein Globalness, and experimentally demonstrate how globalness can be used to effectively compare explainers. These results illustrate connections between both explainer fidelity and explainer robustness.

Under review as a conference paper at ICLR 2023
AXIOMATIC EXPLAINER LOCALITY WITH OPTIMAL TRANSPORT
Anonymous authors Paper under double-blind review
ABSTRACT
Explainability methods have been notoriously difficult to evaluate and compare. Because of this, practitioners are often left guessing as to which explainer they should use for their task. Locality is one critical property of explainers which grants insight into the diversity of produced explanations. In this paper, we define set of axioms which align with natural intuition regarding globalness, the inverse of locality. We then introduce a novel measure of globalness, Wasserstein Globalness, which uses optimal transport to quantify how local global given explainer is. Finally, we provide theoretical results describing the sample complexity Wasserstein Globalness, and experimentally demonstrate how globalness can be used to effectively compare explainers. These results illustrate connections between both explainer fidelity and explainer robustness.
INTRODUCTION
Machine Learning (ML) models are increasingly complex and capable of impressive performance in several domains. However, models become more complex, they also become less interpretable. For this reason, researchers have begun to explore the topic of explainability, where model decisions are assigned an explanation. These explanations come many forms, but often indicate how important each feature is towards the model prediction.
Explainers need be trustworthy for their explanations be valuable. However, ML practitioners have very little information at their disposal when deciding which explainer is right for them. Unlike traditional ML models, whose accuracy can computed on held-out testing datasets, there is no obvious metric by which meaning we cannot directly compute the accuracy of an explainer.
compare explainers. Ground-truth explanations are rarely known,
Some authors have tried to argue that their explainer is best by proposing variety of pscudo-accuracy metrics. When explaining image classifiers, for example, one may follow the lead of Zhang et al. (2018) and Wang et al. (2020), by evaluating explainers based saliency map's 's's energy around the object of interest. This kind of evaluation-metric is far from perfect, as it penalizes explainers from using scene context and is heavily biased towards concentrated saliency. In addition, these psuedo-accuras feature-atrribution for image data, when in reality, there are many other types of explanations and many other types of data. Clearly, it would be of great interest to the explainability community to be able to compare and contrast general-purposs explainers for any ML task.
how well they concentrate their
metrics are only applicable for the specific task of
Globalness is a property that can be used to compare and contrast explainers. When one explanation fully explains the model's behavior, we call this global explanation. In the past, models were typically explained globally. For example, feature selection would be done globally, meaning it would generate a single group of salient features for the entire dataset (Song et al,, 2010 Yu & Liu 2004, John et al., 1994 Dy & Brodley 2004). More recently, in order to explain complex black-box models, it has become common to generate instance-wise explanations rather than a single explanation for the entire model. In this case, the explainer outputs local explanations which apply only subset of the model inputs. Since this distinction between local explainers and global explainers emerged, researchers have begun to acknowledge locality/globalness
property of explainers.
Globalness a meaningful property of explainers because indicates how uniform the explanations are. In some cases, we expect even desire all explanations to be similar to another. Globalness

---

Under review as a conference paper at ICLR 2023
is also related to the concept of stability and robustness. The robustness of explainer is limited by its globalness, and we can even use an explainer's s globalness to measure its local bocal-robustness, since an explainer that is robust to small changes in the imput will be near-global in local neighborhood of the input space. This is another way in which a measure of globalness can facilitate a better understanding of our explainers, and allow to compare/contrast their behavior.
In this paper, we study the property of globalness. We provide a novel way to measure it, and present several examples of how to use it in practice. To our knowledge, this is the first work that has provided a formal measure for explainer locality/globannsss We believe that this advances the field by enabling a more thorough analysis of various explanation techniques. With our measure of globalness, one can compare and contrast explainers in a way that was previously difficult and heuristic.
The contributions of the paper are as follows. First, we introduce intuitive axiomatic properties which align with human intuition surrounding the notion of locality/globalness. Second, we propose Wasserstein Globalness, a novel measure of globalness, that satisfies all these properties. We also present theoretical results regarding the sample complexity of estimating Wasserstein Globalness. Finally, through our experiments, we demonstrate how explainers can be differentiated by their globalness, and make connection between globainess and adversarial robustness.
2 RELATED WORK
As mentioned before, the community has struggled to effectively evaluate explainers, since world data rarely comes with ground-truth explanations. For this reason, several authors have turned to synthetic data, where the relevance of features are known (Chen et al.] 2018 (017). Instead of accuracy, others seek to describe explainer properties like transparency, sparsity, or robustness, in order to inform ML practitioners' choice of explainer (Zhang et al. 2021]. Because of the interaction between the data and explainer, we often want to describe properties of an explainer when applied to specific data sample, like locality. A measure of globalness which accounts for both the data and explainer would provide quantitative method for comparing and contrasting different explainers.
Several authors have discussed the locality/globalness of explainers. For example, Deampee,Doshii Velez & Kim (2017) describe two types of explainers: local and global. They claim that global explainers are useful for scientific understanding or bias detection, and local explainers are useful for understanding specific model predictions. Zhhang et al. (202I) surveys the interpretability literature and categorizes explainers along several axes. One of these axes is related to the locality of the explainer, where explainers are either "global", "local", or "semi-local". The distinctions provided by Doshi--elezz & Kim (017) Zhang al. (202T) are categorical rather than continuous. Though the community has begun to consider this property, there is still no formal continuous measure of locality/globalness.
One common explainer, LIME, requires the user to specify kernel width which is roughly related to locality/globalnss (Ribeiro et al. 2016). Anchors are rule-based explanation given by constrained optimization problem, where the objective is the "coverage" of the anchor (Ribeiro al. 2018). An anchor's ' coverage indicates how broadly applicable the rule is, thus this is also directly related to locality/globalnes,, but is tied directly into the objective of the explainer.
Some authors, Ribeiro et al. (2016), offer ways to construct a single global explanation from many local explanations, thereby granting higher-level understanding of the model. While this offers explanations at two levels of globalness, we emphasize that this is again only binary distinction. While works like this acknowledge the property of locality and its importance to explainability, they offer no way to quantify the locality of an explainer.
The need for a measure like this becoming increasingly apparent as more researchers study the theory of explainability. For example. Li et al. (2020) define object called the neighborhood disjointedness factor in order to study the generalization of finite-sample based local approximation explainers. Neighborhood disjointedness measures roughly how far apart points are from one another, and could be applied to explanations as a measure of globalness. However, this is limited to a small class of explainers, and does not apply to general explanation frameworks.
We advance the existing literature by formalizing the property of explainer globalness, and proposing method for measuring it in practice. This is new tool that the machine learning community can
2

---

Under review as a conference paper at ICLR 2023
to better understand how explainers interact with data. We can order explainers by their globalness, identify grouped explanations, and compare explainers.
METHOD FORMULATION
There are number of different ways one could measure the locality/globalness of an explainer. In order to guarantee the quality of our approach, will first discuss the properties which should be satisfied by any measure explainer globalness. Scction 3.2] these will be stated as axioms which align with our intuition about globalness. In Section 3.33 approaches fall short according to these properties. Then, in Section 3.4. we will give our definition of globalness and demonstrate that it satisfies all of these axioms.
will briefly review why other reasonable
3.1 NOTATIONS
Consider a black-box prediction model f : - + Y R where X C Rd is the space of model inputs, Y is the target, d the dimension of the data, and is the number of class labels. An explainer for this model is function E X' - E, where the outputs of the explainer are called explanations, and comprise the space E. The space & is either subset of Rd for feature-attribution explainers, or the vertices of a hypercube, {0, 1 d for feature-selection explainers. We will use P(E) to denote the set of probability distributions over the space of explanations, E. Let μ E P(E)) be one such probability distribution, generated by passing the data through our explainer. A metric measure space (mm-space) a 3-tuple (E, de, u), where is the space of explanations, and de is a distance metric between explanations.
Explanation Framework &
Distance Measure
Feature Attribution
Rd
Cosine
Feature Selection
{0, 1ja
Hamming
Table Recommended metric spaces for common explanation frameworks.
For P(X) and g E P(Y). I[ g) will denote the set of joint distributions in P(X x Y) which marginalize to and p. 8₂ will denote the Dirac measure centered at æ, which takes the value and 0 elsewhere. We will denote the push-forward measure of by & ((A) u( 166-101 (A)). For case of reference, a summary of the notation used in this paper is in Appendix Table
(()))
3.2 DESIRED PROPERTIES OF A GLOBALNESS MEASURE
In this section, we introduce the desired properties of general globalness measure G P(E) R. which assigns real value to distribution of explanations. Our properties state that measure should be non-negative, continuous, convex, and should encode similarity between explanations.
Property (Non-negativity) For all mm-spaces (E, p), the globalness non-negative, i.c.. G(z)) 2 0. Property 2 (Fully-local measure) Let be the uniform distribution on Then
/ U G(p) 0
Property 2 states that the uniform distribution gives zero globalness, and Property T states that this is the Hinimum-globalness distribution for a given (E, dg).
Property 3 (Fully-global measure) A fully global explainer is one which produces the same ex- planation for every input, meaning its distribution of explanations can be represented by Dirac distribution 6z0- Formally, there exists an To E for which (5,))) 2 Gp(p) for all E P(E).
Property (Continuity) Let {pn} be sequence of probability distributions which converge in distribution to Then (p(,)) Gp()).
This property claims that the globalness should be continuous with respect to the input distribution. In practice, the underlying distribution μ is often unknown, and we need to approximate with empirical distribution ya, where the number of samples. Propeery 4 guaranttess that G can
3

---

Under review as a conference paper at ICLR 2023
be approximated by sampling from μ and finding the globalness of empirical distribution pau Theorem 2 discussed in Section 4 will characterize the rate of this convergence in more detail.
The need to account for distance between explanations. Before we introduce the next property, we will motivate it by discussing the geometry of &. A measure of globalness should take into account notion of difference between explanations, this can be captured by introducing distance metric, dg. For an illustrative example as how globalness is related the geometry of explanations, refer to Figure T Both distributions yield one of two explanations with equal probability. These two explanations are at a hamming distance of for the left explainer, and a distance two for the right explainer. Since the explanations more varied for the right explainer, it is less global.
More global
Less Global
0.5
0.5
0.4
022
02
0.1
0.1
Feature 2
Feature
Feature
Feature
Figure Two distributions of feature-sclection explanations on {0, 1) (red), the vertical axis represents the probability of seeing that explanation. The left distribution is more global because it outputs explanations which are closer to one another in the metric space (E, dg).
The need for isometry invariance. The previous example demonstrated that globalness should account for the distance between explanations. Additionally, when these explanations change, but the distances between them do not, the globalness should not change. In Figure [ show example of this. The two distributions in this figure are equivalent up to a re-ordering of the features. Clearly, would consider both of these to be equally global, because one distribution simply reflection of the other. Transformations like this, rotations and reflections, are referred to as isometries. The following property states that a measure of globalness should be invariant to isometries.
Property 5 (Isometry invariance) Let Te,de be the group of distance- preseecvng measurable maps Then,
Tæ.d (Ø Elde(1, v) de (dbx),, ((1)), Vr, E
Then G' is invariant with respect to the group TE,ds:
G(u) = G(d+H) for all 6 E Te.de
(3)
Additionally, let S denote all the measurable bijective maps v E + E. Then G is not invariant S:
There exists some ab e S for which G(u) # G(v+μ)
(4)
This axiom states that globalness is invariant to isometries of the explanations E, but not arbitrary per- mutations. First, Equation stated that G should be invariant to distance-preserving transformations of the explanation-space. Because globalness measures roughly how similar the set of explanations are to one another, any rigid transformation (rotation, reflection, etc.) of the space of explanations will have no effect on the globalness. Recall that we want globalness measure roughly how similar the explanations are. This notion of similarity is directly encoded by the distance de, hence an isometry which does not change these distances should not change the globalness.
There one more situation that we want our properties to address: when two sets of explanations are combined, how does the globalness change? ? The following property answers this question.
Property 6 (Convexity) If the explanations from two explainers are combined, the combined distri- bution will yield a lower globalness than the average globalness of the original explainers. Formally, let R = XP + (1 - X)Q for some X E [0, 1) and P, Q E P(E). Then G S XG(P) + (1 -))((Q). This property states that G is convex with respect to distribution μ. implying that mixture of two explainers is less global than the average of each explainer's 's globalness. By mixture, we mean probabilistically generating explanation Fg (I:) with probability p or generating explanation Fz
4

---

Under review as a conference paper at ICLR 2023
Isometric Distributions
0.5
0.5
0.3
0.3
02
0.0
0.0
Feature
Feature
Feature
Feature
Figure 2: An example of two of explanations. Though the distributions are different, they can be obtained from one another by simply swapping the ordering of the features. This corresponds to reflection, which is an isometric transformation.
with probability (1 p) independently for each E N. Concavity is common requirement of diversity measures. For example in genetics, where biological diversity is measured, the diversity of two populations combined is expected to be greater than the diversity among one population (Ricotta 2003). We instead require convexity because globalness inversely related to the diversity of explanations.
3.3 CANDIDATE MEASURES
We have considered several "candidate" approaches for G. We will now discuss some of these, and illustrate why they fall short as measure of globalness.
Shannon Entropy. Since globalness is related to how diverse the explanations are, might initially consider using the Shannon Entropy of the explainer's outputs as a measure of locality (Shannon 1948).
H(X) = E[
(5)
Entropy is maximized when all the outcomes are equally likely, and minimized when the explainer only generates one explanation for all of the data, clearly satisfiying Properties 1 - 3. In fact, entropy satisfies several our desiderata in Section 3.2 but fails on |Property Entropy cannot distinguish between how similar explanations are to another.
F-Divergences. One might also consider measuring globalness with an f-divergence (Csiszár between the distribution of explanations μu, and the uniform distribution, U:
1964
((x). Q(x))
Dy(P)) E f((
(6)
Remark (Alternative approaches). Shannon entropy and
f-d f-divergences both violate Property 5
F-divergences fail for the same reason that entropy failed. They consider only the probability of each explanation and ignore the degree of similarity between them. The proof of this remark is given in Appendix E
3.4 THE PROPOSED APPROACH
The framework presented in this paper is very general. Only metric space (E, dg) needs to be defined. For common explanation frameworks, recommended choices for de are listed in Table Given a metric space (E, dg), we propose to measure the p-Wasserstein Globalness of μ., denoted (p) as the Wasserstein distance between and the uniform measure on (Kantorovich) 1960). Intuitively, the Wasserstein distance metric between distributions P and Q measures how much "work" " one would have to do to move probability mass from until it matches Q. For this reason, sometimes called the earth-movers distance when p
5

---

Under review as a conference paper at ICLR 2023
Definition (Wasserstein Globalness).
Gr(n) = dw (1,
inf ((z,))= Ide (2, 8)1
(8)
nEll(H,U)
For all of our experiments, we will have p 1, but p is left general in the remainder of the paper, with the exception of Theorem where affects the convergence of approximate solutions to Equation 7
Theorem 1. The Wasserstein Globalness Gp PPE) R satisfies Properties 1-7, Where P(E) is
the set of probability distributions over E.
The proof of Theorem iis provided in Appendix C
Because it satisfies these intuitive properties, the Wasserstein Globalness satisfactory way quantify the globalness of explainer.
4 SAMPLE COMPLEXITY
In practice, we do not work with the true distributions y and U, but their discrete approximations. The discrete approximation for a distribution is defined as pN = 인 Σi ox. for X; ~ p.
iid
Using discrete approximation for the uniform distribution allows to estimate the Wasserstein Globalness with a discretized formulation:
Definition (Empirical Wasserstein Globalness).
Gp(m)= = dw (M, UN)
(9)
In this section we address the question how accurately Gp (p(N)) approximates (p))). Fortunately, there related works that have studied the convergence of empirical distributions under the Wasserstein metric, notably by Fournier & Guillin (2015); Dereich et al. (2013). These prior results provide the foundation for the following theorem.
Theorem 2. Let p be the order of the Wasserstein Globalness, and let M2(H) Sge (=]))((x)) E (0, d/2), # d/(d - p). q z
dp d-p" then:
Bl[G,(uN) Gp(r))] S Kp.a M, a (M) NN (1-00/// (10)
From this theorem, increasing d will lead to looser bound, meaning we need more samples as N increases. This is
immediately that (r) (
0 N However,
problem if we want to compute Wasserstein Globalness in high-dimensional problem settings like
computer vision tasks. To scale to high dimensions, we utilize the Sinkhorn Distance (Cuturi 2013
which is described in Appendix B
5 EXPERIMENTS
Wasserstein Globalness allows compare and contrast explainers. In this section, demonstrate certain insights it can provide about both our data and our explainer. In this section, we will present two kinds of experiment. The first demonstrates how globalness can help identify clustered explanations. The second type of experiment uses globalness to identify how locally-stable an explainer is. The code used in all following experiments will be made publicly available.
Datasets. Our first experiment uses simple synthetic dataset where the ground-truth explanations are known. To create this dataset, we generate d Gaussian clusters in Rd. Each cluster has a single (known) relevant feature. We use two other datasets in these experiments, which contain high- dimensional image data. The first, MNIST (Deng 2012), is comprised of grayscale images depicting handwritten digits 0-9. The last dataset, CIFAR-10 Krizhevsk (2009). ; is comprised of 32x32 color images. The targets in CIFAR-10 are of ten classes of common objects.

---

Under review as a conference paper at ICLR 2023
Jeppec Soundary noots
HMST results
1425
0:544
stegeted Grnders
08
0.53
0522
8275
0584
022
ACCISOO
0082
970
Sumber of Chsses
Sitl of taharara
Scan of aotanacoo
(a) Identifying Grouped Explanations.
(b) Jagged Boundary Results.
Figure 3: (a) Identifying Grouped Explanations. The results of computing Wasserstein Globalness for variable number of classes, on the MNIST dataset. As we explain an increasing amount of digits, the explanations become more local. (b) The results of the bagged-boundary experiment. The accuracy of both the explainers and the model (left) decrease as the Wasserstein Globalness (right) becomes more varied. These results aggregated across 15 total runs (new data each time). We show the mean + one standard deviation of Wasserstein Globalness.
Explainers. Before describing the experiments, we will briefly discuss the explanation methods used the experiments. The Gradients (Simonyan al. 2013) method directly uses the gradient of class score with respect to the input image as a saliency map. The Integrated Gradients method accumulates this gradient at several points along a path from some baseline input to the input to be explained (Sundararajan et al., 2017). The Expected Gradients method extends integrated gradients by allowing a distribution over baselines, and computing the expected value of integrated gradients with respect to this distribution (Erion et al. 2021). The SmoothGrad method smooths the gradients with a Gaussian kernel to alleviate the high frequency fluctuations commonly seen in partial derivatives (Smilkov approximates shapley values by extending the DeepLIFT algorithm (Shrikumar et al. (017]] LIME (Ribeiro al.) 2016) forms local-approximation on iistance-weiighed samples from the input X. Guided Backpropagation (Springenberg et al. 2015)
al., 2017). The Deep-SHAP method (Lundberg & Lee 2017)
the decision boundary by training a linear model
modifies the gradient approach by adding a signal that prevents the backwards flow of negative gradients during backpropagation.
Identifying Grouped Explanations. In some problems, one may observe that certain groups of inputs receive similar explanations. In this case, it could be beneficial to determine how many such groups exist. Our technique allows us to answer such questions because the number of clusters inversely related to globalness. In this section, demonstrate this relationship empirically. Specifically, we observe that the Wasserstein Globalness decreases as we add more classes because the explanations for images in a given class will receive similar explanations. We train simple MLP classifier on the previously described MNIST dataset, with no regularization and hidden layers of size 300, 500, and 300 respectively. We cross-entropy loss, and train for 5 epochs.
For all image-data in our experiments, including MNIST, we use a feature attribution explainer. Feature attribution explainers score each feature according its importance. Because of this, explanations fall in IR rather than {0, d We again follow the canonical distance metric from Table I] we will endow our metric space de) with the cosine distance: deos (a, y) T |
Figure shows the results of this experiment. As expected, each explainer correctly decreases its Wasserstein Globalness as we increase the number of digits explained.
Identifying Local Explainer Stability. If explanations change drastically after a very small change to the input, this explainer may be considered less trustworthy because we expect our explainers to be locally-stable. Of course, the degree of this stability depends on the specific problem in question. We will later discuss how this explainer-stabile Wasserstein Globalness in a small neighborhood around a single data point, we can see how robust the explainer is to such perturbations. The following experiments demonstrate this insight.
is related to the classifier's stability. By evaluating
7

---

Under review as a conference paper at ICLR 2023
Original Data
Original Boundary
Jageed Boundary idegree
200
15
00-
00
-0.4
-0.4
30
-05
0.0 10
3.5 07
1.0
13 0. 12
Feature
Figure 4: A visualization of the synthetic data in IR Data points are illustrated as dots, with color corresponding class label. Each of the two clusters utilizes different feature to discriminate between the two classes. The original data is shown (left), alongside zoomed-in visualization of the original boundary (middle), and a zoomed-in in visualization of our bonged-b modification (right).
In our first stability experiment, we use the synthetic data shown in Figure 4 We attempt to fool the explainer by making the originally-straight decision boundary jagged, thereby obscuring the true relevant feature. To make this boundary jagged, we randomly visit point then flip all nearby points x' to the label of æ, ensuring that visited points x and x' are never revisited. This data modification is illustrated in Figure I and pseudocode is provided in Appendix [A] Algorithm In this experiment, we select the feature with the highest attribution, and use the hamming distance in our explanation space: des (3r, v) = E; 1 (arq F yi). The results in Figure 3b demonstrate how Accezp is related the explainer globalness, G. in this synthetic-data experimeni. We expect that, as explainers become less accurate, the variance of their globalness increases. Accurate explainers, on the other hand, should yield globalness values which are consistently close to the ground-truth globalness.
In Figure 3b we see the relationship between Accezp and Wasserstein Globalness. As the explainer accuracy decreases, the Wasserstein Globalness varies farther from the ground-truth Globalness. Furthermore, the integrated gradients method is more robust to the jagged boundaries, and this is reflected in it having the smallest standard deviation of Wasserstein Globalness.
We conduct our next experiment on the CIFAR- 10 dataset (Krizhevsky 2009). This experiment demonstrates how globalness can be used as an indicator of explainer robustness in local neigh- borhood. First we train a CNN on the dataset as our classification model. We generate a set of inputs by taking one CIFAR-10 image and performing 300 independent adversarial attacks on that image. Each adversarial input is explained, and we evaluate the Wasserstein Globalness for that set of adversarial inputs. This is repeated several times, each time allowing stronger adversarial attacks. Each time we evaluate Wasserstein Globalness we are evaluating it in a larger local neighborhood of X. By doing this, can examine how globalness differs for different explainers. Though do not know the ground-truth globalness for these images, we expect the globalness to be maximum in this setting, since feature importances should not be affected by very small perturbations. We are primarily interested in perturbations which are small enough that the classifier still performs well. Intuitively, we hope that, long as the classification does not change, the reason for the classification should also remain roughly constant. Figures 5 and show that, even with only 300 explanations, in 32 32 x dimensions, the Wasserstein Globalness can differentiate the explainers.
we can see how explanations change after the adversarial attacks. This figure also illustrates that the perturbed images are not too different from the original image. Though the images are perceptually near-identical, the explanations can be seen to change, leading to change in globalness. In F6gure we observe exactly how this globalness changes for the batch explanations at each a. The deep-SHAP explainer is the most robust local perturbations (Wasserstein Globalness close to maximum for a variety of σ), and that the Gradient and SmoothGrad explainers are the most sensitive. This result is not surprising, as SmoothGrad injects noise in order to perform the smoothing operation, and the Gradient approach is the most naive, forming the basis for many of the other approaches.
In Figure 5
8

---

Under review as a conference paper at ICLR 2023
Gradients a 0.355
Smoothöred o 09923
Gradients - 20003
Gradieets Q a 1002
e 0.000
Q 1.2000
0.10
6 0 3970
G O 3828
6- 0.993
G- 59822
6 09977
o- 0.00
G O 2017
G O 8881
G- 0971
G- 0484
G RREE
0.000
6 03375
6 03842
6 09524
G 09628
6 09788
Figure 5: The perturbed input image shown in the left column. The remaining columns correspond to different explanation methods, and show an explanation overlayed on the original image.
100
10
0.99
Warsersten Giocaleess
0.98
Model Accurscy
0.97
0.96
Deep
Expected Gradients
0.55
SmoothGrad
Gradients
0.4
Integrated Gradients
0.000
0005
0010
0.155
0.000
0.25
0.000
0.000 0005
0.10
0.20 0.055 Adversarial Radius
0015
0.000
Adversarial Radius
Figure 6: The Wasserstein Globalness for each explainer in our CIFAR-10 experiment (left). The results of 11 separate runs plotted in the same color for each explainer. The bold line with markers of that color represents the mean across all 11 runs. We also indicate the classifier accuracy (right) in order to indicate the point which the perturbations become too strong for the explainer.
CONCLUSION
Explainability is branch of machine learning that not only facilitates trust between the user and the model, but also allows us to learn from our models. Thus, is critical that we understand our explainers use them reliably. In this paper, we have argued that globalness a theoretical property of explainers which be valuable to the explainability community. We introduced Wasserstein Globalness, model-agnostic method for measuring the globalness of explainers which satisfies intuitive axioms regarding the notion of globalness. Finally, presented experimental results which demonstrate how globalness can be used to differentiate explainers and choose the best one for given task.

---

Under review as a conference paper at ICLR 2023
ETHICS STATEMENT
As ML models grow in both complexity and popularity, they have also become less interpretable. These complex models often exhibit surprising behavior. In extreme cases, they can propagate racial and gender bias and even put humans in physical danger. However, even the benign cases erode trust between the user and the model. ML practitioners are responsible for the harmful effects of their models, and are therefore are obliged to build reliable, trustworthy systems.
Explainability is one way in which we can begin to build trustworthy models. It is crucial that we seek a better understanding of uninterpretable models in order to deploy them safely in the real world, whether that means building interpretable models in the first place or explaining them after the fact. In this paper, we discuss theoretical property of explainability methods, and present technique for measuring this quantity in practice. We intend for our globalness measure to facilitate responsible practices by enabling humans to better understand their models. This understanding can facilitate responsible decisions, enable practitioners to remove harmful behavior from their systems, and increase trust in the models themselves.
REFERENCES
Jianbo Chen, Mitchell Stem, Martin Wainwright, and Michael I Jordan. Kernel feature selection via conditional covariance minimization. Advances in Neural Information Processing Systems, 30, 2017.
Jianbo Chen, Le Song, Martin Wainwright, and Michael Jordan. Learning to explain: information- theoretic perspective on model interpretation. In International Conference on Machine Learning. pp. 883-892. PMIR, 2018
Imre Csiszár. Eine informationsthhcortiiche ungleichung und ihre anwendung auf beweis der ergodizitaet markoffschen ketten. Magyer Tudd. Akad. Mat. Kutato Int. Koezl., 8:85-108, 1964.
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In C. J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 26. Curran Asso- ciates, Inc., 2013. URL httpppss/pprcceeeddngg neurips. /eeurr/ps.cccppape///033/// pdf
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, pp. 141-142, 2012.
Steffen Dereich, Michael Scheutzow, and Reik Schottstedt. Constructive quantization: Approximation by empirical measures. In Annales de I'IHP Probabilités et statistiques, volume 49, pp. 1183-1203, 2013.
Finale Doshi-Veles and Been Kim. Towards rigorous science of interpretable machine learning. arXiv preprint arXiv: 1702:0070020600 2017.
Jennifer G Dy and Carla E Brodley. Feature selection for unsupervised learning. Journal of machine learning research, 5(Aug):845-88) 2004.
Gabriel Erion, Joseph D Janizek, Pascal Sturmfels, Scott M Lundberg, and Su-In Lee, Improving performance of deep learning models with axiomatic attribution priors and expected gradients. Nature machine intelligence, 3(7):220-631, 2021.
Rémi Flamary, Nicolas Courty, Alexandre Gramfort, Mokhtar Z. Alaya, Aurélie Boisbunon, Stanislas Chambon, Laetitia Chapel. Adrien Corenflos, Kilian Fatras, Nemo Fournier, Léo Gautheron, Nathalie T.H. Gayraud, Hicham Janati, Alain Rakotomamonjy, levgen Redko, Antoine Rolet, Antony Schutz, Vivien Seguy, Danica Sutherland, Romain Tavenard, Alexander Tong, and Titouan Vayer. Pot: Python optimal transport. Journal of Machine Learning Research, 22(78):1 2021. URL http:
wwtp://pppppppppppe/tttt wtwp:eerww/////oeeee 2001/100-5111110 html
Nicolas Fournier and Arnaud Guillin. On the rate convergence in wasserstein distance of the empirical measure. Probability Theory and Related Fields, 162(3):707-738, 2015.
10

---

Under review as a conference paper at ICLR 2023
George H John, Ron Kohavi, and Karl Pfteger. Irrelevant features and the subset selection problem. In Machine learning proceedings 1994. pp. 121-129. Elsevier, 1994.
L. V. Kantorovich. Mathematical methods of organizing and planning production. Management Science, 664):366-422, 1960. ISSN 00251909) 15265501. URL http://wwwwjstooororoooo.o stable/ (table/26270B5
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Jeffrey Li, Vaishnavh Nagarajan, Gregory Plumb, and Ameet Talwalkar. A learning theoretic perspective on local explainability. In International Conference on Learning Representations, 2020.
Scott M Lundberg and Su-In Lee. unified approach interpreting model predictions. Advances in neural information processing systems, 30, 2017.
Facundo Mémoli. Gromov- distances and the metric approach to object matching. Founda- tions of Computational Mathematics, 11(4):417-477, 2011. URL http:
de /t/bb//at jet///ounal//////aa /otinaalssscom// foom/ foomll html Memolill
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Anchors: High-precision model-agnostic explanations. In Proceedings of the AAAI conference artificial intelligence, 2018.
Carlo Ricotta. Additive partition of parametric information and its associated 3-diversity measure. Acta Biotheoretica, 5102 2003.
C. E. Shannon. A mathematical theory of communication. The Bell System Technical Journal, 27(3): 379 423, 1948. doi: 10.1002/j..5533 1550)).1338--30 7305.1948.... 1948.1901333 wtt.tt013.....
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In International conference machine learning, pp. 3145- 3153. PMLR, 2017.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprini arXiv:1312.6034, 2013.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv: arXiv: 1706.0382, 2017.
Fengxi Song. Zhongwei Guo, and Dayong Mei. Feature selection using principal component analysis. In 2010 international conference on system science, engineering design and manufacturing informatization, volume 1, pp. 27-30. IEEE, 2010.
J Springenberg, Alexey Dosovitskiy, Thomas Brox, and M Riedmiller. Striving for simplicity: The all convolutional net. In ICLR (workshop track), 2015.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pp. 3319-3328, PMLR, 2017.
Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu. Score-cam: weightee visual explanations for convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 24-25, 2020.
Lei Yu and Huan Liu. Efficient feature selection via analysis of relevance and redundancy. The Journal of Machine Learning Research, 2:05--12 2004.
Jianming Zhang, Sarah Adel Bargal, Zhe Lin, Jonathan Brandt, Xiaohui Shen, and Stan Sclaroff. Top-down neural attention by excitation backprop. International Journal of Computer Vision, 126 (00):104-1100) 2018
11

---

Under review as a conference paper at ICLR 2023
Yu Zhang, Peter Tino, Ales Leonardis, and Ke Tang. A survey neural network interpretability. IEEE Transactions on Emerging Topics in Computational Intelligence, pp. 726-742, oct 2021. doi: 10. 10011194 URL BEtoas//eeb.oorr/..11DDSeerree. 2021
3100641
12

---

Under review as a conference paper at ICLR 2023
A
APPENDIX
Table 2 provides a summary of the notation and terminology used in the paper.
Symbol
Name
O#
Push-forward Measure
1
Indicator Function
6z
Dirac distribution (centered at x)
E
Expected Value
Set of Explanations
X
Space of model inputs
E
Explainer
P(X)
Set of probability distributions over X
U
Uniform Distribution
(I(.Q)
Set of joint distributions which marginalize P and
R
Real numbers
(F))
Frobenius Inner Product
dg
distance between explanations
mm- mm-space
metric-measure space
Table 2: Summary of notation used in the paper
Algorithm
describes the bngged-boundard data modification shown in Figure
Algorithm
1: Let be a finite data sample.
2: Let label(-) give the label of each point in X.
3: Let N₂ be neighborhood around point æ.
4: while not all points visited do
5: for unvisited x do
6: for unvisited x' N₂ do
7:
label(x') +- label(x) x' visited
8:
9:
end for
10: end for
11:
end while
We measure the wall clock timing of the three different computationally expensive tasks. Because these quantities vary with the explainer used and the random input image, present the average both the explainers and 10 randomly selected CIFAR-10 10 images. These results are shown Figure 3
Task
Average Time (s)
Adversarial Perturbations
56.
Generating Explanations
19.42
Computing Globalness
6.97
Table 3: Average wall clock time for the CIFAR- 10 experiment. As expected, computing globalness less computationally expensive than generating the explanations themselves.
Figure shows all of the perturbed images from the CIFAR-10 experiment presented in 5 Figure is formed from subset of the rows in Figure Figure X depicts another set of explanations, with different original input image.
In figure 11 show example of the different perturbed images for single adversarial radius
13

---

Under review as a conference paper at ICLR 2023
inputs - 0.000
expGrad G 0.9949
deepshap 1.0000
integrated gradients
vanillagrad 0.9999
smoothgrad G 0.9877
G 0.9999
0.003
0.9993
0.9978
0.007
G 0.9933
0.9980
G 0.9981
0.9926
G 0.9845
0.010
G 0.9923
0.9968
0.9969
0.9916
G 0.9842
0.013
G 0.9912
0.9958
G 0.9958
0.9885
G 0.9811
0.017
G 0.9903
G 0.9947
G 0.9949
0.9856
G 0.9751
0.020
G 0.9883
0.9927
0.9892
0.9835
0.9669
== 0.023
G = 0.9865
G 0.9908
G 0.9845
0.9654
G 0.9564
0.027
G 0.9851
G 0.9891
G 0.9811
0.9591
G 0.9481
0.9877
0.9592
Figure 7: An extended version of
B APPROXIMATING WASSERSTEIN GLOBALNESS IN HIGH- DIMH-DIMNSSIONS
Computing the Wasserstein distance traditionally involves solving a linear programming problem which does not scale well. the previous section, we discussed the sample complexity of (uw)). This empirical estimate also scales poorly with the dimension of the data. However, several techniques have been developed to efficiently compute Wasserstein distances in high dimensions. One such technique, called regullaziee optimal transport, relies on regularizing the entropy of the joint distribution म E ITP,.Q. Q)
The Wasserstein distance can be written optimization over the of product measures IT(P, Q). In its discrete form, the expectation [. (x, y)] from Equation 7 can be written as a Frobenius inner product between the matrix of distances D, and the transport plan R.
dy (p.4) = min (R, D)
(1I)
Rell(p.a)
The sinkhom distance, introduced to the ML community by Cuturi 2013),, utilizes technique called entropic regularization. The term (RR)) Ens (rglog(ra.)-
transport problem adds an additional regularization
14

---

Under review as a conference paper at ICLR 2023
inputs 0.000
smoothgrad G 0.9990
deepshap 1.0000
integrated gradients
vanillagrad - 1.0001
expGrad
G 1.0000
0.9900
0.9993
0.007
G 0.9984
0.9986
G 0.9994
0.9932
G 0.9898
0.010
G 0.9994
0.9978
G 0.9988
0.9913
G 0.9891
0.013
G 0.9985
0.9967
G 0.9973
0.9871
G 0.9887
0.017
G 0.9988
0.9955
G 0.9981
0.9807
G 0.9880
0.020
G 0.9992
0.9941
0.9974
0.9837
0.9870
0.023
G 0.9985
G 0.9929
G 0.9963
0.9831
G 0.9864
0.027
G 0.9981
G 0.9913
G 0.9951
0.9823
G 0.9852
0.9902
0.9823
Figure 8: Another set explanations for an image of deer from CIFAR-I0 dataset.
dsinkhorn = min (R, (). XQ(R) RELl(p.a)
(12)
This encourages the transport plan to have less mutual information with the marginals and By searching just these "smooth" transport plans, have convexified the optimization problem, and now can use matrix scaling algorithms to solve it There are also existing theoretical guarantees that dyinkhorn will be close to dw. for which we refer the reader Cuturi (2013).
By utilizing entropic regularization, we can find good solutions faster. However, we can always decrease the strength of this regularization in order to eliminate noise from the computation. As have discussed, this is especially useful when we work with high-dimensional data, as is done in 5
C PROOF OF THEOREM
We will now restate and prove that the axioms from Section 3.2 apply to the definition of G' in Section
For our experiments, compute the sinkhorn algorithm using available optimal transport solvers available Flamary al. (02T).
15

---

Under review as a conference paper at ICLR 2023
inputs o- 0.000
smoothgrad G 0.9991
deepshap
integrated gradients
vanillagrad - 1.0001
expGrad
1.0000
G 1.0001
0.9891
0.9983
0.9943
0.007
G 0.9992
0.9968
G 0.9983
0.9900
G 0.9891
0.010
G 0.9987
0.9957
G 0.9980
0.9889
G 0.9888
0.013
G 0.9988
0.9943
G 0.9979
0.9849
G 0.9884
0.017
G 0.9986
0.9930
G 0.9967
0.9773
G 0.9878
0.020
G 0.9985
0.9911
G 0.9961
0.9760
G 0.9860
0.023
G 0.9977
0.9881
G = 0.9936
0.9762
G 0.9839
0.027
= 0.9971
G 0.9863
G 0.9919
0.9773
G 0.9825
0.9841
0.9776
Figure Another set of explanations for an image ship from the CIFAR-10 Dataset.
Property states that Gp(m) 2 0 V/u.
Proof. Since dw distance metric, it is non-negative by construction. Therefore Gp(P) dw (P.U) is non-negative.
Property Let {mur} be a sequence probability distributions which converges distribution to j.. Then Gp(ln) Gp(u).
Proof.
By remark 2.4 in (Mémoli) 2011).
|dl (A, B) dyr (An, Ba)l (A, (B, Br)
(13)
This implies that
ldy (H, U) dy (Mn, Un)l S dy un) + (U,
(14)
However, U does not need to be approximated, therefore
U Un [Gr(u) Gp(un))
(d U) dl (Mm, U)
dy (H.
16

---

Under review as a conference paper at ICLR 2023
inputs 0.000
smoothgrad G 0.9992
deepshap 1.0000
integrated gradients
vanillagrad 0.9996
expGrad 0.9925
G 1.0001
0.9980
0.9962
0.007
G 0.9993
0.9960
G 0.9978
0.9938
G 0.9907
0.010
G 0.9993
0.9948
G 0.9977
0.9919
G 0.9899
0.013
G 0.9984
0.9933
G 0.9967
0.9895
G 0.9889
0.017
G 0.9990
0.9915
G 0.9953
0.9875
G 0.9877
0.020
G 0.9990
0.9896
0.9953
0.9855
G 0.9857
0.023
G 0.9983
G 0.9879
G 0.9938
0.9833
G 0.9843
0.027
G 0.9969
G 0.9859
G 0.9932
G 0.9814
G 0.9831
0.9850
0.9807
Figure 10: Another set explanations for an image of a truck from the CIFAR-10 dataset.
Figure 11:
A visualization of the pertubed images used in our CIFAR-10 experiment. correspond to the last row in Figure
These
But An converges in distribution to μ. meaning di (H, (M) + = >Gp(p) Gp(un) +
From this proof we can see that, not only does G,((u)) converge to Gp(u), but it converges least as quickly as Pr - μ.
Property 3 states that there exists an Do for which Gr( 2 Gp(n) Vp P(E). Furthermore, will also show that, for this zo, 820.p < Su.p Vx E E, where Su.p the p-eccentricity, defined in
Mémoli 2011 as Sz.p Jx d (x, x' da'
Roughly, the p-eccentricity indicates how far a point the of the points in E.
17

---

Under review as a conference paper at ICLR 2023
P-eccentricity appears in this property in order to characterize the 2y for which San achieves maximum globalness.
Proof. Without loss of generality, assume Szo. 2 Sz.ps Vz E &. Any discrete distribution can be represented as a delta-train P = E; x;0(2;) such that X; 2 O, Vi and Σ, A; = 1. By Property we know that Gp(P) s Σ, (,G((((z)) But for Dirac measure, reduces
Gp(5x0) =1/8 d' ())r) 1/8
(15)
So
Gp(P) S Σ X1Strop S max88;; 820.p"
(16)
This proves the axiom for any discrete distribution. But general distributions can be approximated with these weighted delta-trains, so by Property 4, so in the limit, this proof holds for general distributions.
Property states that = U - G(u)= =0
Proof. First note that de Q) a metric. From the identity of indiscernibles axiom distance metrics, (I, v) = 0 >
This directly implies that dw (,s, U) 0 < == U.
We will now prove the first part of Property 5) restated below:
Let Te,de be the group of isometries of (E, de) defined in Equation
Then Gp is Te invarrint.
Proof. We need to show that Gp(()) = Gp(d+1)) Vo E Te,d-
One consequence of the definition of push-forward measure that
P Powg(Oz)
(x)) (x(x)
E am
[ox)]
EE z~
[x]]
(17)
Consider the set of measure couplings (feasible transport plans) between and v, denoted II(,,) v)
Let ha I(,, v) > l1 Qyv) be given by
h o (A, B) AA
(18)
Since he is bijective, and since we have shown in 17 that Ex [xc] (x0))(())((0()) it holds that
Gp(1)= inf Eeany~ (dr =Ell(u.U)
(19)
Because o is an isometry,
inf Bee.grhhtmmll Φu)) 1/p =)l((
(20)
Because Egwn [z] E Ex ( [d(()),
inf E (e ld' /r xell(,U)
(21)
Finally, since he is bijection between 11(,, U) and I1(0u U))
inf reM(onndaa)
E Ba (d v)) 1/p
= G
18

---

Under review as a conference paper at ICLR 2023
We will now prove the second part of Property 5 which states that 3Φ: : - E of Te for which Gp(0+H) + Gy(H), for some J..
Proof. Consider Φ E o: - & such that dz(2, y) dg(dz, σy) for some constant c, and some 2, y E &. Note that Φ is clearly not in de the group of isometries of (E, d,
Like before, we can write:
Gp(n)= inf (d y) =EIT().))
(22)
inf =eN(oguddnU)
E (a [d (z, (e.u)]
(23)
But since @ is not isometric:
inf rell@gu,b0U)
Bla dc (x. //
(24)
# Gp(0#H)
(25)
Property 6 (convexity) Let R be convex combination of measures P and Q. That is, R XP + 2)Q for some À E [0, 1]. Then G(R) S XG(P) + (1 - A)G()).
Proof. Let P,Q be probability distributions. Let Φp be the optimal transport plan between P and U. Similarly, let Φa be the optimal transport plan between Q and U. Then the convex combination of P and Q is a feasible transport plan between R = XP (1 X)Q and U,
Gz(X)=
inf E B (x,
(26)
=El(aU))
Blp.)) ld (d" ///
(27)
AB(z.g)mop dd (d. (2) 1/r + (1 - A) (d //
(28)
AGp(P) + (1- (Q)
(29)
D PROOF OF THEOREM 2
We will first introduce two lemmas to simplify the proof of theorem 2 The first lemma concerns the error incurred by approximating μ with finite samples. The second lemma concerns the error incurred by approximating U with finite samples. By bounding both of these errors, obtain an upper bound on the eтror of using both mamte-sample approximations.
Let p (0, d/2). Also, let q & d/(d- p). Let M²(H) .gee (x))) Then for some constant Cp.ad which depends only on p.q, and d,
Lemma 1.
(R) Gp(uN)]] s (p) p/d + N w-(a-1)/o)
(30)
Proof. From the triangle inequality,
dew (uN,U) U) S (u, +
(e (uN,
(31)
From Fournier & Guillin 2015). Eldy H|) S OpnaMte (N) p/d N (o-n//))))
So
[[G]) Gp(()) s Cn -N-00-1//9
(32)
19

---

Lemma 2. Ler p E (0, 2), and q > dp d-p' Then for some constant kipa which depends only on p and Under review as a conference paper at ICLR 2023
Similarly, invoke the triangle inequality with dw (p, U) on the LHS:
dw (u,UU U) I d (uN, U) + d (UN,
(33)
And, making the same argument, get:
So
Gş(n)= E[GG(un)] S CnaaMMi N-00001/9)
(34)
Finally, combining 32 and 34
G Gp (p(N))]] (pgaMMi/ N-
This theorem related to Property 4 in the sense that the empirical distribution B converges to J.. This theorem is concerned with the rate at which Gp(uN) converges, whereas Property 4 simply states that it will converge.
Perhaps the most important consequence if is that [[]G,(()= - Gp(un)]] - as +
E Gp((p( (eow)>GGp(uNi))) Gp(uN)l] S Kpall f [=]l)))(x))
1/9 --1//
(35)
Proof. From the triangle inequality:
dw(uM UN) S dw(uN, U) dw (UN,U, dw (uN, Un) - dw(uN, & (UNUN.
(36)
(37)
From Dereich et al. 2013), there exists some constant Kp.q such that U)]
Kp-a Jan [==][1dp)(z)]] 179 N-1d/
Fldw (un, UN) - dw (uN, U)] s Bldy (UN, S Kp [sw [[=]))pp())) N/
E[G,(uN) - Gp(un)] s Kp,a Ies [x=1[<p((()) 1/9 N/ N
Similarly:
dw(uN, U) S. dw(uN, UN) + (UN(UNUU) (ww, U) - CUN,NNU dw(UUNUU)
So we have that:
El
- ( S Kp I |||||| [10l1)d((0)] N/2
(38)
We will now combine the previous lemmas to get final bound on the approximation error incurred by using discrete approximations for both f and U.
Proof. The following one-line proof a natural consequence of lemmas
and
BlIG,(uN)- - Gp(m)] S Bl]G,(uN) -an0--G((Nlll) + Gp(ux)]]
S Kpal { ||~|| [[=])))((z)] 1/9 1/4 CCpgaMpP p/d - ((s--)))
= =El(GG(uN) - Gy(1)]] S Kp. (1) Cp (M) -p/d + +NN 10-0011//
1
20

---

Under review as a conference paper at ICLR 2023
E PROOF OF REMARK 3.3
Proof. Let v E S be permutation of the set of explanations.
By Equation 17 direct consequence of the definition of push-forward measure, we have that:
Ex [[((()))] B (l
(39)
for some function g-
Now observe that both entropy and f-divergences, as defined in Equations and respectively, can be written in the form above. Entropy is obtained by directly letting g(x) -log(2)
H(X) = E--n(--lua [og])(()))
(40)
The f-divergence between and U is equivalent by allowing g(x) = [( and allowing the constant to be the uniform probability of element x, ((r).
U(x)
D(U = Exan [5(
}
(41)
u(xx)
21


Reviews:
[{"review_id": "xsigEky26eE", "reviewer": "Reviewer_8L8j", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper introduces Wasserstein Globalness, a model-agnostic method for measuring the globalness (the inverse of locality) of explainers.\n", "summary_of_the_review": "A potentially interesting paper that introduces a concept and evaluates it on simple test scenarios. \n", "strength_and_weaknesses": "*Strengths*\n\n* The idea becomes rather clear even for a non-expert, and the paper could be of interest to the explainability community.\n\n* The paper reads well and is easy to follow.\n\n*Weaknesses*\n\n* As a non-expert in explainability, I found it hard to position the work or assess the novelty/originality of this paper.\n\n* The experiment are rather simple, which is good for illustrative purposes, but leaves open questions regarding relevance and generality of the analysis.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "nhzivjNQzJ", "reviewer": "Reviewer_arKF", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper promotes the use of the Wasserstein distance between the distribution of explanations produced by an explainer algorithm and the uniform distribution as a way to access whether the explainer is local (provides sample specific information) or global (class specific). Five desired properties are introduced to motivate the choice, and proofs are provided that the proposed use of the Wasserstein distance concurs with these properties.", "summary_of_the_review": "A clear paper with a solid and reasonably well-presented idea on an important topic. However, it lacks some demonstration of the claims and does not systematically explore the usefulness of the proposed method.", "strength_and_weaknesses": "# Strengths\n\n1.  Clear motivation of the chosen measure with the 5 properties.\n2.  An important problem the paper aims to address.\n3.  Empirical demonstration with some theoretical justification, and the code provided.\n4.  A well written manuscript.\n\n# Weaknesses\n\n1.  Some claims do not take into account the existing literature.\n    1.  In the intro it is claimed that there are no obvious metrics to compare the explainers, yet the field of developing such metrics is a highly active one.\n        1.  Remove and Retrain ROAR: Sara Hooker et al. A benchmark for interpretability methods in deep neural networks. In Advances in Neural Information Processing Systems, pages 9737–9748, 2019\n        2.  Retain and Retrain RAR: Md Rahman, et al. Interpreting models interpreting brain dynamics. Scientific Reports, 12(1):1–15, 202\n        3.  Accuracy Information Curves, Softmax Information Curves: Andrei Kapishnikov, Subhashini Venugopalan, Besim Avci, Ben Wedin, Michael Terry, and Tolga Bolukbasi. Guided integrated gradients: An adaptive path method for removing noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5050–5058, 202\n    2.  The abstract claims introduction of a novel measure of globalness, yet I've come across this concept already in the paper below. Admittedly the method in that paper was developed to demonstrate an application. This work may be a mathematical treatment of that method. Please explain.\n        1.  Md Rahman, et al. Interpreting models interpreting brain dynamics. Scientific Reports, 12(1):1–15, 202\n2.  Judging by experiments, sensitivity of the Wasserstein measure to globalness is low.\n    1.  In all experiments the changes of the measure are within hundredth units, while the variance, where shows is much larder.\n    2.  In Figure 6 accuracy drops to random, while the measure only slightly decreases and even then not for all methods.\n    3.  What is the source of variability in assessing the distribution of explainer predictions? Multiple classifiers each trained with random seeds and explained at each sample? Multiple samples on the same classifier? Something else?\n3.  Lack of demonstration of the computational complexity and wall clock timing of the method.\n    1.  Wasserstein distance is computationally demanding to compute, especially in high dimensions. There's a danger, that the proposed method will be intracktable in practice despite for toy problems. The paper would benefit from demonstrations on the issue.\n4.  The meaning of the distance of the value of it in separating one method from another in terms of behavior of explaners at individual input samples is not clearly demonstrated.\n    1.  It is unclear what does it mean for the value to be 0.99 or 0.98, what changes in detected explainers? Figure 5 contains examples of saliency maps, yet interpretation or connection of the proposed G measure with what is displayed is not clear.\n    2.  It is clear, that if the metric is close to 0 all saliency maps for each sample must have very few things in common, and a high value of the metric means there is a single global per class feature. Rigorous demonstration that this is indeed the case would help evaluate the value of the proposed method.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "3MzUxBNC4p", "reviewer": "Reviewer_CcYU", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies the problem of measuring globalness of neural network explainers. The work considers the globalness of explanations from a theoretical angle and inspected a list of properties that a metric of globalness should satisfy. ", "summary_of_the_review": "Overall, this work studies the interesting problem of measuring globalness of explanations in a more rigorous way. However, there is a disjoint between the analysis and the proposed approach. It is also unclear the benefit of the this new metric of globalness. The writing of the work should be improved. ", "strength_and_weaknesses": "Strength: \n\n+ The work considers globalness of explanations from a theoretical perspective. I don't know this area well, but there is no such theoretical study before, I feel this contribution is solid. \n+ The work analyzes a list of properties that a metric of globalness should satisfy. \n\nWeaknesses:\n\n- The analysis is somewhat shallow. I feel that most analyses are not far from my intuitive understanding. \n- The proposed approach of Wasserstein Globalness is not examined against those properties analyzed in the first part of the work.\n- The writing of the submission is poor. It is hard to understand,  partially because it contains many typos. \n\nBesides these weaknesses, I don't see a clear goal of experiments. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject