Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanations
Authors: Tessa Han
Suraj Srinivas
Himabindu Lakkaraju
Venue: NeurIPS 2022 Accept (2022)
Keywords: explainability
transparency

Abstract: A critical problem in the field of post hoc explainability is the lack of a common foundational goal among methods. For example, some methods are motivated by function approximation, some by game theoretic notions, and some by obtaining clean visualizations. This fragmentation of goals causes not only an inconsistent conceptual understanding of explanations but also the practical challenge of not knowing which method to use when.

In this work, we begin to address these challenges by unifying eight popular post hoc explanation methods (LIME, C-LIME, KernelSHAP, Occlusion, Vanilla Gradients, Gradients × Input, SmoothGrad, and Integrated Gradients). We show that these methods all perform local function approximation of the black-box model, differing only in the neighbourhood and loss function used to perform the approximation. This unification enables us to (1) state a no free lunch theorem for explanation methods, demonstrating that no method can perform optimally across all neighbourhoods, and (2) provide a guiding principle to choose among methods based on faithfulness to the black-box model. We empirically validate these theoretical results using various real-world datasets, model classes, and prediction tasks.

By bringing diverse explanation methods into a common framework, this work (1) advances the conceptual understanding of these methods, revealing their shared local function approximation objective, properties, and relation to one another, and (2) guides the use of these methods in practice, providing a principled approach to choose among methods and paving the way for the creation of new ones.

Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanations
Tessa Han Harvard University Cambridge, MA thangg. harvard. edu
Suraj Srinivas Harvard University Cambridge, MA ssrinivasdseas www.aaaass.saaa.aaaaaaaaaaaa www.aarrad...... edu
Himabindu Lakkaraju Harvard University Cambridge, MA hlakkarajuhhbs edu
Abstract
A critical problem in the field of post hoc explainability the lack of a common foundational goal among methods. For example, some methods are motivated by function approximation, some by game theoretic notions, and some by obtaining clean visualizations. This fragmentation of goals causes not only an inconsistent conceptual understanding of explanations but also the practical challenge not knowing which method to use when.
In this work, we begin to address these challenges by unifying eight popular post hoc explanation methods (LIME, C-LIME, KernelSHAP, Occlusion, Vanilla Gradients, Gradients x Input, SmoothGrad, and Integrated Gradients). We show that these methods all perform local function approximation of the black-bor box model, differing only in the neighbourhood and loss function used perform the approximation. This unification enables us to (1) state a no free lunch theorem for explanation methods, demonstrating that method can perform optimally across all neighbourhoods, and (2) provide guiding principle to choose among methods based on faithfulness to the black-box model. We empirically validate these theoretical results using various real-world datasets, model classes, and prediction tasks.
By bringing diverse explanation methods into common framework, this work (1) advances the conceptual understanding of these methods, revealing their shared local function approximation objective, properties, and relation to one another, and (2) guides the use of these methods in practice, providing principled approach to choose among methods and paving the way for the creation of new ones.
Introduction
As machine learning models become increasingly complex and are increasingly deployed in high- stakes settings (e.g.. medicine [1,, law [2], and finance [3)). there is growing emphasis on understand- ing how models make predictions that decision-makers (e.g., doctors, judges, and loan officers) can assess the extent to which they can trust model predictions. To this end, several post hoc explanation methods have been developed, including LIME 44) C-LIME [5], SHAP (. Occlusion 77. Vani 1la Gradients [8) Gradient x Input [9, SmoothGrad 10] and Integrated Gradients [[]]] However, different methods have different goals. Such differences lead to both conceptual and practical challenges to understanding and using explanation methods, thwarting progress in the field.
From conceptual standpoint, the misalignment of goals among methods leads to inconsistent view of explanations. What an explanation? This is unclear as different methods have different notions of explanation. Depending the method, explanations may be local function approximations (LIME and C-LIME), Shapley values (SHAP), raw gradients (Vanilla Gradients), raw gradients
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

scaled by the input (Gradient x Input), de-noised gradients (SmoothGrad), or straight- line path integral of gradients (Integrated Gradients). Furthermore, the lack of common mathematical framework for studying these diverse methods prevents systematic understanding of these methods and their properties. To address these challenges, this paper unifies diverse explanation methods under a common framework, showing that diverse methods share common motivation of local function approximation, and uses the framework to investigate and evaluate properties of these methods.
From a practical standpoint, the misalignment of goals among methods leads to the disagreement problem [2]. the phenomenon that different methods provide disagreeing explanations for the same model prediction. Not only do different methods often generate disagreeing explanations in practice, but practitioners do not have a principled approach to select among explanations, resorting to ad hoc heuristics such as personal preference [[2]. These findings prompt one to ask why explanation methods disagree and how to select among them in principled manner. This paper addresses these questions, providing both an explanation for the disagreement problem and a principled approach to select among methods.
Thus, to address these conceptual and practical challenges, we study post hoc explanation methods from a function approximation perspective. We formalize a mathematical framework that unifies and characterizes diverse methods and that provides a principled approach to select among methods, Our work makes the following contributions:
1. We show that eight diverse, popular explanation methods (LIME, C-LIME, KernelSHAP, Occlusion, Vanilla Gradients, Gradient x Input, SmoothGrad, and Integrated Gradients) all perform local function approximation of the black-box model, differing only in the neighbourhoods and loss functions used to perform the approximation.
2. We introduce a no free lunch theorem for explanation methods which demonstrates that no single explanation method can perform local function approximation faithfully across all neighbourhoods, which in turn calls for principled approach to select among methods.
3. To select among methods, we set forth a guiding principle based on function approximation, deeming a method to be effective if its explanation recovers the black-box model when the two are in the same model class (i.e., if the explanation perfectly approximates the black-bor box model when possible).
4. We empirically validate the theoretical results above using various real-world datasets, model classes, and prediction tasks.
2 Related Work
Post hoc explanation methods. Post hoc explanation methods can be classified based on model access (black-box model vs. access to model internals), explanation scope (global vs. local), search technique (perturbation-base
vs. madieett-based), and basic unit of explanation (feature importance vs. rule-based). This paper focuses on local post hoc explanation methods based on feature importance. It analyzes four perturbation-based methods (LIME, C-LIME, Kerne1SHAP, and Occlusion) and four gradient-based methods (Vanilla Gradients, Gradient x Input, SmoothCrad, and Integrated Gradients).
Connections among post hoc explanation methods. Prior works have taken initial steps towards characterizing post hoc explanation methods and the connections among them. Agarwal et al. 5 proved that C-LIME and SmoothGrad converge the same explanation in expectation. Lundberg and Lee 6 proposed a framework based on Shapley values to unify binary perturbation- explanations. Covert al. found that many perturbation- estimating feature importance based on the change in model behavior upon feature removal. In addition, Ancona et al. 14 analyzed four gradient-based explanation methods and the conditions under which they produce similar explanations. However, these analyses are based on mechanistic properties of methods (e.g., Shapley values or feature removal), are limited in scope (connecting only two methods, only perturbation-base methods, or only gradient- based methods). and do not inform when one method is preferable to another. In contrast, this paper formalizes mathematical framework based on the concept of local function approximation, unifies eight diverse methods (spanning perturbation- based and gradient-based methods), and guides the use of these methods in practice.
methods share the property of

---

Properties of post hoc explanation methods. Prior works have examined various properties of post hoc explanation methods, including faithfulness to the black-box model 17. robustness to adversarial attack [18-20 20 15 211, and fairness across subgroups [22]. This paper focuses on explanation faithfulness. Related works [15-77] assessed explanations generated by gradient-baser
methods, finding that they are not always faithful to the underlying model. Different from these works, this paper provides a framework for generating faithful explanations in the first place, theoretically characterizes the faithfulness of existing methods in different input domains, and provides principled approach to select among methods and develop new ones based on explanation faithfulness.
Explanation as Local Function Approximation
In this section, we formalize the local function approximation framework and show its connection to existing explanation methods. We start by defining the notation used in the paper.
Notation. Let f X' + y be the black-box function we seek to explain in a post hoc manner, with input domain X' (e.g., X' = _d or {0, 132)) and output domain y (e.g., y = IR or [0, 1). Let G= [g X' }) be the class of interpretable models used generate local explanation for by selecting suitable interpretable model E g.
We characterize locality around point X X' using a noise random variable & which is sampled from distribution Z. Let Xg = Xg @ & be a perturbation of X₂ generated by combining x₂ and using a binary operator (e.g., addition, multiplication). Lastly, let {(f, Xx, {) R+ be the loss function (e.g., squared error, cross-entropy) measuring the distance between f and over the noise random variable & around Xo.
We now define the local function approximation framework.
Definition Local function approximation (LFA) of a black-bor box model f on a neighbourhood distribution Z around X, by interpretable model class G and loss function E is given by
g arg min
E ([19,x0,3 8~2
gEG
where valid loss E is such that Eç- z {(ʃ g, Xo, 5) =0 - (xx)== g(xx) 3A 2
The LFA framework is a formalization of the function approximation perspective first introduced by LIME 4 to motivate local explanations. Note that this conceptual framework is distinct from the algorithm introduced by LIME. We elaborate on this distinction below.
(1) The LFA framework requires that f and g share the same input domain X' and output domain ) fundamental prerequisite for function approximation. This implies, for example, that using an interpretable model g with binary inputs (X' {0, 1 }d)) to approximate a black-box model f with continuous inputs (X' Rd), as proposed in LIME, not true function approximation.
(2) By imposing a condition on the loss function, the LFA framework ensures model recovery under specific conditions: 9 recovers ff (i.e., 9" = f) through LFA when f itself is of the interpretable model class G (i.e., f E 9) and perturbations span the input domain of f (i.e., domain(x) = X)). This is key distinction between the LFA framework and LIME (which has such requirement) and guides the characterization of explanation methods in Section +
(3) Efficiently minimizing Equation | requires following standard machine learning methodology of splitting the perturbation data into train validation / test sets and tuning hyper- parameters on the validation set to ensure generalization. To our knowledge, implementations of LIME do not adopt this procedure, making it possible to overfit to a small number of perturbations.
The LFA framework is generic enough to accommodate variety of explanation methods. In fact, we show that specific instances of this framework converge to existing methods, as summarized in Table [ At a high level, existing methods use a linear model g to locally approximate the black-box model different input domains (binary or continuous) over different local neighbourhoods specified by noise random variable & (where & is binary or continuous, drawn from specified distribution, and combined additively or multiplicatively with point Xo) using different loss functions (squared-error or gradient- gradient-matching loss). We discuss the details of these connections in the following sections.
3

---

Explanation Method
Local Neighbourhood 2 around Xo
Loss Function
C-LIME
Xy + E: E(E RR) ~ Normal(O, o²)
Squared Error
SmoothGrad
XD + {; E(E IR) ~ Normal o²)
Gradient Matching
Vanilla Gradients
Xp {: {(E IRd))
Normal(O, o²),
Gradient Matching
Integrated Gradients
{xo: {(E R) Uniform ))
Gradient Matching
Gradients Input
{xo: E(E R) Uniform(a, 1), a
Gradient Matching
LIME KernelSHAP
Xo ① f: E(E {0. 1)3d)= ~ Exponential kernel
Squared Error
Xp E: E(E {0, 1] Shapley kernel Xy O E(E (0, 134) Random one-hot vectors
Squared Error
Occlusion
Squared Error
Table I: Correspondence of existing explanation methods to instances of the LFA framework. Existing methods perform LFA of a black-box model f using the interpretable model class G of linear models where g(x) w x over local neighbourhood Z around point Xo based on loss function L. Exponential and Shapley kernels are defined in Appendix A.1
3.1 LFA with Continuous Noise: Crrddiet-Basedd Explanation Methods
To connect gradient- gradient-based explanation methods to the LFA framework, we leverage the gradient- matching loss function 'ym. We define gm and show that is a valid loss function for LFA.
gy (f.gff.... Nu. E) = V 1 {) V Veg Xxo + E)
(2)
This loss function has been previously used in the contexts generative modeling (where it is dubbed score-matching) 23
and model distillation [16). However, our knowledge, its use in
interpretabilityy novel.
Proposition 1. The gradient-matching loss function fa is a valid loss function for LFA up to constant, i.e., Eę-z gym() (f, 9, xu. =0 f(xç) (xx{) +C v Z, where C R
Proof. If fxx) = g(xe), then V Vxx(xx)= Vxg(xe) and it follows from the definition of fym that fym = 0. Integrating [xf( = Vgg(Xe) gives f(x;) = (xx)) + +C.
Proposition I limplies that, when using the linear model class 9 parameterized by g(x) = w to approximate f. 9" recovers w but not b. This can be fixed by setting b s(0).
Theorem 1. LFA with gradient-matching matching loss is equivalent to (1) Smooth SmoothGrad for additive con- tinuous Gaussian noise, which converges Vanilla Gradients in the limit of small standard deviation for the Gaussian distribution; and (2) Integrated Gradiens entientsfel for multiplicative continu- ous Uniform noise, which converges to Gradient Input in the limit small support for the Uniform distribution.
Proof Sketch. For SmoothGrad and Integrated Gradients, the idea that these methods are exactly the first-order stationary points of the gradient-matching loss function under their respective noise distributions. function is the explanation returned by each method. For Vanilla Gradients and Gradient Input, the result derived by taking the specified limits and using the Dirac delta function to calculate the limit. In the limit, the weights of the interpretable model converge to the explanation of each method. The full proof is in Appendix A.T
other words, the weights of the interpretable model g that minimize the loss
Along with gradient-based methods, C-LIME (a perturbation-based method) is an instance of the LFA framework by definition, using the squared-error loss function. The analysis in this section characterizes methods that use continuous noise. It does not extend to binary or discrete noise methods because gradients and continuous random variables do not apply these domains. In the next section, we discuss binary noise methods.
3.2 LFA with Binary Noise: LIME, KernelSHAP and Occlusion maps
Theorem 2. LFA with multiplicative binary noise and squared-erro loss is equivalent to (1) LIME for noise sampled from an unnormalized exponential kernel over binary vectors; (2) KernelSHAP

---

for noise sampled from an unnormalized Shapley kernel; and (3) Occlusion for noise in the form of one-hot vectors.
Proof Sketch. For LIME and KernelSHAP, the equivalence is mostly by definition: these methods have components that correspond to the interpretable model g and the loss function & of the LFA framework and we need only to determine the local neighbourhood 2. We define the local neighbourhood E using each method's weighting kernel. In this setup, the LFAA framework yields the respective explanation methods in expectation via importance sampling. For Occlusion, the equivalence involves enumerating all perturbations, specifying an appropriate loss function, and computing the resulting stationary points of the loss function. The full proof is in Appendix A.]
3.3 Which Methods Do Not Perform LFA?
Some popular explanation methods are not instances of the LFA framework due to their properties. These methods include guided backpropagation [24], DeconvNet (25), Grad-CAM [26]. Grad- CAM++ (27,, FullGrad
28. and Dee (9. Further details are in Appendix A.2
4
When Do Explanations Perform Model Recovery?
Having described the LFA framework and its connections to existing explanation methods, we now leverage this framework to analyze the performance of methods under different conditions. We introduce a no free lunch theorem for explanation methods, inspired by classical no free lunch theorems in learning theory and optimization. Then, we assess the ability of existing methods to perform model recovery based on which we provide recommendations for choosing among methods.
4.1 No Free Lunch Theorem for Explanation Methods
An important implication of the function approximation perspective is that no explanation can be optimal across all neighbourhoods because each explanation is designed to perform LFA in specific neighbourhood. This is especially true for explanations of non-linear models. We formalize this intuition into the following theorem.
Theorem (No Free Lunch for Explanation Methods). Consider explaining a black-box model f around point Xa using an interpretable model g from model class 9 and a valid loss function E where the distance between J and G is given by d( mingeg maxxex E{S', 0,
Then, for any explanation g" over a neighbourhood distribution ε7 maxe, Elf.g g',""",', (0,E1) S 6, there always exists another neighbourhood E2 ~ Zz such that maxe, E(f.g" xx, E2) 2 d(f.g).
21 such that
Proof Sketch. The idea is that, given explanation obtained by using g to approximate I over a specific local neighbourhood Z, it is always possible to find a local neighbourhood over which this explanation does not perform well (i.e.. does not perform faithful LFA). Thus, no single ex- planation method can perform well over all local neighbourhoods. The proof entails constructing an "adversarial" input for an explanation g" such that g" has a large loss for this input and then creating neighbourhood that contains this adversarial input which will provably have large loss. The magnitude of this loss d(f, 9), the distance between f and the model class G, inspired by the Haussdorf distance. The proof is generic and makes no assumptions regarding the forms of f, G or 2,. The full proof is in Appendix A.3
Thus, an explanation on a finite E, necessarily cannot approximate function behaviour at all other points, especially when G is less expressive than f. which indicated by a large value of d(f, 9). Thus, in the general case, one cannot perform model recovery as & is less expressive than f.
An important implication of Theorem is that seeking find the "best" explanation without specify- ing a corresponding neighbourhood is futile as no universal "best" explanation exists. Furthermore, once the neighbourhood is specified, the best explanation is exactly the given by the corresponding instance of the LFA framework.
In the next section, we consider the special case when d( = 0 (i.e., when f E 9), where Theorem 3 does not apply because the same explanation can be optimal for multiple neighbourhoods and model recovery is thus possible.
5

---

4.2 Characterizing Explanation Methods via Model Recovery
Next, we formally state the model recovery condition for explanation methods. Then, we use this condition guiding principle to choose among methods.
Definition 2 (Model Recovery: Guiding Principle). Given an instance of the LFA framework with black-box model f such that f E 9 and a specific noise type (e.g., Gaussian, Uniform), an explanation method performs model recovery there exists some noise distribution 2 such that L.F returns g" ===
In other words, when the black-box model f itself is of the interpretable model class G, there must exist some setting of the noise distribution (within the noise type specified in the instance of the LFA framework) that is able to recover the black-box model. Thus, in this special case, we require local function approximation to lead to global model recovery over all inputs. This criterion can be thought of as a "sanity check" for explanation methods to ensure that they remain faithful to the black-box model.
Next, we analyze the impact of the choice of perturbation neighbourhood 2, the binary operator @ and the interpretable model class G on an explanation method's s ability to satisfy the model recovery guiding principle in different input domains X. Note that while we can choose 2, O and G, we cannot choose X, the input domain.
Which explanation should I choose for continuous X? We now analyze the model recovery proper- ties of existing explanation methods when the input domain is continuous. We consider methods based on additive continuous noise (SmoothGrad, Vanilla Gradients, and C-LIME), multiplicative con- tinuous noise (Integrated Gradients and Gradient x Input), and multiplicative binary noise (LIME, KernelSHAP, and Dcclusion). For these methods, we make the following remark regarding model recovery for the class of linear models.
Remark 1. For X = Ia and linear models f and 9 where f(x)= =]] and g(x) ww continuous noise methods recover W = () while multiplicative continuous and multiplicative binary noise methods do not and instead recover Wg = W ①
=w]] additive
This remark can be verified by directly evaluating the explanations (weights) of linear models, where the gradient exactly corresponds to the weights.
Note that the inability of multiplicative continuous noise methods to recover the black-box model is not due to the multiplicative nature the noise, but due to the parameterizatioon of the loss function. Specifically, these methods (implicitly) use the loss function {(ff, 9, xo. <) [IV
(c9(<)
Slightly changing the loss function to {(S, g, Xo, {) (xee) - Veg(xe))l i.e., replacing g(E) with g(xe), would enable g" to recover f. This would change Integrated Gradients
to fa-o ((x) (omitting the input multiplication term) and Gradient Input to Vanilla Gradients.
A similar argument can be made for binary noise methods which parameterize the loss function as [(f.g. Xo. {) = - By changing the loss function to ( g. X {) = binary noise methods can recover f for the case described in Remark [ However, binary noise methods for continuous domains are unreliable, as there are cases where, despite the modification to f, model recovery is not guaranteed. The following is an example of this scenario.
I|
Remark 2. For X = Rd, periodic functions f and g where f(x) = g(x)
Ei sin(w,, @ xx) and 1=1 sin( din(w,, gt. 3 x.). and an integer n. binary noise methods do not perform model recovery Am
for W
This is because, for the conditions specified, sin(w,,Xo,) = sin(= = sin = 0, i.c., Xx,) outputs zero for all binary perturbations, thereby preventing model recovery. In this case, the discrete nature of the noise makes model recovery impossible. In general, discrete noise is inadequate for the recovery of models with large frequency components.
Which explanation should I choose for binary X²? In the binary domain, continuous noise methods are invalid, restricting the choice of methods to binary noise methods. For reasons discussed above, methods with perturbation neighbourhoods characterized by multiplicative binary perturbations (e.g.. LIME, Kerne1SHAP, and Occlusion) only enable g" to recover f in the binary domain. Note that

---

the sinusoidal example in Remark 2 does not apply in this regime due to the continuous nature of its domain.
Which explanation should I choose for discrete X²? In the discrete domain, continuous noise methods are also invalid. In addition, binary noise methods (e.g., LIME, KernelSHAP and Occlusion) cannot be used either because model recovery is not guaranteed in the sinusoidal case (Remark 2 following similar logic that presented for continuous noise. Note that none of the existing methods in Table _ perform general discrete perturbations, suggesting that these methods are not suitable for the discrete domain. Thus, in the discrete domain, user can apply the LFA framework to define a new explanation method, specifying an appropriate discrete noise type. In the next section, we discuss more broadly about how one can use the LFA framework to create novel explanation methods.
4.3 Designing Novel Explanations with LFA
The LFA framework not only unifies existing explanation methods but also guides the creation of new ones. To explain given black-box model prediction using the LFA framework, a user must specify the (1) interpretable model class G. (2) neighbourhood distribution 2, (3) loss function E, and (4) binary operator @ to combine the input and the noise. Specifying these four components completely specifies an instance of the LFA framework, thereby generating an explanation method tailored a given context.
To illustrate this, consider a scenario in which user seeks to create a sparse variant of SmoothGrad that yields non-zero gradients for only small number of features ""SparseSmoothrraa signing SharseSmoothrral only requires the addition of a regularization term to the loss function used in the SmoothGre instance of the LFA framework (e.g.. l (SmoothGrad + || |/ Vxg((<))l()) at which point, sparse solvers may be employed to solve the problem. Note that, unlike Smoo thGrad, SparseSmoothhrre does not have a closed form solution, but that is not an issue for the LFA framework. More generally, by allowing customization of (1), (2), (3), and (4), the LFA framework creates new explanation methods through "variations on a theme".
De-
We summarize Section $4 as a table in Appendix A.4 and discuss the practical implications of Section O by providing the following recommendation for choosing among explanation methods.
Recommendation for choosing among explanation methods. general, choose methods that satisfy the guiding principle of model recovery in the input domain in question. For continuous data, use additive continuous noise methods (e.g., SmoothGrad, Vanilla Gradients, C-LIME) modified multiplicative continuous noise methods (e.g., Integrated Gradients, Gradient x Input) as described in Section $4.2 For binary data, use binary noise methods (e.g.. LIME, KernelSHAP, Occlusion). Given that methods that discrete noise do not exist, in case of discrete data, design novel explanation methods using the LFA framework with discrete noise neighbourhoods. Within each input domain, choosing among appropriate methods boils down determining the perturbation neighbourhood most suitable in the given context.
5 Empirical Evaluation
In this section, we present an empirical evaluation of the LFA framework. We first describe the experimental setup and then discuss three experiments and their findings.
5.1 Datasets, Models, and Metrics
Datasets. We experiment with two real-world datasets for two prediction tasks. The first dataset is the life expectancy dataset from the World Health Organization (WHO) [29]. It consists of countries' demographic, economic, and health factors from 2000 to 2015, with 2,938 observations for 20 continuous features. We use this dataset to perform regression, predicting life expectancy. The other dataset is the home equity line of credit (HELOC) dataset from FICO [30 consists of information on HELOC applications, with 9,871 observations for 24 continuous features. We this dataset to perform classification, predicting whether an applicant made payments without being 90 days overdue. Additional dataset details are described in Appendix A.5
Models. For each dataset, we train four models: simple model (linear regression for the WHO dataset and logistic regression for the HELOC dataset) that can satisfy conditions of the guiding prin-
7

---

Metrics. To measure the similarity between two vectors (e.g.. between two sets of explanations or between explanation and the true model weights), we use L1 distance and cosine distance. L1 distance ranges between [0, oc) and is 0 when two vectors are the same. Cosine distance ranges between [0, 2| and is 0 when the angle between vectors is 0 (or 360"). For both metries, the SmoothGGrad vs. Gradients
Integrated Gradients vs.
Gradient x Input
16
0.25
1134
9024
00503
0.009
6972
1.441
1,637
0.20
12
8.007
6.000
8770
L0732
1.020
11
Grdart
1069
004775
1999
9009
1.6072
0.30
04
91-
Sroethrrd
146091
1.215
0.-
0.3
0.1
0.05
D:03
0
0.50
0.65
0.00
0.95
Standard Dovittinn not of No (M00, oño
Livee coouttound Dno of lasue diratarme
(b)
(c)
Figure 1: Correspondence between existing explanation methods and instances the LFA framework. (a) Heatmap of average LI distance between pairs of explanations. Boxplots of L1 distance between explanations of (b) SmoothGrad and Vanilla Gradients and (c) Integrated Gradiente and Gradient Input. The lower the LI distance, the more similar explanations are. Results indicate that existing explanation methods are instances of the LFA framework.
ciple and three more complex models (neural networks of varying complexity) that are more reflective of real-world applications. Model architectures and performance described Appendix A.3
lower the value, the more similar two given vectors are.
5.2 Experiments
Here, describe the setup of the experiments, present results, and discuss their implications.
Experiment Existing explanation methods are instances of the LFA framework. First, we compare existing methods with corresponding instances of the LFA framework to assess whether they generate the same explanations. To this end, we use seven methods to explain the predictions of black-box models for 100 randomly-selected test set points. For each method, explanations are computed using either the existing method (implemented by Meta's Captum library (II or the corresponding instance of the LFA framework (Table The similarity of given pair of explanations is measured using LI distance and cosine distance.
The LI distance values for neural network with three hidden layers trained on the WHO dataset are shown in Figure indicating that explanations generated by existing methods and corresponding instances of the LFA framework are very similar. Figures D and show that explanations generated by instances of the LFA framework corresponding SmoothGrad and Integrated Gradient: converge to those of Vanilla Gradients and Gradient x Input, respectively. Together, these results demonstrate that, consistent with the theoretical results derived in Section $ existing methods are instances of the LFA framework. In addition, the clustering of the methods in Figure I indicates that, consistent with the theoretical analysis in Section $4. for continuous data, SmoothGrad and Vanilla Gradients generate similar explanations while LIME, Kerne1SHAP, Occlusion, Integrated Gradients, and Gradient x Input generate similar explanations. We observe similar results across various datasets, models, and metrics (Appendix A.6.I].
Figure a lowest L1 distance values appear in the diagonal of the heatmap,
Experiment 2: Some methods recover the underlying model while others do not (guiding principle). Next, we empirically assess which existing methods satisfy the guiding principle, i.e., which methods recover the black-box model f when f is of the interpretable model class G. We specify setting in which f and g are of the same model class, generate explanations using each method, and assess whether g recovers for each explanation. For the WHO dataset, we set

---

Weights of vs. Weights of
Weights of g vs. Weights of rx Input
-
1.25
1.25
Diszence
1.00
2.00
0.75
0.50
0.50
0.00
0.00
LINE
KametSHAP
Occhuston
intggated
Grestets
Gradeen
Smoothaale
Vanita Grodilnnte
LINE
KenatSHAl
Integnate
Explanation
Explonation
(b)
Figure 2: Analysis of model recovery. The lower the LI distance, the more similar g's weights are to (a) f's weights or (b) f's weights multiplied by the input. Results indicate that, for continuous data, additive continuous noise methods recover f's weights, satisfying the guiding principle, while multiplicative binary and multiplicative continuous noise methods do not, recovering f's weights multiplied by the input instead.
Perturbations
Continuous Perturbations
Difference Mooe Predictoo
UIEE
Rtsctue Differince of Modal Prodection
KerneSHAP
0.05
Occlusion
Gradients
0.04
SmeothGrad
Vanilla Gradients
0.02
0.00
0.00
2
10
(a)
(b)
Figure 3: Perturbation tests perturbing bottom & features using (a) binary or (b) continuous noise. The lower the curve, the better method identifies unimportant features. Results illustrate the no free lunch theorem, i.e., no single method performs best across all neighborhoods.
and y to linear regression models and generate explanations for 100 randomly-selected test set points. Then, for each point, we compare g's weights with s's gradients alone or with J's gradients multiplied by the input because, based on Section $ some methods generate explanations on scale of gradients while others the scale of trussent-iime--inut.. Note that, for linear regression, f's gradients are f's weights.
Results are shown in Figure Consistent with Section $ for continuous data, SmoothGrad and Vanilla Gradients recover the black-box model, thereby satisfying the guiding principle, while LIME, KernelSHAP, Occlusion, Integrated Gradients, and Gradient Input do not. We observe similar results for the HELOC dataset using logistic regression models for f and (Appendix A.6.2).
Experiment 3: No single method performs best across all neighbourhoods (no free lunch theorem). Lastly, we perform a set of experiments to illustrate the no free lunch theorem in $ We generate explanations for black-box model predictions for 100 randomly-selected test set points and evaluate the explanations using perturbation tests based on top-k or bottom-k features. For perturbation
Section
based on top-k features, the setup is follows. For a given data point, k and explanation, we identify the top-k features and either replace them with zero (binary perturbation) or add Gaussian noise to them (continuous perturbation). Then, we calculate the absolute difference in model prediction before and after perturbation. For each point, generate one binary perturbation (since such perturbations are deterministic) and 100 continuous perturbations (since such perturbations are random), computing the average absolute difference in model prediction for the latter. In this setup, methods that better identify important features yield larger changes in model prediction. For perturbation tests based bottom-k features, follow the same procedure

---

but perturb the bottom-k features instead. In this setup, methods that better identify unimportant features yield smaller changes in model prediction.
Results of perturbation tests based on bottom-k features performed on explanations for a neural net- work with three hidden layers trained on the WHO dataset are displayed in Figure 5 Consistent with the no free lunch theorem in Section $ LIME, KernelSHAP, Occlusion, Integrated Gradients, and Gradient x Input perform best on binary perturbation neighbourhoods (Figure (3)) while SmoothGrad and Vanilla Gradients perform best on continuous perturbation neighborhoods (Figure Bb). We observe consistent results across perturbation test types (top-k and bottom-k)), datasets, and models (Appendix A.6.3). These results have important implications: one should carefully consider the perturbation neighborhood not only when selecting a method generate expla- nations but also when selecting a method to evaluate explanations. In fact, the type of perturbations used to evaluate explanations directly determines explanation method performance.
6 Conclusions and Future Work
In this work, we formalize the local function approximation (LFA) framework and demonstrate that eight popular explanation methods can be characterized as instances of this framework with different local neighbourhoods and loss functions. We also introduce the no free lunch theorem for explanation methods, showing that no single method can perform optimally across all neighbourhoods, and provide a guiding principle for choosing among methods.
The function approximation perspective captures the essence of an explanation simplification of the real world (i.c., black-box model) that is nonetheless accurate enough to be useful (i.c., predict outcomes of set of perturbations). When the real world is "simple", an explanation should completely capture its behaviour, a hallmark expressed precisely by the guiding principle. When the requirements of two explanations are distinct (i.e., they are trained predict different sets of perturbations), then the explanations are each accurate in their own domain and may disagree, phenomenon captured by the no free lunch theorem.
Our work makes fundamental contributions. We unify popular explanation methods, bringing diverse methods into a common framework. Unification brings conceptual coherence and clarity: diverse explanation methods, even those seemingly unrelated to function approximation, perform LFA but differ in the way they perform it. Unification also enables theoretical simplicity: to study diverse explanation methods, instead of analyzing each method individually, one can simply analyze the LFA framework and apply the findings to each method. An example of this is the no free lunch theorem which holds true for all instances of the LFA framework. Furthermore, our work provides practical guidance by presenting a principled approach to select among methods and design new ones.
Our work also addresses key open questions in the field. In response to criticism about the lack of consensus in the field regarding the overarching goals of post hoc explainability (32), our work points to function approximation as a principled goal. It also provides an explanation for the disagree- ment problem 112 i.e., why different methods generate different explanations for the same model prediction. According to the LFA framework, this disagreement occurs because different methods approximate the black-box model over different neighbourhoods using different loss functions.
Future research includes the following directions. First, we analyzed eight popular post hoc explana- tion methods and this analysis could be extended to other methods. Second, our work focuses on the faithfulness rather than interpretability of explanations. The latter is encapsulated in the "inter- pretable" model class G. which includes all the information about human preferences with regards to interpretability. However, it is unclear what constitutes an interpretable explanation and elucidating this takes not only conceptual understanding but also human-computer interaction research such as user studies. These are important directions for future research.
Acknowledgements
The authors would like to thank the anonymous reviewers for their helpful feedback and the following funding agencies for supporting this work. This work is supported in part by NSF awards #IIS- 2008461 and #IIS-2040988
and research awards from Google, JP Morgan. Amazon, Harvard Data Science Initiative, and D Institute at Harvard. H.L. would like to thank Sujatha and Mohan Lakkaraju for their continued support and encouragement. T.H. is supported in part by an NSF GRFP fellowship. The views expressed here are those of the authors and do not reflect the official policy or position of the funding agencies.
10

---

References
[1] Kun-Hsing Yu, Andrew L Beam, and Isaac S Kohane. Artificial intelligence in healthcare. Nature Biomedical Engineering, 2(10):11--33, 2018.
[2] Robert Walters and Marko Novak. Artificial intelligence and law. In Cyber Security, Artificial Intelligence, Data Protection & the Law, pages 39-69. Springer, 2021.
[3] Longbing Cao. Al in finance: Challenges, techniques, and opportunities. ACM Computing Surveys (CSUR), 55(3):1-38, 2022.
[4] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "Why should I trust you?" Explaining the predictions of any classifier. International Conference on Knowledge Discovery and Data Mining, 2016.
[5] Sushant Agarwal, Shahin Jabbari, Chirag Agarwal, Sohini Upadhyay, Steven Wu, and Himabindu Lakkaraju. Towards the unification and robustness of perturbation and gradient based explanations. International Conference on Machine Learning. 2021.
[6] Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 2017.
[7] Matthew Zeiler and Robert Fergus. Visualizing and understanding convolutional networks. 1.119913 2013.
[8] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional net- works: Visualising image classification models and saliency maps. Workshop International Conference on Learning Representations, 2014.
[9] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. International Conference on Machine Learning, 2017.
[10] Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smooth- Grad: Removing noise by adding noise. arXiv:1706.038/ 2017.
[11] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. International Conference on Machine Learning. 2017.
[12] Satyapriya Krishna*, Tessa Han*, Alex Gu, Javin Pombra, Shahin Jabbari, Steven Wu, and Himabindu Lakkaraju. The disagreement problem in explainable machine learning: A practi- tioner's perspective. arXiv:2202.01660 2022.
[13] Ian Covert, Scott Lundberg, and Su-In Lee. Explaining by removing: A unified framework for model explanation. Journal of Machine Learning Research, 2021.
[14] Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross. Towards better understanding of gradient-basee attribution methods for deep neural networks. International Conference on Learning Representations, 2018
[15] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in Neural Information Processing Systems, 2018.
[16] Suraj Srinivas and François Fleuret. Knowledge transfer with Jacobian matching, International Conference Machine Learning, 2018.
[17] Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. A benchmark for in- terpretability methods in deep neural networks. Advances in Neural Information Processing Systems, 2019.
[18] Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. AAAI Conference on Artificial Intelligence, 2019.
[19] Dylan Slack, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. Fooling LIME and SHAP: Adversarial attacks on post hoe explanation methods. AAAI/ACM Conference on AI, Ethics, and Society, 2020.

---

[20] Ann- Ann-Kathrin Dombrowski, Maximilian Alber, Christopher Anders, Marcel Ackermann, Klaus- Robert Müller, and Pan Kessel. Explanations can be manipulated and geometry to blame. Advances in Neural Information Processing Systems, 2019.
[21] David Alvarez-Melis and Tommi Jaakkola. On the robustness of interpretability methods. arXiv: 1806, 08049, 2018.
[22] Jessica Dai, Sohini Upadhyay, UIrich Aivodji, Stephen Bach, and Himabindu Lakkaraju. Fairness via explanation quality: Evaluating disparities in the quality of post hoc explanations. AAAI/ACM Conference on AI, Sociery, and Ethics, 2022.
[23] Aapo Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 2005.
[24] Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXiv: /411.6641. 2015.
[25] Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. Learning deconvolution network for semantic segmentation. arXiv: 04366, 2015.
[26] Ramprasaath Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: Visual explanations from deep networks via gradient-bast based localization. IEEE International Conference on Computer Vision, 2017.
[27] Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth Balasubramanian. Grad- CAM++: Generalized gradient-based visual explanations for deep convolutional networks. IEEE Winter Conference on Applications of Computer Vision (WACV), 2018.
[28) Suraj Srinivas and François Fleuret. Full-gradient representation for neural network visualization. Advances Neural Information Processing Systems, 2019.
[29] World Health Organization (WHO). Life expectancy dataset. Global Health Observatory Data Repository, 2018
[30] FICO. Home equity line of credit (HELOC) dataset. Explainable Machine Learning Challenge, 2019.
[31] Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, and Orion Reblitz- Richardson. Captum: A unified and generic model interpretability library for PyTorch, 2020.
[32] Zachary Lipton. The mythos of model interpretability: In machine learning, the concept of interpretability both important and slippery. Queue, 2018.
[33] Weili Nie, Yang Zhang, and Ankit Patel. theoretical explanation for perplexing behaviors of backpropagation-baase based visualizations. International Conference on Machine Learning, 2018 88.
[34] Suraj Srinivas and Francois Fleuret. Rethinking the role of gradient- attribution methods for model interpretability. International Conference on Learning Representations, 2021.
[35] Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS One, 2015.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes] See Abstract and Section $
(b) Did you describe the limitations of your work? [Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Section $
12

---

(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Sections and Appendix.
(b) Did you include complete proofs of all theoretical results? [Yes] See Sections and Appendix.
S
3. If you experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL.? [Yes] We include zip file with the code in the supplementary material. The code can also be found at the following repository:
bttpp://githule Cot.aaa.c//AAAAPGGGRAAAAA
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] See Figure and Appendix.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix.
4. If you are using existing assets (c.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you eite the creators? [Yes] See Section
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent obtained from people whose data you'le 're using/curating? [N/A] We do not directly obtain data from individuals.
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [Yes] To our knowledge, the data contains no such information nor content. See Appendix.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A] We did not use crowdsourcing nor did we conduct research with human subjects.
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A] We did not use crowdsoureing nor did we conduct research with human subjects.
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A] We did not use crowdsourcing nor did we conduct research with human subjects.
13


Reviews:
[{"review_id": "WwLNn_TpfYU", "reviewer": "Reviewer_vX68", "summary": "The authors present a framework for local function approximation explanations (LFA) for generating explanations of black box ML models. They present theoretical results that place several existing explanation methods from the literature within the LFA framework; they provide a variety of theoretical and computational results demonstrating the value of this framework.\n", "questions": "None", "limitations": "The authors sufficiently addressed the limitations of their work.", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The authors very clearly present their conceptual framework (LFA). Their theoretical results are helpful (and intiutive, though this isn't bad). Their theoretical and computational results clearly demonstrate the utility of LFA.\n\nIn the context of explaining ML models, their contributions are a bit narrow: this paper focuses on explanations based on local function approximations, and they use fidelity as a measure of explanation \"goodness\". But the authors are extremely thorough in exploring this region of ML explanations. Overall, I don't have anything bad to say about this paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "U8f3sbCTVBD", "reviewer": "Reviewer_JzYe", "summary": "The authors propose the local function approximation (LFA) framework and prove that numerous local explanations are an instance of this framework. In their proposed no-free lunch theorem of explanations, the authors show that no single explanation can outperform the rest of the explanations in all neighbors around the local instances. The claims are shown across numerous datasets and models. \n\nOverall, I vote for the acceptance of the paper. I do believe that the proposed approach is sound and interesting. However, I suspect that some of the assumptions in the paper need more careful refinement. \n\nI am open to changing my score in case the authors can provide sound answers to the questions I have written in the questions section. ", "questions": "* I agree with the authors that the loss formulation of LFA resembles LIME (Equation 1). But we also know that LIME does much. LIME does feature selection before training the surrogate model. Can the authors explain whether the claims and findings still apply?  \n\n* Can the authors provide a similar visualization as Figure 3 to show the permutation tests for cases where important features were removed? I think it is important to understand the model recovery in those cases as well. ", "limitations": "Line 364-370: The authors provide some limitations of their work about including a subset of all explanation techniques. As a reader, I was keen to know a more detailed discussion on 1) limitations of their proposed framework, e.g. where LFA formulation cannot capture explanation techniques 2) Limitations of the assumptions in LFA framework for some application scenarios. ", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: \n\n* The problem of unifications explanation techniques is an essential and important topic of study\n* The authors have built upon numerous other related works and include formalism to strengthen their arguments.\n* The evaluation includes numerous explanation techniques and datasets\n\nLimitations:\n \n* The paper is dense and was very hard for me to read. I kindly suggest to the authors to try to help readers navigate all the theories in their work by including more explanations and not relying on compact forms of writing as theorems (especially in Section 3)\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Fohqo3Kuv0n", "reviewer": "Reviewer_oxiS", "summary": "This work presents a framework called local function approximation that can be used to unify local feature importance explanations (based on gradients and perturbations). ", "questions": "a) Can you mention a few practical uses of the LFA framework for data scientists or practitioners ? \n\nb) Is it possible to overcome any limitation in any of the explanation methods proposed in the literature through the LFA framework ?\n\nc) If my use case of explainability is debugging a model or identifying bias. Can you elaborate how the LFA framework could help me decide which explanation method I should use for these use cases. \n\nd) Could you please elaborate if the LFA framework could help me evaluate the different explanation methods that it unifies and how? \n\n\n\n\n\n", "limitations": "See above. ", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The overall usefulness of the LFA framework is questionable. \n\nBased on the current work, a framework that unifies perturbation methods and gradient based methods is \"good to have\" in terms of understanding all the methods together and perhaps designing new methods in future. The idea of characterising explanation methods via model recovery which is also good. However, whether the framework can help practitioners decide or guide which explanation method to use for which dataset is still not clear. \n\nSome of the ideas discussed related to perturbation methods are partially known in AI explainability community. The fact that one could fit a model alternate to ridge regression for LIME or one could do train/test on the perturbed dataset generated by LIME, etc. or may be use an alternate loss function, etc. \n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "-aHr08Fo5M8", "reviewer": "Reviewer_Fevw", "summary": "In this paper, the authors propose a generic framework which encapsulates different local explanation techniques as special cases of their LFA framework (linear function approximation). They introduce the no free lunch theorem here within the perspective of local explanations claiming that no single explanation method can perform local function approximation across all neighborhoods. Some of the key experimental results include comparison between Captum's explanation and the explanation from proposed approach by comparing the proximity of these. In addition, perturbation based tests have also been done to reinforce the notion of the no free lunch theorem", "questions": "1) In line with the review above, I would expect authors to add on what insight is this no free lunch theorem providing to the XAI community which was not known before ?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1) Paper is easy to follow and the authors have supported their claims through benchmarked results. Overall coverage of local explainers within their framework is also reasonably exhaustive. \n\n**Weakness**\n\n1) While the no free lunch theorem here might appear a really novel contribution, I don't find it particularly appealing as papers in the past Dylan et al (Fooling LIME and SHAP - AIES 2020) and several others have demonstrated that neighborhood samplers are always prone to issues such as adversarial attacks, bias, etc. I believe the no free lunch theorem is just a fancier way of presenting something already known in the community.\n\n2) I would have liked to see some more results on how LFA is more robust to issues such as overfitting etc (due to train/dev/test and hyper parameter tuning advantages, etc). Its a simple experiment but can enhance value of the paper possibly.\n\n**Post Rebuttal**\n\nI have changed my score after reading the author response to my queries.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}]

Author rebuttal:

Decision: Accept