GLIME: General, Stable and Local LIME Explanation
Authors: Zeren Tan
Yang Tian
Jian Li
Venue: NeurIPS 2023 spotlight (2023)
Keywords: Explanation
LIME
Stability
Local fidelity
Interpretability
TL;DR: A new framework, which allows flexible sampling distribution, improves stability and local fidelity of LIME both theoretically and empirically.

Abstract: As black-box machine learning models become more complex and are applied in high-stakes settings, the need for providing explanations for their predictions becomes crucial. Although Local Interpretable Model-agnostic Explanations (LIME) \cite{ribeiro2016should} is a widely adopted method for understanding model behavior, it suffers from instability with respect to random seeds \cite{zafar2019dlime, shankaranarayana2019alime, bansal2020sam} and exhibits low local fidelity (i.e., how the explanation explains model's local behaviors) \cite{rahnama2019study, laugel2018defining}. Our study demonstrates that this instability is caused by small sample weights, resulting in the dominance of regularization and slow convergence. Additionally, LIME's sampling approach is non-local and biased towards the reference, leading to diminished local fidelity and instability to references. To address these challenges, we propose \textsc{Glime}, an enhanced framework that extends LIME and unifies several previous methods. Within the \textsc{Glime} framework, we derive an equivalent formulation of LIME that achieves significantly faster convergence and improved stability. By employing a local and unbiased sampling distribution, \textsc{Glime} generates explanations with higher local fidelity compared to LIME, while being independent of the reference choice. Moreover, \textsc{Glime} offers users the flexibility to choose sampling distribution based on their specific scenarios.

GLIME: General, Stable and Local LIME Explanation
Zeren Tan Tsinghua University thutzri 1 0190gmail. com
Yang Tian
Jian Li Tsinghua University lapordge@gmail.cm com
Tsinghua University tyanyang44@gmil..com
Abstract
As black-box machine learning models grow in complexity and find applications in high-stakes scenarios, it is imperative provide explanations for their predic- tions. Although Local Interpretable Model-agnostic Explanations (LIME) [22] is widely adpoted method for understanding model behaviors, is unstable with and exhibits low local fidelity (i.e., how well the explanation approximates the model's local behaviors) [21 [6]. Our study shows that this instability problem stems from small sample weights, leading to the dominance of regularization and slow convergence. Additionally, LIME's sampling neighborhood is non-local and biased towards the reference, resulting in poor local fidelity and sensitivity to reference choice. To tackle these challenges, we introduce GLIME, enhanced framework extending LIME and unifying several prior meth- ods. Within the GLIME framework, we derive an equivalent formulation of LIME that achieves significantly faster convergence and improved stability. By employing a local and unbiased sampling distribution, GLIME generates explanations with higher local fidelity compared to LIME. GLIME explanations are independent of reference choice. Moreover, GLIME offers users the flexibility to choose a sampling distribution based on their specific scenarios.
respect to random seeds (35 24
3
Introduction
Why a patient is predicted to have brain tumor [10] Why a credit application is rejected ! Why picture identified as electric guitar [2] As black-box machine learning models continue to evolve in complexity and are employed in critical applications, it is imperative to provide explanations for their predictions, making interpretability central concern In response to this imperative, various explanation methods have been proposed [39, 26
19 22
30 aiming to provide insights 28
into the internal mechanisms of deep learning models.
Among the various explanation methods, Local Interpretable Model-agnostic Explanations (LIME) (2 has attracted significant attention, particularly in image classification tasks. LIME explains predictions by assigning each region within an image a weight indicating the influence of this region the output. This methodology entails segmenting the image into super-pixels, illustrated in the lower-left portion of Figure Ta introducing perturbations, and subsequently approximating the local model prediction using linear model. The approximation achieved by solving weighted Ridge regression problem, which estimates the impact (i.e., weight) of each super- super-pixel on the classifier's s output.
Nevertheless, LIME has encountered significant instability due to its random sampling procedure
35 24 3. In LIME, set of samples perturbing the original image is taken. As illustrated in Figure 1a LIME explanations generated with two different random seeds display notable disparities, despite using large sample size (16384). The Jaccard index, measuring similarity between two explanations on a scale from 0 to 1 (with higher values indicating better similarity), is below 0.4. While many prior studies aim enhance LIME's stability, some sacrifice computational time for stability 24 40). and others may entail the risk of overfitting [35 The evident drawback of unstable
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

LIME
GLIME-Ginomia
GUME-Gaues
Bandom Seod
Nsl
Random Seed 2
Betena
Jaccard Index
0.38
1.00
0.95
LIME
GUMME-GausS
LIME
GUME-Gmas
(a) LIME and GLIME explanations for Ima- geNet image the tiny Swin-Transformer with two random seeds. a 0.25 and sample size of 16384 are employed. The numerical value presented below the second row represents the Jaccard Index between the two explanation maps.
(h) Distribution LIME and CLIME-GAUSE in 2D example (left) and 1D example explanations (right). LIME samples are constrained to one side of r and are more distant from x. Additionally, LIME explanations exhibit variability with distinct reference points.
Figure I: GLIME enhances stability and local fidelity compared to LIME. (a) LIME demonstrates instability with the default parameter a, while GLIME consistently provides meaningful explanations. (b) LIME samples from biased and non-local neighborhood, limitation overcome by GLIME.
explanations lies in their potential to mislead end-users and hinder the identification of model bugs and biases, given that LIME explanations lack consistency across different random seeds.
In addition to its inherent instability, LIME has been found to have poor local fidelity [6 21 As depicted in Figure Ia the 1t2 value for LIME on the sample image approaches zero (refer also to Figure 4b). This problem arises from the non-local and skewed sampling space of LIME, which is biased towards the reference. More precisely, the sampling space of LIME consists of the corner points of the hypercube defined by the explained instance and the selected reference. For instance, in the left section Figure 1b. only four red points fall within LIME's sampling space, yet these points are distant from x. As illustrated in Figure 3 the L distance between LIME samples of the input and x approximately 0.7| 0.7||x||2 on ImageNet. Although LIME incorporates weighting function to enforce locality, an explanation cannot be considered as local if the samples themselves are non-local, leading to a lack of local fidelity in the explanation. Moreover, the hypercube exhibits bias towards the reference, resulting in explanations designed to explain only portion of the local neighborhood. This bias causes LIME to generate different explanations for different references, as illustrated in Figure Tb (refer to Appendix A.4 for more analysis and results).
To tackle these challenges. we present GLIME local explanation framework that generalizes LIME and five other methods: KernelSHAP [19], SmoothGrad (28). Gradient (36. DLIME [35], and ALIME 24 Through a flexible sample distribution design, GLIME produces explanations that are more stable and faithful. Addressing LIME's instability issue, within GLIME, we derive an equivalent form of LIME, denoted as GLIME- SIMME--INOMIAL,... by integrating the weighting function into the sampling distribution. GLIME-BINOMIA ensures exponential convergence acceleration compared LIME when the regularization term is presented. Consequently, demonstrates improved stability compared to LIME while preserving superior local fidelity (see Figure 4p. Furthermore, GLIME enhances both local fidelity and stability by sampling from local distribution independent of any specific reference point.
HOMEE-BNOMMAL
In summary, our contributions can be outlined as follows:
We conduct an in-depth analysis to find the source of LIME's instability, revealing the interplay between the weighting function and the regularization term as the primary cause. Additionally, we attribute LIME's suboptimal local fidelity to its non-local and biased sampling space.
We introduce GLIME as more general local explanation framework, offering a flexible design for the sampling distribution. With varying sampling distributions and weights, GLIME serves as a generalization of LIME and five other preceding local explanation methods.

---

By integrating weights into the sampling distribution, we present specialized instance of GLIME with binomial sampling distribution, denoted as GLIME-BINON B---OIIOOML.. We demonstrate that GL.ME-BINONE BINOMIAT, while maintaining equivalence to LIME, achieves faster convergence with significantly fewer samples. This indicates that enforcing locality in the sampling distribution is better than using a weighting function.
With regard to local fidelity, GLIME empowers users to devise explanation methods that exhibit greater local fidelity. This is achieved by selecting local and unbiased sampling distribution tailored to the specific scenario in which GLIME is applied.
2 Preliminary
2.1 Notations
Let X' and y denote the input and output spaces, respectively, where C RD y We specifically consider the scenario in which X' represents the space of images, and f X' serves as a machine learning model accepting an input x E X. This study focuses on the classification problem, wherein produces the probability that the image belongs certain class, resulting in y = [0,
Before proceeding with explanation computations, a of features { derived by applying transformation x. For instance, { VE=1 could represent image segments (also referred as super-pixels in LIME) or feature maps obtained from a convolutional neural network. Alternativll, {st) may correspond to raw features, i.e., itself. In this context, || -lo--- ll ||1, and || ll2 denote the fo. &1, and f2 norms, respectively, with @ representing the element-wise product. Boldface letters are employed denote vectors and matrices, while non-boldface letters represent scalars or features. Bx (e) denotes the ball centered at x with radius e.
2.2 A brief introduction LIME
In this section, we present the original definition and implementation of LIME 22 in the context of image classification. LIME, as a local explanation method, constructs a linear model when provided with an input x that requires explanation. The coefficients of this linear model serve as the feature importance explanation for x.
Features. For an input x, LIME computes feature importance vector for the set of features. In the image classification setting. for an image x, LIME initially segments x into super-pixels sI...., Sd using a segmentation algorithm such as Quickshift 32 Each super-pixel is regarded as feature for the input x.
Sample generation. Subsequently, LIME generates samples within the local vicinity of x as follows. First, random samples are generated uniformly from {0, 1)d. The j-th coordinate z/J for each sample z' is either 1 or O, indicating the presence absence of the super-pixel 8y. When s, is absent, is replaced by a reference value rj. Common choices for the reference value include a black image, blurred version of the super-pixel, or the average value of the super-pixel [29 22 g). Then, these z' samples are transformed into samples in the original input space I P by combining them with
(81,..., sa) using the element-wise product as follows: z x @ z' + r (1 x') where the vector of reference values for each super- super-pixel, and @ represents the element wise product. In
other words, z E is an image that is the same as x, except that those super-pixels sy with z/ = are replaced by reference values.
Feature attributions. For each sample z' and the corresponding image z, we compute the prediction (z). Finally, LIME solves the following regression problem to obtain feature importance vector (also known as feature attributions) for the super-pixels
LIME
arg min Brgg
+ A/
(I)
where z = x X - r (1 z'), T(2')()))) exp[- - 2/113/02), and o is the kernel width parameter.
Remark 2.1. In practice, we draw samples {z]] from the uniform distribation Uni( 1)d) to estimate the expectation in Equation 1) In the original LIME implementation (22), X a/n a
3

---

constant a > O. This choice has been widely adopted in prior studies 40 use IMM to represent the empirical estimation of w!MME
8
79
5
20
34y.
We
2.3 LIME is unstable and has poor local fidelity
Instability. To capture the local characteristics of the neighborhood around the input x, LIME utilizes the sample weighting function x(.) to assign low weights to samples that exclude numerous super-pixels and, consequently, are located far from x. The parameter a controls the level of locality, with small o assigning high weights exclusively to samples very close to x and a large ø permitting notable weights for samples farther from well. The default value for in LIME is 0.25 for image data. However, as depicted in Figuur Ia. LIME demonstrates instability, a phenomenon also noted in
prior studies [3 24 34] As showed in Section 4, this instability arises from small values, leading to very small sample weights and, consequently, slow convergence.
Poor local fidelity. LIME also suffers from poor local fidelity [66 21. The sampling space of LIME is depicted Figure lb) Generally, the samples in LIME exhibit considerable distance from the instance being explained, as illustrated in Figure 3] rendering them non-local. Despite LIME's incorporation of weights to promote locality, it fails to provide accurate explanations for local behaviors when the samples themselves lack local proximity. Moreover, the sampling space of LIME is influenced by the reference, resulting in a biased sampling space and a consequent degradation of local fidelity.
3
A general local explanation framework: GLIME
3.1 The definition of GLIME
We first present the definition of GLIME and show how it computes the explanation vector w GLIME Analogous to LIME, GLIME functions by constructing model within the neighborhood of the input x, utilizing sampled data from this neighborhood. The coefficients obtained from this model are subsequently employed as the feature importance explanation for x.
Feature space. For the provided input x X C RDD the feature importance explanation is computed for a set of features s = ($1,.... sa) derived from applying a transformation to x. These features s can represent image segments (referred to as super-pixels in LIME) or feature maps obtained from a convolutional neural network. Alternatively, the features s can correspond to raw features, i.e., the individual pixels of x. In the context of LIME, the method specifically operates on super-pixels.
Sample generation. Given features a sample z' can be generated from the distribution P defined on the feature space (e.g., are super-pixels segmented by a segmentation algorithm such Quickshift [32 and p = Uni( {0, 1 in LIME). It's important to note that z' may not belong to X' and cannot be directly input into the model f. Consequently, we reconstruct z E R D in the original input space for each z and obtain fzz) (in LIME, reference r is first chosen and then z = x O z' +r (1-1 zz)). Both z and z' are then utilized compute feature attributions.
Feature attributions. For each sample z' and its corresponding 2, we compute the prediction I f(z). Our aim is to approximate the local behaviors of f around using function g that operates on the feature space. can take various forms such as a linear model, decision tree, or any Boolean function operating on Fourier bases (37.. The loss function (f(z), (zz)) quantifies the approximation gap for the given sample z'. In the case of LIME, g(²') = z'. and {((()) g(z)) (f(z) ((fe)))gzz))) g(z'))² To derive feature attributions, the following optimization problem is solved:
W GLIME
arg min Ex iT (2))))(((()))))))))))))))) AR(v):
(2)
where x--) is a weighting function and R(-) serves as a regularization function, e.g., || (which is used by LIME). We use w GLIME to represent the empirical estimation of w GLIME.
|| 1|2 or
||
Connection with Existing Frameworks. Our formulation exhibits similarities with previous frame- works 22 12] The generality of GLIME stems from two key aspects: (1) GLIME operates within a broader feature space Rdd in contrast to (22) which is constrained to (0, 1) and [12) which is confined to raw features in D (2) GLIME can accommodate a more extensive range of distribution choices tailored to specific use cases.

---

3.2 An alternative formulation of GLIME without the weighting function
Indeed, we can readily transform Equation 2 into an equivalent formulation without the weighting function. While this adjustment simplifies the formulation, it also accelerates convergence by sampling from the transformed distribution (see[Section 4.1 Figuure 4a). Specifically, we define the transformed sampling distribution as P(z') =
w(z m(z')'zz) Utilizing P as the sampling distribution,
[(x(t)((t)dt'
Equation can be equivalently expressed as
X I(v),, z= B=
W GLIME arg min E x' (l..p(((()))))(1))))
It is noteworthy that the feature attributions obtained by solving Equation 3 are equivalent to those obtained by solving Equation 2 (see Apeennixxx B.1 for formal proof). Therefore, the use of iT-) in the formulation is not necessary and can be omitted. Hence, unless otherwise specified, GLIME refers to the framework without the weighting function.
3.3 GLIME unifies several previous explanation methods
This section shows how GLIME unifies previous methods. For a comprehensive understanding of the background regarding these methods, kindly refer Appendix A.6
LIME [22] and GLIME-BINOMIAN
the case of LIME, initiates the explanation process
by segmenting pixels T1. ID into super-pixde $1. Sa. The binary vector z' p
Uni 13d) signifies the absence or presence of corresponding super-pixels. Subsequently, z = x r (1 z'). The linear model g(z') = v vT zz' defined on {0, 1]d. For image explanations,
((f(z). (f(z) g(z')) and the default setting is (z') exp(-I z z'llo/²²), ft(v) 1 [22]. Remarkably, LIME is equivalent to the special case GLIME- COOME-BINOMIAL without the weighting function (see (epenndx B.2 for the formal proof). The sampling distribution of GLIME-
BINOMIAL defined as P(z', ||| llo k) k/o² /[1 =l//")d where k 0, b,.,....d d. This distribution is essentially Binomial distribution.
To generate sample z' E {0, 1]dd independently draw z/K E {0, 1} with P(z; 1) 1/(1 + e-/o²" for i 1,..., The feature importance vector obtained by solving Equation 3alviingEuuatinn 3 Jnder FLIME-BINOMEN
can
denoted as w Binoniil our framework, the formulation of KernelSHAP aligns with that of LIME, with the only difference being R(v)= = 0 and π (zz)) = (d - 1)/(( 1o)) || (lo - ||z'|lo)).
KernelSHAP [9
SmoothGrad 28 SmoothGrad functions on raw features, specifically pixels in the case of an image. Here, z x, where z' (N0 o I). The loss function L( ((f())((())) is represented by the squared loss, while (zz)) = 1 and R(v) = 0, as established in Appendix B.6
Gradient [36]. The Gradient explanation is essentially the limit of SmoothGrad as approaches 0.
DLIME 33 DLIME functions raw features, where P is defined over the training data that have the same label with the nearest neighbor of x. The linear model g(z') = v Tz z is employed with the square loss function é and the regularization term R(v)= = 0.
ALIME (24. ALIME employs an auto-encoder trained on the training data, with its feature space defined as the output space of the auto-encoder. The sample generation process involves introducing Gaussian noise to x. The weighting function in ALIME is denoted as π Tzz') = cxp(-IlAE
AE(z') Ill,, where AE(-) represents the autoeen The squared loss function is chosen the loss function and no regularization function is applied.
4 Stable and locally faithful explanations in GLIME
4.1 GLIME-BINOMIAN converges exponentially faster than LIME
To understand the instability of LIME, we demonstrate that the sample weights in LIME are very small, resulting in domination of the regularization term. Consequently, LIME tends to produce explanations that are close to zero. Additionally, the small weights in LIME lead to considerably slower convergence compared GL.IME. BINOMIAL, despite both methods converging to the same limit.
5

---

0= 0.25
a= 0.50
0= 1.00
0= 5.00
144
1-1
110
0.125
11
Binomial
LIME
waight (dotted
84-
weight (eiglt(coolee) (cblled)
probability
0:122
0.00
woight w
probebillty
(dolted)
0.-
0.02
0.75
0.4
62-
0000
-34
0.00
0002
-22
0.
0.00
020
0.50
0.75
1.00
60
0.00
0.25
0.52
0.70
1.00
0000
380
0.2
0.55
100
0.30
0.00
0.25
2.50
075
1.00
12/u/d
luld
[
Figure 2: The distribution and the weight of || |z'1|o- In LIME, the distribution of ||z'|o follows a binomial distribution, and independent of a. In BILMMMB,,, when small, || concentrates around d, while in LIME, most samples exhibit negligible weights except for the all-one vector As a increases, the distributions in BOMME--IOOMIAL converge to those in LIME, and all LIME samples attain non-negligible weights.
Small sample weights in LIME, The distribution of the ratio of non-zero elements to the total number of super-pixels, along with the corresponding weights for LIME and
is depicted in Figure 2 Notably, most samples exhibit approximately d/2 non-zero elements. However, when a takes values such as 0.25 or 0.5, a significant portion of samples attains weights that are nearly zero. For instance, when a 0.25 and ||z||o d/2, Txx') reduces to exp(-8d), which is approximately 10-70 for d = 20. Even with |z'1lo d 1, π (z') equals e-16 approximating 10 Since LIME samples from Uni( ((0, 1)d) the probability that sample z' has ||z'|lo = d - or d is approximately Consequently, the sample estimation of the expectation in Equation I tends to be much smaller than the true expectation with high probability and is thus inaccurate (see Aepeendix B.3 more details). Given the default regularization strength À = 1. this imbalance implies the domination of the regularization term in the objective function of [Equaion explanations close to zero in such cases, diminishing their meaningfulness and leading to instability.
10-5 when d = 20. Therefore, most samples have very small weights.
As result, LIME tends to yield
GLIME converges exponentially faster than LIME in the presence of regularization, Through the integration of the weighting function into the sampling process, every sample uniformly carries a weight of 1, contributing equally to ro[Equation 3 Our analysis reveals that GLIME requires substantially fewer samples than LIME to transition beyond the regime where the regularization term dominates. Consequently, GLIME BINOMIAL converges exponentially faster than LIME. Recall that w.ME and W.GLIME represent the empirical solutions of Equation 1 andd Equation 3 respectively, obtained by replacing the expectations with the sample average. w RINCmRITA the empirical solution of
Equation 3) with the transformed sampling distribution P(z', ||z'|lo = k) = k /(1 + ez/a²)e where i 0, 1....... d. In the subsequent theorem, we present the sample complexity bound for LIME (refer to (oppenddxx f.4 proof).
Theorem 4.1. Suppose samples {z/}= Uni(10, 13d) are used to compute the LIME explanation.
For any > 0,5 (0, 1), if n 2, 2/0" log(4d/6)), & n, have P(] WLME
Z 5. w wLiME = lim,- WIME
Next, we present the sample complexity bound for GLIME (refer to Appendix B.5 for proof).
Theorem 4.2. Suppose z'~P such that the largest eigenvalue of z'z''" is bounded by R and _lz' = (a1 I + | Var(z' (z')T)llz )ll S v², (z' fz))a < M for some M > O. 3 are i.i.d. samples from P and are used to compute GLIME explanation w GLIME, For any e > 0.8 E (0, 1), ifn = lie M2 33,1 log(4d/0) where 7 is function of X, d, on, 02 we have
GLIME GLIME w GLIME c) 2 1 8. w GLIME limm-tx GLIME
Since GLIME- BINOMIAL samples from binomial distribution, which is sub-Gaussian with param- eters M Vd. 2, a 11((1 e -1/o"" ), a2 1/(1+ e e-1/=")² and 7(11, a2, do2/o*
(refer to Appendix B.5 for proof), we derive the following corollary:
Corollary 4.3. Suppose {z]}²_ are i..dd samples from P(z', .|z2/11/ llo = k) = 2/0 /11+c 1.... d and are used compute n = 2,03 2/00² log(Ad/8)), we have P[[] Binomial w Binomial)
explanation. For any 0,5 E (0,1), if
|| < t) > 8.
Binomial
lim w Binomial

---

Court
LIME
Count
Count
Binomial IVE-Gausa
Uniform
08
04
62
00
03
02
04
65
03
02
04
00
1x
1x
Figure 3: The distribution of sample distances to the original input. The samples produced by LIME display a considerable distance from the original input x, whereas the samples generated by GLIME demonstrate more localized distribution. LIME has a tendency to overlook sampling points that are in close proximity to x.
Comparing the sample complexities outlined in Theorem 4.1 and Corollary 4.3) it becomes evident that LIME necessitates an exponential increase of exp(d, (d, o ") more samples than for convergence. Despite both LIME and GLIME-BINOMIAL samples being defined on the binary set {0, J. the weight π (z') associated with sample z' in LIME is notably small. Consequently, the square loss term in LIME is significantly diminished compared to that in SLLMEE-BINOMIA. This situation results in the domination of the regularization term over the square loss term, leading to solutions that are close to zero. For stable solutions, it is crucial that the square loss term is comparable to the regularization term. Consequently, GLIME- GLIME-BINOMIAL BINOMIAL requires significantly fewer samples than LIME to achieve stability.
4.2 Designing locally faithful explanation methods within GLIME
Non-local and biased sampling in LIME. LIME employs uniform sampling from (0, 1)d and subsequently maps the samples to the original input space X' with the inclusion of a reference.
Despite the integration of a weighting function enhance locality, the samples {z: generated by LIME often exhibit non-local characteristics, limiting their efficacy in capturing the local behaviors of the model f (as depicted in FFiure 3). This observation aligns with findings in [16] 21 which demonstrate that LIME frequently approximates the global behaviors instead of the local behaviors of f. As illustrated earlier, the weighting function contributes to LIME's instability, emphasizing the need for explicit enforcement of locality in the sampling process.
Local and unbiased sampling in GLIME. In response these challenges, GLIME introduces sampling procedure that systematically enforces locality without reliance on a reference point. One approach involves sampling z' P (0, o²I) and subsequently obtaining
z'. This method, referred to as GLUME-GAUSS, utilizes a weighting function π(-) (-) = 1, with other components chosen mirror those of LIME. The feature attributions derived from this approach successfully mitigate the aforementioned issues. Similarly, alternative distributions, such as P = Laplace O0, a) or
P = Uni([-a, o)d), can be employed, resulting in explanation methods known as GLIME-LAPLACE and GL.MEE-UIIOOMM respectively.
4.3 Sampling distribution selection for user-specific objectives
Users may possess specific objectives they wish the explanation method to fulfill. For instance, if a user seeks enhance local fidelity within neighborhood of radius c, they can choose distribution and corresponding parameters aligned with this objective (as depicted in [Fiurre 5). The flexible design of the sample distribution GLIME empowers users to opt for distribution that aligns with their particular use cases. Furthermore, within the GL.ME framework, it is feasible to integrate feature correlation into the sampling distribution, providing enhanced flexibility. summary, GL.ME affords users the capability to make more tailored choices based their individual needs and objectives.
7

---

Experiments
Dataset and models. Our experiments are conducted on the ImageNet datase Specifically, randomly choose 100 classes and select an image at random from each class. The models chosen for explanation are R [ and the tiny Swin-Transformer [8] (refer to Appendix A.7 for results). Our implementation is derived from the official implementation of LIME The default segmentation algorithm in LIME, Quickshift [32], is employed. Implementation details of our experiments provided in Appeendix A.I For experiment results on text data, please refer to Aopppeddix A.9 For experiment results ALIME, please refer to Appendixx A.8
Metrics. (1) Stability: To gauge the stability of an explanation method, we calculate the average top-K K Jaccard Index (JI) for explanations generated by 10 different random seeds. Let Wi...., W10 denote the explanations obtained from 10 random seeds. The indices corresponding to the top-K largest values in w, are denoted as Ri.K The average Jaccard Index between pairs of Ri. and Ry.:K is then computed, where JI B) ( BI/|AU BJ.
(2) Local Fidelity: To evaluate the local fidelity of explanations, reflecting how well they capture the local behaviors of the model, we employ two approaches. For LIME, which uses non-local sampling neighborhood, we use the R² score returned by the LIME implementation for local fidelity assessment 33.. Within GLIME, we generate samples (z, | and 12/ from the neighborhood (c).). The squared difference between the model's output and the explanation's 's output on these samples is computed. Specifically, for a sample z, we calculate (f(z) w zz'²² for the explanation w. The local fidelity of an explanation w at the input x is defined as 1/(1+ + (f(.). - -w zz))), following the definition in [16]. To ensure a fair comparison between different distributions in GLIME, we set the variance parameter of each distribution to match that of the Gaussian distribution. For instance, when sampling from the Laplace distribution, we use Laplace(O, a v/v,), and when sampling from the uniform distribution, we use Uni([-V3a, )
5.1 Stability of LIME and GLIME
LIME's instability and the influence regularization/veighting In Figure 4a it is evident that 1) demonstrates greater stability compared to its
LIME without the weighting function (LIME weighted counterpart, especially when a is small (e.g.. a = 0.25, 0.5). This implies that the weighting function contriibtes to instability in LIME. Additionally, we observe that LIME without regularization [LIME+A== 0) exhibits higher stability than the regularized LIME, although the improvement is not substantial. This is because, when is small, the sample weights approach zero, causing the Ridge regression problem to become low-rank, leading to unstable solutions. Conversely, when a is large, significant weights are assigned to all samples, reducing the effectiveness of regularization. For instance, when 0 = 5 and d = 40, most samples carry weights around 0.45, and even samples with only one non-zero element left possess weights of approximately 0.2. In such scenarios, the regularization term does not dominate, even with limited samples. This observation is substantiated by the comparable performance of LIME, LIME+* 1, and LIME-) when 1 and 5. Further results are presented in Appendix A.2
Enhancing stability in LIME with GLIME. In Figure 4a it is evident that LIME achieves Jaccard Index of approximately 0.4 even with over 2000 samples when using the default o = 0.25. In contrast, provide stable explanations with only 200-400 samples.
both GLIME- -IMME-INNOIALL and GLIME-GAUS Moreover, with an increase in the value of or, the convergence speed of LIME also improves. However, GLIME-BINOMES consistently outperforms LIME, requiring fewer samples for comparable stability. The logarithmic scale of the horizontal axis in Figure 4a highlight the exponential faster convergence of GLIME compared to LIME.
Convergence of LIME and -IMMM-BIOOMIAL to a common limit. Figure 8 of Appendix A.3) we explore the difference and correlation between explanations generated by LIME and GLIME- BINOMIAL. Mean Squared Error (MSE) and Mean Absolute Error (MAF) are employed as metrics to quantify the dissimilarity between the explanations, while Pearson correlation and Spearman rank correlation assess their degree of correlation. As the sample size increases, both LIME and GLIME-
I 'Code hct.l//iiiiiiiiiaaaaagggg/aaaaae hattpss//ggtcub..ccc//arroo/e///
Stable-and -ade-and-Lose Local Loccl-LIMEE- Expt-EExpaaaniion anation
8

---

0.25
0.50
1.4
GLIME-Binomal
Top- 20 Lancard Index
Top-20 Jaccard
0.-
LIME
LIME+A
0.-
LIME+) =e
-84
GLIME-Binomal GLIME-Gause
GLIME-UUitom
0.-
GLIME-Laplaoe
# samples (log ecale)
semples (log scale)
-= 1.00
d 5.00
04-
Top-20 Jacseard Index
0.8
Top- R0--20 Juceard Indox
0.8
04-
0.4
0.5
02-
samples (log scale)
semples scaie)
03
035
(a) Stability of various methods. The reported met- ric the Top-20 Jaceard index. Instances of LIME with no regularization and weighting are denoted as LIME++A = 0 and LIME++T = respectively. LIME exhibits instability when is small, whereas GLIME demonstrates enhanced stability across differ- ent values. Notably, LIME achieves greater stability without weighting or regularization when a small. Conversely, regularization and weighting have mini- mal impact the stability of LIME when is large.
(b) R² comparison between LIME and various GLIME methods with different sampling distribu- tions. We utilize 2048 samples to compute explana- tions and the corresponding R for each image and each method. LIME exhibits nearly zero R when a 0.25 and 0.5, suggesting minimal explanatory power. In general, the R² of LIME consistently lower than that of GLIME, underscoring GL.ME's en- hancement in local fidelity.
Figure 4: GLIME consistently enhances stability and local fidelity compared to LIME across various values of 0.
--0 0.25
d- 0.5
g=
10000
6.000
2:005
05400
GLUNE-GaUSS
10615
39900
MEEE-E Laplaos
Unform
Figure 5: Local fidelity of GLIME across different neighborhood radii. Explanations produced under a distribution with a standard deviation of a demonstrate the ability to capture behaviors within local neighborhoods with radii exceeding a.
BINOMIAL exhibit greater similarity and higher correlation. The dissimilarity in their explanations diminishes rapidly, approaching zero when is significantly large (e.g., a 5).
5.2 Local fidelity of LIME and GLIME
Enhancing local fidelity with GLIME. A comparison of the local fidelity between LIME and the explanation methods generated by GLIME is presented in Figure 4b) Utilizing 2048 samples for each image to compute the R² score, GLIME consistently demonstrates superior local fidelity compared to LIME. Particularly, when signifying that the linear approximation model (w LIME z is nearly constant. Through the explicit integration of locality into the sampling process, GLIME significantly improves the local fidelity of the explanations.
0.25 and 0.5, LIME exhibits local fidelity that is close to zero,
Local fidelity analysis of GLIME under various sampling distributions. In Figure 5
assess the local fidelity of GLIME employing diverse sampling distributions: AN II), Laplace(O,, a o/v2)..

---

0.30
LIME
0.30
LIME
GLIME
GLIME
0.25
0.25
Dersity
0.20
Density
0.20
0.15
0.15
0.10
0.00
4
5
Score
(a) luuma-interpretabliity results for accurate pre- dictions. Participants consistently rated GLIME sig- nificantly higher than LIME, indicating that GLIME serves more effective tool for explaining model predictions.
(b) Human-interpretablitty results for incorrect predictions. Participants consistently rated GL.MEE significantly higher than LIME, indicating that GL.ME excels in identifying the model's emrors more effec- tively than LIME.
Figure 6: GLIME helpes explaining model predictions better than LIME.
and Uni(I-V3o, )3o The title of each sub-figure corresponds to the standard deviation of these distributions. Notably, we observe that the value of a does not precisely align with the radius the intended local neighborhood for explanation. Instead, local fidelity tends to peak at larger values than the corresponding a. Moreover, different sampling distributions achieve optimal local fidelity for distinct values. This underscores the significance of selecting appropriate distribution and parameter values based on the specific radius the local neighborhood requiring an explanation. Unlike LIME, GLIME provides the flexibility to accommodate such choices. For additional results and analysis, please refer to Appendix A.5
5.3 Human experiments
In addition numerical experiments, conducted human-interpretabilit experiments to evaluate whether GLIME provides more meaningful explanations to end-users. The experiments consist two parts, with 10 participants involved in each. The details of the procedures employed in conducting the experiments is presented in the following.
1. Can GLIME improve the comprehension of the model's predictions? To assess this, we choose images for which the model's predictions are accurate. Participants are presented with the original images, accompanied by explanations generated by both LIME and GLIME. They are then asked evaluate the degree of alignment between the explanations from these algorithms and their intuitive understanding. Using a 1-5 scale, where 1 indicates significant mismatch and 5 signifies a strong correspondence, participants rate the level of agreement.
Can GLIME assist in identifying the model's errors? To explore this, we select images for which the model's predictions are incorrect. Participants receive the original images along with explanations generated by both LIME and GLIME. They are then asked to assess the degree to which these explanations aid in understanding the model's behaviors and uncovering the reasons behind the inaccurate predictions. Using a 1-5 scale, where indicates no assistance and signifies substantial aid, participants rate the level of support provided by the explanations.
Figure 6) presents the experimental results. When participants examined images with accurate model predictions, along with explanations from LIME and GLIME, they assigned an average score of 2.96 to LIME and 3.37 to GLIME. On average, GLIME received score 0.41 higher than LIME. Notably, in seven out of the ten instances, GLIME achieved higher average score than LIME.
In contrast, when participants examined images with incorrect model predictions, accompanied by explanations from LIME and GLIME, they assigned an average score of 2.33 to LIME and 3.42 to GLIME. Notably, GLIME outperformed LIME with an average score 1.09 higher across all ten images. These results strongly indicate that GLIME excels in explaining the model's 's behaviors.
10

---

Related work
Post-hoc local explanation methods. In contrast to inherently interpretable models, black-box models can be explained through post-hoc explanation methods, which are broadly categorized as model-agnostic or model-specific. Model-specific approaches, such as Gradient [2) SmoothGrad 28], and Integrated Gradient [30), assume that the explained model is differentiable and that gradient access is available. For instance, SmoothGrad generates samples from a Gaussian distribution centered at the given input and computes their average gradient to mitigate noise. On the other hand, model-agnostic methods, including LIME [22] and Anchor [23), aim to approximate the local model behaviors using interpretable models, such linear models or rule lists. Another widely-used model-agnostic method, SHAP [9)], provides a unified framework that computes feature attributions based on the Shapley value and adheres several axioms.
Instability of LIME. Despite being widely employed, LIME is known to be unstable, evidenced by divergent explanations under different random seeds B3
34 38 Many efforts have been devoted to stabilize LIME explanations. Zafar et al. (35] introduced a deterministic algorithm that utilizes hierarchical clustering for grouping training data and k-nearest neighbors for selecting relevant data samples. However, the resulting explanations may not be good local approximation. Addressing this concern, Shankaranaray ana al.
[24] trained an auto-encoder function as a more suitable weighting function in LIME. Shi et al. 25 incorporated feature correlation into the sampling step and considered more restricted sampling distribution, thereby enhancing stability. Zhou et [40] employed hypothesis testing framework to determine the necessary number of samples for ensuring stable explanations. However, this improvement came the expense of substantial increase in computation time.
Impact of references. LIME, along with various other explanation methods, relies references (also known as baseline inputs) to generate samples. References serve as uninformative inputs meant represent the absence of features 4 30 26). Choosing an inappropriate reference can lead to misleading explanations. For instance, if a black image is selected as the reference, important black pixels may not be highlighted [15 6) The challenge lies in determining the appropriate reference, as different types of references may yield different explanations [14] 6 15 In (15), both black and white references are utilized, while (77 employs constant, noisy, and Gaussian blur references simultaneously. To address the reference specification issue, O proposes Expected Gradient, considering each instance the data distribution as a reference and averaging explanations computed across all references.
7 Conclusion
In this paper, we introduce GLIME, novel framework that extends the LIME method for local feature importance explanations. By explicitly incorporating locality into the sampling procedure and enabling more flexible distribution choices, GLIME mitigates the limitations of LIME, such as instability and low local fidelity. Experimental results on ImageNet data demonstrate that GLIME significantly enhances stability and local fidelity compared to LIME. While our experiments primarily focus on image data, the applicability of our approach readily extends to text and tabular data.
8
Acknowledgement
The authors would like thank the anonymous reviewers for their constructive comments. Zeren Tan and Jian Li are supported by the National Natural Science Foundation of China Grant (62161 146004). Yang Tian supported by the Artificial and General Intelligence Research Program of Guo Qiang Research Institute at Tsinghua University (2020GQG1017).
References
[1] Alejandro Barredo Arrieta, Natalia Diaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Ben- jamins, et al. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information fusion, 58:82-115, 2020.
11

---

[2] David Bachrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus-Robert Müller How to explain individual classification decisions. The Journal of Machine Learning Research, 11:1803-1831, 2010.
[3] Naman Bansal, Chirag Agarwal, and Anh Nguyen. Sam: The sensitivity of attribution methods hyperparameters. In Proceedings of the ieee/cvf conference on computer vision and pattern 2020.
recognition, pages
[4] Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, Klaus- Rous-R Müller, and Woj- ciech Samek. Layer-wise relevance propagation for neural networks with local renormalization layers. In Artificial Neural Networks and Machine Learning-ICANN 2016: 25th International Conference on Artificial Neural Networks, Barcelona, Spain, September 6-9, 2016, Proceedings, Part II 25, pages 63-71. Springer, 2016.
[5] Ian Covert, Scott Lundberg, and Su-In Lee. Explaining by removing: A unified framework for model explanation. The Journal of Machine Learning Research, 22(1):9477-9566 2021.
[6] Gabriel Erion, Joseph D Janizek, Pascal Sturmfels, Scott M Lundberg, and Su-In Lee. Improving performance of deep learning models with axiomatic attribution priors and expected gradients. Nature machine intelligence, 3(7):620-63) 631, 2021.
[7] Ruth C Fong and Andrea Vedaldi. Interpretable explanations of black boxes by meaningful perturbation. In Proceedings of the IEEE international conference on computer vision, pages 3429 2000-4377,22200 2017.
[8] Damien Garreau and Dina Mardaoui. What does lime really see in images? In International conference on machine learning, pages 3620-3629. PMLR, 2021.
[9] Damien Garreau and Ulrike von Luxburg. Looking deeper into tabular lime. arXiv preprint arXiv:2008. 1092, 2020.
[10] Loveleen Gaur, Mohan Bhandari, Tanvi Razdan, Saurav Mallik, and Zhongming Zhao. explanation-driven deep learning model for prediction of brain tumour status using mri image data. Frontiers in Genetics, page 448, 2022.
[11] Rory Mc Grath, Luca Costabello, Chan Le Van, Paul Sweeney, Farbod Kamiab, Zhao Shen, and Freddy Lecue. Interpretable credit application predictions with counterfactual explanations. arXiv preprint 1rXiv:1811.0522. 2018.
[12] Tessa Han, Suraj Srinivas, and Himabindu Lakkaraju. Which explanation should choose? a function approximation perspective to characterizing post hoc explanations. arXiv preprint urXiv:2206.0/254. 2022.
[13] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[14] Saachi Jain, Hadi Salman, Eric Wong, Pengchuan Zhang, Vibhav Vineet, Sai Vemprala, and Aleksander Madry. Missingness bias in model debugging. arXiv preprint arXiv:2204.0894, 2022.
[15] Andrei Kapishnikov, Tolga Bolukbasi, Fernanda Viégas, and Michael Terry. Xrai: Better attributions through regions. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4948-4957, 2019.
[16] Thibault Laugel, Xavier Renard, Marie-Jeanne Lesot, Christophe Marsala, and Marcin Detyniecki. Defining locality for surrogates in post-hoc interpretablity. arXiv preprint 17Xiv:1806.07445 2018
[17] Wu Lin, Mohammad Emtiyaz Khan, and Mark Schmidt. Stein's lemma for the reparameteriza- tion trick with exponential family mixtures. arXiv preprint arXiv: 2019.
[18] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference computer vision, pages 10012-10022, 2021.
12

---

[19] Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances neural information processing systems, 30, 2017.
[20] Christoph Molnar. Interpretable machine learning. Lulu. com, 2020.
[21] Amir Hossein Akhavan Rahnama and Henrik Boström. A study of data and label shift in the lime framework. arXiv preprint arXiv: 2019.
[22] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135-1 114, 2016.
[23] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Anchors: High-precision model- agnostic explanations. In Proceedings of the AAAI conference on artificial intelligence, vol- ume 32, 2018.
[24] Sharath M Shankaranarayana and Davor Runje. Alime: Autoencoder based approach for local interpretability. In Intelligent Data Engineering and Automated Learning- -DEALL 2019: 20th International Conference, Manchester, UK, November 14-16, 2019, Proceedings, Part I 20, pages 454-663. Springer, 2019.
[25] Sheng Shi, Xinfeng Zhang, and Wei Fan. A modified perturbed sampling method for local interpretable model-agnostic explanation. arXiv preprint arXiv:2002.07334 2020.
[26] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In International conference on machine learning, pages 3145-3153. PMLR, 2017.
[27] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint 1xXiv:3312.6044 2013.
[28] Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smooth- grad: removing noise by adding noise. arXiv preprint arXiv: 0382500066085 2017.
[29] Pascal Sturmfels, Scott Lundberg, and Su-In Lee. Visualizing the impact of feature attribution baselines. Distill, 5(1): 2020.
[30] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pages 3319-3328. PMLR, 2017.
[31] Joel A Tropp et al. An introduction to matrix concentration inequalities. Foundations and Trendy@ in Machine Learning, 8(1-2):1-230, 2015.
[32] Andrea Vedaldi and Stefano Soatto. Quick shift and kernel methods for mode seeking. In Computer Vision-ECCV 2008: 10th European Conference on Computer Vision, Marseille, France, October 12-18, 2008, Proceedings, Part IV 10, pages 705-718. Springer, 2008
[33] Giorgio Visani, Enrico Bagli, and Federico Chesani. Optilime: Optimized lime explanations for diagnostic computer algorithms. arXiv preprint 0.XXv:20060057/44 2020.
[34] Giorgio Visani, Enrico Bagli, Federico Chesani, Alessandro Poluzzi, and Davide Capuzzo. Statistical stability indices for lime: Obtaining reliable explanations for machine learning models. Journal of the Operational Reseurch Society, 73(1):91-101, 2022.
[35] Muhammad Rehman Zafar and Naimul Mefraz Khan. Dlime: deterministic local interpretable model-agnostic explanations approach for computer-aided diagnosis systems. arXiv preprint 1906. 10263, 2019.
[36] Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part 13, pages 818-833. Springer, 2014.
[37] Yifan Zhang, Haowei He, and Yang Yuan. Consistent and truthful interpretation with fourier analysis. arXiv preprint arXiv:2210.17423 2022.
13

---

[38] Yujia Zhang, Kuangyan Song, Yiming Sun, Sarah Tan, and Madeleine Udell. why should you trust my explanation? understanding uncertainty in lime explanations. arXiv preprint arXiv: 1904. /2991, 2019.
[39] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921 2016.
[40] Zhengze Zhou, Giles Hooker, and Fei Wang. S-lime: Stabilized-lime for model explanation. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 2429-2438, 2021.
14

---

A More discussions
A.1 Implementation details
Dataset selection. The experiments use images from the validation set of the ImageNet- Ik dataset. To ensure consistency, fixed random seed (2022) is employed. Specifically, 100 classes are uniformly chosen at random, and for each class, an image is randomly selected.
Models. The pretrained models used are sourced from torchvision models, with the weights parameter set IMAGENETIK_V.
Feature transformation. The initial step involves cropping each image to dimensions of (224, 224, 3). The quickshift method from scikit- is then employed to segment images into super-pixels, with specific parameters set as follows: and Landom_seed-200 modified fixed random seed. Consistency in the random seed ensures that identical images result in the same super-pixels thereby isolating the source of instability to the calculation of explanations. However, for different images, they are still segmented in different ways.
max_ foxx_dis=200,, ratio=0., 2, This approach aligns with the default setting in LIME, except for the
Computing explanations. The implemented procedure follows the original setup in LIME. The side_color parameter is configured as None, causing the average value of each super-pixel to act as its reference when the super-pixel is removed. The distance iistance_metrriee metrricc is explicitly set to 12, as recommended for image data in LIME [22]. The default value for alpha in Ridge regression is 1, unless otherwise specified. For each image, the model f infers the most probable label, and the explanation pertaining to that label is computed. Ten different random seeds are utilized to compute explanations for each image. The random_ seed parameter in both the Lime ImageExplaal lainerr and the explain ins tance function set to these specific random seeds.
A.2 Stability of LIME and GLIME
Figure 7 illustrates the top-1, top-5, top-10, and average Jaccard indices. Importantly, the results presented in FFigure Jclosely align with those in FFiuure 4al In summary, it evident that GLIME consistently provides more stable explanations compared to LIME.
A.3 LIME and GLIME-BINOMIA BINOMIAL. converge to the same limit
In Figure g the difference and correlation between explanations generated by LIME and GLIME- BINOMIAL are presented. With an increasing sample size, the explanations from LIME and GLIME- BINOMIAL become more similar and correlated. The difference between their explanations rapidly converges zero, particularly when large, such in the case of a = 5. While LIME exhibits a slower convergence, especially with small a, it is impractical to continue sampling until their difference fully converges. Nevertheless, the correlation between LIME and GLIME- BINOMIAL strengthens with an increasing number of samples, indicating their convergence to the same limit as the sample size grows.
A.4 LIME explanations are different for different references.
The earlier work by Jain et al. [14 has underscored the instability of LIME regarding references. As shown in [Section 4.2 this instability originates from LIME's sampling distribution, which relies on the chosen reference references-blace
Additional empirical evidence is presented in Figure 9. Six distinct white, red, blue, yellow image, and the average value of the removed super-pixel (the default setting for LIME) are selected. The average Jaccard indices for explanations computed using these various references are detailed in (Fiiuree 9 The results underscore the sensitivity of LIME to different references.
Different references result LIME identifying distinct features as the most influential, even with a sample size surpassing 2000. Particularly noteworthy is that, with sample size exceeding 2000, the top-1 Jaccard index consistently remains below 0.7, underscoring LIME's sensitivity to reference variations.
15

---

a= 0.25
- 0.50
0.25
or 0.50
1.2
GoneraLLIME-BII -Biomial
GeneraiLIME-
Jepcer Index
Top-5 Jeccard Index
GeneraLLIME BannaaLiM--BBhmaa
Top-5 Jaczerd Index
Jeccerd
LIME+1
LIME
GeneraLLIME- Gauss
LIME 2
LIME+)
LIME+T -
C4
LIME+1 gento
LIME+T
samples (log scale)
samples (log scale)
# samples (log scale)
# samples (log scale)
a- 1.00
a- 5.00
a 1.00
or 5,00
Tep-5 Jacoard Index
Top-5 Jesozard Indext
Jezard
Jezzard
samples (log scale)
samples (log scale)
samples (log scale)
# samples (log scale)
(a) Top-1 1 Jaceard indices for various methods.
(b) Top-5 Jaccard indices for various methods.
d- 0.25
0.50
0 0.25
d 0,50
1.--
GeMeraiLIME-BR -M-onommall GaMeraiLIME- MELLIME-OoR -aass
Jaccard Index
GiMera -Bhomial LimeeriLLMM--OO
Jaccerd Index
LIME
C2
LIME
Top-1 10
C4
LIME+A ol?
Top-1 10
LIME+A -lo
LIME
LIME
12
C4:
samples (log ecsle)
samples (log scale)
@ samples (log scale)
samples (log scale) d' 5500
0- 1.00
d- 5,00
d= 1.00
Jaccard Index
Jaccord Index
Top- 10 Jaccard
Top- 10 Jaccard
2.-
R2-
244
samples (log scsle)
samples (log scale)
# samples (log scale)
samples (log scale)
(c) Top- Jaccard indices for various methods.
(d) Average Jaccard indices for various methods.
Figure 7: T, 5, 10, and average Jaccard indices are computed for various methods. The average Jaccard index is obtained by averaging the top-1 to top-d Jaccard indices.
Poarson
Spearmnn
0.810
0.25
0.54
0:55
0.5
MSE
0.33
5.0
1.0
MAE
0.53
Pearson
Spearmen
6
0.034
0.82
0.50
'/'
samples (log scale)
samples flog scale)
samples (log scale)
samples (log scale)
Figure 8: Difference and correlation between LIME and -INMM-BINOOIAL explanations. Mean Squared Error (MSE) and Mean Absolute Error (MAE) serve as metrics evaluate the dissimilarity berween explanations provided by LIME and
Pearson and Spearman correlation
coefficients are employed to quantify the correlation between these explanations. With an increasing number of samples, the explanations from LIME and GLIME-BINOMIAL tend to show greater similarity. Notably, the dissimilarity and correlation of explanations between LIME and GLIME- BINOMIAL converge more rapidly when a is higher.
16

---

Top-1
Top-5
Tap-10 10
Top-20
588-
1.50
Jencard Index
6.00
Jaccard Indox
6.78
2.70
Japoard
0.00
0.25
2729
0.3
0 1
640
2230
0.00
135
samples (log scale)
samples (log scele)
samples (log scale)
#asmplee (log scale)
Figure 9: The Top-K K Jaccard index of explanations, computed with different references, consistently stays below 0.7, even when the sample size exceeds 2000.
0.25
1.0000
1,0000
1,0000
Fidelit
0.999
0.0000
GeneralLiME Mauusaus
0.3900
0.9600
0.0400
GeneralL IME Uniform
000 000100
0.0002
0.5290
0.0000
0.95
0.9955
100000
Figure 10: The local fidelity of GLIME in the e, neighborhood
A.5
The local fidelity of GLIME
Figure 5 presents the local fidelity of GLIME, showcasing samples from the & neighborhood
1 z - x x||2 S <} around x. Additionally, Figure 10 and Figure 11 illustrate the local fidelity of
GLIME within the & neighborhood {z||zz x|| c) and the Lx neighborhood {z|||z x||0o S c). respectively.
A comparison between Figure 5) and Figure 10 reveals that, for the same GL.ME can explain the local behaviors of f within a larger radius in the é neighborhood compared to the & neighborhood. This difference arises from the fact that {z|lz x|12 e) defines a larger neighborhood compared to {z|lzz - x || S }} with the same radius e.
Likewise, the set [z||z xl|ec I c) denotes a larger neighborhood than {z|lız - x||2 < c). causing the local fidelity to peak at smaller radius e for the fox neighborhood compared to the E2 neighborhood under the same a.
Remarkably, GIAME-LAPLACE consistently demonstrates superior local fidelity compared GL.MEE GAUSS and ULLME-UUIIFRM, Nevertheless, in cases with larger 6, GLIME-GAUSS sometimes surpasses the others. This observation implies that the choice of sampling distribution should be contingent on the particular radius of the local neighborhood intended for explanation by the user.
a= 0.25
0.5
1000
3000
Fidelty (-}
3
0.575
0.363
3.000
0.303
GeneraiL MitrriLLIMEGGuusS
GeneraiLIME--Lagd
2.000
3,800
0.505
600
-
0.675
10'
10'
Figure 11: The local fidelity of GLIME in the loo neighborhood
17

---

A.6 GLIME unifies several previous methods
KernelSHAP 19]. KemnelSHAP integrates the principles of LIME and Shapley values. While LIME employs linear explanation model to locally approximate f. KerneISHAP seeks linear explanation model that adheres to the axioms of Shapley values, including local accuracy, missingness, and consistency [19]. Achieving this involves careful selection of the loss function {(.,) .). the weighting function (-), and the regularization term R. The choices for these parameters in LIME often violate local accuracy and/or consistency, unlike the selections made in KemelSHAP, which are proven to adhere to these axioms (refer to Theorem 2 in (9
Gradient 2 27 This method computes the gradient ff to assess the impact of each feature under infinitesimal perturbation 2 27
SmoothGrad [88]. Acknowledging that standard gradient explanations may contain noise, Smooth- Grad introduces a method alleviate noise by averaging gradients within the local neighbor- hood of the explained input [28). Consequently, the feature importance scores are computed as Bo [ e)].
DLIME [35]. Diverging from random sampling, DLIME seeks a deterministic approach to sample acquisition. In its process, DLIME employs agglomerative Hierarchical Clustering group the training data, and subsequently utilizes k-Nearest Neighbour to select the cluster corresponding to the explained instance. The DLIME explanation is then derived by constructing a linear model based on the data points within the identified cluster.
ALIME [24]: ALIME leverages an auto-encoder to assign weights to samples. Initially, an auto- encoder, denoted as A-).), is trained on the training data. Subsequently, the method involves sampling nearest points to x from the training dataset. The distances between these samples and the explained instance x are assessed using the ε distance between their embeddings, obtained through the application of the auto-encoder AE For sample its distance from is measured as ||AE (2) - (x and its weight is computed as
AE(x) The final explanation derived by solving weighted Ridge regression problem.
A.7 Results on tiny Swin-Transformer 18
The findings on the tiny Swin-Transformer align with those ResNet18, providing additional confirmation that GLIME enhances stability and local fidelity compared to LIME. Please refer to Figure and Figure 14 for results.
Figure 12
A.8
Comparing GLIME with ALIME
While ALIME [24] improves upon the stability and local fidelity of LIME, GLIME consistently
surpasses ALIME, A key difference between ALIME and LIME lies in their methodologies: ALIME employs encoder to transform samples into embedding space, calculating their distance from the input to be explained as [[AE(()) (x II. whereas LIME utilizes a binary vector z E {0, 1]dd to represent sample, measuring the distance from the explained input |] 2|
Because ALIME relies on distance in the embedding space assign weights to samples, there is risk of generating very small sample weights if the produced samples are far from x, potentially resulting instability issues.
In our Image experiments comparing GLIME and ALIME, we utilize the VGG16 model from the repository
as the encoder in ALIME. The outcomes of these experiments are detailed in Table The findings demonstrate that, although ALIME demonstrates enhanced stability compared LIME, this improvement is not substantial the improvement achieved by GLIME, particularly under conditions of small a or sample size.
A.9 Experiment results on IMDb
The DistilBERT model is employed in experimental evaluations on the IMDb dataset, where 100 data points are selected for explanation. The comparison between -INOMIAL and LIME
18

---

== 0.25
0.50
GLIME-Binoma
Top 20 Laocard Index
LIME
Top-20 Jaccard
0.-
LIME+1 =
0.7
LIME+T
124
LIME
GLIME-BBnomil
GLIME-Gaule
04-
GLIME-Unifam
CLIME-Laplaoe
# samples (log ecale)
semples (log scale)
-=
1.00
d 5.00
0.2
Top- 20 Jacecard Indixx
12
0.-
07-
Top-20 Jacsard
0.1
04-
10'
samples (log scale)
semples scaie)
(a) Stability across various methods (tiny Swin- Transformer). The reported metric the Top-20 Jaccard index. LIME+>
and LIME++ represent LIME without regularization and without weighting, respectively. LIME exhibits instability, par- ticularly when small, whereas GLIME demon- strates enhanced stability across varying a values. Notably, the absence of weighting or regulariza- tion, LIME's stability significantly improves when a is small. Conversely, the impact of regularization and weighting LIME's stability marginal when is large.
(b) I7 comparison among LIME and GLIME vari- ants with different sampling distributions (tiny Twin-Transformer) For cach image and method, 2048 samples utilized compute the explana- tion and the corresponding R². Notably, LIME yields nearly zero R when a 0.25 and 0.5, indicating an almost negligible explanation produced by LIME. In contrast, the R2 values of LIME consistently lower than those of GLIME, underscoring that GLIME enhances the local fidelity of LIME.
Figure 12: GLIME markedly enhances both stability and local fidelity compared to LIME actoss various values of a.
Table 1: Top-20 Jaccard Index of GLIME-BINOMIAL, GLIME-GAUSS, and ALIME. GLIME- BINOMIAL and GLIME-GAUSS exhibit significantly higher stability than ALIME, particularly in scenarios with small limited samples.
# samples
128
256
512
1024
GLIME- BINOMIAL
0.952
0.981
0.993
0.998
0.25
GLIME-GAUSS
0.872
0.885
0.898
0.911
ALIME
0.618
0.691
0.750
0.803
GLIME-BINOMIAL
0.596
0.688
0.739
0.772
0 0.5
GLIME-GAUSS
0.875
0.891
0.904
0.912
ALIME
0.525
0.588
0.641
0.688
0.533
0.602
0.676
0.725
GLUME-GAUSS
0.883
0.894
0.908
0.915
ALIME
0.519
0.567
0.615
0.660
GLIME- BINOMIAL
0.493
0.545
0.605
0.661
= 5
GLIME- GAUSS
GLIME-GAUSS
0.865
0.883
0.898
0.910
ALIME
0.489
0.539
0.589
0.640
depicted in Figure 15 usiing the Jaccard Index. Our findings indicate that GLIME-BINOMIAL consistently exhibits higher stability than LIME across range of values and sample sizes. Notably, smaller a values, KL.IM--BINOMIAN demonstrates a substantial improvement in stability compared to LIME.
19

---

d 0.25
d 0.50
J 0.25
d 0.50
GLIME-Binomiel GLIME-GoUL LIME
Top-5 Jeccserd Indeex
Top-5 Jeccerd Inclex
ca-
FLIME-Binooaa GLIME-GaUsS LIME
Jeccard
Top-1 Jecccerd
R.
C.
LIME
LIME-IT
LIME=1=0
LIME-T=1
12-
samples (log scale)
samples (log scale)
@ samples (log scale)
B semples (log scale)
o= 1.00
-= 5.00
or 1.00
or 5.00
Jecccerd Index
Top-S Jeacord Index
Top-5 Jeacard Index
Jeccord
22-
Ce-
7.-
124
12
10"
samples (lgg scale)
samples (log scale)
0 @samples (log scale)
# semples (log scale)
(a) Top-1 Jaccard index of different methods (tiny Swin- Soinn-Tranfformer.
(b) Top-5 Jaccard index of different methods (tiny Swin- Transformer).
o- 0.25
- 0.50
or 0.25
0.50
GLIME-BInomia GLIME-GauRe
GLIME -Biomial
6.8-
GLIME-Gause LIME
LIME
Jeccard Index
Jsccard lindex
Top-10 Jeccard
LIME-) D
Top- Jeccard
LIME-A LIME-IT
+
-0'
13'
samples (log scale)
samples (log scale)
# samples (log scale)
# samples (log scale)
-= 1.00
a 5.00
or 1.00
or 5.00
14
R 1 2
Top- 10-110.00 Jecpard Index
Jeccard Indexx
Jscoard Index
Top- Top-10.0. Jeccard
2 I2
samples (log scale)
samples (log scale)
# samples (log scale)
# samples (log scale)
(c) Top-10 Jaccard index of different methods (tiny Swin- Transformer).
(d) Average Jaccard index of different methods (tiny Swin-Transee Transformer).
Figure 13: Top-1, 5, 10, and average Jaccard indices are calculated for various methods, and the average Jaccard index represents the mean of top-1,.... d indices for the tiny Swin- Transformer.
MSE
MAE
Pearson
Spearman
0.300
6.45-
0,25
0835
0.035
0.5
1.0
0.000
5.0
MAE
0.83-
Dearson
0.00
03010
0.73
0336
626-
0.03
167
12'
1e'
12a
16'
samples (log scale)
samples (log scale)
samples (log scale)
samples (log scale)
Figure 14: Diffference and correlation in LIME and GLINE-BINOMIAL explanations (tiny Swin-Transformer! Mean Squared Error (MSE) and Mean Absolute Error (MAE) quantify the divergence between LIME and BAAMM-BI explanations. Pearson and Spearman correlation coefficients gauge the correlation. With increasing sample size, the explanations tend align more closely. Notably, the difference and correlation converge more rapidly with larger values a.
20

---

3.05
0.7
Swetiid Arromia
Top-20 jeccan kndde.
3.00
methed GUIVE Binombal UNE
0.6
0.55
0.25
0.5
3.0
300
400
500
600
700
800
9.00
1000
sasmplea
(a) Stability comparison between GLIME- BINOMIAL and LIME. The top-20 Jaccard index is reported. illustrating that GLIME displays superior stability compared to LIME across diverse values of Notably, GLIME'S stability remains consistently robust, showing limited sensitivity to changes in a. contrast, LIME exhibits increased stability a values grow larger.
(b) Comparison of H² between LIME and GLIME- BINOMIAL under various sampling distributions. Using 2048 samples for explanation computation, π² values are computed for cach image and method. is notcworthy that when o 0.25, LIME exhibits nearly negligible R values.
Figure 15: GLIME significantly enhances both stability and local fidelity compared LIME across
various a values.
B Proofs
B.1 Equivalent GLIME formulation without x(-)
By integrating the weighting function into the sampling distribution, the problem to be solved is
w GLiME = arg min Exr (x + AR(v)
v
arg min I arg miny
JEc (( AR(v)
Sax (((())
XR(v)
arg min
Jo T(u')P(u)du
A R(v) P(z) = T(2')P(z) Z = / ((u))P(u)du
= arg min
([f(z).g(z) ( + z
Rd
Z
arg min (( + > R(v)
B.2
Equivalence between LIME and GLIME-BINOMIAL
For LIME, P = Uni((O, 13d) and thus P(z', ||z = =k) = kk 00,1,...
dd so that
(k--d)// (0)
ed/02
x(u')P(u)du
(1 +
2/0
2d
2d
Rd
Thus, we have
P(z) (zz)P((z)
(
kkj
Z
Z
(1 el+0//
Therefore, is equivalent LIME.
21

---

B.3 LIME requires many samples to accurately estimate the expectation term in Equation
In Figure it is evident that a lot of samples generated by LIME possess considerably small weights. Consequently, the sample estimation of the expectation in EEquatio tends to be much smaller than the true expectation with high probability. In such instances, the regularization term would have a dominant influence on the overall objective.
Consider specific parameters, such as 0.25, n 1000, = 20 (where and n are the default values in the original implementation of LIME). The probability of obtaining a sample z' with ||z llo = d - 1 or d is approximately d + 2d d+1
= 2 x 10-5. Let's consider a typical scenario where l.f(z) E [0, 1J, and (f(z) - v z²)² is approximately 0.1 for most z, z'. In this case, (02n~Uni( (z)) (
2
2
- v vTz') = 0.1 Ex= elk-aa/0"
2a ~ -10-7 However, if we lack samples z' with | 000 = d - or d, then all samples z with |zl|lo - d - 2 have weights T (2Z) $ exp( - 3/) ~ 1.26 x 10 10-14_ This leads to the sample average 1/2 - क (f)
)²²² < 1.26 x 10-15 < 10 The huge difference between the magnitude of the expectation term in Equation and the sample average of this expectation indicates that the sample average not an accurate estimation of Ext~Uni 1))1) [T) Additionally, under these cireumstances, the regularization term is likely to dominate the sample average term, leading to an underestimation of the intended value of v. In conclusion, the original sampling method for LIME, even with extensively used default parameters, not anticipated to yield meaningful explanations.
v >²) (if we do not get enough samples).
B.4 Proof Theorem 4.1
Theorem B.1. Suppose samples {z Uni({0, 1)d) are used to compute LIME explanation. For any > 0, 8 E (0. 1). if = (e -2, 2. lo log(4d/6) (o)ddd/))).) J), S n, we have TP( TP[ (yLLME wLIMB <
<) 1 - 8. LIME 1im LIME
Proof. To compute the LIME explanation with n samples, the following optimization problem is solved:
LIME
arg min 1
Σ क. (f)) T 22
|
71 1=1
72
Let L 7. E (2))( v x22² + ||vl2 Setting the gradient of with respect to v to
zero, obtain:
-2 (z))) T 21)2 + 2 =
which leads to:
W
LIME
n ( Σ
π (2))22(2)) (+
+ n
+1)
71
Denote En 1 En π (T44 (24)2((z))
T + 2 r, = 1 E. TT T(2)2 (2:), E = lim En,, and
4om n. Then, we have:
w LIME
T n, w LIME = -T-
To prove the concentration of w LIME we follow the proofs in [8 (1) First, we prove the concentration of Σn; (2) Then, we bound | the following inequality:
1 (3) Next, we prove the concentration of Tni (4) Finally, use
| T, E--rI
S 2 ||
Flla 21 | Σ, Σ|I
when Σ (E Σ)| < 0.32 [
Before establishing concentration results, we first derive the expression for Σ.
Expression of Σ.
1 Σ, T 72))(2))+++ A Σn
1 ME
Σ π (2))21 Σ π (Z,
n ((,
3 (Z)))())))))
1 (E(
1/4 (2)
22

---

By taking n - oc, we have
En + E (a1 - a2) + ag
where
=E (x())= = Ey =Bo
01
0/- P2(2 1|||2'|1 A) k)
(k-dt//² k (t)
d
k=0
d
(d-1)
(k=
(6-1)
2d
k=0
(d-l)
1)/o (1-d)/0²
2d
k=U
=e d)//0 (1 (1+
(1 e1
2d
2d
02 =1E Uni )z/2/]
d) Plo = = 11122/10 = x))([1z'11 110 = k)
()2 k(k 11-11_1 1) (+)
d(d 1) 2d
(d-2) (222)
(k-d)/0²
2d
(
2k/2/ 12-d
(-
2d
k=
22 d)/0" (1 + e d
(1 (1te -
2d
2d
By formula, we have
Σ-1= ((a) - az) + +a1l1
(1
02 11 )-1
a1 022
01 02
1
11T
01-02
(3) - 12 811
01
1 a1 -kg
where
81 = (a] (0) + (d - 1))²)
01 + (d 2)02
32 = -a) + (d 1)
022
In the following, we aim to establish the concentration of wLIME
Concentration of Σ.. Considering 0 S <-)) < 1 and z, E {0, 1)", each element within E,, resides within the interval of [0, 2]. Moreover,
1 < ai
(1 + jd-
2d-1
2d
2d
2d
1
(I+e
2d-2
2d
<
02
2d
2d
23

---

1/o² San - = d (1 te d2 2d-
2d
2
4
The elements within Σ are within the range of [0, 1 Consequently, the elements in Σ, Σ fall within the range 2). Referring to the matrix Hoeffding's s incquality BT. holds true that for > 0,
nt2
P( E| 2 t) S 2d exexp(
32d
Bounding |
| d3f (d d).33
Because
d S a1 (d 1)a2 S
2+ (d-1)
++1
2d
4
4
d-1 2d
2+ (d 2)
d
aa + (d 2)02 v
4
we have
01 + (d (d-2))cs -2)a2
|811
2/0
, < 22d
(a1 (2) + (d 1)a2)
a1 02
1821
Q2
1/2
2d
(a1 a2)((1 + (d - 1)a2)
(a1 + (d 1)a2)
33 $ d 22222222/0 2/0²
so that
Σー = d,9 + (de - d) < 2/0² + (d2 - -dd- 2/02 < 2,122d22/0*
Concentration of I,. With lfl 1, all elements within both I, and I exist within the range of [0, 1]. According to matrix Hoeffding's 's inequality [31], for all > 0,
nt²
P r
< 2d exp
8d
Concentration LIME, When | Σ)
S 0.32 81, we have
- S -/ |r Flla + + - E, Σ/
Given that
Σ EE, - Σ)| D E, - - / | -El
Exploiting the concentration of En, where 2 n1 22222222 2/0² log(4d/8) and t
5- 5-222255-00 005.--0-... 1epo-dd-1f 1/02 we have
nt² 32df2
nt²
P( En E112 2 t) < exp
< 2dexp
32d²

---

Therefore, with a probability of at least
2' we have
Σ (E,
3)| - 1 En
Σ < 22/22/
12 Σ
S
0.32
For 2 2d2 22/0² log( (s//4//6) and t2 = 2 dd
de
1/0² the following concen-
tration inequality holds:
7222
P - T| (2) < exp
8d
6
In this context, with probability of at least 1
é 2. we have
E
Tn r
4
Considering T <
vd, we select n 2 ng =
26242 4/0²
log(4d/6) and to =
2 1.52 2--55-220
2d 2/0 e, leading to
1ltg 32d
n2t
5
E( 22 2 (3) S 2d 2dexp
S 2dexp
32.d2
2
With a probability at least 1 - 58/2, we have
Σ 1 En - E S
In summary, we choose 2 max (1 n2, ns). and then for all E > 0,8 E (0.1)
P[ Dn T E ' 2 28
B.5 Proof of Theorem 4.2 and Corollary 4.3
Theorem B.2. Suppose ~P such that the largest eigenvalue of (z') is bounded by R and [ = (a1 - I + 12l1 [ar(zzzz') "ll S v2, 1(z( [())a S M for some M > (z/ are i.dd samples from P and are used to compute GLIME explanation GLIME For any
e <> 0,0 (0, 1), in Qiee- -22
log(4d/)) where 72 = d + (cL d) 31 (an
(d-2)a2)/A3) /00 B3 -02/Bo. Po (a) (a1-)))(a (2) + (d 1)a2)). we have PP GLIME w GLIME ||2
<) 2 6. GLIME lim GLIME
Proof. The proof closely resembles that of Theorem 4.1 Employing the same derivation, deduce that:
== + +2 E-1 (3) [8)- + 3,11
where
a1 + x (d 2)02
02
B1
$2 (a1 + X (2) + (d- 1)az)
(a1 + 1 - (2) + - (d 1)az)'
Given that A < R and [/Var(z(zz)) 1]2 & v according to the matrix Hoeffding's
inequality (31), for all 0:
nt²
P(| E, E112 2 t) < 2d exp
8
25

---

Applying Hoeffding's s inequality coordinate-wist wise, we obtain:
nt2
P( - F|| 2 t) cxp
8M222
Additionally,
Σ 1 d3f + (d² - -d)32 7?
By selecting n 2 n1 = 2² log(4d/6) and & = 2 2 we obtain
n1 t
8
P( - E 2 (1) S 2dexp
8v2
2
with probability of least 8/22.
|>- - -}1
| | Σ1| S 7t1 0.32
Letting n 2 112 2 log(4d/6) and 12 2 2- 11 we have
P - F 2 t2) S 2dcxp (-
n2t3
-
with probability of least 1 8/2.
|T r S st2 S
As |T|| S M, by choosing have
2 n3 2 log /8) and to 2 2-2cM-
we
P( - E12 2 ts) S 2d (- nntt3 8
2v 2
and with a probability of at least 1 - 5/2,
| 1 E., - 2 4
Therefore, by choosing n = max n2, ny). we have
E T, E
'
20) 58
Corollary B.3. Suppose (z.)²= are i.i.d. samples from P(z', 1|22 = =k) e2/0²² /11 yd, 1...., d are used to compute
explanation. For any > 0,5 E 1), if w Binomial
-286 Afo22 log(Ad/6)), we have P([] Vy Binomial
||2 t) 1 8. w Binomial
limp-100 Binomial
Proof. For GLIME-BINON BINOMILL, each coordinate of z' z'(z') follows a Bernuulii distribution, ensuring the bounded variance of both z' =x(x²) and (z' f(z'(( )). Additionally, we have
| Vd.
26

---

al =[((s])] = [|]]] e 1/2
1 e!/0²
P(2 = 11||'1lo 10 = k)F([lz'lo = k)
(+)
(1+ cloood
(1+elo
e2/0²
(1+c e1/02
1 1+ c//02
e4/0²
02 = E [
+
P(2 1, 1111z111 =))(((21o k)
k(k 1) ch)
d(d 1) (1+ el/oogd
k=1 + elloogd
(1 + 1lteo/// 2/02
22/0²
(1 (1+e1/0ap el/0272 = =a
(1 (1+el11) el/02jd
a1 + + (d - 2)02
1
12
(an
(2) + x + (d. 1)02)
1
= E 1100²
(1 e/ae/ S 1/2
0n + 022
01
02
1821 =| - (a1 + X - -22) + X + (d - 1)a2)
a2
12
az
< (ai (2) + + (d - 1)02)²
01
(1 - a1) + (d - 1)a2)²
-1/0² (1 + e2/00/3
2201/0²
0102
(1 (1- (d 1)02)2
1)2
(d 112
Therefore,
dø + (& -d) < 1e te//0² d
d- S 1/0
B.6 Formulation of SmoothGrad
Proposition B.4. SmoothGrad is equivalent to MI.MME with z = z' + x where of ~ (0 I), ((f(z),9(z') = (f(z) =)² and x Tz)) 1, = O.
The explanation returned by GLIME for f at x with infinitely many samples under the above setting is
1
w
12 f(z' +x)] Eg /o
which is exactly SmoothGrad explanation. When 0. w + Vf(x + 2)
27

---

Proof. To establish this proposition, we commence by deriving the expression for the GLIME explanation vector w"
Exact Expression of Σ: For each i 1, n, let 2 ~ (N0,000000 TT). In this context,
Σ (x Y
Ex 2 kd
Σn
Ex Ala
4. 4x (chaa)
This implies
o
Σ = (=/))/)))))
Σ-1
Consequently, we obtain
w' - - - [x= +z)] Exx.voo. z)]
The final equality is direct consequence of Stein's 's lemma
28


Reviews:
[{"review_id": "BfuNgLmGpY", "reviewer": "Reviewer_NTaD", "summary": "The paper introduces GLIME as a solution to tackle the issues of instability and diminished local fidelity encountered in the original LIME method. To address the problem of instability, GLIME employs a novel sampling scheme that guaranteed to have a faster sampling rate. The diminished local fidelity problem is resolved by modifying sampling distribution so that nearby samples have higher probability to be sampled.\n\nDisclaimer: I only read the main text and do not check correctness of the proposed sample complexity argument.", "questions": "In Section 4.3 of the paper, it is stated that the local fidelity problem arises due to the utilization of a high regularization weight. Could this issue be addressed by reducing the regularization weight or, in more extreme cases, completely eliminating the regularization. ", "limitations": "The potential societal impact is not stated in the main paper.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The problem they tackle with is specific and well-formulated. The proposed solution is simple and effective.\n2. Their methods are supported by sample complexity analysis. This analysis not only provides mathematical evidence that the original LIME approach necessitates an exponentially large number of samples to achieve convergence, but also demonstrates that their proposed method requires only a polynomial number of samples, offering a significant improvement in efficiency.", "weaknesses": "One weakness would be limited applicability of the proposed GLIME. The paper only demonstrates it can only be applied to the image domain. As other features from different domains, such as texts or categorical features, are not necessarily to be continuous, GLIME equipped with continuous distributions may not resolve the local fidelity issue.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer NTaD for reviewing our paper and for the insightful comments. We hope our answers will address your concerns.\n\n> **Q1:** One weakness would be limited applicability of the proposed GLIME. The paper only demonstrates it can only be applied to the image domain. As other features from different domains, such as texts or categorical features, are not necessarily to be continuous, GLIME equipped with continuous distributions may not resolve the local fidelity issue.\n\n**A1:** We have conducted experiments on text data. We use the DistilBERT model and select 100 data points from the IMDb dataset as inputs to be explained. We run experiments comparing GLIME-Binomial and LIME, and the results are shown in the table below. The sample size is set to 1024. From the results, it can be seen that GLIME has better local fidelity compared to LIME, especially when $\\sigma$ is small.\n\nGLIME-B: GLIME-Binomial; \n\n|       | $\\sigma=0.25$ |       | $\\sigma=0.5$ |       | $\\sigma=1$ |       | $\\sigma=5$ |       |\n| ----- | ------------- | ----- | ------------ | ----- | ---------- | ----- | ---------- | ----- |\n|       | GLIME-B       | LIME  | GLIME-B      | LIME  | GLIME-B    | LIME  | GLIME-B    | LIME  |\n| $R^2$ | 0.688         | 0.001 | 0.691        | 0.160 | 0.693      | 0.579 | 0.693      | 0.682 |\n\nFor data with discontinuous feature values, the samples $\\mathbf{z^\\prime}$s naturally tend to be far away from the input being explained. However, because of the large distances, the weights of each sample after weighting are very low. This may cause the sum-of-squares term to be dominated by the regularization term, leading to explanations tending towards zero and resulting in a smaller $R^2$ and poorer local fidelity. GLIME does not suffer from this problem, as the sum-of-squares term is not dominated by the regularization term, leading to better local fidelity. We will conduct additional experiments in the future to compare GLIME and LIME.\n\n> **Q2:** In Section 4.3 of the paper, it is stated that the local fidelity problem arises due to the utilization of a high regularization weight. Could this issue be addressed by reducing the regularization weight or, in more extreme cases, completely eliminating the regularization.\n\n**A2:** In our paper, we propose that the instability issue in LIME is attributable to the use of a high regularization weight and low sample weights. To address this concern, we conducted experiments aimed at evaluating whether reducing the regularization parameter $\\lambda$ or increasing the sample weight function $\\pi(\\cdot)$ could improve stability. For a comprehensive analysis of the results and discussion, please refer to Figure 4(a) and lines 278-289. It is important to note that while removing regularization does offer some improvement, it is not as significant as the improvement achieved by GLIME. This is because, in the absence of regularization, the LIME solution involves computing $(Z^\\top WZ)^{-1}$, where $W$ is a diagonal matrix with sample weights on its diagonal. The dependence on $W$ makes LIME sensitive to sample weights. Furthermore, since the sample weights are nearly zero, $Z^\\top WZ$ becomes low-rank, resulting in numerical instability. Additionally, increasing the sample weights to 1 only provides limited improvement in stability for LIME. This is because the large sampling space of LIME causes the obtained explanations to highly depend on the selected samples. GLIME, on the other hand, does not encounter these issues as its solution does not require inverting a low-rank matrix, and it operates within a local sampling space.\n\n> **Q3:** The potential societal impact is not stated in the main paper.\n\n**A3:** The positive impact of GLIME is its ability to enhance the interpretability of machine learning, making the use of machine learning in various fields safer and more trustworthy. It reduces the potential for social security incidents caused by machine learning. It can help users better understand the behavior of machine learning models. The potential negative societal impacts of GLIME are not yet clear at the moment. "}, {"review_id": "gvt8aBVjHg", "reviewer": "Reviewer_DXPw", "summary": "In this paper, the authors present GLIME an approach for explainable ai that generalizes LIME. Here, the authors present a framework that encompasses different explainability methods as instantiations of different aspects such as loss function, sampling function, etc. \n\nThe authors also present an analysis of problematic cases for LIME. More precisely, they show how the interaction of the weighting and regularization can cause instability in the explanations and how the samples generated in LIME might not be close to the original sampling space.\n\nThe paper then presents different sampling procedures and show empirically how they converge and how stable the explanations are given different parameterizations.\n", "questions": "My main concern is regarding the contribution and the comparison to ALIME. Could you mention a bit more in terms of your contributions regarding that method?\nThis does not diminish your theoretical contributions in terms of the convergence in comparison to LIME, however it can potentially make GLIME less appealing to the general user than ALIME.\n\nRegarding your GLIME-gauss, and I believe ALIME is similar, wouldn't the sampling space be too close to the original image we want to explain? It seems like simply a noise addition similar to difussion models.", "limitations": "I don't consider the paper to have potential negative societal impact. \n\n", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "I find the paper insightful, in particular the aspect of the weights becoming all zeros in the standard case for low values of sigma.\n\nThe paper is easy to read, technically sound and guides the reader through the concepts in a solid yet understandable way.\nThe technical contributions are solid and overall provides a good foundation for further research. \n\nOverall the paper is original and I would say significant as it has the potential to become the standard replacement for LIME.", "weaknesses": "The main concern I have is regarding the empirical section. In particular, you mention two main issues with LIME being the interaction of the weighting with regularization and sub-par sampling. However, it would seem like ALIME does not suffer from those two issues. It would be good to see a comparison of GLIME and ALIME in Fig. 4.\n\nThere are some minor improvements I would suggest on the presentation. \nI would suggest you unify the color scheme in Fig. 4 and if possible present as many of the methods in both graphs.\nIs there a typo in the norm of the weighting function in line 171? shouldn't it be 2 and 2?\nThe language on the sub-section in Feature attributions could be improved. \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer DXPw for reviewing our paper and for the insightful comments. We hope our answers will address your concerns.\n\n> **Q1:** You mention two main issues with LIME being the interaction of the weighting with regularization and sub-par sampling. However, it would seem like ALIME does not suffer from those two issues. It would be good to see a comparison of GLIME and ALIME in Fig. 4.\n\n**A1:** Although ALIME improves stability and local fidelity over LIME, GLIME still outperforms ALIME. One major difference between ALIME and LIME is that ALIME uses an encoder to encode samples into embedding space and compute their distance with the explained input in embedding space $\\\\\\|\\mathcal{AE}(\\mathbf{z}) - \\mathcal{AE}(\\mathbf{x})\\\\\\|_2$ while LIME uses a binary vector $\\mathbf{z} \\in \\\\\\{0,1\\\\\\}^d$ to represent a sample, and use $\\\\\\|\\mathbf{1} - \\mathbf{z}\\\\\\|_2$ as the distance between the sample and the explained input. ALIME use distance in embedding space to weight samples. Therefore, if the samples generated by ALIME is distant from $\\mathbf{x}$, sample weights may still be very small and cause instability problem. \n\nWe conduct experiments to compare GLIME and ALIME. We utilize the VGG16 model provided by the repository https://github.com/Horizon2333/imagenet-autoencoder as the encoder in ALIME for our experiments on ImageNet. The table below shows the results of the experiments. It can be observed that the improvement of ALIME is still not as significant as that of GLIME, especially when $\\sigma$ is small or the sample size is small. We will include results of ALIME in Figure 4 in the final version.\n\nGLIME-B: GLIME-Binomial;  GLIME-G: GLIME-Gauss\n\n| # samples | $\\sigma=0.25$ |         |       | $\\sigma=0.5$ |         |       | $\\sigma=1$ |         |       | $\\sigma=5$ |         |       |\n| --------- | ------------- | ------- | ----- | ------------ | ------- | ----- | ---------- | ------- | ----- | ---------- | ------- | ----- |\n|           | GLIME-B       | GLIME-G | ALIME | GLIME-B      | GLIME-G | ALIME | GLIME-B    | GLIME-G | ALIME | GLIME-B    | GLIME-G | ALIME |\n| 128       | 0.952         | 0.872   | 0.618 | 0.596        | 0.875   | 0.525 | 0.533      | 0.883   | 0.519 | 0.493      | 0.865   | 0.489 |\n| 256       | 0.981         | 0.885   | 0.691 | 0.688        | 0.891   | 0.588 | 0.602      | 0.894   | 0.567 | 0.545      | 0.883   | 0.539 |\n| 512       | 0.993         | 0.898   | 0.750 | 0.739        | 0.904   | 0.641 | 0.676      | 0.908   | 0.615 | 0.605      | 0.898   | 0.589 |\n| 1024      | 0.998         | 0.911   | 0.803 | 0.772        | 0.912   | 0.688 | 0.725      | 0.915   | 0.660 | 0.661      | 0.910   | 0.640 |\n\n> **Q2:** Could you mention a bit more in terms of your contributions regarding that method?  Regarding your GLIME-gauss, and I believe ALIME is similar, wouldn't the sampling space be too close to the original image we want to explain? It seems like simply a noise addition similar to diffusion models.\n\n**A2:** Both GLIME-Gauss and ALIME utilize Gaussian sampling and employ Ridge regression to solve linear explanations. However, there are notable differences between the two methods.\n\n* Firstly, ALIME initially trains an auto-encoder and then uses this auto-encoder to compute embeddings for each sample. On the other hand, GLIME does not require the training of an auto-encoder or any calculation of embeddings. Consequently, GLIME is significantly more efficient than ALIME.\n\n* Secondly, in ALIME, sample weights are computed based on the distance between the embeddings of the samples and the embedding of the original input, denoted as $\\mathbf{x}$. These sample weights are then used in a weighted Ridge regression to obtain explanations. Conversely, GLIME does not involve the use of sample weights. In ALIME, sample weights emphasize locality, whereas GLIME enforces locality by employing a sample distribution that more frequently samples samples closer to $\\mathbf{x}$.\n* Based on the comparison of the results between GLIME and ALIME mentioned above, although ALIME has improved stability compared to LIME, GLIME shows even better stability than ALIME, especially when the value of sigma is small and the sample size is small.\n\nAdditionally, GLIME-Gauss introduces Gaussian noise to super-pixels, while diffusion models add noise to pixels. The distance between samples and the original image can be controlled by adjusting the parameter $\\sigma$. In Figure 3, we showcase the distances of samples from the original image, with larger $\\sigma$ resulting in samples being farther from the desired $\\mathbf{x}$.\n\n> **Q3:** I would suggest you unify the color scheme in Fig. 4 and if possible present as many of the methods in both graphs.\n\n**A3:** Thanks for your suggestions. In the final version, we will unify Figure 4(a) and Figure 4(b). However, it is important to clarify that these figures are intended to compare different aspects of LIME and GLIME.\n\nFigure 4(a) aims to demonstrate how GLIME improves stability compared to LIME, while also illustrating the impact of regularization and weighting on the stability of LIME. To provide a comprehensive comparison, we will include LIME with regularization parameter $\\lambda$ set to 0 (LIME$+\\lambda=0$), LIME with $\\lambda = e^{-d/\\sigma^2}$ (LIME$+\\lambda = e^{-d/\\sigma^2}$), and LIME with sample weight function $\\pi$ set to 1 (LIME$+\\pi=1$).\n\nOn the other hand, Figure 4(b) focus on comparing the local fidelity of LIME and GLIME, specifically by including different sample distributions used in GLIME.\n\n> **Q4:**  Is there a typo in the norm of the weighting function in line 171? shouldn't it be 2 and 2? The language on the sub-section in Feature attributions could be improved.\n\n**A4:** Thank you for providing the correction. The accurate expression should be $\\pi(\\mathbf{z^\\prime}) = \\exp(-\\\\\\|\\mathbf{1} - \\mathbf{z}^\\prime\\\\\\|_2^2 /\\sigma^2)$. We will make sure to revise the language in the final version for further improvement."}, {"review_id": "HKmVbR3u3G", "reviewer": "Reviewer_sX7G", "summary": "This paper proposes a new explanation method, GLIME, which provides more general, stable and local LIME explanations over ML models. Specifically, the authors demonstrate that small sample weights cause the instability of LIME, which results in dominance of regularization slow convergence and worse local fidelity.To address those issues, the authors proposed GLIME framework, which takes a slightly different form of LIME. Through rigorous theoretical analysis and some experiments, the authors can demonstrate that GLIME addressed the above issues and outperformed LIME.", "questions": "+ How about the experimental results of using GLIME on other datasets?\n+ How about the empirical comparison between GLIME and LIME in terms of the convergence speed?", "limitations": "Not applicable.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "+ The authors addressed a very important problem, i.e., the well-known instability issue of LIME, and proposed an effective solution to address it.\n+ The authors conducted a rigorous theoretical analysis to support their claims, which is very convincing.\n+ The overall presentation is very clear and easy to follow.", "weaknesses": "+ The experiments are only conducted on one dataset, i.e., ImageNet dataset. It would be better if the authors could show more results on more benchmark datasets\n+ Some properties that are studied in theory for GLIME are not empirically verified. For example, in Section 4.1, the authors showed that their method can converge faster than LIME. Although they provide clear proof for it, the authors did not demonstrate it in experiments. So it would be better if some empirical experiments can cover this.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer sX7G for reviewing our paper and for the insightful comments. We hope our answers will address your concerns.\n\n> **Q1:** The experiments are only conducted on one dataset, i.e., ImageNet dataset. It would be better if the authors could show more results on more benchmark datasets. \n\n**A1:** We have also conducted experiments on text data, utilizing the DistilBERT model. We select 100 data points from the IMDb dataset as inputs for explanation. In our experiments, we compare the performance of GLIME-Binomial and LIME, and the Jaccard Index results are presented in the table below. Our findings indicate that GLIME-Binomial exhibits significantly higher stability than LIME across various values of $\\sigma$ and sample sizes. Particularly, when $\\sigma$ is small, GLIME-Binomial demonstrates a substantial improvement in stability compared to LIME. \n\nGLIME-B: GLIME-Binomial; \n\n| # samples | $\\sigma=0.25$ |       | $\\sigma=0.5$ |       | $\\sigma=1$ |       | $\\sigma=5$ |       |\n| --------- | ------------- | ----- | ------------ | ----- | ---------- | ----- | ---------- | ----- |\n|           | GLIME-B       | LIME  | GLIME-B      | LIME  | GLIME-B    | LIME  | GLIME-B    | LIME  |\n| 258       | 0.533         | 0.406 | 0.533        | 0.407 | 0.533      | 0.421 | 0.533      | 0.509 |\n| 512       | 0.607         | 0.414 | 0.607        | 0.417 | 0.607      | 0.441 | 0.607      | 0.567 |\n| 1024      | 0.679         | 0.431 | 0.683        | 0.437 | 0.683      | 0.480 | 0.683      | 0.637 |\n\nThe $R^2$ values for GLIME and LIME are presented in the table below. The sample size is fixed at 1024. The results indicate that GLIME exhibits superior local fidelity compared to LIME, particularly when $\\sigma$ is small.\n\nGLIME-B: GLIME-Binomial; \n\n|       | $\\sigma=0.25$ |       | $\\sigma=0.5$ |       | $\\sigma=1$ |       | $\\sigma=5$ |       |\n| ----- | ------------- | ----- | ------------ | ----- | ---------- | ----- | ---------- | ----- |\n|       | GLIME-B       | LIME  | GLIME-B      | LIME  | GLIME-B    | LIME  | GLIME-B    | LIME  |\n| $R^2$ | 0.688         | 0.001 | 0.691        | 0.160 | 0.693      | 0.579 | 0.693      | 0.682 |\n\nWe will conduct more experiments in the future and incorporate the results into the final version.\n\n> **Q2:** Some properties that are studied in theory for GLIME are not empirically verified. For example, in Section 4.1, the authors showed that their method can converge faster than LIME. Although they provide clear proof for it, the authors did not demonstrate it in experiments. So it would be better if some empirical experiments can cover this.\n\n**A2:** The comparison is presented in Figure 4(a). In Figure 4(a), we present the stability of GLIME and LIME with respect to random seeds and parameters. GLIME-Binomial, which is equivalent to LIME, is much more stable than LIME, especially when $\\sigma$ is small. Under default setting where $\\sigma=0.25$, GLIME requires 256 sample to have top-20 Jaccard Index $\\approx 0.9$ while with 2048 samples, LIME only has top-20 Jaccard Index$\\approx 0.7$.  This is an empirical evidence that GLIME converges much faster than LIME. Please refer to Section 5.1 and Figure 4(a) for more details. "}, {"review_id": "IUoYYn9G1V", "reviewer": "Reviewer_jFSL", "summary": "This work is embedded in the research on model-agnostic explanations, i.e., to provide the user an understanding on the outputs of otherwise black-box prediction methods without knowing about the model's internals.  While LIME is a popular approach to solve this problem, prior work has demonstrated LIME to suffer from a strong dependence on the random seed, leading to instability and inconsistency. Since reliable, model-agnostic explanations will be a crucial tool for research and application alike to afford the use of otherwise black-box machine learning models, this paper is tackling an important issue considering LIME's popularity yet evident short-comings. GLIME is presented as a step towards more general, stable and consistent model-explanations. Due to the free choice of its sampling distribution and weights, it is shown how GLIME not only improves on LIME but generalizes over other methods.", "questions": "1. What are GLIME's short-comings and what are plans to improve on the method in the future?\n\n2. Has it been evaluated if there is a difference in how human-interpretable GLIME's and LIME's explanations are?", "limitations": "Overall I would say it's a technically sound paper, but for a model-explanation paper I am missing the human-side a bit. My understanding is that the improvements the work shows are only meaningful if the consistent/stable explanations are also still good explanations, which I think is not really discussed here.", "rating": 7, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.The method GLIME is presented very clearly. Not only are text and equations supporting the reader in understanding the method well, but its motivation as successor of LIME in terms of stability and local fidelity are easy to follow and well justified by both the presented related work as well as this paper's own evaluation.\n\n2. The unification of various model-explanation methods not only gives the reader an overview of how these methods relate to each other but shows well how GLIME is not only succeeding at stability and local fidelity but also a more general framework then LIME.", "weaknesses": "1. This work is strongly focused on comparing GLIME and its variants to LIME. While the relation of LIME and GLIME are made clear and well supported by the experiments, a more comprehensive overview on the field of explanation methods other than LIME could help the reader to better understand how GLIME fits into current research. Similarly, a discussion of GLIME's short-comings and an outline of Future Work would reinforce the contribution. Along the same line, a discussion on GLIME's quality as model-explainer and human-interpretability of the achieved results would greatly support the claims.\n\n2. While the figures present the concepts and results of this paper quite well, they could benefit from some additional attention and polishing. For example, Fig 1a misses an explanation of the employed colormap. Fig. 1b shows GLIME-Gauss as blue dot in the legend but not the graphic itself. In Fig. 4a, the legend occupies important parts of the plot such that GLIME-Binomial and GLIME-Gauss curves are hard to see.\n\n3. The use of inline-math can at times be overwhelming, e.g., in Theorem 4.2. While it is important to state all the relevant equations and relations, reverting to display- rather than inline-math for the key concepts might help the reader to better digest the underlying theory and assumptions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer jFSL for reviewing our paper and for the insightful comments. We hope our answers will address your concerns.\n\n> **Q1:** What are GLIME's short-comings and what are plans to improve on the method in the future?\n\n**A1:** From our perspective, the following may be promising for future research:\n\n* Parameter selection: It would be worthwhile to explore algorithms that can adaptively determine the optimal locality parameter $\\sigma$ for each input $\\mathbf{x}$, allowing for more flexible and context-specific explanations.\n\n* Feature correlation: Considering feature correlation in the explanation process could enhance the accuracy and interpretability of the results. Developing algorithms that incorporate feature correlation would be a promising avenue for future research.\n\n> **Q2:** Has it been evaluated if there is a difference in how human-interpretable GLIME's and LIME's explanations are? My understanding is that the improvements the work shows are only meaningful if the consistent/stable explanations are also still good explanations, which I think is not really discussed here.\n\n**A2:** We acknowledge the absence of a comparison between LIME and GLIME in terms of human interpretability in our study. We understand the significance of evaluating the interpretability of these methods and will carefully consider how to conduct such a comparison in future research. LIME, being one of the most widely utilized explanation methods, relies on linear model approximation to explain model behavior. Previous studies on LIME (e.g., ALIME [1], DLIME [2], S-LIME [3])  have primarily focused on addressing its limitations, particularly instability issues, rather than conducting human evaluation experiments. Instead, alternative evaluation metrics have been emphasized.\n\nBased on our current findings, we maintain the belief that GLIME outperforms LIME. A prominent concern with LIME is its high sensitivity to the choice of random seed. Figure 1(a) clearly demonstrates that different random seeds can yield entirely disparate explanation results. In contrast, GLIME consistently produces outputs irrespective of the seed, as indicated by the Jaccard Index. Explanations that are heavily influenced by random seeds may undermine their reliability and significance. Consequently, we argue that GLIME provides more meaningful explanations in comparison to LIME.\n\nNonetheless, we recognize the necessity of conducting experiments to compare these methods in terms of human interpretability. We have plans to conduct relevant experiments in the future to address this aspect. Due to time constraints, we are currently unable to present experimental results, but we assure you that they will be included in the final version of our study.\n\n> **Q3:** A more comprehensive overview on the field of explanation methods other than LIME could help the reader to better understand how GLIME fits into current research.\n\n**A3:** Our main focus is to explore the limitations of LIME and how our proposed method, GLIME, addresses those limitations. Therefore, in the main body of the text, we primarily discuss LIME and similar explanation methods. There are many commonly used methods (e.g., SHAP, Anchor, SmoothGrad) that differ significantly from LIME, which we did not mention. However, we will provide a more detailed introduction to some commonly used explanation methods in the appendix, so that readers can have a clearer understanding of the differences between GLIME and those methods, as well as the applicability of GLIME.\n\n> **Q4:** Fig 1a misses an explanation of the employed colormap. Fig. 1b shows GLIME-Gauss as blue dot in the legend but not the graphic itself. In Fig. 4a, the legend occupies important parts of the plot such that GLIME-Binomial and GLIME-Gauss curves are hard to see. The use of inline-math can at times be overwhelming, e.g., in Theorem 4.2. \n\n**A4:** Thank you for bringing these issues to our attention. We appreciate your feedback and will make the necessary changes to address them. We will also conduct a thorough review and revision of other sections in accordance with your suggestions.\n\n[1] Sharath et al., ALIME: Autoencoder Based Approach for Local Interpretability\n\n[2] Muhammad et al., DLIME: A Deterministic Local Interpretable Model-Agnostic Explanations Approach for Computer-Aided Diagnosis Systems\n\n[3] Zhengze et al., S-LIME: Stabilized-LIME for Model Explanation"}]

Author rebuttal:

Meta-review: All reviewers recommend acceptance and point out that the contribution is of interest to the community.
Decision: Accept (spotlight)