SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In Table 4, we provide the results for SHAP along with all methods for easy comparison. Note that SHAP does not have standard errors since it is computed only once per test point. The INFD values for SHAP are miniscule since SHAP values add up to the predictions by definition. In order to compute GI, CI, Y, CAC, we convert the SHAP values to SHAP attributions [5] first and follow the same approach used by other explanation methods.
Figure 21: Results using realistic perturbations for FMNIST dataset with mean feature importances for all classes:1-10 (T-shirt/top, Trouser, Pullover, Dress, Coat, Sandal, Shirt, Sneaker, Bag and Ankle boot). (a) Mean feature attributions of all images in the class using MeLIME. (b) Mean feature attributions of all images in the class using LINEX. (c) Mean of all images in the class. The  $r$  values show Pearson's correlation between average feature attributions and mean of the original images from the respective classes. We observe that LINEX explanations/attributions exhibit significantly higher correlation with the original images belonging to a particular class (i.e. high CAC).
I Error Analysis of LINEX
We perform error analysis for LINEX to gain better understanding about the method. We choose FMNIST dataset for doing this since, LINEX/real under performs MeLIME in terms of the INFD measure here (see Table 2) more heavily compared to other datasets and so we wanted to investigate the reasons for this. This also happens to be one of the higher dimensional datasets that is intuitive to visualize and understand.
We start by observing that even though LINEX/real underperforms in the INFD metric, the gap is not so great in the GI metric, which suggests that MeLIME may be overfitting explanations here. We also note that in terms of CI,  $\Upsilon$ , and CAC metrics, LINEX/real clearly outperforms MeLIME.
We now choose a sample of images from the dataset where LINEX/real has highest instance-level infidelity numbers and display them in Figure 23. Just looking at the explanations and the corresponding original images visually, it is evident that LINEX/real highlights the prominent features like sleeves and collar in a shirt, handles of the bags, outlines of the boots/shoes, even though the infidelity values are high. However, MeLIME misses out on some of these prominent features and
Figure 22: Results using realistic perturbations for CIFAR10 dataset. We see above images of a dog, a horse, a truck, a bird, a boat and a dog again randomly selected from CIFAR10. The original images are greyed out here so that the (normalized) attributions are clearly visible. As can be seen LINEX attributions seem to consistently focus on salient features as compared to MeLIME. For example for the first dog image we highlight the head, ears and leg, while MeLIME focuses more on the neck and some of the background. For horse too LINEX focuses on head and body, while MeLIME focuses on the legs and neck. For truck both seem to focus on important features. For bird LINEX hones in on the wings, while MeLIME although giving importance to wings also attributes some of the background. The boat image LINEX focuses on the center of the boat, while Melime on the edges and some of the water around the boat. For the dog face image LINEX focuses on the nose, eyes and ears, while Melime focuses on the ears and neck.
focuses only on optimizing the local fit. The fact that LINEX zeroes in on important features also provides additional evidence for the closeness of GI metrics between the two methods, and the better performance of LINEX/real with CI,  $\Upsilon$ , and CAC metrics.
This conclusion is also verified when we look at the performance of LINEX at a class level. In Figure 24, we see two classes one where the infidelity of LINEX is low (i.e. Trousers class) and the other where its infidelity is high (i.e Shirt class). As can be seen since the Trousers class has examples with less superfluous features (viz. varied designs) focusing on which might reduce infidelity but are not critical for determination of the class, LINEX does better in terms of infidelity on the prior. However, although infidelity is higher for the latter Shirt class it does much better on other metrics such as GI, CAC, CI and  $\Upsilon$  indicating that LINEX truly focuses on robust features.
J Ablation Analysis of Important Features for Various Explanation Methods
We wanted to analyze the most challenging case for us in the reported experiments which is on the FMNIST dataset where we are more worse than MeLIME in terms of INFD than any of the other setups. We thus assess if the features deemed important - those with the largest coefficients - by the explanation methods are indeed important for the black box model to make their predictions. To assess this, we set the we set a fraction of features (pixel values) corresponding to the top coefficients of MeLIME and LINEX/realistic to a baseline value and run the modified images again through the black box model - this is what we mean by ablation here. The baseline value here was chosen to be -1 since that is the value of the background pixels. We then used two measures to assess the quality of explanations - higher values being better for both. The first measure is mean absolute error between the predicted scores before and after ablation, corresponding to the original predicted class. The second measure is the fraction of images that changed their predicted class after ablation. We
Table 4: Comparing the different methods (including SHAP) using metrics infidelity (INFD), generalized infidelity (GI), coefficient inconsistency (CI), class attribution consistency (CAC) and unidirectionality (Y).
Dataset	Method	INFD ↓	GI ↓	CI ↓	Y ↑	CAC ↑
IRIS	LIME	0.015 ± 0.011	0.132 ± 0.042	0.319 ± 0.132	0.646 ± 0.040	0.667 ± 0.167
S-LIME	0.015 ± 0.010	0.077 ± 0.011	0.143 ± 0.045	0.704 ± 0.037	0.878 ± 0.034
LINEX/rand	0.013 ± 0.009	0.052 ± 0.008	0.044 ± 0.013	0.802 ± 0.043	0.921 ± 0.042
NB/rand	0.040 ± 0.010	0.067 ± 0.003	0.319 ± 0.132	0.646 ± 0.040	0.667 ± 0.167
MeLIME	0.008 ± 0.003	0.049 ± 0.018	0.219 ± 0.108	0.629 ± 0.013	0.464 ± 0.100
LINEX/real	0.009 ± 0.003	0.029 ± 0.003	0.024 ± 0.002	0.744 ± 0.044	0.942 ± 0.023
NB/real	0.058 ± 0.022	0.034 ± 0.000	0.219 ± 0.108	0.629 ± 0.013	0.464 ± 0.100
MAPLE	0.009 ± 0.001	0.038 ± 0.004	0.261 ± 0.033	0.458 ± 0.032	0.586 ± 0.035
LINEX/mpl	0.013 ± 0.000	0.020 ± 0.000	0.026 ± 0.002	0.694 ± 0.008	0.929 ± 0.004
SHAP	0.007	0.197	0.248	0.664	0.524
MEPS	LIME	0.158 ± 0.066	0.214 ± 0.041	0.005 ± 0.001	0.981 ± 0.006	NA
S-LIME	0.158 ± 0.066	0.214 ± 0.042	0.005 ± 0.001	0.974 ± 0.008
LINEX/rand	0.130 ± 0.052	0.164 ± 0.021	0.003 ± 0.001	0.979 ± 0.006
NB/rand	0.275 ± 0.062	0.311 ± 0.079	0.005 ± 0.001	0.981 ± 0.006
MAPLE	0.063 ± 0.000	0.067 ± 0.000	0.007 ± 0.000	0.957 ± 0.000	NA
LINEX/mpl	0.098 ± 0.001	0.094 ± 0.001	0.007 ± 0.000	0.950 ± 0.000
SHAP	0.000	0.091	0.009	0.940	NA
FMNIST	LIME	0.162 ± 0.003	NA	NA	NA	NA
S-LIME	0.142 ± 0.003
LINEX/rand	0.149 ± 0.002
NB/rand	0.207 ± 0.000
MeLIME	0.001 ± 0.000	0.277 ± 0.000	0.007 ± 0.000	0.769 ± 0.000	0.327 ± 0.000
LINEX/real	0.100 ± 0.002	0.304 ± 0.001	0.002 ± 0.000	0.780 ± 0.000	0.649 ± 0.001
NB/real	0.017 ± 0.000	0.446 ± 0.000	0.007 ± 0.000	0.769 ± 0.000	0.327 ± 0.000
SHAP	0.000	1.962	0.589	0.551	0.038
CIFAR10	LIME	0.191 ± 0.005	NA	NA	NA	NA
S-LIME	0.185 ± 0.002
LINEX/rand	0.186 ± 0.002
NB/rand	0.208 ± 0.001
MeLIME	0.100 ± 0.003	0.412 ± 0.007	0.014 ± 0.000	0.546 ± 0.003	NA
LINEX/real	0.090 ± 0.005	0.279 ± 0.001	0.006 ± 0.000	0.679 ± 0.004
NB/real	0.103 ± 0.002	0.398 ± 0.004	0.014 ± 0.000	0.546 ± 0.003
SHAP	0.003	1.376	0.398	0.512	NA
Rotten Tomatoes	LIME	0.079 ± 0.036	NA	NA	NA	NA
S-LIME	0.075 ± 0.035
LINEX/rand	0.069 ± 0.032
NB/rand	0.241 ± 0.007
MeLIME	0.029 ± 0.001	0.391 ± 0.000	0.000 ± 0.000	0.999 ± 0.000	0.909 ± 0.000
LINEX/real	0.053 ± 0.000	0.361 ± 0.000	0.000 ± 0.000	1.000 ± 0.000	0.953 ± 0.001
NB/real	0.035 ± 0.000	0.535 ± 0.000	0.000 ± 0.000	0.999 ± 0.000	0.909 ± 0.000
SHAP	0.000	0.384	0.008	0.999	0.015
see from Figure 25 that LINEX/realistic substantially outperforms MeLIME in both these measures, clearly demonstrating the relevance of features chosen by our method to the black box.
K Error Analysis of LINEX based on Ablation
Highlighting stable features for examples near non-linearities is a key strength of LINEX. However, in some cases for examples near class boundaries it may ignore sensitive features as we show in this demonstration.
In Figure 26, we show 6 examples that are appear to be close to class boundaries. We ablate pixels corresponding to top  $15\%$  of important features chosen by MeLIME and LINEX/realistic using the approach discussed in Section J. Ablation based on MeLIME importances meaningfully changes classes, whereas ablation by LINEX importances does not. The changes in prediction for MeLIME ablation for the six images are respectively from Dress to Trouser, Sneaker to Sandal, Pullover to Dress, Sneaker to Sandal, Bag to Pullover, and Sneaker to Sandal. The new class assignment looks
reasonable looking at the ablated images. We also see that the changes in class probabilities for the original class  $(p)$  are much higher after MeLIME ablation compared to LINEX/realistic ablation.
MeLIME ablated images for the first example has structures that look like trouser legs, for the second, fourth and sixth examples the area around the heel is more open making the original sneaker look like a sandal, for the third example, there is a hole in the hooded part of the pullover making it resemble a dress. The fifth example is classified as a pullover possibly because of the elongated structures on the sides that look like hands.
Note that such cases of LINEX under performing are rare though as is confirmed by its superior performance in Figure 25.
L Understanding Behavior of LIME and LINEX with Synthetic Data
We consider explaining the behavior of a function of two variables  $x$  and  $y$  with Class 1 sandwiched between Class 0 (see Figure 27). The third (or vertical) axis denotes the probability of being in Class 1. Clearly,  $x$  is the only important feature here that determines the class label.
From Figure 27 (left), we see that the LIME (here MeLIME) would be the same as LIME since the space is flat and all points are realistic) feature attributions at points  $a, b,$  and  $c$  will provide importance to  $x$  feature for small as well as large kernel width (1 and 2 respectively) neighborhoods. For point  $c$ , in the interior of the Class 0, the attributions are stable across kernel widths. However for points  $a$  and  $b$  close to the boundary of classes, the attributions for small kernel width and large kernel width neighborhoods differ significantly along the  $x$  direction. This shows the instability of LIME explanations near boundaries of classes for different kernel widths.
In contrast in Figure 27 (right), we see that the LINEX explanation constructed for the two kernel widths provides stable feature attributions for all points  $a, b, c$ . For  $a$  and  $b$ , LINEX will conservatively pick a smaller feature attribution along the  $x$  direction since the function changes rapidly in its neighborhood. As such though LINEX will still pick the feature in the  $x$  direction in this scenario.
M Variation of feature attributions with  $\gamma$
Based on the proof of Theorem 1, if for a feature the optimal attributions have opposite sign for each of the two environments, then  $\gamma$  can be made arbitrarily small (except 0) or large and the output of Algorithm 1 should still be the same which is 0 as the Nash Equilibrium is  $\pm \gamma$ . If the optimal attributions are the same sign then we should still get the same output from Algorithm 1 as long as  $\gamma \geq \min(|w_{1i}|, |w_{2i}|)$  since the attribution from our algorithm is the minimum of those values. When  $\gamma < \min(|w_{1i}|, |w_{2i}|)$  then the feature attributions will smoothly reduce as  $\gamma$  reduces.
We demonstrate this behavior in Figure 28 using an example from the IRIS dataset with random perturbations using the same setting as in Section 5. In the experiments in Section 5, we set  $\gamma = 0.329$  which is the maximum absolute value based on a linear fit to each environment. As  $\gamma$  increases beyond 0.329, the attributions are unchanged demonstrating robustness. Same holds true while reducing  $\gamma$  up to 0.165 beyond which we see smooth reduction in the attribution values. Qualitatively, similar behavior is seen for other examples too. Because we set  $\gamma$  pessimistically (ignoring constraints) to a high value, we can expect our reported performances in the paper to be robust across many values of  $\gamma$ .
N Convergence of LINEX procedure and comparisons
We demonstrate based on a synthetic example how Algorithm 1 and provides a unidirectional explanation. We generate synthetic data using a function in  $\mathbb{R}^2$  (Figure 29(left)). The function gently rises with increasing  $y$  values, and along  $x$  it is flat first, then rises abruptly and then falls gradually. We want to obtain robust attributions of this function at the point  $x = 1.0$ ,  $y = 0.0$ , which is close to the end of the rising edge along  $x$  direction.
As we can imagine, since the slope changes abruptly along  $x$  direction near the point, it should be ideally excluded from an explanation intended towards recourse based on a linear proxy. Otherwise, the explanation will not generalize in the neighborhood of this point. On the other hand, the  $y$  direction should be included since the function changes smoothly along  $y$  throughout.
To generate explanations We first create two environments centered at the example to explain with variances 0.5 and 2.0. Now independently fitting to these environments leads to feature attributions that are  $\{-0.033, 0.098\}$  and  $\{0.084, 0.102\}$ . Appending the two environments the attributions are  $\{0.029, 0.095\}$ , whereas with LINEX, the attributions would be  $\{0.0, 0.093\}$ . Thus, LINEX effectively eliminates the feature with high variability or abrupt changes. The behavior of the coefficients for each environment as LINEX converges is shown in Figure 29(right). As such, one can also see the convergence is fast.
O Limitations
Like any other posthoc explainable AI method there is no way to surely say that LINEX exactly reflects the true reasoning behind a black box classifier in arbitrary applications. It also is somewhat slower than LIME as shown in section A given the game theoretic nature of the algorithm, where its stability and unidirectionality hopefully offsets the additional time required. On the flip side, given its favorable properties in terms of recovering explanations it could be used to violate privacy which may be concerning from a social standpoint.
MeLime:  $r = 0.121$  INFD=0.001 LINEX/real;  $r = 0.334$  INFD=0.169
MeLime:  $r = 0.093$  INFD=0.001 LINEX/real:  $r = 0.338$  INFD=0.216
MeLime:  $r = 0.112$  INFD=0.001 LINEX/real:  $r = 0.539$  INFD=0.201
MeLime:  $r = 0.114$  INFD=0.001 LINEX/real:  $r = 0.412$  INFD=0.189
MeLime:  $r = 0.393$  INFD=0.001 LINEX/real:  $r = 0.713$  INFD=0.180
MeLime:  $r = 0.162$  INFD=0.001 LINEX/real:  $r = 0.528$  INFD=0.161
Figure 23: Error analysis for a chosen set of examples in FMNIST using MeLIME and LINEX/real methods. The three columns are the MeLIME feature attributions, LINEX/real feature attributions, and the original images. The rows correspond to different examples. We show the Pearson's correlation coefficient between feature attributions and mean of the original images from the respective classes  $(r)$  and instance-level infidelity (INFD) measures. LINEX seems to highlight important features like stripes in the t-shirt, handles of the bags, outlines of the boots/shoes more prominently, while MeLIME seems to overfit to the data while missing out on highlighting some key features prominently.
Trouser class - less infidelity (INFD)
Shirt class - more infidelity (INFD)
Figure 24: We see above that infidelity is lower for Trousers class for LINEX as compared with the Shirts class. A reason for this is that the trousers are more plain with less superfluous features such as the different designs in shirts. Since LINEX focuses on robust features focusing excessively on the designs is not critical for it to determine a shirt, albeit focusing on these designs might reduce infidelity. Advantage of it relying on robust features is however apparent when we look at other metrics such GI, CAC, CI and  $\Upsilon$  as seen in Table 2 where it is much closer to or superior to MeLIME.
Figure 25: Ablation analysis to determine if the features deemed important by the explanation methods are actually considered important for prediction by the black box model. We see that features chosen by LINEX impact the prediction of the black box model much more than those chosen by MeLIME. This is true with respect to both MAE measure (left) between the predicted probabilities before and after ablation for winning (or argmax) class, and the change in predicted classes (right) before and after ablation. Higher values here mean that the features chosen by the explanations are more relevant for the black box to make its predictions. The maximum value of both measures is 1.0.
Figure 26: Error analysis for a chosen set of examples in FMNIST using MeLIME and LINEX/realistic methods, using ablation of important features. Each row shows results for a particular image. The columns show the: (a) MeLIME coefficients, (b) LINEX/realistic coefficients, (c) the original image along with its predicted class (cls.) and predicted probability for that class  $(p)$ , (d) the image after MeLIME ablation along with the predicted probability for the original class  $(p)$  and the new class prediction (cls.), and (e) the image after LINEX/realistic ablation along with the predicted probability for the original class  $(p)$  and the new class prediction (cls.). The changes in prediction for MeLIME ablation for the six images are respectively from Dress to Trouser, Sneaker to Sandal, Pullover to Dress, Sneaker to Sandal, Bag to Pullover, and Sneaker to Sandal. No changes in classes are seen for LINEX ablation.
Figure 27: LIME (left) and LINEX (right) feature attributions for three points  $(a, b, c)$  for a synthetic data where we have Class 1 sandwiched between Class 0. For LIME, the different colors pink and blue correspond to feature attributions obtained with the small and large kernel width neighborhoods. Note how explanations for LIME change significantly (in magnitude) by kernel widths near the class boundaries, whereas the LINEX explanation remains stable, where it still picks up the important feature.
Figure 28: Feature attributions for the four features for an example in the IRIS dataset are shown above when varying  $\gamma$ . We used the same setting as in Section 5 for this experiment. The attributions increase smoothly as  $\gamma$  increases and stay constant after  $\gamma \geq \min(|w_{1i}|, |w_{2i}|) \forall i$ .
Figure 29: Left side: Explaining a scalar function in  $\mathbb{R}^2$  at the point indicated by the triangle. The point is centered at  $x = 1.0$ ,  $y = 0.0$ . The two environments are created by sampling multivariate normals with variances 0.5 and 2.0 respectively (samples not shown) centered at this point. Right side: Convergence of individual environment attributions. The attributions for first feature  $(x)$ ,  $w_{1,0}$  and  $w_{2,0}$ , converge to  $\gamma$  and  $-\gamma$  leading to the optimal attribution of 0. For the second feature  $(y)$  the optimal attribution  $(w_{1,1} + w_{2,1})$  converges to a positive value.
(a) INFD - S-LIME - mean smoothing
(c) INFD - S-LIME - MoM smoothing
(b) INFD - S-LIME - median smoothing
(d) GI - S-LIME - mean smoothing
(e) GI - S-LIME - median smoothing
(f) GI - S-LIME - MoM smoothing
(g) CI - S-LIME - mean smoothing
(h) CI - S-LIME - median smoothing
(i) CI - S-LIME - MoM smoothing
(j)  $\Upsilon$  -S-LIME - mean smoothing
(m) CAC - S-LIME - mean smoothing
Figure 30: Effect of various smoothing schemes on S-LIME's performance based on 5 environments (since median-of-means (MoM) is just median for 2) with the same setup described in Suppl. D. for the IRIS dataset. As can be seen median and MoM perform worse than the mean on INFD and similar to it on other metrics. Thus, this does not change the takeaways from the main paper. See Figure 4 in the supplement for legend. 36
(k)  $\Upsilon$  - S-LIME - median smoothing
(n) CAC - S-LIME - median smoothing
(1)  $\Upsilon$ -S-LIME - MoM smoothing
(o) CAC - S-LIME - MoM smoothing