Selective Explanations
Authors: Lucas Monteiro Paes
Dennis Wei
Flavio Calmon
Venue: NeurIPS 2024 poster (2024)
Keywords: explainability
selective prediction
amortization
interpretability
shapley
Primary area: interpretability_and_explainability
TL;DR: We propose selective explanations to detect when amortized explainers produce low-quality explanations and introduce an optimized method to improve their quality.

Abstract: Feature attribution methods explain black-box machine learning (ML) models by assigning importance scores to input features. 
These methods can be computationally expensive for large ML models. To address this challenge, there have been increasing efforts to develop amortized explainers, where a ML model is trained to efficiently approximate computationally expensive feature attribution scores. Despite their efficiency, amortized explainers can produce misleading explanations. In this paper, we propose selective explanations to (i) detect when amortized explainers generate inaccurate explanations and (ii) improve the approximation of the explanation using a technique we call explanations with initial guess. Selective explanations allow practitioners to specify the fraction of samples that receive explanations with initial guess, offering a principled way to bridge the gap between amortized explainers (one inference) and more computationally costly approximations (multiple inferences). Our experiments on various models and datasets demonstrate that feature attributions via selective explanations strike a favorable balance between explanation quality and computational efficiency.

Selective Explanations
Lucas Monteiro Paes Harvard University lucaspaes0g harvardd. edu
Dennis Wei IBM Research dwei @us i.mm com
Flavio P. Calmon
Harvard University
f flaviodseas hharvard..... edu
Abstract
Feature attribution methods explain black-box machine learning (ML) models by assigning importance scores input features. These methods can be computa- tionally expensive for large ML models. To address this challenge, there have been increasing efforts develop amortized explainers, where ML model is trained to efficiently approximate computationally expensive feature attribution scores. Despite their efficiency, amortized explainers can produce misleading explanations. In this paper, we propose selective explanations to (i) detect when amortized explainers generate inaccurate explanations and (ii) improve the approxi- mation of the explanation using technique we call explanations with initial guess. Selective explanations allow practitioners to specify the fraction of samples that receive explanations with initial guess, offering a principled way to bridge the gap between amortized explainers (one inference) and more computationally costly approximations (multiple inferences). Our experiments datasets demonstrate that feature attributions via selective explanations strike a favorable balance between explanation quality and computational efficiency.
various models and
Introduction
Large black-box models are increasingly used to support decisions in applications ranging from online content moderation [26]. hiring [12], and medical diagnostics [35]. In such high-stakes settings, the need explain "why" model produces given output has led to growing number of perturbation-basee feature attribution methods [22, 29, 27, 23, 2, 40). These methods use input perturbations to assign numerical values each input feature (e.g.. words in text) model uses, indicating their influence on the model prediction. They are widely adopted in part because they work in the black-box setting with access only model output (i.e., without gradients). However, existing feature attribution methods can be prohibitively expensive for the large models used in the current machine learning landscape (e.g., language models with billions of parameters) since they require significant number of inferences for each individual explanation.
Recent literature has introduced two main approximation strategies speed up existing feature attribution methods for large models: (i) employing Monte Carlo methods to approximate explanations with fewer computations [22, 29, 5, 24], and (ii) adopting amortized approach, training separate model to "mimic" " the outputs of reference explanation method [16, 6, 38, 32, 3, 33J. Monte Carlo approximations can yield accurate approximations for attributions but may converge slowly, limiting their practicality for and online applications. In contrast, amortized explainers require only one inference per explanation, making them efficient for large black-box models and online explanations. However, as shown in Figure 1, amortized explainers can produce highly diverging explanations from their reference due lack of precision in approximations. Aiming to benefit from Monte Carlo and amortized explainers, we propose selective explanations to answer the questions:
(Q1) When amortized explanations inaccurate
(Q2) How can we improve inaccurate amortized explanations using additional computations?
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

[CLS] Benor than most chain pozza ok. Innii got a crust, Whach was nice and
[CLS) Bemal Ban most chain pizza, BS
[CLE] Bentn most pizza IIS BNe.
immiii got thin crer which waso rice and
irnnti get a thin crust, which vas and
crispy, only Eril greasy di liing acdeents not
crispy only
gressy ok Bng ediarts, not
crisyy only littl greasy. ok ingreed entss not
amazing on the cheese, dind had kind of blund
amazing on the choese, and had kind of blund
amazing on the cheese, and had kind of brand
crust small quess that Boesntt sound too good.
crust immi guess it at docsntt sound too good.
crust unni guoss thit docon't sound too good. but roally promite ItS better than any national
but raally promise Ehs better than any national
bul roally promise itas better than any national
chain pizza you rind in town, B also has mayy
chain pizza you'll rind tosin it also has really
chain pazza you'll find town, also has really
friendly ladd back almoephtare.. [SEP]
trienity las boakk atmosspiore [SEP)
frendly laks black atmosphere. [BEP]
(a) Amortized (MSE 0.31)
(b) Target Explanation
(c) Selective (MSE 0.07)
Fig. I: Amortized explainer (a) compared with a target explainer (SHAP [22]) (b) and our selective explanation method (c). All methods flag imput parts that contribute to the YelpLLM predicting the given example is Negative Review. We observe that both target and selective explanations attribute "not amazing" for the negative review (blue), while the amortized explainer misses this term.
To answer (Q1) and (Q2), we propose selective explanations, a method that bridges Monte Carlo and amortized explanations. The selective explainer first trains model that "learns to select" which data points will receive inaccurate amortized explanations, and then performs additional computations to further approximate target explanations. The key idea behind the selective explanation method is to use Monte Carlo explanations only for points that would receive inaccurate amortized explanations; see Figure 2 for the workflow of selective explanations. The code for generating selective explanations can be found at https: / //gittb..tttt//////// com/ www.ssottteirooaaee........aaa http://tttteiooaeeeeeeeeeeiiiiie http:/eeeeeeeeeeeeeeeeeeeeeeeeee anations inss
The ideas of predicting selectively and providing recourse with a more accurate but expensive method (usually human feedback) have been explored in classification and regression [28, 10, 7, 9, 11]. To our knowledge, however, these ideas have not been applied to feature attribution methods. We make two contributions in this regard that are relevant for selective prediction more generally. (1) Selective prediction uses quality metrics to identify input points for which the predictor (the amortized explainer in our case) would produce inaccurate outputs and recourse is needed. The nigh-ddmensiooal nature of explanations requires to develop new quality metrics (Section 3) suitable for this setting. (2) Instead of providing recourse with a Monte Carlo explanation alone, we use an optimized method called explanations with initial guess (Section 4) that combines amortized and Monte Carlo explanations in a optimized manner, improving the approximation to the target explanation beyond that of either method individually.
Our overall contribution (3) is to combine (1) and (2) in the form of selective explanations, providing explanations with initial guess to improve inaccurate amortized explanations. We validate our selective explanations approach on two language models as well as tabular datasets demonstrating its ability to (i) detect inaccurate explanations from the amortized explainer, (ii) enhancing amortized explanations even when Monte Carlo explanations are inaccurate, and (iii) improving the worst explanations from the amortized model.
Ingurs 1o Explained
Poss 20 Brough aroriizdd DxgASTWeY 37d Selact Purstiion
Compuro Manto Caro explanddons whon anorfiadd acplination are Lize-Gullty
Use Expltritiino w Iniiil Eucol irprivv austesetion quality
A0ta))
Arail
Ligh Quily Dppimattee
Experaloon
Use Eqular improve
The contonsatan funstion
MSF
Deleeen salaska and LM220t explanatons
Useligg en-ou
Taol
Qight
Epplcaaioon
the
Fig. 2: Workflow of selective explanations.

---

2 Problem Setup & Background
We aim explain the predictions of fixed probabilistic black-box model h that predicts h(x)) (hn (ny) (z:) and outputs argmax jey ht (z: E y using a vector of features a: (x1 Tdl E R4. The user specifies an output of interest y E y (usually y argmaxJey (z)) and our goal is to efficiently explain Why would h output y for given x? We consider dataset D { y: ) |==1
comprised of N > 0 samples divided into three parts: Derain for training / and the explainers, Dea for calibration and validation, and Dreat for testing. Thus, D Derain U caal U Ureast. Moreover,
for subset 5 {+1 ... ( [d] we write æ A (Kir.-.,, Vit s))))
Feature Attribution Methods, also called explainers, are functions Id y + Rd that assess the importance of each feature for the model's (h) prediction to be for a given input vector æ. We consider three types of explainers:
(i) Target explainers that use a large number of computations to provide explanations (e.g.. SHAP with 2d inferences from model h) [22, 29], denoted by Target(æ:, y);
(ii) Monte Carlo explainers that approximate fixed target explainers using n inferences from model h per explanation [22, 24], denoted by MC" (æz, y):
(iii) Amortized explainers are trained to approximate the target explanations using only one inference [6, 38], denoted by Amor(æ. y).
Remark 1. Monte Carlo and amortized explainers aim to approximate the target explanation and are benchmarked on this approximation. We evaluate the performance of Monte Carlo and amortized explainers by computing their distance and correlation to Target(æ, y). The usefulness of target explanations (e.g.: SHAP and Lime) has been validated by user studies and automated metries in [22, 29, 13, 37, 30, 31]. Therefore, we call higher-quality the explanations that closely approximate the computationally expensive target and lower-quality the one that diverge from the target.
In practice, we measure the quality of a given explanation that aims to approximate the target explanation using loss (or distortion) function & Rd x Rd + R, e.g., mean square error (MSE) and Spearman's s correlation. The goal of selective and Monte Carlo explanations to approximate the target explanations while decreasing the number of computations, i.e., to minimize ((SE(z, y) Target( x, y)) with few model inferences.
We define selective expluiners to provide better approximations to target explanations bridging the gap between Monte Carlo and amortized explainers.
Definition 1 (Selective Explainer). For a given model h, an amortized explainer Amor, a Monte Carlo explainer MC". a combination function d IRd - R. and selection function Ta Rd {0, 1} (parametrized by a), we define the selective explainer SE(z, as
/Amor(æ, y)
if (1)) 1, (a())) = 0.
SE(ær y)
(1)
(2))r)mmr( y) + (1 An (z.,)
When Ta O, selective explanations output explanations with initial guess (Definition 2). Expla- nations with initial guess linearly combine amortized and Monte Carlo explanations to leverage information from both and provide higher-quality explanations than either explainer alone. Selective explanations heavily depend on three objects that we define in this work and that are covered in the rest of the paper: (i) an uncertainty metric (Section 3), (ii) selection function (Section 3), and (iii) combination function (Section 4).
Uncertainty metrics (5i. output the likelihood of the amortized explainer producing low-quality explanation for an input. Lower (xx) indicates a higher-quality explanation for æ. We propose two uncertainty metrics: Deep and Learned Uncertainty (Section 3).
Selection function (ra) is a binary rule that outputs for higher-quality amortized explana- tions and for lower-quality ones based on the uncertainty metric. We define Ta to ensure a fraction a of inputs receive amortized explanations. Smaller a implies higher-quality selective explanations but also more computations (Section 3).
Combination function (Ax) optimally linearly combines amortized and Monte Carlo expla- nations to minimize MSE from target explanations (Theorem 1). We propose explanations with initial guess and fit A to optimize explanation quality (Section 4).
3

---

Algorithm Building a Selective Explainer
Require: Datasets: Dtrain. Deas Explainers: Amor, MC", MC" Coverage:
Ensure: Selection function: Ta- Combination function: An-
1: Fit the uncertainty metric 8n using Derasn. Amor, and MC" (using (4) or (5))
2: Compute la using Dea (7)
3: Define the selection function Ta using 8/2 and ta (6)
4: Define bins Qi [ ta,+1) for partition a; = - &
for i E [k + 1] (9)
5: For E [A: + 1] Compute X; as in (12) using Dent, Amor, MC", and MC"
6: Define (x) = =>an (x)))((x)) E Qi] as in (9)
7: return Tax (x))
Algorithm I describes the procedure to compute the uncertainty metric, selection function, and combination function using the results we describe in Section and 4. Although selective explanations can be applied to any feature attribution method, we focus on Shapley values since they are widely used and most amortized explainers are tailored for them [16, 38, 6]. We discuss how selective explanations can be applied to LIME and provide more details feature attribution methods in Appendix B. Next, we describe specific feature attribution methods that we use as building blocks for selective explainers of the form (1).
Shapley Values (SHAP) [22] target explainer that attributes value de for each feature æ; in = (Z1, Id) which is the marginal contribution of feature z; if the model was to predict
1 Σ
d (d=1) |S|
(.
(Mg()5)(())) hy(ars))
SCI//(3
SHAP has several desirable properties and is widely used. However, as (2) indicates, computing Shapley values and the attribution vector Target(æ: y) (01 y). "a., y)) requires 2 in- ferences from h, making SHAP impractical for large models where inference is costly. This has motivated several approximation methods for SHAP, discussed next.
Shapley Value Sampling (SVS) [24] is Monte Carlo explainer that approximates SHAP by restricting the sum in (2) to uniformly sampled permutations of features performing md + inferences. We denote SVS that samples m feature permutations as SVS-m.
Kernel Shap (KS) [22] is a Monte Carlo explainer that approximates Shapley values using the fact that SHAP can be computed by solving a weighted linear regression problem using input perturbations resulting in inferences. We refer to Kernel Shap using n inferences as KS-n.
Stochastic Amortization [6] is an amortized explainer that uses noisy Monte Carlo explanations to learn target explanations. Covert et al. [6] trained an amortized explainer a model class F (multilayer perceptrons) Amor I to take (x, y) and predicts an explanation Amor(æ, y) Target(æ; a y) by minimizing the L2 norm from Monte Carlo explanations MC" (xx y). Specifically, the amortized explainer is given by
Amor E argminn feg
Σ (a.g)) Duis
[f y) - MC" (a, y)
Amortized Shap for LLMs [38] is amortized explainer similar to stochastic amortization but tai- lored for LLMs. Yang et al. [38] train linear regression the LLM embeddings [er (e.),..... G|as (a:) to minimize the L2 norm from Monte Carlo explanations MC" (x, y) and define the amortized ex- plainer as Amor(z, y) = (W,e (x) + by..... W [ + by). where W,, is a matrix and by E IR.
We use stochastic amortization to produce amortized explainers for tabular datasets and amortized Shap for L.LMs to produce explainers for I.IM predictions. Both explainers are trained using SVS-12 MC". High-quality and Monte Carlo explanations computed using the Captum library [18].

---

3 Selecting Explanations
This section defines key concepts for selective explainers: (i) uncertainty metries 8y for amor- tized explanations and (ii) selection functions (Ta) to predict when amortized explanations closely approximate target explanations based on the uncertainty metrics.
Uncertainty Metries for High- Tigh-Dimensional Regression: An uncertainty metric is function tailored for the model that takes x and outputs a real number 8p (b(2) that encodes information about the uncertainty of the model h in the prediction for æ. Generally, if 8n (x) < (xx")) then the model is more confident about the prediction h(z) than h(x') [10, 28]. Existing uncertainty metrics cater to (i) classification [28, 10. 7. 11] and (ii) one-dimensional regression [39, 34, 11, 17]. but none specifically address high-dimensional regression - which is our case of interest (d-dimensional explanations). Next, we propose two uncertainty metrics tailored to high-dimensional outputs: (i) Deep uncertainty and (ii) Learned uncertainty.
Deep Uncertainty is inspired by deep ensembles [19], a method that uses an ensemble of models to provide confidence intervals for the predictions of one model. We run the training pipeline for the amortized explainer described in (3) k times, each with a different random seed, resulting in k different amortized explainers Amor ..... Am.r". We define the deep uncertainty
Deep h
(x)
A
Var (Amor' (æ); (amor'(x)),,. (a);,..,AAor (m):
(4)
dk.
Here, Var (a1,..., .x) is the variance of the sample (q......a)] and Amor (æ); indicates the i-th entry of the feature attribution vector Am)) Hence, deep uncertainty is the average (across entries) of the variance (across all trained amortized explainers) for the predicted attributions.
If the deep uncertainty for point zero, then the amortized explainers produce the same feature attribution. On the other hand, if the deep uncertainty is high, then the feature attributions vary widely across the amortized explainers. Intuitively, the points with higher deep uncertainty are more affected by random seed change, implying more uncertainty in the explanation.
While the Deep Uncertainty approach offers principled method for estimating the uncertainty of the amortized explainer by leveraging an ensemble models, is computationally expensive due to the need for training, serving, and running multiple models. This overhead be prohibitive in practice, especially for large-scale applications. To mitigate this issue, we propose Learned Uncertainty, which, although less grounded, requires training and serving only single model.
Learned Uncertainty uses data to predict the amortized explainer uncertainty at an input point æ. We choose (the loss function) between two explanations to be MSE. The learned uncertainty metric is a function in the class I (multilayer perceptron in our experiments) such that
Learn %
E argmii #EF
|s(x) - (Amor(æ; y), y), MC" (z) ?
(5)
Ideally, instead of using the Monte Carlo explanation MC" as the reference in (5), we would like to use target explanations, i.e., L (Amor(æ; y), Target(æ;; y)). However, these computationally expensive explanations are usually not available. Thus, we resort to using Monte Carlo explanations.
For large language models, the textual input is encoded in a sequence of token embedding [ez (z)..., f|zi (æ) such that (xx) E IRd for i E [xx]] In this case, we use the mean (i.e., "mean- pooling") of the token embeddings to train the learned uncertainty metric instead of æ.
We analyze the performance of the proposed uncertainty metrics in Section 5.1, showing that it can be used to detect inaccurate explanations from the amortized explainer. Our results indicate that the proposed uncertainty metrics are (i) strongly correlated with how accurate amortized explanations are and (ii) closely approximate the best possible uncertainty measure the Oracle with knowledge of the approximation quality (Figure 3). Next, we define the selection function that allows practitioners to set coverage (percentage of points) that will receive amortized explanations.
5

---

Selection functions: a selection function is the binary qualifier (Ta) that thresholds the uncertainty metric by to given by
{
if (x) & to Quigh-quality approximations) if s > ta (low-quality approximations)
Ta (x) -
(6)
Intuitively, ta is the maximum uncertainty level tolerated by the user. In practice, if the output of the selection function is (high-quality approximations), we use the explanations from the amortized model because it is probably close to the target; if the output of the selection function is (low-que low-quality approximations). we use explanations with initial guess (see Definition bellow) to improve the explanation provided to the user. The threshold la is chosen to be the a-quantile of the uncertainty metric to ensure that at least fraction of points receive a computationally cheap explanation the coverage. Specifically, given a, we calibrate la in the calibration dataset D Deai and compute it as
ta - min i, such that Pr(s)(2) S <t] 2 at 1ER
cal
where Prest is the empirical distribution of the calibration dataset. For discussions on selecting coverage with guarantees the number of inferences for selective explanations, see Appendix C. Remark 2. A property of selective predictions [10], which transferred to selective explanations, is that it is possible to control the explainer's performance via the threshold la with guaranteed performance without providing predictions for all points. This result is displayed in Figure 3.
Explanations with Initial Guess
We have introduced methods to detect points likely to receive amortized explanations that poorly approximate the target. This raises the question: How can we improve the explanations for these points? One approach is to simply use Monte Carlo (MC) explanations instead of amortized ones. However, this ignores potentially valuable information already computed by the amortized explainer. In this section, we propose a more effective solution called explanations with initial guess, which combines amortized and Monte Carlo explanations to improve quality.
Explanation with Initial Guess uses an optimized linear combination of the amortized explanation with more computationally expensive method the Monte Carlo explainer to improve the quality of the explanation. We formally define explanations with initial guess next.
Definition 2 (Explanation with Initial Guess). Given Monte Carlo explainer MC" (x: y), and combination function Å Rd + IK that reflects the quality of the amortized explanation Amor, we define the explanation with initial guess as
(G y) A (m(r))(o((( ( y) + (1 - Au (zx) MC (æ; y)-
(8)
0, selective explanations use the explanation with initial guess (1) to improve low-quality amortized explanations, i.e., SE(x.1 y) = (G y).
Recall that when To
(a) (x)
Defining explanations with initial guess as the linear combination between the amortized and the Monte Carlo explanations is inspired by the literature on shrinkage estimators [21, 20J that use an initial guess (Amor(æ, 3) in our case) to improve the estimation MSE in comparison to only using the empirical average (a role played by MC (x. y) in our case). Next, we tune A to minimize the MSE from target explanations.
Optimizing the Explanation Quality: Our goal is for explanations with initial guess to approximate the target explanations, i.e., [rr y) -Target(r,, y)ll. To achieve this goal, we optimize the function A as follows.
First, since Target is unavailable, we use another Monte Carlo explanation MC" to approximate Target. MC" is different from MC" and potentially more computationally expensive but not necessarily. Importantly, MC" only needed beforehand when computing \, not at prediction time. In our experiments, we use SVS-12 for MC"".
Second, quantize the range of the uncertainty metric sn into bins aggregate points with similar uncertainty and define the bins Q₁ by a partition ( = On < 02 <.. < Om = 1 of [0,1]
Q2 e [ta; tass)) ) Vi [m - 1]
(9)

---

where ta, is defined as in (7). We then define the combination function to be
X (x: = X; if sx()) E Qi
(10)
A is chosen to optimize the asplanation-uualit for points with similar uncertainty, A, is given by:
X
- argmin AER
Σ (2. (k())
IG y) MC (ar, v)
(11)
The constant is only computed once per bin and stored. At explanation time, when we provide explanations with initial guess (i.e., when Ta (x) = 0) (8), we lookup the bin for the point being explained and use the associated A;-
Theorem provides a closed-form solution for X4-
Theorem 1 (Optimal An). Let 01 02 < ... < Om and define Qi as in (9). Then the
solution to the optimization problem in (11) is given by
E(e y) (MC" (æ, y) MC" (ær, v), MC (xx, y) - Amor(x;y) (a)e(;
A
(12)
Σ (a. (a.g))eD... Amor( y) MC (x, y)
#n (x)):
The range of uncertainty functions is quantized for main reasons. First, the uncertainty metric 8y encodes the amortized explainer's s uncertainty for each point æ. This uncertainty quantification should be reflected in the choice of Aς. Quantizing the range of sΙ, allows to group points with similar uncertainty levels and optimize Å for each group separately. Second, quantizing the range of 8h enables to have multiple point per bin Qi allowing us compute A minimize the MSE in each bin.
We use the Monte Carlo explainer MC" because: (i) mentioned above, assume don't 't have access to target explanations due to computational cost and (ii) even when using this Monte Carlo explainer, show that in all bins, A approximates well the optimal combination function computed assuming access to target explanations from Target defined as
1
aargmin
Σ (x. XE[O,1]
|IG((, y) Target(æ; 3)
(2)
Specifically, Theorem 2 shows that X; Re A with high probability. Appendix E shows the formal version of the Theorem along with the proof's for results this section.
Theorem (Informal A, X,Pr). (i) MC is sufficiently different from the amortized explainer Amor and (ii MC¹ approximates the target explanations Target then X and X.Pt are close with high-probabiility for all bins Qi, i.e.,
}>: 14= with probability at least 1 - e
for C > 0 and Qil the number of points in the validation datasel DD eat that are in bin Qi.
5 Experimental Results
This section analyzes the performance of selective explanations and its different components (i) uncertainty measures and (úi) explanations with initial guess. All results are showed in terms of MSE from target explanations, check Appendix D for the same results using Spearman's 's Rank Correlation.
Experimental Setup:
We generate selective explanations and evaluate their MSE and Spearman's correlation compared to the target explanation computed using large number of inferences Al- though our results hold for any feature and data attribution method, in this section, we focus Shapley values due to its frequent use and prevalence the literature on amortized explainers [16, 6, 38]. Seaborn [36] is used to compute 95% confidence intervals using the bootstrap method.
We provide details how target explanations computed in Appendix D.I.
7

---

Table Pearson's and Spearman's 's correlation between the proposed uncertainty measures and the MSE of amortized explanations from target SHAP explanations. Standard deviation in parenthesis.
Datasets
Correlation
Uncertainty Metric UCI-Adult
UCI-News
Toxigen
Yelp
Deep
0.37 (0.03)
0.40 (0.03)
0.54 (0.04)
0.52 (0.04)
Pearson's
Learned
0.36 (0.03)
0.18 (0.03)
0.89 (0.02)
0.72 (0.03)
Deep
0.50 (0.03)
0.43 (0.03)
0.69 (0.03)
0.55 (0.04)
Spearman's
Learned
0.69 (0.02)
0.23 (0.03)
0.93 (0.02)
0.77 (0.03)
Datasets & Tasks: We use four datasets: two tabular datasets UCI-Adult [1] and UCI-News [8], and two text classification datasets Yelp Review [41] and Toxigen [14]. We use 4000 samples from each dataset due the cost of computing target explanations for evaluation. Models: For the tabular datasets, we train a multilayer perceptron [15] to learn the desired task. We use the HuggingFace Bert-based model textattack/ wwwerrrbaeeeucsseeeeeeeeeeeeeeee weweeedd-yll---plriiiee and the Roberta-ban based model tomh/ fomm/toxigee toxiren_robbrta roberta [14] for the Toxigen. Uncertainty metries:
[25] for the Yelp dataset
we train 20 amortized explainers per task compute the deep uncertainty.
5.1 Uncertainty Measures & Explanations with Initial Guess
Correlation Between MSE and Uncertainty Metries: Table presents Pearson's and Spearman's correlation between the uncertainty metrics (Deep and Learned Uncertainty) and the MSE from amortized and target explanations. The table shows that the proposed uncertainty metrics positively correlate with the MSE, i.e., low uncertainty implies low MSE. Additionally, Spearman's 's correlation specially high in the language models used in the Toxigen and Yelp datasets, our main object of interest. This finding indicates that the uncertainty metrics might perform especially well when detecting inaccurate amortized explanations attributed to the predictions language models.
Detecting High-MSE Explanations using Uncertainty: Figure 3 presents the MSE of amortized explanations (y-axis) which are predicted to be higher-quality at a coverage level (x-axis), i.c., MSE of points such that Ta (x) 1. The Oracle* is computed by sorting examples from smallest highest MSE the optimal selection. The random selector chooses covered points uniformly at random. Figure shows that both deep and learned uncertainty metrics successfully identify examples that receive lower and higher-accuracy amortized explanations, also suggested by Table 1. Surprisingly, learned uncertainty can identify points that will receive low-accuracy amortized explanations almost as accurately as the optimal Oracle for the language models ((c) and (d)).
For more details on implementation, please see Appendix D.1.
'The oracle computationally expensive hecause requires target explanations.
0.12
MSE in Covered
MSE Covered
0.030
MSE in Corered
0.10
0.03
0.025
0.030
0.08
0.020
MSE in
0.015
0.06
0.005
0.005
25
100
0.02
75 Coverage (%)
100
Coverage (%)
Coverage (1)
(a) UCI-Adult
(b) UCI-News
(c) Yclp Review
(d) Toxigen
Selectors
Random
Leam
Oracle
Deep
Fig. 3: Coverage (a) vs. MSE of covered points. A point is covered when its amortized explanation predicted to higher-quality, i.e., Ta (x: 1. When 100% then all points are covered and the MSE of covered points is the average MSE for the amortized explainer.

---

0:12
0.015
0.10
M8E
0.020
0.035
0.08
0.05
0.005
0.090
0.06
0.025
0.000
0
20
40
60
80
100
20
40
60
100
0
20
40
60
80
300
20
40
100
Fractio
Computations
Computations
pututaaioon
Fraction Extra Computations
(a) UCI-Adult
(b) UCI-News
(c) Yelp Review
(d) Toxigen
Selector:
Explanations Using:
Improve Lower Quality Using:
Leam
Amertized
Monte Carlo
Explanation w/ Initial Guess
Monte Carlo Only
Fig. Fraction (1 -1-a0 a) of points which explanations receive additional computations (x-axis) MSE of selective explanations w.r.t target explanations (y-axis) with coverage a. Naive uses A = while Initial guess Å in (12). MSE is computed across all points in the test dataset. Yelp Review and Toxigen use SVS (12) as Monte Carlo explanations while UCI-Adult and UCI-News use KS (32).
Explanations with Initial Guess vs. Monte Carlo Figure we compare selective explanations improving quality of non-covered points using (i) explanations with initial guess and (ii) Monte Carlo explanations, when amortized explanations are inaccurate (Aa = 0). Case I: When the MSE from the Monte Carlo is smaller than from the amortized explainer ((a) and (c)), explanations with initial guess results in a smaller MSE compared to only using Monte Carlo. Case 2: When the MSE in Monte Carlo is larger than the amortized explanation MSE ((b) and (d)), only using Monte Carlo increase the MSE while explanations with initial guess reduces the MSE. Together, Cases and suggest that even when lower quality, explanations contain valuable information that be leveraged by explanations with initial guess to improve explanation quality.
5.2 Efficacy of Selective Explanations
Worst Case Performance Improvement: Figure 5 shows the MSE of selective explanations for the points receiving the highest MSEs. The figure suggests that selective explanations significantly decrease the worst-case MSE of amortized explanations. With just 20% coverage the MSE decreases consistently across datasets. Remarkably, when providing explanations with initial guess for 20% of the samples in the Yelp dataset (Figure 5 (c)), selective explanations result in MSE for the worst 5% of points that is about 30% smaller than the original amortized explanations - this is even more pronounced in the UCI datasets.
Improved Inferences vs. Quality Trade-off: Figure presents the trade-off between number of inferences per explanation and MSE from target explanations using selective and Monte Carlo explanations. The MSE decreases with the number of inferences and selective explanations Pareto dominates Monte Carlo explanations. We also show an "Oracle" that knows priori how optimally
0.20
0,20
0.56
0.39
car
0.20
604
0:24
0:22
0:25
0.22
500%
0.30
25.
1714
1974
45
5074
1072
0.50
40%
105
1074
Compucat
(a) UCI-Adult
(b) UCI-News
(c) Yelp Review
(d) Toxigen
Fig. 5: MSE for the 5% of explanations with the highest MSE the test dataset (y-axis) for selective explanations with varying fraction of points with extra computations (x-axis). Selective explanations are shown (i) black solid bar using the Learned uncertainty and (ii) striped black bar using Deep uncertainty. Dashed red line shows the MSE of amortized explanations in worst 5% explanations.

---

Oracle
Oracle
Gracle
0.015
SE (Leam)
0.03
SE (Learn)
0.10
SE (Leam]
SE (Leam)
Amertized
Amertized
Amortized
0.10
Amortized
NSE
0.010
SVS 131
8 0.02
SVS
SVS
g
0.08
SVS (25)
SVS (12)
SVS (25)
SVS (12)
SV5 (12)
(12)
SVS (50)
SVS (50)
0.06
0.06
0.04
0.000
0.02
50
0
200 400
600
2500
5000
7500
2000
4000
6000
Number of Inferences In)
Number of Inferences In)
Number of Inferences (n)
Number of Inferreccss (n)
(a) UCI-Adult
(b) UCI-News
(c) Yelp Review
(d) Toxigen
Fig. 6: Number of inferences (X-axis) vs. MSE (y-axis). Black curve shows the performance of selective explanations using Learned uncertainty. Purple curve connects Shapley Value Sampling (SVS) with parameters 12, 25, and 50 sequentially until all samples receive SVS-50 explanations and amortized explanations. The red curve is the Oracle that optimally trades off MSE and inferences.
route samples in terms of MSE and inferences. We simulate this oracle by pre-computing SVS explanations with parameters 12, 25, and 50, and selecting the one with the smallest MSE from the target SHAP explanation while manteining the average number inferences shown x-axis. Remarkably, Figure 6 shows that selec- tive explanations closely approximate the Oracle curve, indicating that, on these benchmarks, our method has near- oracle trade-off between the number of inferences and MSE.
100
100
High qualy
Knswrr kim. faceume
Accraa 194
Sdccoon Deop
Secdin Deea
Anceane 33%
Aicaa 2000
Improved Local Fidelity: Figure 7 shows that selective explanations in- crease the local fidelity of the amortized explainer and that local fidelity increases with the fraction of points that receive ad- ditional computations (recourse). Both Yelp and Toxigen models receive expla- nations with initial guess using SVS-12.
of beeu Nututed
hadkknn thee Nuttated
4)
(a) Yelp Review
Toxigen
Fig. 7: Model accuracy (y-axis) when removing the to- kens with the highest attribution scores according to the amortized explainer (black), selective explanations with varying coverage and target explanations (red).
6 Final Remarks
Conclusion: We propose selective explanations that first identify which inputs would receive low-quality but computationally cheap explanation (amortized) and then perform additional model inferences to improve the quality of these explanations. We propose explanations with initial guess to improve the quality of explanations by combining amortized explanations with more expensive explanations Monte Carlo using an optimized combination function, improving the explanation performance. Selective explanations provide new framework for approximating expensive feature attribution methods. Our experiments indicate that selective explanations (i) efficiently identify points that the amortized explainer would produce low-quality explanations, (ii) improves the quality of the worst-quality amortized explanations, (iii) improves the trade-off between computational cost and explanation quality, and (iv) improves the local fidelity of amortized explanations.
Limitations: Selective explanations can be applied any feature attribution method for which amortized and Monte Carlo explainers were developed. However, our empirical results focus on Shapley values. We leave the application of selective explanations to other attribution methods for future work. Additionally, we do not explore image classifiers, which may also interest the interpretability community. Also, we do not explore selective explanations for Generative Language models due to the lack of amortized explainers for such application.
10

---

7 Acknowledgments
The authors thank Amit Dhurandhar for early discussions on the trustworthiness of amortized explainers. This material is based upon work supported by the National Science Foundation under awards CAREER-1845855 CIF-1900750, CIF-2231709 and SIF-23112677 FAI-2040880, and also an Apple Scholar Fellowship. The views expressed here those of the authors and do reflect the official policy or position of the funding agencies.
References
[1] Barry Becker and Ronny Kohavi. Adult. UCI Machine Learning Repository, 1996. DOI: http://b..rrr/10224422///X3
[2] Jianbo Chen and Michael I. Jordan. Ls-tree: Model interpretation when the data are linguis- tic. ArXiv, abs/1902.04187, 2019. URL https://api. semanticscholar. wwwranniiiccolla.rrr/o/ooooee org/CorpusID: 60441455.
[3] Yu-Neng Chuang, Guanchu Wang, Fan Yang, Quan Zhou, Pushkar Tripathi, Xuanting Cai, and Xia Hu. Cortx: Contrastive framework for real-time explanation. ArXiv, abs/2303.02794, 2023.
URL https: h/ap:::///ppi semanticscholes org/ Org/CorpusID 257365448
[4] Ian Covert and Su-In Lee. Improving kernelshap: Practical shapley value estimation via linear regression. International Conference on Artificial Intelligence and Statistics, 2020. URL
https://api.
sew.nti https://ap..emmatcccccoooo
org/ CorppssD::2225533000
[5] Ian Covert and Su-In Lee. Improving kernelshap: Practical shapley value estimation using linear regression. In Arindam Banerjee and Kenji Fukumizu, editors, Proceedings of The 24th International Conference Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pages 3457-3465. PMLR, 13-15 Apr 2021. URL https: // //proceedings.. mlr. press/ caees/vv30//cee v130/covert21a html.
[6] Ian Covert, Chanwoo Kim, Su-In Lee, James Zou, and Tatsunori Hashimoto. Stochastic amortization: A unified approach to accelerate feature and data attribution, 2024.
[7] Ran Yaniv et al. On the foundations of noise-free selective classification. Journal of Machine Learning Research, 11(5). 2010.
[8] Kelwin Fernandes, Pedro Vinagre, Paulo Cortez, and Pedro Sernadela. Online News Popularity. UCI Machine Learning Repository, 2015, DOI: http://doo.rrg/102244/////////
[9] Aditya Gangrade, Anil Kag, and Venkatesh Saligrama. Selective classification via one-sided prediction. In International Conference on Artificial Intelligence and Statistics, pages 2179- 2187. PMLR, 2021.
[10] Yonatan Geifman and Ran El- Yaniv. Selective classification for deep neural networks. In
I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https: https///////oooeeiiing wtwp:eeeedddnnnnnnereeeeeeeeeeee peurips.co. wttpippp.cpppppp...pp /atls///ppper//////1///// 0a2422285599
[11] Yonatan Geifman and Ran El-Y El-va..v. Selectivenet: A deep neural network with an integrated reject option. International conference on machine learning, pages 2151-2159, PMLR, 2019.
[12] Preetam Ghosh and Vaishali Sadaphal. Jobrecogpt explainable job recommendations using Ilms, 2023.
[13] Samit Kumar Ghosh and Ahsan H Khandoker. Investigation on explainable machine learning models to predict chronic kidney diseases. Scientific Repots, 14(3687), 2024. doi: 10.1038/ -1159-024454375-- URL attps://doo org/oogg///. 101. 51141159988------4------443955

---

[14] Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. ToxiGen: A large-scale muchine--eneratte dataset for adversarial and implicit hate speech detection. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, ed- itors, Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guisties ( (Volume 1: Long Papers), pages 3309-3326, Dublin, Ireland, May 2022. Associ- ation for Computational Linguistics. doi: / //aclanthology org 2022/2022.o0 acl aol-oon... 234.
URL https:
[15] Simon Haykin. Neural networks: a comprehensive foundation. forr Prentice Hall PTR, 1994.
[16] Neil Jethani, Mukund Sudarshan, Ian Connick Covert, Su-In Lee, and Rajesh Ranganath. FastSHAP: Real-time shapley value estimation. In International Conference on Learning Representations, 2022. URL https: /ttp::////pp/eeee/eeeeee nww//feuun/eeee teet:/oormmmddd22___.. 2g2GG_2233T
[17] Wenming Jiang, Ying Zhao, and Zehan Wang. Risk-controlled selective prediction for regression deep neural network models. 2020 International Joint Conference on Neural Networks (IJCNN) pages 1-8. IEEE, 2020.
[18] Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, and Orion Reblitz- Richardson. Captum: A unified and generic model interpretability library for pytorch. 2020.
[19] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
[20] HH Lemmer. Note shrinkage estimators for the binomial distribution. Communications in statistics-heery and methods, 10(10):017-1127, 1981.
[21] HH Lemmer. From ordinary to bayesian shrinkage estimators. South African Statistical Journal, 15(1):57-72, 1981.
[22] Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predic- tions. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 4765-4774. Curran Associates, Inc., 2017. URL http://papers. wwws.cc/ppaaeeeeeeeeeee wewpaacc-tto-nnteeeeeeeeeeee
pdf.
[23] Vivek Miglani, Aobo Yang, Aram H. Markosyan, Diego Olanoo and Narine Kokhlikyan. Using Captum to explain generative language models, 2023.
[24] Rory Mitchell, Joshua Cooper, Eibe Frank, and Geoffrey Holmes. Sampling permutations for shapley value estimation. Journal of Machine Learning Research, 23(43):1-46 2022. URL http://jmlx
org/ wwgg/appers/223/2111 0439. html.
[25] John Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. Textattack:
A framework for adversarial attacks, data augmentation, and adversarial training in nlp. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 119-126, 2020.
[26] OpenAl. using-
Using gpt-4 for content moderation.
Accessed: 2024-05-01.
[27] Lucas Monteiro Paes, Dennis Wei, Hyo Jin Do, Hendrik Strobelt, Ronny Luss, Amit Dhurandhar, Manish Nagireddy, Karthikeyan Natesan Ramamurthy, Prasanna Sattigeri, Wernrr Geyer, and Soumya Ghosh. Multi-level explanations for generative language models. arXiv:2403. 14459, wargg/aa///////
2024. URL httpt::///aaaaet
org/ abs abt////b////...//////////// 14459.
[28] Stephan Rabanser, Anvith Thudi, Kimia Hamidieh, Adam Dziedzic, and Nicolas Papernot. Selective classification via neural network training dynamics. ArXiv, 2ttp/220551332.... 2022. URL https: //apii www.sti org/ COoppssDD:2400000
12

---

[29] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should I trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13- 2016, pages 1135-1144, 2016
[30] Yao Rong, Tobias Leemann, Thai-Trang Trang Nguyen, Lisa Fiedler, Peizhu Qian, Vaibhav Unhelkar, Tina Seidel, Gjergji Kasneci, and Enkelejda Kasneci. Towards human-centered explainable ai: survey of user studies for model explanations. IEEE Trans. Puttern Anal. Mach. Intell., 46 (4):2104-2122, November 2023. ISSN 0162-8828. doi: 10. N0.109//PAMM22203333311 URL https://doi. org/10. 1109 / 2023. 3331846.
[31] Ahmed M. A. Salih, Zahra Raisi-Estabragh, Ilaria Boscolo Galazzo, Petia Radeva, Stef- fen E. Petersen, Karim Lekadir, and Gloria Menegaz. A perspective on explainable ar- tificial intelligence methods: Shap and lime. Advanced Intelligent Systems, 2023. URL https://api. stw.a.isssmmttiicoolllaa
Trgg/orpusIDD: ( 258461143.
[32] Patrick Schwab and Walter Karlen. Cxplain: Causal explanations for model interpretation under uncertainty. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https: p/rocee//prooceedingg neur ips. cc/paper fiw.e/ /lper/pper////// Paper Pap.r.pdd.
[33] Robert Schwarzenberg, Nils Feldhus, and Sebastian Möller. Efficient explanations from empiri- cal explainers. In Jasmijn Bastings, Yonatan Belinkov, Emmanuel Dupoux, Mario Giulianelli, Dieuwke Hupkes, Yuval Pinter, and Hassan Sajjad, editors, Proceedings of the Fourth Black- boxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 240-249, Punta Cana, Dominican Republic, November 2021. Association for Computational Linguis- tics. doi: 10. 10.18653/v1/202/ plackboxnlp-1.177 URL https: /aaaaaaaallooooooooooooooooo org blackboxnlp-1.11 17.
or0/ 2021.
[34] Abhin Shah, Yuheng Bu, Joshua K Lee, Subhro Das, Rameswar Panda, Prasanna Sattigeri, and Gregory Wornell. Selective regression under fairness criteria. International Conference on Machine Learning, pages 19598-19615, PMLR, 2022.
[35] Sheng Wang, Zihao Zhao, Xi Ouyang, Qian Wang, and Dinggang Shen. Chatcad: Interactive computer-aided diagnosis on medical image using large language models, 2023.
[36] Michael L. Waskom. seaborn: statistical data visualization. Journal of Open Source Software 6(60):3021, 2021. doi: 10.21105/joss.0300 03021. URL http://///01..r/1100110///////// 03021.
[37] Hilde J. P. Weerts, Werner van lpenburg, and Mykola Pechenizkiy. A human-grounded evaluation of shap for alert processing. ArXiv, abs/1907.03324, 2019. URL https:
/ //api.semantaei..e mer.ssiamnnnichoolatt
org/CorppsII: 1958333477
[38] Chenghao Yang, Fan Yin, He He, Kai-Wei Chang, Xiaofei Ma, and Bing Xiang. Efficient shapley values estimation by amortization for text classification. In Annual Meeting of the Association for Computational Linguistics, 2023. URL sstps://apps semanticscholar org/CorpusID:
258987882.
[39] Ahmed Zaoui, Christophe Denis, and Mohamed Hebiri. Regression with reject option and application to knn. Advances in Neural Information Processing Systems, 33:20073-20082. 2020.
[40] Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In David Fleet, Tomas Pajdla, Bernt Schicle, and Tinne Tuytelaars, editors, Computer Vision ECCV 20/4, pages 818-833, Cham, 2014. Springer International Publishing. ISBN 978-3-319- 10590-1.
[41] Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-Ievel convolutional networks for text classification. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, ed- itors, Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015. URL https: /ttpapp::///poceeee http://///oceeeeeeeeeeee www.iinggn..e.ureeeeeeeeeeereeee wwwrips.cc./eeeeeeeeeeeeoooo.eee wttplpp.ccpppprrreeeee httee//fleee//pee///////111 250c6601117775fff33ccbbbbbb66e33 Paper pdf.
13

---

A Overview
In this supplementary material provide the following information:
Appendix discuss other high-quality and Monte Carlo explainers.
Appendix C discuss a guide to select the coverage when the agent providing selective explanations has a budget for the average number of inferences to provide an explanation.
Appendix D shows
experimental results on selective explanations.
Appendix shows the proofs for the theoretical results in Section 4.
B Additional Explanation Methods
In this section, we deseribe high-quality, Monte Carlo, and amortized explainers with further details.
B.1 High-Quality Explainers
Shapley Values (SHAP) [22] is high-quality explainer that attributes value Φ; for each feature I; in x = (II,,.,,a)) which is the marginal contribution of feature x; if the model was to predict (2).
1 d
Σ d
|S| (==1)
-1 (Ng())))())) hy(ars))
Φ (1 y)
(13)
SCI//(0
SHAP has several desirable properties and is widely used. However, (2) indicates, computing Shapley values and the attribution vector Target( x y) = (Φ1(2, y).... Ød y)) requires 2d in- ferences from h, making SHAP impractical for large models where inference is costly. This has motivated several approximation methods for SHAP, discussed next".
Local Interpretable Explanations (Lime). Lime is another feature attribution method [29] widely used to provide feature attributions. relies on selecting combinations features, removing these features from the input to generate perturbations, and using these perturbations to approximate the black box model locally by a linear model. The coefficients of the linear model are considered to be the attribution of each feature. Formally, given weighting kernel π (S) and penalty function Ω the attribution produced by lime are given by
(Ø, a) = argmin Σ T(S) (((xs) - an @ERd, .EE SC(d]
-
Σ
(14)
SS
where Target(æx, y) é. As SHAP, compute the feature attributions using lime, we need to perform large number of model inferences, which is prohibitive for large models.
B.2 Monte Carlo Lime
Shapley Value Sampling (SVS) [24] is Monte Carlo explainer that approximates SHAP by restricting the sum in (2) to specific permutations feature. SVS computes the attribution scores by uniformly sampling features permutations S1,..., Sm restricting the sum in (2) and performing n = md + inferences. We denote SVS that samples m feature permutations by SVS-m.
Kernel Shap (KS) [22] is Monte Carlo explainer that approximate the Shapley values using the fact that SHAP can computed by solving the optimization problem
Tt
(Ø. a) argmin
(S.)
h(rss)) ag Σ Φ0
(15)
dERd aaE
JES,
using (()) (S) (S)) and where MC y) = Φ. Kernel Shap samples n > 0 feature
combinations S1,.., ..., Sn and define the feature attributions be given by the coefficients d. We refer to Kernel Shap using inferences as KS-n. We use the KS-n from the Captum library [18] for our experiments.
WWe also discuss Lime and its amortized version in Appendix B
14

---

Sample Constrained Lime. To approximate the attributions from Lime, we consider the sample- contained version of (15). Instead of sampling all feature combinations [d]. we only uniformly sample fixed number n of feature combinations S Sn For our experiments, shown in the appendix, we use the Sample Constrained Lime from the Captum library [18].
B.3 Amortized Explainers
Stochastic Amortization [6] is Amortized explainer that uses noisy Monte Carlo explanations to learn high-quality explanations. Covert et al. [6] trained an amortized explainer Amor E F in hypothesis class (we use multilayer perceptrons) that takes an input and predicts an explanation. Specifically, taking the amortized explainer to be the solution of the training problem given in (3).
Amor E arginin
Σ (o.y)E
1f y) MC" (æ, y)
(16)
fE9
We are interested in explaining the predictions of large models for text classification. However, the approach in (3) is only suitable for numerical inputs. Hence, we follow the approach from Yang et al. [38] to explain the predictions of large language models, explained next.
Amortized Shap for LLMS [38] a Amortized explainer similar to the one (3) but tailored for LLMs.. First, the authors note that they can use the L.M to write all input texts ær sequence of token embedding (z)), G|m) (x)] where ex(x) E 12d denotes the LLM embedding for the i-th token contained the input text æ and [x] is the number of tokens in the input text. Second, they
restrict I in (3) to be the set of all linear regressions that take the token embeddings and output the
token attribution score. Then, they solve the optimization problem in
120
W E argmin
Σ
| cz(2) + MC" (x.,))
(17)
WERd BEB
(m. E
and define the amortized explainer as Amor(ar) = (WT (x)) + b..., W WTe (x) b).
We use stochastic amortization produce amortized explainers for tabular datasets and Amortized Shap for LLMs to produce explainers for LLM predictions. Both explainers are trained using SVS-12 MC".
C Selecting Coverage for a Given Inference Budget
Providing explanations with initial guess in- creases the number of model inferences from when using solely the amortized explainer However, a practitioner may have budget of inferences, i.e., a maximum average number of infer- ences they are willing perform to provide explanation. We formalize the notion of inference budget in Definition 3.
Determining Coverage from Inference Budget:
1.
Definition 3 (Inference Budget). Denote by N(SE(x, y)) the number of model inferences to produce the explanation SE(æ, y). The inference budget Noudget N is the maximum average number of inferences a practitioner is willing to perform per explanation, i.c., it is such that
Noudget > E IN(SE (x, y))]
(18)
Once an inference budget Noudget is defined, the coverage should be set to follow it. In Proposition we show the minimum coverage for the selective explanations to follow the inference budget.
Proposition (Coverage for Inference Budget). Lei Noudget be the inference budget, and assume that the Monte Carlo method MC (x, y) uses n model inferences. Then, the coverage level a should be chosen such that
n + Noudget
min a, such that TE (N(SE(( y))] < N Neudget- eE
(19)
n
Recall that SVS-m performs 1+ dm inferences (x Rd), and KS-m performs m inferences.
15

---

D More Experimental Results
In this section, we (i) give further implementation details and (ii) discuss further empirical results.
D.1 More Details on Experimental Setup
High- Bigh-Quality Explanations: We define the high-quality explanations for the tabular datasets to be given by Kernel Shap with as many inferences as needed for convergence, using the Shapley Regression library [4]. For the textual dataset, following [38], define the high-quality explanations to be given by Kernel Shap using 8912 model inferences per explanation.
Amortized Explainers: For the tabular datasets, we use the amortized explainer from [6] that we describe in Section 2. Specifically, we use multilayer perceptrom model architecture to learn the shapley values for the tabular datasets. For the textual datasets, we use the linear regression on token-level textual embeddings to learn the shapley values, as described in Section 2. Both amortized models learn from the training dataset of explanations generated using Shapley Value Sampling from the Captum library [18] with parameter 12, i.e., SVS-12.
Uncertainty Metries: We test the two proposed uncertainty metrics in Section 3, namely, deep uncertainty and uncertainty learn. For deep uncertainty, we the training pipeline for the amortized explainers 20 times for each dataset we perform experiments on, resulting in 20 different the multilayer perceptrom as the hypothesis class with only one hidden layer. The hidden layer was composed of 3d neurons where is the dimension of the input vector æ E Rd The uncertainty learn metric was trained on Derain: the same training dataset as the amortized explainers.
amortized explainer that we to compute (4). For uncertainty learn,
Dataset sizes: We use 4000 samples from each dataset due to computational limitations on the computation of high-quality explanations used to evaluate selective explanations. All explanations were computed using the Captum library [18]. The dataset with = 4000 samples was partitioned in three parts, D Drrain with 50% of points, Deai with 25% of points, and Drest with the other 25% of points.
Computational Resources: All experiments were in A100 40 GB GPU. For each dataset, compute different Monte Carlo explanations. For the UCI-News dataset, the high quality explanations took 4:30 hours be generate until convergence while for UCI-Adult took 3:46 hours. For the tabular datasets, all other Monte Carlo explainers were generated in less than hour. For the language models, the high-quality explanations with 8192 model inferences, took 18:51 hours for the Toxigen dataset and 20:00 hours for the Yelp Review datasets. The other used Monte Carlo explanations took proportional (to the number of inferences) time be generated.
D.2 Uncertainty Measures Impact on Spearman's 's Correlation
Figure 8 shows in the x-axis the coverage (a) and in the y-axis the average Spearman's 's correlation of the selected amortized explanations from high-quality explanations using deep uncertainty (with 20 models) and the uncertainty learn to select low-quality explanations. The Oracle is computed by sorting examples by the smallest to higher MSE and computing the average Spearman's 's correlation in the bottom x-axis points accordingly to the MSE and is the best that can be done in terms of MSE.
Figure 8 shows that the Oracle and proposed uncertainty metries don't always select the points with the smallest Spearman's 's correlation first. This implies that MSE and Spearman's 's correlation don't 't always align, i.e., there are points with high MSE and high Spearman's 's correlation at the same time. However, we note that the uncertainty learns selector can be applied to any metric é as we define in (5) including Spearman's correlation and any combination of Spearman's correlation and MSE aiming to approximate both metrics. Moreover, when the smallest MSE aligns with the highest Spearman's 's correlation, i.e., the oracle is decreasing Spearman" 's correlation when the coverage increases (Figure 8 (a) and (c)), the proposed uncertainty metries also accurately detect the low-quality explanations in term of Spearman's 's correlation.
'The oracle computationally expensive because it requires access high-quality explanations.
16

---

Spcaranns in Covered
Secaman's Covered
Spraman's in Covemid
in Coverd
0.9
0.35
0.15
0.30
0.10
0.25
0.7
0.25
0.00
25
50
75
100
25
50
75
100
25
50
75
100
25
50
75
100
Coverage 0%)
Coverage 0%)
Coverage (%)
Coverage (8)
(a) UCI-Adult
(b) UCI-News
(c) Yelp Review
(d) Toxigen
Selectors
Learn
Deep
Fig. 8: Coverage vs. Spearman's ' correlation from the high-quality explanation. Coverage is the per- centage of the points that the selection function predicts that will receive higher-quality explanation, ie., (x) = 1. When coverage is 100% Spearman's correlation is the average performance for the amortized explainer.
D.3
The Effect of Explanations with Initial Guess
In Figure 9 compare explanations with initial guess (Definition 2) to only using the Monte Carlo to provide recourse to the low-quality explanaitons, i.e., Å = O we call Naive. In all tested cases, Spearman's correlation of the Monte Carlo method is comparable explainer. Although selective explanations optimized for MSE by using explanations with initial guess (Definition 2), observe that the Spearman's 's correlation of selective explanations close or larger than the naive method, once again, demonstrating the efficacy of selective explanations.
larger than the amortized
Searman's Corrdatio
0.90
Corristioon
0.
0.475
Speaman''s Correcaiion
Cormlcttio
0.85
0.450
0.80
0.425
0.22
0.75
0.400
0.3
0.70
0.20
0.375
0.2
0.65
0.18
60
80 Computations
0
20
60
80 Computation
100
40
80 Computations
100
Computations
UCI-Adult
(b) UCI-News
Yelp Review
Toxigen
Fig. 9: Fraction of the population that receive explanations with initial guess (x-axis) vs. their Spearman's correlation from the high-quality explanations (y-axis), Naive uses '₁, while initial guess uses explanations with initial guess, i.e. when Av, is given in (12).
D.4 Spearman's correlation Explanation of initial guess in the worst explanations
Figure 10 shows the Spearman's 's rank correlation selective explanations for the points receiving explanations with the smallest correlation. The figure shows that selective explanations significantly decrease the worst-case Spearman's rank correlation of amortized explanations. With just 20% coverage the Spearman's 's rank correlation increases consistently across datasets. Remarkably, when providing explanations with initial guess for 50% of the samples in the Yelp dataset (Figure 10 (c)), selective explanations result in explanations with positive correlation with target explanations in the worst 5%. At the same worst 5% of points, amortized explanations are negatively correlated with target explanations.
D.5 Performance for Different Monte-Carlo Explainers
Figure 11 shows how the MSE and Spearman's correlation behave accordingly with the quality the Monte Carlo explainer. We compare Kernel Shap and Shapley Value Sampling in all experiments. We observe that when the quality of the Monte Carlo explainer increases, the quality of the Selective ex- planation also increases. i.c., the MSE decreases and the Spearman's 's correlation increases. Morcover,
17

---

0.0
0.8
2.33
2,000
Spearmee Wi
0.0
0.00
-1025
3.85
0.0
2.0
24.8
30%
100%
-243
50%
131%
-2130
100%
Prcton farr Comparatione
Prarton Eorr Compuestione
Facntor Betrs Computaiiin
Imortin AI Eara Compurations
(a) UCI-Adult
(b) UCI-News
(c) Yelp Review
(d) Toxigen
Fig. 10: Spearman's correlation for the 5% of explanations with the smallest correlation in the test dataset (y-axis) for selective explanations with varying fraction points with extra computations
(x-axis). Selective explanations are shown in (i) black solid bar using the Learned uncertainty and (ii) striped black bar using Deep uncertainty. Dashed red line shows the MSE of amortized explanations in worst 5% explanations.
3.322
4.82
9.50
0551
315. 11
1:322
300
11:)
0:00
0.50
0.50
2500
FX510
1.25
032
(20
(a) UCI-Adult
(b) UCI-News
(c) Yelp Review
(d) Toxigen
Fig. 11: MSE (top) and Spearman's 's correlation (bottom) for selective explanations using different
Monte Carlo explainers.
we also observe diminishing returns, i.e., after certain point, increasing the quality of the Monte Carlo explanations doesn't 't lead to tailored increase in performance. For example, observe the SVS method in the tabular datasets Figure 1I (a) and (b). We also observe that providing explanations with initial guess has a high impact on both Spearman's 's correlation and MSE when only providing recourse toa small fraction of the population. For example, when providing explanations with initial guess for 20% of the population using SVS-12 in the Yelp Review dataset, Figure 11 (c), increases the Spearman's 's correlation in more than 50% (from 0.2 more than 0.3).
D.6 Time Sharing Using Selective Explanations
Figure 6 presents the trade-off between number of inferences per explanation and MSE from target explanations using selective and Monte Carlo explanations. addition to what is shown in Figure we also show selective explanations using the Deep uncertainty metric as a selector.
18

---

Oracle
Oracle
0.015
SE (Learn)
0.03
SE (Learn)
SE (Deep)
Amortized
Amortized
0.010
SVS (3)
MSE
SVS (3)
MSE
0.02
SVS (6)
SVS (6)
SVS (12)
SVS (12)
0.005
0.01
0.000
0
50
100
150
0.00
Number of Inferences (n)
200
400
600
Number of Inferences (n)
(a) UCI-Adult
(b) UCI-News
Oracle
0.12
Oracle
SE (Learn)
0.10
SE (Leam)
SE (Deep)
0.10
SE (Deep)
Amortized
0.08
Amortized
MSE
SVS (12)
MSE
0.08
SVS (12)
SVS (25)
SVS (25)
0.06
SVS (50)
0.06
SVS (50)
0.04
0.04
0.02
2500 5000 7500
2000
4000
6000
Number of Inferences (n)
Number of Inferences (n)
Yelp Review
(d) Toxigen
Fig. 12: Number of inferences (X-axis) vs. MSE (y-axis). Black curve shows the performance of selective explanations using Learned uncertainty. Purple curve connects Shapley Value Sampling (SVS) with parameters 12, 25, and 50 sequentially until all samples receive SVS-50 explanations and amortized explanations. The red curve is the Oracle that optimally trades off MSE and inferences
E Proofs of Theoretical Results
Theorem (Optimal An). Let 01 < 02 <.. < 0m and define Qi as in (9). Then, X that solves the optimizution problem in (11) is given by
E E(my)) (MC (x, y) MC² (x, y) MC (x, y) Amor(x, ())
A (n(e)EQ
(20)
Σ (( |Amor( y) MC (æ, y) (x)))E
Proof. First, recall that
SE(æ, v) MC" (æ, y)) 2
X A argmin
Σ
(21)
AER
( en (x))
argmin AER
Σ (D. 8n (x)
AAmor(ao y) + (1 Ax, MCC (a.
(22)
19

---

Note that the function in (22) is convex in A; therefore, if the derivative of it with respect to À is zero, then the lambda that achieves the zero gradient the minima. So, let's derivate (22) find A,.
Σ
|AAmor(z; y) + (1 A)MC" (x, y) MC" (z, w)
(23)
dd D % (m)EQ.
= 2
Σ
X||MC y) Amor(a: y)1l²
(24)
(m. * (z)EQ.
2 Σ
(M," y) MC" (x, v). MC" y) Amor(z, 3))
(25)
(m.y))Dmm 94 (2))E.
From (25) we conclude the proof by showing that
E(ayy)D (MC" (x. y) MC" (x, v). MC (z, y) - Amor( y)) (x)c)
A = ==
(26)
Σ
(D. (m))E
|MCC (æ, y) Amor(æ y)ll²
Theorem 2 (A: R ). Let the Monte Carlo explanation used to provide recourse MC" to be different enough from the amortized explainer, i.e., TE [[MMC (X,CC(X,XXX) X,Y) Amor( X, Y)
0. Also, assume that MC² is good Monte Carlo approximation for the high-quality explainer | [IMM (X,Y) Target( (x, Y)1I 1²" for e Lap=" Recall that a RRd, If 11 Targett(, y)1 Cd some the explanations are bounded, i.e., ||MC" (x. f)1l ||Amor 8)1l. C > 0 then
Target, ie., E
Pr[lA- x e -21 -
(27)
where Q2i the number of points the validation dataset D vat that are the bin Qi-
Proof. Denote |Q:1 = 1( y) Dral, s.t. (ax) &iH
start by showing that if [||MC* Amor(X,X)III = μ then
Pr
Σ
|IMC y) Amor(z:, y)1l² <
(28)
Q2
2
(m.y) Ex)
Pr
Σ D en (m))Q.
|MC (x, y) - Amor( y)12 2
(29)
Q
- 14 Se
AC4
(30)
Where the inequality in (30) follows from Hoeffding's inequality and the fact that:
MMC (z, y) Amor( 2) < ||MC (æ, v) ||Amor(æ y) < 2Cd.
(31)
Second, recall that E ||MMC (X Y) Target(X, y)[[] 1r- de2 Then, have that
Target((x (z. v) Amor( v)1l > 2 21
Pr
Σ
(32)
IQil
4
DD (x))eE
20

---

Targeet(a; y) Amor( (z, y)ll 11¹ 2 2214²
Pr
11*
(33)
(m. 4A (ar)EE
Target((x y) Amor( y)11 118 222
Pr
(34)
Qi
20
ED (x))E
-14,410.1
Se
(35)
Where the inequality in (35) follows from Hoeffding's inequality and the fact that:
Target(æ y) Amor( y)1l s <|aarett y)ll ||Amor(æ, y) S 2Cd.
(36)
Third, notice by directly applying Theorem | and replacing the Monte Carlo explanation by the high-quality explanation, we have that
E(ay)eD (x, y) - Target(æ: y), MC" (ax y) Amor(a: yz) (D))E
A opt
(37)
Σ ED |MCC (æ, y) Amor (x, 22
(m)EQi
Hence, we can write A A as
AxPE
Arl
(38)
( (MC" (x, 8) Target( y), MCM (x: y) Amor( y))
(coeEQ
(39)
Σ ( (()))QQ
MMC (z, y) Amor(x, y)
1/2
Σ Ei |MC (x, v) - Target( y)11 MC" (x, y) - Amor(x, y)1l (a))
(40)
Σ (D.))
MMC (z, y) Amor(æ, y)
sh (m)
where the last inequality (40) comes from the in (40) by A, i.c.,
inequality. Denote the denominator
Σ
||MC (x, y) Amor(æ, 2) A.
( *A (m)E
Lastly, notice that MC (x, y) is sampled independently of MC" (x. y) and that Target(æ: y) is deterministic. Therefore:
Pr[1Ap00 opt - 2
(41)
1/2
E(argieD |MC" (x. v) Target( v) (æ, v) Amor(æ, v) (n()eQQ
< Pr
Σ EiaweDD... (x(m).Q
MMC (a. y) Amor(æ, 1
(42)
E(enyeDo
IMC" (x, y) Target(æ,, y) (x. y) Amor(æ,y)1B [x(e)
Pr
(43)
2
21

---

Σ
De 5h (m()EQQ
|MC" (x, v) Target( y)1 MC" (æ, (a," y) Amo y)1
Sr
2
A
2
x Pr
A
2
Σ
(a D.. en (x))Q.
|MC (z, v) Target(æ:, y)]l Amor(a:, y)11
+ Pr
2
A
2
X Pr
A
(44)
2
u2
SPr
Σ
IMC" (æ, y) Target(æ, | IMC" y) Amor (2) 2 e²
4
S/. Ex)
+ Pr
(45)
10.1 102
-,
<e
+
1O0C
(46)
Where the inequality in (42) is a direct application of 40, the inequality in (44) comes from simply conditioning, the inequality in (45) comes from the fact that probabilities rid of the first term in (45) (first out of lines) and the fourth term in (45) (forth out of lines) and the fact that MC" (ax, y) is sampled independently of MC" (at, y) and that Target(ar, y) is deterministic. Finally, the last inequality in (46) comes from applying (30) and (35).
bounded by getting
Hence, from (46), we conclude that
11t 2000d
Pr|lAr
X
4Ca
(47)
Proposition 2 (Coverage for Inference Budget). Let Noudget 2 1 be the set inference budget, and assume that the Monte Carlo method MC" (xx y) uses n model inferences. Then, the coverage level should be chosen such that
Noudget
argmin [I(SSE) y))l S Noudget} oE(0.1]
(48)
Recall that Shapley Value Sampling with parameter m performs dm inferences (æ E Rd),, and Kernel Shap with parameter m performs m inferences.
Proof. Let (0, 1], then a portion of examples receive explanations from the amortized explainer, i.e., they receive one inference, and - a portion of examples receive explanations with initial guess, i.c., model inferences. Therefore, the expected number of model inferences per instance is given by (49).
E IN(S((z, v))] + (1 (n + order the inference budget to be followed, it is necessary that E [N(SE(<,, y))l = a + (1 (n + 1) < N budget-
(49)
(50)
From (50), conclude that:
a 1 Noudget
(51)
n
Hence,
71 + 1 N Nougget
argmin a E (E v)) Noudget
(52)
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our claims are supported both by experiments (Section 5) and theoretical results (Section 4),
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation in the end of Section 6.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
23

---

While we don't show all assumptions in Theorem 2 in the main paper for simplicity, we do provide the necessary assumptions in Appendix For all the other results, we provide the the necessary requirements.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide all necessary information in Section 5 and also in Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
24

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We share the code to generate selective explanations with the Conda environ- ment in an yml file run in the supplementary material. We also share a Github repository where the code is shared.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
(hbllii/guidees/oeeetttmiss setpbbmmsssooooocccc for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ //niss.ae /twpps.cccpppppeeee/o/
wwtec//giiee///ee Caaeae//ddSSSubbbeie iimmmissionnn onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permirted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We do share this informations. In addition, these and others implementation details can be verified in our publicly available code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report error bars using Seaborn [36] with 95% confidence using the bootstrap method.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
25

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report it in Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: ptttps://neurppii cc/ppblicc/thicccuueeeeeeeeeeeee
Answer: [Yes]
Justification: Yes, does.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: We propose method to improve explanation quality feature attribution methods which social impact is not clear beyond the impacts of the attribution methods themselves.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
26

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We don't release novel data or models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All datasets and models used are referenced.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
27

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasttss has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Yes, they are.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
28

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "wQPTkdPrDC", "reviewer": "Reviewer_XcVA", "summary": "The paper proposes a method, termed \"Selective Explanations,\" aimed at improving the quality of explanations generated by amortized explainers in machine learning. The authors introduce a technique that detects low-quality explanations and employs a combination of amortized and Monte Carlo methods to enhance them. The approach leverages an \"explanations with initial guess\" technique, allowing a trade-off between computation speed and explanation quality. The proposed method is validated across different datasets, showing that it can improve the poorest quality explanations typically provided by amortized explainers.", "questions": "See weaknesses.", "limitations": "Given the current state of the submission, the reasons to reject slightly outweigh the reasons to accept.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The concept of using a selective approach to manage computational resources while improving explanation quality is compelling and timely.\n2. The proposed method is intuitive and easy to understand.\n3. The paper is well written.", "weaknesses": "1. The experimental design does not strongly support the major claims on the reduction of computational cost. Evaluations on the tradeoff between explanation quality and computational cost would be helpful.\n2. The relationship between uncertainty and explanation quality is unclear, it would be better to have empirical or theoretical proof on their correlations. In addition, the proposed uncertainty measurement also introduces computational overhead when “run the training pipeline for the amortized explainer described in (3) k times” (line 139).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review. Your feedback will positively impact the updated version of our paper and we will include all of your comments. We also appreciate that you found our manuscript well-written, our method intuitive and easy to understand, and that our idea of Selective Explanations is compelling and timely. We address your comments below. We hope you can engage with us during the discussion period.\n\n**Q1)  “The experimental design does not strongly support the major claims on the reduction of computational cost. Evaluations on the tradeoff between explanation quality and computational cost would be helpful.”**\n\nWe appreciate this comment and we kindly ask that you refer to Q1 in the global answer to all reviewers above. The TL;DR is: the trade-off between computational complexity (given by number of model inferences) and quality of explanations (MSE from high-quality explanations) is captured in Fig. 4 and 10 of the paper. In particular, Fig. 10 shows that our method significantly reduces the number of inferences required to achieve a given explanation quality in comparison with Monte Carlo explanations – specifically SVS. Remarkably, Fig. 10 shows that Selective Explanations performs very close to an oracle that \"knows\" which explainer to use for a given MSE and constraint on number of inferences. We will make this clearer by bringing Fig. 10 and the associated discussion to the main body of the paper instead of the appendix. Please, check the global answer (Q1) for more details.  \n\n**Q2) “The relationship between uncertainty and explanation quality is unclear, it would be better to have empirical or theoretical proof on their correlations.”**\n\nAgain, thank you for this thoughtful comment. Here, again, we kindly ask for you to refer to the global answer for all reviewers above and the attached pdf, which empirically shows a high correlation between uncertainty and explanation quality. TL;DR: Figure 3 shows the relationship between uncertainty and explanation quality by demonstrating that the amortized explanations with smallest uncertainty also have the smallest MSE from high-quality explanations and vice-and-versa. However, we agree with you that more direct evidence is needed. To address this, we computed this correlation, in a new Table 1 (attached in the PDF) and will add it to the main paper. This table presents positive Pearson and Spearman’s correlations between our uncertainty measures and MSE and a strong monotonic relationship between both quantities, especially for the language datasets. This evidence reinforces the effectiveness of our approach in detecting lower-quality explanations in the main tasks of interest. Again, please, check the global answer (Q2) for more details.\n\n**Q3)  “In addition, the proposed uncertainty measurement also introduces computational overhead when “run the training pipeline for the amortized explainer described in (3) k times” (line 139).”**\n\nWe agree with you, the Deep Uncertainty measure introduces a computational overhead. Please note that this computational overhead is exactly the reason we proposed a second method, referred to as \"Learned Uncertainty,\" which only needs to be trained once. Our experiments suggest that the Learned Uncertainty provides a better performance for selective explanations of language models – our main use case of interest. We will make this distinction clearer in the paper by adding the following paragraph after line 148: \n\n“To address the computational overhead of Deep Uncertainty and directly target low-quality explanations, we introduce Learned Uncertainty. This alternative method requires only one training run, drastically reducing computational costs. Learned Uncertainty is optimized to predict discrepancies between amortized and high-quality explanations.”\n\nWe proposed Deep Uncertainty because this method is directly inspired by Deep Ensembles [1] used for distribution-free uncertainty quantification. It achieves favorable performance (see Figs. 3, 4, 5, and 6), but it indeed comes with a computational overhead. We introduce it because of its connection with Deep Ensembles traditionally used for uncertainty quantification. \n\n[1] Lakshminarayanan et al. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. NIPS 2017.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vcvOemXuAA", "reviewer": "Reviewer_kYJk", "summary": "This paper proposed a feature attribution method that detects when amortized explainers generate low-quality explanations and improves the explanations with their linear interpolation of themselves and expensive high-quality explanations.\nTo detect the low-quality explanations of the amortized explainers, the proposed method measures the uncertainty of the explanations.\nThen, based on a selection function defined by the uncertainty, it selects either using the explanations by the amortized explainers or using the linear interpolation of the explanations by the amortized explainers and Monte Carlo explainers.\nThe experiments were conducted on tabular and text datasets, and the experimental results showed that the proposed method achieved better accuracy of explanations than using the amortized explainers and the Monte Carlo explainers solely.", "questions": "- **Learned uncertainty:** The loss $\\ell(\\mathrm{Amor(x;y)}, \\mathrm{MC}^n(x,y))$ in (5) is the same as the objective of the amortized explainer $\\mathrm{Amor(x;y)}$ in (3).\nTherefore, if the amortized explainer is fitted enough to the training data, the loss $\\ell$ is consistently near zero, resulting in the learned uncertainty function $s$ being consistently near zero, too.\nThis learned uncertainty function does not seem to work well at an inference phase. What is the justification for (5)?\n- What does 'Random' in Figure 3 mean?", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This work is the first to propose a feature attribution method in the \"selective\" setting, which can improve the quality of the explanations with Monte Carlo explanations when amortized explainers generate low-quality explanations. Since the proposed method is an extension of selective classification and regression, which are successful and well-studied approaches, it is easy to imagine it working well.\n- The formalization of the proposed method is almost appropriate except that I am concerned about the learned uncertainty (5), and it is described clearly.", "weaknesses": "- In common, the quality of explanations has a trade-off between computational efficiency and accuracy. Although the experimental results show that the proposed method can improve the accuracy of explanations, those on computational efficiency are not investigated.  \n- The generated explanations are quantitatively evaluated. However, due to the lack of qualitative evaluation, it is unclear how good the explanations generated by the proposed method are actually from a user perspective.\n\nThe following are a minor point:\n- If (5) is MSE, should use $\\|| \\cdot \\||$ instead of $| \\cdot |$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your thoughtful review. Your feedback will positively impact the final version of our paper! We address your questions below.\n\n**Q1) ”Although the experimental results show that the proposed method can improve the accuracy of explanations, those on computational efficiency are not investigated.”**\n\nThank you very much for this comment! Please kindly refer to the global answer to all reviewers above. TL;DR: the trade-off between computational complexity (given by number of model inferences) and quality of explanations (MSE from higher-quality explanations) is captured in Fig. 4 and 10 of the paper. Particularly, Fig. 10 (reproduced in the attached PDF) shows that our method performs very close to an oracle that \"knows\" which explainer to use for a given MSE and constraint on number of inferences. We will make this clearer by bringing Fig. 10 and the associated discussion to the main body of the paper instead of the appendix. Please, check the global answer (Q1) for more details.\n\n**Q2) “The generated explanations are quantitatively evaluated. However, due to the lack of qualitative evaluation, it is unclear how good the explanations generated by the proposed method are actually from a user perspective.”**\n\nThank you for this great point. The main reason for the lack of user evaluation is that we are approximating expensive-to-obtain yet established explanation methods that were extensively tested and evaluated by humans such as LIME [1] and SHAP [2]. Given these established results, we follow the methodology in most papers that aim to approximate feature attribution methods [3, 4, 5, 6] and report both MSE and Spearman’s Correlation to SHAP explanations computed until convergence. For low MSE and high Spearman Correlation with SHAP (which our Selective Explanation method achieves), the performance of our method is essentially the same as SHAP. In other words, our goal is to approximate expensive-to-obtain SHAP explanations as closely as possible, while lowering the required number of inference and, thus, computational cost.\n\nNevertheless, we acknowledge this point, and recognize that – since we aim to approximate methods such as SHAP – our method is ``high quality\"' insofar that SHAP is high-quality, and will note this caveat in the paper.\n\n[1] Ribeiro et al. \"Why Should I Trust You?\": Explaining the Predictions of Any Classifier. KDD 2016.\n\n[2] Lundberg et al. A Unified Approach to Interpreting Model Predictions. NIPS 2017.\n\n[3] Jethani et al. FastSHAP: Real-Time Shapley Value Estimation. ICLR 2022.\n\n[4] Schwab and Karlen. CXPlain: Causal Explanations for Model Interpretation under Uncertainty. NeurIPS 2019.\n\n[5] Yang et al. Efficient Shapley Values Estimation by Amortization for Text Classification. ACL 2023.\n\n[6] Covert et al. Stochastic Amortization: A Unified Approach to Accelerate Feature and Data. Attribution. arXiv:2401.15866.\n\n\n**Q3) “If (5) is MSE, should use ||⋅||  instead of |⋅|.”**\n\nThank you for this careful point. Notice that $\\ell(\\text{AMOR}(x;y), \\text{MC}(x; y))$ is a scalar. Hence, $||.||$ and $|.|$ are equivalent.\n\n**Q4) “Learned uncertainty: The loss in (5) is the same as the objective of the amortized explainer in (3). Therefore, if the amortized explainer is fitted enough to the training data, the loss is consistently near zero, resulting in the learned uncertainty functions being consistently near zero, too. This learned uncertainty function does not seem to work well at an inference phase. What is the justification for (5)?”**\n\nThank you for this great point. In the case there is overfitting, the selection function might not be sufficiently accurate (neither the amortized explainer), as you mentioned. In such cases, we advise the users to reserve a validation dataset to fit the uncertainty function. We highlight that we did not observe overfitting in any of our experiments where we train the uncertainty measure using the training dataset, and train the amortized explainers until convergence on the same training dataset, and evaluate their performance on a test set. However, this does not preclude the risk of overfitting in other settings. We will add your comment to our limitations section and advise on the use of a validation set instead of the same training dataset.\n\n**Q5) \"What does 'Random' in Figure 3 mean?\"**\n\n\"Random\" means that we select explanations uniformly at random instead of using the uncertainty metrics. This leads to an average MSE that is near the MSE of the amortized explainer and is  independent of the coverage. We will add this description in line 255 of our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "00021mgpIO", "reviewer": "Reviewer_tCbB", "summary": "This paper introduces a novel method called selective explanations for improving the efficiency and accuracy of feature attribution methods for black-box machine learning models. The key contributions are:\n\n- A zero-cost proxy to evaluate the adversarial robustness of deep neural networks without training.\n- A selective explanation method that detects when amortized explainers generate low-quality explanations and improves them using explanations with initial guess.\n- An optimization approach for combining amortized and Monte Carlo explanations to improve explanation quality.\n- Comprehensive evaluation on both tabular and language model datasets, demonstrating improved explanation quality with reduced computational cost.\n\nThe authors argue that their method addresses the challenges of computational expense in existing feature attribution methods while maintaining or improving explanation quality.", "questions": "- How does the performance of selective explanations scale with extremely large models (e.g., models with billions of parameters)?\n- Have you explored using more advanced uncertainty estimation techniques, such as those based on Bayesian neural networks?\n- How sensitive is the method to the choice of amortized explainer? How might it perform with different types of amortized explainers?\n- Could the selective explanation approach be extended to other types of explanation methods beyond feature attribution (e.g., example-based explanations)?\n- How might the method be adapted to handle streaming data or online learning scenarios where the underlying model is continuously updating?", "limitations": "The authors discuss limitations of their work at the end of Section 6. They acknowledge that the method is currently focused on Shapley values and has been primarily tested on specific types of models and datasets. They also note potential challenges in applying the method to image classifiers. These limitations are reasonably addressed, though a more detailed discussion of potential failure modes or edge cases could have been beneficial.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Novel approach: The selective explanation method offers a new paradigm for balancing efficiency and accuracy in feature attribution.\n- Theoretical foundation: The paper provides rigorous mathematical formulations and proofs for key components.\n- Comprehensive evaluation: The method is tested on multiple datasets and model types, with comparisons to various baselines.\n- Practical impact: The approach significantly reduces computational cost while maintaining or improving explanation quality.\n- Flexibility: The method can be applied to different types of feature attribution techniques and model architectures.", "weaknesses": "- Limited exploration of very large models: While the method is tested on language models, it's not clear how well it scales to extremely large models (e.g., GPT-3 scale).\n- Dependence on amortized explainers: The method's effectiveness relies on the quality of the underlying amortized explainer.\n- Computational overhead: While more efficient than full Monte Carlo methods, the selective approach still requires additional computation compared to pure amortized methods.\n- Sensitivity to hyperparameters: The impact of various hyperparameters (e.g., uncertainty thresholds, combination function parameters) is not thoroughly explored.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We appreciate that you found our work novel. We also appreciate that you found our evaluation comprehensive and our method flexible. We answer your questions and comments next.\n\n**Q1)  “How does the performance of selective explanations scale with extremely large models (e.g., models with billions of parameters)?”**\n\nOur results indicate that Selective explanations scale well to very large models. We observed a better performance improvement when using Selective Explanation for larger models - the language models for text classification (Figures 3 and Figure 4 (c) and (d)). This is because we use the higher quality embeddings of these larger models to train the amortized explainer and the uncertainty metrics. \n\n**Q2) “Have you explored using more advanced uncertainty estimation techniques, such as those based on Bayesian neural networks?”**\n\nThis is an excellent suggestion. We agree with you that other techniques for uncertainty quantification such as those based on Bayesian neural networks could potentially improve uncertainty estimation by providing more calibrated uncertainty estimates. In fact, our Deep uncertainty metric (Eq. 4) is directly inspired by this approach. However, Bayesian methods induce a higher cost that may be prohibitive in some applications. For example using ensembles of models as in deep ensembles [1] requires retraining the same model pipeline multiple times – a limitation we also observe in our proposed Deep uncertainty metric (Eq. 4). Moreover, our ultimate goal is to select the examples that receive higher (lower) quality explanations relative to a computationally-expensive-to-obtain method such as SHAP. In Learned uncertainty we “learn” from data which points will receive higher (lower) quality explanations instead of using a Bayesian approach. This leads to a method that is optimized to achieve our goal because it is trained to predict the examples with higher MSE relative to the high-quality explanations and decrease the computational overhead by only training a model once.\n\n[1] Lakshminarayanan et al. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. NIPS 2017.\n\n\n**Q3) “How sensitive is the method to the choice of amortized explainer? How might it perform with different types of amortized explainers?”**\n\nThe main objective of selective explanations is to improve the quality of the explanations provided by both amortized and Monte Carlo explainers by combining both methods when the amortized explainer fails. Therefore, in the case that the amortized explainer has poor performance, selective explanations would require more usage of the Monte Carlo explainer to achieve favorable performance (i.e., small MSE from high-quality explanations), thus increasing the computational cost per explanation. Hence, the choice of amortized explainer heavily impacts the quality vs. computational cost of selective explanations. \n\n**Q4) “Could the selective explanation approach be extended to other types of explanation methods beyond feature attribution (e.g., example-based explanations)?”**\n\nThe core idea of selective explanations is identifying low-quality predicted explanations and using a more computationally expensive method to improve their quality. Therefore, selective explanations could be extended to other explanation types. For example, with example-based explanations, an uncertainty metric could be developed to identify cases where selected examples are likely to be unrepresentative, and thus a different explanation method should be used.\n\n**Q5) “How might the method be adapted to handle streaming data or online learning scenarios where the underlying model is continuously updating?”**\n\nTo handle streaming data or online learning, the components of the selective explanation method would need to be updated incrementally. Specifically, the amortized explainer, the combination function (Eq. 12), and the uncertainty measures (Eq. 4 and 5) would need to be continuously updated to reflect the changes in the model being explained.\n\n**Q6) “Computational overhead: While more efficient than full Monte Carlo methods, the selective approach still requires additional computation compared to pure amortized methods.”**\n\nThe selective explanation method combines amortized and Monte Carlo explanations, introducing an additional computational cost. However, we believe this is a feature instead of a flaw, since it allows flexibility in choosing the fraction of samples that receives extra computations. Figure 5 shows that selective explanations significantly improve the performance of the amortized explainer, especially for the samples with worst-performing explanations. For example, Figure 5 (a) demonstrates that providing explanations with an initial guess to 50% of the data improves the Spearman’s correlation of the worst 10% amortized explanations from 0 (no correlation) to almost 0.6 (strong correlation).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jfFwub7MAi", "reviewer": "Reviewer_xNyJ", "summary": "The paper presents a novel framework for explaining black box models through a selective explainer. The selective explainer unifies two different explanation methods: the former relies on amortized explainers, which are easier to compute but provide lower-quality explanations; the latter can provide higher-quality explanations but is more costly to compute. By identifying when the first method is enough to have a good explanation, the selective explainer can trade off (depending on the user’s needs) when relying on amortized explainers or a more complex explanation method. Experiments on benchmark data show the effectiveness of the approach.", "questions": "Q1 - What is a low-quality explanation under the proposed framework? I think this is the core question underlying the work. However, the authors are not directly addressing this question, as, in my opinion, assessing the low quality of a certain explanation requires defining who is the final user of the explanation.\n\nQ2 -  As far as I know, an unstable explanation might be due to the underlying uncertainty around a prediction, but also due to the approximations required to compute the explanation. Is there a way to disentangle these two kinds of uncertainty? I think this is something that the method should take into consideration.\n\nQ3—As discussed in W2, what the authors call a high-quality explanation seems to be obtained by a more costly (as it relies on some exact computation) explanation method. Is my understanding correct? If so, how do you ensure that the high quality explanation is a good explanation of why a certain prediction has been made?\n\nA few other details:\n\nLine 80: you refer to explanations with initial guesses.  I would personally prefer to have the definition here rather than in the next section. \nLines 152-154: can you clarify and discuss the drawbacks of this choice?", "limitations": "NA", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper considers an interesting problem, i.e. how to combine cheap-to-obtain explanations and expensive-to-obtain ones;\n2. The theoretical analysis seems sound.", "weaknesses": "w1. The methodology is based on never-discussed assumptions, i.e. a more uncertain/less stable explanation is a low-quality explanation. I think the authors should discuss this assumption further. Moreover, I guess such a measure is potentially correlated with whether we explain correct or incorrect ML predictions, which is something that is never discussed/taken into account and lacks appropriate references.\nw2.  The terminology is misleading: the paper calls expensive-to-obtain explanations high-quality explanations. I think this misleads the reader because there is never an actual comparison between expensive explanations and their actual quality in explaining the underlying ML model. \nw3. The experimental evaluation is limited to quantitative measures. I think the paper would greatly benefit from including a user study, as humans must evaluate the quality of explanations provided by the novel selective explainer. For example, [Longo et al., 2024] provide a few reasons why human user studies should always be conducted when considering XAI methods.\nw4. The paper does not take into account relevant related work on uncertainty quantification of explanation methods. For instance, see [Zhao et al, 2021] and [Slack et al., 2021].\n\n[Longo et al., 2024 ] Longo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Del Ser, J., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A. and Jiang, R., 2024. Explainable artificial intelligence (XAI) 2.0: A manifesto of open challenges and interdisciplinary research directions. Information Fusion, p.102301.\n[Slack et al, 2021] Dylan Slack, Anna Hilgard, Sameer Singh, Himabindu Lakkaraju:\nReliable Post hoc Explanations: Modeling Uncertainty in Explainability. NeurIPS 2021: 9391-9404\n[Zhao et al., 2021] Xingyu Zhao, Wei Huang, Xiaowei Huang, Valentin Robu, David Flynn:\nBayLIME: Bayesian local interpretable model-agnostic explanations. UAI 2021: 887-896", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review! It was really insightful and will definitely improve the final version of our paper. We will implement your suggested changes, and hope you engage with us during the discussion period. We address your comments below.\n\n**Q1) “The terminology is misleading: the paper calls expensive-to-obtain explanations high-quality explanations.”**\n\nThis is a great point. We do agree with you that expensive-to-obtain explanations is a more precise term and will change “high-quality” to “expensive-to-obtain” everywhere in the revised version of our paper. Throughout the paper we assume that “high-quality” explanations are the ones computed until convergence – for example, SHAP with exponentially many inferences.\nThese computationally expensive SHAP explanations have been extensively tested both quantitatively and via user studies [1, 2] – although SHAP does have limitations as well [3]. We will add the following paragraph on the quality of expensive-to-obtain explanations to the limitations section:\n\n“Selective explanations aim to approximate expensive-to-obtain explanation methods such as SHAP with an exponential number of inferences. Therefore, its efficacy is limited to the performance of the explanation method being approximated. The general methodology behind Selective Explanations is flexible, and can be applied to approximate other expensive-to-obtain feature attribution methods. We encourage the reader to use Selective Explanations to approximate the computationally expensive explanation method that best fits their application, noting that, ultimately, the quality of explanation will be limited by the chosen approximand.”\n\n**Q2) “The methodology is based on never-discussed assumptions, i.e. a more uncertain/less stable explanation is a low-quality explanation.”**\n\nThank you for raising this important point. We address it in the global response to all reviewers due to its importance, and we kindly ask you to refer to the text above (see answer to Q2 above). TL;DR: Figure 3 illustrates the relationship between uncertainty and explanation quality, showing that explanations with the lowest uncertainty also have the lowest MSE (and vice-versa), where MSE is with respect to \"higher-quality explanations\" (SHAP computed until convergence). We agree that this correlation would be better supported by more direct evidence. To address this, we will add a new Table 1 (see attached rebuttal PDF) to the main paper, which presents Pearson and Spearman correlations between our uncertainty measures and MSE. This table provides evidence of strong positive correlations and monotonic relationships between MSE and uncertainty estimates, particularly for language datasets. The monotonic relationship underlies our approach's effectiveness in identifying lower-quality explanations by using uncertainty estimates. For more details, please refer to the global answer (Q2).\n\n**Q3) “The experimental evaluation is limited to quantitative measures.”**\n\nThe main reason for the lack of user evaluation is that our main goal is to approximate widely-studied yet expensive-to-obtain explanation methods such as LIME (with a large number of model inferences) and SHAP (computed until convergence) [1, 2]. Given their established results, we follow the methodology in recent papers focused on accelerating attribution methods [5 – 8] and report both MSE and Spearman’s Correlation to these expensive-to-obtain explanations. When MSE is low and Spearman Correlation is high with converged SHAP explanations (which our Selective Explanation method achieves), the performance of our method is essentially the same as SHAP.\n\n**Q4) “The paper does not take into account relevant work on uncertainty quantification of explanation methods. For instance, see [Zhao et al, 2021] and [Slack et al., 2021].”**\n\nThanks for suggesting these papers. We'll discuss them in our related work section. Our approach differs in two key ways from the mentioned papers:\n\n1. These papers study uncertainty for SHAP and LIME variants. We calculate uncertainty for the amortized explainer - a model that takes data points as input and directly outputs explanations with only one inference.\n\n2. Main contribution: Our focus isn't on the uncertainty measures themselves; they only serve the intermediate purpose of finding amortized explanations that severely differ from expensive-to-obtain explanations.\n\n**Q5)  “As far as I know, an unstable explanation might be due to the underlying uncertainty around a prediction, but also due to the approximations required to compute the explanation. Is there a way to disentangle these two kinds of uncertainty?”**\n\nWe also believe that prediction uncertainty and approximations are sources of unstable explanations. However, we are computing the uncertainty for the amortized explainer, not the black-box model being explained ($h$ in our notation). Our method prioritizes practical improvement of the approximation of expensive-to-obtain explanations over the disentanglement of uncertainty sources – which we believe is an important direction of future work.\n\n[1] Lundberg et al. A Unified Approach to Interpreting Model Predictions. NIPS 2017.\n\n[2] Antwarg et al. Explaining Anomalies Detected by Autoencoders Using SHAP. Expert Syst. Appl. Vol 186.\n\n[3] Slack et al. Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods. AIES 2020.\n\n[4] Ribeiro et al. \"Why Should I Trust You?\": Explaining the Predictions of Any Classifier. KDD 2016.\n\n[5] Jethani et al. FastSHAP: Real-Time Shapley Value Estimation. ICLR 2022.\n\n[6] Schwab and Karlen. CXPlain: Causal Explanations for Model Interpretation under Uncertainty. NeurIPS 2019.\n\n[7] Yang et al. Efficient Shapley Values Estimation by Amortization for Text Classification. ACL 2023.\n\n[8] Covert et al. Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attribution. arXiv:2401.15866.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
### Global Rebuttal

Thank you very much to the reviewers for their effort! We are pleased that you found the paper well-written, the problem setting interesting, and our theoretical analysis sound (all reviewers), recognized that we are the first to propose "selective" feature attribution (reviewers kYjk and tCbB), and that our method is intuitive and easy to understand (reviewer XcVA).
 
Next, we answer two questions that were common across reviewers regarding computational cost and the relationship between MSE and uncertainty metrics.

**Q1) What is the tradeoff between explanation quality and computational cost?**

We thank reviewers XcVA and kYJk for bringing this up. The trade-off between computational cost (given by number of inferences) and quality of explanations (MSE w.r.t. a converged SHAP explanation) is captured in Fig. 4 and 10 of the paper. In particular, Fig. 10 (replicated in the rebuttal PDF) shows that the Selective Explanation method's computational cost is close to an oracle that "knows" which explainer to use for a given MSE. We will bring Fig. 10 and the associated discussion into the main body of the paper. We give details next.

1. Fig. 10 compares Selective Explanations with Monte Carlo methods in terms of computational cost (x-axis: number of model inferences) and explanation quality (y-axis: MSE w.r.t. high-quality explanations). The number of inferences serves as a proxy for computational cost, as GPU/CPU usage time scales linearly with it. Our results demonstrate that Selective Explanations achieve lower MSE than Monte Carlo methods for the same computational cost.

In our Selective Explanation method, we predict which samples should be routed to a more computationally expensive explanation method. We compare this method against an "oracle" who knows a priori how to optimally route samples in terms of MSE. We simulate this oracle by pre-computing SVS explanations with parameters 12, 25, and 50, and selecting the one with the smallest MSE from the target SHAP explanation. The oracle is simulated for comparison purposes only. Remarkably, Fig. 10 shows that selective explanations closely approximate the Oracle curve, indicating that, on these benchmarks, our method has a near-oracle trade-off between the number of inferences and MSE.

2. Fig. 4 also compares computational cost (percentile with recourse ($1 - \alpha$) in x-axis) and explanation quality (MSE from converged SHAP explanation, y-axis). Here, percentile with recourse is a proxy for computational cost because it controls the fraction of the dataset that will receive more computationally-expensive explanations, i.e., Monte Carlo or explanations with initial guess. These explanation methods are more costly than amortized methods because they require a larger number of inferences while the amortized explainer only requires one inference.

Fig. 4 demonstrates that Selective Explanations achieve lower MSE than Monte Carlo methods using the same number of inferences. In Fig. 4 (a) and (c), Selective Explanations can match the performance of the more expensive method while using only 50\% of the computational resources. Fig. 4 (b) and (d) show an even stronger result: Selective Explanations improve upon amortized explanations even when the more computationally expensive method has a higher MSE than the amortized method. This improvement is particularly significant for the lower-quality amortized explanations, as illustrated in Fig. 5.

**Q2) What is the relationship between our uncertainty metrics and explanation quality?**

We thank reviewers XcVA and xNyJ for this question. Figure 3 shows the relationship between our proposed uncertainty metrics (deep and learned uncertainty) and explanation quality (measured by MSE). MSE serves as a proxy for explanation quality as it measures the difference from SHAP explanations computed until convergence. Additionally, we included a new Table 1 (see attached PDF) showing Pearson’s (linear) and Spearman’s (monotonic) correlation measures between the uncertainty metrics and MSE, which will be added to the paper. These results show that our uncertainty metrics are consistently positively correlated with explanation quality across different tasks.

1. In Fig. 3, the x-axis represents coverage ($\alpha$), the fraction of examples with the lowest predicted uncertainty, while the y-axis shows the average MSE for these examples. For instance, with $\alpha = 25$\%, the y-axis shows the average MSE for the 25\% of examples with the lowest predicted uncertainty.

Fig. 3 indicates that examples with higher uncertainty metrics also have higher MSE because as coverage increases, the MSE of the selected explanations also increases. We compare our method to an "Oracle" that has access to the MSE of the amortized explanations and, when the coverage decreases, removes exactly those explanations with the highest MSE. Our approach closely matches the Oracle for both language tasks, Yelp Review and Toxigen datasets, demonstrating that our uncertainty measures are nearly optimal in these cases.

2. Table 1 (in the attached PDF) shows both Pearson and Spearman’s correlation coefficients between our uncertainty measures (Deep and Learned) and the explanation quality (measured by MSE) across the datasets. The table shows that our uncertainty measures achieve positive correlations across all datasets and uncertainty methods, indicating that as uncertainty increases, so does the MSE (lower explanation quality). We also observe strong correlations in many cases, particularly for the Learned uncertainty method on the Toxigen dataset (Pearson: 0.89, Spearman: 0.93). Moreover, Spearman’s correlation is constantly higher than Pearson’s, suggesting a strong monotonic relationship between uncertainty and explanation quality – which is our main interest since we aim to detect and rank lower-quality explanations.

Meta-review: The paper presents a novel framework for a more flexible way of feature attribution in black box models, called the "selective explanations". The selective explainer tries to bridge two different explanation methods -- a) amortized explainers that are easier to compute but of lower quality, and b) the traditional higher-quality explanations that are costly to compute. The new method works simply by identifying when a) generates low-quality explanations, and improving them using an "explanations with initial guess" technique. In the end, the method allows a trade-off between computation speed and explanation quality. 

The reviewers shared a similar set of questions/concerns: 1) explanation and computation tradeoff is missing; 2) unvalidated so-called "high-quality" explanations (which in the paper are interchangeably with "expensive" explanations; 3) the relationship between our uncertainty metrics and explanation quality is not discussed. For 1) there was a figure in the appendix that the authors plan to move to main text. For 2) the authors will stop suing the two terms interchangeably. But in my opinion the well established expensive methods such as SHAP is generally acknowledged to be high quality. A clarification in the text that this is what they regard as high-quality would be great. For 3), the authors added further experiments, which to me successfully addressed the concern. 

To me, the initial draft was missing some important discussions, and thanks to the reviewers, the revised version is much better. However, two of the reviewers rating negatively seem to have an anchoring bias and refuse to update their scores, or engage further. To my eyes, the updated draft (with the promised changes) does meet the standard of NeurIPS acceptance.
Decision: Accept (poster)