An Additive Instance-Wise Approach to Multi-class Model Interpretation
Authors: Vy Vo
Van Nguyen
Trung Le
Quan Hung Tran
Gholamreza Haffari
Seyit Camtepe
Dinh Phung
Venue: ICLR 2023 poster (2023)

Abstract: Interpretable machine learning offers insights into what factors drive a certain prediction of a black-box system. A large number of interpreting methods focus on identifying explanatory input features, which generally fall into two main categories: attribution and selection. A popular attribution-based approach is to exploit local neighborhoods for learning instance-specific explainers in an additive manner. The process is thus inefficient and susceptible to poorly-conditioned samples. Meanwhile, many selection-based methods directly optimize local feature distributions in an instance-wise training framework, thereby being capable of leveraging global information from other inputs. However, they can only interpret single-class predictions and many suffer from inconsistency across different settings, due to a strict reliance on a pre-defined number of features selected. This work exploits the strengths of both methods and proposes a framework for learning local explanations simultaneously for multiple target classes. Our model explainer significantly outperforms additive and instance-wise counterparts on faithfulness with more compact and comprehensible explanations. We also demonstrate the capacity to select stable and important features through extensive experiments on various data sets and black-box model architectures.

Published as a conference paper at ICLR 2023
AN ADDITIVE WISSE APPROACH TO MULTI- CLASS MODEL INTERPRETATION
Vy Vo" Van Nguyen' Trung Le
Quan Hung Tran Gholamreza HafTari'
Seyit Camtepe
Dinh Phung
1 Monash University, Australia
Adobe Research, USA
3 CCIIR''s Dataól, Australia
VinAl Research, Vietnam
ABSTRACT
Interpretable machine learning offers insights into what factors drive a certain prediction of a black-box system. A large number of interpreting methods fo- cus on identifying explanatory input features, which generally fall into two main categories: attribution and selection. popular attribution- -aaedd approach is exploit local neighborhoods for learning -ppeinc explainers in an additive manner. The process thus inefficient and susceptible to poorly-conditioned sam- ples. Meanwhile, many selection-based methods directly optimize local feature distributions in an aistance-wise training framework, thereby being capable leveraging global information from other inputs. However, they only interpret single-class predictions and many suffer from inconsistency across different set- tings, due to a striet reliance pre-defined number of features selected. This work exploits the strengths of both methods and proposes a framework for learning local explanations simultaneously for multiple target classes. Our model explainer significantly outperforms additive and instance-wise counterparts on faithfulness with more compact and comprehensible explanations. We also demonstrate the capacity to select stable and important features through extensive experiments on various data sets and black-bor model architectures.
INTRODUCTION
Black-box machine learning systems enjoy remarkable predictive performance at the cost of inter- pretability. This trade-off has motivated a number of interpreting approaches for explaining the behav- ior of these complex models. Such explanations are particularly useful for high- stakes applications such healthcare (Caruana et al. 2015 Rich 2016), cybersecurity (Nguyen al. 102T) or criminal investigation (Lipton 2018). While model interpretation can be done in various ways (Mothilal et al. 2020 Bodria al. 2021), discussion will focus feature importance saliency-based approach that is, to assign relative importance weights to individual features w.r.t the model's prediction on input example. Features here refer input components interpretable to humans; for high-dimensional data such as texts or images, features can be a bag of words/phrases or a group of pixels/super-pixet (Ribeiro al. 2016). Explanations are generally made by selecting top K features with the highest weights, signifying K most important features to black-box's 's decision. Note that this work tackles feature selection locally for an input data point, instead of generating global explanations for an entire dataset.
abundance of interpreting works follows the removal-baue based explanation approach (Covert et al. 2021), which quantifies the importance of features by removing them from the model. Based on how feature influence summarized into an explanation, methods in this line of works can be broadly categorized as feature attribution and feature selection. In general, attribution methods produce relative importance scores to each feature, whereas selection methods directly identify the subset of features most relevant to the model behavior being explained. One popular approach to learn attribution is through an Additive model (Ribeiro et al., 2016)] Zafar & Khan
2019: Zhao
et al. 2021). The underlying principle is originally proposed by LIME (Ribeiro al. 2016) which

---

Published as a conference paper at ICLR 2023
learns regularized linear model for each input example wherein each coefficient represents feature importance scores. LIME explainer takes the form of a linear model w. 2 where & denotes neighboring examples sampled heuristically around the input]] Though highly interpretable themselves, additive methods are inefficient since they optimize individual explainers for every input. As opposed to the instance-specific nature of the additive model, most of the feature selection methods
developed instance-wisely (Chen et al., 2018 Bang et al.) 2021; Yoon et al., 2019 Jethani et al. 2222la). Instance-wise frameworks entail global training of model approximating the local distributions over subsets of input features. Post-hoc explanations can thus be obtained simultaneously for multiple instances.
Contributions. In this work, we propose a novel strategy integrating both approaches into an additive instance-wise framework that simultaneously tackles all issues discussed above. The framework consists of 2 main components: an explainer and a feature selector. The explainer first learns the local attributions of features across the space of the response variable via multi-class explanation module denoted as W. This module interacts with the input vector in an additive manner forming a linear classifier locally approximating the black-box decision. To support the learning of local explanations, the feature selector constructs local distributions that can generate high-quality neighboring samples on which the explainer can be trained effectively. Both components are jointly optimized via backpropagation. Unlike such works (Chen et al., 2018 [Bang et al. [202T) that sensitive to the choice of K as a hyper-paramen parameter, our learning process eliminates this reliance (See Appendix G for detailed analysis on why this is necessary).
Our contributions are summarized as follows
We introduce AIM- an Additive Instance masse approach to Multi-class model interpretation. Our flexibility
model explainer inherits merits from both families of methods: model-agnosticisst while supporting efficient interpretation for multiple decision classes. To the best of our knowledge, we are the first to integrate additive and instance-wise approaches into an end-to-end amortized framework that produces such multi-class explanation facility.
Our model explainer is shown produce remarkably faithful explanations of high quality and compactness. Through quantitative and human assessment results, we achieve superior performance over the baselines on different datasets and architectures of the black-box model.
RELATED WORK
Early interpreting methods are gradient-basse in which gradient values are used to estimate attribution scores, which quantifies how much a change in an input feature affects the black-box's 's prediction in infinitesimal regions around the input. It originally involves back-propagation for calculating the gradients of the output neuron w.r.t the input features (Simonyan et al.) 2014). This early approach however suffers from vanishing gradients during the backward pass through ReLU layers that can downgrade important features. Several methods are proposed to improve the propagation rule (Bach) jet al., 2015; Springenberg et al.] 20114; Shrikumar et al. 2017, (Sundararaaan et al. 2017). Since explanations based on raw gradients tend to be noisy highlighting meaningless variations, a refined approach is sampling- bampling-based based gradient. in which sampling is done according to a prior distribution for computing the gradients of probability Bachrens et al. (2010) expectation function (Smilkov et al.) 2017 Adebayo et al. 2018). Functional Information (FI) (Gat et al.] 2202)) is the state-of-the-art in this line of research applying functional entropy compute feature attributions. FI is shown to work on auditory, visual and textual modalities, whereas most of the previous gradient- methods are
solely applicable to images.
A burgeoning body of works in recent years can be broadly categorized as removal-based explanation (Covrrt al. (02TT. Common removal techniques include replacing feature values with neutral or user-defined values such as zero or Gaussian noises (Zeiler & Fergus 2014 Dabkowski & Gal 2017, Fong & Vedaldi [2017] Petsiuk et al. [0188 Fong et 2019), marginalization of distributions over input features (Lundberg & |ee]]201 7
Covert et al. 2020 Datta et al., 2016). or substituting held-out feature values with samples from the same distribution (Ribeiro et al. 2016). The output explanations are often either attribution-bant
2017
or selection-based In addition to additive models
is binary representation vector of an input indicating the presence/absence of features. The dot-product operation is equivalent summing up feature weights given by the weight vector w, giving rise additivity.
2

---

Published as a conference paper at ICLR 2023
that estimate feature important via the coefficients of the linear model, feature attributions can be calculated using Shapley values (Datta et al.] 2016 Lundberg & Lee 2017 Covert et al. 2020) or directly obtained by measuring the changes in the predictive probabilities or prediction losses when adding or excluding certain features (Zeiler & Fergus 2014 Schwab & Karlen 2019). On the other hand, selection based works straightforwardly determine which subset of features
important or unimportant to the model behavior under analysis (Chen et al. 20118) Yoon et al. 2019; Bang et al.) Jethani et al. 2021a Nguyen et al. 2021 2022). Explanations are made by either selecting features with the highest logit scores obtained from the learned feature distribution, or specifying a threshold between 0 and 1 to decide on the most probable features. Most selection methods adopt amortized optimization, thus post-hoc inference of features for multiple inputs can be done very efficiently. In contrast, attribution-
approaches are mostly less efficient since they process input examples individually. There have been methods focusing improving the computational cost of these models (Dabkowski & Gal. 2017
Schwab & Karlen 2019; Jethani et al. (021b).
Recently, there is an emerging interest in integrating the instance-wise property into an additive framework to better exploit global information. For a given input, Plumb et al. (2018); Yoon et al. (2022) in particular learn surrogate model assigning weights to training examples such that those more similar or relevant to the input are given higher weights. A locally interpretable model (that is often LIME-based) is subsequently trained on these samples return feature attributions. AAgaraal let al.] (202T) further proposes a neural additive framework that constructs an explainer in a form of a linear combination of neural networks. Despite their potential, these methods have only been reported to work on tabular data.
3 PROPOSED METHOD
3.1 PROBLEM SETUP
In the scope of this paper, we limit the current discussion classification problems. Consider data set of pairs (X, Y) where X ~ (- is the input random variable and Y is characterized by the conditional distribution m (Y) |X) obtained
the predictions of a pre-trained black-box classifier for the response variable. The notation m stands for model, indicating the predictive distribution of the black-box model, to be differentiated from the ground- -ruth distribution. We denote E Rd as an input realization with d interpretable features and predicted label Y c (1,.... C}. Given an input ar, we obtain the hard prediction from the black-box model as y/m = argmax, P.. (Y e X .)).
While earlier methods generate a single d dimenssiia weight vector w. assigning the importance weights to each feature, we define an explainer& & : ıR + _dxC mapping an input æ to a weight matrix W. E dx with the entry W representing the relative weights of the ith feature of æ to the predicted label j E {1,...,Cr C}.
E(x) W..
Given a training batch, LIME (Ribeiro et al. 2016 in particular trains separate explainers for every input, thus cannot take advantage of the global information from the entire dataset. In line with the instance-wise motivation, our explainer & is trained globally over all training examples to produce local explanations with respect to individual inputs simultaneously, which also seeks to effectively enable global behavior (c.g., two similar instances should have similar explanations). As E is expected to be locally faithful to the black-box model, we optimize on the local neighborhood around the input x. This region is constructed via feature selection module. We now explain how this is done.
3.2 TRAINING OBJECTIVES
Let eeze E (0, 1]d be a random variable with the entry z' 1 indicating the feature ith is important to the black-box's predictions. With respect to employ selector S RR + [0, 1]d that outputs S(x) = ue such that T := P(zz = 1 X = x), = 1,.... d.
Through the probability vector ææ. the selector helps define local distribution on a local space of samples zω @ x with Xe ~ MultiBernoulli(T.. (=a) and element-wise product O. The selector S is also a learnable module, and we want it to generate well-behaved local samples that focus more valuable featurewlatributiions of x. Intuitively, if the feature i of æ contributes more to the predictions of the black-box model, i.e., z R 1, the explainer is expected to give higher assignments to the row
3

---

Published as a conference paper at ICLR 2023
vector W To mimic how the black-box model behaves towards different attributions, propose to minimize the cross-entropy loss between the prediction of the black-box model on local examples
Zn O x and the prediction of the explainer binary vectors 2n via the weight matrix Wa as
L Eg CE (im. softmmx W z0))
where CE is the cross-entropy function and Um = argmax, (Y = 2 2).
To make the process continuous and differentiable for training, the temperature-depente
Gumbel-
Softmax trick (Jang et al.) 2016 Maddison et al. (0166) is applied for relaxing Bernoulli variables
22 In particular, the continuous representation z is sampled from the Concrete distribution as
[2 2ie Concrete )²)):
expl log T2 + Ga)/T}
(log( T + TT) + e Tú. + Ga)/T)}
with temperature т, random noises G3 and GN1 independently drawn from Gumbel distribution G log( log ut), u, ~ Uniform(0,1) 1).
Given the corresponding prediction Im = argmax, (Y c žω ② x)) L1 now becomes
C Eg
CE (jms softmax W 22)
Since zu is binary vector indicating the absence/presence of features, 2 C indeed acts local perturbation, which generally concurs with the principle of LIME model. However, different from LIME, we amortize the explainer & to produce the weight matrices W, locally approximating the black-box model with linear classifiers operating on input neighborhoods. Furthermore, we replace LIME's uniform sampling strategy with learnable local distribution offered by the selector S.
We argue that heuristic sampling is inadequate for our purpose. As d gets large, realizing the space of 2d possible binary patterns infeasible. Given the fact that the number of binary patterns that actually approximate the original prediction is arbitrarily small, it is thus very difficult for such simple linear separator as one used in LIME learn useful patterns within finite sampling rounds. While diversity in these samples is desirable for learning attributions for individual decision classes, we also want the explainer encourage the selector to yield more of the samples that contain the features that best approximate the model behavior on the original input, we propose the following information -thoorrtttc approach.
focus more on relevant features the original prediction you To
Let æs denote the sub-vector formed by the subset of K most important features $ (i..... ig C {l....,d) dd (i: < i2 <... < ig). Thus, The can now viewed as the probability that the ith feature of x appears in S. Given random vector Xs R we maximize the mutual information.
P (Y Xs) P.m
T(xs: Y)
log
log
Ex ESJ Eylxs [og P..
Xs) + Constant.
Based on the following inequality, we can obtain variational lower bound for [[Xg;; Y) via generic choice of conditional distribution Qs(YY) Xs)
Exsxs [log Pm (Y Xx)] Eylxs [log Qs (Y) Xx)] + KL (Pm (Y Xx), Qs (Y Xx))
Eyjx, [log Os (Y) Xg)],
where KL represents the Hillbacck-eibble divergence.
worth noting that the purpose of using the mutual information L2X (Chen et al. 2018) and our AIM framework are different. L2X uses the mutual information to directly select valuable features/attriuttions. Meanwhile, in our work, the role of the selector is to balance exploration with exploitation. Stochastic sampling yields various examples that produce different predictions, and E exploits such variation to learn feature attributions w.r.t multiple classes. Simultancously, maximizing [[X;; Y) encourages the selector S to produce meaningful probability vector focusing more
4

---

Published as a conference paper at ICLR 2023
the selected subset of attributions that can well approximate the full full-input decision. In Appendix we show that an explainer with learnable local distributions performs significantly better than one optimized on heuristic examples.
Maximizing the mutual information in Eq.
(3) can therefore be relaxed to maximizing the variational
lower bound Exxaexx By log Q
Xs) We parametrize Q with a function approximator
the element-wiss product Σnm to approximate 2g. a: contains discrete features (e.g., words), we embed feature (e.g., a selected word) in with learnable embedding vector, wherein feature not in $ is replaced with vector. With the prediction = argmax. Pm (Y = e æ), our second objective is given as
such that Q2(2s)) ( Notice that
C2 Eg CE (yms ( x))}
The final objective. We parametrize & S and 9 with three neural networks of appropriate capacity. All networks ε, S and 9 are jointly optimized over total parameters and globally on the training set. We further introduce regularization term over W encourage sparsity and accordingly compact explanations. The final objective function is now given as
ming [ + L2 + 8 E
[2.1]
(5)
where || ||2,1 is the group norm 2, 1, and ,3 are balancing coefficients loss terms. and are subject to tuning since highly compressed representation can cause information loss and harm faithfulness. Figure summarizes our framework as follows
(em.culic..))
Binary samplez,
W 3
hm
Input
x. 1Rd
Selector
Concrete sampler
Softmax ø
Black Box
Approximator
a(W 2,)
8m
Ym
G(2 xx)
Explainer
Mark W
(rowe) (cokumas)
Relaved
l
fg
CR Dmn a (W 2,)]
L CE Dim GG, xx]
Figure 1: An illustration of AIM pipeline. Left: Given input æ. the explainer & produces local multi-class explanation module W. in which each entry W representing the relative weight the ith feature of to the predicted label j {1,..., C}. & is optimized on local space of perturbations around æ. Such a space is constructed via the feature selector that is simultaneously optimized to generate high-quality local distribution containing well-behaved neighboring samples. The binary sample z ~ Multi-Bernoulli/ TT)) is passed through Gumbel-Softmay sampler for relaxation. We end up with the explanation matrix W. = and relaxed samples Σn Right: The figure illustrates how these output components interact with each other and the input to form the first and second loss objectives given in Eq. and (4). The final objective in Eq. (5) combines L1 and Le with an additional sparsity term to induce compactness. CE is the cross-entropy function. Ø, ', and a denote the element-wise product, inner product and softmax operation respectively.
3.3 INFERENCE
standard inference strategy is to choose top K features with the highest weights, with K determined in advance. In framework, the explainer outputs weight matrix W., size d x C' (recall that d is the number features and the number of target classes). We obtain the black-bon label j Vm argmax,I,,, (Y weight vector. Features can then be derived accordingly. Though it is intuitive to use n. directly for the explanation, doing this may require specifying certain threshold 0 E [0: Since मæ represents
predicted X x)) and select the corresponding column W as the
5

---

Published as a conference paper at ICLR 2023
local distributions, choosing the thresholds individually for each input is daunting while setting global threshold for all imputs is sub-optimal. Moreover, the selection of an ith feature using Tee (i.e., Th 8) is independent for each feature, so when combined, they do not guarantee the resulting subsets of features can well approximate the black-box's ' decisions. On the other hand, the explainer looks into input features all-in-once settle with good subsets of features. Appendix Appendix[pprovieeee provides evidence that inference according to Wa is the optimal strategy.
4 EXPERIMENTS
We conducted experiments on various machine learning classification tasks. In the main paper, we focus on NLP classifiers since we believe text data is the most challenging modality. In the following. we discuss the experimental design for textual data.
The Large Movie Review Dataset IMDB (Maas et al. 2011 consists of 50, 000 movie reviews with positive and negative sentiments. The black-box classifier is a bidirectional GRU (Chen al. 2018) that achieves 85.4% test accuracy.
Sentiment Analysis:
Hate Speech Detection: HateXplain is an annotated dataset of Twitter and Gab posts for hate speech detection (Mathew et al., 202T). The task to classify a post either be normal or to contain hate/offensive speech. The black-box model is bidirectional LSTM (Gers et al., 2000) stacked under a standard Transformer encoder layer (Vaswani et al. 2017
attention heads.
The best test accuracy obtained is 69.6%.
Topic Classification: AG is collection of more than 1 million news articles. AG News corpus (Zhang et al., 2015) constructed by selecting largest classes from the original dataset: World, Sports. Business, and Sci/Tech. We train word-level convolution neural network (CNN) (LeCun et al. 1995 as black-box model. obtains 89.7% accuracy on the test set.
See Appendix A for additional details on our experimental setup and model design. Appendix E| further demonstrates the remarkable capability of AIM for generalizing on images and tabular data. Code and data for reproducing our experiments are published at /tttps://gitubb com/
isVY08/AIM/
4.1 PERFORMANCE METRICS & BASELINE METHODS
The task of saliency- -ased explainer is to find the subset of input features S that best mimics the black-box's 's predictions on the original input. Following the suggestions from Robnik-Sikonja & Bohanec| (2018) on desiderata of explanations and related works (Ribeiro et al. 2016) Chen et al. 2018:
Schwab Schwwbchhwbb quantitative evaluation of word- explanations. See Appendix B for implementation details.
& Karlen 2019 Situ al.) 1222 Gat et al. 2022). Table presents the metrics for
For text classification tasks, compare our method against baselines that have done extensive experiments on textual data: L2X (Chen et al.) 2018), LIME (Ribeiro et al., 2016), VIBI (Bang et al.. 2021) and FI (Gat al., 2022). Regarding model architectures, note that AIM has been intentionally designed to match those of L2X and VIBI to assure fair comparison. For each method, we tune the remaining byper-paramettes over wide range of settings and report the results for which Faithfulness is highest (See Appendix
4.2 RESULTS
We compare the performance of methods by assessing the extent to which the set of 10 best features satisfies the criteria discussed in Table I Except for AIM and FI that do not treat K as hyper- parameter, all the other baselines are optimized at K = 10. Table rreportss the average results over model initializations. We here show that our method AIM consistently outperforms the baselines on all metrics while achieving a remarkably high level of faithfulness of over 90% across datasets. AIM effectively approximates the black-box predictions with only 10 features, which demonstrates the sufficiency of the selected feature sets. Given the vast combinatorial space of possible subsets of
VVBI (Bang et al. 2021 measures fidelity via the prediction accuracy of the approximator model, whereas conduct a host-hoc comparison of the black-hox's predictions on the original and masked input.

---

Published as a conference paper at ICLR 2023
Table I: Description of quantitative evaluation metrics.
Property
Definition
Metrie
Description
Hidelity
How well does the explanation approximati the prediction of the black box model?
Faithfulness
Degree of agreement between the prodiction given the full document and the prodiction given the sclected words in 3. A higher value meuns the explanations are strongly black-box's prediction.
Post-agc
Accurac
relevant
Brevity
How conciso is the explanaion?
Brevity
Number of clusters of duplicates or semantically related words formed over S. A lower value means the takens are semantically polarizing and compact.
Comprehensiiility
How well do humans understand the explanation?
Purity
Proportion of stopwords and punctuation included in S. lower proportion is equivalent more meaningful feature
Stability
How similor are the explaration for similat instanees?
Interscction Union (loli)
Praportion of overlapping words in the explanations of two similar documents. We expect the selected features for two such examples overlap in great quanity.
Degrec of importance
How well does the explatation reflect the importance of features parts the explanation?
Positive A
Difference in the confidence of the black-box model in prediction before and after masking importunt words given S. A higher value indicates S contains important features.
log-odds
Degres importance
How does explanation reflect the importance of features parts the explanation?
Negative log-odkts
Difference the confidence of the black-box model in a prediction before and after masking unimportant words ie.. words
lower value indients features contained in S are unimportant.
features, we believe the capacity to efficiently search for a sufficient set of features is what makes AIM stand out from the existing works. Examining A log-odds, it is observed that our top 10 features are deemed more important since removing them causes the largest drops in confidence of the black-box model in the original prediction (on the full document). Given an input containing only important features, interestingly there is even a slight increase in confidence when the black-box models make that prediction. Table 2 also reports the average training time (in minutes) for each method. Since AIM is trained in an instance-wis manner, AIM matches L2X and VIBI in terms of time efficiency, whereas LIME and FI are extraordinarily time-consumin cons-cnnn due the nature of additive models. Learning local explanations instance instance-wisely wisely also enables AIM to leverage global information, thereby supporting stability (via % IoU) better the baselines.
Table 2: Performance of all methods on datasets at K = 10. 1 Higher is better. + Lower is better.
Explainer
AIM (ours)
L2X
LIME
VIBI
IMDB
Purity (%)
0.20
00227
36.55+00.3
30.86+0.20
30.27:+0.86
Brevity
2.48=0.01
2.51
7.73:0.16
3.8611..23
3.66:10.75
Faithfulness (
99.62=0.02
84.80=0.08
79.00:0.21
56.80+0.009
71.70-10.36
1oU (
0.09
4.50+0.14
1.44-1002
0,59+0.01
3.04++0.01
Positive A log-odds 1
7.03
2.92+0.11
2.25:0.18
0.09:10.34
2.38:12.35
Negative log-odds
0.02
2.6320.33
5.7410006
8.4710.36
7.15+11.26
Training time (minutes)
11.19
6.90
551.42
8.48
311.42
HateXplain
Purity (%)
19.78=2.54
21.87:0.14
37.73+0.17
33.91:10.29
33.13:10.09
Brevity
0.21
4.3620.15
7.5910.20
4.56110.28
4.23:10.00
Faithfulness (5)
1.17
75.32=0.03
80.56-20.11
67.2540.29
66 66.28-40.68
IoUl (
0.30
3.42+0074
4.40+0.00
2.37-10.88
3.04:10.06
Positive A log-odds 1
4.28=
2.81
3.1810.11
L4L10.13
1.1110.02
Negative A log-odds J
-.14
1.15=0,27
L077-0.13
2.50.1.11
2.59-10.02
Training time (minutes)
3.13
2.43
222.17
2.15
162.17
AG News
Purity (%)
3.83=0.05
6.64:20.25
29.15::0006
21.15:0.25
27.61:10.03
Brevity
3.09
3.9420.18
8.7620.16
4.07+0.03
4.91-10.00
Faithfulness (
97.92-0.05
90.13+0..26
86.64+0.10
66.58-10.36
76.10-10.11
IoU ?
6.08=0.00
6.01
3.52+:0.2
1.6810.02
3.18:10.03
Positive A log-odds 1
7.14=0.01
4.362:0.88
138800.17
0.03+10.2
0.49-10.22
Negative A log-odds 1
0.00
0.28
0.60=2..1
3.88:10.10
2.24:10.00
Training time (minutes)
11.44
22.08
137.02
15.36
30.22
As K increases, the explamation is expected to be more faithful to the black-box model. Since the mechanism of L2X, VIBI, or LIME requires training new model with the corresponding K, as shown in Figure 2 it may however not guarantee the monotonic behavior. Appendix G analyzes this property in L2X explanations in more detail. We choose to investigate L2X here only since it is the best pessfrerfornnn among the baselines. It is shown that the performance of L2X does not always satisfy monotonicity w.r.t K on newly trained model: different choices of K can yield different
7

---

Published as a conference paper at ICLR 2023
feature rankings e.g., the features picked by model trained top may be considered irrelevant by one trained on top 10. AIM strietly avoids such inconsistency as our framework is not sensitive to K.
Table 3 additionally provides 4 examples of the features chosen by AIM in IMDB dataset. This helps shed light on why the black-box model makes a certain prediction, especially the wrong one. A comprehensive qualitative comparison with the baselines on multiple examples can further be found in Appendix C [ While explanations from additive models (LIME and Fl) are contaminated with a
AIM
LINE
VIBI
100
Falthtamess [N6
larger volume of neutral words, instancewwis methods (L.2X and VIBI) tend to select more meaningful features. AIM stands out with the strongest compactness by picking up all duplicates and synonyms without compromising predictive performance. Note that LIME suffers from low brevity mainly because its algorithm extracts unique words explanations. This also means LIME's feature sets tend to be more diverse than the other methods and thus should be more faithful. Our experiment nevertheless shows that this is not the case.
25
Selected Features DO
Figure 2: Faithfulness of explanation models at different values of K on IMDB dataset.
Table 3: Ground-truth labels and labels predicted by the black-box model on IMDB movie reviews given in the first two columns. 10 most relevant words selected by AIM are highlighted in yellow.
Truth
Model
Key
positive
pasitive
this movie was pleasant surprisee for me. in all bonesty, the previews looked borrible, up until the point where thompson and alan rickman appeared. so rented with reservation, but thoroughly enjoyed this movic, had great acting, a few good plot twrises, and, of counse, emma thompson and alan rickman. it's detinirelly worth checking out.
negative
negotive
movic ever produced. word plat, lettng, special elfects. be prepared if you want to watch this, the only way to get enjoyment out is to light a match and burn the tupe it, knowing it will the hands of any sine person again.
may just be
positive
nogutive
to me, "anatooii"" is certainly one of the better movics i have seen. i don't think "anatomie" was primatily intended to be horrot movic but movic quotiionng the othics of science. you watch with that in mind, it turns into really good film. the only innoying bit was the awfill voice dubbing for the english version. how can you expect any now- german person listen these unbearable perman accents for hours let native english speakers do the talking or use subcitles instead!!
negative
pasitive
have seen this movie several times, it sure is one of the cheapest action flicks the eighties. so, i think many channel when they come across
viewers would definiell "dragon hunt" is made for you. the main characters (the menamara swins) are sporting great moustaches and look so ridiculous their camouflage dresses,
but, if you into great mash,
the best kicking his enemies into sirvam this movic really awful, but then again, is a greal purty tape!
when one of then gets shoe the leg and is still
4.3 HUMAN EVALUATION
We additionally conduct human experiment to evaluate whether the words selected as explanation convey sufficient information about the original document to human users. We ask 30 university students to infer the sentiments of 50 IMDB movie reviews, given only 10 key words obtained from an explainer for each review. To avoid confusion, only examples where the black-box model predicts correctly are considered (See Appendix F the setup). We assess whether the sentiment
Table 4: Human evaluation results on IMDB dataset of AIM, L2X, and LIME.
Explainer
AIM
L,XX
LIME
Human accuracy
90.10%
83.03% 12.22%
84.13%
% Neutral
8.41%
19.22%
inferred by humans is consistent with the actual label of movie review: human accuracy. Some reviews are judged as "neutral / can't 't decide", because the selected key words are neutral, or because positive and negative words are comparable in quantity. We exclude these neutral examples when computing the average accuracy for a participant, but record the proportion of such examples as a proxy measure for purity. The final accuracy is averaged over multiple participants and reported in
All qualitative examples presented in our work are randomly selected from the outputs of the model initialization with the best Faithfulness.
8

---

Published as a conference paper at ICLR 2023
Table 4 It is consistent with quantitative results that explanations from AIM are perceived to be more informative and contain fewer neural features, thus being more comprehensible to human users.
4.4 MULTI-CLASS EXPLANATION
A novel contribution of our work is the capability of simultancously explaining multiple deci- sion classes from a single matrix Wa. Whereas existing methods often require re- re-training or re-optimization to predict different class, our explainer produces class-specific explanations in single forward pass: given a learned W.. select the column j (Wa)) corresponding to the target class to be explained. To the best of our knowledge, are the first propose explanation module with such a facility.
We assess the quality of multi-class explanations via two modifications of Faithfulness and IoU, The former metric Class-specitic Faithfulness measures whether the black-box prediction on the explanations aligns with the class being interpreted. The latter Pairwise IoU evaluates the overlapping ratio of words in the explanations for pair of decision classes. provides the average results for these metries, in comparison with LIME and FI. AIM performs surprisingly well on binary classification tasks with the selected feature sets nearly distinctive to each class i.e., overlapping words account only for less than 4%. Faithfulness 98. 98.09% of on the first class, for example, means that given the explanations, the black-box model predicts label 0 for 98.09% of testing examples. Meanwhile, the performance of LIME and FI seems to be no better than random and sensitive to the distribution of classes in the datasets. However, the task gets more challenging more classes are involved. Since AG News is a dataset of news articles from 4 topics, is sometimes difficult to clearly distinguish text between two classes, which we suspect leads higher overlapping ratio, thereby harming faithfulness. Regardless, the success on IMDB and Xptain demonstrates the potential of supporting counterfactual explanations that seek to determine which features black-box classifier attends to when predicting certain class.
Table 5: Quality of multi-class explanations from AIM, LIME and FI.
Metric
Class-specific Faithfulness (%) f
Pairwise IoU (%)
Target lahel
IMDB
AIM LIME
98.09=00066
98.96=0.05
0.41 00.22
50.02:10.15
50.48-0.16
15.69:10.01 92.34++0.08
FI
86.32-0.229
15.62+0.36
HateXplain
AIM LIME
87.2610.229
88,59 1.88
1.69.0..11
24.3010.12
14.151.0.11
66.91110.02
FI
41.21+0.50
60.35+0.40
76.52+00.05
AG News
AIM LIME
73.88
79.13
54.56 0.18
84.73+ 0.38
9.24 0.03
22.8911.011
26.45=0.06
25. 25.00=0.02
26.71:0.01 26.71=#0001
51.09=0.00
FI
25.07=0.43
25.00=0.44
18-0.0
35.60+0.07
CONCLUSION AND FUTURE WORK
We developed AIM a novel model interpretation framework that integrates local additivity with instance-wise feature selection. The approach focuses on learning attributions across the target output space, based on which to derive important features maximally faithful to the black-box model being explained. We provide empirical evidence further proving the quality of our explanations: compact yet comprehensive, distinctive to each decision class and comprehensible to human users. Exploring causal or counterfactual explanations, especially within our multi-class module is potential research avenue. Though extension to regression problems and other modalities such as audio or graphical data is straightforward, our future work will conduct thorough experiments on these modalities along with comprehensive comparisons with related baselines. Furthermore, our paper currently focuses on word-level explanations, so there is chance of discarding positional phrasal information (e.g.. idioms, phrasal verbs). This can be resolved through chunk-level or sentence-lewe explanations, which will be tackled in future works of ours.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
Trung Le and Dinh Phung were supported by the US Air Force grant FA2236-21-1-40099 Trung Le was also supported by the ECR Seed grant of Faculty of Information Technology, Monash University,
REFERENCES
Mohan S Acharya, Asfia Armaan, and Aneeta S Antony. A comparison of regression models for prediction of graduate admissions. 2019 international conference on computational intelligence in data science (ICCIDS), pp. 1-5. IEEE, 2019.
Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018.
Rishabh Agarwal, Levi Melnick, Nicholas Frosst, Xuezhou Zhang, Ben Lengerich, Rich Caruana, and Geoffrey Hinton. Neural additive models: Interpretable machine learning with neural nets. Advances in Neural Information Processing Systems, 34:4699-4711, 2021.
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojeiech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7):e0130140, 2015.
David Bachrens, Timon Schrocter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus- Robert Müller. How to explain individual classification decisions. The Journal of Machine Learning Research, 11:1803-1834 2010.
Seojin Bang, Pengtao Xie, Heewook Lee, Wei Wu, and Eric Xing. Explaining black-box by using deep variational information bottleneck approach. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 11396-118 1404, 2021.
Steven Bird. Nltk: the natural language toolkit. In Proceedings of the COLING/ACL 2006 Interactive Presentation Sessions, pp. 69-72, 2006.
Francesco Bodria, Fosca Giannotti, Riccardo Guidotti, Francesca Naretto, Dino Pedreschi, and Salvatore Rinzivillo. Benchmarking and survey of explanation methods for black box models. arXiv preprint 1007 2021.
Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 2 1 ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1721-1730, 2015.
Jianbo Chen, Le Song, Martin Wainwright, and Michael Jordan. Learning explain: An information- theoretic perspective on model interpretation. In International Conference on Machine Learning, pp. 883-892. PMLR, 2018.
lan Covert, Scott M Lundberg, and Su-In Lee. Understanding global feature contributions with additive importance measures. Advances in Neural Information Processing Systems, 33:17212- 17223, 2020.
Ian Covert, Scott M Lundberg, and Su-In Lec. Explaining by removing: unified framework for model explanation. J. Mach. Learn. Res., 22:209-11 1, 2021.
Piotr Dabkowski and Yarin Gal. Real time image saliency for black box classifiers. Advances in neural information processing systems, 30, 2017.
Anupam Datta, Shayak Sen, and Yair Zick. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. 2016 IEEE symposium on security and privacy (SP), pp. 598-617. IEFF, 2016.
Ruth Fong, Mandela Patrick. and Andrea Vedaldi. Understanding deep networks via extremal perturbations and smooth masks. Proceedings of the IEEE/CVF international conference on computer vision, pp. 2950-2958, 2019.
10

---

Published as a conference paper at ICLR 2023
Ruth C Fong and Andrea Vedaldi. Interpretable explanations of black boxes by meaningful perturba- tion. In Proceedings of the IEEE international conference on computer vision, pp. 3429-3437 2017.
Itai Gat, Nitay Calderon, Roi Reichart, and Tamir Hazan. A functional information perspective on model interpretation. In International Conference Machine Learning, pp. 7266-7278. PMLR, 2022.
Felix A Gers, Jürgen Schmidhuber, and Fred Cummins. Learning forget: Continual prediction with Istm. Neural computation, 12(10):2451-247) 2000.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterizatioon with gumbel-softmax. arXiv preprint arXiv: 1.Xiv:661... 1.01 144, 2016.
Neil Jethani, Mukund Sudarshan, Yindalon Aphinyanaphongs, and Rajesh Ranganath, Have learned to explain?: How interpretability methods can learn to encode predictions in their inter- pretations. In International Conference on Artificial Intelligence and Statistics, pp. 1459-1467. PMLR, 2021a.
Neil Jethani, Mukund Sudarshan, Ian Connick Covert, Su-In Lee, and Rajesh Ranganath. Fastshap: Real-time shapley value estimation. In International Conference on Learming Representations, 2021b.
Ron Kohavi et al. Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid. In Kdd, volume 96, pp. 202-207, 1996.
Yann LeCun, Yoshua Bengio, et al. Convolutional networks for images, speech, and time series, The handbook of brain theory and neural networks, 3361(10):1995, 1995.
Zachary C Lipton. The mythos of model interpretability: In machine learning, the concept of interpretability both important and slippery. Queue, 16(3):31-57, 2018.
Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017.
Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pp. 142-150, 2011.
Chris Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: continuous relaxation of discrete random variables. arXiv preprint arXiv: 11.07/1111.00122 2016.
Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. Hatexplain: A benchmark dataset for explainable hate speech detection. Proceedings of the AAAI Conference on Artificial Intelligence, 35(17))14867-14877 May 2021. URL https 77ojs www..aaaaa..o.. wwwaaiioor//idexxx pttp://AA./aaticce//eeee///////s
George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11): 39-41, 1995.
Ramaravind Mothilal, Amit Sharma, and Chenhao Tan. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference Fairness, Accountability, and Transparency, pp. 607-617, 2020.
Van Nguyen, Trung Le, Olivier De Vel, Paul Montague, John Grundy, and Dinh Phung. Information- theoretic source code vulnerability highlighting. In 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-8, 2021. doi: 10. 10.1109/JCNN52387.. 1...:3877...1..93999999
Van Nguyen, Trung Le, Chakkrit Tantithamthavom,, John Grundy, Hung Nguyen, Seyit Camtepe, Paul Quirk, and Dinh Phung. An information-thewne and contrastive learning- based approach for identifying code statements causing software vulnerability, 2022. URL https: //arxiv brg/abbs/200 /00bb//220... 10414
11

---

Published as a conference paper at ICLR 2023
Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.0742)) 2018.
Gregory Plumb, Denali Molitor, and Ameet S Talwalkar. Model agnostic supervised local explana- tions. Advances in neural information processing systems, 31, 2018.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should trust you? explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Michael L Rich. Machine learning, automated suspicion algorithms, and the fourth amendment. University of Pennsylvania Law Review, pp. 871-929, 2016.
Marko Robnik- -hoonk-Sikonja and Marko Bohanee. Perturbation-busee explanations of prediction models. In Human and machine learning, pp. 159-175. Springer, 2018.
Patrick Schwab and Walter Karlen. Cxplain: Causal explanations for model interpretation under uncertainty. Advances in Neural Information Processing Systems, 32, 2019.
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In International conference on machine learning, pp. 3145- 3153. PMLR, 2017.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. In In Workshop International Conference on Learning Representations. Citeseer, 2014.
Xuelin Situ, Ingrid Zukerman, Cecile Paris, Sameen Maruf, and Gholamreza Haffari. Learning to explain: Generating stable explanations fast. In Proceedings of the 59th Anmual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing ( (Volume 1: Long Papers), pp. 5340-53400000 5355, 2021.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. ICML Workshop, 2017.
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXiv preprint 1rXiv:1412.6800 2014.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pp. 3319-33222 PMLR, 2017.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint 1911::1110.03771,, 2019.
Jinsung Yoon, James Jordon, and Mihaela van der Schaar. Invase: Instance-wise variable selection using neural networks. In International Conference Learning Representations, 2019.
Jinsung Yoon, Sercan Arik, and Tomas Pfister. Limis: Locally interpretable modeling using instance- wise subsampling. 2022.
Muhammad Rehman Zafar and Naimul Mefraz Khan. Dlime: deterministic local interpretable model-agnostic explanations approach for camputer-aided diagnosis systems. In Proceedings of SIGKDD Workshop on Explainable AI/ML (XAI) for Accountabiliry, Fairness and Transparency. ACM, 2019.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European conference computer vision, pp. 818-833. Springer, 2014.
12

---

Published as a conference paper at ICLR 2023
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015.
Xingyu Zhao, Wei Huang, Xiaowei Huang, Valentin Robu, and David Flynn. Baylime: Bayesian local interpretable model-agnostic explanations. In Uncertainty in Artificial Intelligence, pp. 887-896. PMLR, 2021.
13

---

Published as a conference paper at ICLR 2023
A EXPERIMENTAL DESIGN
We now discuss the model design for each component in our framework. We parametrize &, and by three deep neural network functions. Since our input X is discrete, every network contains a learnable embedding layer. The explainer & passes the embedded inputs into three 250-dimensional dense layers and outputs W after applying ReLU non-linearity. The selector S is composed of one bidirectional LSTM of 100 dimension and three dense layers of the same size. Each layer is stacked between a Dropout layer and an activation. The upper layers use ReLU while Sigmoid is natural choice for the final one. Regarding the network G, after feeding the inputs into its embedding layer, we process the outputs through a 250-dimensional convolutional layer with kernel size followed by max- poollini layer over the sequence length. The last layer is dense layer of dimension 250 together with Softmax activation. We use the same architecture for all tasks and train our model with Adam optimizer at r 0.2 and learning rate of 0.001. We tune the coefficients a, B via grid search to achieve an adequate balance of faithfulness and compression.
Table [Jdeaaiss data splits and best hyperparameters used in our experiments for classification (IMDB/ / HateXplain AG News) and image recognition tasks (MNIST/ Fashion-MNIST), and are the balancing coefficients on the loss terms in the final training objective. For every dataset, tune and A via grid search with values in {0.1, 0.5, 1, 1.5, 1.8, 2} and {1c-2, 3, 1e-41 respectively, and the setting that yields the highest Faithfulness selected. In Table 12 we provide detailed empirical results showing superior performance is insensitive to
choices.
Table 6: Dataset statistics and hyperparameters.
Dataset
Train/Dev/Test
No. of features
a
/
IMDB
25000/20000/555
400
1.8
le-3
HateXplain
15000/ /40000/411// 1029
200
0.1
le-3
AG News
120000/ /6000/00001/ 1220
400
0.1
1e-4
MNIST
14000/4623/31
16
0.5
le-3
Fashion-MNiS
15000/3000/300
16
0.5
le-3
B PERFORMANCE METRICS
We here discuss the implementation details of our quantitative metrics for text explanation tasks. Recall that saliency-based approaches produce explanations in the form of subset of K most important features $ given by the weight vector.
B. PURITY
For text classification tasks, we observe that an explainer sometimes selects stopwords or punctuation important features, which incomprehensible from human user's perspective. An effective explainer should reduce the likelihood of picking such "contaminated" features. Purity quantifies the proportion of stopwords and puncluation included in S. We obtain the collection of stopwords and punctuation via NLTK package (Bird) 2006
B.2 BREVITY
Given a subset S. we define an explainer achieving brevity if the subset contains closely related features. For textual data, expect the chosen features to contain a large number of duplicates and/or synonyms. We introduce cluster ratio to quantify brevity. Specifically, we first collect a database of semantically related words through WordNet (Miller 1995). We group tokens in S into clusters of synonyms (including duplicates), then calculate the average number of clusters formed over K tokens.
14

---

Published as a conference paper at ICLR 2023
B.3 FAITHFULNESS
Faithfulness measures the degree of agreement between the black-box's prediction given the explana- tion and the prediction given the original input. When fed into the black-box model, the explanation a set of discrete features, is reconstructed into a similar representation vector with the original input where features in retained and those not in & are masked by zero paddings. Faithfulness is standard criterion to evaluate the quality of textual explanations and commonly adopted in various literature, including our baseline papers (Ribeiro et 2016 Chen al. 2018: Situ et al. 2021
Gat
et al. 2022
B.4 STABILITY
One desirable property of a good model explainer is Stability the ability to produce the same explanations given similar examples. In the context of text explanations, the subsets of selected important words are expected to overlap in large quantities for two similar documents. We evaluate explanation stability through simplified implementation of the measure Intersection over Union (IoU) originally proposed in (Situ et al. (02TT.
Given an example in the test set, first search for the nearest neighbors Mzz). The neighboring documents are defined to (1) have the same -black-box predicted) label and (2) be either lexically or semantically similar. We adopt the ratio of overlapping tokens as a proxy metric for lexical similarity. Semantic similarity is measured via cosine similarity of their BERT representations, obtained by summing over the token representations of the last hidden state produced by pre-trained BERT uncased base open sourced by Hugging Face (Wolf et al. 2019). We then select set of 20 distinctive neighbors, consisting of top 10 semantically and top 10 lexically similar documents. Let Vy and Va respectively denote the subsets of top K tokens selected for the instance and its neighbor 2', IoU is given as
Σ Z'EN(x)
Ue O Ual De U Uar
[
To eliminate the effect of poor initialization, for each model explainer, we evaluate the model initialization with the highest faithfulness and compare the stability of top 10 explanations. Noticing that explainers sometimes favor a large number stopwords, which may overestimate the measure, exclude such tokens in the feature when computing Stability.
B.5 A LOG-ODDS
Given an example A log-odd((z) measures the change in the confidence of the black-box's prediction before and after masking features in an explanation (Schwab & Karlen 2019 Situ al. T)2T). Given the original input vector the black-box model outputs the predictive distribution P = c x) with label E {1,..., C}. Let ym argmax = denote the predicted label.
Alog-odds log-odds(PP,,) (ym x)) log-odds(PPm T)),
where log-odds(P)) log - and क denotes the masked representation. Positive A log-odds refers the input version where mask important features i.e., features in S. Negative A log-odds refers to the input version where mask unimportant feature i.c., all features not in S. This is also the input version used to evaluate Faithfulness.
15

---

Published as a conference paper at ICLR 2023
C QUALITATIVE COMPARISON
This section presents 12 additional qualitative examples to examine the quality of explanations of all model explainers. These examples are randomly selected from the oulputs of the model initialization with the best faithfulness. Examples 9 12 are particularly dedicated to illustrate multi-class explanations. Across all examples, we again demonstrate that our explanations are strongly consistent with black-box 's predictions, highly compact (by covering duplicates and synonyms) and distinctive to each decision class.
Original document: this movie was a pleusant surprise for me. all honesty, previews looked horrible, up until the point where emma thompson and alan rickman appeared. so i rented it with reservation, but i thoroughly enjoyed this movie. it had great acting, a few good plot twists, and, of course, emma thompson and alan rickman. its definitely worth checking out.
Truth: positive Model: positive
Explainer
Key words
AIM
enjoyed, great, emma, emma, definitely, pleasant, reservation, good, twists, and
L2X
enjoyed, pleasant, definitely, great, checking, rented, emma, surprise, worth worth, great, enjoyed, and, it, checking, definitely, surprise, movie, thompson horrible, great, rickman, definitely, honesty, surprise, rented, reservation, the,
LIME
VIBI
acting
FI
horrible, acting, movie, pleasant, great, thoroughly, enjoyed, where, was
Original document: this may just be the worst movie ever produced. worst plot, worst acting, worst special effects...be prepared if you want to watch this. the only way to get enjoyment out of it is light a match and burn the tape of it, knowing will never fall into the hands of any sane person again.
Truth: negative-
Model: negative
Explainer
Key words
AIM
worst, worst, worst, worst, plot, acting, any, just, if. to worst, worst, worst, match, any, only, worst, light, plot, tape
L2X
LIME
worst, special, of, acting, match, the, prepared, enjoyment, to, tapeX
VIBI
burn, worst, knowing, tape, the, the, special,
FI
worst, tape, plot, worst, be, never, burn, worst
Original document: to me, "anatomie" " is certainly one of the better movies have seen. dont think "anatomie" was primarily intended be horror movie but movie questioning the ethics of science. if you watch it with that mind, turns into a really good film. the only annoying bit was the awful voice dubbing for the english version. how can you expect any non- non-german person listen to these unbearable german accents for two hours? let native english speakers do the talking or use subtitles instead!!
Truth: positive Model: negative
16

---

Published as a conference paper at ICLR 2023
Explainer
Key words
AIM
awful. ?, anatomie, anatomie, unbearable, annoying, dubbing, any, "nt", horror
L2X
awful, accents, instead, seen, good, do, or, f,
LIME
awful, can, non, seen, the, really, speakers, into, instead, annoying
VIBI
anatomie, questioning, anatomie, german, native, ethics, intended, instead, let,
movie
FI
?, really, me, science, any, better, to, can, let, was
Original document: have seen this movie several times, it sure is one of the cheapest action flicks of the eighties. so, think many viewers would definitely change the channel when they come across this one. but, if you are into great trash, "drugon hunt" is made for you. the main characters (the menamara twins) are sporting great moustaches and look so ridiculous in their camouflage dresses. one of the best scenes is when one of then gets shot in the leg and is still kicking his enemies into nirvana. this movie is really awfil,, but then again, it is great party tape!
Truth: negative Model: positive
Explainer
Key words
AIM
great, great, great, nirvana, best, definitely, still, times, and, and seen, great, tape, ridiculous, best, great, great, sporting, awful, !
L2X
LIME
great, is, and, best, party, it, made, the, really, awful
VIBI
sporting, moustaches, menamara, nirvana, into, great, dragon, tape, great,
party
FI
would, great, are, ridiculous, across, have, tape, in
Original document: this is not competition against whites we are in this together if you want to criticise other whites this only makes it impossible to ever beat these spics and sand niggers from taking over just as the niggers have.
Truth: hate/offensive speech
Model: hate/offensive speech
Explainer
Key words
AIM
niggers, niggers, sand, whites, whites, only, impossible, are, these
L2X
sand, beat, niggers, spics, niggers, whites, criticise, makes, competition, want
LIME
niggers, this, whites, are, is, ever, have, makes, we, against
VIBI
impossible, whites, ever, beat, over, a, niggers, this, to, taking
if. is, against, this, it, just, spics, niggers, makes
Original document: all it does boomerpost while pretending to slav and screeches anytime anyone says anything about jews like anyone other than a boomer would be retarded enough to believe slavs do not know their own 20 century history at the hands of the chosenites.
Truth: normal Model: hate/offensive speech
17

---

Published as a conference paper at ICLR 2023
Explainer
Key words
AIM
boomer, jews, screeches, retarded, a, a, would, enough, their. hands
L2X
retarded, jews, th, boomer, 2, enough, slav, pretending, history, screeches
LIME
jews, believe, all, it, anytime, boomer, hands, boomerpost, retarded, 20 slav, slavs, anyone, own, their, jews, screeches, know, anything, it says, be, all, chosenites, O, a, about, pretending
VIBI
FI
oracle statements peoplesofts chief executive accused oracle of spreading misleading information about his stock sales and threatened to for defamation.
7. Original document: peoplesoft chief threatens
Truth: business
Model: business
Explainer
Key words
AIM
stock, executive, sales, chief, chief, peoplesoft, peoplesoft, misleading, state- ments, defamation
L2X
stock, sales, chief. peoplesoft, executive, "s", statements, peoplesofl, orucle, his his, sales, oracle, chief, executive, about, accused, threatens, misleading, and desamation, spreading, statements, threatens, execulive, sales, oracle, chief. peoplesoft
LIME
VIBI
FI
stock, chief, oracle, accused, peoplesoft, oracle, about, misleading, spreading. chief
blunkett gets tougher on drugs new police powers to prosecute offenders for possession if they test positive for drugs when they are arrested, even if the only drugs they have are in their bloodstream, are to be announced this week.
8. Original document:
Truth: world Model: sports
Explainer
Key words
AIM
test, offenders, positive, tougher, when, only, they, they, they, powers
L2X
blunkeut, police, test, possession, only, bloodstream, positive, arrested, offenders,
have
LIME
test, offenders, they, tougher, drugs, the, for, gets, positive, are prosecute, if. for, offenders, gets, tougher, the, even
VIBI
FI
offenders, they, powers, week, they, drugs, the, test, on, bloodstream
18

---

Published as a conference paper at ICLR 2023
9. Original document: fda oks scientist publishing vioxx data (ap) ap the food and drug adminis- tration has given a whistle-bile boowerr scientist permission to publish data indicating that as many as 139,000 people had heart attacks that may be linked to vioxx, the scientists lawyer said monday.
Explainer
Topic
Key words
AIM
World
attacks, people, lawyer, permission, linked, heart, ap, ap, whistle, indicating
AIM
Sci/Tech
scientist, scientist, scientist, data, data, may, many, be, ap. ap ap, scientist, data, drug, people, monday, vioxx, attacks, s, may monday, vioxx, food, drug, data, said, scientist, people, lawyer, had may, oks, indicating, administration, vioxx, "s", publishing, +, many,
LIME
World
LIME
Sci/Tech
FI
World
Fl
Sci/Tech
oks, administration, may, drug, "s", ap, scientist, ap, that, many
10. Original document: tivo net loss widens; subscribers grow tivo inc. inc.(iivo.o: quote, profile, research), maker of digital television recorders, monday said its quarterly net loss widened as boosted spending to acquire customers, but subscribers to its fee-based tv service rose.
Explainer
Topic
Key words
AIM
Business
profile. quote, rose, quarterly, maker, widened, based, boosted, grow, fee
AIM
Sports
loss, loss, recorders, widened, television, monday, ;. subscribers, subscribers, but
LIME
Business
its, quote, digital, profile, widened, loss, said, monday, spending, customers
LIME
Sports
service, its, research, maker, profile, but, television, rose, monday, boosted
FI
Business
widens, profile, rose, tivo, ), research, tivo.0
FI
Sports
profile, rose, widens, quarterly, fivo, digital, its
19

---

Published as a conference paper at ICLR 2023
11. Original document: i could not believe the original rating i found when i looked up this film, 9.5? unfortunately it looks like am not alone. the film, is slow and boring really, one of the sad things is that if the film had been given realistic rating of around 5 or 6 then the expectation would not have been so high. unfortunately, this was not the case, so when watching the film, and seeing the poor story and acting, am left giving it 3/10 score. vinnie jones is superb in lock stock, and also snatch, and he plays great hard man, however, he should stick to this role. its a bit like when stallone and schwarzenegger have done comedy films, they just don't It work. neither can he play lead actor, he plays better as supporting or otherwise. when he plays lead, his acting talents are too 'in view' and shown up as not really very good. mean machine is another good example of this.
Explainer
Sentiment
Key words
AIM
Negative
poor, unfortumately, boring, 3/10, ?. otherwise, looks, acting, acting, looked
AIM
Positive
superb, great, also, very, bit, realistic, plays, plays, plays, supporting poor, work, acting, given, lead, sud, one, rating, comedy, too the, good, plays, when, not, unfortunately. acting, boring, then, case another, is, unfortunately, not, poor, very, schwarzenegger, otherwise, given, slow
LIME
Negative
LIME
Positive
FI
Negative
Fl
Positive
another, is, unfortunately, poor, very, not, schwarzenegger, given, otherwise, slow
12. Original document: the finest short i've ever seen. some commentators suggest it might have been lengthened, due to the density of insight it offers. there's 's irony in that comment and little merit. the acting is all up to noonan and he carries his thankless character perfectly. might have preferred that the narrator be less "recognizuble", but the gravitas lent is pitch perfect. this is a short for people who read, for those whose "bar" set high and for those who recognize that living culture that celebrates stupidity and banality can forge contrary and bitter defenders of beauty. beautiful short film. fwiw: was pleased at the picasso reference, since once believed that picasso was just another art whore with little talent; like, i assume, most people until the day saw some drawings he made when he was 12. picasso was a finer draftsman and a brilliant artist at that age than many artists will ever become in lifetime. understood immediately why he had to make the art he became known for.
Explainer
Sentiment
Key words
AIM
Negative
stupidity, talent, acting, suggest, just, been, banality, make, might, might
AIM
Positive
perfect, brilliant, perfectly, artists, beautiful, finest, pleased, cele- brates, beauty, reference
LIME
Negative
and, he, ever, offers, a, it, short, defenders, in, saw
LIME
Positive
and, who, became, beautiful, he, is, short, the, i merit, suggest, immediately, acting, offers, pitch, insight, ., is
FI
Negative
FI
Positive
merit, suggest, immediately, acting, pitch, offers, insight
20

---

Published as a conference paper at ICLR 2023
D ABLATION STUDY
Generally, our framework involves both an explainer and a feature selector. The explainer & aims produce multi-class explanation module W, x directly used infer features. The role of the selector is to learn good local distributions to generate high-quality local perturbations to train &. Here we study various setups for AIM demonstrate that the proposed method yields the optimal performance. We seek to answer the following questions:
1. Does inference from the probability vector of the selector S give better result than inference from Wa the explainer E?
2. Is the explainer necessary component?
3. Is using samples from learnable local distributions better than using heuristic samples?
To validate these hypotheses, analyze 3 different approaches IMDB dataset. Table compares the quality of explanations produced under these setups with the performance level achieved under our proposed method on metries: Faithfulness, Purity and Brevity.
D.1 INFERENCE FROM SELECTOR
In the original framework, experiment with the strategy of inferring features from the output probability vector Tz. Following the same approach, we rank features Te a decreasing order and conventionally select the top 10. We initially argue that the selector operates each feature independently, thus does not guarantee good performance when combining features into a single explanation. Table 7 supports this argument in that this approach leads to nearly 20% drop in Faithfulness.
D.2 TRAINING SELECTOR ONLY
The L2X and VIBI frameworks only contain feature selector from which accordingly infer explanations. We investigate whether training the explainer jointly is necessary or if the selector simply does the job. Recall our final objective function
mino c. L2 + B E.
W (allla.)
where L1 is used to train the explainer and optimize Wz. Removing the role of explainer, we omit Li and the third loss term. We then only train the selector according to L2 with the support of the approximator G. It can be seen from Table that AIM again under-performy under this setup and the performance does not differ much from the first scenario above. We further note that our method employs the Gumbel trick for Bernoulli sampling, while L2X applies it for Categorical sampling over features. This explains why despite having the Selector only, L2X can still search for better combinations of features.
21

---

Published as a conference paper at ICLR 2023
D.3 TRAINING EXPLAINER ONLY
Lastly, we analyze the importance of learning local distributions via the selector compared to using heuristic sampling. In our framework, we expect the selector helps mitigate the risk of ill-conditioned local samples observed in LIME. To validate this hypothesis, we modify our framework to mimic LIME: We first exclude the S and To train E, we uniformly sample local perturbations, denoted as Zz, with the number of non-zero elements also uniformly drawn at random.
E is now optimized purely on L1, which is modified
L = Eg
CE (Jm. softmax WW Re))
Table 7 shows that this heuristic approach does significantly worsen the explainer performance. This proves the effectiveness of optimizing the selector S together with the explainer so that can assist E in learning faithful feature attributions and distinctively across decision classes.
Table 7: Ablation study of AIM on IMDB dataset. Proposed method.
Method
Purity (%)
Brevity
Faithfulness (%)
Inference from Explainer*
8.22:10.20
2.08
99.62:10022
Inference from Selector
1..72::1.56
3.15=10114
80.23
Training Selector only 29.77#3.79
26.48:t0.16
3.09:10.05
80.71=0.34
Training Explainer only
3.96=0.57
61.66:1.900
E EXPERIMENTS ON IMAGES AND TABULAR DATA
E.
IMAGE
We first describe the experiments for interpreting image recognition machine learning systems. The MNIST and Fashion-MNIST dataset respectively consist of 28 x 28 gray-scale images of handwritten digits and article clothing images. We train two simple neural networks digits 0, 1, 2 and Fashion-MNIST images of Tsshir//rouueer/Pllovrr Both networks have the same architecture: 2 convolution layers kernel size followed by 2 dense layers with output Softmax activation. The model on MNIST achieves 97. .5% test accuracy while that on Fashion-Man MNIST gains 95.9%.
subset of MNIST
E.1.1 PIXEL-BASED EXPLANATION
In this section, investigate the potential of AIM framework visual tasks by comparing AIM with two popular image explamation methods: Integrated Gradients (IG) (Zeiler Fergus| (014) and Kernel SHAP (Lundberg & Lee 2017). To keep it consistent with these baselines, consider each pixel to be a feature and the goal is to find the optimal local subset of pixels for each example x that approximate the black-box prediction on the full image.
We compare Faithfulness scores of top K selected features with E {200, 300, 400]. We again approximate the explanation with the input variant where the features not selected masked by zeros. The results are averaged over model initializations. AIM has been shown to be superior on texts and Table Sable&hhere here demonstrates that AIM framework can also work reasonably well on images.
E. 1.2 SUPERPIXEL-BASED EXPLANATION
We now consider groups of pixels features. We split each image into 4 x 4 patches size 7 x 7, resulting in a total of 16 features. Table reports faithfulness of superpixel- explanations on MNIST and Fashion-MNIST test sets, averaged over 5 model initializations. Randomly selected examples of various scenarios are additionally presented for qualitative investigation. We find that the selected features particularly useful to explain wrong decisions in terms of what spurious signals the black- box model relies on to make predictions e.g., the round shape to predict digit 0, or the rectangular pattern the bottom predict Trouser instead of T-shirt.
22

---

Published as a conference paper at ICLR 2023
Table 8: Faithfulness (%) of AIM, IG and SHAP on pixel-based explanation.
K
200
300
400
MNIST
AIM
96.16
96.95
97.30
Integrated Gradients
99.10
99.52
99.52
Kernel SHAP
67.33
67.33
67.33
Bushion-MNISSS
AIM
90.66
93.13
97.60
Integrated Gradients
92.93
94.43
94.43
Kernel SHAP
52.30
59.63
62.77
Table 9: Faithfulness (%) of AIM superpixel-based explanation.
K
5
8
10
MNIST
92.67
94.60
95.08
Fashion-MNISS
88.46
92.23
92.42
Model
Truth
Model
Truth 2
Model
I
2
Truth
Model 2
Truth
Model 2
Truth 2
Model 0
1
a
C
Truth T-shirt
Model Pullover
Truth T-shir
Model Trouser
Truth Trouser Model Pullover
Truth Tishirt
Model T-shirt
Truth Trouser
Model Trouser
Truth Pullover Model Pullover
Figure 3: Explanations the black-box model's predictions based on top most relevant superpixels.
23

---

Published as a conference paper at ICLR 2023
E.2 TABULAR DATA
We here demonstrate the extensibility of AIM framework on tabular data. We experimented with two real-world datasets: Admission (Acharya al. 2019) (classifying whether a graduate application is successful on 12 features) and Adult (Kohavi et al. 1996) (predicting whether income exceeds 50K/year 7 features). For every test example, the task select the top most relevant features to the original prediction from black-box classifier.
Evaluation metrics. want to evaluate how well top K important features can approximate the prediction the full input. In the main paper, this is done via the Faithfulness metrie: the consistency between the black-box predictions on the full input and the imput variant where non- selected features masked. For texts and images, masking is often done by zero replacement. However, for tabular data, zero values do not indicate the absence of features. We therefore implement mean masking and noise masking strategies: the former replaces unimportant features with the mean value of each feature over the test set; the latter replaces them with random uniform noise E [-1, 1].
We additionally report the Positive Log-odds scores, which measure the drop in the black-box confidence scores before and after masking the top K features. A selection of features is deemed important when Faithfulness and Positive A Log-odds metries both high.
Results. We evaluate the top K = 6 and K = 3 features (half the number of features) respectively Admission and Adult datasets. The following table summarizes the average results over model initializations and 10 initializations of random noises. The black-box architectures are given in parentheses. We compare AIM against popular tabular baselines: LIME (Ribeiro et al. (016)). INVASE (Yoon et al. 2019) and MAPLE (Plumb et al.) 2018). Here demonstrate the selected features from AIM do encode sufficient information to yield a prediction consistent with the full input while being more robust to perturbations.
Table 10: Performance of AIM, LIME, MAPLE and INVASE tabular datasets
Method
Faithfulness (%)
Faithfulness
(%)
Positive A Log-odds (Mean masking) f
Positive A Log-odds (Noise masking)
(Mean masking)
(Noise masking)
Admission Random Forest)
AIM LIME
94.00
92.20
1.12
1.17
93.00
91.20
0.99
1.16
MAPLE
94.00
91.30
0.74
1.03
INVASE
91.00
87.50
0.92
0.99
Adult (Logistic Regression)
AIM
99.00
86.26
1.82
1.82
LIME
86.63
83.27
1.21
1.19
MAPLE
91.72
81.52
0.93
0.94
INVASE
81.93
67.90
0.78
0.79
24

---

Published as a conference paper at ICLR 2023
F HUMAN EVALUATION
We ask 30 university students to infer the sentiments of 50 IMDB movie reviews, each of which is given only 10 key words obtained from each explainer. Each participant is presented with 3 sections, containing output examples from AIM, L2X and LIME respectively. Each section displays 50 sets of 10 key words corresponding belongs to which method is hidden and the ordering of examples within section is randomized.
different movie reviews. The information on which section
Movie Reviews Sentiment Evaluation
The survey has sections. each section, you will be given 50 lines of texts. Each line contains few words extracted from movie review. Given only such words, please infer the original sentiment of the movie review (Positive, Negative, Neutral Can't decide).
Switch account
Requuied
Based on the following words only, guess the original sentiment of the movie
review
Neutral / Can't
Positive
Negative
decide
[bad, poorly, total, saving, needless, whatever, far, not, not, onlyl
[ridiculous, supposed, plot, plot, no, positive, anything, could, any, any]
3. [awful, bad, bad, bad, horrible, grade, turkey, instead, reason, dialog]
[amazing, job, holds, powerful, especially, humanity, message, true, seen, will]
Figure 4: Human Evaluation Interface
25

---

Published as a conference paper at ICLR 2023
G MODEL MONOTONICITY OVER K
The following table reports the performance L.2X trained at different values of K. It highlights the fact that careful choice of K hyperparameter crucial, and larger K does not necessarily yield better results. This is undesirable since in fact, larger K increases the chance of selecting meaningless features (lower purity) while does not guarantee faithfulness will go up accordingly. We also provide qualitative examples showing inconsistencies for selecting top K' features i.e, the rankings of features vary across settings. For instance, though qualitatively considered an important feature, the word amazing in example is selected in top but does not appear in top 10 and even ranks ninth in top 20. The same pattern can be observed across examples.
Table Performance of I.2X when trained at values of K for all datasets. Performance of AIM under the same setup is reported for comparison.
Explainer
L2X
AIM
K
Purity (%)
Brevity |
Faithfulness (%)
Purity (%)
Brevity +
Faithfulness (%)
IMDB
5
10.99=0.200
1.68:0.18
83.42-+0.08
5.91=#0.06
1.68:+0.00
99.02=0.03
10
12.89=0.27
2.5110.14
04.8010.08
8.22:0020
2.48100.11
99.62=0.02
20
19.23=0.24
5,21=0,10
14.14:10.06
12,46:40.26
4.30+10.22
99.88=0.01
HateXplain
5
22.74=0.16
2.65
70.17+00.07
13.94++1.44
2.59++0.5
85.15-0.81 0.81
10
21.87=0.14
4.3620.15
75.32-0..33
19.78-2.54
3.88:1.21
92.98=1117
20
26.63=0.10
7.25
82.90:0.19
18.85:3.70
5,38+1061
95.22=1.32
AG News
5
4.29:10.18
2.
85.79:10.09
1.29-0.03
2.10:10.00
94.78-0.08
10
6.64::0.25
3.94:-0.18
90.13-40.26
3.83--0.05
3,39:10.00
97.92-0.05
20
13,10=0.20
5.99:0.15
88.82:00.15
13,08:0.03
6.53:10.01
99.26=0.06
1. Original document: i saw Riverlance the new show and loved it frfrrmm the very first moment! it is an energetic tribute to Irish dance filled with brilliant dancing, music and choreography! the leads, Jean Butler and Colin Dunne had me captivated with their exquisite dancing! may they always keep shining and keep dancing. their on stage chemistry was amazing, and the unity between them on stage was obvious. they look like they were made to dance with each other! this show is my absolute favourite, and probably always will be. long live Riverdance!
Truth: positive- Model: positive
K
Key words
5
loved, favourite, energetic, brilliant, amazing
10
energetic, loved, favourite, tribute, brilliant, first, moment, dancing, butler, obvious
20
energetic, loved, butler, favourite, choreography, !, dancing, moment, amazing, other, dancing, captivated, brilliant, probably, Riverdance, is, tribute, music,
26

---

Published as a conference paper at ICLR 2023
2. Original document: how much longer will the west continue to put up with all of this shit from retarded third worlders and liberal cucks before we revert to a frontier mentality and just start the real violence.
Truth: hatc/offensive speech Model: hatc/offensive speech
K
Key words
5
retarded, third, cucks, frontier, violence
10
third, retarded, cucks, worlders, frontier, mentality, violence, liberal, shit, and
20 worlders, third, retarded, and, frontier, mentality, to, from, will, longer, pud, a, liberal, the, continue, shit, west, and, much, up
Yahoo and SBC extend partnership and plan new services Yahoo and SBC communications have agreed to collaborate to extend some of the online services and content they currently provide PC users mobile phones and home entertainment devices.
3. Original document:
Truth: sci/tech Model: sci/tech
K
Key words
5
Yahoo, Yahoo, users, phones, online
10
phones, online, Yahoo. users, Yahoo, devices, communications, mobile, collaborate, agreed
20 services, entertainment, collaborate, content, extend, and, and, PC, services. Yahoo, extend, some, have, partnership, provide, agreed, home, phones, the, and
4. Original document: riding high on the success of "rebel without cause", came tidal wave of teen movies. arguably this one of the best. a very young Mcarthur excels here as the not really too troubled teen. the story concentrates more perceptions of delinquency. than any traumatic occurrence. the supporting cast is memorable, Frankenheimer directs like an old pro. just a story of young man that finds others take his actions much too seriously.
Truth: positive Model: positive
K Key words
5
best, memorable, one, troubled, the
10
best, memorable, too, teen, more, riding, high, seriously, young
20 memorable, best, [PAD], very, one, high, seriously, troubled, any, perceptions, story, teen, is, young, directs, on, traumatic, success
5. Original document: i thought i was going to watch another friday the 13th or a halloween rip off. but was surprised, its about 3 psycho kids who kill, theres not too many movies like that, can think of mikey, children of the corn and few others, its not the greatest horror movie but its least worth a rent.
Truth: negative Model: negative
K
Key words
5
surprised, least, 3, greatest, about
10
not, least, surprised, rent, ., was, [PAD], thought, worth surprised, least, (PAD], think, halloween, worth, going, . it, rent, was, mikey, children. a, about, but, "s", " [PAD], "s"
20
27

---

Published as a conference paper at ICLR 2023
H HYPER-PARAMETER TUNING
H.1
AIM
In our experiments, the only hyperparameters subject tuning are loss coefficients a and While seeks to balance exploration and exploitation of local samples as discussed in the previous sections, controls the magnitude of |[ 12,1 for stable backpropagation.
We here would like to demonstrate that our framework is not highly sensitive to hyperparameters. As reported, 3 is chosen at le across most text and image datasets, while a can vary within {0.1, 0.5, 0.8).
The following table reports the performance of our models under various settings of a (averaged over 5 initializations). It can be that there is significant variation in the performances compared to our reported results (highlighted in bold). For the purpose clarity, we only display the results for crucial metries: Faithfulness and A Log-odds scores.
Table 12: AIM Hyper-arrameter Tuning.
Cx
Faithfulness (%) f Positive A log-odds 1
Negative A log-odds +
IMDB
0.1
99.61
9.34
1.89
0.5
99.56
8.83
-0.09
1.0
99.41
6.65
0.98
1.5
99.50
7.31
1.98
1.8
99.62
7.53
-0.20
2.0
99.10
8.35
1.49
HateXplain
0.1
92.98
4.98
-1.40
0.5
92.38
5.11
-1.75
1.0
92.50
5.87
-1.82
1.5
92.52
5.02
-1.16
1.8
91.28
4.28
-1.36
2.0
90.67
3.16
-0.44
AG News
0.1
97.92
7.14
-1.09
0.5
96.03
7.16
-1.01
1.0
97.89
7.14
-1,09
1.5
97.03
7.10
-0.88
1.8
97.42
7.10
-1.08
2.0
97.83
7.09
-1.07
H.2 BASELINES
This section provides performance results of the baseline methods under different parem-errreeter settings. Note that black-box architecture for IMDB dataset is different from ones reported in Chen et al. 22088) Bang et al. (2021) and Gat et al. (2022): L2X adopts CNN while VIBI and FI opt for LSTM. Since AIM is not GRU-based either, black-box model is chosen to be bidirectional GRU in order to examine whether these models can explain different kinds of black-box architectures.
The table below lists all of the remaining hyper-pane parameersss subject tuning and their corresponding model performance. We tune all baselines via grid search over the following ranges and report the average results over initializations. For the purpose clarity, only display the results for metrics: Purity, Brevity and Faithfulness. When there is trade-off among these metrics, Faithfulness chosen be the deciding criterion. The best settings for each method are presented in bold and their corresponding results are reported in the main paper.
28

---

Published as a conference paper at ICLR 2023
H.2.1 L2X
For L2X, we tune 7 which the Gumbel-Softmax temperature. L2X has only one loss term, so no loss term coefficient needs tuning.
Table 13: L2X Hyper-parameters Tuning,
Purity (%) 4
Brevity
Faithfulness (%)
IMDB
0.1
20.15:10.28
3.12:1011
83.10:0.335
0.2
11.62:10.18
2.46:10.16
84.12:1.117
0.5
12.79=#0.36
2.64+0.21
84.40=00007
0.7
12.89=0.27
2.51
84.80
HateXplain
0.1
21.87++0.14
4.36+-0.15
75.32-1-0.03
0.2
23.52++0.3
4.34
75.22++0.33
0.5
17.46=10.15
3.48
67.54
0.7
16.5
2.9310.06
55.78
AG News
0.1
6.56+1019
3.80+10.09
89.41=0.088
0.2
6.64
3.94
9..1
0.5
6.01+1.07
3.12
89.08
0.7
12.54:0.24
3.72+0.13
87.04
H.2.2 LIME
The relevant hyper-parameters of LIME for text explanations include kernel width (used define proximity function) and number of sampling perturbations N. As shown in Figure increasing N leads better faithfulness, yet at the cost of an exponential climb computing times. our maximum capacity, we follow the authors' suggestion setting N 5000 for all experiments. We examine kernel width in (15, 20, 25, 30.35).
Table 14: LIME
Tuning.
Kernel Width
Purity
Brevity J
Faithfulness
IMDB
35.9511.007
7.03
67.00110.04
20
36.5510.113
7.13
79.00110221
25
36.20110.20
7.34.0.35
74.00.10.35
30
37.7510..15
7.39
69.00.10.17
37.40.10.37
7.57
70.00.10.30
HateXplain
15
37.88-10.13
7.
80.27-10.06
20
37.85-10.06
7.36
19.49-10.19
37.74+0.19
7.63-0.14
13.94+0.11
30 38.6440.114
7.85-0.06
80.27+0.09
35
0.17
7.59
80.56+0.11
AG News
28.70.10.08
8.65
86.12.1033
20
29.1910.12
8.77.0..1
85.46.10.22
25
29.15.0..06
8.76
86.6410.110
30
29.50.10.06
8.75
84.8710.03
29.00.1..01
8.76.0.02
85.46.10.12
29

---

Published as a conference paper at ICLR 2023
Faithfulness
Computing time
25
224
Computing time (s)
Faithfulness (%)
70
2
65
2
60
2
55
2
100
500
1000
3000
5000
No. of perturbations per instance
Figure 5: Faithfulness of LIME vs. the number of random perturbations and average processing time on single CPU over 1000 random test samples in IMDB dataset. The presented model trained at K 10 and kernel width of 20.
H.2.3 VIBI
In terms of model architecture, VIBI offers multiple options for the approximator. In our experiments, LSTM approximator gives the highest accuracies for both IMDB and AG News, while CNN works best on HateXplain. For the remaining hyper-parameters we tune the Gumbel-Softmax temper- ature t within {0.2, 0.5, 0.7). The objective function of VIBI further has two loss terms where 3 is the weight of the second one, controlling brevity of the explanation. We explore 3 within (0.1, 0.3, 0.5, 1.0, 1.5, 2.0).
H.2.4 FI
For FI, the relevant hyper-parameter is the number of sampling perturbations N. Since Fl is very time-expensive, we tune N over 3 values (50, 100, 200}.
30

---

Published as a conference paper at ICLR 2023
Table 15: VIBI Hyper-parameterr Tuning.
B
Purity (%) 4
Brevity + Faithfulness (%) 1
IMDB
0.2
0.1
3..33+00200
3.81
55.96++0.10
0.5
0.1
30.48+00.3
3.86+00114
55.64++..11
0.7
0.1
30.20+00008
3.83+0029
56.58++0.35
0.2
0.3
30.20=0.334
3.82::0.37
56.66:#0226
0.5
0.3
30.27+0.37
3.83
55.80++0.12
0.7
0.3
30.59+0.12
3.85+00144
55.84+0.10
0.2
0.5
30.46++0.06
3.82+0012
56.00=00.1
0.5
0.5
30.86=00.20
3.86:10.23
56.80=10.09
0.7
0.5
31.10:L0.39
3.85
55.18++0.04
0.2
1.0
30.881L0.3
3.90
56.56110.35
0.5
1.0
30.81
3.84
56.34+:.22
0.7
1.0
30.90+0.19
3.88+0.24
55.78=00.15
0.2
1.5
30.90=0.26
3.87+0.36
55.98=0.34
0.5
1.5
31.04:0..16
3.88+0.41
55.82+00116
0.7
1.5
31.21+0.14
3.90+0.00
55.70+10.36
0.2
2.0
31.46+0.23
3.89
55.94+0.17
0.5
2.0
30.59+0.30
3.88+0.21
55.56+0.34
0.7
2.0
31.12::0.39
3.88+0029
55.7240.13
Table 16: VIBI Hlper-parameterr Tuning.
B
Purity (%) 4
Brevity +
Faithfulness (%) 1
HateXplain
0.2
0.1
32.77+0.31
4.34
66.67+0014
0.5
0.1
33.55+0..15
4.37++0.30
65.99++0.28
0.7
0.1
30.70+10.22
4.13+:0.40
63.95+:.20
0.2
0.3
32.77=0.27
4.40+10.3
65.2110.06
0.5
0.3
24.50+0..1
3.44
60.06+00.17
0.7
0.3
33.10+0.33
4.30++0.13
64.43+0.29
0.2
0.5
34.63+0.111
4.47+0005
65.01+0.07
0.5
0.5
30.35+0.33
4.10+:0.0
62.68+10.24
0.7
0.5
32.55+0.227
4.32+0.32
65.99+00.3
0.2
1.0
19.69
3.93+0.24
64.43+0.12
0.5
1.0
33.78+0022
4.48+00.9
64.04+0.30
0.7
1.0
33.8440.38
4.47+00.1
66.57+0.11
0.2
1.5
27.39-0.24
3.83+0.37
61.8110.38
0.5
1.5
35.22-00.26
4.52:10.19
65.60110.11
0.7
1.5
33.00+0.02
4.33+.0.18
65.99-0.19
0.2
2.0
33.45+0.12
4.36110.77
66.08+0010
0.5
2.0
33.9110.29
4.56110.2
67.25110.29
0.7
2.0
32.7510.23
4.30
66.47-10.00
31

---

Published as a conference paper at ICLR 2023
Table 17: VIBI Hyper-parameters Tuning.
B
Purity (%) 4
Brevity + Faithfulness (%) f
AG News
0.2
0.1
19.70=0.005
3.99::0.32
62.43+20.99
0.5
0.1
20.30+t0.09
4.02
65.66++0.35
0.7
0.1
0.25
4.07
66.58+0.36
0.2
0.3
19.36=t0.12
3.82+0.13
62.37+0005
0.5
0.3
19.60=0.32
3.99:40.06
63.55++028
0.7
0.3
19.68+0.114
3.88
63.62++0.02
0.2
0.5
20.53+0.23
3.95
62.17+0.13
0.5
0.5
15.84+0.09
3.53+0020
60.26-t0.31
0.7
0.5
17.66+0.09
3.72
61.05++0.2
0.2
1.0
21.42+0.32
4.08
64.87+0.32
0.5
1.0
19.07+0.24
3.80+0.03
6..76+00.24
0.7
1.0
20.41+00099
3.97
61.45+0025
0.2
1.5
18.91
3.87+0.05
64.87+0.16
0.5
1.5
19.61#0.14
3.92+:.25
63.75+0.17
0.7
1.5
19.44
3.91
61.25+0.18
0.2
2.0
20.20+
4.00
64.01+0.111
0.5
2.0
18.89+0.04
3.85+0.04
63.88+00.2
0.7
2.0
18.1440.19
3.81
62.63+00.3
Table 18: Hyper-parameters Tuning.
N
Purity (%) 4
Brevity J Faithfulness (%) 1
IMDB
50
25.94
3.21+00.01
69.26-t0.47
100
30.27+00.86
3.66-10.75
71.70++0.36
200
33.8
3.87+
68.78++0.36
HateXplain
32.84.10.11
4.24+:0.01
65.57+L0.81
100
33.13+0.09
4.23+-0.00
66.28
200
33.09+0.05
4.25+t0.01
66.12+t0.31
AG News
27.6240.02
4.90+E0.00
75.75=+0.08
100
27.61+0.03
4.91-:0.00
76.01=10.11
200
27. 27.63+10.00
4.91+00.00
76.01+10.06
32


Reviews:
[{"review_id": "lXWWqgDrbj", "reviewer": "Reviewer_Vmy2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new framework for ML interpretation. The goal is to choose a good subset of features (words/pixels) of the input data  to explain its prediction from a black-box classifier. The model contains three networks: a selector that selects important features, an explainer that learns an importance matrix between every feature and class, and an approximator used in the information theoretic lower bound. Through very extensive experiments (mostly on text data), the paper shows improvement of the proposed method compared to baseline methods on multiple evaluation metrics and settings. ", "summary_of_the_review": "Based on the comments above, I think this paper presents an interesting method for feature-based interpretability with extensive evaluation, and therefore I vote for acceptance. ", "strength_and_weaknesses": "**Strength**\n\nThe paper is clearly written. It is easy to understand and mathematically solid at the same time. The proposed networks and loss functions are very well motivated and make a lot of sense to me. The experiments are extensive: the paper studies several text classification tasks, all  being very useful in practice. There are a number of meaningful evaluation metrics, and the proposed method achieves huge improvement on all of these metrics. In the appendix, there are also extensive studies on loss functions, hyperparameters, and other tasks such as image classification.\n\n**Weakness**\n\nIt is unclear to me how $K$ is reflected in training. $K$ is used to define $\\mathbb{S}$ but later $x_{\\mathbb{S}}$ is approximated by $\\tilde{z}_x\\odot x$. It is unclear how accurate such approximation is, and it is irrelevant with $K$ anymore. Will constraints like letting $\\\\|\\mathcal{S}(x)\\\\|_1\\approx K$ work? \n\nSome minor things on notation. In eq(2) it would be more rigorous to use $\\tilde{y}_m$ instead of $\\bar{y}_m$. In addition, the symbol $\\circ$ is usually used to represent composition of functions, and the symbol $\\odot$ is usually used for element-wise product. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "lVhiwlDczuJ", "reviewer": "Reviewer_RWdr", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "- The authors proposed an interpretable model that combines the additive and instance-wise feature selection approaches.\n- More specifically, the proposed methods jointly train three components (explainer, selector, and approximators) to identify important instance-wise features and provide corresponding explanations per each class.\n- The proposed methods show consistent performance improvements across multiple text and image datasets over alternatives.", "summary_of_the_review": "Overall, this paper is easy to read and understand. Proposed method makes sense and the results are promising.\nBut it would be great if the authors can provide the tabular results to show the superiority over other works in tabular domains.\nAlso, it would be good to clarify the second loss better.", "strength_and_weaknesses": "Strength:\n- The proposed method can provide not only instance-wise feature importance, but also corresponding additive explanations. Also it can provide explanation for multiple classes simultaneously.\n- The experimental results are promising and consistently better than alternatives.\n\nWeakness:\n- The proposed methods can be applicable to any data types. As the authors said, there are many related works on tabular data. In that case, it would be good to provide the superiority of the proposed method on tabular data as well. \n- It is unclear how L2 can prevent to converge \\pi_x = 1 for all components. It would be great if the authors can explain better on it.\n- There are some other related works that also do not need to set K value for instance-wise feature selection (e.g., Yoon et al, 2019). So, it would be good to tone down on this contribution. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "GY0j99Syn4z", "reviewer": "Reviewer_qCbz", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work develops a new method to explain predictions from black-box ML models. The method learns two separate modules to perform the following: 1) output an attribution score for each feature and each class (similar to methods like LIME), and 2) output selection probabilities for each feature (similar to methods like L2X). Thus, the method (AIM) is thought to bridge two categories of explanation methods. (The authors refer to these categories as \"additive\" and \"instance-wise\", but these are incorrect names for the groups of methods they're referring to.) \n\nThe experiments compare the method to L2X, LIME and two other methods (VIBI, FI) and find encouraging results.", "summary_of_the_review": "The authors propose a new method to combine aspects of existing attribution- and selection-based methods. The method involves a complicated joint training routine with three separate modules, only one of which is used to generate explanations. The method lacks theoretical justification (what the method is optimizing for, e.g., in terms of mutual information) but obtains some positive experimental results. ", "strength_and_weaknesses": "--- Strengths ---\n\n- It's an interesting idea to merge selection- and attribution-based methods, as most existing model explanation work falls into one of these categories.\n- The AIM method provides fast explanations because it's trained in an amortized fashion (similarly to L2X). \n- The experimental results suggest strong performance. \n\n--- Weaknesses ---\n\nWriting and prior work\n- The authors frequently refer to two categories of explanation methods, \"additive\" and \"instance-wise.\" This is poor choice of terminology, neither term seems to mean what the authors intend. \"Instance-wise\" is synonymous with \"local\" (i.e., explaining individual predictions), and *all the methods discussed here* are instance-wise/local. And \"additive\" doesn't cover multiple methods supposedly in that category, such as vanilla gradients (Simonyan et al., 2013) or SmoothGrad (Smilkov et al., 2017). I think the correct terminology would be additive methods --> \"attribution methods\" and instance-wise --> \"selection methods.\" See [1] for a review of how such methods are related (section 6).\n- The authors further conflate instance-wise feature selection with the amortized fashion in which the selectors are often trained (e.g., in L2X). There are multiple parts of the paper revealing this confusion, see the abstract for example: \"instance-wise methods directly optimize local feature distributions in a global training framework, thereby being capable of leveraging global information from other inputs.\" This seems to allude to the explainer training routine, but you don't technically need a trained selector model: you could instead follow a greedy algorithm for each example to be explained, for example.\n- The authors mention that prior \"instance-wise methods\" (read: selection-based methods) have a strict reliance on selecting a fixed number of features. Not true, INVASE and Real-X both address this [2, 3].\n- Regarding LIME, the authors write that \"there is a chance that the perturbed examples behave undesirably, for example, to change the original prediction.\" That's not undesirable, it's exactly the point of LIME..... Seeing how the prediction changes helps determine how much the model depends on each feature.\n- The authors write that \"additive methods are inefficient since they optimize individual explainers for every input.\" We can't do this for every sentence in the paper, but I'll just pick this one to highlight some more issues. i) Many \"additive methods\" (read: attribution methods) are not based on optimization, e.g., nothing is being *optimized* on a per-input basis in Integrated Gradients. ii) Some of these methods are inefficient, like LIME, but not all. E.g., most gradient-based methods are on the order of a couple forward passes, so you can't make a blanket statement that they're all slow. iii) Earlier works have addressed the problem of slow attribution methods: [4], [5], and [6] are three examples of works proposing the use of learned explainer models. iv) This suggests that attributions methods are inherently slower than selection methods, but selection methods aren't naturally fast (see comment above about how you don't need a learned selector) - they just happen to have leveraged amortized learning earlier than attribution methods. \n\n[1] Covert et al., \"Explaining by Removing: A Unified Framework for Model Explanation\" (2021)\n\n[2] Yoon et al, \"INVASE: Instance-wise variable selection using neural networks\" (2018)\n\n[3] Jethani et al, \"Have We Learned to Explain?: How Interpretability Methods Can Learn to Encode Predictions in their Interpretations\" (2021)\n\n[4] Dabkowski et al, \"Real time image saliency for black box classifiers\" (2017)\n\n[5] Schwab & Karlen, \"Cxplain: Causal explanations for model interpretation under uncertainty\" (2019)\n\n[6] Jethani et al, \"FastSHAP: Real-Time Shapley Value Estimation\" (2021)\n\nMethod\n- I can follow the training procedure outlined by the authors, but ultimately I don't understand what we expect the result to be. After implementing the joint training procedure, does the result have anything to do with the information communicated by each feature, for example? Many well-regarded methods (L2X, Integrated Gradients, SHAP) provide some theoretical characterization of their results, including what properties they satisfy or what they represent in terms of information theory. This approach does not, and it has hyperparameters that can significantly affect the results ($\\alpha, \\beta$) with little thought put into how they should be selected. Explaining ML models is an ambiguous problem with many existing solutions, so at this point solutions with weak theoretical support are unlikely to gain traction. \n- It's odd that the method requires training both the selector and explainer, but that the experiments only use the explainer. It seems like this decision was made based on which version of the method performed best in experiments (Appendix D), not by design. It seems a bit wasteful and overcomplicated to train both, and perhaps a flaw in the method that the selector actually hurts the results. \n- I suspect that AIM suffers from the same \"prediction encoding\" problem that [3] exposed in L2X: that the explanation can encode the label via its selections, even with a single feature. If so, this would not be a good thing - it would reflect that the predictions with selected features are based more on which features are selected than the information contained in those features. The results in Table 8 and Table 9 with $K = 5$ show that this may indeed be a problem, because the faithfulness values (% agreement with the full-input prediction) are often >90%. Could the authors comment on this? I think a rigorous test for this issue would involve evaluating the method with $K = 1$, and using the Eval-X method from [3].\n\nExperiments\n- Given the abundance of model explanation methods, I don't understand the choice of baselines. L2X I understand, but why not some more common/simple ones like Integrated Gradients [7], occlusion [8] or RISE [9]? SHAP would also be nice to include but has similar computation cost as LIME.\n- The authors write that \"Except for AIM and FI that do not require specifying $K$, all the other baselines are trained at $K = 10$.\" LIME doesn't require specifying $K$ either, right? Again, if you wanted methods like L2X that don't require specifying $K$, there are two of them [2, 3].\n- Can the authors confirm that their faithfulness and log-odds metrics are calculated using the original classifier rather than the new learned one ($\\mathcal{G}$)? For both the text and image datasets?\n- The experiments with MNIST and Fashion-MNIST don't compare AIM with any baselines. Can the authors correct this and include some commonly used methods for images (IntGrad, occlusion, RISE)? \n\n[7] Sundararajan et al, \"Axiomatic attribution for deep networks\" (2017)\n\n[8] Zeiler & Fergus, \"Visualizing and understanding convolutional networks\" (2014)\n\n[9] Petsiuk et al, \"Rise: Randomized input sampling for explanation of black-box models\" (2018)", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster