TOWARDS AN OBJECTIVE EVALUATION OF THE TRUSTWORTHINESS OF CLASSIFIERS
Authors: Manish Chandra
Debasis Ganguly
Venue: Submitted to ICLR 2023 (2023)
Keywords: Model trustworthiness
Explainable AI

Abstract: With the widespread deployment of AI models in applications that impact human lives, research on model trustworthiness has become increasingly important, as a result of which model effectiveness alone (measured, e.g., with accuracy, F1, etc.) should not be the only criteria to evaluate predictive models; additionally the trustworthiness of these models should also be factored in. It has been argued that the features deemed important by a black-box model should be aligned with the human perception of the data, which in turn, should contribute to increasing the trustworthiness of a model. Existing research in XAI evaluates such alignments with user studies - the limitations being that these studies are subjective, difficult to reproduce, and consumes a large amount of time to conduct. We propose an evaluation framework, which provides a quantitative measure for trustworthiness of a black-box model, and hence, we are able to provide a fair comparison between a number of different black-box models. Our framework is applicable to both text and images, and our experiment results show that a model with a higher accuracy does not necessarily exhibit better trustworthiness.

Under review as a conference paper at ICLR 2023
TOWARDS AN OBJECTIVE EVALUATION OF THE TRUSTWORTHINESS OF CLASSIFIERS
Anonymous authors
Paper under double-blind review
ABSTRACT
With the widespread deployment of Al models in applications that impact human lives, research model trustworthiness has become increasingly important, a result of which model effectiveness alone (measured, c.g., with accuracy, FI, etc.) should not be the only criteria to evaluate predictive models; additionally the trustworthiness of these models should also be factored in. It has been argued that the features deemed important by black-box model should be aligned with the human perception of the data, which in turn, should contribute to increasing the trustworthiness of a model. Existing research in XAI evaluates such alignments with user studies the limitations being that these studies are subjective, difficult to reproduce, and consumes large amount of time to conduct. We propose an evaluation framework, which provides quantitative measure for trustworthiness a black-bo box model, and hence, we are able to provide a fair comparison between number of different black-box models. Our framework is applicable both text and images, and our experiment results show that a model with a higher accuracy does not necessarily exhibit better trustworthiness.
INTRODUCTION
Owing to the success and promising results achieved for data-driven (deep) approaches for super- vised learning, there has been growing interest in the AI community apply such models in domains such as healthcare (Asgarian et al. 20188] Spann et al. 2020; Yasodhara et al. 2020), crim- inal justice (Rudin) 2019) and finance (Dion al., 2020). As MI. models become embedded into critical aspects of decision making, their successful adoption depends heavily on how well different stakeholders (e.g. user or developer of ML models) can understand and trust their predictions. As result, there has been recent surge in making ML models worthy of human trust (Wiens et al.) 2019). and researchers have proposed a variety methods to explain ML models to stakeholders (Bhatt et al.) (2020), with examples such as DARPA's Explainable AI (XAD) initiative (Gunning al., 2019) and the "human-interpretable machine learning' community (Abdu et al. 2018).
Although standard evaluation metries exist evaluate the performance of predictive model, there no consistent evaluation strategy for XAI. Consequently, a common evaluation strategy is to show individual, potentially cherry-picked, examples that look reasonable (Murdoch et al. 2019)) and pass the first test of having 'face-validity' (Doshi- (Doshi-Velez & Kim 2018). Moreover, evaluating the ability of explanation convince human is different from evaluating its correctness, e.g., while / et al. (2018) believe that keeping humans out of the loop for evaluation makes it more fair and true the classifier's own view on the problem rather than representing a human's 's view, Gilpin et al. (2018) explain that non-intuitive explanation could indicate either an error in the reasoning of the predictive model, or an error in the explanation producing method.
Visual inspection on the plausibility of explanations, such as anecdotal evidence, cannot make the distinction as to whether a non-intuitive explanation is the outcome an error in the reasoning of the predictive model, that it is an error that could be attributed to an explanation generating model itself. Zhang et al. (2019) identify such visual inspections as one of the main shortcomings when evaluating XAI and state that checking whether explanation "looks reasonable" only evaluates the accuracy of the black box model and is not evaluating the faithfulness of the explanation. These commentaries relate to the inherent coupling of evaluating the black box model's predictive accuracy

---

Under review as a conference paper at ICLR 2023
with explanation quality. As pointed out by Sabbiia-Sikojaa & Bohanec (2018), the correctness of an explanation and the accuracy of the predictive model may be orthogonal.
Although the correctness of the explanation is independent of the correctness of the prediction, visual inspection cannot distinguish between the two. Validating explanations with users can un- intentionally combine the evaluation of explanation correctness with evaluating the correctness of the predictive model. Synthetie datasets are useful for evaluating explanations for black box models Oramas al. (019). By designing dataset in a controlled manner, it should be possible argue, with a relatively high confidence, that a predictive model should reason in a particular way; a set of 'gold' explanations can thus be created in controlled manner using data generation process. Subsequently, the agreement of the generated explanations with these true explanations can be mea- sured. For example, Oramas al. (2019) generate an artificial image dataset of flowers, where the color is the discriminative feature between classes.
Our work compares multiple underlying predictive models in terms of trustworthiness, rather than the XAI methods themselves. Evaluating whether the features deemed to be important by a predictive model conform with those by a human an intrinsically human-centric task that ide- ally requires human studies. However, performing such studies multiple times during the model development phase is not feasible. To this end, the major contributions our work are as follows.
Our Contributions. First, we generate synthetic dataset and its associated ground-truth expla- nations for multi-objective image classification task. We also manually create the ground-truth explanations for two image classification datasets, namely the MNIST '3 vs. 8' classification and the Mlant-Village (Mohanty et al. 2016) disease classification tasks, and for our text experiments we make of dataset of legal documents with existing ground-trute explanation units (Malik al., 2021) (dataset and code will be released).
Second, propose general framework quantify the trustworthiness of black- box model. Our approach is agnostic to both the explanation methodology and the data modality. In our experiments, compare the performance of predictive models both in terms of effectiveness and trustworthi- ness on synthetic and real-world datasets using data from two different modalities - image and text.
RELATED WORK
Several works have used synthetic datasets for evaluating XAI algorithms. Liu et al. 2021 released the XAI-BENCH suite of synthetic datasets along with library for benchmarking feature attri- bution algorithms. The authors argue that their synthetic datasets offer wide variety of parameters which can be configured to simulate real-world data and have the potential to identify subtle fail- ures, such as the deterioration of performance examples of how real datasets can be converted to similar synthetic datasets, thereby allowing XAI methods to be benchmarked on realistic synthetic datasets.
datasets with high feature correlation. They give
Oramas et al. (0199) introduce an8Flower, a dataset specifically designed for objective quantitative evaluation of methods for visual explanation. They generate two synthetic datasets, 'an8Flower- single-6e' and mnaFFlower-doubl-1222 with 6 and 12 classes respectively. In the former, a fixed single part of the object is allowed change color. This color defines the classes of interest. In the latter, a combination of color and the part on which it is located defines the discriminative feature. After defining these features, they generate masks that overlap with the discriminative regions. Then, they threshold the heatmaps at given values and measure the pixel-level intersection over union (loU) of a model explanation (produced by the method to be evaluated) with respect to these masks. We argue that the importance of each pixel as outputted by the XAI model is different, and that this information not captured by a simple technique, such as the pixel-level IoU of a model's explanations relative to the ground-truth explanation masks. In work, we propose two new metrics for evaluating predictive models in terms of their trustworthiness.
Some argue in favor of automated metrics where user involvement is needed, e.g., in the context of usability evaluation in the Human Computer Interaction (HCI) community, Greenberg & Buxton 2008) argue that there is risk of executing user studies in early design phase, since this can quash creative ideas or promote poor ideas. Miller et al. (2017) therefore argues that proxy studies especially valid in early development. Qi et al. (2021) indicate that "evaluating explanations
2

---

Under review as a conference paper at ICLR 2023
8 8
8
(a)
(b)
Figure 1: schematic description of the idea leveraged by the proposed trustworthiness metric on sample image
vs. classification task. a): Overlayed ground-truth (red) and the predicted feature weights (blue) of explanations obtained for two different black-box models. b): A zoomed-in view of selected regions from these two explanations showing the closest ground-truth from pair of predicted points p1 and p2 (left), and ps and p4 the right. The trustworthiness of the model the left is higher than that of the right, because the explanations for the former closely matches the human perceived discriminating feature (shown red). can be seen that the sum of the distances between the two line segments for the figure on the right is higher than that of the left. This results in a higher quantitative value of the metric for the image on the left.
objectively without a human study is also important because simple parameter variations can easily generate thousands of different explanations, vastly outpacing the speed of human studies".
User studies are not only relatively hard to run, they may be of limited value (Poursabzi-Sangdehh et al., 2021) and may suffer from confirmation bias (Wang et al., 2019). Rosenfeld (2021) present four objective XAI measures to quantify XAI effectiveness either on their own or in conjunction with user studies: 'D', the performance difference between the agent's model and the performance of the logic presented as an explanation, 'R', the number of rules in the agent's explanation. 'F', the number of features used to construct the explanation and 'S', the stability of the agent's 's explanation.
Amiri et al. (2000) proposed a methodology to create a synthetic dataset representing ground-truth explanations (GTE), and used it for evaluating LIME. Our work is different in the sense that focus on evaluating predictive models rather than the XAI method. We, therefore, define GTE in terms of human perception of important features, e.g. the patches the leaves of plants that identify a particular disease. Yang & Kim (2019) release a carefully crafted semi-natural image dataset with ground-truth for evaluating interpretability of methods. They also propose three complementary metrics to evaluate interpretability a) Model Contrast Score (MCS), which measures the perfor- mance of interpretability methods across models, b) Input Dependence Rate (IDR), which accounts for when an interpretability method should "react" to two different inputs, and c) Input Indepen- dence Rate (IIR), which concerned with when an interpretability method should not "react" to two different inputs. Our work, however, does not focus on evaluating XAI methods in terms of identifying false positives (features that are incorreetly attributed as important) or false negatives. We rather focus evaluating predictive models with respect to their trustworthiness, keeping the XAI method constant. Yang & Kim 2019) train a model in a controlled manner and establish what regions of the input the model is and isn't 7 attending based on whether they' re able to capture such important and non-important features in their explana- tions of the model. In our work, however, we define GTE in terms of human perception of important features, and then compare different predictive models in terms of the extent to which they agree to this ground- truth for making their predictions.
The XAI methods are then evaluated
3 TRUSTWORTHINESS EVALUATION OF MODELS
Constructing an objective ground-truth of trustworthiness. Existing studies have mainly used the trustworthiness measure to compute the fitness of an XAI model itself, e.g., [hen et al. (2018) compare the stability of feature weight distributions across samples of data instances computed by different explanation methodologies, and report that L2X leads to the most stable results. The novelty of our work lies in objectively evaluating the trustworthiness of black-box models. To this end, our evaluation framework first requires a manually created ground-truth defined for small number of data instances, e.g., for images, this ground-truth is in the form of regions (constituted of
3

---

Under review as a conference paper at ICLR 2023
8
{ (b)
{
:
(a)
(c)
(d)
(c)
Figure 2: illustration how we obtain the ground-truth explanations from manual assessments pro- vided by three different annotators for the 3-vs-8 classification task. Each annotator required to select 3 control points (shown as red dots), based on which region was selected as the ground-truth. The final ground-truth annotation
average image of the three individual ones. The red dots magnified in the figure so that they can seen conveniently. The images, in order from left right, correspond Original Image, b) Annotator 1, c) Annotator 2, d) Annotator 3, e) Ground-trund
(a)
(b)
(c)
(d)
(e)
(I)
Figure 3: Synthetic data and corresponding ground-trun explanations for a multi-objective image classifica- tion task. Task I: Shape classification and Task 2: To classify if filled or not. The images, order from to and Filled='1', b) Ground fround-truth for the shape
right, correspond to a) Generated image where Chape='Ellipss: recognition task, Ground- for recognizing whether filled, d) Generated image where Shape= and Filled='O', Ground-truth for the shape recognition task, and Ground-truth for recognizing whether filled.
pixels) within an image indicative of discriminative features and for texts, is in the form of set of sentences.
A Distance Weighted Overlap Measure. We present approach by referring to data units as pixels being associated with 2D coordinate. However, the same principle also applies for text where each explanation unit (a word or sentence etc.) is specified by single dimension (namely its offset or index in the text). Let 0 an input x Rhx (or simply E IR" for text) to of the & classes, where h and coordinates of an input feature (for an image, and w is an integer offset of a word or a sentence in text). Let denote the objective trustworthiness ground-truth of input as 7(x) (this ground- truth is either manually annotated or synthetically generated, as described later in Section 4.
+ Zy be data-driven black-box model that classifies
the 2D
The ground-truth 7(x) of an input x is a binary matrix of the same dimensions that of x, in other words, T r(x) {0, 11 Let the local feature weight predictions outputted by some XAI method be denoted by 0x.0. For the particular case image classification, the weight distribution corresponds to pixel-level importance and hence is of the same dimension that of x, i.e., Φx.0 E ghx"" Similar to Ribeiro et al. (2016). we select subset of the most important features from the predicted feature weight distribution matrix. A convenient way of doing this is select the top-(100 x k) percentile weights, where E [0, is parameter (we denote this set with the simple notation dv.). The set du thus constitutes tuples of the form (h, w, c). where E [0, 1] the importance of the feature at index (h, w).
The problem with simple overlap (e.g., Jaccard- metric (Oramas al. 2019) is that it cannot take into account the relative distances between the predicted and the ground features, and as a result, may end up penalizing a predicted feature within a near vicinity a ground-truth one. Figure illustrates the idea, where revisit the case-study of '3 8' classifier; the human perceived discriminating feature between a '3' and an '8' is the left part of the looped curve of an '8', the absence of which transforms it to '3'. Figure shows that the model the left results in a more trustworthy prediction, and this is captured by the sum of the distances of predicted (to be an important feature) point from its nearest ground-truth point.
Precision Measure. The first metric that we propose precision- where we measure the fraction of features that are deemed important both by the predictive model and by humans. More concretely, we start by dividing Φù and r(x) into windows of shape m X m. to get of and r(x)"" respectively. For each predicted window, find its closest window in the ground-truth set, and then add up the similarities (inverse distances). We also make use of the feature importance values compute weighted average take into account the confidence of the predictions of feature
4

---

Under review as a conference paper at ICLR 2023
Algorithm 1: Algorithm to compute trustworthiness metrics TWP and TWR.
Input: Ground-tran truth explanations for evaluation ix.. (7(x), ].
Input: A black box model 8, and Incal explanation methodology o, e.g., LIME SHAP etc.
Input: v(x, (. si, The importance pixel (6, image X.. obtained by explanation method.
Input: &
The top-(le 100) pereentile of the feature weights used define the prodicted important features.
Input: mi: size of window for trustworthiness metrics computation
Output:
begin
TWP alt
TWRa1
foreach x in X, do
@x.00 The local explanation for the black hox model 0
Sort each element (h, mi E 9x. by descending values o, and retain the top-(k 100) percentile; call this dx-
TWP
TWR
Ze 0: Ze 0 0// Norma 1 iiaatio ion factora for TWP and TWR
0g +
all m windows die
=(x) Set all m x windows in ((x)
foreach p in 6% if R(,,
c)
then
g +- Clusest window in T>x)" R to the window
c w.)epp
[p]
TWP TWP =dpppd ((p.9)
2p 2, c
foreach in r r(x) do
[( : (N', we') ((x) a/
Closest window in o% to the window g d' E,)
TWR < TWn d' d(G.p)
Zr 7,
TWP /2p TWR TWR/Z,
TWP olI TWP TWP
TWRa: +- -WRR Rall TWR
TWP TWP.l//X.l; TWRe Average over the evaluation
Figure 4: Sample annotated ground-truth explanation from plant disease classification task.
importance. Formally speaking,
Ie c.e d(p.9)
PEdm
TWP
where y is the closest window in r(x)" to the window p, (1)
where d((p, g) is the average distance between the set of points in and those in and e is the average feature importance value of window p. We call this metric TWP (abbv. for TrustWorthiness Precision) because of its similarity with the set-based definition of the precision measure. The
denominator denotes the set of most important (predicted) features, whereas the numerator denotes weighted correctness measure (similar to the true-positives).
Recall Measure. The second metric that we propose measures the fraction of human perceived important features that deemed important by the predictive model under consideraion. For mea- suring soft recall, we start from each window in the ground-truth and find its dual (the closest win- dow in the predicted set). The dual has the importance score d' We then aggregate the similarities

---

Under review as a conference paper at ICLR 2023
XAI
FF
CONY
Dataset
Method
Fidelity
F1
TWP.
TWR,
TWP,
TWR4
Fidelity
F1
TWP:
TWR,
TWP,
TWR
Syn (shape)
LIME
SHAP
0.97
0.56
0.9867
0.7654
0.7672
0.7603
0.7685
0.1303
0.1303
0.1370
0.1222
0.36
0.67
0.9933
0.6749
0.7442
0.7297
0.7586
0.1276
0.1242
0.1342
0.1167
Syn (illie)
LIME
0.34
L,0000
0.9165
0.8000
0.1297
0.1343
0.1207
0.36
1,0000
0.9201
0.9166
0.7488
0.7667
0.1269
0.1327
0.1180
0.1344
3-v3-8
LIME
SHAP
0.93
0.41
0.9859
0.3425
0,4653
0.6197
0.3803
0.0786
0.0587
0.1314
0.0723
0.29
0.46
0.9945
0.6197
0.4084
0.7261
0.4528
0.099
0.0689
0.1270
0.0850
Table 1: Comparison of two predictive models in terms of proposed metrics, for synthetic dataset and MNIST 3 vs dataset. The best trustworthiness values obtained for each model with LIME are bold-faced, whereas the ones obtained with SHAP are both bold hald-faced and underlined.
XAI
AlexNot
ResNet-18
ResNot-34
Method
Fl
Fidelity
TWP, TWR,
FI Fidelity
TWP, TWR,
Pidelity TWP, TWR,
SHAP
0,9670
0.88
0.2237
0.8419
0.9892
0.74
0.1880
0.7678
0.9902
0.79
0.1683 0.7258 0.1120 0.6425
LIME
0.77
0.1156
0.6847
0.67
0.1146
0.6464
0.52
Table 2: Comparison of two predictive models terms of proposed metrics, for plant disease classification task. Similar bold-facing and underline convention as in Table
(inverse distances) over each ground-truth point. More formally,
Egerm e d(g.r)
TWR
where p is the closest window in 0% to the window g-
(2)
Egerg
Adopting a similar naming convention, we name this metric as TWR (abbv. for TrustWorthiness Recall). Algorithm|| (onorithm[[summmann summmriiess the steps of computing TWP and TWR for image domains. For our experiments, we only consider positive SHAP values. This is because negative ones imply that those pixels are contributing negatively towards the prediction but for our ground-truth explanations, only annotate those regions that should contribute positively towards the prediction.
4 EXPERIMENTS
The objective of our experiments is to demonstrate that the proposed metrics provide a comple- mentary dimension of evaluating predictive models in addition to effectiveness, e.g., accuracy, precision, recall etc. A higher value of both TWP, TWR their harmonic means TWF 2TW.-TWR/ TWR/TTTP TWR) (F-score like combination) along with a high accuracy (or F-score) value should be indicative that model is both trustworthy and effective.
Synthetic Dataset. We synthesize a dataset in way such that the 'gold' explanations can be created with a controlled data generation process. Specifically, we generate dataset for multi-task image classification wherein the discriminative attributes are i) the shape type of the figure, and ii) whether the shape is filled or not. In particular, the shape of the objects in our dataset belongs triangle, b) ellipse and c) quadrilateral. After generating
to either of the three following types: an object a particular shape, the object is either filled up or left unfilled. The dimension of cach generated grayscale image is 28 x 28. We formulate the problem as multi-task classification problem, where the task of a predictive model, given an input image, is to classify the shape-type and predict whether the shape is filled not. The ground-truth explanation for the shape recognition task is the boundary of the shape itself, whereas for predicting if a shape is filled or not, the entire interior region is considered to be the ground-truth This is illustrated in Figure 3
MNIST 3 vs 8 Dataset. We use a subset of MNIST hand-written digits dataset comprising images from the two classes and 8, the task being to distinguish between them (Chen et al. 2018). The images of 8 contain two closed loops while those of 3 contain two open loops; hence, the two arcs on the left of the images of 8 that complete the loop are the discriminative regions for 3-vs-8, For ground-truth explanations, top, one at the middle and one at the bottom, such that when everything to the right of the piece-wise linear curve connecting those 3 points is removed, end up with those two discriminative arcs. The
ask the annotators pick three points the images of 8, one the

---

Under review as a conference paper at ICLR 2023
Model (FI)
Metric
GT-1
GT-2
GT-3
GT44
GT-5
Avg
TWP 0.6049
0.3964
0,4525
0,4694
0.3646
0.4576
BERT+LSTM (0.7036)
TWR 0.1709 0.1825
0,1085
0.1504
0.1893
0.1603
TWP 0,6015 0.3155 0,5219
0.4532
0,3502
0.4485
RoBERT&+LSTM (0.7473)
TWR 0.1636 0.1536 0.1351 0.1458 0.1746 0.1545
Table 3: Explanation evaluation on the ILDC dataset: the explanations of the hierarchical transformer- models were obtained with the occlusion- hased method[[ietat (206)) the 5 different versions of the ground- truth annotated 5 different legal experts. 'Avg' denotes the average values computed over the individual measures each the ground-truths.
8
8
(a)
Figure 5: Qualitative model explanations outputted by SHAP for the multi-task classification the syn- thetic dataset and the real datascts. The images, in order from left right, correspond a) 'Shape prediction' SHAP explanation for FF (center) and CONV (right), b) 'Fill prediction' SHAP explanation for FF (center) and CONV (right), and c) SHAP explanation for FF (center) and CONV (right) for the 3-vs-8 classification task.
assignment of images to annotate is done in such a way that one image annotated by exactly annotators. We use the pixel coordinates of the three points get the two left ares mentioned above. We take all three (from three different annotators) such left ares and compute the average of them to obtain our final ground-trui -tuth explanations, as shown in Figure 2
It is worth mentioning that defining the ground truth for instances of some classes is more convenient than defining it for other classes in the same task. For instance, is easier to define the ground-truth for the class '8' (in terms of the presence of certain region within an image) rather than defining for the class '3' because in that case, it is not the presence but the absence that needs to be considered, which is not so convenient. A total of 70 images of hand-written 8 were annotated.
Plant-Village Village Dataset. Plant Plantailllaee (Mohanty et al. 2016) is plant disease classification dataset of 54,309 images across 14 plant species and 38 classes (26-12 discase-healthy), The original task is predict the plant type along with the disease (each combination indicating case, the predictive model should be making use of information both from the patches the leaves (indicative of disease) along with the discriminative features of the leaves themselves (indicative of the plant type). However, this also means that the explanations should then cater to both the combination of the leaf and the disease types, and result they may be more subjective.
class). In such
To simplify the task, we grouped together the individual classes by the disease types to create 'super- classes', e.g., 'potato early blight' and 'tomato early blight' gets merged into single class "carly blight'. Application of this grouping operation along with the removal of classes corresponding to healthy leaves yielded 16 classes total. This contributed to simplifying the process of manually annotating the ground-truth explanations, because the simplified prediction task should then focus only on the patterns of the leaf patches to identify the correct disease. A sample annotated ground- truth explanation is shown in Figure In total, we annotated 10 images from each class.
Indian Legal Documents Corpus (ILDC). The final dataset that the ILDC MMalkk et al., 202I) corpus, which is a collection of case proceedings from the Supreme Court of India (SCI). Given document input, the prediction task identify the decision of
for experiments is

---

Under review as a conference paper at ICLR 2023
(a)
Figure 6: Visualizing sample images from two different ranges of the trustworthiness (TWF) scores on a) AlexNet (FI 0.9670) and b) ResNet-34 (Fl 0.9902). The TWF scores increase one goes from the top- row to the bottom-row. The row shows sample image from the set of images where TW F E [0.5, 0.75), whereas the second row shows sample where TWF F 2 0.75.
FF
CONV
Dataset
TWP TWR
TWP TWR
Syn (Shape) 0.5070 0.7057 0.5609 0.6589
Syn (Filled)
0.5814 0.6741 0.6473 0.6611
3-v8-8
0.2990 0.6143 0.3624 0.5624
Table 4: Ablation study which treats feature importance uniformly, i.e., when and cf to in Equations and respectively). For these results, the XAI method employed was SHAP.
the judiciary bench (acting as the ground-truth). In addition to the ground-truth decision, a separate set of 56 documents (IaDGea)ert) from the test set documents are annotated by different legal experts. Each annotation constitutes set of sentences that a human or an Al model should focus on to arrive at the decision.
Models and Parameter Settings. The objective in experiments is compare the predictive models in terms of their trustworthiness. We do not intend to evaluate the effectiveness of the XAI methods themselves. Therefore, fair comparisons where the explanations are also obtained with the same XAI methodology. e.g., comparing the TWP TWR values computed with different explanation methodologies is not fair. In particular, for our experiments we employ two different explanation methodologies namely SHAP and LIME.
only be made between predictive models
As instances of the black-box model for synthetic dataset and MNIST '3 8 dataset, we employ two different methodologies, namely i) a feed-forward network with one hidden layer (hereafter, re- ferred to FF model), and ii) 2D convolution layers in addition the feed-forward ones (hereafter, referred to CONV model). More specifically, the FF model has one hidden layer of 8 neurons and the CONV model has 2 convolution layers of kernel size 3, the first one having the number of output channels of and the second one 5. The convolution layers are followed by fully-connected layer with neurons and finally an output layer specific to the task (sigmoid for the 3-vs-8 classi- fier, whereas softmax of dimension 3 and a sigmoid for the multi-class objective of shape and fill prediction). For plant disease classification task, we employ AlexNet, ResNet-18 and ResNet-34. We fine-tune the instances of these models pretrained from the ImageNet datset. For ILDC, two hierarchical transformer models with RoBERTa and BERT as the transformer units. The [CLS] representations of each text chunk (refer to Malik al. (021) for more details the text chunking) are then fed into BiGRU and then through a dense layer to yield sigmoid probability. We run our experiments on ILDCmatts dataset and use the same hyper-parameter settings as used by the authors. As the explanation method, we use the occlusion method (Li et al.) (016)) as also used in Malik et al. (2021). Our experiments with LIME used the number of image segments as 20 for the synthetic and MNIST 3 vs 8 datasets, whereas this was set to 1000 for the plant disease classification task owing to the larger image sizes.
To find how faithfully the local explanation models approximate the primary models, we randomly sample 100 instances from the test set and compute the difference of accuracy between the original predictions and the new predictions after masking out the top 25% important features (as ranked by by explanation method). These fidelity scores (Pope et al. 2019) are reported in
Results.
Tables and 2 They show that the difference in accuracy is always > 29%. thereby establishing
8

---

Under review as a conference paper at ICLR 2023
the faithfulness of the XAI models- LIME and SHAP for the tasks in our experiments. Figure shows the pixel-level explanations as outputted by SHAP for both FF and CONV models, for the synthetic dataset. It can be noted that both the models attend to the interiors of the shapes for task (fill predictionn. But when it comes to task (shape prediction), the FF model seems to be attending the relevant pixels better than the CONV model. Specifically; for the fill prediction task, a majority of the high intensity red points (high SHAP values) lie on the circumference or in close proximity of it. However, for the shape prediction task, such points are spread out farther away from the human-intee interpretable regions. Our proposed metrics are able to capture this behaviour as depicted in Table It should be noted that despite the TWP and TWR values being slightly different when we use SHAP versus LIME, the relative rank of the models based on the performance with respect to these metrics still the same (compare TWP and TWR columns between the consecutive rows, e.g., rows and 2 etc.).
An important observation worth noting is that despite having higher accuracy, the CONV model fails to attend to the human-interpretabl regions within the images. Therefore, somewhat surprisingly, the FF model's way of reasoning is more in parity with that of a human in comparison to a 2D convolutional network.
Figure 5 shows the pixel-level explanations as outputted by SHAP for both FF and CONV models, for real dataset. It can be noted that the CONV model assigns a substantially higher importance to the two left ares (see Figure 2 than the FF model. Therefore, in this task, the abstractions from the data that CONV learns agrees more with that of human's 's than is the case for FF. Unlike the synthetic dataset, here the model with higher accuracy attends to the pereeieee discriminatory regions in better manner as compared to the lower accuracy one. This is again reflected in our proposed metrics, as depicted in Table regardless of the choice of the XAI model.
F shows the pixel-level explanations as outputted by SHAP for AlexNet and ResNet-34 mod- els. on the plant disease classification task. It can be observed that for predictions yielding the highest TWF values (the bottom-most row), AlexNet shows better agreement with the ground- truth than ResNet-34, as can be seen from the presence of a considerable quantity of red regions (i.e., where the model focuses most) even outside the leaf. This shows that a model with higher effectiveness may not necessarily be attending to a set of data units that a human would focus This is also seen for the top row Figure [6 where we see that a model with lower effectiveness (AlexNet) despite showing the presence of attention regions outside the leaf (thus the lower TWF scores) shows a denser distribution of focus regions inside the leaf nonetheless. In contrast, in ResNet-34 the distribution of focus regions is not only sparse but even outside the leaf.
Table 3 shows the comparison of TWP and TWR scores for the hierarchical transformer model ex- planations obtained using occlusion method with 5 different ground-truth annotations, for ILDC dataset. Observations similar to the plant disease classification task can also be made for the Court Judgement Prediction and Explanation (CJPE) task, i.e., model with lower effectiveness (BERT+LSTM) yields higher trustworthiness scores (TWP and TWR).
Ablation Study. We now measure the metrics TWP and TWR without using identical feature importance, and see if this is able to produce a reversal in the relative trustworthiness estimation across models. In other words, the Φ set in this setup now comprises elements of the form (h, w, 1), ie., the importance e of each pixel is set to uniform value of 1. The results of this ablation, using SHAP as the fixed XAI method, is shown in Table 4 We observe that, contrary to the true metric values of T the TWR values in Table of of the CONV model on the 3-vs-8 classification task is worse than FF, which shows a reversal of the relative ordering of trustworthiness. This, in turn, shows that taking into account the feature weights themselves obtained from an explainer model is indeed important.
Concluding Remarks. In this study, we proposed a general approach to quantifying the trustwor- thiness of a predictive model and apply it on synthetic dataset and three real datasets. We proposed two new metrics for the quantitative evaluation of model trustworthiness, and demonstrated that the model with higher effectiveness may not necessarily be attending the most human- interppraree features, thereby hinting towards the importance of looking beyond model effectiveness measures, such accuracy, precision, recall etc.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Ashraf Abdul, Jo Vermeulen, Danding Wang, Brian Y. Lim, and Mohan Kankanhalli. Trends and trajectories for explainable, accountable and intelligible systems: An hei research agenda. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI '18. pp. 1-18, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 9781450356206. doi: 10.1145//17355743174155 B173574.3174158
URL
11457
Shideh Shams Amiri, Rosina O. Weber, Prateek Goel, Owen Brooks, Archer Gandley, Brian Kitchell, and Aaron Zehm. Data representing ground-truth explanations to evaluate XAI methods. CoRR, abs/2011 .99992, 2020.
wttp://wwwwrgor///o///o//o 2trg//bb/2211... 09892
Azin Asgarian, Parinaz Sobhani, Ji Chao Zhang, Madalin Mihaileseu, Ariel Sibilia, Ahmed Bi- lal Ashraf, and Babak Taati. A hybrid instance-based transfer learning method. ArXiv, abs/1812.0100 2018.
Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, José M. F. Moura, and Peter Eckersley. Explainable machine learning in deployment. In Proceedingy of the 2020 Conference on Fairness, Accountability, and Trans- parency, FAT* '20, pp. 648 657,,,775 New York, NY, USA, 2020. Association for Computing Ma- doi: 10.1145/33510953335666
chinery. ISBN 978 47876763363
htw.hhtts::/deeo orgg
10.1145/3351088 3375624
Jianbo Chen, Le S., Martin W., and Michael Jordan. Learning to explain: An information-theoreticc perspective on model interpretation. Proc. of ICMI.' 18, volume 80, pp. 882-891, 2018.
M.F. Dixon, I. Halperin, and P. Bilokon. Machine Learning in Finance: From Theory to Prac-
tice. Springer International Publishing, 2020. ISBN 9783030410681. hoLlhttss://boot
google co. in/books http://oooooocoooooooooooee 1d=0pruDAAAAEAJ
Finale Doshi-Velez and Been Kim. Considerations for Evaluation and Generalization in Inter- pretable Machine Learning, pp. 3-17. Springer International Publishing, Cham, 2018, ISBN 978- 3-319-9813144.
doi: 10. 1111100/977883311-8811111 978 378-3-319-9811--
URL sttps://aci.orgg/
Leilani H. Gilpin, David Bau, Ben Z Yuan, Ayesha Bajwa, Michael Specter. and Lalana Kagal. Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA), pp. 80-89. IEEE, 2018.
Saul Greenberg and Bill Buxton. Usability evaluation considered harmful (some of the time). In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '08, pp. 111-120, New York, NY, USA, 2008. Association for Computing Machinery. ISBN 9781605580111. doi: 1357054.135701
(1145//135705.13577777 URL
David Gunning, Mark Stefik, Jaesik Choi, Timothy Miller, Simone Stumpf, and Guang-Zhong Yang. Xaix#*20014explaiaabbee artificial intelligence. Science Robotics, 4(37) 1200, 2019. URL https://wwwt hwwp://wwwwwc/o wwwwwcewww.eccee.....e..e org/ doi/ http://...1////rrrotttoooo aayp:tiiss.aa...00000
Jiwei Li, Will Monroe, and Dan Jurafsky. Understanding neural networks through representation
erasure. CoRR, abs/1612.08220, 2016.
Yang Liu, Sujay Khandagale, Colin White, and Willie Neiswanger. Synthetic benchmarks for sci- entific research in explainable machine learning. In Thirty-fifth Conference on Neural Infor- mation Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL https: //oeereview.
d d 1vr 14ffhFg
Vijit Malik, Rishabh Sanjay. Shubham Kumar Nigam, Kripa Ghosh, Shouvik Guha, Amab Bhat- tacharya, and Ashutosh Modi. Ilde for cipe: Indian legal documents corpus for court judgment prediction and explanation. In Proceedings of the 2021 Conference The Joint Conference
10

---

Under review as a conference paper at ICLR 2023
of the 59th Annual Meeting of the Association for Computational Linguisties and the 11th In- ternational Joint Conference on Natural Language Processing (ACL-IJCNLP 2021), Bangkok, Thailand (Online), 2021. Association for Computational Linguistics.
Tim Miller, Piers Howe, and Liz Sonenberg. Explainable AI: Beware of inmates running the asylum. In IJCAI 2017 Workshop on Explainable Artificial Intelligence (XAI), 2017. URL http:// ion- inma tes .
srople.eng unimelb edu, httpt//meeee/ppp////ppo////////a
Sharada Prasanna Mohanty, David P. Hughes, and Marcel Salathé. Using deep learning for image- based plant disease detection. CoRR, abs/ Mass/600.03166 169,,,,,,, 2016. URL fttp://arxiv:
abs fass/1604. 03169
W. James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu. Defini-
tions, methods, and applications in interpretable machine learning. Proceedings of the Na- fional Academy of Sciences, 1 16(44))220011-22000 2019. doi: 10. 10.1073/pnas. 111.////a..900055111111111 1900654116. URL doi abs /ttp://////ab//.111///////////// 10w../as...pp.........m 990055116
https: http://wwwwwpttt.
Jose Oramas, Kaili Wang, and Tinne Tuytelaars. Visual explanation by interpretation: Improving
visual feedback capabilities of deep neural networks. In International Conference on Learning
Representations, 2019. URL https:tpp// http::///peneeiieeeeeeeemee net.ieewwweeeteeeeeeeeeeeeeeee fwr.moormmmoom htte:///oooummmniiiiii
Vitali Petsiuk, Abir Das. and Kate Saenko. Rise: Randomized input sampling for explanation of
black-box models. In Proceedings of the British Machine Vision Conference (BMVC), 2018.
Phillip Pope, Soheil Kolouri, Mohammad Rostami, Charles E. Martin, and Heiko Hoffmannn Explainability methods for graph convolutional neural networks. In 2019 IEEE/CVF Conference 2019. doi: 10.1109/
on Computer Vision and Pattern Recognition (CVPR), pp.
Forough Poursabzi- Sangdehh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Wort- man Vaughan, and Hanna Wallach. Manipulating and measuring model interpretability. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI '21, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450380966. doi: URL/ttps:: //o::////oo.///// org/10 1141/3111111111 344431444515
01144//341174.4455555
Zhongang Qi, Saeed Khorram, and Fuxin Li. Embedding deep networks into visual explanations. Artificial Intelligence, 103:103435, 2021. doi: 10.1016fj.artim22200.0444. URL https:
Marco Tulio Ribeiro. Sameer Singh, and Carlos Guestrin. "why should trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference Knowledge Discovery and Data Mining, KDD '16, pp. 1135- 1144, New York, NY, USA, 2016. Association for Computing Machinery.
M. Robnik-Sikonja and Marko Bohanec. Certurbation-based explanations of prediction models. In Human and Machine Learning, 2018
Avi Rosenfeld. Better metrics for evaluating explainable artificial intelligence. Proceedings of the 20th International Conference on Aulonomous Agents and MultiAgent Systems, AAMAS '21 pp. 45-50, Richland, SC, 2021. International Foundation for Autonomous Agents and Multiagent Systems. ISBN 978 450383073.
Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1:206-211 225, 05 2019. doi: 10.
1038/642256-019-0048-x.
Ashley Spann, Angeline Yasodhara. Justin Kang, Kymberly Watt, Bo Wang, Anna Goldenberg, and Mamatha Bhat. Applying machine learning in liver disease & transplantation: A comprehensive review. Hepatology, 71, 01 2020. doi: 10.1002/hep.311 103.
Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y. Lim. Designing theory-driven user- centric explainable ai. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI '19, pp. 1-15, New York, NY, USA, 2019. Association for Com- puting Machinery. ISBN 9781450359702. //doi.org/// 100.0011155332000000 330079900000333803
doi: 00.1145//29000533000833 URL https:

---

Under review as a conference paper at ICLR 2023
Jenna Wiens, Suchi Saria, Mark Sendak, Marzyeh Ghassemi, Vincent Liu, Finale Doshi- Velez, Kenneth Jung, Katherine Heller, David Kale, Mohammed Saced, Pilar Ossorio, Sonoo Thadaney- Israni, and Anna Goldenberg. Do no harm: roadmap for responsible machine learning for health care. Nature Medicine, 25, 08 2019. doi: 10.
Mengjiao Yang and Been Kim. BIM: towards quantitative evaluation of interpretability methods with ground truth. CoRR, 09701
2019. URL http:///arxiv.oog//a////000
Angeline Yasodhara, Victor Dong, Amirhossein Azhie, Anna Goldenberg, and Mamatha Bhat. Iden- tifying modifiable predictors of long-term survival in liver transplant recipients with diabetes us- ing machine learning. Liver Transplantation, 27, 10 2020. doi: 10. 10.1002/1.25930.
Hao Zhang, Jiayi Chen, Haotian Xue, and Quanshi Zhang. Towards unified evaluation of explana- tion methods without ground truth, 2019, URL httpp/ss/aar....rrr/////11......
12


Reviews:
[{"review_id": "lEgf-c2JS5", "reviewer": "Reviewer_KrGW", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work develops a framework to evaluate the trustworthiness of a model using XAI tools and TWP, TWR that are developed in this work. Then, it illustrates an application of their framework to several real-world settings.   ", "summary_of_the_review": "This work has important logical flaws and I cannot recommend this work for publication even after a major review. ", "strength_and_weaknesses": "Strengths: \n\n-- Interesting and important topic. As rightly pointed out having XAI methods are not sufficient to argue whether a model is trustworthy or not. \n-- This work is not about evaluating XAI models, it is about the evaluation of a trained ML model. This adds a new dimension to XAI literature. \n\nWeaknesses:\n\n-- there are critical logical flaws in the arguments and models. \n-- the models developed in this work do not solve the problem raised in the motivation. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "aBbdA0fwswX", "reviewer": "Reviewer_G8bv", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a quantitative measure of trustworthiness of a black-box model trained on text or image datasets. This measure requires a set of ground-truth explanations to compare them against explanations provided by an explainable artificial intelligence (XAI) method applied to a black-model model. After introducing their measure, the authors test their measure on a set of datasets. ", "summary_of_the_review": "The authors have tried to provide a solution to a very difficult problem. Both proposed measures, in their current states, are only ad-hoc solutions.", "strength_and_weaknesses": "Strength\n\nIt is quite important to come up with a measure for trustworthiness of trained models. The authors have motivated their study nicely, and presented their measures in a clear manner.\n\nWeaknesses\n\nBoth measures are straightforward. This could be an advantage, but this also causes them to overly simplify the concept of providing explanations, or for that matter, measuring trust. The ground-truth explanations for images are obtained from experts. There could be a few concerns there: First, creating a binary matrix for an image is a tedious task that is prone to errors. Second, when looking at an image, the humans look at the pattern that are caused by a collection of pixels not just single points (e.g., a skin lesion where the doctors look at the symmetry or the size of the lesion). When presented with same images that are transformed (e.g., rotated) the ground-truth would completely change.\n\nA few questions for the authors:\n\n1. SHAP also returns negative weights. In that case, do you use the absolute values of these weights as the feature importance?\n\n2. As authors clearly stated on page 6, \"[...] it is not the presence but the absence that needs to be considered [...]\" This is very important, and I believe cannot be addressed with the current measures. Am I right?\n\n3. Both measures are obtained by averaging over all samples in the evaluation set. Could the sample variance or the outliers would be of more interest than averages?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "W9651b_SE3", "reviewer": "Reviewer_isth", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors provide a quantitative measure to check the correspondence of feature importance as detected by a model with human perception of a feature's importance, which is incorporated in a framework to quantify model trustworthiness. ", "summary_of_the_review": "I enjoyed reading the paper because it addressed the problem with clarity, and due to the simplicity of the defined measures", "strength_and_weaknesses": "+ The paper is well motivated and well written\n+ Recent work seems to have been discussed\n+ The measures are simple but intuitive\n+ Experiments are conducted on real datasets\n\n- It would be nice if the measures can be incorporated to produce both trustworthy and effective clusters", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "TpVqHo3dOFC", "reviewer": "Reviewer_8Lv7", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new framework for measuring trustworthiness of a model including new proposed metrics that can evaluate various models comparingly. The framework is applicable to text and image domains and is validated through experiments on datasets from each domain accordingly. The results showed that a model with higher accuracy might not be trustworthy according to the proposed metrics and framework introduced in the paper.", "summary_of_the_review": "Overall, the problem considered is interesting, but the paper has some flaws in terms of lacking some more in depth discussions along with technicality. ", "strength_and_weaknesses": "**Strengths:**\n1. The problem considered in this paper is interesting.\n2. The framework is applicable to text and image domain.\n3. Paper is well-written and easy to follow.\n4. The collected datasets in this paper can be useful to the community.\n\n**Weaknesses:**\n1. The approach needs ground truth explanation collection for each task which I am not sure how feasible might be.\n2. In some cases this ground truth collection might not be easy in cases where expert knowledge is required. What do authors think about these types of overheads. Please discuss.\n3. The paper is not technically rigor and am not sure about the extend of novelty introduced in this work.\n4. Authors (and ultimately the metric and framework proposed in this work) assume that the explanation methods are themselves complete and reliable such that the measures are being build upon these explanation methods. More discussion and justification needs on this issue.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject