IAEval: A Comprehensive Evaluation of Instance Attribution on Natural Language Understanding
Authors: Peijian Gu
Yaozong Shen
Lijie Wang
Quan Wang
Hua Wu
Zhendong Mao
Venue: EMNLP 2023 Findings (2023)
Keywords: evaluation; instance attribution
TL;DR: We propose a systematic and comprehensive evaluation scheme to evaluate instance attribution methods.

Abstract: Instance attribution (IA) aims to identify the training instances leading to the prediction of a test example, helping researchers understand the dataset better and optimize data processing. While many IA methods have been proposed recently, how to evaluate them still remains open. Previous evaluations of IA only focus on one or two dimensions and are not comprehensive. In this work, we introduce IAEval for IA methods, a systematic and comprehensive evaluation scheme covering four significant requirements: sufficiency, completeness, stability and plausibility. We elaborately design novel metrics to measure these requirements for the first time. Three representative IA methods are evaluated under IAEval on four natural language understanding datasets. Extensive experiments confirmed the effectiveness of IAEval and exhibited its ability to provide comprehensive comparison among IA methods. With IAEval, researchers can choose the most suitable IA methods for applications like model debugging.

IAEval: A Comprehensive Evaluation of Instance Attribution on Natural Language Understanding
Peijian Gu Yaozong Shen Lijie Wang Quan Wang Hua Wu Zhendong Mao
Universityy of Science and Technology of China, Hefei, China
"Baidu Inc., Beijing, China
MOE Key Laboratory of Trustworthy Distributed Computing and Service, Beijing University of Posts and Telecommunications, Beijing, China
wangquan@bupt. edw. cn gpj123@mail www.ssttcc.... bustc.edu.c. cn shenyaozong@baidu. com
Abstract
the model's predictions, but also help researchers debug or optimize models, such as finding problem- atic training data and improving the data quality.
Instance attribution (IA) aims to identify the training instances leading to the prediction of test example, helping researchers understand the dataset better and optimize data processing. While many IA methods have been proposed recently, how to evaluate them still remains open. Previous evaluations of IA only focus on one two dimensions and not compre- hensive. In this work, we introduce IAEval for IA methods, a systematic and comprehensive evaluation scheme covering four significant re- quirements: sufficiency, completeness, stability and plausibility. We elaborately design novel metries to measure these requirements for the first time. Three representative TA methods are evaluated under IAEval on four natural lan- guage understanding datasets. Extensive exper- iments confirmed the effectiveness of LAEval and exhibited its ability provide comprehen- sive comparison among IA methods. With IAE- val, researchers can choose the most suitable IA methods for applications like model debugging.
Plenty of IA methods have been proposed re- cently (Koh and Liang, 2017; Yeh et al., 2018; Barshan et al., 2020; Pruthi et al., 2020). For a test example, these methods calculate a influence score for each training data, reflecting its influence the prediction of the test example shown in the left part of Figure 1, The top ranked training instances are extracted as evidences. While all these meth- ods are potentially chosen to identify evidences, how to evaluate them still remains open. Hanawa et al. (2021) proposed a randomized-test and two heuristics to test whether IA methods satisfy mini- mal requirements for evidences. Karthikeyan and Søgaard (2021) proposed to evaluate by detecting poisoned training examples. Besides, Pezeshkpour et al. (2021) evaluated IA methods by comparing the prediction difference after removing evidences and a randomized-test However, all these existing methods consider solely one or two perspectives of evidences such as faithfulness, and they ignored other significant requirements that ideal evidences were expected to meet. It still lacks a holistic eval- uation for IA methods.
1 Introduction
Along with the fast development and wide applica- tion of deep neural networks, a lot of interpretabil- ity methods have been proposed to explain the models' predictions, which help people understand the reason behind the models' success and limi- tations (Ribeiro et al., 2016; Lundberg and Lee, 2017; Linardatos et al., 2020; Hanawa et al., 2021). Such interpretability or attribution methods can be primarily divided into two categories: Feature At- tribution (FA) and Instance Attribution (IA). While FA highlights important input features or tokens that support the prediction of the input, IA identi- fies the training instances leading to the predictions of test examples. The evidences obtained by attribu- tion methods can not only help people understand
In order to address the above problems, we pro- pose IAEval, the first systematic and comprehen- sive evaluation scheme for IA methods. IAEval consists of our elaborately designed metries quanti- fying and measuring four important requirements for TA, including sufficiency, completeness, stabil- ity and plausibility. Although these requirements have been deeply discussed for FA Young et al., 2020; Wang et al., 2022), we apply them to IA for the first time by refining their definition and de- signing metrics for measurement thoroughly. We demonstrate the overall framework of IAEval in the middle part of Figure 1. With IAEval, the evaluation of the IA methods can be conducted in a methodical and holistic manner. Furthermore,
"Equal Contribution.
Work done during internship at Baidu Inc.. =Corresponding author: Quan Wang.

---

IAEval
Sufficiency
Completeness
8%
dis eos
210 the pove 1:0
dis
Paur
Curves
Stability
Plausibility
Spcarman
Levenshtein
Figure I: The overall pipeline of evaluation of TA methods. The left part shows the basic structure of TA methods. They extract evidences according to the influence scores of each training data on a test example. The middle part is our proposed systematic evaluation scheme IAEval. IAEval evaluates TA methods with four sets metrics covering four significant requirements for IA. The right part is the comparison of the evaluated methods under our IAEval. The comparison is comprehensively conducted on four distinct dimensions.
owing IAEval's 's diverse evaluation dimensions, in-depth analyses can be conducted each IA method. Through such analyses, possible to unveil potential drawbacks of a method that may not be detected solely through evaluation from a single perspective (Seeing Section 7.1).
provide comprehensive evaluation and help with unveiling potential drawbacks of method through nulti-persppecive analysis.
With IAEval, researchers can choose the most suitable IA methods for specific applications such as annotation checking or training data reduction.
We evaluate IA methods with IAEval on four natural language understanding datasets in two languages: SST-2 (Socher al., 2013), MNLI (Williams et al., 2018) in English and LCQMC (Liu et al., 2018), Chnsenticorp (Tan and Zhang, 2008) in Chinese. We evaluate three representative IA methods: Feature Similarity (FS), Gradient Sim- ilarity (GS) (Charpiat et al., 2019) and Representer Point Selection (RPS) (Yeh et al., 2018). The sys- tematic nature of IAEval enables make thor- ough comparison from four perspectives among these IA methods in the right part of Figure 1. With IAEval, researchers can refer to its evaluation results and choose better IA methods to acquire higher-quality evidences, helping with specific ap- plications.
Related Work
Instance attribution. Recent attribution meth- ods can be primarily divided into two categories: Feature Attribution (FA) and Instance Attribution (IA). In this work, focus IA methods which calculate influence scores of each training data for test examples. Influence functions (IF) (Koh and Liang, 2017) approximate the importance of each training data via the products of gradients and the inverse Hessian matrix. Barshan et al. (2020) pro- poses Relative Influence to mitigate the overlap- ping problem of IF. Pruthi et al. (2020) measures the importance by tracing the loss change of test example caused by each training data. Similarity- based methods directly measure the training data's 's importance by calculating the similarity features (Caruana et al., 1999) or gradients (Charpiat et al., 2019) between training data and test example. Rep- resenter Point Selection (RPS) (Yeh et al., 2018) propose to measuring influence score by linearly decomposing the pre-activation prediction of test example with regard to each training instance. With the influences scores, the training data are ranked accordingly and the top ranked instances are iden- tified as evidences.
Our key contributions are summarized as fol- lows:
We propose IAEval, the first systematic and comprehensive evaluation scheme for IA methods. IAEval covers four significant re- quirements including sufficiency, complete- ness, stability and plausibility. We design novel metrics to measure these four signifi- cant requirements for the first time.
Experiments on four natural language under- standing datasets two languages validate the effectiveness of IAEval. Compared to evalua- tions solely with a single perspective, IAEval
Evaluation of attribution methods. With the advancement of attribution methods, evaluation

---

of these methods gradually becomes increasingly significant. For FA methods, many datasets with human-annotated evidences have been published for evaluation (De Young et al., 2020; Mathew et al., 2021; Camburu et al., 2018; 8; Rajani et al., 2019; Wang et al., 2022). Meanwhile, many studies give their views on the properties that an evidence should satisfy, such compactness and sufficiency (Lei et al., 2016; De Young et al., 2020), comprehen- siveness (Kass et al., 1988; Yu al., 2019), plau- sibility and faithfulnessDingg and Koehn, 2021; Jacovi and Goldberg, 2020). But for IA methods, there is no manually annotated evaluate datasets, and there are few evaluation metrics. Hanawa et al. (2021) proposes a randomized-test and two heuris- tics to evaluate the IA methods. Karthikeyan and Søgaard (2021) points out the illness of previous metrics and propose evaluate by detecting poi- soned training examples. Pezeshkpour et al. (2021) analyzes the correlation of different IA methods and makes comparison according to the prediction difference by removing the identified evidences and randomization test. The above evaluations of IA methods consider solely one or two perspec- tives. A comprehensive evaluation covering multi- ple perspectives is indispensable necessity for iA methods.
to tailor them for IA for the first time. We refine their definition and design novel metrics for mea- surement. We describe in detail each requirement as well as the corresponding metrics as follows.
4.1 Sufficiency
Definition. The selected evidence sufficient if they contain enough information for model make the correct prediction. In other words, the model can make the same prediction only based on evidences as based on all training instances.
Metries. Given m test examples, we use the con- sistency between the prediction Dt supported by the selected evidences and the prediction Oe sup- ported by the full training dataset measure the sufficiency of the selected evidences. Specially, we train two models, one is based on evidences, and the other is trained on full data. The sufficiency score (denoted as S%) is shown in Equation 1:
1 m
5%
I(ū 9t) 100%, (1)
where I equals to 1 only if úr - 5t. The higher the 5% is, the more sufficient the evidences are.
We also calculate the consistency between two prediction probabilities obtained by two models: Pt, and Pty. We compute the Euclidean distance and cosine distance between two probabilities shown in Equation 2 and 3; Meanwhile,
3 Preliminaries
Firstly, we introduce background knowledge that used the later sections. Since most of the exist- ing IA methods are conducted on the classification tasks, we mainly focus on text classification tasks. Given training dataset D = {z,, ys 311e-1,2..... for a task, where z; is the input sentence sentence pair and y, is the label, we train classification model O We denote the learned representation of ay as fi and the classifier linear layer as A. Given test examples (#t))=112....r IA methods quantify the influence score each training data x; on the prediction of each testing example it, denoted as score [x, 21). For each test example Ht, the topk training instances with the greatest influence scores are selected as the evidences {ft.j j==12.....
Paif to measure the difference between them in Equation 4. Lower distance and difference scores imply higher sufficiency.
1
m
diSeac
m Σ
10r - D.11
(2)
discos
(1 cos (pt,
(3)
1
Pdiff
Pr.ge
Pr.je
(4)
4 LAEval
4.2 Completeness
IAEval covers four significant requirements for IA methods including sufficiency, completeness, sta- bility and plausibility. While these requirements have been discussed for FA YDe et al., 2020; Wang et al., 2022), we conduct the optimization
Definition.
The selected evidences are complete if all the instances that can support test example's prediction are selected, i.e., the remaining train- ing data can not support the prediction of the test example.

---

Metrics. We sort the training instances accord- ing to their influence scores, and then divide the training data evenly into pieces. We then sample the last yuu examples of a data piece as the sub- set as well as the top Nsub examples of the whole training data. As a result, we get N+1 subsets of data representing different influence score rankings, The influences scores in i-th subset are higher than those in the (i + 1)-th subset. Then for each subset, we train a model and get prediction for test example with the model. We calculate the sufficiency score 5% of cach sampling point and report the curve of the 5% with regard to the influence score rankings. To be more concise, we quantify the completeness, denoted as C%. by calculating the maximum dif- ference between the average 5% on both sides of a certain division point in Equation 5:
Metries. The plausibility measures how much the rationales provided by the model align with human-annotated rationales. The golden instance- level evidences are difficult for annotating, as it is unrealistic to annotate an influence score for each training data for each test examplc, especially with large training set. To simplify the measure of this evaluation, we propose two metrics for plau- sibility: semantic similarity and pattern similarity between the selected evidence and the test example. We use the cosine similarity of hidden represen- tations to measure semantic similarity (Reimers and Gurevych, 2019). Meanwhile, we use Leven- shtein distance (Levenshtein, 1966) between the dependency trees of the evidence and the test ex- ample to measure the pattern similarity (McCoy et al., 2019). Higher cosine similarity and lower Levenshtein distance imply higher plausibility.
n Σ
C%
ISn
max
n2
S
Preparations
w=1
N-1
5.1 Evaluated Methods
Σ 1
S,%,,
(5)
N +1
In this work, we evaluate three representative IA methods: Feature Similarity, Gradient Similarity (Charpiat et al., 2019) and Representer Point Selec- tion (Yeh et al., 2018). We briefly describe their main ideas and the calculation of influence scores.
where Sa% refers to the 5% of the w-th subset. Higher C% implies better completeness.
4.3 Stability
Definition. An attribution method is stable if it provides similar evidences for similar inputs.
Feature Similarity (FS) A simple assumption is that higher feature similarity between exam- ples reflects higher influence. We consider the cosine similarity between the learned representa- tion fa and ft and the influence score is defined as: [( .r)) = cos(fs, ft).
Metrics. We use the similarity of evidences for similar test examples as the metric for stability. We calculate the instance-level overlap rate, denoted as 0%, between the evidences { } of top& most similar test example pairs in Equation 6:
Gradient Similarity (GS) Charpiat et al. (2019) pointed out that the similarity of gradients between two examples reflected how much the change in one would affect the other. We consider the cosine similarity of the gradients VoL(x, yi) of the clas- sifier lincar layer 0. The influence score is defined as: [[x,, 2)) = cos ( VeL(xs, 3:), V VoL(zt, 11)).
0%
1 N
Σ (2too)
H } ( x 100%.
(6)
Following Pezeshkpour et al. (2021), for the topN most similar test example pairs, we also re- port the Spearman correlation between the influ- ence rankings over a fixed subset of training set induced from each pair. Higher overlap rate and Spearman correlation imply higher stability.
Representer Point Selection (RPS) Yeh et al. (2018) proved that the pre-activation predictions of test example p θ) can be decomposed with regard to each training data: 4 θ) a,, 2z). Such decomposition deemed as the influence of each training data on the test example. The influence score is defined as: -1 8((#.w) f( ft- (0) ZAm (0),,,
4.4 Plausibility
Plausibility requires that the identi- fied evidences are in line with human cognition and can be understood and accepted by humans.
Definition.

---

5.2 Setup
are effective and can better distinguish LA methods from random test. FS and GS perform fairly close across four datasets, while RPS performs slightly inferior to them on three datasets. For LCQMC, RPS performs much worse on all four metircs.
Datasets. We conduct experiments on sentiment analysis dataset SST-2 (Socher et al., 2013) and natural language inference dataset MNLI (Williams et al., 2018). We also experiment on two Chinese datasets: sentiment analysis dataset Chnsenticorp (Tan and Zhang, 2008) and similarity alignment dataset LCQMC (Liu et al., 2018). The statisties of these datasets are listed in Appendix A.
Completeness.
For each test example, we extract 10 examples for each subset and train models. We then caleulate S% of every rank point in Section 4.2. We show the curve of 5% with regard the rankings in Figure 2. A complete IA method should perfectly separate the whole dataset into two parts. One of them contains all of the evidences while the other one contains few or zero evidences. The perfect completeness curve should look like a 'z' shape. which reaches 100% at the beginning, drops dramatically to 0% at a demarcation point and re- mains 0% to the end. Figure 2 clearly shows that the curve of FS is more close to perfect. GS has a good curve overall, but there remains flaws at the bottom part. RPS performs worse but still normal on English datasets, and it looks like out of shape on Chinese datasets. The above observations can also be validated through C% in Table la. While FS achieves higher C% than GS, the score of RPS is significantly lower than the other two methods except on MNLI where all three methods achieve relatively close scores.
Models. We fine-tune the classification models based on RoBERTa- base (Liu al., 2019) for two English datasets and Chinese RoBERTa-wwwn (Cui et al.. 2020) for two Chinese datasets. All our experiments are conducted with 5 different seeds and the average results are reported.
Test Examples. Pretrained models like RoBERTa (Liu et al., 2019) possess certain degree of zero/few-shot capability. They are able to predict some examples correctly without or with very few training examples. As a result, the prediction of such simple examples may not depend on any training example and the corresponding evidences are probably meaningless. We avoid to evaluate on these simple data. To filter out them, we randomly sample 100 training examples, fine-tune models accordingly, and make predictions on the dev set. We repeat the procedure for 5 rounds, getting 5 different models and predictions. The examples with 4 and more correct predictions in the 5 rounds are considered simple and neglected. We then randomly sample 100 examples from the remaining data in dev set as our test examples.
Stability. We select the top 100 most similar pairs from our test examples according to the co- sine similarity of the hidden representations. The cosine similarity is calculated with sentence-BERT (Reimers and Gurevych, 2019) on English datasets and Text2vec' on Chinese datasets. We report the overlap rate of their top 1000 ranked evidences in Table 1b. A fixed data subset of size 1000 are sam- pled from the training set. For each test pair, the Spearman correlation between the influence rank- ings is calculated on this subset and the average results are reported. We observe that under the Spearman correlation metric, FS and RPS exhibit relatively close performance, while GS performs the worst, indicating its inferior stability compared to the other methods. Besides, FS and GS have similar overlap rate for all the datasets while RPS achieves an abnormally high score.
6 Experiments
With our evaluation scheme IAEval, we make comprehensive comparison among the aforemen- tioned IA methods from four distinct perspectives. We report the performance of these methods with regard to each important requirements in Table
6.1 Main results
Sufficiency. We train models with top 100 ranked evidences identified by each method well as 100 randomly sampled training data and make predic- tions. We compute our metrics for each models with the model that is trained with the full training data. We report results in Table la. All three meth- ods perform remarkably better than random sam- pling, which indicates our metrics for sufficiency
For each text example, we select top 1000 evidences, and report the average cosine similarity of their representations and Levenshtein
Plausibility.

---

SST-2
MNLI
Method
S%
diseue +
discos
Pdiff
C%
5%
dissue
discos +
Paiff
C%
FS
98.6
0.426
0.091
0.301
91.7
98.4
0.597
0.202
0,484
90.6
GS
99.8
0.432
0.094
0.302
87.7
98.2
0.593
0.206
0.481
87.6
RPS
99
0.470
0.116
0.327
69.5
98.4
0.616
0.222
0.498
92.6
random
28
0.701
0.312
0.495
N/A
32.6
0.760
0.395
0.616
N/A
Chnsenticorp
LCQMC
Method
S%
diseue +
discos
Pdiff
C%
S%
diseuc
discos +
Pdiff
C%
FS
100
0.094
0.004
0.053
90.6
97.8
0.110
0.007
0.045
75.0
GS
100
0.104
0.005
0.052
88.6
98
0.110
0.008
0.047
64.1
RPS
93
0.213
0.035
0. 1.146
58.5
0.478
0.206
0.323
30.4
random
55.8
0.495
0.250
0.438
N/A
47.4
0.620
0.280
0.427
N/A
(a) Sufficiency and Completeness
SST-2
MNLI
Chnsenticorp
LCQMC
Method
Spearman
0%
Spearman
0%
Spearman
0%
Spearman
0%
FS
0.0478
10.33
0.0292
0.31
0.0278
30.53
0.0399
1.20
GS
0.0299
10.34
0.0265
0.04
0,0253
30.68
0.0378
1.20
RPS
0.0400
65.95
0.0314
31.78
0.0330
57.40
0.0322
44.03
(b) Stability
SST-2
MNLI
Chnsenti.
LCQMC
Method FS
Cosine.
Leven.
Cosine.
Leven.
Cosine.
Leven.
Cosine.
Leven.
0.3045
19.15
0.1556
35.71
0.6110
85.61
0.4060
12.53
GS
0.3036
19.14
0.1528
35.80
0.6108
85.60
0,4059
12.54
RPS
0. 1.44
17.65
0.1291
37.19
0.5750
91.24
0.3654
12.66
(c) Plausibility
Table 1: The overall performance of the evaluated TA methods under IAEval. means the smaller the better. The best results in the corresponding regime are shown in bold. a) Sufficiency performance and completeness performance. 5% quantifies the completeness performance addition to the completeness curves in Figure 2. All the reported results averaged over 5 different b) Stability performance. c) Plausibility performance.
40
20
26
Sanc PN11
BaYk NE11
Fanic PUin
Bank EVE1T
SST-2 (b) MNLI
(c) Chosenticorp
LCQMC
Figure 2: Completeness performance of evaluated IA methods. We report the S% of each rank point on the four evaluated datasets. the reported results are averaged over 5 different runs.

---

10000
2000
20de
Sall
232
402
500
(a) Gradient
(b) Feature Dot-produet
Figure 5% the evaluated methods based on different evidence sizes LCQMC.
Figure 3: The distribution a) the scale the gradients of the loss function with respect the pre-activation logits of each training data. and b) the averaged scale of feature dot-products on LCQMC.
function with respect to the pre-activation logits of each training data. The other is the feature dot- product of training instances and test example. We discover that the gradient part plays dominant role in the influence score as shown in Figure 3. The majority of feature dot-products have the largest scale: 10², and they are correlated with the test example. However, only few amount of gradients have the largest scale: 10-6, and it is noteworthy that the gradients are independent of the test ex- amples. As a result, when we multiply these two factors, those small amount training data with high gradients will constantly get high influence score regardless of the test example. This explains why the identified evidences by RPS have sig- nificant high overlap rate, shown in Section 6. We examined the top 1000 evidences of RPS and discovered that data with gradient of scale 10-6 dominant the evidences with proportion of 98%. Such observation verifies our explanation.
distance of their dependency trees in Table lc. The cosine similarity is calculated in the same way as in the stability part. The dependency parsing is done with CoreNLP (Manning al., 2014) En- glish datasets and DDParser (Zhang al., 2020) on Chinese datasets. As shown Table Ic the performance FS and GS is relatively close un- der both metrics, while RPS performs worse than the other two methods except for the Levenshtein distance in SST-2. Such anomaly may be due to the incompleteness of sentences in SST-2 and the dependency is relatively broken compare other datasets.
6.2 Comparison of methods
After the evaluation with IAEval, we claim that FS performs relatively better among the evaluated IA methods. GS is inferior to FS in completeness and stability. RPS performs significantly worse than expectation. The poor performance and abnormal pattern exhibited by RPS our metrics prompt us undertake de-thpth analysis, which leads us reasonable explanation in Section 7.1 that can account for its underperformance and anomaly.
Furthermore, posit high gradient may be at- tributed to annotation issues. We randomly sam- pled 100 training data with gradient of scale 10-6 and examined the annotation. We find that 26% of the sampled data are labeled incorrectly. leading to the poor performance RPS. We also exam- ined the annotation of 100 training data randomly sampled from whole training data. The annotation error rate is 5%. Such observation inspires that RPS can be used extract illness data, helping with annotation checking.
7 Analysis
In this section, we first provide explanation for the poor performance and anomaly of RPS on LCQMC. To illustrate the robustness of TAEval, conduct ablation studies with regard to evidence size. Furthermore, we validate the effectiveness of IAEval with an empirical analysis.
The overall explanation is derived from a com- prehensive assessment of performance across four perspectives. Without such multitude of evalu- ation angles, explanations for the issue would be partial and the limitations would not be unveiled.
7.1 Why RPS has poor performance? We investigate the reason for the poor perfor- mance and abnormally high overlap rate. First let's recall the influence score of RPS: I(x;, 21) -1 aC(z.)) f. fc. Apart from constants, consists ZAn (p]
7.2 The influence of the size of evidence set
We analyze whether the size of evidence set affects IAEval. The conclusions are consistent on all the datasets and we report the results on LCQMC.
two components. One the gradient of the loss

---

1.5
30
24
25%
40%
40%
80h
3325
66
14
40.
(a) Evidence size = 50
(b) Evidence size = 100
(c) Evidence size 500
(d) Evidence size 1000
Figure 5: Completeness curves of the evaluated methods with different evidence sizes LCQMC.
Cosine Similarity
Method
10
50
100
500
1000
10
50
100
500
1000
FS
0.00
0.04
0.04
0.54
1.20
0.4104
0.4063
0.4057
0,4060
0.4060
GS
0.00
0.02
0.04
0.54
1.20
0.4094
0.4058
0.4053
0.4058
0.4059
RPS
29.60
31.92
35.04
41.10
44.03
0.3710
0.3718
0.3701
0.3657
0.3654
Table 2: The results of stability and plausibility of the evaluated methods with different evidence size on LCQMC.
For sufficiency, we report the 5% of the evalu- ated methods based on different evidence sizes in Figure 4. The results of FS and GS are relatively constant across different evidence sizes while the RPS meets performance drop as the evidence size grows. Such drop could be attributed to the grow- ing quantity of the training data with high gradient in Section 7.1 and the predictions will be bi- ased towards them. As to completeness reported in Figure 5, the completeness curves of the evaluated methods are similar across different evidence sizes. We also report the overlap rate for stability and cosine similarity for plausibility with different evi- dence sizes in Table 2 and the results are consistent across different evidence sizes.
SST-2
MNLI
Chnsenti.
LCQMC
Ayg.
Full
95.0
86.4
93.4
86.0
90.2
FS
91.6
80.8
92.8
84.8
87.5
GS
92.6
79.2
92.2
84.6
87.1
RPS
45.0
31.8
38.0
50.4
41.3
Table 3: accuracy on the target set. Full represents the fully-trained model. The best results of the evaluated methods are shown in bold.
source set. We report the results of the evaluated methods in Table 3, as well as the accuracy of the fully-trained model.
As Table shows, the evidences selected by FS and GS are high-quality and have competitive performance with the full training data, while RPS performs much worse, Such observation is consis- tent with the conclusion of TAEval, verifying its effectiveness. The bad performance of RPS may be due to the duplication and relatively bad quality of its extracted evidences as in Section 7.1. Since the high-quality of evidences obtained by FS and GS, choose more representative and effective training instances for the given test set, reduce annotation costs and training time.
To summarize, the above observations illustrate that EAE is consistent and robust with regard to the evidence size.
7.3
Validation effectiveness
To validate the effectiveness of IAEval, duct empirical analysis by simulating training data reduction. We assess the quality of the ex- tracted evidences of each IA method for training data reduction and determine if is consistent with the conclusion from IAEval. We build target set by randomly sampling 500 dev examples for each dataset and build source set by combining the top10 evidences of each target example for each method?. The quality of the evidences from each
Conclusion
We propose IAEval, the first systematic and com- prehensive evaluation scheme for TA methods. We design novel metrics to quantify and measure four important requirements for the first time, including sufficiency, completeness, stability and plausibility. Extensive experiments validate the effetiveness of IAEval. With IAEval, researchers can make com- prehensive comparison among IA methods, gain
method is measured by the accuracy on the target set, with a model fine-tuned on the corresponding
We do not eliminated duplicates and the size of every source set constantly 5000.

---

thorough understanding of their strengths and weaknesses and choose the most suitable methods for specific applications.
pages 657-668, Online. Association for Computa- tional Linguistics.
Jacob Devlin, Ming-Was Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv:
Limitations
We conduct all the experiments text/text pair classification tasks while the performance of IA methods on other NLU tasks such as Question An- swering are still unstudied. We evaluate three rep- resentative IA methods and leave the others for future work because of the high computational cost (influence functions (Koh and Liang, 2017)) or the similarity of methods (dot-product v.s. cosine sim- ilarity). Moreover, our experiments are based on the base-size models while the sizes of models are extremely large nowadays. The performance of IA with LLMs may be possible future research directions.
Jay De Sarthak Jain, Nazneen Fatema Rajani. Eric Lehman. Caiming Xiong. Richard Socher, and Byron C Wallace. 2020. Eraser: benchmark to evaluate rationalized nip models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguisties, pages
Shuoyang Ding and Philipp Koehn. 2021. Evaluating saliency methods for neural language models. In Pro- ceedings the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5034-5052 Online. Association for Computational Linguisties.
Kazuaki Hanawa, Sho Yokoi, Satoshi Hara, and Ken- taro Inui. 2021. Evaluation of similarity- bassd ex- planations. In International Conference Learning Representations.
Acknowledgements
We would like to thank all the reviewers for their valuable advice to improve this work. This research is supported by National Science Found for Excel- lent Young Scholars under Grant 62222212 and the General Program of National Natural Science Foundation of China under Grant 62376033.
Alon Jacovi and Yoav Goldberg. 2020. Towards faith- fully interpretable NIP systems: How should we define and evaluate faithfulness? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4198 4205, On- line. Association for Computational Linguisties.
K Karthikeyan and Anders Søgaard. 2021. Revisit- ing methods for finding influential examples. arXiv preprint arXiv:21 20ff::211..00 211.006883.
References
Elnaz Barshan, Marc-Etienne Brunet, and Gintare Karolina Dziugaite. 2020. Relatif: Identifying explanatory training samples via relative influence. In International Conference on Artificial Intelligence and Statistics, pages 1899-1909, PMLR.
Robert Kass, Tim Finin, al. 1988. The need for user models in generating expert system explanations. International Journal of Expert Systems, 1(4).
Pang Wei Koh and Percy Liang, 2017. Understanding black-box predictions via influence functions. In International conference on machine learning, pages 1885-1894, PMLR.
Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-snli: Natu- ral language inference with natural language expla- nations. Advances in Neural Information Processing Systems, 31.
Tao Lei, Regina Barzilay. and Tommi Jaakkola. 2016. Rationalizing neural predictions. In Proceedings of the 2016 Conference on Empirical Methods in Nat- ural Language Processing, pages 107-117, Austin, Texas. Association for Computational Linguisties.
Rich Caruana, Hooshang Kangarloo, John David Dion- isio, Usha Sinha, and David Johnson. 1999. Case- learning meth-
based explanation of non-case- ods. In Proceedings of the AMIA Symposium, page 212. American Medical Informatics Association.
V. I. Levenshtein. 1966. Binary Codes Capable of Cor- recting Deletions, Insertions and Reversals. Soviet Physicy Doklady, 10:707.
Guillaume Charpiat, Nicolas Girard, Loris Felardos, and Yuliya Tarabalka. 2019. Input similarity from the neural network perspective. Advances in Neural Information Processing Systems, 32.
Pantelis Linardatos, Vasilis Papastefanopoulos, and Sotiris Kotsiantis. 2020. Explainable ai: A review of machine learning interpretability methods. Entropy. 23(1):
Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang, and Guoping Hu. 2020. Revisiting pre-trained models for Chinese natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings,
Xin Liu, Qingeai Chen, Chong Deng, Huajun Zeng Jing Chen, Dongfang Li, and Buzhou Tang. 2018. Leqme: A large-scale chinese question matching corpus. In

---

Proceedings of the 27th international conference on computational linguistics, pages 1952-1962.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng. and Christopher Potts. 2013, Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empiri- cal methods in natural language processing, pages 1631-1642.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.
Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv: 7907. 1692.
Yu Sun, Shuohuan Wang, Shikun Feng, Siyu Ding. Chao Pang, Junyuan Shang. Jiaxiang Liu, Xuyi Chen, Yanbin Zhao, Yuxiang Lu, et al. 2021. Ernie 3.0: Large-scale knowledge enhanced pre-training for lan- guage understanding and generation. arXiv preprint N.Ni:2107.02137.
Scott M Lundberg and Su-In Lee. 2017. A unified ap- proach to interpreting model predictions. Advances neural information processing systems, 30.
Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David Mc- Closky. 2014. The stanford corenlp natural language processing toolkit. In Proceedings of 52nd annual meeting of the association for computational linguis- fics: system demonstrations, pages 55-60.
Songbo Tan and Jin Zhang. 2008. An empirical study of sentiment analysis for chinese documents. Expert Systems with applications, 34(4):2622-2622
Lijie Wang. Yaozong Shen. Shuyuan Peng, Shuai Zhang. Xinyan Xiao, Hao Liu, Hongxuan Tang. Ying Chen, Hua Wu, and Haifeng Wang. 2022. A fine-grained interpretability evaluation benchmark for neural NLP. In Proceedings of the 26th Conference on Computa- tional Natural Language Learning (CoNIL.). pages 70-84, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukher- jee. 2021. Hatexplain: A benchmark dataset for ex- plainable hate speech detection. In AAAL
Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuris- tics natural language inference. the 57th Annual Meeting of the Association for Com- putational Linguistics, pages 3428 -344,,4, Florence, Italy, Association for Computational Linguisties.
Proceedings of
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sen- tence understanding through inference. In Proceed- ings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume (Long Papers), pages 112-1122.
Pouya Pezeshkpour, Sarthak Jain, Byron C Wallace, and Sameer Singh. 2021. An empirical comparison of instance attribution methods for nlp. Proceedings of the 2021 Conference of the North American Chap- ler of the Association for Compulational Linguistics: Human Language Technologies. pages 967-975.
Chih-Kuan Yeh, Joon Kim, Ian En-Hsu Yen, and Pradeep K Ravikumar, 2018. Representer point selec- tion for explaining deep neural networks. Advances in neural information processing systems, 31.
Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. 2020. Estimating training data influ- ence by tracing gradient descent. Advances in Neural Information Processing Systems, 33:19920-19930
Mo Yu. Shiyu Chang, Yang Zhang, and Tommi Jaakkola. 2019. Rethinking cooperative rationalization: In- trospective extraction and complement control. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNLA) pages 4094- 4103, Hong Kong, China. Association for Computa- tional Linguisties.
Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Explain your- self! leveraging language models for commonsense reasoning. In Proceedings of the 57th Anmual Meet- ing of the Association for Computational Linguistics, pages 4932-4942, Florence, Italy. Association for Computational Linguistics.
Shuai Zhang, Lijie Wang, Ke Sun, and Xinyan Xiao. 2020. A practical chinese dependency parser based on a large-scale dataset. arXiv preprint arXi::2009.00901
Nils Reimers and Iryna Gurevych. 2019. Sentence-bert Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLL) pages 3982-3992.
A
Datasets
The statistics of the datasets we evaluate on.
Marco Tulio Ribeiro, Sameer Singh. and Carlos Guestrin. 2016. why should trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135- 1 1144.
B Models
We also conduct experiments to show the robust- ness of IAEval across models in the same language. We report the results of base-uncased

---

Dataset
#Train
#Dev
#Test
SST-2 (Sucher al., 2013)
67.349
872
1.821
MNLI (Williams et al., 2018)
392,702
9,815
9.796
Chnsenticorp (Tan and Zhang, 2008)
9,600
1,200
1,200
LCQMC (Liu et al., 2018)
238,766
8,802
12,500
Table 4: Statistics of the evaluated datasets, We rank the influence scores over the whole training set.
et al., 2018) and RoBeRTa-basc(Liu et al., 2019) on English; Emii-3.00 et al., 2021) and RoBERTa- wwm-ext((uui et al., 2020) on Chinese in Table 5, The consistent results across models indicate the robustness of IAEval across models in the same language.

---

SST-2
LCQMC
BERT
Ernie
Method
S%
diseuc +
discos +
diff 4
C%
8%
diseuc
discos
diff C%
feature
98.4
0,291
0.042
0,206
84.8
99.6
0.219
0,022
0.146
79.8
gradient
99.8
0.288
0.04
0.2
57.5
98.8
0.223
0.024
0.15
65.5
RPS
80.4
0,515
0.178
0.36
46.3
90.1
0.403
0.09
0.283
31.7
RoBERTa
RoBERTa
Method
8%
dissuc
discos
diff +
0%
8%
disouc
discos
diff
C%
feature
98.6
0.426
0.091
0.301
91.7
97.8
0.11
0.007
0.045
75.0
gradient
99.8
0.432
0.094
0,302
87.7
98
0.11
0.008
0.047
64.1
RPS
99.0
0.47
0.116
0.327
69.5
65
0.478
0.206
0.323
30.4
Table 5: Sufficiency and completeness performance of different models the same language.


Reviews:
[{"review_id": "FgG9TWCG9R", "reviewer": "Reviewer_VKZW", "summary": "", "questions": "There are some questions embedded in the \"Reasons To Reject\" above.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your insightful review. \n\nFor concern 1:\n\nInstance attribution (IA) methods are an important branch of interpretability. Since interpretability is for humans, the results of interpretability methods, i.e., the identified evidences in IA, should be in line with human cognition and be understood and accepted by humans. The concept of plausibility has been widely discussed within the interpretability community as in Strout et al (2019) (https://aclanthology.org/W19-48.pdf#page=70), Jacovi and Goldberg (2020) (https://aclanthology.org/2020.acl-main.386/) and Hanawa et al (2021) (https://openreview.net/forum?id=9uvhpyQwzM_), etc. Since plausibility refers to how the explanation is convincing to humans, it is important to evaluate such requirement.\n\nThe usefulness and effectiveness of IA methods you mentioned are prone to be reflected in other branches in IAEval such as sufficiency (measuring the prediction consistency between models trained on only evidences and the original model). However, for some domains such as medication or law, the demand for plausibility would be rather high. It is quite necessary to evaluate the plausibility of IA methods for such reason.\n\n\n\nFor concern 2:\n\nWe believe that when the evidences share similar semantics or have similar structures with the test example, humans can easily recognize these commonalities, thus understanding and accepting the evidences. Conversely, if commonalities cannot be found, humans may feel confused and struggle to understand the evidences. Similarity measurement serves as a way to measure how commonalities can be recognized by humans. Thus, the differences in pattern and cosine (semantic) similarity can reflect differences in plausibility and can represent if the examples can be understood by humans or not.\n\n\n\nFor concern 3:\n\nPlease note that our goal is not to select the best IA method through comparison; our goal is to propose a comprehensive evaluation scheme that evaluates IA methods from four dimensions including sufficiency, completeness, stability, and plausibility. Additionally, our validation of effectiveness is presented in Section 7.3 rather than in comparing to the random sampling baseline. The scores of random sampling in Table 1(a) serve more like a lower bound for the metrics, helping people grasp the scale of the current metrics. In Section 7.3, we validate the effectiveness of IAEval with an empirical study by simulating the application of training data reduction. In the empirical study, the performance or rankings of the IA methods are consistent with the results of IAEval. The effectiveness of IAEval is thus validated.\n\nIn fact, the evaluated metrics in IAEval can align with various demands of practical applications such as sufficiency for training data reduction or plausibility for rationale presentation. IAEval allows users to choose the most suitable IA method for their specific application. What’s more, through experimental results, it can be observed that no method consistently performs well across dimensions. This indicates that there is still room for improvement of IA methods. IAEval can provide valuable guidance for the future design and advancement of IA methods.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Instance attribution is a hard and important problem, and understanding how to measure progress in the space is an important topic.", "reasons_to_reject": "- I'm not sure I agree with some of the axes. For example, why does \"plausibility\" (\"requires that the identified evidences are in line with human cognition and can be understood and accepted by humans.) matter? If the goal is to better understand what training instances a model is using to predict a particular test point, why is this related to human understanding at all? In other words, it's completely conceivable to me that a useful and effective instance attribution method has low plausibility, since it's faithful to what the model is actually doing (and the model itself uses features that are uninterpretable to humans)\n- Similarly, I'm not sure that I agree with some of the methods used to measure these axes. Going again to plausibility, how is the difference between pattern and cosine similarity supposed to represent if the examples can be understood by humans or not?\n- The stated goal is to compare between IA methods, but what makes one IA method better than another? The authors argue that their conclusions \"experiments validate the effectiveness of IAEval\", but comparing to the random sampling baseline seems like a pretty weak baseline to compare against. In general, I'm not entirely convinced that if you run two IA methods through IAEval, that it really tells you much that'd be useful for comparing these methods, which in general lowers my excitement about this work.", "questions_to_authors": "There are some questions embedded in the \"Reasons To Reject\" above.", "paper_topic_and_main_contributions": "This paper proposes an evaluation framework for instance attribution methods. The authors argue that methods should be evaluated along four axes: sufficiency, completeness, stability and plausibility. The authors evaluate a variety of instance attribution methods (feature similarity, gradient similarity, and representer point selection).", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bAWeN5Iugv", "reviewer": "Reviewer_sPvk", "summary": "", "questions": "Two questions about stability evaluation:\n\nA1. When you select the top N similar examples, did you also control for the prediction to be the same? I think this could eliminate some part of the errors from (1) and (2) I mentioned in the first weakness point.\n\nA2. For (3), do you think it could cause interference to use the similarity of hidden representations both for assigning instance importance (for FS method) and for retrieving top-N instances? From your description, it's not clear to me whether those two similarity metrics are the same (I doubt that it's not, given that FS method does not perform well on the stability metric). Do you think your conclusion would change with an alternative similarity metric?\n\nThen a few other things:\n\nB. For the simple example filtering described in L335-352, how many examples are still left after the filtering?\n\nC. I'm not sure I quite follow the motivation of section 7.3. To me this seems like re-doing the sufficiency evaluation?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your insightful review.\n\nFor the 3 points in concern 1:\n\n(1) Although the similarity of topN cannot be fully guaranteed, we believe that the current cosine similarity method is prone to capture some similar features. We conduct a statistical analysis and find that all topN pairs actually share the same prediction. Such observation indicates that the similarity metric to retrieve topN similar examples can capture some similar features. Besides, the significant performance drop (e.g., from ﻿77.10% to 0% on Chnsenticorp) between the topN and bottomN similar pairs also indicates the effectiveness of the similarity metric.\n\n(2) The impact of human label error on gradient based methods like GS is inevitable. But since the predictions of the topN similar pair are the same (seeing (1)), it will mitigate such impact to some extent, as the reviewer pointed out in Q.A1. It is impossible to fully eliminate the impact of human label error for gradient based methods such as GS or influence functions, etc.\n\n(3) The similarity metric we use to retrieve topN examples is the same with FS. After eliminating the interference, the conclusion of stability evaluation still holds. Seeing R.A2. for more details.\n\n\n\nFor concern 2:\n\nIAEval provides a systematic and comprehensive evaluation scheme from four dimensions. In  Pezeshkpour et al. (2021), the contents related to IAEval mainly consider one dimension and with different designs. Additionally, our analysis and application studies are distinct to theirs.\n\nIAEval evaluates IA methods from four important dimensions: sufficiency, completeness, stability and plausibility. In Pezeshkpour et al. (2021), they mainly considered metrics related to sufficiency while the remaining three important dimensions in IAEval (i.e., completeness, stability, plausibility) are not included in their work. The evaluation of IAEval is more systematic and comprehensive than Pezeshkpour et al. (2021)'s.  Besides, the design of metrics for sufficiency are totally different. They propose remove-n prediction change and a randomized-test while the metrics for sufficiency in IAEval evaluate the prediction consistency between model trained on evidences and the original model. Our metrics are more efficient since we train models only on evidences while they train models on the left training data, much more than the evidences. \n\nThe analysis and application studies of our paper is also distinct to Pezeshkpour et al. (2021). They analyze the correlation and computation complexity of IA methods and conduct a artifact-detection application study. In our paper, we analyze the drawback of RPS and conduct application studies on training data reduction.\n\nFrom the above comparison, it is evident that our evaluation is distinct from Pezeshkpour et al. (2021). The goal of IAEval and the evaluated metrics are novel compared to theirs.\n\n \n\nR.A1:  When we select the top N similar examples, we do not control the predictions to be the same. But after conducting statistical analysis, we found that all pairs actually share the same prediction. Such observation indicates that the similarity metric to retrieve topN similar examples can capture some similar features.\n\n\n\nR.A2: The similarity metric we use to retrieve topN examples is actually the same with FS. We believe such similarity metric could guarantee the similarity of topN pairs to some extend and the discussions in (1) demonstrate that. All the IA methods are evaluated on the same topN examples retrieved with the similarity metric.\n\nHowever, due to our oversight, we did not realize that it overlaps with the evaluated method FS. As pointed out by the reviewer, it will cause interference for FS. To eliminate such interference, we change the similarity metrics to retrieve top-N instances to the cosine similarity of hidden representations from the pretrained sentence transformers (Sentence-bert and Text2vec). The results are in the following table.\n\n|        | SST2         |         | MNLI     |         |\n| ------ | ------------ | ------- | -------- | ------- |\n| Method | Spearman     | Overlap | Spearman | Overlap |\n| FS     | 0.0478       | 10.33   | 0.292    | 0.31    |\n| GS     | 0.0299       | 10.34   | 0.265    | 0.04    |\n| RPS    | 0.04         | 65.95   | 0.314    | 31.78   |\n|        | Chnsenticorp |         | LCQMC    |         |\n| Method | Spearman     | Overlap | Spearman | Overlap |\n| FS     | 0.0278       | 30.53   | 0.0399   | 1.85    |\n| GS     | 0.0253       | 30.68   | 0.0378   | 1.91    |\n| RPS    | 0.033        | 57.4    | 0.0322   | 35.41   |\n\nAs the table shows, although the change of similarity metric to retrieve topN examples affects the metric scores, the conclusions of the stability evaluation still hold.\n\n\n\nR.B: The remaining examples are as follows (remaining/total): LCQMC: 923/﻿8,802; Chnsenticorp: 1116 /1200; SST2: 296/872; MNLI: 6655/9815. Due to the varying complexity of the datasets, the extent of filtering also differs.\n\n\n\nR.C: The motivation of Section 7.3 is to validate the effectiveness of IAEval with an empirical study by simulating training data reduction. The data granularity and purpose of experiments in Section 7.3 is different from sufficiency evaluation. The sufficiency evaluation in Section 6 evaluates the evidences of each test example individually, assessing the performance of each IA method in sufficiency. However, the experiments in Section 7.3 treat evidences of all the test examples as a whole, assessing the performance of each IA method in training data reduction. The former is designed for evaluation within IAEval while the latter is designed for a specific application. Only by assessing the consistency of the results from these two experiments with different purposes can we verify the effectiveness of IAEval. Therefore, the experiments in Section 7.3 are not re-doing sufficiency evaluation.\n\n\n\nThank you again for pointing out the missing references and typo issues. We will discuss the refered papers in related work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Each evaluation metric is guided by a clearly motivated goal, which makes the proposal clear and organized.\n\n2. The evaluations are done on four different datasets, including two Chinese datasets, which improves the generality of the conclusions.\n\n3. The experiment and analysis part is thoughtfully designed. I especially find the RPS case study insightful and helpful for the readers to understand how to use those different metrics. The choice to sift out easy test examples for the experiments (discussed in P5) is also a good one.", "reasons_to_reject": "1. I have some doubts about the proposed stability evaluation. There is no doubt that this property is desirable, but there are many ways this could err. To give a few examples I can think of (1) top N similarity does not really yield similar examples; (2) human label error cause the wrong gradient to be evaluated (for GS); (3) choice of similarity metric to retrieve top N examples interfering with evaluated methods (for FS).\n\n2. The novelty might be deemed as slightly incremental compared to Pezeshkpour et al. (2021), but this is only a minor concern.", "questions_to_authors": "Two questions about stability evaluation:\n\nA1. When you select the top N similar examples, did you also control for the prediction to be the same? I think this could eliminate some part of the errors from (1) and (2) I mentioned in the first weakness point.\n\nA2. For (3), do you think it could cause interference to use the similarity of hidden representations both for assigning instance importance (for FS method) and for retrieving top-N instances? From your description, it's not clear to me whether those two similarity metrics are the same (I doubt that it's not, given that FS method does not perform well on the stability metric). Do you think your conclusion would change with an alternative similarity metric?\n\nThen a few other things:\n\nB. For the simple example filtering described in L335-352, how many examples are still left after the filtering?\n\nC. I'm not sure I quite follow the motivation of section 7.3. To me this seems like re-doing the sufficiency evaluation?", "paper_topic_and_main_contributions": "This paper designs a comprehensive evaluation framework for Instance Attribution (IA) methods. The designed evaluation framework is centered around four desired properties discussed in the feature attribution evaluations, namely: sufficiency, completeness, stability, and plausibility. The evaluation was done on four different datasets featuring different text classification tasks and on three Instance Attribution methods, feature similarity (FS), gradient similarity (GS), and Representer Point Selection (RPS).\n\nResults show that FS and GS perform the best, while RPS suffers from some performance issues. Through the analysis, the paper demonstrates that the four properties as outlined in the evaluation framework can provide a comprehensive assessment of IA methods' performance.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "L237/238: latex seems off for the two $n_{sub}$"}, "extra_text": {"Missing_References": "There is another paper in the FA literature that discusses the desired property for evaluation: Jacovi and Goldberg (2020) (https://aclanthology.org/2020.acl-main.386/) which was adopted by an analysis work: Ding and Koehn (2021) (https://aclanthology.org/2021.naacl-main.399.pdf). I found the motivation and design are similar enough to this work so probably worth discussing in the related work."}}, {"review_id": "ZoE7qqfctp", "reviewer": "Reviewer_wyYh", "summary": "", "questions": "My concerns are listed in Reasons to Reject", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your insightful review.\n\n\n\nFor concern 1:\n\nThe consistency across models is important and we provide the results of our metrics of different models in one language. For English, we report bert-base-uncased and roberta-base. For Chinese, we report ernie-3.0-base-zh and roberta-wwm-ext. We only report experiments on SST-2 and LCQMC due to space limitations. The results are as follows:\n\nSufficiency and completeness:\n\n|        |  SST2   |         |         |        |      |   LCQMC   |         |         |        |      |\n| :----: | :-----: | :-----: | :-----: | :----: | :--: | :-------: | :-----: | :-----: | :----: | :--: |\n|        |  bert   |         |         |        |      | ernie-3.0 |         |         |        |      |\n| Method |   S%    | dis_euc | dis_cos | p_diff |  C%  |    S%     | dis_euc | dis_cos | p_diff |  C%  |\n|   FS   |  98.4   |  0.291  |  0.042  | 0.206  | 84.8 |   99.6    |  0.219  |  0.022  | 0.146  | 79.8 |\n|   GS   |  99.8   |  0.288  |  0.04   |  0.2   | 57.5 |   98.8    |  0.223  |  0.024  |  0.15  | 65.5 |\n|  RPS   |  80.4   |  0.515  |  0.178  |  0.36  | 46.3 |   90.1    |  0.403  |  0.09   | 0.283  | 31.7 |\n|        | roberta |         |         |        |      |  roberta  |         |         |        |      |\n| Method |   S%    | dis_euc | dis_cos | p_diff |  C%  |    S%     | dis_euc | dis_cos | p_diff |  C%  |\n|   FS   |  98.6   |  0.426  |  0.091  | 0.301  | 91.7 |   97.8    |  0.11   |  0.007  | 0.045  |  75  |\n|   GS   |  99.8   |  0.432  |  0.094  | 0.302  | 87.7 |    98     |  0.11   |  0.008  | 0.047  | 64.1 |\n|  RPS   |   99    |  0.47   |  0.116  | 0.327  | 69.5 |    65     |  0.478  |  0.206  | 0.323  | 30.4 |\n\nStability:\n\n|        |   SST2   |         |          |         |   LCQMC   |         |          |         |\n| :----: | :------: | :-----: | :------: | :-----: | :-------: | :-----: | :------: | :-----: |\n|        |   bert   |         | roberta  |         | ernie-3.0 |         | roberta  |         |\n| Method | Spearman | Overlap | Spearman | Overlap | Spearman  | Overlap | Spearman | Overlap |\n|   FS   |  0.0537  |  42.57  |  0.0978  |  44.70  |  0.1032   |  11.99  |  0.1716  |  23.09  |\n|   GS   |  0.0378  |  42.78  |  0.0462  |  44.69  |  0.0458   |  12.00  |  0.0940  |  23.10  |\n|  RPS   |  0.0618  |  92.00  |  0.0985  |  96.99  |  0.1304   |  91.2   |  0.1637  |  97.07  |\n\nPlausibility:\n\n|        |  SST2  |       |         |       |   LCQMC   |       |         |       |\n| :----: | :----: | :---: | :-----: | :---: | :-------: | :---: | :-----: | :---: |\n|        |  bert  |       | roberta |       | ernie-3.0 |       | roberta |       |\n| Method | Cosine | Leven | Cosine  | Leven |  Cosine   | Leven | Cosine  | Leven |\n|   FS   | 0.2571 | 18.29 | 0.3045  | 19.15 |  0.7856   | 12.34 |  0.406  | 12.53 |\n|   GS   | 0.2564 | 18.3  | 0.3036  | 19.14 |  0.7856   | 12.34 | 0.4059  | 12.54 |\n|  RPS   | 0.1832 | 18.58 | 0.1454  | 17.65 |  0.7801   | 13.03 | 0.3654  | 12.66 |\n\nAs shown in the tables, the performances are consistent across models. Although the specific scores may vary due to different capacity of models, the conclusions are basically consistent.\n\n \n\nFor concern 2:\n\nWe will rephrase our claim to “most of the the existing IA methods are conducted on the classification tasks” to avoid overclaim.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The design of each perspective in IAEval is well-inspired and documented. Multiple datasets and tasks are evaluated in the carefully designed evaluation experiment, and the result consistency (on one model, see comments in Reason to Reject) is maintained across the evaluation. ", "reasons_to_reject": "The IA methods are not model-agnostic, so I would highly recommend the authors show consistent results of different models for the same language (instead of just using one model for one language). The current setup does not fully evaluate and justify the helpfulness of different IA methods, which should be the expected outcome when you are proposing an evaluation metric. \nBy following the current framing of the paper of proposing an evaluation metric for IA methods, it is necessary to show consistent results of different models. This being said I am able to change the rating based on the rebuttal if the authors have additional results. \n\nAs mentioned in the limitation, the experiments are conducted only on classification tasks. The authors made the claim 'all the existing IA methods are conducted on the classification tasks' (line 177), which seems overclaimed and I would suggest the authors to rephrase as long there exist any other IA methods conducted on non-classification tasks.\n\n\n", "questions_to_authors": "My concerns are listed in Reasons to Reject", "paper_topic_and_main_contributions": "The authors proposed an evaluation framework for Instance Attribution (IA) approaches. The framework covers four perspectives: sufficiently, completeness, stability, and plausibility. The evaluation is on 4 datasets with 3 IA methods. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Quality, clarity, originality and significance

TLDR: The paper proposes a suite of measures -- sufficiency, completeness, stability, and plausibility, to evaluate instance attribution explanations, which would benefit future work on instance attribution methods. An analysis using those is performed on four different classification tasks and three instance attribution methods. The reviewers appreciate the importance of the work and the thoughtful design of measures and experiments. The downside of the work is regarding the plausibility measure, which the reviewers have found both not needed and, most importantly, not correctly designed. Besides, the authors have addressed most reviewers' comments and provided additional results to support their claims further. 

Pros:
1. Clarirty/Quality - the design of each metric in IAEval is well motivated and documented (sPvk, wyYh)
2. Significance - instance attribution is an important problem, the work can facilitate future work by providing means to measure progress in the space (VKZW)
3. Quality - the experiment and analysis part is thoughtfully designed, the evaluations are done on four different datasets, improving the generality of observations (sPvk), and the results are consistent across metrics (wyYh)

Cons:
1. Quality  - VKZW raised a concern about plausibility evaluation that was also acknowledged by the other reviewers. The authors should reconsider including plausibility in their suite of measures as 1) the reviewers are debating whether it is always an important property of an explanation and 2) **most importantly, the current way of measuring plausibility does not reflect that an instance is plausible to humans**. In case of missing human annotations, this property should be abandoned. See also the discussion on faithfulness vs. plausiblity in Jacovi et al. where plausibility is defined as "how convincing the interpretation is to humans" where automated measures do not directly measure that.
2. Clarirty -  the authors have addressed most of the concerns of the reviewers and are kindly requested to include all clarifications and additional results in the final version and to tone down their claims that the reviewers found to be overstated.

Jakovi et al. Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness?
Decision: Accept-Findings