Representer Point Selection via Local Jacobian Expansion for Post-hoc Classifier Explanation of Deep Neural Networks and Ensemble Models
Year: 2021

Abstract:
Explaining the influence of training data on machine learning model predictions is a critical tool for debugging models through data curation. A recent appealing and efficient approach for this task was provided via the concept of Representer Point Selection (RPS), i.e. a method the leverages the dual form of l2 regularized optimization in the last layer of the neural network to identify the contribution of training points to the prediction. However, two key drawbacks of RPS-l2 are that they (i) lead to disagreement between the originally trained network and the RPS-l2 regularized network modification and (ii) often yield a static ranking of training data for test points in the same class, independent of the test point being classified. Inspired by the RPS-l2 approach, we propose an alternative method based on a local Jacobian Taylor expansion (LJE). We empirically compared RPS-LJE with the original RPS-l2 on image classification (with ResNet), text classification recurrent neural networks (with Bi-LSTM), and tabular classification (with XGBoost) tasks. Quantitatively, we show that RPS-LJE slightly outperforms RPS-l2 and other state-of-the-art data explanation methods by up to 3% on a data debugging task. More critically, we qualitatively observe that RPS-LJE provides stable and individualized explanations that are more coherent to each test data point. Overall, RPS-LJE represents a novel approach to RPS-l2 that provides a powerful tool for sample-based model explanation and debugging.

heading: 1 Introduction
text: Deep learning as well as ensemble methods such as XGBoost [2] have revolutionized the field of machine learning and led to unprecedented levels of accuracy in a variety of data-driven prediction applications. However, it can be extremely challenging to debug these complex methodologies when they make incorrect predictions. The first step in this debugging process is explaining the prediction, which can stem from a variety of interpretive processes. For example, we may seek to understand the key features [13, 16] or salient regions [18, 20] that led to a prediction, or in a more recent line of work, to understand the influence of the training data on test data predictions [1, 11, 15, 23].
In this paper we focus on a particularly appealing (and efficient) method for understanding the impact of training data on test predictions, namely Representer Point Selection (RPS) [23], which leverages an application of the representer theorem [17] to deep neural networks. Specifically, it uses the dual form of l2 regularized optimization in the last layer of the neural network to identify the contribution of training data to the test prediction. Compared to model-agnostic approaches [1, 9, 10] that estimate
∗Contributions were made while the author was at the University of Toronto. †Affiliate to Vector Institute of Artificial Intelligence, Toronto.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
data influence after optimization, the RPS approach directly integrates with the prediction model to provide a high-fidelity white-box interpretation of the prediction. Compared to the influence function-based approach [11], the RPS is more computationally efficient as it focuses on the last layer of a neural network instead of all parameters.
While holding many advantages, we note the existing RPS approach faces two key drawbacks. First, as the RPS approach requires fine-tuning the last layer of the neural network with l2 regularization, it leads to a disagreement between the originally trained network and the l2 regularized network and is furthermore highly sensitive to l2 regularization strength (cf. Figure 1(a)). Thus, it is hard to justify if the explanation produced by RPS is for the original model or the modified model. Second, we observe that the explanations produced by RPS are nearly identical for all test samples that are classified into the same category, which provides more of a class-level rather than instance-level explanation for RPS (cf. Figure 1(b)). While the RPS approach is appealing in principle, both of these drawbacks significantly harm the utility of the RPS explanation approach in practice.
To address these issues, this paper presents an alternative derivation for Representer Point Selection (RPS) based on a Local Jacobian Taylor expansion (LJE), which corrects for both aforementioned problems. We empirically compared RPS-LJE to the RPS-l2 on image classification (with ResNet [6]), text classification (with Bi-LSTM), and credit analysis (with XGBoost [2]) tasks. Quantitatively, we show RPS-LJE outperforms RPS-l2 and other state-of-the-art data explanation methods by to up 3% on a data debugging task. Qualitatively — and perhaps most importantly — the RPS-LJE provides stable (i.e., no need for l2 tuning) and diverse explanations that are more coherent to the test data.

heading: 2 Representer Points Selection for Explaining Deep Neural Networks

heading: 2.1 Preliminaries
text: In a machine learning context, representer theorems [17] loosely state that under certain conditions, model prediction ŷt of a test sample xt can be expressed as a linear combination of kernel evaluations K(xi,xt) between each training point xi and the test sample xt such that
ŷt = n∑ i αiK(xi,xt) , (1)
where αi is the weight of the training data point i that is independent from the test point t.
As the representer theorem linearly separates the contribution of training samples to the prediction, it has been introduced into the deep learning model interpretation research [23] for tracking the training data influence on predictions. Concretely, the current Representer Point Selection (RPS) approach [23] introduces an l2 regularization term (for the last layer’s parameters ΘL ⊂ Θ) into a model’s objective function, which facilities its derivation such that it fulfills Equation 1 by setting
αi = − 1
2λn ∂L(xi, yi,Θ) ∂ΘLφ(xi) and K(xi,xt) = φ(xi)Tφ(xt) , (2)
where λ denotes the hyper-parameter of l2 regularization term, n denotes the number of total training samples, and φ(xi) and φ(xt) are the representations of input xi and xt at layerL. As this explanation approach works on the last layer of a network and is efficient to compute, it demonstrates a significant advantage over Influence Function-based approaches [11] in terms of computational efficiency.

heading: 2.2 Caveats of Current Representer Points Selection
text: As mentioned, to facilitate its derivation, the current RPS approach introduces an l2 regularization term into the computation, which inevitably violates the saddle point assumption its derivation relies on. To address this, the RPS conducts fine-tuning on the trained model with a new objective function
Θ∗L = arg min ΘL
{ 1
n n∑ i
L(Φ(xi,Θ),Φ(xi,Θgiven)) + λ‖ΘL‖2 } , (3)
where Φ(xi,Θgiven) and Φ(xi,Θ) represents the post-activation prediction of xi of the given model and fine-tuned model, respectively. Correspondingly, the kernel weights in the expression of Equation 2 are updated as
αi = − 1
2λn ∂L(xi, yi,Θ∗) ∂Θ∗Lφ(xi) . (4)
As the data importance factor αi is computed with respect to the l2-modified parameter set Θ∗, the explanations correspond to the modified model instead of the original model. Indeed, the gap between the two sets of parameters is sensitive to the hyper-parameter setting of the l2 regularizer. This reflects our previous observation in Figure 1(a) showing that prediction explanations can vary significantly based on different l2 regularizer weightings.
Also, by inspecting the expression of training data contribution
Θ∗Lφ(xt) = αiK(xi,xt) = − 1
2λn ∂L(xi, yi,Θ∗) ∂Θ∗Lφ(xi) φ(xi) Tφ(xt)
= − 1 2λn ∂L(xi, yi,Θ∗) ∂Θ∗Lφ(xi) ∂Θ∗Lφ(xi) ∂Θ∗L φ(xt) = − 1 2λn ∂L(xi, yi,Θ∗) ∂Θ∗L︸ ︷︷ ︸
dominant term
φ(xt), (5)
we note the first-order derivative of the loss of a single training example is the dominant term that suppresses a small difference of test example representation φ(xt) among test samples in the same class. Figure 2(a) shows the strong positive correlation between the dominant term and the relevant order of the training data contribution. This supports our observation in Figure 1(b) that explanations tend to be identical for the test samples being classified into the same category making RPS more of a class-level than instance-level explanation that defeats the intent of uncovering instance-level prediction errors!

heading: 3 Representer Point Selection via Local Jacobian Expansion
text: In an effort to preserve the conceptual and computational advantages of the previously discussed RPSl2 methodology while improving fidelity of the explanations to the original model and encouraging test instance-level explanation, we now present a novel derivation of Representer Point Selection through a Local Jacobian Expansion (RPS-LJE). Similar to RPS-l2, RPS-LJE also expresses the pre-activation prediction outcome of a well-trained classification model (near the saddle point) as a linear combination of kernel evaluations between the test sample and the training points. Specifically, we use a first-order Taylor expansion on the Jacobian matrix in our derivation, which avoids the problems introduced by imposing the additional l2 regularization term in RPS-l2. First, we begin with the formal problem definition.
Consider a classifierMΘ† (the target model in the following context) that has learned to map input observation xt ∈ IRd to an output space yt ∈ {1 · · · k}, whose pre-activation prediction3 M̄Θ† could be formulated as
ŷt = M̄Θ†(xt) = Θ † Lφ(xt), (6)
where bold yi ∈ IRk represents the prediction vector (one element for a class). Our goal of prediction explanation is to reformulate Equation 6 into the format of Equation 1 such that the contribution of each training data point on the prediction is linearly separable.
While the above setting appears restricted, it represents a large group of machine learning models commonly used in practice; many well-known models, such as ResNet [6] for images, Transformers [22] for text, and even XGBoost [2] for tabular classification tasks, can be expressed in this simple formulation leveraging a feature embedding stage followed by a pre-activation linear prediction stage.

heading: 3.1 First-order Taylor Expansion on Jacobian Matrix
text: We begin by presuming we are given a well-trained target modelMΘ† , whose loss derivative with respect to the decision making parameter ΘL is close to a saddle point such that
0 ≈ n∑ i=1 ∂L(xi, yi,Θ) ∂ΘL ∣∣∣∣∣ ΘL=Θ † L . (7)
Here, we assume that the loss term L is twice-differentiable and strictly convex with respect to the last linear layer parameters ΘL. In the case of non-convexity, we can adopt the approach proposed in the Influence Function [11] to form a convex quadratic approximation of the loss by introducing a damping term .
With any parameter assignment Θ∗L close to the model parameter Θ † L, we can further rewrite Equation 7 via a first-order Taylor expansion such that
0 ≈ 1 n n∑ i=1 ∂L(xi, yi,Θ) ∂ΘL ∣∣∣∣ ΘL=Θ∗L︸ ︷︷ ︸
∇ΘLL(xi,yi,Θ∗)
+(Θ†L −Θ ∗ L)
[ 1
n n∑ i=1 ∂L2(xi, yi,Θ) ∂Θ2L ∣∣∣∣ ΘL=Θ∗L ] ︸ ︷︷ ︸ HΘ∗
L
def = 1n ∑n i=1∇2ΘLL(xi,yi,Θ ∗)
. (8)
This expansion yields the model parameter Θ†L as a linear combination of functions of each training data point
Θ†L = Θ ∗ L − n∑ i=1 1 n H−1Θ∗L∇ΘLL(xi, yi,Θ ∗) + ξ, (9)
3For the trained classifier, the predictions before and after activation are consistent.
where ξ is a negligible error term. Intuitively, this equation shows the model parameter Θ†L could be reconstructed through a one-step gradient descent from the nearby parameter assignment Θ∗L with a dynamic learning rate 1nH −1 Θ∗L (as a matrix).
We remark that the expression in Equation 9 represents the contribution of each data point as a linearly separable function, which is fundamentally different from recording gradients at training time as a data importance score [15]. That is, the gradients during model training are sensitive to both training order and optimizer settings.
For Equation 9 to hold after the Taylor expansion, Θ∗L has to be close to the model parameter Θ † L. Therefore, we propose to estimate Θ∗L through a one-step stochastic gradient ascent from the trained model using any optimizer (Adam, RMSProp, etc), such that
n∑ i=1 L(xi, yi,Θ∗) > n∑ i=1 L(xi, yi,Θ†) (10)
and Θ∗L is maintained close to the original model parameter Θ † L with a small loss shift.

heading: 3.2 Representer Point Selection with Local Jacobian Expansion (RPS-LJE)
text: With the derivation of Equation 9, we can now reformulate the pre-activation prediction of a test point xt as a weighted linear combination of kernels leading to our key result of the final form of RPS-LJE (see Appendix B for a more detailed derivation):
Θ†Lφ(xt) = n∑ i=1 [ 1 n Θ∗L − 1 n H−1Θ∗L∇ΘLL(xi, yi,Θ ∗) ] φ(xt)
= n∑ i=1 [ Θ∗L 1 φ(xi)n − 1 n H−1Θ∗L ∂L(xi, yi,Θ∗) ∂Θ∗Lφ(xi) ] ︸ ︷︷ ︸
αi
φ(xi) Tφ(xt)︸ ︷︷ ︸
K(xi,xt)
. (11)
Similar to the claim of the original RPS using an l2 norm (RPS-l2), when a training data xi is close to the test point xt in the representation space with a large positive value αik , the prediction score for class k is increased. On the other hand, when the αik is a large negative value, the prediction score for class k is then decreased.
The first notable difference between our derivation of RPS-LJE and the original RPS-l2 is that the αi term now contains an inverse of second-order derivative that estimates the correlation among the parameter entries in Θ†L. This modification mitigates the risk of over-weighting training data points with a large predictive error that causes a small number of data points to dominate the explanations as discussed in Section 2.2 and Figure 1(b). Figure 2(b) illustrates the effects of correction.
The second difference is that the prediction explanation of our derivation of RPS-LJE is faithful w.r.t. the original modelMΘ† instead of the l2-regularized modelMΘ∗ . To clarify, the left hand side of of Equation 11 is different with that of RPS-l2 in Equation 5. In addition, data importance factor αi in the new derivation no longer depends on the l2 regularization hyper-parameter. Hence, the changes in the RPS-LJE framework directly address the problem described in Figure 1(a).

heading: 3.3 Relation to Influence Function-based Interpretation
text: The Influence Function method [11] estimates the prediction importance of each training data point by up-weighting the data points with small perturbation that results in the following final expression
Iup,loss(xi,xt) = −∇ΘLL(xt, yt,Θ†)T H−1Θ†L ∇ΘLL(xi, yi,Θ†)︸ ︷︷ ︸
Iup,params(xi) def = dL(xt,yt,Θ†)
d
∣∣∣ =0
. (12)
If we rewrite the above equation by expanding the first-order derivatives with chain rule such that
Iup,loss(xi,xt) = − ∂L(xt, yt,Θ†) ∂Θ†Lφ(xt) H−1 Θ†L ∂L(xi, yi,Θ†) ∂Θ†Lφ(xi) φ(xi) Tφ(xt), (13)
we note the expression is very similar to RPS-LJE proposed in this work (as described in Equation 11) except two subtle differences:
• The Influence Function includes a multiplicative factor ∂L(xt,yt,Θ †)
∂Θ†Lφ(xt) that is relevant to the test
point, whereas the RPS-LJE has an additive factor 1nΘ ∗ Lφ(xt) that is also only relevant to the test
point. • The derivative in the Influence Function respects the original model parameters Θ†, whereas
RPS-LJE’s derivatives apply to Θ∗.
Ultimately, these differences help explain the comparative performance of these methods in our experiments (cf. Section 4.3), but — because they are also subtle — they help explain the similarity of explanations produced by RPS-LJE and Influence Functions for image classification explanation.

heading: 4 Experimental Evaluation
text: We perform a range of experiments with multiple datasets (and corresponding model architectures) and evaluate the performance of RPS-LJE against the original RPS-l2 as well as Influence Functionbased approaches. Note that all three methods do not specifically require access to the training history, which is rarely available in deployment settings. In contrast, TracIn [15], which requires access to training checkpoints, is not in the scope of our comparison. The goal of these experiments is to demonstrate that the alternative derivation of RPS described in this work, RPS-LJE, successfully addresses the two critical drawbacks of RPS-l2 and leads to substantial performance improvement on multiple use cases, including data debugging and model behavior explanation. All code to reproduce these results is publicly available on Github.4

heading: 4.1 Sanity Check of Representer Point Selection with Local Jacobian Expansion
text: Before describing our quantitative analysis, we first start with a sanity check to show the proposed RPS-LJE approach indeed addresses the problem we highlighted earlier in Section 2.2 and Figure 1. Concretely, we repeat our showcase in Figure 3 using RPS-LJE with the same test examples and target model. Here, we highlight the following observations:
• As shown in Figure 3(a), the RPS-LJE can produce an individualized explanation for test samples in the same category. This observation reflects our previous description in Figure 2(b), where the term αiφ(xi) in RPS-LJE no longer dominates the explanation ranking.
4https://github.com/echoyi/RPS_LJE
• To test the sensitivity of one-step gradient descent in RPS-LJE (used to search Θ∗L as described in Section 3.1), we experimented with multiple learning rates γ in a large range γ ∈ {5e-4 · · · 1e-3, 5e-3 · · · 1e-2}. As shown in Figure 3(b), the explanations vary little with the learning rate.

heading: 4.2 Fixing Mislabeled Training Examples (Data Debugging)
text: In this experiment, we simulate an application scenario where human experts need to inspect the data annotation quality of the training set that directly impacts the model performance on the test samples.
We simulate data debugging on three classification tasks, including 1) binary image classification with ResNet [6] (ResNet-20) on CIFAR-10 [12] dataset (horses vs. cars) presented in the RPSl2 paper [23], 2) sentiment analysis with Bi-LSTM [7] on IMDB [14] dataset, and 3) credit risk identification with XGBoost on German Credit dataset [4]. The datasets are intentionally corrupted by randomly flipping 20-30 percent of the data points’ labels, naturally resulting in low test accuracy. Our goal is to identify which data points’ label corruption are more harmful and correct them as early as possible. With the partially corrected dataset (after each 5% of checking), we retrain the models and record the test accuracies for each task. Experiments are repeated for ten random split and corruptions on CNN and XGBoost and five random split and corruptions on RNN; we report the average result. For RPS approaches (LJE and l2), we pick data points that have the largest selfprediction contribution |αiK(xi,xi)| 5 as suspicious corrupted data points. For Influence Functions, we use self-influence (see Equation 12) as the score of ranking. For the random baseline, we picked the data to check randomly.
Figure 4 shows the experimental results. Here, we highlight the following observations:
• RPS-LJE either slightly dominates RPS-l2 or performs comparably to it. This demonstrates that it is an effective data debugging tool comparable to as RPS-l2, since both show a significant performance gap to the random baseline on all three tasks. • RPS-LJE shows slightly better performance than both Influence Function and RPS-l2 on all tests. For the image classification task (Figure 4(a)), the RPS-LJE shows 3% better performance (around 60 more mislabelled data identified) than original RPS-l2 after searching through 40% of all training data. For the sentiment analysis task (Figure 4(b)), the performance improvement is about 1% (around 35 more mislabelled data found than the others).
5We note that, for RPS-l2, there is no difference on using |αi| or |αiK(xi,xi)| to select corrupt data points.

heading: 4.3 Understand Model Behavior through Prediction Explanation
text: In this section, we qualitatively analyze the prediction explanation ability of candidate sample-based explanation models on the three tasks mentioned in Section 4.2. While we compare the explanation results in this experiment, we only intend to compare and contrast the three explanation approaches.
CIFAR-10 Image Classification with ResNet
In Figure 5, we visualize the top-4 training points (both positively and negatively) that have the strongest influence on the prediction of test samples from Influence Function, RPS-l2, and the proposed RPS-LJE approaches on the image classification model. Here, we make the following key observations:
• The explanation produced by RPS-LJE is more similar to that of Influence Function than RPS-l2 but with some differences in explanation order. This observation aligns with our previous conjecture in Section 3.3 that the RPS-LJE’s formula of computing training data importance appears similar to that of Influence Functions. • Explanations produced by RPS-l2 contain a lot of repetition. Training example ID-3291 and ID-7655 are in all three test samples’ explanation lists.
IMDB Sentiment Analysis with LSTM
Table 1 shows the top-3 explanations produced by the candidate explanation approaches on the IMDB sentiment analysis model (Bi-LSTM). Here, we make the following key observations:
• Among the three explanation approaches, RPS-LJE’s explanation is more coherent to the properties of the test points. For instance, the explanations of RPS-LJE for Sample 1 all start with “This”, which hints to the auditor that the model has made a generalization (whether correct or not) that positive sentiment sentences start with “This” based on the identified training data. Similarly, for Sample 1 and 3, we see the explanations follow the same narrative style by starting with “I”. • Explanations provided by Influence Functions generate similar explanations (2 out of the top 3) with RPS-LJE. But Influence Functions sometimes produce confusing explanations. For example, Sample 2 has a positive comment, but Explanation 1 provided by the Influence Function has a negative sentiment. Similarly, Sample 3 and its Explanation 3 have the opposite sentiment.
German Credit Risk Analysis with XGBoost
Table 2 lists the predictions of three test samples in the German Credit dataset through the XGBoost model with corresponding explanations produced by RPS-LJE and Influence Function. Here, we
removed RPS-l2 from the candidate list as it requires fine-tuning the model with l2 normalization, which is incompatible with tree ensemble models. Here, we highlight the following observations:
• The explanation produced by RPS-LJE is more similar to the test point than that of Influence Function in the sense of sharing similar feature values. For instance, the “Checking Account" values of the RPS-LJE explanation always align with the test samples, but that of the Influence Function does not. • Influence Function tends to provide diverse explanations for each sample case, where the explanations produced by it show more or fewer differences.

heading: 5 Conclusion and Discussion
text: We presented an approach for explaining the impact of training data on a test prediction, called Representer Point Selection via Local Jacobian Expansion (RPS-LJE). Our approach aimed to correct two drawbacks of the existing Representer Point approach (RPS-l2) [23], namely that it often 1) produces identical explanations for different instances in the same class and 2) produces highly varying explanations based on the strength of an l2 regularization modification to the original model. We began by observing these problems empirically and then analyzing the RPS-l2 derivation to reveal the technical source of these problems. We then proposed corrections to derive a novel form of RPS based on a local Jacobian Taylor expansion that addresses the technical limitations of the RPS-l2.
We conducted multiple experiments that quantitatively and qualitatively analyzed the proposed RPSLJE against existing state-of-the-art approaches, RPS-l2 and Influence Function. Our experiments empirically show that the proposed RPS-LJE fulfilled our expectation in terms of correcting RPS-
l2’s problems. It produces individualized explanations instead of a class-level explanation and quantitatively performs comparably to, or outperforms, existing data-explanation approaches.

heading: 6 Scope of Application
text: This paper presents a novel method of sample-based model explanation for classifiers including deep neural networks and ensemble models. As stated in Section 3, RPS-LJE is applicable to classification models with a linear last layer before the activation function. While this requirement may seem restrictive, many widely-adopted classifiers satisfy them.
In Section 3.1, we state our assumption that the given model is well-trained (near the saddle point), and thus the gradient of loss with respect to the parameters is close to zero. RPS-l2 [23] and Influence Function [11] also makes the same assumption. The assumption often holds in practice, as many classification models will be trained to near-convergence before deployment. However, one exception to this assumption would be when early stopping is used as a form of regularization.

heading: 7 Broader Impacts
text: Deep learning as well as ensemble methods such as XGBoost have shown exceptional performance in a variety of data-driven prediction applications including image classification, sentiment classification, and risk classification (cf. Section 4). This paper proposes a methodology to explain these complex classifiers via estimating the influence from each training datum on model predictions.
Sample-based explanation methods are critically important for validating and improving classifiers. For example, explanations can increase the interpretability and transparency of a model’s decision making process, and thus help to assess the fairness of the model. Furthermore, monitoring training data quality can facilitate the debugging process and thus improve model performance. One potential negative impact is that the use of training data for explanations may raise privacy concerns in some situations; however, measures for data anonymization may (partially) help mitigate such issues.
Acknowledgments and Disclosure of Funding
Yi Sui was funded by a University of Toronto Dean’s Spark award to Scott Sanner. Ga Wu was funded by a Canadian NSERC Discovery Grant award to Scott Sanner.

Review summaries:
This paper addresses the problem of model explainability by looking at which training examples were responsible for the model’s predictions. The contributed model takes inspiration from RPS and changes its computation of the data importance factor such that it becomes less sensitive to the choice of hyper-parameters and leads to higher diversity in the explanations. The proposed RPS-LJE is compared against two baseline approaches from the state of the art.

The aim of this paper is to identify the contribution of the training data to the prediction. This work highlights the short comings of the existing Represented Point Selection (RPS-$l_2$) both empirically and by analyzing the derivation of RPS$-l_2$, and proposes an improved method called RPS-LJE. RPS-LJE is a clear improvement over RPS-$-l_2$ both qualitatively and quantitatively. Qualitatively, it is observed that the explanations produced by RPS-LJE isn't repetitive like RPS$-l_2$. Quantitatively, it performs comparably to, or better that the existing data-explanation approaches.

The paper proposes a new representer decomposition based on the approximate optimal condition. This address issues for RPS-L2 by 1. this does not rely on regularization strength 2. this provides an explanation that varies much more than RPS-L2 across different testing points. They verify this in their experiments.

This paper addresses sample based explainable AI, where a model prediction at test time is explained by producing a ranked list of training set instances.  They build upon a method called Representer Point Selection, which develops a representer theorem wherein contribution to test loss is decomposed into terms involving the features of the last layer the trained network.  The authors address two shortcomings of the method by developing  a local Jacobean Taylor expansion that alleviates both.  They present experiments on data set cleaning (in line with the original RPS paper), as well as on three other tasks commonly evaluated with sample based explainability.