CoRTX: Contrastive Framework for Real-time Explanation
Authors: Yu-Neng Chuang
Guanchu Wang
Fan Yang
Quan Zhou
Pushkar Tripathi
Xuanting Cai
Xia Hu
Venue: ICLR 2023 poster (2023)
Keywords: Interpretability
explainability
real-time explanation
feature attribution
feature importance ranking

Abstract: Recent advancements in explainable machine learning provide effective and faithful solutions for interpreting model behaviors. However, many explanation methods encounter efficiency issues, which largely limit their deployments in practical scenarios. Real-time explainer (RTX) frameworks have thus been proposed to accelerate the model explanation process by learning an one-feed-forward explainer. Existing RTX frameworks typically build the explainer under the supervised learning paradigm, which requires large amounts of explanation labels as the ground truth. Considering that accurate explanation labels are usually hard to obtain, due to constrained computational resources and limited human efforts, effective explainer training is still challenging in practice. In this work, we propose a COntrastive Real-Time eXplanation (CoRTX) framework to learn the explanation-oriented representation and relieve the intensive dependence of explainer training on explanation labels. Specifically, we design a synthetic strategy to select positive and negative instances for explanation representation learning. Theoretical analysis show that our selection strategy can benefit the contrastive learning process on explanation tasks. Experimental results on three real-world datasets further demonstrate the efficiency and efficacy of our proposed CoRTX framework.

Published as a conference paper at ICLR 2023
CORTX: CONTRASTIVE FRAMEWORK FOR REAL-TIME EXPLANATION
Yu-Neng Chuang*1, Guanchu Wang*I, Fan Yang', Quan Zhou?, "aoouu,Pustiee Pushkar Tripathi", Xuanting Cañ², and Xia Hu-
'Rice University, "Meta Platforms, Inc. (ynchuang, guanchu. wang, fyang, xia hu) edu, (quanz, pushkart, saixuanting)e com
ABSTRACT
Recent advancements in explainable machine learning provide effective and faithful solutions for interpreting model behaviors. However, many explanation methods encounter efficiency issues, which largely limit their deployments in practical scenarios. Real-time explainer (RTX) frameworks have thus been proposed to accelerate the model explanation process by learning one-feed-forward explainer. Existing RTX frameworks typically build the explainer under the supervised learn- ing paradigm, which requires large amounts of explanation labels as the ground truth. Considering that accurate explanation labels are usually hard to obtain due to constrained computational resources and limited human efforts, effective explainer training is still challenging in practice. In this work, propose COntrastive Real-Time eXplanation (CoRTX) framework to learn the cetaaaationoorinnteei representation and relieve the intensive dependence of explainer training on expla- nation labels. Specifically, we design a synthetic strategy to select positive and negative instances for the learning of explanation. Theoretical analysis show that our selection strategy can benefit the contrastive learning process on explanation tasks. Experimental results three real-world datasets further demonstrate the efficiency and efficacy of our proposed CoRTX framework. Our source code is available at: betps://gitaie com. ynchuang Saanchuann/CoTTX77222 CCRTTX720
INTRODUCTION
The remarkable progress in explainable machine learning (ML) significantly improves the model transparency to human beings (Du et al. 2019). However, applying explainable ML techniques to real-time scenarios remains to be challenging task. Real-time systems typically require model explanation to be not only effective but also efficient (Stankovic et al. 1992p. Due to the requirements from both stakeholders and social regulations (Goodman & Flaxman 2017 Floridi 2019), the efficient model explanation is necessary for the real-time ML systems. such as the controlling systems (Steel & Angwin| 2010), online recommender systems (Yang et al., 2018), and healthcare monitoring systems (Gao et al. 2017). Nevertheless, existing work on non-amortized explanation methods has high explanation latency, including LIME (Ribeiro et al. 2016), KernelSHAP (Lundberg & Lee 2017). These methods rely on either multiple perturbations or backpropagation in deep neural networks (DNN) for deriving explanation (Covert & Lee 202T Liu et al. 202T), which is time-consuming and limited for deployment in real-time scenarios.
Real-time explainer (RTX) frameworks have thus been proposed to address such efficiency issues and provide effective explanations for real-time systems (Dabkowski & Gal 2017 Jethani al. 20216). Specifically, RTX learns an overall explainer on the training set by using the ground-truth explanation labels obtained through either exact calculation or approximation. RTX provides the explanation for each local instance via single feed-forward process. Existing efforts on RTX can be categorized into two lines of work. The first line (Schwab & Karlen 2019 Jethani et al.) 2021b) Covert et al. 2022) explicitly learns an explainer to minimize the estimation error regarding to the amproximated explanation labels. The second line (Dabkowski Gall 2017 Chen al. 2018 Kanehira & Harada
2019) trains feature mask generator subject to certain constraints on pre- pre-defined label distribution.
'These authors contributed equally this work.

---

Published as a conference paper at ICLR 2023
Despite the effectiveness of existing RTX frameworks, recent advancements still rely the large amounts of explanation labels under the supervised learning paradigm. The computational cost of obtaining explanation labels is extremely high (Roth 1988 RTX's deployment in real-world scenarios.
Winter 2002). which thereby limits the of each data instance without any ground-truth explanation label. The latent explanation of an instance is defined as a vector encoded with explanation information. Contrastive learning has been widely exploited for improving the learning processes of downstream tasks by providing well-pretrained representative embeddings (Arora et al., 2019 He et al. 2020). In particular, srisntoeeeted selection strategies of positive and negative pairs (Chen al., 2020, Khosla et al., 2020) can shape the representation properties through contrastive learning. Motivated by the such contrastive scheme, CoRTX develops an explanation-oriente contrastive framework to learn the latent explanation, with the goal of further fine-tuning an explanation head in the downstream tasks.
To tackle the aforementioned challenges, propose COntrastive Real-Time eXplanation (CoRTX) framework based on the contrastive learning techniques. CoRTX aims to learn the latent explanation
CoRTX learns the latent explanation to deal with the explanation tasks by minimizing the contrastive loss (Van den Oord et al. 2018). Specifically, CoRTX designs a synthetic positive and negative sampling strategy to learn the latent explanation. The obtained latent explanation can then be transformed to feature attribution ranking by fine-tuning corresponding explanation head using tiny amount of explanation labels. Theoretical analysis and experimental results demonstrate that CoRTX successfully provide the effective latent explanation for feature attribution and ranking tasks. Our contributions can be summarized as follows:
CoRTX provides a contrastive framework for deriving latent explanation, which can effectively reduce the required amounts of explanation label;
Theoretical analysis indicate that CoRTX can effectively learn the latent explanation over the training set and strictly bound the explanation error;
Experimental results demonstrate that CoRTX can efficiently provide explanations to the target model, which is applicable to both tabular and image data.
2
PRELIMINARY
2.1 NOTATIONS
We consider an arbitrary target model f(-) to interpret. Let input feature be = "r Fvll E X. where denote the value of feature i for <i< M. The contribution of each feature to the model output can be treated as a cooperative game on the feature set X. Specifically, the preceding difference f( - f(ās) indicates the contribution of feature i under feature subset S C U \ {i}. where U is the entire feature space. The overall contribution of feature is formalized as the average preceding difference considering all possible feature subsets S, which can be formally given by
s (x) == (scun(() [f(ãsu()) f(Es]
(1)
where zs= x x (1- x, denotes the perturbed sample, S= 1s E (0, 1 is a masking vector of S, and I, = E|x x P(x)] denotes the reference values from feature distribution P(z). The computational complexity of Eqquationngroon grows exponentially with the feature number M, which cumbers its application to real-time scenarios. To this end, we propose an efficient explanation framework for real-time scenarios in this work.
2.2 REAL-TIME EXPLAINER
Different from the existing non-amortized explanation methods (Lundberg & Lee 2017 Lomeli et al. 2019) that utilize local surrogate models for explaining data instances, RTX trains global model to provide fast explanation via one feed-forward process. Compared with the existing methods, the advantages of RTX mainly lie in two folds: (1) faster explanation generation; and (2) more robust explanation derivation. Generally, existing RTXs attempt to learn the overall explanation distribution using two lines of methodologies, which are shapley-sapping. based approaches (Wang et al. 2021 2018
Jethani et al. 2021b; Covert et al., 2022) and feature-selection based approaches (Chen et al.
"Other statistic measurement can also be adopted for generating the reference value.
2

---

Published as a conference paper at ICLR 2023
Derivation
Fine-tuning
Latent Explanation Learning
Fine-tuning with few labels
Positive
Few Labels
Anchor
h;
Negative
g (g)
n 0g) Explanation Hcad
Explanation Results
Explanation Encoder
Figure The explanation pipeline of the CoRTX framework, where h, denotes the latent explanation.
Dabkowski & Gal 2017 Kanehira Harada 2019). The first line enforces the explainer to simulate given approximated Shapley distribution for generating explanation results. A representative work, FastSHAP (Jethani et al.) (2021b), exploits a DNN model to capture the Shapley distribution among training instances for efficient real-time explanations. The second line assumes the specific pre-defined feature patterns or distributions, and formulates the explainer learning process based on the given assumptions. One of the work (Chen et al. 2018) provides feature masking generator for real-time feature selection. The training process of the mask generator is under the constraint of and ceature-selection
the given ground-truth label distribution. Both caapley-sampligg methods highly depend on explanation labels during the training phase. Although labels may bring strong supervised information to the explanation models that benefit explanation tasks, generating high-quality labels typically requires high computational time, which makes the labels hard to acquire. In this work, we propose an unsupervised learning paradigm CoRTX, which can significantly reduce the dependency the explanation labels. The proposed CoRTX benefits the explanation tasks by exploiting a contrastive framework to generate latent explanations without relying on the labels.
2.3 LIMITATION OF SUPERVISED FRAMEWORK
Supervised RTX typically relies on large quantities of ground- truth explanation labels, which limits its application in practice. We experimentally show that supervised RTX suffers from the performance degradation when explanation labels insuffi- cient. Specifically, taking the Shapley value as the ground truth, supervised RTX learns minimize the cross entropy training set, and then estimates the feature ranking on the testing for evaluation. The results on Census dataset (Dua & Graff 2017) demonstrate the intensive dependence of supervised RTX on the amount of explanation labels. The performance of fea- ture ranking versus the exploitation ratio of explanation label is shown in Figure [ where the implementation details are intro- duced in Appendix [ Here, we observe that the performance of feature ranking significantly enhances as the ratio of label increases. Supervised RTXs outperform FastSHAP, which is trained with approximated labels, under 5% label usage. However, the complexity of calculating Shapley values generally grows exponentially with the feature number, which makes it challenging train RTX with large amount of labels. We propose CoRTX in this paper to train the explainer in light-label manner, where only small amount of explanation labels are used for fine-tuning the downstream head.
the
Bank
Sepervint REX (a Lagee)
Superrieed DEX (6-tager)
PaeSSAAP
0.01
3.05
01
0.55
05 0.76 10
Proportion
Figure The Rank ACC v.s. the explanation label ratio in RTX.
CONTRASTIVE REAL-TIME EXPLANATION
We systematically introduce the CoRTX framework in this section. Figure iillsttrasss the overall pipeline of CORTX. In particular, our pipeline first uses an explanation encoder to learn the latent explanation in contrastive way, and then explanation head is further tuned with small amount
3

---

Published as a conference paper at ICLR 2023
of explanation labels. An explanation-oriented positive augmentation and negative sampling strategy is designed for the latent explanation learning process.
3.1 POSITIVE DATA AUGMENTATION TOWARD SIMILAR EXPLANATION
Different from the conventional data augmentation (Liu et al. 2021 for representation learning, CoRTX develops an explanation-orientee data augmentation strategy for training RTX, where the augmented positive and negative instances of explanation are proved be beneficial (Kim al.) (olranDhuandha et al. 2018). Specifically, considering an anchor data instance a E A, the set of synthetic positive instances x is generated via m times of independent perturbations on a::
x+ (S
(1 18-S,) G æt S B(M, A), s fimm) m),
(2)
where S, is sampled from M-dim binomial distribution, and controls the probability density,
CoRTX provides the ioplanation-orientte data augmentation by selecting the compact positive instances / from X Intuitively, ă* is expect to obtain the most similar model explanation as the corresponding which means the feature attribution between 2 and æ are highly similar. However, the greedy selection of x+ needs the explanation score of each instance in X which is hard to access in practice. To tackle the issue, propose Theorem to guide the selection of xx* through set of synthetic positive instances without accessing the explanation scores. The proof of Theorems | shown in Appendix A
Theorem 1 (Compact Alignment). Let f() be K Ceciisschiteoonnnnnnooii funcciioon and ((x))
[ (zt).... ................... ,0. (a)) be the feature importance scores, where dx(x) == Esgun (
f( Given perturbed positive instance ã E satisfying minigigM (x(x)) 0, the explana-
tion difference [<x())) ( l22 is bounded by the prediction difference [f(x) - f()) as
(x) 2 S (1+ (2)) [(=) + VM
where 70 = Ky ||21|| and K 2 is the Lipschitz constant function f(-).
Theorem basically shows that the explanation difference can be bounded by the prediction difference between the anchor and the perturbed positive instance æ. The compact positive instance ă in Theorem | can derive the minimal difference of model outputs and further result similar explanation to the anchor æ;. Following Theorem CoRTX selects क. X by holding the minimum preceding difference between of f(x) and (z:)) Formally, the selection strategy can be indicated as:
= =rg min xcE
1f(x) - f(x.)).
where f(-) is the target model for explanation.
Selecting excessively positive instances has been proved to degrade contrastive learning (Zhu et al.) 202T). The excessively positive instances refer to those very similar the anchor instance. Regarding our x may become excessively positive when selected from the universe set, X is set to be much smaller than the entire universe set, where X |= < 2M For example, we set | = 300 < 296 on the Bankruptcy dataset (Liang et al.) 2016). In practice, CoRTX uses Equation to search the compact positive instance x from X++ where the excessively positive instances may be avoided. In this way, CoRTX can ensure hard positive instances to be as many as possible, which benefits the latent explanation learning process.
3.2 EXPLANATION CONTRASTIVE LOsS
The proposed CoRTX adopts the contrastive learning (He et al. 2020 Chen al. 2020
generate
the latent explanation by training an explanation encoder g(- 0g R Rdd Here, a positive pair
and several negative pairs are exploited during the training g(- (g). A positive pair includes the anchor æ; and compact positive instance ã. A negative pair contains the anchor æ; and a randomly selected instance æ (j * i), where the explanations are mostly different. Let h, = g(æ; fg). h =( 0g) be the latent explanation of the positive pair, and h; h, g(æ, 0g) be the latent explanation for negative pair. CoRTX trains the encoder g(- 0g) by maximizing the dot similarity
II ipschitz continuity be refered to Virmaux & Scaman 2018)
4

---

Published as a conference paper at ICLR 2023
of positive pairs and minimizing the dot similarity of negative pairs. Following the criterion, g(: 0g))
essentially minimizes the contrastive loss as follows:
exp (h: (T)
c = log
(5)
Σラ= exp (h+ h2/TT)
where r is a temperature hyper-parameter (Wu et al. 2018) and N denotes the batch size for training.
To provide the explanation for ʃ(-) from the learned latent explanation in g(- 0g), explanation
head vn(- (n)) : I2 RM further tuned based on the certain explanation tasks. Specifically, 77( 0,,) is tuned with small amount of explanation labels (e.g., the Shapley values), where the computational cost relatively affordable in real-world settings. 17(- 0,)) specifically designed to cope with the different scenarios of model explanation. According to the effects of Equation and Equation 5 CoRTX enforces h, to be similar to h and dissimilar h,, which is consistent with the relationship of their model explanations. This enables the learned representation to contain the explanation information. In this work, CoRTX is studied under two common explanation scenarios, i.e, feature attribution task and feature ranking task. To ensure the explanation quality of CoRTX, we here provide Theorem 2 to show that the explanation error of CoRTX is bounded with theoretical guarantees. The proof of Theorem i provided in Appendix B
Theorem 2 (Explanation Error Bound). Let m[- be Kip-sitchhttz continuous, and Φ
(g((- 0g)
(6,,) be the estimated explanations. If the encoder g(-10g) is well-trained, satisfying [x() - 2 (x) SEERT, then the explanation error on testing instance 2x can be bounded as:
(xx) (<x))l2 & (1+ (2H))l(E - f( + VMTu E +++ K, - - ll2, (6)
where o6 is compact positive instance, h g()x and 7n K K|||0a|12.
Theorem shows that the upper bound of explanation error depends on four different terms, which
are Th, training error E, (f)z) )l. and ||, - h ll.. CoRTX contributes to minimize the
upper bound by explicitly minimizing [f(x) f(z))) and ||h hæs l2 follows:
CoRTX selects the compact positive instances following the selection strategy in Equation which obtains the minimal value of [fx)) [
CoRTX maximizes the similarity between positive pairs while training the encoder g(-10g).
which minimizes the value of |ih he: ll2.
Besides the two terms mentioned above, the error E is naturally minimized after g(- (g)) is trained, and "/2 keeps constant for the given ʃ(..). Thus, we can theoretically guarantee the explanation error bound of CoRTX by Equation 6 This ensures the capability of CoRTX to provide effective explanations to the target model over the testing set.
3.3 ALGORITHM OF CORTX
The outline of CoRTX is given in Algorithm [ CoRTX follows Equation for compact positive selection (lines 4-5), and learns the explanation encoder g(- 0g)) according to Equation 5 (line 6). The training iteration terminates when g(: 0g) converged. Overall, g(- 8g) learns the latent explanation for each instance x = [2], FMM E X, and n7- 0,,) is then fine-tuned with small amount of explanation labels for particular tasks. The algorithmie process of CoRTX can be expressed
as 77 (g(((; 0g) 0g) in general.
Algorithm 1: Real-Time Explainer Training with CoRTX
Input:
Target model f and input feature values æ = [#1,... [MM]
2 Output:
Estimated explanation values of each feature (6), (e.,...,AA..
3
while not convergence do
Generate the synthetic positive instances using x from Equation
Select compact positive instance Ø by Equation 4 and the set of negative instances (æ,
1 i}.
Update hes = g( 0g) with x+ and æ, to minimize loss function given by Equation
end
Fine-tune (he 00,) with small amount explanation labels.
5

---

Published as a conference paper at ICLR 2023
4 EXPERIMENTS
In this section, we conduct experiments to evaluate the performance of CoRTX, aiming to answer the following three research questions: RQI: How does CoRTX perform on explanation tasks in terms of the efficacy and efficiency compared with state-of-the-art baselines? RQ2: Is the latent explanation from CoRTX effective on fine-tuning the explanation head? RQ3: Does synthetic augmentation contribute the explanation performance of CoRTX?
4.1 DATASETS AND BASELINES
Datasets. Our experiments consider two tabular datasets: Census (Dua & Graff 2017) with 13 fea- tures, Bankruptey (Liang al. 2016) with 96 features, and one image dataset: CIFAR-1O (rrizhessky et al. (009) with x pixels. The preprocessing and statisties of three datasets are provided in Ap- pendix C Baseline Methods. In Census and Bankruptey datasets, CoRTX compared with two RTX methods, i.e., Supervised RTX and FastSHAP (Jethani et al. 2021b). as well as two non-amortized explanation methods, i.e., KernelSHAP (KS) (Lundberg & Lee 2017) and Permutation Sampling (PS) (Mitchell et al.] (202I). In CIFAR-10 10 dataset, CoRTX is compared with FastSHAP and other non-amortized methods, including DeePSHAP (Lundberg & Lee 2017). Saliency (Simonyan et al. 2013), Integrated Gradients (IG) (Sundararajan et 2017), SmoothGrad (Smilkov et al. 2017). and GradCAM (Selvaraju et al. 2017). More details about the baselines can be found in Appendix
4.2 EXPERIMENTAL SETTINGS AND EVALUATION METRICS
In this part, we introduce the experimental settings and metrics for evaluating CoRTX. The considered explanation tasks and implementation details are shown as follows.
Feature Attribution Task. This task aims to test the explanation performance on feature attribution. We here implement CoRTX-MSE on fine-tuning the explanation head ();;; 02y)- Given g(ær; Og),
the explanation values are predicted through [Ø1,-.. @Mr] (g((;; 0g) 02,). where Φ; indicates the attribution scores of feature i. Let [Ø1,-. Φarl be the explanation label of an instance I,.
CoRTX-ME -MSE learns 77(- (,,)) by minimizing the mean-square loss LASE = M Σ}= (Ø) - +,) To evaluate the performance, we follow the existing work (Jethani al. 2021b to estimate the e2-error
of each instance on the testing set, where the ez-error is calculated by V V (Ø) - م
Feature Importance Ranking Task. This task aims to evaluate the explanation performance on feature ranking index. We here implement CoRTX-CE for fine-tuning the explanation head n) 0,,). Given g(æ; fg), the feature ranking index is generated by F1. ... F = ngg(2; 0g) (,)) for each instance D4. The predicted feature ranking index [fi. {rr] given by f, = arg maxi, for 1 < j < M. where arg max(-) returns the index with the maximal importance score. Let [r1, rM be the ground-trui ranking label, CoRTX-CE learns 7. 0,y) by minimizing the loss function LCE = E)e CrossEntropy (fy, onehot (r,)). To evaluate the feature importance ranking, we follow existing work (Wang et al. 2022), where the ranking accuracy of each instance is given by
Rank AC 1e,=12) )/( 1. 4).
Evaluation of Efficiency. The algorithmic throughput exploited to evaluate the speed of explaining process (Wang et al.) 2022) Teich & Teich 2018). Specifically, the throughput is calculated by NEa, where Nust and foual denote the testing instance number and the overall time consumption of fuos
explanation derivation, respectively. Higher throughput indicates a higher efficiency of the explanation process. In experiments, twstll measured based on the physical computing infrastructure given in Appendix E.3
Implementation Details. In experiments, CoRTX is implemented under three different prediction models f()). AutoInt (Song et al. 2019) is adopted for Census data, MLP model is used for Bankruptcy data, and R8 is employed for CIFAR-10. labels, we calculate the exact Shapley values for Census dataset. For the other two datasets, we utilize the approximated methods to generate explanation labels (Covert & Lee 2021 Jethani et al., 202 due to the high time complexity of exact Shapley value calculation. The estimated values
for the ground-truth explanation
The manking labels are sorted groud--ruth truth explanation scores g., exact or approximated Shapley values.)

---

Published as a conference paper at ICLR 2023
KATZASHHAP
KeneBHFAP
Thrnglos
LEP
PS Caltric-B AISE
017
CaltFAc-C CE
FistSHAP
10
100 150
0.1
0.3
0.5
0000
0.005
11.000
0.05
0.000
0.0
0.00
0:10
0.30
0.35
(a) Census Income.
(b) Bankruptcy.
(c) Census Income.
(d) Bankruptey.
Figure 3: Explanation throughput versus ranking accuracy on Census Income (a) and Bankruptey
dataset (b). Explanation throughput versus l2-error on Census Income (C) and Bankruptcy dataset (d).
600
0/2
Sepersied CaRTX--IE PLESHAAP
FLetSHAP
Bankk
6:20
CaREX-CB
030
012
000
135
021
Propectin.
Piopurtion
Propeclin
Projection
(a) Census Income.
(b) Census Income.
(c) Bankruptcy.
(d) Bankruptcy.
Figure 4: Explanation performance with different proportions of explanation label usage on two tabular datasets. CoRTX outperforms SOTA RTX baselines by using only 5% of labels.
from Antithetical Permutation Sampling (APS) (Mitchell al.) 2021 Lomeli al. 2019) and
KS are proved to infinitely close to the exact Shapley values (Covert & Lee (021) when they involve sufficient samples. In experiments, employ APS Bankruptcy and KS CIFAR-10 for obtaining explanation labels. More details are introduced in Appendix
4.3
TABULAR EXPERIMENTS
4.3.1
EXPLANATION EFFICACY AND EFFICIENCY (RQ1)
We compare CoRTX with the baseline methods on deriving model explanations. The results in feature attribution and feature ranking are shown by lz-error versus throughput and Rank ACC versus throughput, respectively. The reported results of CoRTX adopts 25% of the explanation labels on fine-tuning the explanation head. The performance comparisons are illustrated in Figures 3 where (a), (b) demonstrate the performance on feature ranking and (c), (d) show the performance on feature attribution. According to the experimental results, we have the following observations:
CoRTX vs. Non-Amortized Methods: Compared with the non-amortized explanation methods with large times of model evaluations, CoRTX achieves competitive explanation performance. In Census data, we observe CoRTX is competitive to PS and KS with respectively 2 and 20 of model evaluations. For KS and PS, it is noted that there is significant decrease in Rank ACC and an obvious increase ee-error as the throughput grows. This indicates that KS and PS both suffer from undesirable trade-off between the explanation speed and performance. In contrast, the proposed CoRTX achieves both efficient and effective explanations for the target model.
CoRTX vs. FastSHAP: CoRTX outperforms FastSHAP on Rank ACC and ey-error under the same level of throughput. This observation shows that CoRTX provides more effective explanation under the scenario of RTX.
CoRTX-MSE vs. CoRTX-CE: CoRTX consistently provides effective solutions for the two explanation tasks. CoRTX- is competitive on the feature attribution and ranking task, while CoRTX- performs even better than CoRTX-MSE in the ranking task. The superiority of CoRTX. CE on ranking task largely results from the loss function in fine-tuning, where appropriate tuning loss can help enhance the explanation quality for certain tasks.
7

---

Published as a conference paper at ICLR 2023
CoRTX
FastSHAP
KernelSHAP
Saliency
IG
Smoothgrad
Gradcam
Deepshap
Truck
Bird
Airplane
Figure Explanations generated on CIFAR-10 Dataset.
4.3.2 CONTRIBUTIONS ON LATENT EXPLANATION (RQ2)
In this experiment, we study the effectiveness of latent explanation in deriving explanation results. Given the obtained latent explanation, Figure demonstrates the explanation performance of the RTX frameworks with different label usage ratios. We compare CoRTX with FastSHAP and Supervised RTX and summarize the key observations below:
Effectiveness of Encoding: CORTX-MSE and CoRTX-CE consistently outperform the Supervised RTX on different label proportions. The results on two explanation tasks show that CoRTX provides effective latent explanation h; in tinettunin the head nh; 0n))
Sparsity of Labels: CoRTX-MSS and CoRTX-CE can provide effective explanation outcomes when explanation labels are limited. The results indicate that CoRTX can be potentially applied to large-scale datasets, where the explanation labels are computationally expensive to obtain.
Effectiveness of Labels: CoRTX and Supervised RTX consistently outperform FastSHAP when having more than 5% explanation labels. FastSHAP synchronously generates the explanation labels with low-samp times during the training. In contrast, CoRTX and Supervised RTX are trained with high-quality labels, which are the exact Shapley values or the approximated ones with high-sampling times. The results indicate that high- high-quality explanation labels can significantly benefit the learning process of the explainer, even when exploiting only 5% explanation labels.
4.3.3 ABLATION STUDIES ON SYNTHETIC POSITIVE AUGMENTATIONS (RQ3)
We further conduct the ablation studies on synthetic positive augmentation in Section 3.1 aiming prove its validity in learning latent explanation. Since negative pairs are randomly selected in CoRTX, here mainly focus the positive aug- mentation in our studies. The ablation studies are conducted under both attribution and ranking tasks Census dataset. In the experiments, CoRTX is compared with two methods: CoRTX w/o Compact Alignment (CA) and CoRTX w/ Max- imum Alignment (MA). To be concrete, CoRTX w/o CA re- places Equation with the existing settings of contrastive learn- ing, which is random masking strategy. CoRTX w/ MA replaces the minimum selecting strategy in Equation 4 with the maximum one synthetic instance set, which is randomly masked from the anchor instance, The remaining settings fol- low the same CoRTX. Figure 6 demonstrates the compar- isons among CoRTX w/o CA, CoRTX w/ MA and CoRTX. We observe that CoRTX outperforms CoRTX w/o CA and CoRTX w/ MA under different proportions of explanation label usage. This demonstrates that the designed synthetic positive augmentation significantly benefits the efficacy of explanation results.
0.0
4.70
4000
Lab
c.m
Proportion
60060
cairi w/ MA
cairz w/t CA Cairx
poo
nam
noxe
Progertion
Figure 6: Ablation Study Posi-
tive Augmentations.
4.4 IMAGE EXPERIMENTS (RQI)
Unlike the tabular cases, image data is typically composed of high-dimension pixel features. To further test the efficacy of latent explanation, evaluate CoRTX CIFAR-10 dataset. We compare CoRTX with FastSHAP and six non-amortized image attribution methods. The explanation results of CoRTX mne- using 5% of ground-truth explanation labels. CoRTX and FastSHAP output
8

---

Published as a conference paper at ICLR 2023
Top-1 Accuracy
Log-odds
Top Accuracs
0.8
Exclusion
Inclusion
Exclusion
Inclusion
CoRTX
0.011
0.774 0.12
0.010
4.86 + 0.028
0.4
FasSHAPP
0.420 + 0.015
0.782 + 0.015
5,010 + 0.012
4,817 + 0.026
KerneSHHAP
0.395
0.012
0.769 + 0.012
5,517 + 0.009
4,829 + 0.024
Saliency
0.735 0.113
5.869 0.010
4.415 +: 0.025
50 Exclusion 100
IG
0,559 = 0.012
0,740 + 0.012
6.159 = 0.011
4,439 + 0.025
Smoothgrad
0,471 = 0.011
0.738 + 0.012
5.679 = 0.008
4,370 = 0.026
Gradeom
0.563 0.012
0.741 0.012
6.109 0.011
4,446 + 0.024
Top 1 Accuracy
0.8
Deepshup
0.554 0.012
0.742 1 0.012
6.124 + 0.009
4,441 + 0.023
0.
Table 1: Exclusion and Inclusion AUCs. The evaluation scores are calculated from the bootstrapping average scores of 20 times repetitions. The explanation methods perform better when ob- taining lower Exclusion AUC and encountering higher Inclusion AUC on both Top-1 Accuracy and Log-odds.
0.4
Sreshared
25
50
75
100
Inclusion %
Figure 7: AUC on CIFAR-10
2x2 super-pixel attributions (Jethani et al. 2021b for explaining the prediction of image classification.
More details are provided in Appendix E
4.4.1 A CASE STUDY OF CORTX
We visualize the explanation results in Figure 5 It shows that CoRTX, FastSHAP, and GradCAM are able to highlight the relevant objects corresponding to the image attribution labels. Specifically, CoRTX provides better numan-understandable explanation results. In contrast, observe that the important regions localized by FastSHAP are larger than relevant objects. Moreover, GradCAM highlights the regions only on tiny relevant objects. This makes FastSHAP and GradCAM less numan-understandabb for the explanation. These observations validate that the explanations from CoRTX are more human-unden numan-undderstaanaaii understandable. More case studies are available in Appendix H
4.4.2 QUANTITATIVE EVALUATION
To quantitatively investigate the explanation results on CIFAR-10, we test CoRTX with the exclusion and inclusion comparing with FastSHAP well as Shapley- and gradient-based non-an amortized methods. Following the experimental settings from existing work (Petsiuk et al., 2018 Jethani al., 20211). we utilize the AUC of Top-1 accuracy and the AUC of Log-odds as the metries for evaluation (Jethani et al. 20216). Specifically, the testing images are orderly masked according to the estimated feature importance scores. Once the important pixels are removed from the input images, Top-1 accuracy and Log-odds are expected to drop drastically, obtaining low exclusion AUC and high inclusion AUC. More details about Log-odds are given Appendix G Figure and Table damoosttde the results of the exclusion and inclusion under 1000 images. We observe that CoRTX outperforms all other baselines in two evaluations, which obtains the lowest exclusion AUC of Top-1 Accuracy and the lowest inclusion AUC of Log-odds. CoRTX is also competitive with the state-of-the-ar baseline, FastSHAP, on the other two evaluations. Besides, CoRTX performs better than non-amortized baseline methods in all four evaluations.
CONCLUSIONS
In this paper, we propose a real-time explanation framework, CoRTX, based on contrastive learning techniques. Specifically, CoRTX contains two components, i.c., explanation encoder and explanation head, which learns the latent explanation and then provides explanation results for certain explanation tasks. CoRTX introduces an explanation-ories explanation. An explanation head can be further fine-tuned with small amount of explanation labels. Theoretical analysis indicates our proposed data augmentation and contrastive scheme can effectively bind the explanation error. The experimental results on three datasets demonstrate that CoRTX outperforms the state-of-the. baselines regarding the explanation efficacy and efficiency. As for future directions, consider extending CoRTX multiple domains obtain the general pre-trained latent explanation for downstream scenarios. It is also promising to explore healthcare applications with CoRTX framework, where the explanation labels require intensive human knowledge.
data augmentation strategy for learning the latent

---

Published as a conference paper at ICLR 2023
6 REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our experiments and benefit the research community, we provide the source code of CoRTX along with the publicly available dataset. The detailed experiment settings. including hyper- rrper-parameters baselines, and datasets, are documented in Appendix to Appendix E Our source code accessible https://github.com http://hwww..oo///cccaaaa/////// 720
REFERENCES
Sanjeev Arora, Hrishikesh Khandeparkar, Mikhail Khodak, Orestis Plevrakis, and Nikunj Saun- shi. A theoretical analysis of contrastive unsupervised representation learning. arXiv preprint arXiv:1902.09229, 2019.
2019.
Neclam Chandolikar, Chaitanya Joshi, Pratcek Roy, Abhiject Gawas, and Mini Vishwakarma. Voice recognition: comprehensive survey, In 2022 International Mobile and Embedded Technology Conference (MECON), pp. 45-51. IEEE, 2022.
Jianbo Chen, Le Song, Martin Wainwright, and Michael Jordan. Learning to explain: An information- theoretic perspective on model interpretation. In International Conference on Machine Learning, pp. 883-892. PMLR, 2018.
Ting Chen, Simon Korbblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. International conference on muchine learning, pp- 1597-1607. PMLR, 2020.
Yu-Neng Chuang, Guanchu Wang, Fan Yang. Zirui Liu, Xuanting Cai, Mengnan Du, and Xia Hu. Efficient xai techniques: A taxonomic survey. arXiv preprint arXiv:2302.03225, 2023.
Ian Covert and Su-In Lee. Improving kemelshap: Practical shapley value estimation using linear regression. In International Conference on Artificial Intelligence and Statistics, pp. 3457-3465. PMLR, 2021.
Ian Covert, Chanwoo Kim, and Su-In Lee. Learning to estimate shapley values with vision trans- formers. arXiv preprint arXiv:2206.0533 05288, 2022.
Piotr Dabkowski and Yarin Gal. Real time image saliency for black box classifiers. Advances in neural information processing systems, 30, 2017.
Amit Dhurandhar, Pin- Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam, and Payel Das. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. Advances in neural information processing systems, 31, 2018
Mengnan Du, Ninghao Liu, and Xia Hu. Techniques for interpretable machine learning, Communica- tions of the ACM, 63(1):68-77 2019.
Dheeru Dua and Casey Graff. UCI machine learning repository. (2007), 2017. URL http:// archive. ics. uci edl/m1]
Luciano Floridi. Establishing the rules for building trustworthy ai. Nature Machine Intelligence, 1(6): 261-262, 2019.
Christopher Frye, Damien de Mijolla, Tom Begley, Laurence Cowton, Megan Stanley, and Ilya Feige. Shapley explainability on the data manifold. arXiv preprint arXiv:2006.01272, 2020.
Jun Gao, Ninghao Liu, Mark Lawley, and Xia Hu. An interpretable classification framework for information extraction from online healtheare forums. Journal of healthcare engineering, 2017, 2017.
Bryce Goodman and Seth Flaxman. European union regulations on algorithmic decision-making and "right explanation". Al magazine, 38(3):50-57, 2017.
Kaiming He, Haoqi Fan, Yuxin Wu. Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and patlern recognition, pp. 9729-9738, 2020.
10

---

Published as a conference paper at ICLR 2023
Neil Jethani, Mukund Sudarshan, Yindalon Aphinyanaphongs, and Rajesh Ranganath. Have learned to explain?: How interpretubility methods can learn to encode predictions in their inter- pretations. In Interaational Conference Artificial Intelligence and Statistics, pp. 1459-1467. PMLR, 2021a.
Neil Jethani, Mukund Sudarshan, Ian Connick Covert, Su-In Lee, and Rajesh Ranganath. Fastshap: Real-time shapley value estimation. In International Conference on Learning Representations, 2021b.
Atsushi Kanehira and Tatsuya Harada. Learning to explain with complemental examples. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp- 8603-8611, 2019.
Andrei Kapishnikov, Tolga Bolukbasi, Fernanda Viégas, and Michael Terry. Xrai: Better attributions through regions. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4948-49511 2019.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in Neural Information Processing Systems, 33: 13:18661-18673 2020.
Been Kim, Rajiv Khanna, and Oluwasanmi O Koyejo. Examples are not enough, learn to criticize! criticism for interpretability. Advances in neural information processing systems, 29, 2016.
Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, et al. Captum: unified and generic model interpretability library for pytorch. arXiv preprint arXiv:2009.07866 2020.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
K Kwei-Herng Lai, Daochen Zha, Junjie Xu, Yue Zhao, Guanchu Wang, and Xia Hu. Revisiting time series outlier detection: Definitions and benchmarks. In Thirty-fifth conference on neural information processing systems dalasets and benchmarks truck (round 1).
Yuening Li, Zhengzhang Chen, Daochen Zha, Kaixiong Zhou, Haifeng Jin, Haifeng Chen, and Xia Hu. Autood: Neural architecture search for outlier detection. In 2021 IEEE 37th International Conference on Data Engineering (ICDE), pp. 2117-2122. IEEE, 2021.
Deron Liang, Chia-Chi Lu, Chih-Fong Tsai, and Guan-An Shih. Financial ratios and corporate governance indicators bankruptey prediction: A comprehensive study. European Journal of Operational Research, 252(2)5561-572 2016.
Yang Liu, Sujay Khandagale, Colin White, and Willie Neiswanger. Synthetic benchmarks for scientific research in explainable machine learning. 2021.
Maria Lomeli, Mark Rowland, Arthur Gretton, and Zoubin Ghahramani. Antithetic and monte carlo kernel estimators for partial rankings. Statistics and Computing. 1975):117711177 2019.
Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Proceed- ings of the 31st international conference neural information processing systems, pp. 4768 4777, 2017.
Shervin Minaee, Amirali Abdolrashidi, Hang Su, Mohammed Bennamoun, and David Zhang, Bio- metrics recognition using deep learning: A survey. Artificial Intelligence Review, pp. 1-49, 2023.
Rory Mitchell, Joshua Cooper, Eibe Frank, and Geoffrey Holmes. Sampling permutations for shapley value estimation. arXiv preprint arXiv:2 104.1119999 2021.
Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.00 2018.

---

Published as a conference paper at ICLR 2023
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should trust you? explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1 1135-1144 2016.
Yao Rong, Tobias Leemann, Vadim Borisov, Gjergji Kasneci, and Enkelejda Kasneci. A consistent and efficient evaluation strategy for attribution methods. In International Conference on Machine Learning, pp. 13770-18793 PMLR, 2022a.
Yao Rong. Tobias Leemann, Thai-trang Nguyen, Lisa Fiedler, Tina Seidel, Gjergji Kasneci, and Enkelejda Kasmeci. Towards human-centered explainable ai: User studies for model explanations. arXiv preprint arXiv:22 1584, 2022b.
Alvin E Roth. Introduction the shapley value. The Shapley value, pp. 1-27, 1988.
Luis M. Ruiz, Federico Valenciano, and Jose M. Zarzuelo. The family of least square values for transferable utility games. Games and Economic Behavior, 24(1):109-130 1998. ISSN 0899-8256.
doi: https::/doi..or//1... wwwp:///.....00000e..0..0.. 1997.0622. 0622. URL https: //ww://wwwww.mo sci encedi rect com/
Patrick Schwab and Walter Karlen. Cxplain: Causal explanations for model interpretation under uncertainty. Advances in Neural Information Processing Systems, 32, 2019.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das. Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 6186626, 2017.
Weichen Shen. Deepetr: Rasy-to--se,,modulrr and extendible package of deep-learning based ctr models. htttps://ggthiii www.:///wwwwwwwwee..eeecccc deepctr 2017.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprini arXiv:13/2.603: 13/2.6034, 2013.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.0000 03825, 2017.
Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. Autoint: Automatic feature interaction learning via self-attentive neural networks. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1161-1170, 2019.
John Stankovic et al. Real-time computing. Byle, pág, pp. 155-162, 1992.
Emily Steel and Julia Angwin. On the web's cutting edge, anonymity in name only. The Wall Street Jourmal, 4, 2010.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International Conference on Machine Learning, pp. 3319-3328. PMLR, 2017.
David A Teich and Paul R Teich. Plaster: A framework for deep learning performance. Technical report, Tech. rep. TIRIAS Research, 2018.
Aaron Van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv e-prints, pp. a8Xiv-1807, 2018.
Aladin Virmaux and Kevin Scaman. Lipschitz regularity of deep neural networks: analysis and efficient estimation. Advances in Neural Information Processing Systems, 31, 2018.
Chuan-Ju Wang, Yu-Neng Chuang, Chih-Ming Chen, and Ming-Feng Tsai. Skewness ranking opti- mization for personalized recommendation. In Conference on Uncertainty in Artificial Intelligence, pp. 400-409 PMLR, 2020.
Guanchu Wang, Yu-Neng Chuang, Mengnan Du, Fan Yang, Quan Zhou, Pushkar Tripathi, Xuanting Cai, and Xia Hu. Accelerating shapley explanation via contributive cooperator selection. arXiv preprint 2206 208.229900. 2022.
12

---

Published as a conference paper at ICLR 2023
Rui Wang, Xiaoqian Wang, and David Inouye. Shapley explanation networks. arXiv preprint arXiv:2004.0229,, 2021.
Eyal Winter. The shapley value. Handbook of game theory with economic applications, 3:2025-2054, 2002.
Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non- parametric instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3733-3742, 2018.
Fan Yang. Ninghao Liu, Suhang Wang, and Xia Hu, Towards interpretation of recommender systems with sorted explanation paths. In 2018 IEEE International Conference Data Mining (ICDM), pp. 667-676. IEFE, 2018.
Daochen Zha, Louis Feng, Bhargav Bhushanam, Dhruv Choudhary, Jade Nie. Yuandong Tian, Jay Chae, Yinbin Ma, Arun Kejariwal, and Xia Hu. Autoshard: Automated embedding table sharding for recommender systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4461 16611447,, 2022.
Rui Zhu, Bingehen Zhao, Jingen Liu, Zhenglong Sun, and Chang Wen Chen. Improving contrastive learning by visualizing feature transformation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10306-10315, 2021.
13

---

Published as a conference paper at ICLR 2023
APPENDIX
A PROOF OF THEOREM
We prove Theorem 1 in this section.
Theorem 1 (Compact Alignment). Let (() be K g-LLppchitzeoonntinnee continuous function and (xx)
( f(ms)] Given perturbed positive instance ã Ex satisfying minigigM (x 0, the explama-
be the feature importance scores, where 0z(x)) == Escu [f
tion difference ({<(() Φ(x)ll2
l is bounded by the prediction difference [f(x) - f()) as
( (x) 2 I (1 22 - f(z)) VM
where 7o K ||a2r|12 and K is the Lipschitz constant of function f(-).
Proof. In order to calculate the importance score (( == = (1) [f)) - f(ās]] of the feature subset S, we recast the formulation under the expression of S= E {0, 10 as follows:
(x) = Es [S(S [1], O a + -1- (..)) S(S o x + (1 S) o x.)] (8)
where S U [1]; denotes not to mask feature i-th from perturbed masking S. Following Equation 8,
now discuss explanation difference each feature. Without lost of generality, we consider the explanation difference of single feature as follows,
[al/Bollliee----S000000111----13 (a))
(+)
-(SS00ic m,
< | -((8) / o x.
1
(B (1))
2M-1
- (1-=))
Following Equation 9 the upper bound of the explanation difference can be derived by
(x) 4,
(x)
Σ
(--
M
+
Ky S oF S0022
M-
S
+
Kyl[S о -5
ox.)
-aa)--((Su)) /(S 08 (1- o ex.)l)
2
+ K |
(10)
$1 ipschitz continuity be refered to Virmaux & Scaman
2018
14

---

Published as a conference paper at ICLR 2023
Note that the difference of prediction scores [f(x) -ā(ā) is equal to the summation of contribution
score among all features (i.e., [f(z) f()) =1E== 6,D) Since we consider the assumption
minigigM D; (z) 2 0, we have Σ 1=1 d,1 = Σn= 16./. In this manner, we have the upper bound
of the explanation difference $ x
( [[2 2 follows:
| ((())
(
+((x)) (
M
(6. + Ky K,|[=|||)
M
M
$
Σ
(6.)² + v2. (Ky(12)12)
Σ 6, +
Σ [
{ll(,, "2, _2r]ll2 + v 1 [f()) - [()) + M
{1[ 82, dmmlli + V270 (1f(x) f())1} + M1o
(1 + - VM
where = Ky|1æ112-
B PROOF OF THEOREM 2
Theorem 2 (Explanation Error Bound). Let 17(110)) be K,p-ipschitzz continuous, and (.)) (g(-l (0))))) be the estimated explanations. If the encoder g(-10g) is well-trained, satisfying
((()) SEER,, then the explanation error on testing instance V& can be bounded as:
{ (ax) (2xx)l2 & (1 (2xx)) - f(=t)) + VMy + E -Knl - - ll2, (11)
where ot is a compact positive instance, h = g()) and 7 K11121112-
Proof. Without loss of generality. consider E norm evaluate the distance between predicted explanation scores and ground truth explanation scores. For all 2₁ E C. we have
|| 2 (xx) (xx))l2 = (x+s) 4 + + ( + क5 (xx)) (xxx)ll
< (x<x) - 4b || ( 4 (क l| + ( ( (xx))l
S (1 v2++) f( + M + + Knllg(Tx)) - g(ax)]l2
(1 + 22 - f(=t) + MM+ + E + Kn h llz
C DETAILS ABOUT DATASETS
The experiments conducted two tabular datasets and one image dataset. The details of the datasets are provided as follows:
Census Income: collection of human social information with 26048 samples for training and validating; and 6513 samples for testing. Each instance has five continuous features and eight categorical features.
Bankruptey: A financial dataset contains 5455 samples of companies in the training set and validating set: and 1364 instances for the testing set. Each sample has 96 features characterizing each company and whether it went bankrupt.
CIFAR-10: An image dataset with 60000 images in 10 different classes, where each image is composed of 32x32 pixels. We follow the original dataset division on training, validating, and testing process.
15

---

Published as a conference paper at ICLR 2023
D DETAILS ABOUT BASELINE METHODS
Some of the baseline algorithmss are implemented under the open-source package and the hyper- parameters are all decided with the optimal model convergence. In our experiments, the RTX frameworks, such as CoRTX and FastSHAP, are typically built on the training set and evaluated on the testing dataset.
Tabular Dataset: We here provide some detailed information about the baselines on tabular dataset. Supervised RTX: A supervised IT'X-based MI.P model trains with raw features of data instances and ground-truth explanation labels from scratch. The layer number in Supervised RTX is set to be six for purpose of fair comparison to CoRTX. FastSHAP: A state-of-the-ar RTX method which adopts an DNN model to universally learn the distribution of approximated Shapley label (Jethani et al. 2021b). KernelSHAP (KS): The model proposes the weighted linear regressions to estimate the Shapley additive explanations from the prediction model scores (Kokhlikyan al.] 2020). Permutation Sampling (PS): PS estimates the feature attribution based on calculating the sensitivity gap of the scores from prediction model while inputting randomly masking data features (Mitchell et al.) 2002)) To achieve the competitive performance on the baselines, we demonstrate the following settings to the KS and PS in the tabular experiments. On Census Income dataset, the numbers of model evaluations on KS is set from 2 to 2 and the numbers of model evaluations on PS is given from 2 to 2 On Bankruptcy dataset, the of model evaluations on KS is set from 2 to 2 and numbers of model evaluations on PS are from 2 to 2 For a fair comparison, CoRTX and Supervised RTX adopt the same loss function while conducting on the same explanation task.
We evaluate the results with two non-amortized Shapley-based estimators: DeepSHAP and KS, where the first work utilizes a linear composition rule estimate the Shapley values on the non-lincar neural networks. Moreover, gradient-based methods are known for their efficiency compared to Shapley-based estimators. We list some gradient-base baselines: Saliency, Integrated Gradients (IG), SmoothGrad, and GradCAM. The mentioned four approaches derive the explanation based on the gradient values of prediction models. Lastly, to evaluate the efficacy of the image explainers, adopt state-of ofate-of-theeart baseline, FastSHAP, to test the explanation generating speed and accuracy. To achieve the better performance on baselines, the evaluation times on baselines basically decided on grid search once achieving competitive performance.
Image Dataset:
E EXPERIMENT DETAILS
We conduct the experiments on tabular dataset and image datasets. For the tabular dataset, we consider two common explanation tasks: feature attribution and importance ranking task. For the image dataset, the explanation results are presented using heatmap in the case study and evaluated by including and excluding the important pixels for perturbed image classification.
E. EXPERIMENTS ON TABULAR DATASET
We verify the feature ranking task by using CoRTX-CE and the feature attribution task by exploiting CORTX-MSE. The experiment on each tabular dataset follows the pipeline of Target Model Training, Explanation Benchmarks and Explainer Implementing. Each step is shown as follows.
Target Model Training: We exploit different prediction models f--) on three datasets to evaluate the model-agnostic property of CoRTX. Autolnt (Song al. 2019) is adopted for the Census Income dataset, and the MLP model for the Bankruptcy dataset. The two prediction models are trained package (Shen 2017). Due
until the convergence, and the implementation is based on the DeepCTR " to the tasks in two datasets, a binary cross entropy loss is given as the loss function of AutoInt and MLP model for the Census Income and Bankruptcy datasets, respectively. All hypre-parameters on target model training are decided by grid search throughout the classification results, including model layers, hidden units, etc.
Explanation Benchmarks: A brute-force algorithm for calculating the exact Shapley value is adopted for the Census Income dataset as the explanation labels. However, the Bankruptcy dataset contains
https://captum..r
https://github.
wwt.://w......iccccccce shenwei chen/ deepctrr
16

---

Published as a conference paper at ICLR 2023
many features (96 features), so it is hard to gain the exact Shapley value for the Bankruptey dataset due to the extremely high computational cost. We hereby adopt the proximity of Shapley values as the explanation labels by using Antithetical Permutation Sampling(APS) (Mitchell et al. [021]] Lomeli et al., 2019) to convergence. This because APS has shown to converge to exact Shapley values when countering high permutation times tabular datasets. In this work, set the model permutation times as 2 x 10 to generate the ground-truth labels.
Explainer Implementation: The explainer is implemented on the learning of latent explanation and fine-tuning. The explanation encoder and the explanation head are designed with the model structures and are learned with the et al. 2021b Dhurandhar et al.) 20188) we adjust the feature importance scores via additive efficient normalization (Ruiz et al. 1998) for the feature attribution task.
in T Following the existing works (Jethani
E.2 EXPERIMENTS ON IMAGE DATASET
Our experiments the image dataset conducted under CORTX-MSE. We focus on the fea- ture attribution task on the image datasets, which follows the pipeline of Target Model Training. Explanation Benchmarks and Explainer Implementing. Each step is shown follows.
Target Model Training: We utilize ResNet- 18 as the prediction models f(-) in CIFAR-10 10 dataset. We train the target model from scratch until the convergence. All apprrpparaettess model training are decided by grid search throughout the classification results and follow the training setting of the target model from (Jethani al. 202
Explanation Benchmarks: We calculate the approximation to Shapley values as the training labels by adopting KS until the convergence (Covert & Lee| 2021 Jethani al., 2021b). The estimated values from KS can infinitely approach exact Shapley values when encountering the optimal conver- gence (Lundberg & Lee 2017). In this work, we set the model permutation times as 2048 to generate the ground-truth labels.
The explanation encoder and the explanation head are designed with the model structures and are learned with the hyper-parameters in Table Following the existing work (Jethani al., 202Tb), we adopt the surrogate model with the same structure as ResNet- 18. The surrogate model is distilled from the original prediction model with masked input images to overcome the effect of out-of-diitribution in masking perturbation (Frye et al.) 2020). Following existing work (Jethani et al.] 20211), we adjust the feature attribution scores through additive efficiintt normalization (Ruiz et al. 1998) for generating the feature importance scores. Moreover, we follow the setting from the existing work (Jethani et al.) (2021b) to conduct the bootstrapping evaluation. The total testing candidates are chosen to be 1000 and we randomly select 666 instances with 20 times repetitions for getting the bootstrap AUCs and Log-odds. Following the previous work in computer vision domain, the outcomes of CoRTX are post post-processed through the moving average for smoothing visualization.
Explainer Implementation:
Dataset
Census Income
Bankrupcy
CIFAR- 10
Target Model
AutoInt
MLP
ResNet-
Explanation Encoder g(-)
3-layer MLP
6-layer MLP
ResNet-18
Optimizer
Adam
Adam
Adam
Explanation
Synthetic Positive Set Batch Size
30
300
300
Representation
1024
1024
1024
Learning
Learning Rate
10-3
x 10-3 3
5 10-3
Temperature /
0.02
0.03
0.02
Binomial Distribution
0.5
0.5
0.5
Reference Value
Mean Value
Mean Value
Zero Value
Explanation Head 17(-)
3-layer MLP
3-layer MLP 10 to 10-5
3- MLP 10 to 10
Fine-tuninig
Weight DecaycoRTx-MSS
10 to 10-6 0
Weight Decay
Table 2: Sxyper-parameters and model structures settings in CoRTX.
17

---

Published as a conference paper at ICLR 2023
E.3 COMPUTATION INFRASTRUCTURE
For a fair comparison of testing algorithmic throughput, the experiments are conducted based on the following physical computing infrastructure in Table
Device Attribute
Value
Computing infrastructure
GPU
GPU model
Nvidia-A40
GPU number
1
GPU Memory
46068 MB
Table 3: Computing infrastructure for the experiments.
F RELATED WORK
Machine learning has been widely applied in variety of domains, such recommender systems (Zha et al. 2022, Wang et al., 2020), anomaly detection (Li et al. 2021 Lai et al.), and voice recogni- tion Chandolikar et al.) 2022. Minaee al., 2023) Despite the advancements in MI., providing transparency in DNN models still remains a challenge, especially in yielding real-time model expla- nations for the deployment on real-time systems. Existing work aims accelerate the derivation of explanations via two different ways: non-amortized methods and real-time explainers.
Non-amortized Method. Existing work of non-amortized methods can be categorized into three groups The first group of methods adopts linear regressions to fit the non- -mortizd explanation, such as LIME (Ribeiro et al., 2016) and KS (Lundberg & Lee, 2017). Another group of methods adopts the preceding difference of the value function for the explanation, such as RISE (Petsiuk et al., 2018), Permutation Sampling (Mitchell et al. 2021) and SHEAR (Wang et al. 2022). The last group estimates the gradient towards the input data for the explanation, such the GradCAM (Selvaraju
cct al. 2017). Integrated Gradient (Sundararajan et al. 2017) and SmoothGrad (Smilkov et all 2017). Even though the amnotizdd method can provide faithful explanation for DNN models, this group of methods suffers from high computational complexity since each data instance requires one non- amortized explainer yield the explanation. Some gradient-based methods, such as (Sundararajan et al. 2017 Smilkov et al.] 2017). are important explanation methods that provide a relatively faster explanation. However, gradient-based methods need extra time to perform the sampling process on each instance while generating the model explanation. The explanation derivation time is still highly dependent on the number of sampling and tested instances, which means that gradient- based methods are insufficient to be the real-time explainer.
Real-time Explainer (RTX). Unlike the amortized methods, the RTX framework maintains a unified explainer to generate the explanation among each data instance. This way, the explanation can be generated via a single feed-forward process of the explainer, which is much faster than the non-amortized methods but sightly degrades the efficacy. 'he learning strategy of the RTX framework formulates the explainer learning by giving strong assumptions on prior feature distribution (Chen ct al. 2018 Dabkowski & Gal 2017; Kanehira & Harada 2019). Existing work (Chen et al. 2018) utilizes the instance-wiss wise feature selection by maintaining a feature masking generator via maximizing the mutual information between selected features and corresponding labels. A well-trained feature masking generator is able to provide real-time explanation under a single feed-forward process. Another framework of RTX to adopt the exact or approximated Shapley values the ground-truth labels to learn the explainers (Wang et al. 2021 Jethani et al. 20216) Covert et al., 2022 Schwab & Karlen 2019). However, the exploitation of exact Shapley values suffers from extremely high computational complexity. To address this problem. FastSHAP (Jethani et al. 2021b) proposes a Conte-Carlo-baseee method to learn the explainer under the RTX framework. Specifically, FaPtSHAP generates the approximated Shapley values by randomly sampling batches of feature masks during the training process. Meanwhile, it updates the explainer to minimize the mean-square error between the overall contribution scores of masked features and outputs from the DNN explainer. FastSHAP enforces the explanation performance without utilizing the ground Shapley value, which can be supported by the experiment results. The choice between depend on the use case (Rong et al.) 2022b] CCuang et al., (023), where there exists a trade-off between efficacy and efficiency non-amortized methods and RTX.
amortizzz methods and RTX may
18

---

Published as a conference paper at ICLR 2023
G MEASUREMENT OF Log-odds
Definition, Log-odds indicates the confidence of the prediction given by the target model f(). Formally, Log-odds is defined the log-likelihood ratio of f(x) (Schwab & Karlen 2019) as follows,
Log-odds log 1 1-p"
(12)
where p is the output probability of the target model f(z)..
Inclusion and Exclusion AUC of Log-odds. To evaluate the estimated feature attribution on image data, we conduct the exclusion and inclusion tests according to the feature importance scores. Once important features are removed from input instances, the target model f(x) expected to drop drastically Log-odds which leads to low exclusion AUC. On the contrary. it is expected to have a high inclusion AUC of Log-odds since the important features are orderly added back to the input instances of the target model. The remaining experimental settings follow FastSHAP (Jethani et al. 2021b).
Inclusion and Exclusion AUC of ALog-odds: Besides exploiting Log-odds measuring the performance, we additionally test CoRTX with ALog-odds on the inclusion and exclusion task. ALog-odds calculates the preceding difference between the Log-odds of original input instances and masked instances, Different from Log- odds and Top-1 Accuracy, Log-odds has the opposite indication where higher exclusion AUC and lower inclusion AUC represent better performance. Table 4 shows the results of the exclusion and inclusion under 1000 images. We observe that CoRTX outperforms other non-amortized baseline methods on both inclusion task and exclusion task. CoRTX performs better than FastSHAP on the inclusion task and is competitive with FastSHAP on the exclusion task.
Loog-odds
Exclusion
Inclusion
CoRTX
2.790 0.060
1.615 + 0.026
FastSHAP
2.896 = 0.045
1.642 = 0.033
KernelSHAP
2.449 = 0.055
1.687 = 0.031
Saliency
2.075 = 0.052
2.055 = 0.036
IG
1.818 = 0.053
2.040 = 0.039
Smoothgrad
2.287 = 0.054
2.111 = 0.036
Gradcam
1.824 + 0.049
2.062 + 0.037
Deepshap
1.870 0.054
2.040 + 0.035
Table 4: Exclusion and Inclusion AUCs on Log-odds.
19

---

Published as a conference paper at ICLR 2023
H ADDITIONAL RESULTS ON IMAGE DATASET
We demonstrate more explanation results on CIFAR-10 generated by CoRTX compared to other baselines. The results show that CoRTX can identify more human-understandable explanations toward other baselines.
CORTX
FastSHAP
KerneiSHAP
Saliency
IG
Smoothgrad
Gradcam
Deepshap
Airplane
Truck
Dog
Horse
Truck
Ship
Dog
Frog
Horse
Airplaine
Deer
Truck
Figure 8: Explanations generated on CIFAR-10 Dataset.
20

---

Published as a conference paper at ICLR 2023
1 ADDITIONAL BASELINE EXPERIMENTS
We conduct more experiments on CIFAR-10 dataset to compare CoRTX with two more baseline methods: RISE (Petsiuk al., 2018 and XRAI (Kapishnikov et al. 2019). All hyper-parameters are decided with the optimal model convergence. The results show that CoRTX can provide more human-understandable explanations than RISE and XRAI.
CoRTX
RISE
XRAI
Truck
Bird
Airplane
Figure 9: Comparison with the generated Explanation on CIFAR-10 Dataset.
We further conduct the experiments to compare 1.2X (Chen et al. 2018) with CoRTX on the Adult dataset. The hyper-parameters of 1.2X and CoRTX are all decided with the optimal model convergence. In the experiment, we choose Rank ACC and Faithfulness (Liu et al.) (02T) as the metric, as the outputs from L2X are not Shapley- tasks without any ground truth. As shown in Table [5] CoRTX outperforms L2X on two metrics, indicating the better capability for deriving model explanation.
values. Faithfulness the metric evaluate explanation
Methods 1.2X (Chen et al. 2018)
Rank ACC
Faithfulness
0.0847 + 0.0011
0.542
CORTX-MSE
0.6368 + 0.0067
0.835
CoRTX-CE
0.7680 + 0.0031
Table 5: Comparison with I.2X Adult dataset.
21

---

Published as a conference paper at ICLR 2023
J EVALUATION OF CORTX ON SURROGATE MODELS
Besides the quantitative evaluation setting in Section 4.4.2 that follows (Rong et al. 2022a). We further conduct experiments by retraining the surrogate model, where the experiment settings follow from (Jethani et al., 2021b a). The following table shows the results of the exclusion and inclusion AUC under 1000 images. We observe that CoRTX outperforms other amortized baselines on both inclusion and exclusion tasks. CoRTX performs better than FastSHAP (a SOTA of supervised RTX method) on the exclusion task and competitive with FastSHAP on the inclusion task. The results reveal similar pattern as using the original prediction model in the quantitative experiments, which proves that CoRTX has the capability to derive effective model explanations on either of the experiment settings.
Top-1 Accuracy
Exclusion
Inclusion
CoRTX
0.372 + 0.012
0.772 + 0.012
FastSHAP
0.386 = 0.012
0.767 #707=00013 0.013
KernelSHAP
0.406 + 0.011
0.767 0.012
Saliency
0.543 0.012
0.703
IG
0.581 = 0.012
0.704
Smoothgrad
0.480 0.012
0.695 0.012
Gradcam
0.573 + 0.012
0.719 + 0.013
Deepshap
0.573 0.012
0.706 + 0.012
Table 6: Evaluation of CoRTX on Surrogate Models.
K. 12-ERROR OF PS/KS WITH HIGH PERMUTATION
We conducted additional experiments on the Adult dataset to show that PS/KS with high permutation is very similar to the exact values of Shapley values, especially when the perturbation is above 4000 times. This shows that high perturbated KS/PS is sufficient to generate the ground truth. There is no need to do higher permutation sampling for KS/PS to generate the ground truth. This is because the error remains extremely small as well, but execution time grows drastically.
Number of permutation KernelSHAP (L2-error)
2000
4000
6000
8000
0.0038
0.0027
0.0021
0.0018
APS (e2-error)
0.00031
0.00025
0.00020
0.00017
KernelSHAP (sec/instance)
0.3351
0.6816
0.9869
1.3089
APS (sec/instance)
0.0621
0.1
0.1705
0.2222
Table 7: Eg-error of PS/KS with High Permutation.
L ADDITIONAL NOTATION CLARIFICATION
In this section, Table & shows the detailed information of the notations in Section 4.2 for clarification.
Notation
Definition & Description
Dimension
Predicted ranking index score list of feature æ; from CoRTX.
f
Each index score in iy represents the score of feature z; that is ranked at this index.
f, R
Predicted ranking index from CoRTX,
f,
fy E R
where i, arg max it for 1 ISj K The ground-truth ranking label sorted from Shapley values.
r, E RM
Table 8: Notations Section 4.2.
22

---

Published as a conference paper at ICLR 2023
M 22-ERROR AND EXECUTION TIME OF PS/KS WITH HIGH PERMUTATION
We conduct an experiment on the Adult dataset and indicate the explanation performance and the execution time of PS/KS and CoRTX the following table. It is observed that there is trade-off between the speed and performance of KS/PS. A fair comparison should keep the execution time as close as possible. According to the results, CoRTX generates the model explanation in around 0.00015 seconds per instance, while nigh-permutated PS/KS requires at least 2000x longer execution time. We believe that comparing CoRTX with low-permutated KS/PS fairer than with nigh-permutated KS/PS. All experimental settings follow FastSHAP (Jethani et al. 2021b).
KS-300
KS-2000
KS-4000
KS-6000
KS-8000
CoRTX-MSE
R2-error
0.0189
0.0038
0.0027
0.0021
0.0018
0.007
Time (sec)
0.1477
0.3351
0.6816
0.9869
1.3089
0.00015
APS-300
APS-2000
APS-4000
APS-6000
APS-8000
CoRTX-MSE
22-error Time (sec)
0.0171
0.00031
0.00025
0.00020
0.00017
0.007
0.0380
0.0621
0.1164
0.1705
0.2222
0.00015
Table 9: error and execution time of PS/KS with high permutation.
23


Reviews:
[{"review_id": "qHiPzOM13ZM", "reviewer": "Reviewer_A6qm", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors proposed a method to obtain a good and fast explanation at the testing time. In particular, the authors focused on the approach that learns an explainer model to mimic the “accurate explanation” at test time (e.g., predict the Shapley value).\nThe authors pointed out that a major issue of existing methods is the requirement of many ground-truth explanation labels (i.e., fully accurate Shapley value for each sample), which can be computationally expensive.\nThe authors proposed a contrastive learning method to learn useful explanation-oriented representations so that they can learn the explainer effectively with only a few explanation labels. The main contribution is the synthetic strategy to select positive and negative instances.\nThe authors conducted various experiments to verify the efficiency of the proposed method.", "summary_of_the_review": "The authors focused on an important problem of acquiring fast explanations at test time, although the scope is limited to Shapley-liked explanations. The proposed method is yet simple but effective and well backed up theoretically and experimentally. The weaknesses seem fixable, and under the condition that these points improve, I will recommend acceptance.", "strength_and_weaknesses": "Strength\nThe paper focused on an important problem.\nThe synthetic strategy to select positive instances for contrastive learning is reasonable and well backed up, theoretically and experimentally.\nThe experiments are well established to show the effectiveness of the proposed method.\n\nWeakness\nThe related works are not sufficient compared to the scope of the title and abstract. In particular, the authors mainly focused on such Shapley-value and aimed to predict these values in test time. However, many other approaches (e.g., gradient-based saliency methods) can also provide fast real-time explanations. Some discussions are needed.\nI would prefer more intuition/explanation of how contrastive learning, especially the instances selection strategy, is useful in learning effective representation for the explainer.\nIn the abstract, the authors claimed that “accurate explanation labels are hard to obtain due to limited human efforts.” Is this true? Since the explanation is of the black-box model, I do not see how humans take part in producing the explanation labels.\nIn Figure 5, it would be useful to include the result of SHAP so that the readers can evaluate how the proposed method’s explanation compares to the ground-truth explanation.\nIn the Feature Importance Ranking Task (Section 4.2), the equation to obtain hat{r}j is not clear (although I can guess what the authors are going to do).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "M_L0KNK8bCl", "reviewer": "Reviewer_ZCbd", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes Constrastive Real-time Explanation (CoRTX) framework to avoid encountering efficiency issues in practical scenarios. Specifically, to avoid the requirement of large amounts of explanation labels, this paper designs a synthetic strategy to select positive and negative samples for contrastive explanation representation learning. With experiments on three datasets, and theoretical analysis on both the positive sample selection method and contrastive explanation error bound, the proposed method CoRTX demonstrates efficiency and efficacy.", "summary_of_the_review": "Overall, I think the studied problem is essential, and the proposed method is novel and reasonable. The motivations are good, and the experiments are clear, comprehensive, and convincing. The theoretical analysis also helps much in understanding how contrastive learning help in explanation. Though the claims in motivations are not entirely convincing, I believe this is a good paper.\n", "strength_and_weaknesses": "- This paper is well-written, and the structure is clear. The authors extend from the problem in explainable machine learning of the huge requirement of labels to why unsupervised contrastive learning can help explanation with clear theoretical analysis.\n- Settings and evaluation metrics in experiments are good. The formulated three questions do help understand the motivation and effectiveness of this method, including the efficiency and efficacy of CoRTX, and whether the extracted contrastive explanation representation is still effective when fine-tuning, also the ablations on synthetic positive augmentations are good.\n- The generality of the proposed approach is well demonstrated by considering experimental scenarios with various modalities, i.e., experiments on images and tabular data.\n\nCons:\n\n- With the mention of \"real-time\" and \"in practice\" in the Intro, this paper focus on the real-time scenario, which will easily remind the readers of industrial-level real-time explanations, e.g., plug-and-play model or tool. However, if one involves a training phase, so-called \"real-time\" is not achievable, even though only the explanation head is trained in CoRTX. Although the proposed components of light-label manner and synthetic positive augmentation do reduce the computational effort and prevent the instances from being too hard in contrastive learning, it is still not the same as what I perceive as real-time for real-world scenarios. Besides, the evaluated datasets are not \"in practice\". To me, this is a good paper on explainable machine learning, but not a paper on \"real-time\". I would recommend that the authors change this paper's stance and contribution in another way rather than \"real-time\".\n- Sections 2.2 and 2.3 depict the motivation of the whole method, i.e., combining the advantages of two conventional approaches and reducing the dependency on labels. Although the motivation is intuitive, the content is not convincing enough. I would recommend the authors could write in a better way to couple unsupervised contrastive learning with explainable ML. For example, labels may bring strong supervised information into the training phase, affecting the inductive bias of the model, making the model favor more readily available label information over the information that may help in explanation, where the contrastive learning has already proved that models can learn features that are beneficial for downstream tasks without relying on labels.\n- The experiments and models are well designed, but there may be a lot of room for tuning in terms of parameters and design, such as the choice of the explanation head. The choice of the number of fully connected layers may have an impact, as also the dimensions of these layers, e.g., multiple projection heads of SimCLR.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "LYjc_Poefb", "reviewer": "Reviewer_MLHw", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a contrastive real-time explanation framework CoRTX, it is a framework based on contrastive learning to learn model explanations using contrastive learning and fine-tuning a corresponding explanation head using a small amount of ground truth explanation labels. Here the paper used Shapley value as the ground truth.\nThe contrastive framework consists of the following given:\n(a) Train an explanation encoder to produce an explanation representation \n - Given a sample $x$, a set of synthetic positive instances  $X$ is generated via perturbations on x given by equation 2. \n - From set $X$ select $\\tilde{x}^+$ that minimizes equation 4. So $\\tilde{x}^+$ is the synthetic sample that gives the closet function output to $x$ when given to the target model.\n-  Select a set of samples from the training dataset negative samples (any samples other than the original one x).\n- Train the encoder by minimizing the typical contrastive loss in equation 5.\n(b) When the encoder is trained, an explanation head is added which is a model that takes as an input the output of the encoder and produces the Shapley values as an output. This entire architecture is then finetuned with Shapley value as ground truth labels.\n\nTwo versions of CoRTX were introduced CoRTX-MSE where the model was trained to minimize the L2 distance and CoRTX-CE where the model is trained to minimize the cross entropy of the ranking.\n\nThe paper performed experiments on two tabular datasets Census and Bankruptcy, and one image dataset CIFAR-10. For ground truth explanations the paper used Shapley values for Census dataset, Antithetical Permutation Sampling for Bankruptcy and kernalShap for  CIFAR-10 as an approximation of Shapley values.", "summary_of_the_review": "Overall, the idea of estimating Shapley values is important.\nMy main concerns in the paper are as follows:\n\n- Why is representation produced by the encoder an \"explanation representation\"?\n- Using an approximation of Shapley values as ground truth if Shapley values are too expensive to compute that even practically computing 5% of a large training dataset like imagenet is unfeasible then the overall idea of this paper is not valid but using an estimation of Shapley values to estimate the Shapley values seems incorrect.\n- The paper mainly compares this idea of supervised RTX which is a model trained on all Shapley values and produces an explanation, however, this model does not really exist (please correct me if I am wrong and cite the paper accordingly) it is a made-up baseline introduced by the paper. Because as mentioned in the paper the ground-truth explanations are too expensive to generate in general.\n- Some of the experimental section are unclear please see the weakness section.", "strength_and_weaknesses": "Strength:\n--\n- The paper is well-written and easy to follow.\n- The need for a fast reliable explanation method is crucial.\n\nWeakness:\n--\n- I am not convinced this framework is telling us anything about how a model is making a prediction, the only time the model in question is actually used to find the positive samples in the contrastive loss. Why are the produced representations \"Explanation representation\"  unless I am missing something the produced representations are simply just different representations. And until they are fine-tuned with ground truth explanations they have nothing to do with model explanations.  The FASTShap paper was sold as a faster way to calculate an estimate of the Shapley values of a model. This might be the case here but claiming that the representation produced by the encoder is an explanation representation is not well supported.\n- I strongly disagree with the notation of the ground truth explanations, especially when using an approximation of Shapley values as ground truth as done in two of the datasets in the experimental section.\n- The \"Supervised RTX\" or \"SOTA RTX\" as explained in the appendix as \"A supervised RTX-based MLP model trains with raw features of data instances and ground-truth explanation labels from scratch\" Where was this introduced? Which paper I could not find a reference for?\n- Why was there inconsistency when choosing ground truth I understand that Shapley values are expensive to calculate but why APS on one dataset and KS on another?\n\n- Experiments:\n    -  For \"EXPLANATION EFFICACY AND EFFICIENCY\": I find it very strange to be comparing with methods (PS/KS) that the paper has considered as ground-truth explanations. Is the rank accuracy and error reported against the actual Shapley values?\n    - For \"CONTRIBUTIONS ON EXPLANATION REPRESENTATION\" The paper is mainly comparing with Supervised RTX which again I am not sure which paper proposes this method (if this method has not been previously proposed by another paper then basically, this is a made-up baseline, and not realistic).\n    -  For \"QUANTITATIVE EVALUATION\" how is the masking and accuracy drop measured do you check the drop on the existing method or create an evaluation model similar to (Jethani et al., 2021)?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "HMRTFRvQiS", "reviewer": "Reviewer_bBUe", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a new real-time explainer framework called COntrastive Real-Time eXplanation (CoRTX).\nThe main goal of the algorithm is to limit the dependence of real-time explainers on predefined labels and to improve the throughput (number of samples/time) of the algorithm.\nCoRTX designs a positive and negative synthetic sampling strategy to learn an explanation representation in the form of feature importance.\nA positive sample is a masking of the original data to explain where some features are replaced by zero values.\nThis explanation representation is shown to be an approximation of Shapley values with a set of theorems.\nThe architecture of the method is very similar to that of an autoencoder, but with only the encoder and a loss of similarity.\nLearning is performed encouraging representations of positive samples to be close, while those of negative samples to be far away.\nAfter learning the representations, a fine-tuning process with a limited amount of ground truth labels is carried out to refine the explanations.", "summary_of_the_review": "The paper is not very clear and needs a thorough rewrite; the experiments part needs further experiments to fully support the claims.", "strength_and_weaknesses": "Strengths:\n+ The results are very good and CoRTX outperforms the tested methods. The evaluations were performed correctly, including the evaluation of errors.\n+ Quantitative evaluation of the feature importance map is well conducted according to current standards.\n+ Good number of data sets tested to validate the claims.\n\n- The fine tuning part is done according to the tested metric. This is done only for CoRTX and is not fair to compare against other methods tested.\n- Two types of approaches were described in the introduction: methods that learn an explainer based on labels and supervised methods. CoRTX is a semi-supervised method, but only the first type of methods is tested. I would like to see a comparison with methods such as Dabkowski & Gal 2017, Chen et al. 2018 and Kanehira & Harada 2019, presented in the introduction and never mentioned again. CoRTX uses a similar framework as Chen et al. but the comparison was not made.\n- The saliency maps produced (Figure 5) are very different from those compared. I suggest a comparison with masking methods such as RISE (https://es.sonicurlprotection-fra.com/click?PV=2&MSGID=202210211531580778554&URLID=35&ESV=10.0.18.7423&IV=C019EBF33BCE13D9800D210395403F1C&TT=1666366319343&ESN=UynqeUYHEpb%2Ba0WBCa0%2FCeGBiM%2Fl9mITsoFXluKYkDY%3D&KV=1536961729280&B64_ENCODED_URL=aHR0cHM6Ly9naXRodWIuY29tL2VjbGlxdWUvUklTRQ&HK=7F3046409B02770A56F86E7E471588356C1EC6136067365613FD327EB358B71D) or XRAI (https://es.sonicurlprotection-fra.com/click?PV=2&MSGID=202210211531580778554&URLID=34&ESV=10.0.18.7423&IV=CC9762D41E2E67C8D730D7F2EE59A2AE&TT=1666366319343&ESN=kNkzfYKleQHNaDAOrggEOdhqRAXo1UQxQGUwd7mAT7g%3D&KV=1536961729280&B64_ENCODED_URL=aHR0cHM6Ly9naXRodWIuY29tL2h1bW1hdC9zYWxpZW5jeQ&HK=225CD5F9DF0A3A0AAC78077D46BE0581FB3486DED2BEC28FDA05130432137D5B) that produce saliency maps very similar to those of CoRTX.\n- The qualitative assessment of saliency maps has not been adequately evaluated. In particular, Section 4.4.1 states that \"CoRTX provides better explanation results and highlights particular regions that are essential for model predictions.\" However, shifting salience toward the subject is not always a symptom of better explanations. There is no certainty that the subject is important for prediction. In particular, there is an article entitled \"Sanity check for saliency maps\" by Adebayo et al. that shows this detrimental behavior of methods such as Smoothgrad (tested in this article).\n- The fine-tuning part appears to be optional from the text, but instead is required to produce a saliency map of size M. The representations output by the encoder have size d and cannot be used as saliency maps because d << M.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster