Original: C O RTX: C ONTRASTIVE F RAMEWORK FOR R EAL - TIME
Revised: C O RTX: C ONTRASTIVE F RAMEWORK FOR R EAL - TIME

Original: E XPLANATION
Revised: E XPLANATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Recent advancements in explainable machine learning provide effective and faithful solutions for interpreting model behaviors.
Revised: Recent advancements in explainable machine learning provide effective and faithful solutions for interpreting model behaviors.

Original: However, many explanation methods encounter efficiency issues, which largely limit their deployments in practical scenarios.
Revised: However, many explanation methods encounter efficiency issues, which largely limit their deployments in practical scenarios.

Original: Real-time explainer (RTX) frameworks have thus been proposed to accelerate the model explanation process by learning an one-feed-forward explainer.
Revised: Real-time explainer (RTX) frameworks have thus been proposed to accelerate the model explanation process by learning an one-feed-forward explainer.

Original: Existing RTX frameworks typically build the explainer under the supervised learning paradigm, which requires large amounts of explanation labels as the ground truth.
Revised: Existing RTX frameworks typically build the explainer under the supervised learning paradigm, which requires large amounts of explanation labels as the ground truth.

Original: Considering that accurate explanation labels are usually hard to obtain, due to constrained computational resources and limited human efforts, effective explainer training is still challenging in practice.
Revised: Considering that accurate explanation labels are usually hard to obtain, due to constrained computational resources and limited human efforts, effective explainer training is still challenging in practice.

Original: In this work, we propose a COntrastive RealTime eXplanation (CoRTX) framework to learn the explanation-oriented representation and relieve the intensive dependence of explainer training on explanation labels.
Revised: In this work, we propose a COntrastive Real-Time eXplanation (CoRTX) framework to learn the explanation-oriented representation and relieve the intensive dependence of explainer training on explanation labels.

Original: Specifically, we design a synthetic strategy to select positive and negative instances for explanation representation learning.
Revised: Specifically, we design a synthetic strategy to select positive and negative instances for the learning of explanation.

Original: Theoretical analysis show that our selection strategy can benefit the contrastive learning process on explanation tasks.
Revised: Theoretical analysis show that our selection strategy can benefit the contrastive learning process on explanation tasks.

Original: Experimental results on three real-world datasets further demonstrate the efficiency and efficacy of our proposed CoRTX framework.
Revised: Experimental results on three real-world datasets further demonstrate the efficiency and efficacy of our proposed CoRTX framework.

Original: Our source code is available at: https://anonymous.4open.science/r/CoRTX-720-BDE
Revised: Our source code is available at: https://anonymous.4open.science/r/CoRTX-720-BDE

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: The remarkable progress in explainable machine learning (ML) significantly improves the model transparency to human beings (Du et al., 2019).
Revised: The remarkable progress in explainable machine learning (ML) significantly improves the model transparency to human beings (Du et al., 2019).

Original: However, applying explainable ML techniques to real-time scenarios remains to be a challenging task.
Revised: However, applying explainable ML techniques to real-time scenarios remains to be a challenging task.

Original: Real-time systems typically require model explanation to be not only effective but also efficient (Stankovic et al., 1992).
Revised: Real-time systems typically require model explanation to be not only effective but also efficient (Stankovic et al., 1992).

Original: Due to the requirements from both stakeholders and social regulations (Goodman & Flaxman, 2017; Floridi, 2019), the efficient model explanation is necessary for the real-time ML systems, such as the controlling systems (Steel & Angwin, 2010), online recommender systems (Yang et al., 2018), and healthcare monitoring systems (Gao et al., 2017).
Revised: Due to the requirements from both stakeholders and social regulations (Goodman & Flaxman, 2017; Floridi, 2019), the efficient model explanation is necessary for the real-time ML systems, such as the controlling systems (Steel & Angwin, 2010), online recommender systems (Yang et al., 2018), and healthcare monitoring systems (Gao et al., 2017).

Original: Nevertheless, existing work on non-amortized explanation methods have high explanation latency, including LIME (Ribeiro et al., 2016), KernelSHAP (Lundberg & Lee, 2017).
Revised: Nevertheless, existing work on non-amortized explanation methods have high explanation latency, including LIME (Ribeiro et al., 2016), KernelSHAP (Lundberg & Lee, 2017).

Original: These methods rely on either multiple perturbations or backpropagation in deep neural networks (DNN) for deriving explanation (Covert & Lee, 2021; Liu et al., 2021), which is time-consuming and limited for deployment in real-time scenarios.
Revised: These methods rely on either multiple perturbations or backpropagation in deep neural networks (DNN) for deriving explanation (Covert & Lee, 2021; Liu et al., 2021), which is time-consuming and limited for deployment in real-time scenarios.

Original: Real-time explainer (RTX) frameworks have thus been proposed to address such efficiency issues and provide effective explanations for real-time systems (Dabkowski & Gal, 2017; Jethani et al., 2021).
Revised: Real-time explainer (RTX) frameworks have thus been proposed to address such efficiency issues and provide effective explanations for real-time systems (Dabkowski & Gal, 2017; Jethani et al., 2021).

Original: Specifically, RTX learns an overall explainer on the training set by using the ground-truth explanation labels obtained through either exact calculation or approximation.
Revised: Specifically, RTX learns an overall explainer on the training set by using the ground-truth explanation labels obtained through either exact calculation or approximation.

Original: RTX provides the explanation for each local instance via a single feed-forward process.
Revised: RTX provides the explanation for each local instance via a single feed-forward process.

Original: Existing efforts on RTX can be categorized into two lines of work.
Revised: Existing efforts on RTX can be categorized into two lines of work.

Original: The first line (Jethani et al., 2021; Covert et al., 2022) explicitly learns an explainer to minimize the estimation error regarding to the approximated explanation labels.
Revised: The first line (Jethani et al., 2021; Covert et al., 2022) explicitly learns an explainer to minimize the estimation error regarding to the approximated explanation labels.

Original: The second line (Dabkowski & Gal, 2017; Chen et al., 2018; Kanehira & Harada, 2019) trains a feature mask generator subject to the certain constraints on pre-defined label distribution.
Revised: The second line (Dabkowski & Gal, 2017; Chen et al., 2018; Kanehira & Harada, 2019) trains a feature mask generator subject to the certain constraints on pre-defined label distribution.

Original: Despite the effectiveness of existing RTX frameworks, recent advancements still rely on the large amounts of explanation labels under the supervised learning paradigm.
Revised: Despite the effectiveness of existing RTX frameworks, recent advancements still rely on the large amounts of explanation labels under the supervised learning paradigm.

Original: The computational cost of obtaining explanation labels is extremely high (Roth, 1988; Winter, 2002), which thereby limits the RTX’s deployment in real-world scenarios.
Revised: The computational cost of obtaining explanation labels is extremely high (Roth, 1988; Winter, 2002), which thereby limits the RTX’s deployment in real-world scenarios.

Original: To tackle the aforementioned challenges, we propose a COntrastive Real-Time eXplanation (CoRTX) framework based on the contrastive learning techniques.
Revised: To tackle the aforementioned challenges, we propose a COntrastive Real-Time eXplanation (CoRTX) framework based on the contrastive learning techniques.

Original: CoRTX aims to learn the  explanationrepresentation of each data instance without any ground-truth explanation label.
Revised: CoRTX aims to learn the latent explanation of each data instance without any ground-truth explanation label.

Revised: The latent explanation of an instance is defined as a vector encoded with explanation information.

Original: Contrastive learning has been widely exploited for improving the learning processes of downstream tasks by providing well-pretrained representative embeddings (Arora et al., 2019;
Revised: Contrastive learning has been widely exploited for improving the learning processes of downstream tasks by providing well-pretrained representative embeddings (Arora et al., 2019;

Original: He et al., 2020).
Revised: He et al., 2020).

Original: In particular, taskoriented selection strategies of positive and negative pairs (Chen et al., 2020; Khosla et al., 2020) can shape the representation properties through contrastive learning.
Revised: In particular, task-oriented selection strategies of positive and negative pairs (Chen et al., 2020; Khosla et al., 2020) can shape the representation properties through contrastive learning.

Original: Motivated by the such contrastive scheme, CoRTX develops an explanation-oriented contrastive framework to learn the explanation representation, with the goal of further fine-tuning an explanation head in the downstream tasks.
Revised: Motivated by the such contrastive scheme, CoRTX develops an explanation-oriented contrastive framework to learn the latent explanation, with the goal of further fine-tuning an explanation head in the downstream tasks.

Original: CoRTX learns the  explanationrepresentation to deal with the explanation tasks by minimizing the contrastive loss (Van den Oord et al., 2018).
Revised: CoRTX learns the latent explanation to deal with the explanation tasks by minimizing the contrastive loss (Van den Oord et al., 2018).

Original: Specifically, CoRTX designs a synthetic positive and negative sampling strategy to learn the explanation representation.
Revised: Specifically, CoRTX designs a synthetic positive and negative sampling strategy to learn the latent explanation.

Original: The obtained  explanationrepresentation can then be transformed to feature attribution or ranking by fine-tuning a corresponding explanation head using a tiny amount of explanation labels.
Revised: The obtained latent explanation can then be transformed to feature attribution or ranking by fine-tuning a corresponding explanation head using a tiny amount of explanation labels.

Original: Theoretical analysis and experimental results demonstrate that CoRTX can successfully provide effective  explanationrepresentation for feature attribution and ranking tasks.
Revised: Theoretical analysis and experimental results demonstrate that CoRTX can successfully provide effective latent explanation for feature attribution and ranking tasks.

Original: Our contributions can be summarized as follows:
Revised: Our contributions can be summarized as follows:

Original: • CoRTX provides a contrastive framework for deriving explanation representation, which can effectively reduce the required amounts of explanation label; • Theoretical analysis indicate that CoRTX can effectively learn the  explanationrepresentation over the training set and strictly bound the explanation error; • Experimental results demonstrate that CoRTX can efficiently provide explanations to the target model, which is applicable to both tabular and image data.
Revised: • CoRTX provides a contrastive framework for deriving latent explanation, which can effectively reduce the required amounts of explanation label; • Theoretical analysis indicate that CoRTX can effectively learn the latent explanation over the training set and strictly bound the explanation error; • Experimental results demonstrate that CoRTX can efficiently provide explanations to the target model, which is applicable to both tabular and image data.

Original:  P RELIMINARY 2. N OTATIONS
Revised:  P RELIMINARY 2. N OTATIONS

Original: We consider an arbitrary target model f ( · ) to interpret.
Revised: We consider an arbitrary target model f ( · ) to interpret.

Original: Let input feature be x = [ x 1 , · · · , x M ] ∈ X , where x i denote the value of feature i for 1 ≤ i ≤ M .
Revised: Let input feature be x = [ x 1 , · · · , x M ] ∈ X , where x i denote the value of feature i for 1 ≤ i ≤ M .

Original: The contribution of each feature to the model output can be treated as a cooperative game on the feature set X .
Revised: The contribution of each feature to the model output can be treated as a cooperative game on the feature set X .

Original: Specifically, the preceding difference f ( (cid:101) x S∪{ i } ) − f ( (cid:101) x S ) indicates the contribution of feature
Revised: Specifically, the preceding difference f ( (cid:101) x S∪{ i } ) − f ( (cid:101) x S ) indicates the contribution of feature

Original: i under feature subset S ⊆
Revised: i under feature subset S ⊆

Original: U \ { i } , where U is the entire feature space.
Revised: U \ { i } , where U is the entire feature space.

Original: The overall contribution of feature i is formalized as the average preceding difference considering all possible feature subsets S , which can be formally given by
Revised: The overall contribution of feature i is formalized as the average preceding difference considering all possible feature subsets S , which can be formally given by

Original: [Equation]
Revised: [Equation]

Original: where (cid:101) x S = S ⊙ x
Revised: where (cid:101) x S = S ⊙ x

Original: +( 1 − S ) ⊙ x r denotes the perturbed sample, S = 1 S ∈ { 0 , 1 } M is a masking vector of S , and x r =
Revised: +( 1 − S ) ⊙ x r denotes the perturbed sample, S = 1 S ∈ { 0 , 1 } M is a masking vector of S , and x r =

Original: E [ x | x ∼ P ( x )] denotes the reference values 1 from feature distribution P ( x ) .
Revised: E [ x | x ∼ P ( x )] denotes the reference values 1 from feature distribution P ( x ) .

Original: The computational complexity of Equation 1 grows exponentially with the feature number M , which cumbers its application to real-time scenarios.
Revised: The computational complexity of Equation 1 grows exponentially with the feature number M , which cumbers its application to real-time scenarios.

Original: To this end, we propose an efficient explanation framework for real-time scenarios in this work.
Revised: To this end, we propose an efficient explanation framework for real-time scenarios in this work.

Original: 2. R EAL - TIME E XPLAINER
Revised: 2. R EAL - TIME E XPLAINER

Original: Different from the existing non-amortized explanation methods (Lundberg & Lee, 2017; Lomeli et al., 2019) that utilize local surrogate models for explaining data instances, RTX trains a global model to provide fast explanation via one feed-forward process.
Revised: Different from the existing non-amortized explanation methods (Lundberg & Lee, 2017; Lomeli et al., 2019) that utilize local surrogate models for explaining data instances, RTX trains a global model to provide fast explanation via one feed-forward process.

Original: Compared with the existing methods, the advantages of RTX mainly lie in two folds: (1) faster explanation generation; and (2) more robust explanation derivation.
Revised: Compared with the existing methods, the advantages of RTX mainly lie in two folds: (1) faster explanation generation; and (2) more robust explanation derivation.

Original: Generally, existing RTXs attempt to learn the overall explanation distribution using two lines of methodologies, which are Shapley-sampling-based approaches (Wang et al., 2021;
Revised: Generally, existing RTXs attempt to learn the overall explanation distribution using two lines of methodologies, which are Shapley-sampling-based approaches (Wang et al., 2021;

Original: Jethani et al., 2021; Covert et al., 2022) and feature-selection-based approaches (Chen et al., 2018;
Revised: Jethani et al., 2021; Covert et al., 2022) and feature-selection-based approaches (Chen et al., 2018;

Original: Dabkowski & Gal, 2017; Kanehira & Harada, 2019).
Revised: Dabkowski & Gal, 2017; Kanehira & Harada, 2019).

Original: The first line enforces the explainer to simulate a given approximated Shapley distribution for generating explanation results.
Revised: The first line enforces the explainer to simulate a given approximated Shapley distribution for generating explanation results.

Original: A representative work, FastSHAP (Jethani et al., 2021), exploits a DNN model to capture the Shapley distribution among training instances for efficient real-time explanations.
Revised: A representative work, FastSHAP (Jethani et al., 2021), exploits a DNN model to capture the Shapley distribution among training instances for efficient real-time explanations.

Original: The second line assumes the specificpre-defined feature patterns or distributions, and formulates the explainer learning process based on
Revised: The second line assumes the specific

Original: [Figure]
Revised: [Figure]

Original:  the given assumptions.
Revised: pre-defined feature patterns or distributions, and formulates the explainer learning process based on the given assumptions.

Original: One of the work (Chen et al., 2018) provides a feature masking generator for real-time feature selection.
Revised: One of the work (Chen et al., 2018) provides a feature masking generator for real-time feature selection.

Original: The training process of the mask generator is under the constraint of the given ground-truth label distribution.
Revised: The training process of the mask generator is under the constraint of the given ground-truth label distribution.

Original: Typically, Shapley-sampling-based approaches are more effective but suffer from high computational cost when examining larger feature subsets, while feature-selection-based methods are easy-to-train but constrained under the specific distributions.
Revised: Both Shapley-sampling-based and feature-selection-based methods highly depend on explanation labels during the training phase. Although labels may bring strong supervised information to the explanation models that benefit explanation tasks, generating high-quality labels typically requires high computational time, which makes the labels hard to acquire.

Original: In this work, we propose an unsupervised framework CoRTX, achieving the advantages from both sides.
Revised: In this work, we propose an unsupervised learning paradigm CoRTX, which can significantly reduce the dependency on the explanation labels.

Revised: The proposed CoRTX benefits the explanation tasks via exploiting a contrastive framework to generate latent explanations without relying on the labels.

Original: 2. L IMITATION OF S UPERVISED F RAMEWORK
Revised: 2. L IMITATION OF S UPERVISED F RAMEWORK

Original: [Figure]
Revised: [Figure]

Original: Supervised RTX typically relies on large quantities of groundtruth explanation labels, which limits its application in practice.
Revised: Supervised RTX typically relies on large quantities of groundtruth explanation labels, which limits its application in practice.

Original: We experimentally show that supervised RTX suffers from the performance degradation when explanation labels are insufficient.
Revised: We experimentally show that supervised RTX suffers from the performance degradation when explanation labels are insufficient.

Original: Specifically, taking the Shapley value as the ground truth, supervised RTX learns to minimize the cross entropy on the training set, and then estimates the feature ranking on the testing set for evaluation.
Revised: Specifically, taking the Shapley value as the ground truth, supervised RTX learns to minimize the cross entropy on the training set, and then estimates the feature ranking on the testing set for evaluation.

Original: The results on Census dataset (Dua & Graff, 2017) demonstrate the intensive dependence of supervised RTX on the amount of explanation labels.
Revised: The results on Census dataset (Dua & Graff, 2017) demonstrate the intensive dependence of supervised RTX on the amount of explanation labels.

Original: The performance of feature ranking versus the exploitation ratio of explanation label is shown in Figure 1, where the implementation details are introduced in Appendix D. Here, we observe that the performance of feature ranking significantly enhances as the ratio of label increases.
Revised: The performance of feature ranking versus the exploitation ratio of explanation label is shown in Figure 1, where the implementation details are introduced in Appendix D. Here, we observe that the performance of feature ranking significantly enhances as the ratio of label increases.

Original: Supervised RTXs outperform FastSHAP, which is trained with approximated labels, under 5% label usage.
Revised: Supervised RTXs outperform FastSHAP, which is trained with approximated labels, under 5% label usage.

Original: However, the complexity of calculating
Revised: However, the complexity of calculating

Original: Shapley values generally grows exponentially with the feature number, which makes it challenging to train RTX with a large amount of labels.
Revised: Shapley values generally grows exponentially with the feature number, which makes it challenging to train RTX with a large amount of labels.

Original: We propose CoRTX in this paper to train the explainer in a light-label manner, where only a small amount of explanation labels are used for fine-tuning the downstream head.
Revised: We propose CoRTX in this paper to train the explainer in a light-label manner, where only a small amount of explanation labels are used for fine-tuning the downstream head.

Original:  C ONTRASTIVE R EAL - TIME E XPLANATION
Revised:  C ONTRASTIVE R EAL - TIME E XPLANATION

Original: We systematically introduce the CoRTX framework in this section.
Revised: We systematically introduce the CoRTX framework in this section.

Original: Figure 2 illustrates the overall pipeline of CoRTX.
Revised: Figure 2 illustrates the overall pipeline of CoRTX.

Original: In particular, our pipeline first uses an explanation encoder for the contrastive explanation representation learning, and then an explanation head is further tuned with a small amount of explanation labels.
Revised: In particular, our pipeline first uses an explanation encoder to learn the latent explanation in a contrastive way, and then an explanation head is further tuned with a small amount of explanation labels.

Original: An explanation-oriented positive augmentation and negative sampling strategy is designed for the  explanationrepresentation learning process.
Revised: An explanation-oriented positive augmentation and negative sampling strategy is designed for the latent explanation learning process.

Original: 3. P OSITIVE D ATA A UGMENTATION TOWARD S IMILAR E XPLANATION
Revised: 3. P OSITIVE D ATA A UGMENTATION TOWARD S IMILAR E XPLANATION

Original: Different from the conventional data augmentation (Liu et al., 2021) for representation learning,
Revised: Different from the conventional data augmentation (Liu et al., 2021) for representation learning,

Original: CoRTX develops an explanation-oriented data augmentation strategy for training RTX, where the augmented positive and negative instances of explanation are proved to be beneficial (Kim et al., 2016; Dhurandhar et al., 2018).
Revised: CoRTX develops an explanation-oriented data augmentation strategy for training RTX, where the augmented positive and negative instances of explanation are proved to be beneficial (Kim et al., 2016; Dhurandhar et al., 2018).

Original: Specifically, considering an anchor data instance x ∈ X , the set of synthetic positive instances X + is generated via m times of independent perturbations on x :
Revised: Specifically, considering an anchor data instance x ∈ X , the set of synthetic positive instances X + is generated via m times of independent perturbations on x :

Original: [Equation]
Revised: [Equation]

Original: where S i is sampled from the M -dim binomial distribution, and λ controls the probability density.
Revised: where S i is sampled from the M -dim binomial distribution, and λ controls the probability density.

Original: CoRTX provides the explanation-oriented data augmentation by selecting the compact positive instances (cid:101) x
Revised: CoRTX provides the explanation-oriented data augmentation by selecting the compact positive instances (cid:101) x

Original: + from X + .
Revised: + from X + .

Original: Intuitively, (cid:101) x
Revised: Intuitively, (cid:101) x

Original: + is expect to obtain the most similar feature attribution as the corresponding x .
Revised: + is expect to obtain the most similar model explanation as the corresponding x , which means the feature attribution between (cid:101) x + and x are highly similar.

Original: However, the greedy selection of (cid:101) x
Revised: However, the greedy selection of (cid:101) x

Original: + needs the explanation score of each instance in X + , which is hard to access in practice.
Revised: + needs the explanation score of each instance in X + , which is hard to access in practice.

Original: To tackle the issue, we propose Theorem 1 to guide the selection of (cid:101) x
Revised: To tackle the issue, we propose Theorem 1 to guide the selection of (cid:101) x

Original: + through a set of synthetic positive instances without accessing the explanation scores.
Revised: + through a set of synthetic positive instances without accessing the explanation scores.

Original: The proof of Theorems 1 is shown in Appendix A.
Revised: The proof of Theoremsis shown in Appendix A.

Original: Theorem 1 ( Compact Alignment ) .
Revised: Theorem 1 ( Compact Alignment ) .

Original: Let f ( x ) be a K f -Lipschitz 2 continuous function and Φ ( x ) =
Revised: Let f ( x ) be a K f -Lipschitz 2 continuous function and Φ ( x ) =

Original: [ ϕ 1 ( x ) , · · · , ϕ M ( x )] be the feature importance scores, where ϕ i ( x ) :
Revised: [ ϕ 1 ( x ) , · · · , ϕ M ( x )] be the feature importance scores, where ϕ i ( x ) :

Original: =
Revised: =

Original: E S⊆ U \{ i } (cid:2)
Revised: E S⊆ U \{ i } (cid:2)

Original: f ( (cid:101) x
Revised: f ( (cid:101) x

Original: S∪{ i } ) − f ( (cid:101) x S ) (cid:3) .
Revised: S∪{ i } ) − f ( (cid:101) x S ) (cid:3) .

Original: Given a perturbed positive instance (cid:101) x ∈X + satisfying min 1 ≤ i ≤ M ϕ
Revised: Given a perturbed positive instance (cid:101) x ∈X + satisfying [Table] [Equation] [Table]

Original: i ( (cid:101) x ) ≥ 0 , the explanation difference ∥
Revised: [Equation] the explanation difference ∥

Original: Φ ( x ) − Φ ( (cid:101) x ) ∥ 2 is bounded by the prediction difference | f ( x ) − f ( (cid:101) x ) | as
Revised: Φ ( x ) − Φ ( (cid:101) x ) ∥ 2 is bounded by the prediction difference | f ( x ) − f ( (cid:101) x ) | as

Original: [Equation]
Revised: [Equation]

Original: where γ 

Revised: where γ

Original: =
Revised: =

Original: K f ∥ x ∥ 2 and K f ≥ 0 is the Lipschitz constant of function f ( · ) .
Revised: K f ∥ x ∥ 2 and K f ≥ 0 is the Lipschitz constant of function f ( · ) .

Original: Theorem 1 basically shows that the explanation difference can be bounded by the prediction difference between the anchor x and the perturbed positive instance (cid:101) x .
Revised: Theorem 1 basically shows that the explanation difference can be bounded by the prediction difference between the anchor x and the perturbed positive instance (cid:101) x .

Original: The compact positive instance (cid:101) x
Revised: The compact positive instance (cid:101) x

Original: + in Theorem 1 can derive the minimal difference of model outputs and further result in a similar explanation to the anchor x i .
Revised: + in Theorem 1 can derive the minimal difference of model outputs and further result in a similar explanation to the anchor x i .

Original: Following Theorem 1, CoRTX selects (cid:101) x
Revised: Following Theorem 1, CoRTX selects (cid:101) x

Original: + i ∈ X
Revised: + i ∈ X

Original: + by holding the minimum preceding difference between of f ( x ) and f ( (cid:101) x i ) .
Revised: + by holding the minimum preceding difference between of f ( x ) and f ( (cid:101) x i ) .

Original: Formally, the selection strategy can be indicated as:
Revised: Formally, the selection strategy can be indicated as:

Original: [Equation]
Revised: [Equation]

Original: where f ( · ) is the target model for explanation.
Revised: where f ( · ) is the target model for explanation.

Original: Selecting hard positive instances has been proved to benefit contrastive learning (Grill et al., 2020; Zhu et al., 2021).
Revised: Selecting excessively positive instances has been proved to degrade contrastive learning (Zhu et al., 2021).

Original: The hard positive instance is expected to be nearly positive to the anchor instance.
Revised: The excessively positive instances refer to those very similar to the anchor instance.

Original: However, our (cid:101) x
Revised: Regarding our (cid:101) x

Original: + may become excessively positive when selected from the universe set of X + . Following this criterion, X + is set to be much smaller than the entire universe set,
Revised: + may become excessively positive when selected from the universe set, X + is set to be much smaller than the entire universe set,

Original: where |X + | = m ≪ 2 M .
Revised: where |X + | = m ≪ 2 M .

Original: For example, we set |X
Revised: For example, we set |X

Original: + | =300 ≪ 2 96 on the Bankruptcy dataset (Liang et al., 2016).
Revised: + | =300 ≪ 2on the Bankruptcy dataset (Liang et al., 2016).

Original: In practice, CoRTX uses Equation 4 to search the compact positive instance (cid:101) x
Revised: In practice, CoRTX uses Equation 4 to search the compact positive instance (cid:101) x

Original: + from X + , where the excessively positive instances may be avoided.
Revised: + from X + , where the excessively positive instances may be avoided.

Original: In this way, CoRTX can ensure hard positive instances to be as many as possible, which benefits the  explanationrepresentation learning process.
Revised: In this way, CoRTX can ensure hard positive instances to be as many as possible, which benefits the latent explanation learning process.

Original: 3. E XPLANATION C ONTRASTIVE L OSS
Revised: 3. E XPLANATION C ONTRASTIVE L OSS

Original: The proposed CoRTX adopts the contrastive learning (He et al., 2020;
Revised: The proposed CoRTX adopts the contrastive learning (He et al., 2020;

Original: Chen et al., 2020) to generate the  explanationrepresentation by training an explanation encoder g ( · | θ g ) :
Revised: Chen et al., 2020) to generate the latent explanation by training an explanation encoder g ( · | θ g ) :

Original: R M → R d .
Revised: R M → R d .

Original: Here, a positive pair and several negative pairs are exploited during the training of g ( · | θ g ) .
Revised: Here, a positive pair and several negative pairs are exploited during the training of g ( · | θ g ) .

Original: A positive pair includes the anchor x i and a compact positive instance (cid:101) x
Revised: A positive pair includes the anchor x i and a compact positive instance (cid:101) x

Original: + i .
Revised: + i .

Original: A negative pair contains the anchor x i and a randomly selected instance x j ( j ̸ = i ), where the explanations are mostly different.
Revised: A negative pair contains the anchor x i and a randomly selected instance x j ( j ̸ = i ), where the explanations are mostly different.

Original: Let h i
Revised: Let h i

Original: = g ( x i | θ g ) , (cid:101)
Revised: = g ( x i | θ g ) , (cid:101)

Original: h
Revised: h

Original: + i
Revised: + i

Original: =
Revised: =

Original: g
Revised: g

Original: ( (cid:101) x
Revised: ( (cid:101) x

Original: + i | θ g ) be the  explanationrepresentation of the positive pair, and h i , h j = g ( x j | θ g ) be the representation for a negative pair.
Revised: + i | θ g ) be the latent explanation of the positive pair, and h i , h j = g ( x j | θ g ) be the latent explanation for a negative pair.

Original: CoRTX trains the encoder g ( · | θ g ) by maximizing the dot similarity of positive pairs and minimizing the dot similarity of negative pairs.
Revised: CoRTX trains the encoder g ( · | θ g ) by maximizing the dot similarity of positive pairs and minimizing the dot similarity of negative pairs.

Original: Following the criterion, g ( · | θ g ) essentially minimizes the contrastive loss as follows:
Revised: Following the criterion, g ( · | θ g ) essentially minimizes the contrastive loss as follows:

Original: [Equation]
Revised: [Equation]

Original: where τ is a temperature hyper-parameter (Wu et al., 2018) and N denotes the batch size for training.
Revised: where τ is a temperature hyper-parameter (Wu et al., 2018) and N denotes the batch size for training.

Original: To provide the explanation for f ( · ) from the learned representation in g ( · | θ g ) , an explanation head η ( · | θ η ) : R d → R M is further tuned based on the certain explanation tasks.
Revised: To provide the explanation for f ( · ) from the learned latent explanation in g ( · | θ g ) , an explanation head η ( · | θ η ) : R d → R M is further tuned based on the certain explanation tasks.

Original: Specifically, η ( · | θ η ) is tuned with a small amount of explanation labels (e.g., the Shapley values), where the computational cost is relatively affordable in real-world settings.
Revised: Specifically, η ( · | θ η ) is tuned with a small amount of explanation labels (e.g., the Shapley values), where the computational cost is relatively affordable in real-world settings.

Original: η ( · | θ η ) is specifically designed to cope with the different scenarios of model explanation.
Revised: η ( · | θ η ) is specifically designed to cope with the different scenarios of model explanation.

Revised: According to the effects of Equation 4 and

Revised: Equation 5, CoRTX enforces h i to be similar to (cid:101)

Revised: h

Revised: + i and dissimilar to h j , which is consistent with the relationship of their model explanations.

Revised: This enables the learned representation to contain the explanation information.

Original: In this work, CoRTX is studied under two common explanation scenarios, i.e., feature attribution task and feature ranking task.
Revised: In this work, CoRTX is studied under two common explanation scenarios, i.e., feature attribution task and feature ranking task.

Original: To ensure the explanation quality of CoRTX, we here provide Theorem 2 to show that the explanation error of CoRTX is bounded with theoretical guarantees.
Revised: To ensure the explanation quality of CoRTX, we here provide Theorem 2 to show that the explanation error of CoRTX is bounded with theoretical guarantees.

Original: The proof of Theorems 2 is provided in Appendix B.
Revised: The proof of Theorems 2 is provided in Appendix B.

Original: Theorem 2 ( Explanation Error Bound ) .
Revised: Theorem 2 ( Explanation Error Bound ) .

Original: Let η ( ·| θ η ) be K η -Lipschitz continuous, and ˆ Φ ( · )
Revised: Let η ( ·| θ η ) be K η -Lipschitz continuous, and ˆ Φ ( · )

Original: =
Revised: =

Original: η ( g ( ·| θ g ) | θ η ) be the estimated explanations.
Revised: η ( g ( ·| θ g ) | θ η ) be the estimated explanations.

Original: If the encoder g ( ·| θ g ) is well-trained, satisfying ∥ Φ ( x ) − ˆ Φ( x ) ∥ 2 ≤E ∈ R + , then the explanation error on testing instance x k can be bounded as:
Revised: If the encoder g ( ·| θ g ) is well-trained, satisfying ∥ Φ ( x [Equation] − ˆ Φ

Original: [Equation]
Revised: [Equation]

Revised: x

Original: [Equation]
Revised: [Equation]

Revised: 2 ≤E ∈ R + , then the explanation error on testing instance x k can be bounded as:

Revised: [Equation]

Revised: [Equation]

Original: where (cid:101) x
Revised: where (cid:101) x

Original: + k is a compact positive instance,
Revised: + k is a compact positive instance,

Original: h x k = g
Revised: h x k = g

Original: ( x k | θ g ) , and γ k = K f
Revised: ( x k | θ g ) , and γ k = K f

Original: ∥ x k ∥ 2 .
Revised: ∥ x k ∥ 2 .

Original: Theorem 2 shows that the upper bound of explanation error depends on four different terms, which are γ k , training error E , | f ( x ) − f ( (cid:101) x + k ) | , and ∥ h (cid:101) x
Revised: Theorem 2 shows that the upper bound of explanation error depends on four different terms, which are γ k , training error E , | f ( x ) − f ( (cid:101) x + k ) | , and ∥ h (cid:101) x

Original: k − h x k ∥ 2 .
Revised: k − h x k ∥ 2 .

Original: CoRTX contributes to minimize the upper bound by explicitly minimizing | f ( x ) − f ( (cid:101) x + k ) | and ∥ h (cid:101) x
Revised: CoRTX contributes to minimize the upper bound by explicitly minimizing | f ( x ) − f ( (cid:101) x + k ) | and ∥ h (cid:101) x

Original: k − h x k ∥ 2 as follows:
Revised: k − h x k ∥ 2 as follows:

Original: • CoRTX selects the compact positive instances following the selection strategy in Equation 4, which obtains the minimal value of | f ( x ) − f ( (cid:101) x + k ) | ; • CoRTX maximizes the similarity between positive pairs while training the encoder g ( ·| θ g ) , which minimizes the value of ∥ h (cid:101) x
Revised: • CoRTX selects the compact positive instances following the selection strategy in Equation 4, which obtains the minimal value of | f ( x ) − f ( (cid:101) x + k ) | ; • CoRTX maximizes the similarity between positive pairs while training the encoder g ( ·| θ g ) , which minimizes the value of ∥ h (cid:101) x

Original: k − h x k ∥ 2 .
Revised: k − h x k ∥ 2 .

Original: Besides the two terms mentioned above, the error E is naturally minimized after g ( · | θ g ) is trained, and γ k keeps constant for the given f ( · ) .
Revised: Besides the two terms mentioned above, the error E is naturally minimized after g ( · | θ g ) is trained, and γ k keeps constant for the given f ( · ) .

Original: Thus, we can theoretically guarantee the explanation error bound of CoRTX by Equation 6.
Revised: Thus, we can theoretically guarantee the explanation error bound of CoRTX by Equation 6.

Original: This ensures the capability of CoRTX to provide effective explanations to the target model over the testing set.
Revised: This ensures the capability of CoRTX to provide effective explanations to the target model over the testing set.

Original: 3. A LGORITHM OF C O RTX
Revised: 3. A LGORITHM OF C O RTX

Original: The outline of CoRTX is given in Algorithm 1.
Revised: The outline of CoRTX is given in Algorithm 1.

Original: CoRTX follows Equation 4 for compact positive selection (lines 4-5), and learns the explanation encoder g ( · | θ g ) according to Equation 5 (line 6).
Revised: CoRTX follows Equation 4 for compact positive selection (lines 4-5), and learns the explanation encoder g ( · | θ g ) according to Equation 5 (line 6).

Original: The training iteration terminates when g ( · | θ g ) is converged.
Revised: The training iteration terminates when g ( · | θ g ) is converged.

Original: Overall, g ( · | θ g ) learns the  explanationrepresentation for each instance x
Revised: Overall, g ( · | θ g ) learns the latent explanation for each instance x

Original: = [ x 1 , · · · , x M ] ∈ X , and η ( · | θ η ) is then fine-tuned with a small amount of explanation labels for particular tasks.
Revised: = [ x 1 , · · · , x M ] ∈X , and η ( · | θ η ) is then fine-tuned with a small amount of explanation labels for particular tasks.

Original: The algorithmic process of CoRTX can be expressed as η ( g ( x i | θ g ) | θ η ) in general.
Revised: The algorithmic process of CoRTX can be expressed as η ( g ( x i | θ g ) | θ η ) in general.

Original: [Table]
Revised: [Table]

Original: [Table]

Original: [Table]

Original: [Table]
Revised: [Table]

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we conduct experiments to evaluate the performance of CoRTX, aiming to answer the following three research questions: RQ1: How does CoRTX perform on explanation tasks in terms of the efficacy and efficiency compared with state-of-the-art baselines?
Revised: In this section, we conduct experiments to evaluate the performance of CoRTX, aiming to answer the following three research questions: RQ1: How does CoRTX perform on explanation tasks in terms of the efficacy and efficiency compared with state-of-the-art baselines?

Original: RQ2: Is the  explanationrepresentation from CoRTX effective on fine-tuning the explanation head?
Revised: RQ2: Is the latent explanation from CoRTX effective on fine-tuning the explanation head?

Original: RQ3: Does synthetic augmentation contribute to the explanation performance of CoRTX?
Revised: RQ3: Does synthetic augmentation contribute to the explanation performance of CoRTX?

Original: 4. D ATASETS AND B ASELINES
Revised: 4. D ATASETS AND B ASELINES

Original: Datasets.
Revised: Datasets.

Original: Our experiments consider two tabular datasets: Census (Dua & Graff, 2017) with 13 features, Bankruptcy (Liang et al., 2016) with 96 features, and one image dataset: CIFAR-10 (Krizhevsky et al., 2009) with 32 × 32 pixels.
Revised: Our experiments consider two tabular datasets: Census (Dua & Graff, 2017) with 13 features, Bankruptcy (Liang et al., 2016) with 96 features, and one image dataset: CIFAR-10 (Krizhevsky et al., 2009) with 32 × 32 pixels.

Original: The preprocessing and statistics of three datasets are provided in Appendix C. Baseline Methods.
Revised: The preprocessing and statistics of three datasets are provided in Appendix C. Baseline Methods.

Original: In Census and Bankruptcy datasets, CoRTX is compared with two RTX methods, i.e., Supervised RTX and FastSHAP (Jethani et al., 2021), as well as two non-amortized explanation methods, i.e., KernelSHAP (KS) (Lundberg & Lee, 2017) and Permutation Sampling (PS) (Mitchell et al., 2021).
Revised: In Census and Bankruptcy datasets, CoRTX is compared with two RTX methods, i.e., Supervised RTX and FastSHAP (Jethani et al., 2021), as well as two non-amortized explanation methods, i.e., KernelSHAP (KS) (Lundberg & Lee, 2017) and Permutation Sampling (PS) (Mitchell et al., 2021).

Original: In CIFAR-10 dataset, CoRTX is compared with FastSHAP and other non-amortized methods, including DeepSHAP (Lundberg & Lee, 2017), Saliency (Simonyan et al., 2013), Integrated Gradients (IG) (Sundararajan et al., 2017), SmoothGrad (Smilkov et al., 2017), and
Revised: In CIFAR-10 dataset, CoRTX is compared with FastSHAP and other non-amortized methods, including DeepSHAP (Lundberg & Lee, 2017), Saliency (Simonyan et al., 2013), Integrated Gradients (IG) (Sundararajan et al., 2017), SmoothGrad (Smilkov et al., 2017), and

Original: GradCAM (Selvaraju et al., 2017).
Revised: GradCAM (Selvaraju et al., 2017).

Original: More details about the baselines can be found in Appendix D.
Revised: More details about the baselines can be found in Appendix D.

Original: 4. E XPERIMENTAL S ETTINGS AND E VALUATION M ETRICS
Revised: E XPERIMENTAL S ETTINGS AND E VALUATION M ETRICS

Original: In this part, we introduce the experimental settings and metrics on evaluating CoRTX.
Revised: In this part, we introduce the experimental settings and metrics on evaluating CoRTX.

Original: The considered explanation tasks and implementation details are shown as follows.
Revised: The considered explanation tasks and implementation details are shown as follows.

Original: Feature Attribution Task.
Revised: Feature Attribution Task.

Original: This task aims to test the explanation performance on feature attribution.
Revised: This task aims to test the explanation performance on feature attribution.

Original: We here implement CoRTX-MSE on fine-tuning the explanation head η
Revised: We here implement CoRTX-MSE on fine-tuning the explanation head η

Original: ( x i | θ η ) .
Revised: ( x i | θ η ) .

Original: Given g ( x i | θ g ) , the explanation values are predicted through [ˆ ϕ 1 , · · · , ˆ ϕ M ] =
Revised: Given g ( x i | θ g ) , the explanation values are predicted through [ˆ ϕ 1 , · · · , ˆ ϕ M ] =

Original: η ( g ( x i | θ g ) | θ η ) , where ˆ ϕ i indicates the attribution scores of feature i .
Revised: η ( g ( x i | θ g ) | θ η ) , where ˆ ϕ i indicates the attribution scores of feature i .

Original: Let [ ϕ 1 , · · · , ϕ M ] be the explanation label of an instance x i ,
Revised: Let [ ϕ 1 , · · · , ϕ M ] be the explanation label of an instance x i ,

Original: CoRTX-MSE learns η ( · | θ η ) by minimizing the mean-square loss L MSE = 1 M (cid:80) Mj
Revised: CoRTX-MSE learns η ( · | θ η ) by minimizing the mean-square loss L MSE = 1 M (cid:80) Mj

Original: =1 (cid:0) ˆ ϕ j − ϕ j (cid:1) 2 .
Revised: =1 (cid:0) ˆ ϕ j − ϕ j (cid:1) 2 .

Original: To evaluate the performance, we follow the existing work (Jethani et al., 2021) to estimate the ℓ 2 -error of each instance on the testing set, where the ℓ 2 -error is calculated by (cid:113)(cid:80) Mj
Revised: To evaluate the performance, we follow the existing work (Jethani et al., 2021) to estimate the ℓ 2 -error of each instance on the testing set, where the ℓ 2 -error is calculated by (cid:113)(cid:80) Mj

Original: =1 (cid:0) ϕ
Revised: =1 (cid:0) ϕ

Original: j − ˆ ϕ j (cid:1) 2 .
Revised: j − ˆ ϕ j (cid:1) 2 .

Original: Feature Importance Ranking Task.
Revised: Feature Importance Ranking Task.

Original: This task aims to evaluate the explanation performance on feature ranking index.
Revised: This task aims to evaluate the explanation performance on feature ranking index.

Original: We here implement CoRTX-CE for fine-tuning the explanation head η ( · | θ η ) .
Revised: We here implement CoRTX-CE for fine-tuning the explanation head η ( · | θ η ) .

Original: Given g ( x i | θ g ) , the feature ranking index is generated by [ˆ r 1 , · · · , ˆ r M ] =
Revised: Given g ( x i | θ g ) , the feature ranking index is generated by [ˆ r 1 , · · · , ˆ r M ] =

Original: η ( g ( x i | θ g ) | θ η ) for each instance x i .
Revised: η ( g ( x i | θ g ) | θ η ) for each instance x i .

Original: The predicted feature ranking index [ˆr 1 , · · · , ˆr M ] is given by ˆr j = arg max
Revised: The predicted feature ranking index [ˆr 1 , · · · , ˆr M ] is given by ˆr j = arg max

Original: ˆ r j for 1 ≤ j ≤ M , where arg max( · ) returns the index with the maximal importance score.
Revised: ˆ r j for 1 ≤ j ≤ M , where arg max( · ) returns the index with the maximal importance score.

Original: Let [r 1 , · · · , r M ] 3 be the ground-truth ranking label, CoRTX-CE learns η ( · | θ η ) by minimizing the loss function L CE =
Revised: Let [r 1 , · · · , r M ] 3 be the ground-truth ranking label, CoRTX-CE learns η ( · | θ η ) by minimizing the loss function L CE =

Original: (cid:80)
Revised: (cid:80)

Original: Mj
Revised: Mj

Original: =
Revised: =

Original: 1 CrossEntropy(ˆ r j , onehot (r j )) .
Revised: 1 CrossEntropy(ˆ r j , onehot (r j )) .

Original: To evaluate the feature importance ranking, we follow existing work (Wang et al., 2022), where the ranking accuracy of each instance is given by
Revised: To evaluate the feature importance ranking, we follow existing work (Wang et al., 2022), where the ranking accuracy of each instance is given by

Original: Rank
Revised: Rank

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Evaluation of Efficiency.
Revised: Evaluation of Efficiency.

Original: The algorithmic throughput is exploited to evaluate the speed of explaining process (Wang et al., 2022;
Revised: The algorithmic throughput is exploited to evaluate the speed of explaining process (Wang et al., 2022;

Original: Teich & Teich, 2018).
Revised: Teich & Teich, 2018).

Original: Specifically, the throughput is calculated by N test t total , where N test and t total denote the testing instance number and the overall time consumption of explanation derivation, respectively.
Revised: Specifically, the throughput is calculated by N test t total , where N test and t total denote the testing instance number and the overall time consumption of explanation derivation, respectively.

Original: Higher throughput indicates the higher efficiency of the explanation process.
Revised: Higher throughput indicates the higher efficiency of the explanation process.

Original: In experiments, t total is measured based on the physical computing infrastructure given in Appendix E.3.
Revised: In experiments, t total is measured based on the physical computing infrastructure given in Appendix E.3.

Original: Implementation Details.
Revised: Implementation Details.

Original: In experiments, CoRTX is implemented under three different prediction models f ( · ) .
Revised: In experiments, CoRTX is implemented under three different prediction models f ( · ) .

Original: AutoInt (Song et al., 2019) is adopted for Census data, MLP model is used for
Revised: AutoInt (Song et al., 2019) is adopted for Census data, MLP model is used for

Original: Bankruptcy data, and ResNet-18 is employed for CIFAR-10.
Revised: Bankruptcy data, and ResNet-18 is employed for CIFAR-10.

Original: As for the ground-truth explanation labels, we calculate the exact Shapley values for Census dataset.
Revised: As for the ground-truth explanation labels, we calculate the exact Shapley values for Census dataset.

Original: For the other two datasets, we utilize the approximated methods to generate explanation labels (Covert & Lee, 2021; Jethani et al., 2021), due to the high time complexity of exact Shapley value calculation.
Revised: For the other two datasets, we utilize the approximated methods to generate explanation labels (Covert & Lee, 2021; Jethani et al., 2021), due to the high time complexity of exact Shapley value calculation.

Original: The estimated values from Antithetical Permutation Sampling (APS) (Mitchell et al., 2021; Lomeli et al., 2019) and KS are proved to be infinitely closed to the exact Shapley values when they involve sufficient samples (Covert & Lee, 2021).
Revised: The estimated values from Antithetical Permutation Sampling (APS) (Mitchell et al., 2021; Lomeli et al., 2019) and KS are proved to be infinitely closed to the exact Shapley values when they involve sufficient samples (Covert [Figure] [Figure] Lee, 2021).

Original: In experiments, we employ APS on Bankruptcy and KS on CIFAR-10 for obtaining explanation labels.
Revised: In experiments, we employ APS on Bankruptcy and KS on CIFAR-10 for obtaining explanation labels.

Original: More details are introduced in Appendix E.
Revised: More details are introduced in Appendix E.

Original: [Figure]

Original: [Figure]

Original: 4. T ABULAR E XPERIMENTS 4.3. E XPLANATION E FFICACY AND E FFICIENCY (RQ1)
Revised: 4. T ABULAR E XPERIMENTS 4.3. E XPLANATION E FFICACY AND E FFICIENCY (RQ1)

Original: We compare CoRTX with the baseline methods on deriving model explanations.
Revised: We compare CoRTX with the baseline methods on deriving model explanations.

Original: The results in feature attribution and feature ranking are shown by ℓ 2 -error versus throughput and Rank ACC versus throughput, respectively.
Revised: The results in feature attribution and feature ranking are shown by ℓ 2 -error versus throughput and Rank ACC versus throughput, respectively.

Original: The reported results of CoRTX adopts 25% of the explanation labels on fine-tuning the explanation head.
Revised: The reported results of CoRTX adopts 25% of the explanation labels on fine-tuning the explanation head.

Original: The performance comparisons are illustrated in Figures 3, where (a), (b) demonstrate the performance on feature ranking and (c), (d) show the performance on feature attribution.
Revised: The performance comparisons are illustrated in Figures 3, where (a), (b) demonstrate the performance on feature ranking and (c), (d) show the performance on feature attribution.

Original: According to the experimental results, we have the following observations:
Revised: According to the experimental results, we have the following observations:

Original: • CoRTX vs. Non-Amortized Methods : Compared with the non-amortized explanation methods with large times of model evaluations, CoRTX achieves competitive explanation performance.
Revised: • CoRTX vs. Non-Amortized Methods : Compared with the non-amortized explanation methods with large times of model evaluations, CoRTX achieves competitive explanation performance.

Original: In Census data, we observe CoRTX is competitive to PS and KS with respectively 2 9 and 2 10 of model evaluations.
Revised: In Census data, we observe CoRTX is competitive to PS and KS with respectively 2 9 and 2 10 of model evaluations.

Original: For KS and PS, it is noted that there is a significant decrease in Rank ACC and an obvious increase in ℓ 2 -error as the throughput grows.
Revised: For KS and PS, it is noted that there is a significant decrease in Rank ACC and an obvious increase in ℓ 2 -error as the throughput grows.

Original: This indicates that KS and PS both suffer from an undesirable trade-off between the explanation speed and performance.
Revised: This indicates that KS and PS both suffer from an undesirable trade-off between the explanation speed and performance.

Original: In contrast, the proposed CoRTX achieves both efficient and effective explanations for the target model.
Revised: In contrast, the proposed CoRTX achieves both efficient and effective explanations for the target model.

Original: • CoRTX vs. FastSHAP : CoRTX outperforms FastSHAP on Rank ACC and ℓ 2 -error under the same level of throughput.
Revised: • CoRTX vs. FastSHAP : CoRTX outperforms FastSHAP on Rank ACC and ℓ 2 -error under the same level of throughput.

Original: This observation shows that CoRTX provides more effective explanation under the scenario of RTX.
Revised: This observation shows that CoRTX provides more effective explanation under the scenario of RTX.

Original: • CoRTX-MSE vs. CoRTX-CE : CoRTX consistently provides effective solutions for the two explanation tasks.
Revised: • CoRTX-MSE vs. CoRTX-CE : CoRTX consistently provides effective solutions for the two explanation tasks.

Original: CoRTX-MSE is competitive on the feature attribution and ranking task, whileCoRTX
Revised: CoRTX-MSE is competitive on the feature attribution and ranking task, while

Original: -CE performs even better than CoRTX-MSE in the ranking task.
Revised: CoRTX-CE performs even better than CoRTX-MSE in the ranking task.

Original: The superiority of CoRTXCE on ranking task largely results from the loss function in fine-tuning, where appropriate tuning loss can help enhance the explanation quality for certain tasks.
Revised: The superiority of CoRTXCE on ranking task largely results from the loss function in fine-tuning, where appropriate tuning loss can help enhance the explanation quality for certain tasks.

Original: 4.3. C ONTRIBUTIONS ON  E XPLANATIONR EPRESENTATION (RQ2)
Revised: 4.3. C ONTRIBUTIONS ON L ATENT E XPLANATION (RQ2)

Original: In this experiments, we study the effectiveness of  explanationrepresentation in deriving explanation results.
Revised: In this experiments, we study the effectiveness of latent explanation in deriving explanation results.

Original: Given the obtained explanation representation, Figure 4 demonstrates the explanation performance of the RTXframeworks with different label usage ratios. We compare CoRTX with FastSHAP and Supervised RTX and summarize the key observations as below: [Figure]
Revised: Given the obtained latent explanation, Figure 4 demonstrates the explanation performance of the RTX

Original:  u c k
Revised: T r u c k

Original: CoRTX
Revised: CoRTX

Original: FastSHAP KernelSHAP
Revised: FastSHAP KernelSHAP

Original: Saliency
Revised: Saliency

Original: IG
Revised: IG

Original: Smoothgrad Gradcam
Revised: Smoothgrad Gradcam

Original: Deepshap
Revised: Deepshap

Original: [Figure]
Revised: B i r d A i r p l a n e frameworks with different label usage ratios. We compare CoRTX with FastSHAP and Supervised RTX and summarize the key observations as below:

Original: • Effectiveness of Encoding : CoRTX-MSE and CoRTX-CE consistently outperform the Supervised RTX on different label proportions.
Revised: • Effectiveness of Encoding : CoRTX-MSE and CoRTX-CE consistently outperform the Supervised RTX on different label proportions.

Original: The results on two explanation tasks show that CoRTX provides effective  explanationrepresentation h i in fine-tuning the head η
Revised: The results on two explanation tasks show that CoRTX provides effective latent explanation h i in fine-tuning the head η

Original: ( h i | θ η ) .
Revised: ( h i | θ η ) .

Original: • Sparsity of Labels : CoRTX-MSE and CoRTX-CE can provide effective explanation outcomes when explanation labels are limited.
Revised: • Sparsity of Labels : CoRTX-MSE and CoRTX-CE can provide effective explanation outcomes when explanation labels are limited.

Original: The results indicate that CoRTX can be potentially applied to large-scale datasets, where the explanation labels are computationally expensive to obtain.
Revised: The results indicate that CoRTX can be potentially applied to large-scale datasets, where the explanation labels are computationally expensive to obtain.

Original: • Effectiveness of Labels : CoRTX and Supervised RTX consistently outperform FastSHAP when having more than 5% explanation labels.
Revised: • Effectiveness of Labels : CoRTX and Supervised RTX consistently outperform FastSHAP when having more than 5% explanation labels.

Original: FastSHAP synchronously generates the explanation labels with low-sampling times during the training.
Revised: FastSHAP synchronously generates the explanation labels with low-sampling times during the training.

Original: In contrast, CoRTX and Supervised RTX are trained with high-quality labels, which are the exact Shapley values or the approximated ones with high-sampling times.
Revised: In contrast, CoRTX and Supervised RTX are trained with high-quality labels, which are the exact Shapley values or the approximated ones with high-sampling times.

Original: The results indicate that high-quality explanation labels can significantly benefit the learning process of the explainer, even when exploiting only 5% explanation labels.
Revised: The results indicate that high-quality explanation labels can significantly benefit the learning process of the explainer, even when exploiting only 5% explanation labels.

Original: 4.3. A BLATION S TUDIES ON S YNTHETIC P OSITIVE A UGMENTATIONS (RQ3)
Revised: 4.3. A BLATION S TUDIES ON S YNTHETIC P OSITIVE A UGMENTATIONS (RQ3)

Original: [Figure]
Revised: [Figure]

Original: We further conduct the ablation studies on synthetic positive augmentation in Section 3.1, aiming to prove its validity in learning explanation representation.
Revised: We further conduct the ablation studies on synthetic positive augmentation in Section 3.1, aiming to prove its validity in learning latent explanation.

Original: Since negative pairs are randomly selected in CoRTX, we here mainly focus on the positive augmentation in our studies.
Revised: Since negative pairs are randomly selected in CoRTX, we here mainly focus on the positive augmentation in our studies.

Original: The ablation studies are conducted under both attribution and ranking tasks on Census dataset.
Revised: The ablation studies are conducted under both attribution and ranking tasks on Census dataset.

Original: In the experiments, CoRTX is compared with two methods: CoRTX w/o Compact Alignment (CA) and CoRTX w/ Maximum Alignment (MA) .
Revised: In the experiments, CoRTX is compared with two methods: CoRTX w/o Compact Alignment (CA) and CoRTX w/ Maximum Alignment (MA) .

Original: To be concrete, CoRTX w/o CA replaces Equation 4 with the existing settings of contrastive learning, which is a random masking strategy.
Revised: To be concrete, CoRTX w/o CA replaces Equation 4 with the existing settings of contrastive learning, which is a random masking strategy.

Original: CoRTX w/ MA replaces the minimum selecting strategy in Equation 4 with the maximum one on synthetic instance set, which is randomly masked from the anchor instance.
Revised: CoRTX w/ MA replaces the minimum selecting strategy in Equation 4 with the maximum one on synthetic instance set, which is randomly masked from the anchor instance.

Original: The remaining settings follow the same as CoRTX.
Revised: The remaining settings follow the same as CoRTX.

Original: Figure 6 demonstrates the comparisons among CoRTX w/o CA , CoRTX w/ MA and CoRTX.
Revised: Figure 6 demonstrates the comparisons among CoRTX w/o CA , CoRTX w/ MA and CoRTX.

Original: We can observe that CoRTX outperforms CoRTX w/o CA and
Revised: We can observe that CoRTX outperforms CoRTX w/o CA and

Original: CoRTX w/ MA under different proportions of explanation label usage.
Revised: CoRTX w/ MA under different proportions of explanation label usage.

Original: This demonstrates that the designed synthetic positive augmentation significantly benefits the efficacy of explanation results.
Revised: This demonstrates that the designed synthetic positive augmentation significantly benefits the efficacy of explanation results.

Original: 4. I MAGE E XPERIMENTS (RQ1)
Revised: 4. I MAGE E XPERIMENTS (RQ1)

Original: Unlike the tabular cases, image data is typically composed of high-dimension pixel features.
Revised: Unlike the tabular cases, image data is typically composed of high-dimension pixel features.

Original: To further test the efficacy of explanation representation, we evaluate CoRTX on CIFAR-10 dataset.
Revised: To further test the efficacy of latent explanation, we evaluate CoRTX on CIFAR-10 dataset.

Original: We compare CoRTX with FastSHAP and six non-amortized image attribution methods.
Revised: We compare CoRTX with FastSHAP and six non-amortized image attribution methods.

Original: The explanation results of CoRTX are fine-tuned using 5% of ground-truth explanation labels.
Revised: The explanation results of CoRTX are fine-tuned using 5% of ground-truth explanation labels.

Original: CoRTX and FastSHAP output 2 × 2 super-pixel attributions (Jethani et al., 2021) for explaining the prediction of image classification.
Revised: CoRTX and FastSHAP output 2 × 2 super-pixel attributions (Jethani et al., 2021) for explaining the prediction of image classification.

Original: More details are provided in Appendix E.
Revised: More details are provided in Appendix E.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table] [Figure] 
Revised: [Table] [Figure] [Table]

Original: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original:  cc u r a c y
Revised: A cc u r a c y

Original: CoRTXFastSHAPKernelSHAPSaliencyIGSmoothgradGradcamDeepshap
Revised: CoRTXFastSHAPKernelSHAPSaliencyIGSmoothgradGradcamDeepshap

Original: 4.4. A C ASE S TUDY OF C O RTX
Revised: 4.4. A C ASE S TUDY OF C O RTX

Original: We visualize the explanation results in Figure 5.
Revised: We visualize the explanation results in Figure 5.

Original: It shows that CoRTX, FastSHAP, and GradCAM are able to highlight the relevant objects corresponding to the image attribution labels.
Revised: It shows that CoRTX, FastSHAP, and GradCAM are able to highlight the relevant objects corresponding to the image attribution labels.

Original: Specifically,
Revised: Specifically,

Original: CoRTX provides more effective explanation results and highlights the particular regions which are essential to the model predictions.
Revised: CoRTX provides better human-understandable explanation results.

Original: In contrast, we observe that the important regions localized by FastSHAP are larger than relevant objects, which are less accurate for explanation.
Revised: In contrast, we observe that the important regions localized by FastSHAP are larger than relevant objects.

Original: Moreover, GradCAM provides theimportant regions only on tiny relevant objects, making the explanations less effective.
Revised: Moreover, GradCAM highlights the regions only on tiny relevant objects. This makes FastSHAP and GradCAM are less human-understandable for explanation.

Original: Furthermore, CoRTX outperforms the other baselines in most of the examples in case studies.

Original: These observations validate that the explanations from CoRTX are more effective.
Revised: These observations validate that the explanations from CoRTX are more human-understandable.

Original: More case studies are available in Appendix H.
Revised: More case studies are available in Appendix ?? .

Original: 4.4. Q UANTITATIVE E VALUATION
Revised: 4.4. Q UANTITATIVE E VALUATION

Original: To quantitatively investigate the explanation results on CIFAR-10, we test CoRTX with the exclusion and inclusion comparing with FastSHAP as well as Shapley-based and gradient-based non-amortized methods.
Revised: To quantitatively investigate the explanation results on CIFAR-10, we test CoRTX with the exclusion and inclusion comparing with FastSHAP as well as Shapley-based and gradient-based non-amortized methods.

Original: Following the experimental settings from existing work (Petsiuk et al., 2018; Jethani et al., 2021), we utilize the AUC of Top-1 accuracy and the AUC of Log - odds as the metrics for evaluation (Jethani et al., 2021).
Revised: Following the experimental settings from existing work (Petsiuk et al., 2018; Jethani et al., 2021), we utilize the AUC of Top-1 accuracy and the AUC of Log - odds as the metrics for evaluation (Jethani et al., 2021).

Original: Specifically, the testing images are orderly masked according to the estimated feature importance scores.
Revised: Specifically, the testing images are orderly masked according to the estimated feature importance scores.

Original: Once the important pixels are removed from the input images, Top-1 accuracy and Log - odds are expected to drop drastically, obtaining a low exclusion
Revised: Once the important pixels are removed from the input images, Top-1 accuracy and Log - odds are expected to drop drastically, obtaining a low exclusion

Original: AUC and a high inclusion AUC.
Revised: AUC and a high inclusion AUC.

Original: More details about Log - odds are given in Appendix G. Figure 6 and
Revised: More details about Log - odds are given in Appendix G. Figure 6 and

Original: Table 1 demonstrate the results of the exclusion and inclusion under 1000 images.
Revised: Table 1 demonstrate the results of the exclusion and inclusion under 1000 images.

Original: We observe that
Revised: We observe that

Original: CoRTX outperforms all other baselines in two evaluations, which obtains the lowest exclusion AUC of Top-1 Accuracy and the lowest inclusion AUC of Log - odds .
Revised: CoRTX outperforms all other baselines in two evaluations, which obtains the lowest exclusion AUC of Top-1 Accuracy and the lowest inclusion AUC of Log - odds .

Original: CoRTX is also competitive with the state-of-the-art baseline, FastSHAP, on the other two evaluations.
Revised: CoRTX is also competitive with the state-of-the-art baseline, FastSHAP, on the other two evaluations.

Original: Besides, CoRTX performs better than non-amortized baseline methods in all four evaluations.
Revised: Besides, CoRTX performs better than non-amortized baseline methods in all four evaluations.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: In this paper, we propose a real-time explanation framework, CoRTX, based on contrastive learning techniques.
Revised: In this paper, we propose a real-time explanation framework, CoRTX, based on contrastive learning techniques.

Original: Specifically, CoRTX contains two components, i.e., explanation encoder and explanation head, which learns the  explanationrepresentation and then provides explanation results for certain explanation tasks.
Revised: Specifically, CoRTX contains two components, i.e., explanation encoder and explanation head, which learns the latent explanation and then provides explanation results for certain explanation tasks.

Original: CoRTX introduces an explanation-oriented data augmentation strategy for learning explanation representation.
Revised: CoRTX introduces an explanation-oriented data augmentation strategy for learning latent explanation.

Original: An explanation head can be further fine-tuned with a small amount of explanation labels.
Revised: An explanation head can be further fine-tuned with a small amount of explanation labels.

Original: Theoretical analysis indicate our proposed data augmentation and contrastive scheme can effectively bound the explanation error.
Revised: Theoretical analysis indicate our proposed data augmentation and contrastive scheme can effectively bound the explanation error.

Original: The experimental results on three datasets demonstrate that CoRTX outperforms the state-of-the-art baselines regarding to the explanation efficacy and efficiency.
Revised: The experimental results on three datasets demonstrate that CoRTX outperforms the state-of-the-art baselines regarding to the explanation efficacy and efficiency.

Original: As for future directions, we consider to extend CoRTX to multiple domains to obtain general pre-trained  explanationrepresentation for downstream scenarios.
Revised: As for future directions, we consider to extend CoRTX to multiple domains to obtain general pre-trained latent explanation for downstream scenarios.

Original: It is also promising to explore healthcare applications with CoRTX framework, where the explanation labels require intensive human knowledge.
Revised: It is also promising to explore healthcare applications with CoRTX framework, where the explanation labels require intensive human knowledge.

Original: A PPENDIX A P ROOF OF T HEOREM
Revised: A PPENDIX A P ROOF OF T HEOREM

Original: We prove Theorem 1 in this section.
Revised: We prove Theorem 1 in this section.

Original: Theorem 1 ( Compact Alignment ) .
Revised: Theorem 1 ( Compact Alignment ) .

Original: Let f ( x ) be a K f -Lipschitz 4 continuous function and Φ ( x ) =
Revised: Let f ( x ) be a K f -Lipschitz 4 continuous function and Φ ( x ) =

Original: [ ϕ 1 ( x ) , · · · , ϕ M ( x )] be the feature importance scores, where ϕ i ( x ) :
Revised: [ ϕ 1 ( x ) , · · · , ϕ M ( x )] be the feature importance scores, where ϕ i ( x ) :

Original: =
Revised: =

Original: E S⊆ U \{ i } (cid:2)
Revised: E S⊆ U \{ i } (cid:2)

Original: f ( (cid:101) x
Revised: f ( (cid:101) x

Original: S∪{ i } ) − f ( (cid:101) x S ) (cid:3) .
Revised: S∪{ i } ) − f ( (cid:101) x S ) (cid:3) .

Original: Given a perturbed positive instance (cid:101) x ∈X + satisfying min 1 ≤ i ≤ M ϕ
Revised: Given a perturbed positive instance (cid:101) x ∈X + satisfying min 1 ≤ i ≤ M ϕ

Original: i ( (cid:101) x ) ≥ 0 , the explanation difference ∥
Revised: i ( (cid:101) x ) ≥ 0 , the explanation difference ∥

Original: Φ ( x ) − Φ ( (cid:101) x ) ∥ 2 is bounded by the prediction difference | f ( x ) − f (
Revised: Φ ( x ) − Φ ( (cid:101) x ) ∥ 2 is bounded by the prediction difference | f ( x ) − f (

Original: [Equation]
Revised: [Equation]

Original: x ) | as
Revised: x ) | as

Original: [Equation]
Revised: [Equation]

Original: where γ 

Revised: where γ

Original: =
Revised: =

Original: K f ∥ x ∥ 2 and K f ≥ 0 is the Lipschitz constant of function f ( · ) .
Revised: K f ∥ x ∥ 2 and K f ≥ 0 is the Lipschitz constant of function f ( · ) .

Original: Proof.
Revised: Proof.

Original: In order to calculate the importance score Φ i ( x ):=
Revised: In order to calculate the importance score Φ i ( x ):=

Original: E S∼ U \{ i } (cid:2)
Revised: E S∼ U \{ i } (cid:2)

Original: f
Revised: f

Original: ( (cid:101) x
Revised: ( (cid:101) x

Original: S∪{ i } ) − f ( (cid:101) x S ) (cid:3) of the feature subset S , we recast the formulation under the expression of S = 1 S ∈ { 0 , 1 } M as follows:
Revised: S∪{ i } ) − f ( (cid:101) x S ) (cid:3) of the feature subset S , we recast the formulation under the expression of S = 1 S ∈ { 0 , 1 } M as follows:

Original: [Equation]
Revised: [Equation]

Original: where S ∪ [1] i denotes not to mask feature i -th from perturbed masking S .
Revised: where S ∪ [1] i denotes not to mask feature i -th from perturbed masking S .

Original: Following Equation 8, we can now discuss explanation difference by each feature.
Revised: Following Equation 8, we can now discuss explanation difference by each feature.

Original: Without lost of generality, we consider the explanation difference of a single feature i as follows,
Revised: Without lost of generality, we consider the explanation difference of a single feature i as follows,

Original: [Equation]
Revised: [Equation]

Original: Following Equation
Revised: Following Equation

Original: [Equation]
Revised: [Equation]

Original: the upper bound of the explanation difference can be derived by
Revised: the upper bound of the explanation difference can be derived by

Original: [Equation]
Revised: [Equation]

Original: Note that the difference of prediction scores | f ( x ) − f ( (cid:101) x ) | is equal to the summation of contribution score among all features (i.e., | f ( x ) − f ( (cid:101) x ) | = | (cid:80) Mi
Revised: Note that the difference of prediction scores | f ( x ) − f ( (cid:101) x ) | is equal to the summation of contribution score among all features (i.e., | f ( x ) − f ( (cid:101) x ) | = | (cid:80) Mi

Original: =

Revised: =

Original: δ i | ).
Revised: δ i | ).

Original: Since we consider the assumption min 1 ≤ i ≤ M Φ i ( (cid:101) x ) ≥ 0 , we have | (cid:80)
Revised: Since we consider the assumption min 1 ≤ i ≤ M Φ i ( (cid:101) x ) ≥ 0 , we have | (cid:80)

Original: Mi
Revised: Mi

Original: =

Revised: =

Original: δ i | = (cid:80) Mi
Revised: δ i | = (cid:80) Mi

Original: =1 | δ i | .
Revised: =1 | δ i | .

Original: In this manner
Revised: In this manner

Original: [Equation]
Revised: [Equation]

Original: we have the upper bound of the explanation difference ∥
Revised: we have the upper bound of the explanation difference ∥

Original: Φ ( x ) − Φ ( (cid:101) x ) ∥ 2 as follows:
Revised: Φ ( x ) − Φ ( (cid:101) x ) ∥ 2 as follows:

Original: [Equation]
Revised: [Equation]

Original: where γ 

Revised: where γ

Original: =
Revised: =

Original: K f ∥ x ∥ 2 .
Revised: K f ∥ x ∥ 2 .

Original: B P ROOF OF T HEOREM
Revised: B P ROOF OF T HEOREM

Original: Theorem 2 ( Explanation Error Bound ) .
Revised: Theorem 2 ( Explanation Error Bound ) .

Original: Let η ( ·| θ η ) be K η -Lipschitz continuous, and ˆ Φ ( · )
Revised: Let η ( ·| θ η ) be K η -Lipschitz continuous, and ˆ Φ ( · )

Original: =
Revised: =

Original: η ( g ( ·| θ g ) | θ η ) be the estimated explanations.
Revised: η ( g ( ·| θ g ) | θ η ) be the estimated explanations.

Original: If the encoder g ( ·| θ g ) is well-trained, satisfying ∥ Φ ( x ) − ˆ Φ ( x ) ∥ 2 ≤E ∈ R + , then the explanation error on testing instance x k can be bounded as:
Revised: If the encoder g ( ·| θ g ) is well-trained, satisfying ∥ Φ ( x ) − ˆ Φ ( x ) ∥ 2 ≤E ∈ R + , then the explanation error on testing instance x k can be bounded as:

Original: [Equation]
Revised: [Equation]

Original: where (cid:101) x
Revised: where (cid:101) x

Original: + k is a compact positive instance,
Revised: + k is a compact positive instance,

Original: h x k = g
Revised: h x k = g

Original: ( x k | θ g ) , and γ k = K f
Revised: ( x k | θ g ) , and γ k = K f

Original: ∥ x k ∥ 2 .
Revised: ∥ x k ∥ 2 .

Original: Proof.
Revised: Proof.

Original: Without loss of generality, we consider ℓ 2 norm to evaluate the distance between predicted explanation scores and ground-truth explanation scores.
Revised: Without loss of generality, we consider ℓ 2 norm to evaluate the distance between predicted explanation scores and ground-truth explanation scores.

Original: For all x k ∈ C , we have ∥
Revised: For all x k ∈ C , we have ∥

Original: [Equation]
Revised: [Equation]

Original: C D ETAILS ABOUT D ATASETS
Revised: C D ETAILS ABOUT D ATASETS

Original: The experiments are conducted on two tabular datasets and one image dataset.
Revised: The experiments are conducted on two tabular datasets and one image dataset.

Original: The details of the datasets are provided as follows:
Revised: The details of the datasets are provided as follows:

Original: • Census Income: A collection of human social information with 26048 samples for training and validating; and 6513 samples for testing.
Revised: • Census Income: A collection of human social information with 26048 samples for training and validating; and 6513 samples for testing.

Original: Each instance has five continuous features and eight categorical features.
Revised: Each instance has five continuous features and eight categorical features.

Original: • Bankruptcy: A financial dataset contains 5455 samples of companies in the training set and validating set; and 1364 instances for the testing set.
Revised: • Bankruptcy: A financial dataset contains 5455 samples of companies in the training set and validating set; and 1364 instances for the testing set.

Original: Each sample has 96 features characterizing each company and whether it went bankrupt.
Revised: Each sample has 96 features characterizing each company and whether it went bankrupt.

Original: • CIFAR-10:
Revised: • CIFAR-10:

Original: An image dataset with 60000 images in 10 different classes, where each image is composed of 32×32 pixels.
Revised: An image dataset with 60000 images in 10 different classes, where each image is composed of 32×32 pixels.

Original: We follow the original dataset division on training, validating, and testing process.
Revised: We follow the original dataset division on training, validating, and testing process.

Original: D D ETAILS ABOUT B ASELINE M ETHODS
Revised: D D ETAILS ABOUT B ASELINE M ETHODS

Original: Some of the baseline algorithms are implemented under the open-source package 5 and the hyperparameters are all decided with the optimal model convergence.
Revised: Some of the baseline algorithms are implemented under the open-source package 5 and the hyperparameters are all decided with the optimal model convergence.

Original: In our experiments, the RTX frameworks, such as CoRTX and FastSHAP, are typically built on the training set and evaluated on the testing dataset.
Revised: In our experiments, the RTX frameworks, such as CoRTX and FastSHAP, are typically built on the training set and evaluated on the testing dataset.

Original: Tabular Dataset: We here provide some detailed information about the baselines on tabular dataset.
Revised: Tabular Dataset: We here provide some detailed information about the baselines on tabular dataset.

Original: Supervised RTX: A supervised RTX-based MLP model trains with raw features of data instances and ground-truth explanation labels from scratch.
Revised: Supervised RTX: A supervised RTX-based MLP model trains with raw features of data instances and ground-truth explanation labels from scratch.

Original: The layer number in Supervised RTX is set to be six for purpose of fair comparison to CoRTX.
Revised: The layer number in Supervised RTX is set to be six for purpose of fair comparison to CoRTX.

Original: FastSHAP:
Revised: FastSHAP:

Original: A state-of-the-art RTX method which adopts an
Revised: A state-of-the-art RTX method which adopts an

Original: DNN model to universally learn the distribution of approximated Shapley label (Jethani et al., 2021).
Revised: DNN model to universally learn the distribution of approximated Shapley label (Jethani et al., 2021).

Original: KernelSHAP (KS): The model proposes the weighted linear regressions to estimate the Shapley additive explanations from the prediction model scores (Kokhlikyan et al., 2020).
Revised: KernelSHAP (KS): The model proposes the weighted linear regressions to estimate the Shapley additive explanations from the prediction model scores (Kokhlikyan et al., 2020).

Original: Permutation
Revised: Permutation

Original: Sampling (PS): PS estimates the feature attribution based on calculating the sensitivity gap of the scores from prediction model while inputting randomly masking data features (Mitchell et al., 2021).
Revised: Sampling (PS): PS estimates the feature attribution based on calculating the sensitivity gap of the scores from prediction model while inputting randomly masking data features (Mitchell et al., 2021).

Original: To achieve the competitive performance on the baselines, we demonstrate the following settings to the
Revised: To achieve the competitive performance on the baselines, we demonstrate the following settings to the

Original: KS and PS in the tabular experiments.
Revised: KS and PS in the tabular experiments.

Original: On Census Income dataset, the numbers of model evaluations on KS is set from 2 5 to 2 10 , and the numbers of model evaluations on PS is given from 2 4 to 2 10 .
Revised: On Census Income dataset, the numbers of model evaluations on KS is set from 2 5 to 2 10 , and the numbers of model evaluations on PS is given from 2 4 to 2 10 .

Original: On
Revised: On

Original: Bankruptcy dataset, the s of model evaluations on KS is set from 2 3 to 2 11 , and numbers of model evaluations on PS are from 2 3 to 2 10 .
Revised: Bankruptcy dataset, the s of model evaluations on KS is set from 2 3 to 2 11 , and numbers of model evaluations on PS are from 2 3 to 2 10 .

Original: For a fair comparison, CoRTX and Supervised RTX adopt the same loss function while conducting on the same explanation task.
Revised: For a fair comparison, CoRTX and Supervised RTX adopt the same loss function while conducting on the same explanation task.

Original: Image Dataset: We evaluate the results with two non-amortized Shapley-based estimators:
Revised: Image Dataset: We evaluate the results with two non-amortized Shapley-based estimators:

Original: DeepSHAP and KS, where the first work utilizes a linear composition rule to estimate the Shapley values on the non-linear neural networks.
Revised: DeepSHAP and KS, where the first work utilizes a linear composition rule to estimate the Shapley values on the non-linear neural networks.

Original: Moreover, gradient-based methods are known for their efficiency compared to Shapley-based estimators.
Revised: Moreover, gradient-based methods are known for their efficiency compared to Shapley-based estimators.

Original: We list some gradient-based baselines: Saliency,
Revised: We list some gradient-based baselines: Saliency,

Original: Integrated Gradients (IG), SmoothGrad, and GradCAM.
Revised: Integrated Gradients (IG), SmoothGrad, and GradCAM.

Original: The mentioned four approaches derive the explanation based on the gradient values of prediction models.
Revised: The mentioned four approaches derive the explanation based on the gradient values of prediction models.

Original: Lastly, to evaluate the efficacy of the image explainers, we adopt a state-of-the-art baseline, FastSHAP, to test the explanation generating speed and accuracy.
Revised: Lastly, to evaluate the efficacy of the image explainers, we adopt a state-of-the-art baseline, FastSHAP, to test the explanation generating speed and accuracy.

Original: To achieve the better performance on baselines, the evaluation times on baselines is basically decided on grid search once achieving competitive performance.
Revised: To achieve the better performance on baselines, the evaluation times on baselines is basically decided on grid search once achieving competitive performance.

Original: E E XPERIMENT D ETAILS
Revised: E E XPERIMENT D ETAILS

Original: We conduct the experiments on tabular dataset and image datasets.
Revised: We conduct the experiments on tabular dataset and image datasets.

Original: For the tabular dataset, we consider two common explanation tasks: feature attribution and importance ranking task.
Revised: For the tabular dataset, we consider two common explanation tasks: feature attribution and importance ranking task.

Original: For the image dataset, the explanation results are presented using heatmap in the case study and evaluated by including and excluding the important pixels for perturbed image classification.
Revised: For the image dataset, the explanation results are presented using heatmap in the case study and evaluated by including and excluding the important pixels for perturbed image classification.

Original: E. E XPERIMENTS ON T ABULAR D ATASET
Revised: E. E XPERIMENTS ON T ABULAR D ATASET

Original: We verify the feature ranking task by using CoRTX-CE and the feature attribution task by exploiting
Revised: We verify the feature ranking task by using CoRTX-CE and the feature attribution task by exploiting

Original: CoRTX-MSE.
Revised: CoRTX-MSE.

Original: The experiment on each tabular dataset follows the pipeline of Target Model Training ,
Revised: The experiment on each tabular dataset follows the pipeline of Target Model Training ,

Original: Explanation Benchmarks and Explainer Implementing .
Revised: Explanation Benchmarks and Explainer Implementing .

Original: Each step is shown as follows.
Revised: Each step is shown as follows.

Original: Target Model Training : We exploit different prediction models f ( · ) on three datasets to evaluate the model-agnostic property of CoRTX.
Revised: Target Model Training : We exploit different prediction models f ( · ) on three datasets to evaluate the model-agnostic property of CoRTX.

Original: AutoInt (Song et al., 2019) is adopted for the Census Income dataset, and the MLP model is for the Bankruptcy dataset.
Revised: AutoInt (Song et al., 2019) is adopted for the Census Income dataset, and the MLP model is for the Bankruptcy dataset.

Original: The two prediction models are trained until the convergence, and the implementation is based on the DeepCTR 6 package (Shen, 2017).
Revised: The two prediction models are trained until the convergence, and the implementation is based on the DeepCTR 6 package (Shen, 2017).

Original: Due to the tasks in two datasets, a binary cross entropy loss is given as the loss function of AutoInt and MLP model for the Census Income and Bankruptcy datasets, respectively.
Revised: Due to the tasks in two datasets, a binary cross entropy loss is given as the loss function of AutoInt and MLP model for the Census Income and Bankruptcy datasets, respectively.

Original: All hyper-parameters on target model training are decided by grid search throughout the classification results, including model layers, hidden units, etc.
Revised: All hyper-parameters on target model training are decided by grid search throughout the classification results, including model layers, hidden units, etc.

Original: Explanation Benchmarks : A brute-force algorithm for calculating the exact Shapley value is adopted for the Census Income dataset as the explanation labels.
Revised: Explanation Benchmarks : A brute-force algorithm for calculating the exact Shapley value is adopted for the Census Income dataset as the explanation labels.

Original: However, the Bankruptcy dataset contains
Revised: However, the Bankruptcy dataset contains

Original: https://captum.aihttps://github
Revised: https://captum.aihttps://github

Original: many features (96 features), so it is hard to gain the exact Shapley value for the Bankruptcy dataset due to the extremely high computational cost.
Revised: many features (96 features), so it is hard to gain the exact Shapley value for the Bankruptcy dataset due to the extremely high computational cost.

Original: We hereby adopt the proximity of Shapley values as the explanation labels by using Antithetical Permutation Sampling(APS)
Revised: We hereby adopt the proximity of Shapley values as the explanation labels by using Antithetical Permutation Sampling(APS)

Original: (Mitchell et al., 2021;
Revised: (Mitchell et al., 2021;

Original: Lomeli et al., 2019) to convergence.
Revised: Lomeli et al., 2019) to convergence.

Original: This is because APS has shown to converge to exact Shapley values when countering high permutation times on tabular datasets.
Revised: This is because APS has shown to converge to exact Shapley values when countering high permutation times on tabular datasets.

Original: In this work, we set the model permutation times as 2048 to generate the ground-truth labels.
Revised: In this work, we set the model permutation times as 2048 to generate the ground-truth labels.

Original: Explainer Implementation :
Revised: Explainer Implementation :

Original: The explainer is implemented on explanation representation learning  and fine-tuning.
Revised: The explainer is implemented on the learning of latent explanation and fine-tuning.

Original: The explanation encoder and the explanation head are designed with the model structures and are learned with the hyper-parameters in Table 2.
Revised: The explanation encoder and the explanation head are designed with the model structures and are learned with the hyper-parameters in Table 2.

Original: Following the existing works (Jethani et al., 2021; Dhurandhar et al., 2018), we adjust the feature importance scores via additive efficient normalization (Ruiz et al., 1998) for the feature attribution task.
Revised: Following the existing works (Jethani et al., 2021; Dhurandhar et al., 2018), we adjust the feature importance scores via additive efficient normalization (Ruiz et al., 1998) for the feature attribution task.

Original: E. E XPERIMENTS ON I MAGE D ATASET
Revised: E. E XPERIMENTS ON I MAGE D ATASET

Original: Our experiments on the image dataset are conducted under CoRTX-MSE.
Revised: Our experiments on the image dataset are conducted under CoRTX-MSE.

Original: We focus on the feature attribution task on the image datasets, which follows the pipeline of Target Model Training ,
Revised: We focus on the feature attribution task on the image datasets, which follows the pipeline of Target Model Training ,

Original: Explanation Benchmarks and Explainer Implementing .
Revised: Explanation Benchmarks and Explainer Implementing .

Original: Each step is shown as follows.
Revised: Each step is shown as follows.

Original: Target Model Training : We utilize ResNet-18 as the prediction models f ( · ) in CIFAR-10 dataset.
Revised: Target Model Training : We utilize ResNet-18 as the prediction models f ( · ) in CIFAR-10 dataset.

Original: We train the target model from scratch until the convergence.
Revised: We train the target model from scratch until the convergence.

Original: All hyper-parameters on model training are decided by grid search throughout the classification results and follow the training setting of the target model from (Jethani et al., 2021).
Revised: All hyper-parameters on model training are decided by grid search throughout the classification results and follow the training setting of the target model from (Jethani et al., 2021).

Original: Explanation Benchmarks : We calculate the approximation to Shapley values as the training labels by adopting KS until the convergence (Covert & Lee, 2021; Jethani et al., 2021).
Revised: Explanation Benchmarks : We calculate the approximation to Shapley values as the training labels by adopting KS until the convergence (Covert & Lee, 2021; Jethani et al., 2021).

Original: The estimated values from KS can infinitely approach exact Shapley values when encountering the optimal convergence (Lundberg & Lee, 2017).
Revised: The estimated values from KS can infinitely approach exact Shapley values when encountering the optimal convergence (Lundberg & Lee, 2017).

Original: In this work, we set the model permutation times as 2048 to generate the ground-truth labels.
Revised: In this work, we set the model permutation times as 2048 to generate the ground-truth labels.

Original: Explainer Implementation : The explanation encoder and the explanation head are designed with the model structures and are learned with the hyper-parameters in Table 2.
Revised: Explainer Implementation : The explanation encoder and the explanation head are designed with the model structures and are learned with the hyper-parameters in Table 2.

Original: Following the existing work (Jethani et al., 2021), we adopt the surrogate model with the same structure as ResNet-18.
Revised: Following the existing work (Jethani et al., 2021), we adopt the surrogate model with the same structure as ResNet-18.

Original: The surrogate model is distilled from the original prediction model with masked input images to overcome the effect of out-of-distribution in masking perturbation (Frye et al., 2020).
Revised: The surrogate model is distilled from the original prediction model with masked input images to overcome the effect of out-of-distribution in masking perturbation (Frye et al., 2020).

Original: Following existing work (Jethani et al., 2021), we adjust the feature attribution scores through additive efficient normalization (Ruiz et al., 1998) for generating the feature importance scores.
Revised: Following existing work (Jethani et al., 2021), we adjust the feature attribution scores through additive efficient normalization (Ruiz et al., 1998) for generating the feature importance scores.

Original: Moreover, we follow the setting from the existing work (Jethani et al., 2021) to conduct the bootstrapping evaluation.
Revised: Moreover, we follow the setting from the existing work (Jethani et al., 2021) to conduct the bootstrapping evaluation.

Original: The total testing candidates are chosen to be 1000 and we randomly select 666 instances with 20 times of repetitions for getting the bootstrap AUCs and Log - odds .
Revised: The total testing candidates are chosen to be 1000 and we randomly select 666 instances with 20 times of repetitions for getting the bootstrap AUCs and Log - odds .

Original: [Table]
Revised: [Table]

Original: E. C OMPUTATION I NFRASTRUCTURE
Revised: E. C OMPUTATION I NFRASTRUCTURE

Original: For the fair comparison on testing algorithmic throughput, the experiments are conducted based on the following physical computing infrastructure in Table 3.
Revised: For the fair comparison on testing algorithmic throughput, the experiments are conducted based on the following physical computing infrastructure in Table 3.

Original: [Table]
Revised: [Table]

Original: F R ELATED W ORK
Revised: F R ELATED W ORK

Original: Non-amortized methods.
Revised: Non-amortized methods.

Original: Existing work of non-amortized methods can be categorized into three groups.
Revised: Existing work of non-amortized methods can be categorized into three groups.

Original: The first group of methods adopts linear regressions to fit the non-amortized explanation, such as LIME (Ribeiro et al., 2016) and KS (Lundberg & Lee, 2017).
Revised: The first group of methods adopts linear regressions to fit the non-amortized explanation, such as LIME (Ribeiro et al., 2016) and KS (Lundberg & Lee, 2017).

Original: Another group of methods adopts the preceding difference of the value function for the explanation, such as RISE (Petsiuk et al., 2018),
Revised: Another group of methods adopts the preceding difference of the value function for the explanation, such as RISE (Petsiuk et al., 2018),

Original: Permutation Sampling (Mitchell et al., 2021) and SHEAR (Wang et al., 2022).
Revised: Permutation Sampling (Mitchell et al., 2021) and SHEAR (Wang et al., 2022).

Original: The last group estimates the gradient towards the input data for the explanation, such as the GradCAM (Selvaraju et al., 2017), Integrated Gradient (Sundararajan et al., 2017) and SmoothGrad (Smilkov et al., 2017).
Revised: The last group estimates the gradient towards the input data for the explanation, such as the GradCAM (Selvaraju et al., 2017), Integrated Gradient (Sundararajan et al., 2017) and SmoothGrad (Smilkov et al., 2017).

Original: Even though the non-amortized method can provide faithful explanation for DNN models, this group of methods suffers from high computational complexity since each data instance requires one non-amortized explainer to yield the explanation.
Revised: Even though the non-amortized method can provide faithful explanation for DNN models, this group of methods suffers from high computational complexity since each data instance requires one non-amortized explainer to yield the explanation.

Original: Real-time Explainer (RTX).
Revised: Real-time Explainer (RTX).

Original: Unlike the non-amortized methods, the RTX framework maintains an unified explainer to generate the explanation among each data instance.
Revised: Unlike the non-amortized methods, the RTX framework maintains an unified explainer to generate the explanation among each data instance.

Original: This way, the explanation can be generated via a single feed-forward process of the explainer, which is much faster than the non-amortized methods.
Revised: This way, the explanation can be generated via a single feed-forward process of the explainer, which is much faster than the non-amortized methods.

Original: The learning strategy of the RTX framework formulates the explainer learning by giving strong assumptions on prior feature distribution (Chen et al., 2018;
Revised: The learning strategy of the RTX framework formulates the explainer learning by giving strong assumptions on prior feature distribution (Chen et al., 2018;

Original: Dabkowski &
Revised: Dabkowski &

Original: Gal, 2017; Kanehira & Harada, 2019).
Revised: Gal, 2017; Kanehira & Harada, 2019).

Original: Existing work (Chen et al., 2018) utilizes the instance-wise feature selection by maintaining a feature masking generator via maximizing the mutual information between selected features and corresponding labels.
Revised: Existing work (Chen et al., 2018) utilizes the instance-wise feature selection by maintaining a feature masking generator via maximizing the mutual information between selected features and corresponding labels.

Original: A well-trained feature masking generator is able to provide real-time explanation under a single feed-forward process.
Revised: A well-trained feature masking generator is able to provide real-time explanation under a single feed-forward process.

Original: Another framework of
Revised: Another framework of

Original: RTX is to adopt the exact or approximated Shapley values as the ground-truth labels to learn the explainers (Wang et al., 2021;
Revised: RTX is to adopt the exact or approximated Shapley values as the ground-truth labels to learn the explainers (Wang et al., 2021;

Original: Jethani et al., 2021; Covert et al., 2022).
Revised: Jethani et al., 2021; Covert et al., 2022).

Original: However, the exploitation of exact Shapley values suffers from extremely high computational complexity.
Revised: However, the exploitation of exact Shapley values suffers from extremely high computational complexity.

Original: To address this problem,
Revised: To address this problem,

Original: FastSHAP (Jethani et al., 2021) proposes a Monte-Carlo-based method to learn the explainer under the RTX framework.
Revised: FastSHAP (Jethani et al., 2021) proposes a Monte-Carlo-based method to learn the explainer under the RTX framework.

Original: Specifically, FastSHAP generates the approximated Shapley values by randomly sampling batches of feature masks during the training process.
Revised: Specifically, FastSHAP generates the approximated Shapley values by randomly sampling batches of feature masks during the training process.

Original: Meanwhile, it updates the explainer to minimize the mean-square error between the overall contribution scores of masked features and outputs from the DNN explainer.
Revised: Meanwhile, it updates the explainer to minimize the mean-square error between the overall contribution scores of masked features and outputs from the DNN explainer.

Original: FastSHAP enforces the explanation performance without utilizing the ground-truth Shapley value, which can be supported by the experiment results.
Revised: FastSHAP enforces the explanation performance without utilizing the ground-truth Shapley value, which can be supported by the experiment results.

Original: G M EASUREMENT OF Log - odds
Revised: G M EASUREMENT OF Log - odds

Original: Definition.
Revised: Definition.

Original: Log - odds indicates the confidence of the prediction given by the target model f ( x ) .
Revised: Log - odds indicates the confidence of the prediction given by the target model f ( x ) .

Original: Formally, Log - odds is defined as the log-likelihood ratio of f ( x ) (Ashton, 1972) as follows,
Revised: Formally, Log - odds is defined as the log-likelihood ratio of f ( x ) (Ashton, 1972) as follows,

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where p is the output probability of the target model f ( x ) .
Revised: where p is the output probability of the target model f ( x ) .

Original: Inclusion and Exclusion AUC of Log - odds .
Revised: Inclusion and Exclusion AUC of Log - odds .

Original: To evaluate the estimated feature attribution on image data, we conduct the exclusion and inclusion test according to the feature importance scores.
Revised: To evaluate the estimated feature attribution on image data, we conduct the exclusion and inclusion test according to the feature importance scores.

Original: Once the important features are removed from the input instances, the target model f ( x ) is expected to drop drastically on Log - odds which leading to a low exclusion AUC.
Revised: Once the important features are removed from the input instances, the target model f ( x ) is expected to drop drastically on Log - odds which leading to a low exclusion AUC.

Original: On the contrary, it is expected to have a high inclusion AUC of Log - odds since the important features are orderly added back to the input instances of target model.
Revised: On the contrary, it is expected to have a high inclusion AUC of Log - odds since the important features are orderly added back to the input instances of target model.

Original: The remaining experimental settings follows(Jethani et al., 2021).
Revised: The remaining experimental settings follows(Jethani et al., 2021).

Original: Inclusion and Exclusion AUC of ∆ Log - odds : Besides exploiting Log - odds on measuring the performance, we additionally test CoRTX with ∆ Log - odds on the inclusion and exclusion task.
Revised: Inclusion and Exclusion AUC of ∆ Log - odds : Besides exploiting Log - odds on measuring the performance, we additionally test CoRTX with ∆ Log - odds on the inclusion and exclusion task.

Original: ∆ Log - odds calculates the preceding difference between the Log - odds of original input instances and masked instances.
Revised: ∆ Log - odds calculates the preceding difference between the Log - odds of original input instances and masked instances.

Original: Different from Log - odds and Top-1 Accuracy, Log - odds has the opposite indication where higher exclusion AUC and lower inclusion AUC represent better performance.
Revised: Different from Log - odds and Top-1 Accuracy, Log - odds has the opposite indication where higher exclusion AUC and lower inclusion AUC represent better performance.

Original: Table 4 shows the results of the exclusion and inclusion under 1000 images.
Revised: Table 4 shows the results of the exclusion and inclusion under 1000 images.

Original: We observe that CoRTX outperforms other non-amortized baseline methods on both inclusion task and exclusion task.
Revised: We observe that CoRTX outperforms other non-amortized baseline methods on both inclusion task and exclusion task.

Original: CoRTX performs better than FastSHAP on the inclusion task and is competitive with FastSHAP on the exclusion task.
Revised: CoRTX performs better than FastSHAP on the inclusion task and is competitive with FastSHAP on the exclusion task.

Original: [Table]
Revised: [Table]

Original: H A DDITIONAL R ESULTS ON I MAGE D ATASET
Revised: H A DDITIONAL R ESULTS ON I MAGE D ATASET

Original: We demonstrate more explanation results on CIFAR-10 generated by CoRTX compared to other baselines.
Revised: We demonstrate more explanation results on CIFAR-10 generated by CoRTX compared to other baselines.

Original: The results show that CoRTX can identify more accurate regions as the explanationresults toward the significant object related to the image classes.
Revised: The results show that CoRTX can identify more human-understandable explanation toward other baselines.

Original: [Figure]
Revised: [Figure]

Original: CoRTX

Original: FastSHAP

Original: KernelSHAP
Revised: KernelSHAP

Original: Saliency
Revised: Saliency

Original: IG
Revised: IG

Original: Smoothgrad
Revised: Smoothgrad

Original: Gradcam
Revised: Gradcam

Original: Deepshap
Revised: Deepshap

Original: [Figure]
Revised: [Figure]

Revised: I A DDITIONAL B ASELINE E XPERIMENTS

Original: [Figure]
Revised: We conduct more experiments on CIFAR-10 dataset to compare CoRTX with two more baseline methods: RISE (Petsiuk et al., 2018) and XRAI (Kapishnikov et al., 2019). All hyper-parameters are decided with the optimal model convergence. The results show that CoRTX can provide more human-understandable explanations than RISE and XRAI. RISE

Revised: XRAI CoRTX

Original: [Figure]
Revised: [Figure]

Revised: i r d

Original: [Figure]
Revised: [Figure]

Revised: i r p l a n e

Original: [Figure]
Revised: We further conduct the experiments to compare L2X (Chen et al., 2018) with CoRTX on the Adult dataset. The hyper-parameters of L2X and CoRTX are all decided with the optimal model convergence. In the experiment, we choose Rank ACC and Faithfulness (Liu et al., 2021) as the metric, as the outputs from L2X are not Shapley-based values.

Revised: Faithfulness is the metric to evaluate explanation tasks without any ground truth.

Original: [Figure]
Revised: As shown in Table 5, CoRTX outperforms L2X on two metrics, indicating the better capability for deriving model explanation. Methods [Table] J A DDITIONAL N OTATION C LARIFICATION

Revised: In this section, Table 6 shows the detailed information of the notations in Section 4.2 for clarification.

Original: [Figure]
Revised: Notation [Table] ˆ r j Predicted ranking index score list of feature x j from CoRTX.

Revised: Each index score in ˆ r j represents the score of feature x

Revised: j that is ranked at this index.

Revised: ˆ r j ∈ R M ˆr j

Revised: Predicted ranking index from CoRTX, where ˆr j =arg max ˆ r j for 1 ≤ j ≤

Original: [Figure]
Revised: [Table]