Original: Exploiting the Relationship Between Kendall’s Rank Correlation and Cosine Similarity for Attribution Protection
Revised: Exploiting the Relationship Between Kendall’s Rank Correlation and Cosine Similarity for Attribution Protection

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Model attributions are important in deep neural networks as they aid practitionersin understanding the models, but recent studies reveal that attributions can beeasily perturbed by adding imperceptible noise to the input.
Revised: Model attributions are important in deep neural networks as they aid practitioners in understanding the models, but recent studies reveal that attributions can be easily perturbed by adding imperceptible noise to the input.

Original: The non-differentiable
Revised: The non-differentiable

Original: Kendall’s rank correlation is a key performance index for attribution protection.
Revised: Kendall’s rank correlation is a key performance index for attribution protection.

Original: Inthis paper, we ﬁrst show that the expected Kendall’s rank correlation is positivelycorrelated to cosine similarity and then indicate that the direction of attribution isthe key to attribution robustness.
Revised: In this paper, we first show that the expected Kendall’s rank correlation is positively correlated to cosine similarity and then indicate that the direction of attribution is the key to attribution robustness.

Original: Based on these ﬁndings, we explore the vectorspace of attribution to explain the shortcomings of attribution defense methodsusing ` p norm and propose integrated gradient regularizer (IGR), which maximizesthe cosine similarity between natural and perturbed attributions.
Revised: Based on these findings, we explore the vector space of attribution to explain the shortcomings of attribution defense methods using ℓ p norm and propose integrated gradient regularizer (IGR), which maximizes the cosine similarity between natural and perturbed attributions.

Original: Our analysisfurther exposes that IGR encourages neurons with the same activation states fornatural samples and the corresponding perturbed samples.
Revised: Our analysis further exposes that IGR encourages neurons with the same activation states for natural samples and the corresponding perturbed samples.

Original: Our experiments ondifferent models and datasets conﬁrm our analysis on attribution protection anddemonstrate a decent improvement in adversarial robustness.
Revised: Our experiments on different models and datasets confirm our analysis on attribution protection and demonstrate a decent improvement in adversarial robustness.

Original:  Introduction
Revised:  Introduction

Original: Recently, the explainable artiﬁcial intelligence (XAI) has revived since deep neural networks (DNNs)are applied to more security-sensitive tasks such as medical imaging [27], criminal justice [5] andautonomous driving [20].
Revised: Recently, the explainable artificial intelligence (XAI) has revived since deep neural networks (DNNs) are applied to more security-sensitive tasks such as medical imaging [27], criminal justice [5] and autonomous driving [20].

Original: As one of the XAI tools, model attributions explain and measure therelative impact of each feature on the ﬁnal prediction.
Revised: As one of the XAI tools, model attributions explain and measure the relative impact of each feature on the final prediction.

Original: With more non-expert practitioners beinginvolved, it is more important for them to understand and reliably interpret the mechanism behindthe outputs.
Revised: With more non-expert practitioners being involved, it is more important for them to understand and reliably interpret the mechanism behind the outputs.

Original: Besides, EU regulators also start to enforce General Data Protection Regulation formore transparent interpretations on decision making based on AI [10].
Revised: Besides, EU regulators also start to enforce General Data Protection Regulation for more transparent interpretations on decision making based on AI [10].

Original: Therefore, the trustworthyattribution is becoming even more crucial.
Revised: Therefore, the trustworthy attribution is becoming even more crucial.

Original: Although numerous attribution methods have been proposed in recent studies [25, 26, 29, 35, 37] ithas been pointed out that they are vulnerable to attribution attacks.
Revised: Although numerous attribution methods have been proposed in recent studies [25, 26, 29, 35, 37], it has been pointed out that they are vulnerable to attribution attacks.

Original: Different from standard adversarialattacks [3, 9, 18, 22, 31] that focus on misleading classiﬁers to incorrect outputs, Ghorbani et al.
Revised: Different from standard adversarial attacks [3, 9, 18, 22, 31] that focus on misleading classifiers to incorrect outputs, Ghorbani et al.

Original: [8]shows that it is possible to generate visually indistinguishable images which are signiﬁcantly differenton their attributions, but with the same predicted label.
Revised: [8] shows that it is possible to generate visually indistinguishable images which are significantly different on their attributions, but with the same predicted label.

Original: Dombrowski et al.
Revised: Dombrowski et al.

Original: [6] emphasizes on targetedattack that manipulates the attributions to any predeﬁned target attributions while keeping the modeloutputs unchanged.
Revised: [6] emphasizes on targeted attack that manipulates the attributions to any predefined target attributions while keeping the model outputs unchanged.

Original: There are also black-box attacks applied on text explanations [14].
Revised: There are also black-box attacks applied on text explanations [14].

Original: Commonadversarial defense mechanisms such as adversarial training [21] and distillation [23] are not able totackle the attribution attacks; instead, researchers turn their focus on the attribution itself.
Revised: Common adversarial defense mechanisms such as adversarial training [21] and distillation [23] are not able to tackle the attribution attacks; instead, researchers turn their focus on the attribution itself.

Original: As the differences between natural and perturbed attributions are measured by Kendall’s rankcorrelation [15], which reﬂects the ordinal importance among features, i.e. , the proportion of order alignment of attributions between original and perturbed images, a straightforward practice to protectthe attributions against such adversaries is to maximize their Kendall’s rank correlation.
Revised: As the differences between natural and perturbed attributions are measured by Kendall’s rank correlation [15], which reflects the ordinal importance among features, i.e. , the proportion of order alignment of attributions between original and perturbed images, a straightforward practice to protect the attributions against such adversaries is to maximize their Kendall’s rank correlation.

Original: Since
Revised: Since

Original: Kendall’s rank correlation is not differentiable, in previous studies, it is replaced by its differentiablealternatives, such as ` p -distance regularizers [2, 4].
Revised: Kendall’s rank correlation is not differentiable, in previous studies, it is replaced by its differentiable alternatives, such as ℓ p -distance regularizers [2, 4].

Original: However, ` p -distance regularizers are not idealfor Kendall’s rank correlation.
Revised: However, ℓ p -distance regularizers are not ideal for Kendall’s rank correlation.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 1, we found that given ﬁxed ` 1 -distance betweenoriginal and perturbed attributions, their Kendall’s rank correlations are drastically different, whichindicates ` 1 -distance is unstable as a measure of attribution similarity.
Revised: 1, we found that given fixed ℓ 1 -distance between original and perturbed attributions, their Kendall’s rank correlations are drastically different, which indicates ℓ 1 -distance is unstable as a measure of attribution similarity.

Original: Besides, there are also non-
Revised: Besides, there are also non-

Original: ` pbased regularizers, such as using Pearson’s correlation, as the surrogate measurement of Kendall’srank correlation [13], it is shown to be unstable to measure the attribution.
Revised: ℓ p based regularizers, such as using Pearson’s correlation, as the surrogate measurement of Kendall’s rank correlation [13], it is shown to be unstable to measure the attribution.

Original: In this paper, we discover that cosine similarity , as a measurement emphasizing the angle betweentwo vectors, is consistent with Kendall’s rank correlation.
Revised: In this paper, we discover that cosine similarity , as a measurement emphasizing the angle between two vectors, is consistent with Kendall’s rank correlation.

Original: We present a theorem stating that cosinesimilarity is positively correlated with the expected Kendall’s rank correlation.
Revised: We present a theorem stating that cosine similarity is positively correlated with the expected Kendall’s rank correlation.

Original: Based on the discoveryof angular perspective, we then explain the shortcomings of ` p -norm based attribution robustnessmethods and propose integrated gradients regularizer (IGR) , an attribution robustness trainingregularizer that optimizes on the cosine similarity between natural and perturbed attribution.
Revised: Based on the discovery of angular perspective, we then explain the shortcomings of ℓ p -norm based attribution robustness methods and propose integrated gradients regularizer (IGR) , an attribution robustness training regularizer that optimizes on the cosine similarity between natural and perturbed attribution.

Original: Ourfurther analysis shows that optimizing cosine similarity encourages neurons with the same activationstates.
Revised: Our further analysis shows that optimizing cosine similarity encourages neurons with the same activation states.

Original: The contributions of this work are summarized as follows:
Revised: The contributions of this work are summarized as follows:

Original: • We theoretically show that, under certain assumptions, Kendall’s rank correlation betweentwo vectors is positively correlated to their cosine similarity.
Revised: • We theoretically show that, under certain assumptions, Kendall’s rank correlation between two vectors is positively correlated to their cosine similarity.

Original: • We characterize a novel geometric perspective related to the angles between attribution vec- tors that explains the connection between adversarial robustness and attribution robustness for attribution methods fulﬁlling the axiom of completeness [29].
Revised: • We characterize a novel geometric perspective related to the angles between attribution vectors that explains the connection between adversarial robustness and attribution robustness for attribution methods fulfilling the axiom of completeness [29].

Original: • Under the angular perspective, we propose integrated gradients regularizer (IGR) to robustlytrain neural networks.
Revised: • Under the angular perspective, we propose integrated gradients regularizer (IGR) to robustly train neural networks.

Original: Our method is proved to encourage neurons with the same activation states for natural and corresponding perturbed images.
Revised: Our method is proved to encourage neurons with the same activation states for natural and corresponding perturbed images.

Original: • The experimental results show that the proposed IGR regularizer can be embedded into adversarial training methods to improve their performance in terms of both attribution and adversarial robustness and outperform the state-of-the-art attribution protection methods.
Revised: • The experimental results show that the proposed IGR regularizer can be embedded into adversarial training methods to improve their performance in terms of both attribution and adversarial robustness and outperform the state-of-the-art attribution protection methods.

Original: The remainder of this paper is organized as follows.
Revised: The remainder of this paper is organized as follows.

Original: We ﬁrst introduce the notations and previousrelated works in Section 2.
Revised: We first introduce the notations and previous related works in Section 2.

Original: The content starts with the theorem disclosing the relationship between
Revised: The content starts with the theorem disclosing the relationship between

Original: Kendall’s rank correlation and cosine similarity in Section 3.
Revised: Kendall’s rank correlation and cosine similarity in Section 3.

Original: Based on that, we discuss the vector space of attribution in Section 4 and describe the proposed IGR as well as its property regardingneuron activations in Section 5.
Revised: Based on that, we discuss the vector space of attribution in Section 4 and describe the proposed IGR as well as its property regarding neuron activations in Section 5.

Original: Section 6 presents our experimental results and the paper concludesin Section 7.
Revised: Section 6 presents our experimental results and the paper concludes in Section 7.

Original:  Preliminaries and related work
Revised:  Preliminaries and related work

Original: Let { ( x ( i ) , y ( i ) ) } ni =1 denote data points sampled from the distribution D , where x ( i ) 2 R d are inputdata and y ( i ) 2 { 1 , . . .
Revised: Let { ( x ( i ) , y ( i ) ) } ni =1 denote data points sampled from the distribution D , where x ( i ) ∈ R d are input data and y ( i ) ∈ { 1 , . . .

Original: , k } are labels.
Revised: , k } are labels.

Original: A non-bold version x i denotes the i -th feature of vector x ,and the capitalized version X denotes a random variable.
Revised: A non-bold version x i denotes the i -th feature of vector x , and the capitalized version X denotes a random variable.

Original: A classiﬁer is the mapping from inputspace to the logits f : R d ! R k parameterized by ✓ , where f j ( x ) is the j -th entry of f ( x ) , and theclassiﬁcation result of input x is given by the index of maximum logit ˆ y = arg max 1  j  k ( f j ( x )) .
Revised: A classifier is the mapping from input space to the logits f : R d → R k parameterized by θ , where f j ( x ) is the j -th entry of f ( x ) , and the classification result of input x is given by the index of maximum logit ˆ y = arg max 1 ≤ j ≤ k ( f j ( x )) .

Original: 2.1 Attribution methods
Revised: 2.1 Attribution methods

Original: Model attribution, denoted by g ( x ) , studies the importance that the input features contribute towards the ﬁnal result.
Revised: Model attribution, denoted by g ( x ) , studies the importance that the input features contribute towards the final result.

Original: The mostly used attribution methods include perturbation-based [35, 37]and backpropagation-based methods [1], including gradient-based attribution methods [25, 26].
Revised: The mostly used attribution methods include perturbation-based [35, 37] and backpropagation-based methods [1], including gradient-based attribution methods [25, 26].

Original: Inparticular, integrated gradients (IG) [29], one of the gradient-based methods, computes the attributionusing the line integral of gradients from a baseline image a to the input image x weighted by theirdifference, i.e. ,
Revised: In particular, integrated gradients (IG) [29], one of the gradient-based methods, computes the attribution using the line integral of gradients from a baseline image a to the input image x weighted by their difference, i.e. ,

Original: [Equation]
Revised: [Equation]

Original: IG satisﬁes the axiom of completeness which guarantees P i g ( x ) i =
Revised: IG satisfies the axiom of completeness which guarantees (cid:80) i g ( x ) i =

Original: f y ( x )  f y ( a ) .
Revised: f y ( x ) − f y ( a ) .

Original: We omit thebaseline image a in the later parts of this paper, and it is chosen to be a black image, i.e. , 0 , if notspeciﬁcally stated
Revised: We omit the baseline image a in the later parts of this paper, and it is chosen to be a black image, i.e. , 0 , if not specifically stated.

Original: .2.2 Attribution robustness
Revised: 2.2 Attribution robustness

Original: Recent studies reveal the vulnerability of neural networks that, similar to adversarial examples,imperceptible perturbations added to natural images would have signiﬁcantly different attributionwhile their classiﬁcation results remain unchanged [8].
Revised: Recent studies reveal the vulnerability of neural networks that, similar to adversarial examples, imperceptible perturbations added to natural images would have significantly different attribution while their classification results remain unchanged [8].

Original: Heo et al.
Revised: Heo et al.

Original: [12] manipulates the modelparameters instead of input images to disturb attributions and remains high accuracy on classiﬁcations.
Revised: [12] manipulates the model parameters instead of input images to disturb attributions and remains high accuracy on classifications.

Original: Dombrowski et al.
Revised: Dombrowski et al.

Original: [6] makes targeted attack that changes original attributions to any predeﬁnedattributions and gives a theoretical explanation to this phenomenon.
Revised: [6] makes targeted attack that changes original attributions to any predefined attributions and gives a theoretical explanation to this phenomenon.

Original: Engstrom et al.
Revised: Engstrom et al.

Original: [7] points out that robust optimization enhances model representations and interpretability.
Revised: [7] points out that robust optimization enhances model representations and interpretability.

Original: Chen et al.
Revised: Chen et al.

Original: [4] and Boopathy et al.
Revised: [4] and Boopathy et al.

Original: [2] use ` 1 -norm to constrain the distance betweenattributions of natural and perturbed images.
Revised: [2] use ℓ 1 -norm to constrain the distance between attributions of natural and perturbed images.

Original: Sarkar et al.
Revised: Sarkar et al.

Original: [24] proposes a contrastive regularizerthat emphasizes a skewed distribution on true class attribution while a uniform one on negativeclass attribution.
Revised: [24] proposes a contrastive regularizer that emphasizes a skewed distribution on true class attribution while a uniform one on negative class attribution.

Original: Ivankay et al.
Revised: Ivankay et al.

Original: [13] directly optimizes Pearson’s correlation and Singh et al.
Revised: [13] directly optimizes Pearson’s correlation and Singh et al.

Original: [28]uses a triplet loss to minimize the upper bound of the attribution distortion.
Revised: [28] uses a triplet loss to minimize the upper bound of the attribution distortion.

Original: Although the previoustechniques present promising results, none of them exploits the angle between attributions explicitlyfor attribution protection.
Revised: Although the previous techniques present promising results, none of them exploits the angle between attributions explicitly for attribution protection.

Original: The method introduced in this work leverages the relationship between
Revised: The method introduced in this work leverages the relationship between

Original: Kendall’s rank correlation and cosine similarity with a theoretical support for attribution robustness.
Revised: Kendall’s rank correlation and cosine similarity with a theoretical support for attribution robustness.

Original:  Kendall’s rank correlation and cosine similarity
Revised:  Kendall’s rank correlation and cosine similarity

Original: Kendall’s rank correlation, often denoted by ⌧ , is a measurement of the ordinal relationship betweentwo quantities, where two quantities have higher ⌧ when they have more concordant pairs.
Revised: Kendall’s rank correlation, often denoted by τ , is a measurement of the ordinal relationship between two quantities, where two quantities have higher τ when they have more concordant pairs.

Original: Formally,
Revised: Formally,

Original: Kendall’s rank correlation between two vectors x and x 0 can be explicitly computed by
Revised: Kendall’s rank correlation between two vectors x and x ′ can be explicitly computed by

Original: [Equation]
Revised: [Equation]

Original: where d is the dimension of the vectors.
Revised: where d is the dimension of the vectors.

Original: As Kendall’s rank correlation is an important metric toquantify the differences between perturbed and natural attributions, we begin by presenting therelationship between Kendall’s rank correlation and cosine similarity.
Revised: As Kendall’s rank correlation is an important metric to quantify the differences between perturbed and natural attributions, we begin by presenting the relationship between Kendall’s rank correlation and cosine similarity.

Original: It should be highlighted that all the previous attribution robustness studies [2, 4, 8, 13, 28, 33] use Kendall’s rank correlation as a keyperformance index to evaluate the effectiveness of their methods.
Revised: It should be highlighted that all the previous attribution robustness studies [2, 4, 8, 13, 28, 33] use Kendall’s rank correlation as a key performance index to evaluate the effectiveness of their methods.

Original: To enhance attribution robustness, it is equivalent to force the perturbed attribution to have a higher
Revised: To enhance attribution robustness, it is equivalent to force the perturbed attribution to have a higher

Original: Kendall’s rank correlation with the original one.
Revised: Kendall’s rank correlation with the original one.

Original: However, as Kendall’s rank correlation is notdifferentiable, it is difﬁcult to directly optimize it.
Revised: However, as Kendall’s rank correlation is not differentiable, it is difficult to directly optimize it.

Original: It is necessary to ﬁnd an alternative that eitherapproximates to or has a consistent behavior with Kendall’s rank correlation.
Revised: It is necessary to find an alternative that either approximates to or has a consistent behavior with Kendall’s rank correlation.

Original: The following theoremstates that cosine similarity is an appropriate replacement as a regularization term since it is positivelyrelated to the Kendall’s rank correlation (Fig. 2a).
Revised: The following theorem states that cosine similarity is an appropriate replacement as a regularization term since it is positively related to the Kendall’s rank correlation (Fig. 2a).

Original: Theorem 1.
Revised: Theorem 1.

Original: Given a random vector Y =
Revised: Given a random vector Y =

Original: ( y 1 , y 2 , . .
Revised: ( y 1 , y 2 , . .

Original: . , y d ) where y i follows a positive-valueddistribution, and two arbitrary vectors with the same dimension, X, X 0 2 R d that x i , x 0 i  0 , assumethat there exists a sequence S 
Revised: . , y d ) where y i follows a positive-valued distribution, and two arbitrary vectors with the same dimension, X, X ′ ∈ R d that x i , x ′ i ≥ 0 , assume that there exists a sequence S =

Original: = { X i } Ni =1 with X =
Revised: { X i } Ni =1 with X =

Original: X 0 and X [-
-]
Revised: X 0 and X [-
-]′ =

Original: = X N , where the vectors satisfy the118 condition that cos( X i , Y )  cos( X i +1 , Y ) , and each X i +1 can be induced from its previous vector
Revised: X N , where the vectors satisfy the condition that cos( X i , Y ) ≥ cos( X i +1 , Y ) , and each X i +1 can be induced from its previous vector

Original: X i through one of the following two operations,
Revised: X i through one of the following two operations,

Original: (i) arbitrarily exchanging two entries of X i (ii) multiplying one entry in X i by ↵ 2 (0 , 1]
Revised: (i) arbitrarily exchanging two entries of X i (ii) multiplying one entry in X i by α ∈ (0 , 1]

Original: Then Kendall’s rank correlations of Y with X and X 0 have the property that E [ ⌧ ( X, Y )]
Revised: Then Kendall’s rank correlations of Y with X and X ′ have the property that E [ τ ( X, Y )]

Original:  E [ ⌧ ( X 0 , Y )] , where the expectation is taken over Y satisfying cos( X i , Y )  cos( X i + 1 , Y ) .
Revised: ≥ E [ τ ( X ′ , Y )] , where the expectation is taken over Y satisfying cos( X i , Y ) ≥ cos( X i + 1 , Y ) .

Original: The full proof and discussions can be found in Appendix A.
Revised: The full proof and discussions can be found in Appendix A.

Original: In the scenario of attribution robustness,under the above assumption, we denote Y as the natural attribution g ( x ) , and X and X 0 as twoperturbed attributions g ( x 0 ) and g ( x 00 ) .
Revised: In the scenario of attribution robustness, under the above assumption, we denote Y as the natural attribution g ( x ) , and X and X ′ as two perturbed attributions g ( x ′ ) and g ( x ′′ ) .

Original: If the perturbed attribution has a greater cosine similaritywith natural attribution, then their expected Kendall’s rank correlation is also greater.
Revised: If the perturbed attribution has a greater cosine similarity with natural attribution, then their expected Kendall’s rank correlation is also greater.

Original: Explicitlyspeaking, if cos( g ( x 0 ) , g ( x ))  cos( g ( x 00 ) , g ( x )) , then E [ ⌧ ( g ( x 0 ) , g ( x ))]
Revised: Explicitly speaking, if cos( g ( x ′ ) , g ( x )) ≥ cos( g ( x ′′ ) , g ( x )) , then E [ τ ( g ( x ′ ) , g ( x ))]

Original:  E [ ⌧ ( g ( x 00 ) , g ( x ))] .
Revised: ≥ E [ τ ( g ( x ′′ ) , g ( x ))] .

Original: This theorem provides a theoretical foundation that supports using cosine similarity for attributionprotection because it directly links to Kendall’s rank correlation.
Revised: This theorem provides a theoretical foundation that supports using cosine similarity for attribution protection because it directly links to Kendall’s rank correlation.

Original:  Characterization of geometric perspective on attributions
Revised:  Characterization of geometric perspective on attributions

Original: In the last section, we have indicated the relationship between cosine similarity and Kendall’s rankcorrelation.
Revised: In the last section, we have indicated the relationship between cosine similarity and Kendall’s rank correlation.

Original: In this section, we use this relationship to explain (i) the drawbacks of attributionprotections based on ` p -norm, e.g. , min ✓ k g ( x )  g (˜ x ) k p in Chen et al.
Revised: In this section, we use this relationship to explain (i) the drawbacks of attribution protections based on ℓ p -norm, e.g. , min θ ∥ g ( x ) − g (˜ x ) ∥ p in Chen et al.

Original: [4], where x is a naturalsample and ˜ x is a perturbed sample; (ii) the inappropriateness of standard adversarial training forattribution protection and (iii) the limitation of the cosine similarity, i.e. , min ✓ (1  cos( g ( x ) , g (˜ x )))for standard adversarial protection.
Revised: [4], where x is a natural sample and ˜ x is a perturbed sample; (ii) the inappropriateness of standard adversarial training for attribution protection and (iii) the limitation of the cosine similarity, i.e. , min θ (1 − cos( g ( x ) , g (˜ x ))) for standard adversarial protection.

Original: In this discussion, g ( x ) and g (˜ x ) are considered as vectors, andas stated in Theorem 1, a smaller angle between them implies higher attribution robustness.
Revised: In this discussion, g ( x ) and g (˜ x ) are considered as vectors, and as stated in Theorem 1, a smaller angle between them implies higher attribution robustness.

Original: Theattribution method g is assumed to fulﬁll the axiom of completeness 1 , i.e. , f y ( x )  f y ( a ) =
Revised: The attribution method g is assumed to fulfill the axiom of completeness 1 , i.e. , f y ( x ) − f y ( a ) =

Original: P i g ( x ) i .
Revised: (cid:80) i g ( x ) i .

Original: If g ( x ) i  0 for all i , k g ( x ) k 2 = pP i g ( x ) 2 i 
Revised: If g ( x ) i ≥ 0 for all i , ∥ g ( x ) ∥ 2 = (cid:112)(cid:80) i g ( x ) 2 i ≤ (cid:80)

Original: P i g ( x ) i =
Revised: i g ( x ) i =

Original: f y ( x ) .
Revised: f y ( x ) .

Original: In other words, k g ( x ) k 2 is thelower bound of f y ( x ) and larger k g ( x ) k 2 implies higher classiﬁcation accuracy.
Revised: In other words, ∥ g ( x ) ∥ 2 is the lower bound of f y ( x ) and larger ∥ g ( x ) ∥ 2 implies higher classification accuracy.

Original: Thus, minimizingthe angle between g ( x ) and g (˜ x ) and maximizing their magnitudes would respectively enhance theirattributional and adversarial robustness.
Revised: Thus, minimizing the angle between g ( x ) and g (˜ x ) and maximizing their magnitudes would respectively enhance their attributional and adversarial robustness.

Original: Fig. 2b shows a two-dimensional projection for the ease of illustration, where each 2D point representsan attribution of an input.
Revised: shows a two-dimensional projection for the ease of illustration, where each 2D point represents an attribution of an input.

Original: Higher-dimensional cases can be extended in a similar manner.
Revised: Higher-dimensional cases can be extended in a similar manner.

Original: In
Revised: In

Original: Fig.
Revised: Fig.

Original: 2b, g ( x ) is the original attribution of x and the others are its perturbed counterparts.
Revised: 2b, g ( x ) is the original attribution of x and the others are its perturbed counterparts.

Original: Given twoattributions, g ( x 0 ) and g ( x 00 ) , where k g ( x )  g ( x 0 ) 
Revised: Given two attributions, g ( x ′ ) and g ( x ′′ ) , where ∥ g ( x ) − g ( x ′ ) ∥

Original: k
Revised: =

Original: = k g ( x )  g ( x 00 ) k but cos( g ( x ) , g ( x 0 )) >148 cos( g ( x ) , g ( x 00 )) , according to Theorem 1, ⌧ ( g ( x ) , g ( x 0 )) is likely larger than ⌧ ( g ( x ) , g ( x 00 )) ,implying that the attribution of x 0 is likely closer to the attribution of x than that of x 00 .
Revised: ∥ g ( x ) − g ( x ′′ ) ∥ but cos( g ( x ) , g ( x ′ )) > cos( g ( x ) , g ( x ′′ )) , according to Theorem 1, τ ( g ( x ) , g ( x ′ )) is likely larger than τ ( g ( x ) , g ( x ′′ )) , implying that the attribution of x ′ is likely closer to the attribution of x than that of x ′′ .

Original: It explainsthe results in Fig.
Revised: It explains the results in Fig.

Original: 1 and point (i), i.e. , drawbacks of attribution protection based on ` p -norm.
Revised: 1 and point (i), i.e. , drawbacks of attribution protection based on ℓ p -norm.

Original: The standard adversarial training maximizes f y (˜ x ) , where ˜ x is an adversarial example.
Revised: The standard adversarial training maximizes f y (˜ x ) , where ˜ x is an adversarial example.

Original: In Fig.
Revised: In Fig.

Original: 2b,x 000 has large k g ( x 000 ) k 2 , implying that f y ( x 000 ) is also large.
Revised: 2b, x ′′′ has large ∥ g ( x ′′′ ) ∥ 2 , implying that f y ( x ′′′ ) is also large.

Original: In other words, the classiﬁcation labelof x 000 is well protected.
Revised: In other words, the classification label of x ′′′ is well protected.

Original: However, standard adversarial training does not explicitly minimize theangle between g ( x ) and g ( x 000 ) .
Revised: However, standard adversarial training does not explicitly minimize the angle between g ( x ) and g ( x ′′′ ) .

Original: It implies that ⌧ ( g ( x ) , g ( x 000 )) can be small and x 000 can attack theattribution successfully.
Revised: It implies that τ ( g ( x ) , g ( x ′′′ )) can be small and x ′′′ can attack the attribution successfully.

Original: It should be mentioned that adversarial training does improve attribution
Revised: It should be mentioned that adversarial training does improve attribution

Original: [Figure]
Revised: [Figure]

Original: g ( x 0 ) g ( x 00 ) g ⇤ ( ˜ x ) g ( x 000 ) g ( x ) g (˜ x ) gg(b) robustness because it smooths the decision surface [33] although it is not the most ideal one.
Revised: robustness because it smooths the decision surface [33] although it is not the most ideal one.

Original: Itexplains the point (ii).
Revised: It explains the point (ii).

Original: Point (iii) can be explained similarly.
Revised: Point (iii) can be explained similarly.

Original: Since the cosine similarity, or min ✓ (1  cos( g ( x ) , g (˜ x )) , doesnot necessarily enlarge the magnitude of g (˜ x ) , it cannot improve network robustness against standardadversarial attack.
Revised: Since the cosine similarity, or min θ (1 − cos( g ( x ) , g (˜ x )) , does not necessarily enlarge the magnitude of g (˜ x ) , it cannot improve network robustness against standard adversarial attack.

Original: Fig.
Revised: Fig.

Original: 2b shows two networks (the dashed circle and solid circle).
Revised: 2b shows two networks (the dashed circle and solid circle).

Original: cos( g ( x ) , g (˜ x )) =
Revised: cos( g ( x ) , g (˜ x )) =

Original: cos( g ( x ) , g ⇤ (˜ x )) , which implies that the two networks perform the same on attribution protection,but k g ⇤ (˜ x ) k > k g (˜ x ) k , meaning g ⇤ is more robust against the standard adversarial attack from ˜ x ,while g is more vulnerable.
Revised: cos( g ( x ) , g ∗ (˜ x )) , which implies that the two networks perform the same on attribution protection, but ∥ g ∗ (˜ x ) ∥ > ∥ g (˜ x ) ∥ , meaning g ∗ is more robust against the standard adversarial attack from ˜ x , while g is more vulnerable.

Original: To protect against both attribution attack and adversarial attack, in the following section, the proposed
Revised: To protect against both attribution attack and adversarial attack, in the following section, the proposed

Original: IGR is optimized with adversarial loss together, where the former minimizes the angle betweenattribution vectors to perform attribution protection, and the latter maximizes their magnitude to offerstandard adversarial protection.
Revised: IGR is optimized with adversarial loss together, where the former minimizes the angle between attribution vectors to perform attribution protection, and the latter maximizes their magnitude to offer standard adversarial protection.

Original:  Integrated gradients regularizer (IGR)
Revised:  Integrated gradients regularizer (IGR)

Original: Based on the above analysis, in this section, we introduce the integrated gradients regularizer (IGR),which regularizes the cosine similarity between natural and perturbed attributions.
Revised: Based on the above analysis, in this section, we introduce the integrated gradients regularizer (IGR), which regularizes the cosine similarity between natural and perturbed attributions.

Original: 5.1 IGR robust training objective
Revised: 5.1 IGR robust training objective

Original: Since Kendall’s rank correlation and cosine similarity are positively related, we suggest to improveattribution robustness, especially integrated gradients, by maximizing the cosine similarity betweennatural and perturbed attributions, or equivalently, minimizing 1  cos( IG ( x ) , IG (˜ x )) .
Revised: Since Kendall’s rank correlation and cosine similarity are positively related, we suggest to improve attribution robustness, especially integrated gradients, by maximizing the cosine similarity between natural and perturbed attributions, or equivalently, minimizing 1 − cos( IG ( x ) , IG (˜ x )) .

Original: Therefore,we propose the following training objective function incorporating the IGR
Revised: Therefore, we propose the following training objective function incorporating the IGR

Original: [Equation]
Revised: [Equation]

Original: (3) where L is a standard loss function used in robust training, and  is a hyper-parameter.
Revised: where L is a standard loss function used in robust training, and λ is a hyper-parameter.

Original: We will latershow in Section 6 that L can be chosen from existing loss function in robust training, and our IGRwill further improve the robustness upon baseline methods.
Revised: We will later show in Section 6 that L can be chosen from existing loss function in robust training, and our IGR will further improve the robustness upon baseline methods.

Original: In practice, the integral inside IG deﬁnition can be numerically computed by its Riemann sum, and
Revised: In practice, the integral inside IG definition can be numerically computed by its Riemann sum, and

Original: IG is approximated by
Revised: IG is approximated by

Original: [Equation]
Revised: [Equation]

Original: Similar to adversarial training, optimizing the above objective function in Eq.
Revised: Similar to adversarial training, optimizing the above objective function in Eq.

Original: (3) requires perturbedexample ˜ x that maximally diverts its IG from the original counterpart.
Revised: (3) requires perturbed example ˜ x that maximally diverts its IG from the original counterpart.

Original: Such examples can be foundby maximizing the proposed regularizer within its ` p -ball with radius " , i.e. ,
Revised: Such examples can be found by maximizing the proposed regularizer within its ℓ p -ball with radius ε , i.e. ,

Original: [Equation]
Revised: [Equation]

Original: It is noticed that computing the adversarial loss L (˜ x , y ; ✓ ) itself relies on ˜ x , which can be obtainedfrom adversarial attacks,i.e. , ˜ x = arg max
Revised: It is noticed that computing the adversarial loss L (˜ x , y ; θ ) itself relies on ˜ x , which can be obtained from adversarial attacks,

Original:  ˜ x L (˜ x , y ; ✓ ) .
Revised: i.e. , ˜ x = arg max ˜ x L (˜ x , y ; θ ) .

Original: Thus, here we reuse these ˜ x in IGR to avoidrepeatedly using gradient descent methods to ﬁnd the optimum in Eq.
Revised: Thus, here we reuse these ˜ x in IGR to avoid repeatedly using gradient descent methods to find the optimum in Eq.

Original: (5) and speed up the training.
Revised: (5) and speed up the training.

Original: For example, if L is the standard adversarial training loss function, we directly use the examplesgenerated from PGD attack.
Revised: For example, if L is the standard adversarial training loss function, we directly use the examples generated from PGD attack.

Original: The computation cost of IGR is similar to previous proposed methods.
Revised: The computation cost of IGR is similar to previous proposed methods.

Original: The use of Pearson’s correlation regularizer in Ivankay et al.
Revised: The use of Pearson’s correlation regularizer in Ivankay et al.

Original: [13] as the replacement of Kendall’srank correlation is the closest method to ours.
Revised: [13] as the replacement of Kendall’s rank correlation is the closest method to ours.

Original: Ivankay et al.
Revised: Ivankay et al.

Original: [13] suggests that Pearson’s correlationregularizer keeps the ranking of feature constant.
Revised: [13] suggests that Pearson’s correlation regularizer keeps the ranking of feature constant.

Original: However, the statement is not supported by anytheoretical justiﬁcation while we give a theorem that shows cosine similarity is positively related to
Revised: However, the statement is not supported by any theoretical justification while we give a theorem that shows cosine similarity is positively related to

Original: Kendall’s rank correlation.
Revised: Kendall’s rank correlation.

Original: Besides, the Pearson’s correlation is an unstable metric for attributionswith small variances.
Revised: Besides, the Pearson’s correlation is an unstable metric for attributions with small variances.

Original: For a ﬁxed vector, two slightly different inputs  can have drastically different
Revised: For a fixed vector, two slightly different inputs δ can have drastically different

Original: Pearson’s correlation, which easily ﬂuctuate from  1 to 1 .
Revised: Pearson’s correlation, which easily fluctuate from − 1 to 1 .

Original: The detailed discussion can be found in
Revised: The detailed discussion can be found in

Original: Appendix B.
Revised: Appendix B.

Original: 5.2 IGR induces more consistent neuron activation states
Revised: 5.2 IGR induces more consistent neuron activation states

Original: An interesting discovery about IGR is related to neuron activations.
Revised: An interesting discovery about IGR is related to neuron activations.

Original: We found that the activationfunctions in ReLU networks trained with IGR are more often with the same neuron activation statesfor natural sample and corresponding perturbed sample.
Revised: We found that the activation functions in ReLU networks trained with IGR are more often with the same neuron activation states for natural sample and corresponding perturbed sample.

Original: For deep networks with ReLU activations, ifthe pre-ReLU value is positive (negative) for natural sample, the probability of pre-ReLU value beingpositive (negative) for corresponding perturbed sample would increase when trained with IGR.
Revised: For deep networks with ReLU activations, if the pre-ReLU value is positive (negative) for natural sample, the probability of pre-ReLU value being positive (negative) for corresponding perturbed sample would increase when trained with IGR.

Original: Toanalyze this phenomenon, a single-layer neural network with ReLU activation is studied.
Revised: To analyze this phenomenon, a single-layer neural network with ReLU activation is studied.

Original: The resultsfrom this single-layer neural network can be extended to deep networks by stacking multiple layers.
Revised: The results from this single-layer neural network can be extended to deep networks by stacking multiple layers.

Original: Recall that x 2 R d is an input image, and the network function f is parameterized by ( W , u , c ) [-
-] R d ⇥ m ⇥ R m ⇥ R , where W i is the column vector of W , w ij is the ij -th entry of matrix W and u i isthe i -th entry of vector u , i.e. , f ( x ) = u > ReLU ( W > x ) + c .
Revised: Recall that x ∈ R d is an input image, and the network function f is parameterized by ( W , u , c ) [-
-]∈ R d × m × R m × R , where W i is the column vector of W , w ij is the ij -th entry of matrix W and u i is the i -th entry of vector u , i.e. , f ( x ) = u ⊤ ReLU ( W ⊤ x ) + c .

Original: Then, the following proposition holds.
Revised: Then, the following proposition holds.

Original: Proposition 1.
Revised: Proposition 1.

Original: Given a single-layer neural network with ReLU activation, and with the aboveparameterization, if, for all i , W i and u i are all independent and identically distributed randomvariables following Gaussian distributions, i.e., W i i.i.d.
Revised: Given a single-layer neural network with ReLU activation, and with the above parameterization, if, for all i , W i and u i are all independent and identically distributed random variables following Gaussian distributions, i.e., W i i.i.d.

Original: ⇠ N (0 ,  2 w I d ) and u i i.i.d.
Revised: ∼ N (0 , σ 2 w I d ) and u i i.i.d.

Original: ⇠ N (0 ,  2 u ) , andtwo input images that each has small variance, x and ˜ x , then
Revised: ∼ N (0 , σ 2 u ) , and two input images that each has small variance, x and ˜ x , then

Original: [Equation]
Revised: [Equation]

Original: The proof can be found in Appendix A.2.
Revised: The proof can be found in Appendix A.2.

Original: The right-hand side of Eq.
Revised: The right-hand side of Eq.

Original: (6) is called the activationconsistency of natural and perturbed samples.
Revised: (6) is called the activation consistency of natural and perturbed samples.

Original: For the sake of convenience, let the event W > x > 0 be
Revised: For the sake of convenience, let the event W ⊤ x > 0 be

Original: A and W > ˜ x > 0 be B .
Revised: A and W ⊤ ˜ x > 0 be B .

Original: The right-hand side of Eq.
Revised: The right-hand side of Eq.

Original: (6) can be rewritten as P ( A \ B ) / p P ( A ) P ( B ) .
Revised: (6) can be rewritten as P ( A ∩ B ) / (cid:112) P ( A ) P ( B ) .

Original: Since P ( A \ B ) has the upper bound that P ( A \ B )  min { P ( A ) , P ( B ) } , it is obvious that
Revised: Since P ( A ∩ B ) has the upper bound that P ( A ∩ B ) ≤ min { P ( A ) , P ( B ) } , it is obvious that

Original: [Equation]
Revised: [Equation]

Original: and the equality holds when event A happens with the same probability as event B , i.e. , P ( A )=
Revised: and the equality holds when event A happens with the same probability as event B , i.e. , P ( A )

Original:  P ( B ) = P ( A \ B ) ; alternatively, the equality can hold when P ( A c )
Revised: = P ( B ) = P ( A ∩ B ) ; alternatively, the equality can hold when P ( A c )

Original: =
Revised: =

Original: P ( B c ) =
Revised: P ( B c ) =

Original: P ( A c [ B c ) .
Revised: P ( A c ∪ B c ) .

Original: Inother words, maximizing Eq.
Revised: In other words, maximizing Eq.

Original: (6) encourages that the network activates the same set of neurons for xand ˜ x , or deactivates the same set of neurons for x and ˜ x .
Revised: (6) encourages that the network activates the same set of neurons for x and ˜ x , or deactivates the same set of neurons for x and ˜ x .

Original:  Experiments and results 
Revised:  Experiments and results 6.1 Experimental configurations

Original: 6.1 Experimental conﬁgurations

Original: We evaluate the performance of IGR on different datasets, including MNIST [19], Fashion-MNIST[34] and CIFAR-10 [17].
Revised: We evaluate the performance of IGR on different datasets, including MNIST [19], Fashion-MNIST [34] and CIFAR-10 [17].

Original: For MNIST and Fashion-MNIST, we use a network consisting of four
Revised: For MNIST and Fashion-MNIST, we use a network consisting of four

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: convolutional layers followed by three fully connected layers.
Revised: convolutional layers followed by three fully connected layers.

Original: The model is trained by Adam
Revised: The model is trained by Adam

Original: Optimizer [16] with learning rate 10  4 for 90 epochs.
Revised: Optimizer [16] with learning rate 10 − 4 for 90 epochs.

Original: For CIFAR-10, we train a ResNet-18 [11] for120 epochs using SGD [30] with initial learning rate 0.1, momentum 0.9 and weight decay 5 ⇥ 10  4 .
Revised: For CIFAR-10, we train a ResNet-18 [11] for 120 epochs using SGD [30] with initial learning rate 0.1, momentum 0.9 and weight decay 5 × 10 − 4 .

Original: The learning rate decays by 0 .
Revised: The learning rate decays by 0 .

Original: 1 at the 75th and 90th epoch.
Revised: 1 at the 75th and 90th epoch.

Original: All the experiments are run on NVIDIA
Revised: All the experiments are run on NVIDIA

Original: GeForce RTX 3090 2 .
Revised: GeForce RTX 3090.

Original: As discussed in Section 5, IGR is applied with state-of-the-art adversarial training methods: standardadversarial training (AT)[21], TRADES [36] and MART [32].
Revised: As discussed in Section 5, IGR is applied with state-of-the-art adversarial training methods: standard adversarial training (AT)[21], TRADES [36] and MART [32].

Original: L at , L trades and L mart in Table 1 arethe objective functions of these methods, and are regarded as L in Eq.
Revised: L at , L trades and L mart in Table 1 are the objective functions of these methods, and are regarded as L in Eq.

Original: In Table 1, CE denotes thecross-entropy loss and KL denotes the KL-divergence.
Revised: In Table 1, CE denotes the cross-entropy loss and KL denotes the KL-divergence.

Original: BCE is a boosted cross-entropy (see details in
Revised: BCE is a boosted cross-entropy (see details in

Original: Wang et al.
Revised: Wang et al.

Original: Note that both AT and MART generate adversarial examples by maximizing the
Revised: Note that both AT and MART generate adversarial examples by maximizing the

Original: CE loss, while TRADES maximizes the KL-divergence regularizer.
Revised: CE loss, while TRADES maximizes the KL-divergence regularizer.

Original: Following the baseline methods,we directly leverage the perturbed examples generated by their original techniques to compute theintegrated gradients, as well as the IGR, instead of generating our own ones using Eq.
Revised: Following the baseline methods, we directly leverage the perturbed examples generated by their original techniques to compute the integrated gradients, as well as the IGR, instead of generating our own ones using Eq.

Original: Moreover,to ensure fair comparisons, we keep the hyper-parameters the same for models with or without IGR.
Revised: Moreover, to ensure fair comparisons, we keep the hyper-parameters the same for models with or without IGR.

Original: 6.2 Evaluation on attribution robustness
Revised: 6.2 Evaluation on attribution robustness

Original: To evaluate our method under attribution attack, the iterative feature importance attacks (IFIA) usingtop-k intersection as dissimilarity function ( top-k ) [8] is adapted.
Revised: To evaluate our method under attribution attack, the iterative feature importance attacks (IFIA) using top-k intersection as dissimilarity function ( top-k ) [8] is adapted.

Original: IFIA generates perturbations byiteratively maximizing the dissimilarity function that measures the changes between attributions ofimages, while keeps the classiﬁcation results unchanged.
Revised: IFIA generates perturbations by iteratively maximizing the dissimilarity function that measures the changes between attributions of images, while keeps the classification results unchanged.

Original: In this experiment, we perform 200-step
Revised: In this experiment, we perform 200-step

Original: IFIA as in Chen et al.
Revised: IFIA as in Chen et al.

Original: For MNIST and Fashion-MNIST, we choose k = 100 and the perturbationsize " = 0 .
Revised: For MNIST and Fashion-MNIST, we choose k = 100 and the perturbation size ε = 0 .

Original: For CIFAR-10, k
Revised: For CIFAR-10, k

Original: = 1000 and " = 8 / 255 .
Revised: = 1000 and ε = 8 / 255 .

Original: Two metrics are chosen to evaluate theperformance under attribution attack as in Chen et al.
Revised: Two metrics are chosen to evaluate the performance under attribution attack as in Chen et al.

Original: [4]: top-k intersection and Kendall’s rankcorrelation, where top-k intersection counts the proportion of pixels that coincide in the k most 
Revised: [4]: top-k intersection and Kendall’s rank correlation, where top-k intersection counts the proportion of pixels that coincide in the k most important features.

Revised: Each sample is attacked five times and the mean metrics are reported.

Revised: For both metrics, a higher number indicates that the model is more robust under the attack.

Original: More visualization results are given in Appendix C
Revised: More visualization results are given in Appendix C

Original: [Table]
Revised: [Table]

Original: important features.

Original: Each sample is attacked ﬁve times and the mean metrics are reported.

Original: For bothmetrics, a higher number indicates that the model is more robust under the attack.

Original: To compare, attribution protection methods, IG-NORM and IG-SUM-NORM by Chen et al.
Revised: To compare, attribution protection methods, IG-NORM and IG-SUM-NORM by Chen et al.

Original: Smooth Surface Regularization (SSR) [33], Attributional Robustness Training (ART)
Revised: Smooth Surface Regularization (SSR) [33], Attributional Robustness Training (ART)

Original: [28] and Adversarial Attributional Training with robust training loss ( AdvAAT ), are implemented and evaluated onall the datasets.
Revised: [28] and Adversarial Attributional Training with robust training loss ( AdvAAT ), are implemented and evaluated on all the datasets.

Original: A cross-entropy loss trained natural model ( standard ) is also included as a baseline.
Revised: A cross-entropy loss trained natural model ( standard ) is also included as a baseline.

Original: The details of these baseline methods are brieﬂy introduced in Appendix C.2.
Revised: The details of these baseline methods are briefly introduced in Appendix C.2.

Original: From the results in Table 2, we observed the following phenomenons.
Revised: From the results in Table 2, we observed the following phenomenons.

Original: (i)
Revised: (i)

Original: Compared with baselinemethods (AT, TRADES and MART), models trained with IGR outperform their correspondingcounterparts in terms of both top-k intersection and Kendall’s rank correlation.
Revised: Compared with baseline methods (AT, TRADES and MART), models trained with IGR outperform their corresponding counterparts in terms of both top-k intersection and Kendall’s rank correlation.

Original: (ii) Adversarial defensemethods themselves also help the attribution protection, especially improve on Fashion-MNIST and
Revised: (ii) Adversarial defense methods themselves also help the attribution protection, especially improve on Fashion-MNIST and

Original: CIFAR-10 comparing with the standard cross-entropy training.
Revised: CIFAR-10 comparing with the standard cross-entropy training.

Original: (iii)
Revised: (iii)

Original: Compared with other attributionprotection methods, standard adversarial defense methods, including AT, TRADES and MART, areweaker in attribution robustness; however, they achieve comparable or even stronger attributionprotections when training with IGR.
Revised: Compared with other attribution protection methods, standard adversarial defense methods, including AT, TRADES and MART, are weaker in attribution robustness; however, they achieve comparable or even stronger attribution protections when training with IGR.

Original: (iv) TRADES itself has the best attribution protections amongmodels without IGR, and IGR provides the most signiﬁcant boost on TRADES.
Revised: (iv) TRADES itself has the best attribution protections among models without IGR, and IGR provides the most significant boost on TRADES.

Original: (v) Since AdvAATuses Pearson’s correlation as a regularizer, which is close to IGR, and TRADES+IGR outperformsthe other baselines, we apply Pearson’s correlation on TRADES, i.e. , TRADES +AdvAAT, in the
Revised: (v) Since AdvAAT uses Pearson’s correlation as a regularizer, which is close to IGR, and TRADES+IGR outperforms the other baselines, we apply Pearson’s correlation on TRADES, i.e. , TRADES +AdvAAT, in the

Original: Table 2 for the comparison.
Revised: Table 2 for the comparison.

Original: Table 2 shows clearly that TRADES+AdvAAT does not always improveover TRADES and is consistently outperformed by TRADES+IGR.
Revised: Table 2 shows clearly that TRADES+AdvAAT does not always improve over TRADES and is consistently outperformed by TRADES+IGR.

Original: A visualization of attribution robustness is also presented in Figure 3.
Revised: A visualization of attribution robustness is also presented in Figure 3.

Original: It is observed that the attributionof the baseline model is easily corrupted.
Revised: It is observed that the attribution of the baseline model is easily corrupted.

Original: For model trained with IGR, although the magnitudes of
Revised: For model trained with IGR, although the magnitudes of

Original: IG are different from IG of the original images, the directions remain nearly identical, which is alsoaligned with human visual perceptions.
Revised: IG are different from IG of the original images, the directions remain nearly identical, which is also aligned with human visual perceptions.

Original: 6.3 Evaluation on white-box adversarial robustness
Revised: 6.3 Evaluation on white-box adversarial robustness

Original: To evaluate the performance of IGR on adversarial robustness, the trained defense models areevaluated under white-box adversarial attacks, including FGSM [9], PGD [21] and CW 1 [3], where[Figure] all the attacks have the information of the entire models, including architectures and parameters.
Revised: To evaluate the performance of IGR on adversarial robustness, the trained defense models are evaluated under white-box adversarial attacks, including FGSM [9], PGD [21] and CW ∞ [3], where all the attacks have the information of the entire models, including architectures and parameters.

Original: Thenumbers are reported in both natural accuracy (Natural) and adversarial accuracies under FGSM,
Revised: The numbers are reported in both natural accuracy (Natural) and adversarial accuracies under FGSM,

Original: PGD with 20 steps (PGD20), and CW 1 attacks.
Revised: PGD with 20 steps (PGD20), and CW ∞ attacks.

Original: The maximum allowable perturbations are chosento be " = 0 .
Revised: The maximum allowable perturbations are chosen [Figure] to be ε = 0 .

Original: 3 for MNIST and Fashion-MNIST, and " = 8 / 255 for CIFAR-10 as previous studies.
Revised: 3 for MNIST and Fashion-MNIST, and ε = 8 / 255 for CIFAR-10 as previous studies.

Original: The white-box adversarial accuracy results, as well as natural accuracy are reported in Table 3.
Revised: The white-box adversarial accuracy results, as well as natural accuracy are reported in Table 3.

Original: As shown in Table 3, defense methods trained with IGR achieve higher accuracies under all threetypes of attacks upon their corresponding baseline methods, except TRADES+IGR under PGD attackin CIFAR-10.
Revised: As shown in Table 3, defense methods trained with IGR achieve higher accuracies under all three types of attacks upon their corresponding baseline methods, except TRADES+IGR under PGD attack in CIFAR-10.

Original: In the meantime, classiﬁcation accuracies of natural images are also improved in sevenout of nine evaluations.
Revised: In the meantime, classification accuracies of natural images are also improved in seven out of nine evaluations.

Original: This suggests that training with IGR improves adversarial accuracies withoutlosing the generalization of natural accuracies.
Revised: This suggests that training with IGR improves adversarial accuracies without losing the generalization of natural accuracies.

Original: Although IGR is designed for attribution protection,these improvements is considered as a side-effect and a rigorous study of the phenomenon can befuture work.
Revised: Although IGR is designed for attribution protection, these improvements is considered as a side-effect and a rigorous study of the phenomenon can be future work.

Original: 6.4 Evaluation on activation consistency
Revised: 6.4 Evaluation on activation consistency

Original: This section reports the experimental results that verify the claim in Section 5.2 — IGR encouragesthat the network activates the same set of neurons for natural and perturbed samples x and ˜ x .
Revised: This section reports the experimental results that verify the claim in Section 5.2 — IGR encourages that the network activates the same set of neurons for natural and perturbed samples x and ˜ x .

Original: Duringthe experiments, all the pre-activation values are recorded and used to compute the proportion ofnonnegative values.
Revised: During the experiments, all the pre-activation values are recorded and used to compute the proportion of nonnegative values.

Original: Thus, the activation consistency deﬁned on the right-hand side of Eq.
Revised: Thus, the activation consistency defined on the right-hand side of Eq.

Original: (6) can benumerically computed.
Revised: (6) can be numerically computed.

Original: Fig.
Revised: Fig.

Original: 4 compares the activation consistency on the baselines and the corresponding models trainedwith IGR.
Revised: 4 compares the activation consistency on the baselines and the corresponding models trained with IGR.

Original: It is noticed that for all the datasets, the activation consistency of the models trainedwith IGR are consistently greater than the corresponding baseline models, which veriﬁes our theoryin Section 5.2.
Revised: It is noticed that for all the datasets, the activation consistency of the models trained with IGR are consistently greater than the corresponding baseline models, which verifies our theory in Section 5.2.

Original: Moreover, as reported in Table 2, the improvements of AT+IGR in MNIST and
Revised: Moreover, as reported in Table 2, the improvements of AT+IGR in MNIST and

Original: MART+IGR in Fashion-MNIST are not as signiﬁcant as others.
Revised: MART+IGR in Fashion-MNIST are not as significant as others.

Original: The results are also reﬂected onactivation consistency, as the value of activation consistency slightly improves from 0.40 to 0.43 on
Revised: The results are also reflected on activation consistency, as the value of activation consistency slightly improves from 0.40 to 0.43 on

Original: AT+IGR in MNIST and from 0.67 to 0.69 on MART+IGR in Fashion-MNIST, while TRADES+IGRthat boosts the most in attribution robustness also increases the most in activation consistency.
Revised: AT+IGR in MNIST and from 0.67 to 0.69 on MART+IGR in Fashion-MNIST, while TRADES+IGR that boosts the most in attribution robustness also increases the most in activation consistency.

Original:  Conclusions
Revised:  Conclusions

Original: In order to leverage the non-differentiable Kendall’s rank correlation for attribution protection,this paper starts with a theorem indicating the positive correlation between cosine similarity and
Revised: In order to leverage the non-differentiable Kendall’s rank correlation for attribution protection, this paper starts with a theorem indicating the positive correlation between cosine similarity and

Original: Kendall’s rank correlation.
Revised: Kendall’s rank correlation.

Original: We then introduce a geometric perspective to explain the shortcomingsof ` p based attribution defense methods and propose the integrated gradients regularizer to improveattribution robustness.
Revised: We then introduce a geometric perspective to explain the shortcomings of ℓ p based attribution defense methods and propose the integrated gradients regularizer to improve attribution robustness.

Original: It is discovered that IGR encourages networks activating the same set ofneurons for natural and perturbed samples.
Revised: It is discovered that IGR encourages networks activating the same set of neurons for natural and perturbed samples.

Original: Finally, experiments show that IGR can be combined withadversarial objective functions, which simultaneously minimizes the angle between attribution vectorsfor attribution robustness and maximizes their magnitude to offer standard adversarial protection. 
Revised: Finally, experiments show that IGR can be combined with adversarial objective functions, which simultaneously minimizes the angle between attribution vectors for attribution robustness and maximizes their magnitude to offer standard adversarial protection. Acknowledgments and Disclosure of Funding This work is partially supported by the Ministry of Education, Singapore through Academic Research Fund Tier 1, RG73/

Revised: Checklist

Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Revised: [Yes] (b) Did you describe the limitations of your work?

Revised: [N/A] (c) Did you discuss any potential negative societal impacts of your work?

Revised: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Revised: [Yes] 2.

Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Revised: [Yes] 3.

Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Revised: [Yes] (b) Did you specify all the training details (e

Revised: g., data splits, hyperparameters, how they were chosen)?

Revised: [Yes] (c) Did you report error bars (e

Revised: g., with respect to the random seed after running experiments multiple times)?

Revised: [N/A] (d) Did you include the total amount of compute and the type of resources used (e

Revised: g., type of GPUs, internal cluster, or cloud provider)?

Revised: [Yes] 4.

Revised: If you are using existing assets (e

Revised: g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Revised: [Yes] (b) Did you mention the license of the assets?

Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Revised: [N/A] (d) Did you discuss whether

Revised: and how consent was obtained from people whose data you’re using/curating?

Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Revised: [N/A] 5.

Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Revised: [N/A]