Original: Exploiting the Relationship Between Kendall’s Rank Correlation and Cosine Similarity for Attribution Protection
Revised: Exploiting the Relationship Between Kendall’s Rank Correlation and Cosine Similarity for Attribution Protection

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AfﬁliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Model attributions are important in deep neural networks as they aid practitionersin understanding the models, but recent studies reveal that attributions can beeasily perturbed by adding imperceptible noise to the input.
Revised: Model attributions are important in deep neural networks as they aid practitionersin understanding the models, but recent studies reveal that attributions can beeasily perturbed by adding imperceptible noise to the input.

Original: The non-differentiable
Revised: The non-differentiable

Original: Kendall’s rank correlation is a key performance index for attribution protection.
Revised: Kendall’s rank correlation is a key performance index for attribution protection.

Original: Inthis paper, we ﬁrst show that the expected Kendall’s rank correlation is positivelycorrelated to cosine similarity and then indicate that the direction of attribution isthe key to attribution robustness.
Revised: Inthis paper, we first show that the expected Kendall’s rank correlation is positivelycorrelated to cosine similarity and then indicate that the direction of attribution isthe key to attribution robustness.

Original: Based on these ﬁndings, we explore the vectorspace of attribution to explain the shortcomings of attribution defense methodsusing ` p norm and propose integrated gradient regularizer (IGR), which maximizesthe cosine similarity between natural and perturbed attributions.
Revised: Based on these findings, we explore the vectorspace of attribution to explain the shortcomings of attribution defense methodsusing ℓ p norm and propose integrated gradient regularizer (IGR), which maximizesthe cosine similarity between natural and perturbed attributions.

Original: Our analysisfurther exposes that IGR encourages neurons with the same activation states fornatural samples and the corresponding perturbed samples.
Revised: Our analysisfurther exposes that IGR encourages neurons with the same activation states fornatural samples and the corresponding perturbed samples.

Original: Our experiments ondifferent models and datasets conﬁrm our analysis on attribution protection anddemonstrate a decent improvement in adversarial robustness.
Revised: Our experiments ondifferent models and datasets confirm our analysis on attribution protection anddemonstrate a decent improvement in adversarial robustness.

Original:  Introduction
Revised:  Introduction

Original: Recently, the explainable artiﬁcial intelligence (XAI) has revived since deep neural networks (DNNs)are applied to more security-sensitive tasks such as medical imaging [27], criminal justice [5] andautonomous driving [20].
Revised: Recently, the explainable artificial intelligence (XAI) has revived since deep neural networks (DNNs)are applied to more security-sensitive tasks such as medical imaging [26], criminal justice [5] andautonomous driving [19].

Original: As one of the XAI tools, model attributions explain and measure therelative impact of each feature on the ﬁnal prediction.
Revised: As one of the XAI tools, model attributions explain and measure therelative impact of each feature on the final prediction.

Original: With more non-expert practitioners beinginvolved, it is more important for them to understand and reliably interpret the mechanism behindthe outputs.
Revised: With more non-expert practitioners beinginvolved, it is more important for them to understand and reliably interpret the mechanism behindthe outputs.

Original: Besides, EU regulators also start to enforce General Data Protection Regulation formore transparent interpretations on decision making based on AI [10].
Revised: Besides, EU regulators also start to enforce General Data Protection Regulation formore transparent interpretations on decision making based on AI [10].

Original: Therefore, the trustworthyattribution is becoming even more crucial.
Revised: Therefore, the trustworthyattribution is becoming even more crucial.

Original: Although numerous attribution methods have been proposed in recent studies [25, 26, 29, 35, 37] ithas been pointed out that they are vulnerable to attribution attacks.
Revised: Although numerous attribution methods have been proposed in recent studies [24, 25, 28, 34, 36], ithas been pointed out that they are vulnerable to attribution attacks.

Original: Different from standard adversarialattacks [3, 9, 18, 22, 31] that focus on misleading classiﬁers to incorrect outputs, Ghorbani et al.
Revised: Different from standard adversarialattacks [3, 9, 17, 21, 30] that focus on misleading classifiers to incorrect outputs, Ghorbani et al.

Original: [8]shows that it is possible to generate visually indistinguishable images which are signiﬁcantly differenton their attributions, but with the same predicted label.
Revised: [8]shows that it is possible to generate visually indistinguishable images which are significantly differenton their attributions, but with the same predicted label.

Original: Dombrowski et al.
Revised: Dombrowski et al.

Original: [6] emphasizes on targetedattack that manipulates the attributions to any predeﬁned target attributions while keeping the modeloutputs unchanged.
Revised: [6] emphasizes on targetedattack that manipulates the attributions to any predefined target attributions while keeping the modeloutputs unchanged.

Original: There are also black-box attacks applied on text explanations [14].

Original: Commonadversarial defense mechanisms such as adversarial training [21] and distillation [23] are not able totackle the attribution attacks; instead, researchers turn their focus on the attribution itself.
Revised: Common adversarial defense mechanisms such as adversarial training [20] anddistillation [22] are not able to tackle the attribution attacks; instead, researchers turn their focus onthe attribution itself.

Original: As the differences between natural and perturbed attributions are measured by Kendall’s rankcorrelation [15], which reﬂects the ordinal importance among features, i.e. , the proportion of order alignment of attributions between original and perturbed images, a straightforward practice to protectthe attributions against such adversaries is to maximize their Kendall’s rank correlation.
Revised: As the differences between natural and perturbed attributions are measured by Kendall’s rankcorrelation [14], which reflects the ordinal importance among features, i.e. , the proportion of order alignment of attributions between original and perturbed images, a straightforward practice to protectthe attributions against such adversaries is to maximize their Kendall’s rank correlation.

Original: Since
Revised: Since

Original: Kendall’s rank correlation is not differentiable, in previous studies, it is replaced by its differentiablealternatives, such as ` p -distance regularizers [2, 4].
Revised: Kendall’s rank correlation is not differentiable, in previous studies, it is replaced by its differentiablealternatives, such as ℓ p -distance regularizers [2, 4].

Original: However, ` p -distance regularizers are not idealfor Kendall’s rank correlation.
Revised: However, ℓ p -distance regularizers are not idealfor Kendall’s rank correlation.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 1, we found that given ﬁxed ` 1 -distance betweenoriginal and perturbed attributions, their Kendall’s rank correlations are drastically different, whichindicates ` 1 -distance is unstable as a measure of attribution similarity.
Revised: 1, we found that given fixed ℓ 1 -distance betweenoriginal and perturbed attributions, their Kendall’s rank correlations are drastically different, whichindicates ℓ 1 -distance is unstable as a measure of attribution similarity.

Original: Besides, there are also non-
Revised: Besides, there are also non-

Original: ` pbased regularizers, such as using Pearson’s correlation, as the surrogate measurement of Kendall’srank correlation [13], it is shown to be unstable to measure the attribution.
Revised: ℓ pbased regularizers, such as using Pearson’s correlation, as the surrogate measurement of Kendall’srank correlation [13], it is shown to be unstable to measure the attribution.

Original: In this paper, we discover that cosine similarity , as a measurement emphasizing the angle betweentwo vectors, is consistent with Kendall’s rank correlation.
Revised: In this paper, we discover that cosine similarity , as a measurement emphasizing the angle betweentwo vectors, is consistent with Kendall’s rank correlation.

Original: We present a theorem stating that cosinesimilarity is positively correlated with the expected Kendall’s rank correlation.
Revised: We present a theorem stating that cosinesimilarity is positively correlated with the expected Kendall’s rank correlation.

Original: Based on the discoveryof angular perspective, we then explain the shortcomings of ` p -norm based attribution robustnessmethods and propose integrated gradients regularizer (IGR) , an attribution robustness trainingregularizer that optimizes on the cosine similarity between natural and perturbed attribution.
Revised: Based on the discoveryof angular perspective, we then explain the shortcomings of ℓ p -norm based attribution robustnessmethods and propose integrated gradients regularizer (IGR) , an attribution robustness trainingregularizer that optimizes on the cosine similarity between natural and perturbed attribution.

Original: Ourfurther analysis shows that optimizing cosine similarity encourages neurons with the same activationstates.
Revised: Ourfurther analysis shows that optimizing cosine similarity encourages neurons with the same activationstates.

Original: The contributions of this work are summarized as follows:
Revised: The contributions of this work are summarized as follows:

Original: • We theoretically show that, under certain assumptions, Kendall’s rank correlation betweentwo vectors is positively correlated to their cosine similarity.
Revised: • We theoretically show that, under certain assumptions, Kendall’s rank correlation between two vectors is positively correlated to their cosine similarity.

Original: • We characterize a novel geometric perspective related to the angles between attribution vec-
Revised: • We characterize a novel geometric perspective related to the angles between attribution vec-

Original: tors that explains the connection between adversarial robustness and attribution robustness for attribution methods fulﬁlling the axiom of completeness [29].
Revised: tors that explains the connection between adversarial robustness and attribution robustnessfor attribution methods fulfilling the axiom of completeness [28].

Original: • Under the angular perspective, we propose integrated gradients regularizer (IGR) to robustlytrain neural networks.
Revised: • Under the angular perspective, we propose integrated gradients regularizer (IGR) to robustlytrain neural networks.

Original: Our method is proved to encourage neurons with the same activation states for natural and corresponding perturbed images.
Revised: Our method is proved to encourage neurons with the same activationstates for natural and corresponding perturbed images.

Original: • The experimental results show that the proposed IGR regularizer can be embedded into
Revised: • The experimental results show that the proposed IGR regularizer can be embedded into

Original: adversarial training methods to improve their performance in terms of both attribution and
Revised: adversarial training methods to improve their performance in terms of both attribution and

Original: adversarial robustness and outperform the state-of-the-art attribution protection methods.
Revised: adversarial robustness and outperform the state-of-the-art attribution protection methods.

Original: The remainder of this paper is organized as follows.
Revised: The remainder of this paper is organized as follows.

Original: We ﬁrst introduce the notations and previousrelated works in Section 2.
Revised: We first introduce the notations and previousrelated works in Section 2.

Original: The content starts with the theorem disclosing the relationship between
Revised: The content starts with the theorem disclosing the relationship between

Original: Kendall’s rank correlation and cosine similarity in Section 3.
Revised: Kendall’s rank correlation and cosine similarity in Section 3.

Original: Based on that, we discuss the vector space of attribution in Section 4 and describe the proposed IGR as well as its property regardingneuron activations in Section 5.
Revised: Based on that, we discuss the vector space of attribution in Section 4 and describe the proposed IGR as well as its property regardingneuron activations in Section 5.

Original: Section 6 presents our experimental results and the paper concludesin Section 7.
Revised: Section 6 presents our experimental results and the paper concludesin Section 7.

Original:  Preliminaries and related work
Revised:  Preliminaries and related work

Original: Let { ( x ( i ) , y ( i ) ) } ni =1 denote data points sampled from the distribution D , where x ( i ) 2 R d are inputdata and y ( i ) 2 { 1 , . . .
Revised: Let { ( x ( i ) , y ( i ) ) } ni =1 denote data points sampled from the distribution D , where x ( i ) ∈ R d are inputdata and y ( i ) ∈ { 1 , . . .

Original: , k } are labels.
Revised: , k } are labels.

Original: A non-bold version x i denotes the i -th feature of vector x ,and the capitalized version X denotes a random variable.
Revised: A non-bold version x i denotes the i -th feature of vector x ,and the capitalized version X denotes a random variable.

Original: A classiﬁer is the mapping from inputspace to the logits f : R d ! R k parameterized by ✓ , where f j ( x ) is the j -th entry of f ( x ) , and theclassiﬁcation result of input x is given by the index of maximum logit ˆ y = arg max 1  j  k ( f j ( x )) .
Revised: A classifier is the mapping from inputspace to the logits f : R d → R k parameterized by θ , where f j ( x ) is the j -th entry of f ( x ) , and theclassification result of input x is given by the index of maximum logit ˆ y = arg max 1 ≤ j ≤ k ( f j ( x )) .

Original: 2.1 Attribution methods
Revised: 2.1 Attribution methods

Original: Model attribution, denoted by g ( x ) , studies the importance that the input features contribute towards the ﬁnal result.
Revised: Model attribution, denoted by g ( x ) , studies the importance that the input features contribute towards the final result.

Original: The mostly used attribution methods include perturbation-based [35, 37]and backpropagation-based methods [1], including gradient-based attribution methods [25, 26].
Revised: The mostly used attribution methods include perturbation-based [34, 36]and backpropagation-based methods [1], including gradient-based attribution methods [24, 25].

Original: Inparticular, integrated gradients (IG) [29], one of the gradient-based methods, computes the attributionusing the line integral of gradients from a baseline image a to the input image x weighted by theirdifference, i.e. , 
Revised: Inparticular, integrated gradients (IG) [28], one of the gradient-based methods, computes the attributionusing the line integral of gradients from a baseline image a to the input image x weighted by theirdifference, i.e. , [Equation]

Original: [Equation]
Revised: [Equation]

Original: IG satisﬁes the axiom of completeness which guarantees P i g ( x ) i =
Revised: IG satisfies the axiom of completeness which guarantees (cid:80) i g ( x ) i =

Original: f y ( x )  f y ( a ) .
Revised: f y ( x ) − f y ( a ) .

Original: We omit thebaseline image a in the later parts of this paper, and it is chosen to be a black image, i.e. , 0 , if notspeciﬁcally stated
Revised: We omit thebaseline image a in the later parts of this paper, and it is chosen to be a black image, i.e. , 0 , if notspecifically stated.

Original: .2.2 Attribution robustness
Revised: 2.2 Attribution robustness

Original: Recent studies reveal the vulnerability of neural networks that, similar to adversarial examples,imperceptible perturbations added to natural images would have signiﬁcantly different attributionwhile their classiﬁcation results remain unchanged [8].
Revised: Recent studies reveal the vulnerability of neural networks that, similar to adversarial examples,imperceptible perturbations added to natural images would have significantly different attributionwhile their classification results remain unchanged [8].

Original: Heo et al.
Revised: Heo et al.

Original: [12] manipulates the modelparameters instead of input images to disturb attributions and remains high accuracy on classiﬁcations.
Revised: [12] manipulates the modelparameters instead of input images to disturb attributions and remains high accuracy on classifications.

Original: Dombrowski et al.
Revised: Dombrowski et al.

Original: [6] makes targeted attack that changes original attributions to any predeﬁnedattributions and gives a theoretical explanation to this phenomenon.
Revised: [6] makes targeted attack that changes original attributions to any predefinedattributions and gives a theoretical explanation to this phenomenon.

Original: Engstrom et al.
Revised: Engstrom et al.

Original: [7] points out that robust optimization enhances model representations and interpretability.
Revised: [7] points out that robust optimization enhances model representations and interpretability.

Original: Chen et al.
Revised: Chen et al.

Original: [4] and Boopathy et al.
Revised: [4] and Boopathy et al.

Original: [2] use ` 1 -norm to constrain the distance betweenattributions of natural and perturbed images.
Revised: [2] use ℓ 1 -norm to constrain the distance betweenattributions of natural and perturbed images.

Original: Sarkar et al.
Revised: Sarkar et al.

Original: [24] proposes a contrastive regularizerthat emphasizes a skewed distribution on true class attribution while a uniform one on negativeclass attribution.
Revised: [23] proposes a contrastive regularizerthat emphasizes a skewed distribution on true class attribution while a uniform one on negativeclass attribution.

Original: Ivankay et al.
Revised: Ivankay et al.

Original: [13] directly optimizes Pearson’s correlation and Singh et al.
Revised: [13] directly optimizes Pearson’s correlation and Singh et al.

Original: [28]uses a triplet loss to minimize the upper bound of the attribution distortion.
Revised: [27]uses a triplet loss to minimize the upper bound of the attribution distortion.

Original: Although the previoustechniques present promising results, none of them exploits the angle between attributions explicitlyfor attribution protection.
Revised: Although the previoustechniques present promising results, none of them exploits the angle between attributions explicitlyfor attribution protection.

Original: The method introduced in this work leverages the relationship between
Revised: The method introduced in this work leverages the relationship between

Original: Kendall’s rank correlation and cosine similarity with a theoretical support for attribution robustness.
Revised: Kendall’s rank correlation and cosine similarity with a theoretical support for attribution robustness.

Original:  Kendall’s rank correlation and cosine similarity
Revised:  Kendall’s rank correlation and cosine similarity

Original: Kendall’s rank correlation, often denoted by ⌧ , is a measurement of the ordinal relationship betweentwo quantities, where two quantities have higher ⌧ when they have more concordant pairs.
Revised: Kendall’s rank correlation, often denoted by τ , is a measurement of the ordinal relationship betweentwo quantities, where two quantities have higher τ when they have more concordant paris.

Original: Formally,
Revised: Formally,

Original: Kendall’s rank correlation between two vectors x and x 0 can be explicitly computed by
Revised: Kendall’s rank correlation between two vectors x and x ′ can be explicitly computed by

Original: [Equation]
Revised: [Equation]

Original: where d is the dimension of the vectors.
Revised: where d is the dimension of the vectors.

Original: As Kendall’s rank correlation is an important metric toquantify the differences between perturbed and natural attributions, we begin by presenting therelationship between Kendall’s rank correlation and cosine similarity.
Revised: As Kendall’s rank correlation is an important metric toquantify the differences between perturbed and natural attributions, we begin by presenting therelationship between Kendall’s rank correlation and cosine similarity.

Original: It should be highlighted that all the previous attribution robustness studies [2, 4, 8, 13, 28, 33] use Kendall’s rank correlation as a keyperformance index to evaluate the effectiveness of their methods.
Revised: It should be highlighted that all the previous attribution robustness studies [2, 4, 8, 13, 27, 32] use Kendall’s rank correlation as a keyperformance index to evaluate the effectiveness of their methods.

Original: To enhance attribution robustness, it is equivalent to force the perturbed attribution to have a higher
Revised: To enhance attribution robustness, it is equivalent to force the perturbed attribution to have a higher

Original: Kendall’s rank correlation with the original one.
Revised: Kendall’s rank correlation with the original one.

Original: However, as Kendall’s rank correlation is notdifferentiable, it is difﬁcult to directly optimize it.
Revised: However, as Kendall’s rank correlation is notdifferentiable, it is difficult to directly optimize it.

Original: It is necessary to ﬁnd an alternative that eitherapproximates to or has a consistent behavior with Kendall’s rank correlation.
Revised: It is necessary to find an alternative that eitherapproximates to or has a consistent behavior with Kendall’s rank correlation.

Original: The following theoremstates that cosine similarity is an appropriate replacement as a regularization term since it is positivelyrelated to the Kendall’s rank correlation (Fig. 2a).
Revised: The following theoremstates that cosine similarity is an appropriate replacement as a regularization term since it is positivelyrelated to the Kendall’s rank correlation (Fig. 2a).

Original: Theorem 1.
Revised: Theorem 1.

Original: Given a random vector Y =
Revised: Given a random vector Y =

Original: ( y 1 , y 2 , . .
Revised: ( y 1 , y 2 , . .

Original: . , y d ) where y i follows a positive-valueddistribution, and two arbitrary vectors with the same dimension, X, X 0 2 R d that x i , x 0 i  0 , assumethat there exists a sequence S
Revised: . , y d ) where y i follows a positive-valueddistribution, and two arbitrary vectors with the same dimension, X, X ′ ∈ R d that x i , x ′ i ≥ 0 , assumethat there exists a sequence S

Original: =
Revised: =

Original: { X i } Ni =1 with X =
Revised: { X i } Ni =1 with X =

Original: X 0 and X [-
-]
Revised: X 0 and X [-
-]′ =

Original: = X N , where the vectors satisfy the 118 condition that cos( X i , Y )  cos( X i +1 , Y ) , and each X i +1 can be induced from its previous vector
Revised: X N , where the vectors satisfy thecondition that cos( X i , Y ) ≥ cos( X i +1 , Y ) , and each X i +1 can be induced from its previous vector

Original: X i through one of the following two operations,
Revised: X i through one of the following two operations,

Original: (i) arbitrarily exchanging two entries of X i
Revised: (i) arbitrarily exchanging two entries of X i

Original: (ii) multiplying one entry in X i by ↵ 2 (0 , 1]
Revised: (ii) multiplying one entry in X i by α ∈ (0 , 1]

Original: Then Kendall’s rank correlations of Y with X and X 0 have the property that E[ ⌧ ( X, Y )] 
Revised: Then Kendall’s rank correlations of Y with X and X ′ have the property that E

Original:  E [ ⌧ ( X 0 , Y )] , where the expectation is taken over Y satisfying cos( X i , Y )  cos( X i + 1 , Y ) .
Revised: [ τ ( X, Y )] ≥ 123 E [ τ ( X ′ , Y )] , where the expectation is taken over Y satisfying cos( X i , Y ) ≥ cos( X i + 1 , Y ) .

Original: The full proof and discussions can be found in Appendix A.
Revised: The full proof and discussions can be found in Appendix A.

Original: In the scenario of attribution robustness,under the above assumption, we denote Y as the natural attribution g ( x ) , and X and X 0 as twoperturbed attributions g ( x 0 ) and g ( x 00 ) .
Revised: In the scenario of attribution robustness,under the above assumption, we denote Y as the natural attribution g ( x ) , and X and X ′ as twoperturbed attributions g ( x ′ ) and g ( x ′′ ) .

Original: If the perturbed attribution has a greater cosine similaritywith natural attribution, then their expected Kendall’s rank correlation is also greater.
Revised: If the perturbed attribution has a greater cosine similaritywith natural attribution, then their expected Kendall’s rank correlation is also greater.

Original: Explicitlyspeaking, if cos( g ( x 0 ) , g ( x ))  cos( g ( x 00 ) , g ( x )) , then E [ ⌧ ( g ( x 0 ) , g ( x ))]
Revised: Explicitlyspeaking, if cos( g ( x ′ ) , g ( x )) ≥ cos( g ( x ′′ ) , g ( x )) , then E [ τ ( g ( x ′ ) , g ( x ))]

Original:  E [ ⌧ ( g ( x 00 ) , g ( x ))] .
Revised: ≥ E [ τ ( g ( x ′′ ) , g ( x ))] .

Original: This theorem provides a theoretical foundation that supports using cosine similarity for attributionprotection because it directly links to Kendall’s rank correlation.
Revised: This theorem provides a theoretical foundation that supports using cosine similarity for attributionprotection because it directly links to Kendall’s rank correlation.

Original:  Characterization of geometric perspective on attributions
Revised:  Characterization of geometric perspective on attributions

Original: In the last section, we have indicated the relationship between cosine similarity and Kendall’s rankcorrelation.
Revised: In the last section, we have indicated the relationship between cosine similarity and Kendall’s rankcorrelation.

Original: In this section, we use this relationship to explain (i) the drawbacks of attributionprotections based on ` p -norm, e.g. , min ✓ k g ( x )  g (˜ x ) k p in Chen et al.
Revised: In this section, we use this relationship to explain (i) the drawbacks of attributionprotections based on ℓ p -norm, e.g. , min θ ∥ g ( x ) − g (˜ x ) ∥ p in Chen et al.

Original: [4], where x is a naturalsample and ˜ x is a perturbed sample; (ii) the inappropriateness of standard adversarial training forattribution protection and (iii) the limitation of the cosine similarity, i.e. , min ✓ (1  cos( g ( x ) , g (˜ x )))for standard adversarial protection.
Revised: [4], where x is a naturalsample and ˜ x is a perturbed sample; (ii) the inappropriateness of standard adversarial training forattribution protection and (iii) the limitation of the cosine similarity, i.e. , min θ (1 − cos( g ( x ) , g (˜ x )))for standard adversarial protection.

Original: In this discussion, g ( x ) and g (˜ x ) are considered as vectors, andas stated in Theorem 1, a smaller angle between them implies higher attribution robustness.

Original: Theattribution method g is assumed to fulﬁll the axiom of completeness 1 , i.e. , f y ( x )  f y ( a )=
Revised: In this discussion, the attribution method g is assumed to fulfillthe axiom of completeness, i.e. , f y ( x ) − f y ( a )

Original: P i g ( x ) i . If g ( x ) i  0 for all i , k g ( x ) k 2 = pP i g ( x ) 2 i  P i g ( x ) i =
Revised: =

Original: f y ( x ) . In other words, k g ( x ) k 2 is thelower bound of f y ( x ) andlarger k g ( x ) k 2 implies higher classiﬁcation accuracy. Thus, minimizingthe angle between g ( x ) and g (˜ x )  and maximizing their magnitudes would respectively enhance theirattributional and adversarial robustness.
Revised: (cid:80) i g ( x ) i and g ( x ) and g (˜ x ) are considered asvectors. Without loss of generality, we assume f y ( a ) = 0 and use ℓ 2 -norm as the illustration.

Original: Fig.
Revised: Fig.

Original: 2b shows a two-dimensional projection for the ease of illustration, where each 2D point representsan attribution of an input.
Revised: 2b shows a two-dimensional projection for the ease of illustration, where each two-dimensionalpoint represents an attribution of an input.

Original: Higher-dimensional cases can be extended in a similar manner.
Revised: Higher-dimensional cases can be extended in a similarmanner.

Original: In Fig.
Revised: In Fig.

Original: 2b, g ( x ) is the original attribution of x and the others are its perturbed counterparts.
Revised: 2b, g ( x ) is the original attribution of x and the others are its perturbed counterparts.

Original: Given twoattributions, g ( x 0 ) and g ( x 00 ) , where k g ( x )  g ( x 0 ) 
Revised: Given two attributions, g ( x ′ ) and g ( x ′′ ) , where ∥ g ( x ) − g ( x ′ ) ∥

Original: k
Revised: =

Original: = k g ( x )  g ( x 00 )k
Revised: ∥ g ( x ) − g ( x ′′ )

Original: but cos( g ( x ) , g ( x 0 )) >148 cos( g ( x ) , g ( x 00 )) , according to Theorem 1, ⌧ ( g ( x ) , g ( x 0 )) is likely larger than ⌧ ( g ( x ) , g ( x 00 )) ,implying that the attribution of x 0 is likely closer to the attribution of x than that of x 00 .
Revised: ∥ butcos( g ( x ) , g ( x ′ )) > cos( g ( x ) , g ( x ′′ )) , according to Theorem 1, τ ( g ( x ) , g ( x ′ )) is likely larger thanτ ( g ( x ) , g ( x ′′ )) , implying that the attribution of x ′ is likely closer to the attribution of x than that ofx ′′ .

Original: It explainsthe results in Fig.
Revised: It explains the results in Fig.

Original: 1 and point (i), i.e. , drawbacks of attribution protection based on ` p -norm.
Revised: 1 and point (i), i.e. , drawbacks of attribution protection based onℓ p -norm.

Revised: Before discussing the point (ii), it is worthy to mention that, if g ( x ) i ≥ 0 for all i ,

Revised: ∥ g ( x ) ∥ 2 = 149 (cid:112)(cid:80) i g

Revised: ( x ) 2 i ≤ (cid:80) i g ( x ) i =

Revised: f y ( x ) .

Revised: In other words, ∥ g ( x ) ∥ 2 is the lower bound of f y ( x ) .

Original: The standard adversarial training maximizes f y (˜ x ) , where ˜ x is an adversarial example.
Revised: Thestandard adversarial training maximizes f y (˜ x ) , where ˜ x is an adversarial example.

Original: In Fig.
Revised: In Fig.

Original: 2b,x 000 has large k g ( x 000 ) k 2 , implying that f y ( x 000 ) is also large.
Revised: 2b, x ′′′ haslarge ∥ g ( x ′′′ ) ∥ 2 , implying that f y ( x ′′′ ) is also large.

Original: In other words, the classiﬁcation labelof x 000 is well protected.
Revised: In other words, the classification label of x ′′′ iswell protected.

Original: However, standard adversarial training does not explicitly minimize theangle between g ( x ) and g ( x 000 ) .
Revised: However, standard adversarial training does not explicitly optimize the angle betweeng ( x ) and g ( x ′′′ ) .

Original: It implies that ⌧ ( g ( x ) , g ( x 000 )) can be small and x 000 can attack theattribution successfully.
Revised: It implies that τ ( g ( x ) , g ( x ′′′ )) can be small and x ′′′ can attack the attributionsuccessfully.

Original: It should be mentioned that adversarial training does improve attribution 
Revised: It should be mentioned that adversarial training does improve attribution robustnessbecause it smooths the decision surface [32] although it is not the most ideal one. It explains thepoint (ii).

Original: [Figure]
Revised: [Figure]

Original: g ( x 0 ) g ( x 00 ) 
Revised: g ( x 0 ) g ( x 00 ) g ∗ ( ˜ x ) g ( x 000 ) g ( x ) g (˜ x ) gg(

Original: g ⇤ ( ˜ x ) g ( x 000 ) g ( x ) g (˜ x ) gg(b) robustness because it smooths the decision surface [33] although it is not the most ideal one. Itexplains the point (ii).
Revised: [Figure]

Original: Point (iii) can be explained similarly.
Revised: Point (iii) can be explained similarly.

Original: Since the cosine similarity, or min ✓ (1  cos( g ( x ) , g (˜ x )) , doesnot necessarily enlarge the magnitude of g (˜ x ) , it cannot improve network robustness against standardadversarial attack.
Revised: Since the cosine similarity, or min θ (1 − cos( g ( x ) , g (˜ x )) , doesnot necessarily enlarge the magnitude of g (˜ x ) , it cannot improve network robustness against standardadversarial attack.

Original: Fig.
Revised: Fig.

Original: 2b shows two networks (the dashed circle and solid circle).
Revised: 2b shows two networks (the dashed circle and solid circle).

Original: cos( g ( x ) , g (˜ x )) =
Revised: cos( g ( x ) , g (˜ x )) =

Original: cos( g ( x ) , g ⇤ (˜ x )) , which implies that the two networks perform the same on attribution protection,but k g ⇤ (˜ x ) k > k g (˜ x ) k , meaning g ⇤ is more robust against the standard adversarial attack from ˜ x ,while g is more vulnerable.
Revised: cos( g ( x ) , g ∗ (˜ x )) , which implies that the two networks perform the same on attribution protection,but ∥ g ∗ (˜ x ) ∥ > ∥ g (˜ x ) ∥ , meaning g ∗ is more robust against the standard adversarial attack from ˜ x ,while g is more vulnerable.

Original: To protect against both attribution attack and adversarial attack, in the following section, the proposed
Revised: To protect against both attribution attack and adversarial attack, in the following section, the proposed

Original: IGR is optimized with adversarial loss together, where the former minimizes the angle betweenattribution vectors to perform attribution protection, and the latter maximizes their magnitude to offerstandard adversarial protection.
Revised: IGR is optimized with adversarial loss together, where the former minimizes the angle betweenattribution vectors to perform attribution protection, and the latter maximizes their magnitude to offerstandard adversarial protection.

Original:  Integrated gradients regularizer (IGR)
Revised:  Integrated gradients regularizer (IGR)

Original: Based on the above analysis, in this section, we introduce the integrated gradients regularizer (IGR),which regularizes the cosine similarity between natural and perturbed attributions.
Revised: Based on the above analysis, in this section, we introduce the integrated gradients regularizer (IGR),which regularizes the cosine similarity between natural and perturbed attributions.

Original: 5.1 IGR robust training objective
Revised: 5.1 IGR robust training objective

Original: Since Kendall’s rank correlation and cosine similarity are positively related, we suggest to improveattribution robustness, especially integrated gradients, by maximizing the cosine similarity betweennatural and perturbed attributions, or equivalently, minimizing 1  cos( IG ( x ) , IG (˜ x )) .
Revised: Since Kendall’s rank correlation and cosine similarity are positively related, we suggest to improveattribution robustness, especially integrated gradients, by maximizing the cosine similarity betweennatural and perturbed attributions, or equivalently, minimizing 1 − cos( IG ( x ) , IG (˜ x )) .

Original: Therefore,we propose the following training objective function incorporating the IGR
Revised: Therefore,we propose the following training objective function incorporating the IGR

Original: [Equation]
Revised: [Equation]

Original: (3) where L is a standard loss function used in robust training, and  is a hyper-parameter.
Revised: where L is a standard loss function used in robust training, and λ is a hyper-parameter.

Original: We will latershow in Section 6 that L can be chosen from existing loss function in robust training, and our IGRwill further improve the robustness upon baseline methods.
Revised: We will latershow in Section 6 that L can be chosen from existing loss function in robust training, and our IGRwill further improve the robustness upon baseline methods.

Original: In practice, the integral inside IG deﬁnition can be numerically computed by its Riemann sum, and
Revised: In practice, the integral inside IG definition can be numerically computed by its Riemann sum, and

Original: IG is approximated by
Revised: IG is approximated by

Original: [Equation]
Revised: [Equation]

Original: Similar to adversarial training, optimizing the above objective function in Eq.
Revised: Similar to adversarial training, optimizing the above objective function in Eq.

Original: (3) requires perturbedexample ˜ x
Revised: (3) requires perturbedexample ˜ x

Original: that maximally diverts its IG from the original counterpart.
Revised: that maximally diverts its IG from the original counterpart.

Original: Such examples can be foundby maximizing the proposed regularizer within its ` p -ball with radius " , i.e. ,
Revised: Such examples can be found by maximizing the proposed regularizer within its ℓ p -ball with radius ε , i.e. ,

Original: [Equation]
Revised: [Equation]

Original: It is noticed that computing the adversarial loss L (˜ x , y ; ✓ ) itself relies on ˜ x , which can be obtainedfrom adversarial attacks, i.e. , ˜ x = arg max
Revised: It is noticed that computing the adversarial loss L (˜ x , y ; θ ) itself relies on ˜ x , which can be obtainedfrom adversarial attacks, i.e. , ˜ x = arg max

Original: ˜ x L (˜ x , y ; ✓ ) .
Revised: ˜ x L (˜ x , y ; θ ) .

Original: Thus, here we reuse these ˜ x in IGR to avoidrepeatedly using gradient descent methods to ﬁnd the optimum in Eq.
Revised: Thus, here we reuse these ˜ x in IGR to avoidrepeatedly using gradient descent methods to find the optimum in Eq.

Original: (5) and speed up the training.
Revised: (5) and speed up the training.

Original: For example, if L is the standard adversarial training loss function, we directly use the examplesgenerated from PGD attack.
Revised: For example, if L is the standard adversarial training loss function, we directly use the adversarialexamples generated from PGD attack.

Original: The computation cost of IGR is similar to previous proposed methods.
Revised: The computation cost of IGR is similar to previous proposedmethods.

Original: The use of Pearson’s correlation regularizer in Ivankay et al.
Revised: The use of Pearson’s correlation regularizer in Ivankay et al.

Original: [13] as the replacement of Kendall’srank correlation is the closest method to ours.
Revised: [13] as the replacement of Kendall’srank correlation is the closest method to ours.

Original: Ivankay et al.
Revised: Ivankay et al.

Original: [13] suggests that Pearson’s correlationregularizer keeps the ranking of feature constant.
Revised: [13] suggests that Pearson’s correlationregularizer keeps the ranking of feature constant.

Original: However, the statement is not supported by anytheoretical justiﬁcation while we give a theorem that shows cosine similarity is positively related to
Revised: However, the statement is not supported by anytheoretical justification while we give a theorem that shows cosine similarity is positively related to

Original: Kendall’s rank correlation.
Revised: Kendall’s rank correlation.

Original: Besides, the Pearson’s correlation is an unstable metric for attributionswith small variances.
Revised: Besides, the Pearson’s correlation is an unstable metric for attributionswith small variances.

Original: For a ﬁxed vector, two slightly different inputs  can have drastically different
Revised: For a fixed vector, two slightly different inputs δ can have drastically different

Original: Pearson’s correlation, which easily ﬂuctuate from  1 to 1 .
Revised: Pearson’s correlation, which easily fluctuate from − 1 to 1 .

Original: The detailed discussion can be found in
Revised: The detailed discussion can be found in

Original: Appendix B.
Revised: Appendix B.

Original: 5.2 IGR induces more consistent neuron activation states
Revised: 5.2 IGR induces more consistent neuron activation states

Original: An interesting discovery about IGR is related to neuron activations.
Revised: An interesting discovery about IGR is related to neuron activations.

Original: We found that the activationfunctions in ReLU networks trained with IGR are more often with the same neuron activation statesfor natural sample and corresponding perturbed sample.
Revised: We found that the activationfunctions in ReLU networks trained with IGR are more often with the same neuron activation statesfor natural sample and corresponding perturbed sample.

Original: For deep networks with ReLU activations, ifthe pre-ReLU value is positive (negative) for natural sample, the probability of pre-ReLU value beingpositive (negative) for corresponding perturbed sample would increase when trained with IGR.
Revised: For deep networks with ReLU activations, ifthe pre-ReLU value is positive (negative) for natural sample, the probability of pre-ReLU value beingpositive (negative) for corresponding perturbed sample would increase when trained with IGR.

Original: Toanalyze this phenomenon, a single-layer neural network with ReLU activation is studied.
Revised: Toanalyze this phenomenon, a single-layer neural network with ReLU activation is studied.

Original: The resultsfrom this single-layer neural network can be extended to deep networks by stacking multiple layers.
Revised: The resultsfrom this single-layer neural network can be extended to deep networks by stacking multiple layers.

Original: Recall that x 2 R d is an input image, and the network function f is parameterized by ( W , u , c ) [-
-] R d ⇥ m ⇥ R m ⇥ R , where W i is the column vector of W , w ij is the ij -th entry of matrix W and u i isthe
Revised: Recall that x ∈ R d is an input image, and the network function f is parameterized by ( W , u , c ) [-
-]∈ 206 R d × m × R m × R , where W i is the column vector of W , w ij is the ij -th entry of matrix W and u i isthe

Original: i -th entry of vector u , i.e. , f ( x ) = u > ReLU ( W > x ) + c .
Revised: i -th entry of vector u , i.e. , f ( x ) = u ⊤ ReLU ( W ⊤ x ) + c .

Original: Then, the following proposition holds.
Revised: Then, the following proposition holds.

Original: Proposition 1.
Revised: Proposition 1.

Original: Given a single-layer neural network with ReLU activation, and with the aboveparameterization, if, for all i , W i and u i are all independent and identically distributed randomvariables following Gaussian distributions, i.e., W i i.i.d.
Revised: Given a single-layer neural network with ReLU activation, and with the aboveparameterization, if, for all i , W i and u i are all independent and identically distributed randomvariables following Gaussian distributions, i.e., W i i.i.d.

Original: ⇠ N (0 ,  2 w I d ) and u i i.i.d.
Revised: ∼ N (0 , σ 2 w I d ) and u i i.i.d.

Original: ⇠ N (0 ,  2 u ) , andtwo input images that each has small variance, x and ˜ x , then
Revised: ∼ N (0 , σ 2 u ) , andtwo input images that each has small variance, x and ˜ x , then

Original: [Equation]
Revised: [Equation]

Original: The proof can be found in Appendix A.2.
Revised: The proof can be found in Appendix A.3.

Original: The right-hand side of Eq.
Revised: The right-hand side of Eq.

Original: (6) is called the activationconsistency of natural and perturbed samples.
Revised: (6) is called the activationconsistency of natural and perturbed samples.

Original: For the sake of convenience, let the event W > x > 0 be
Revised: For the sake of convenience, let the event W ⊤ x > 0 be

Original: A and W > ˜ x > 0 be B .
Revised: A and W ⊤ ˜ x > 0 be B .

Original: The right-hand side of Eq.
Revised: The right-hand side of Eq.

Original: (6) can be rewritten as P ( A \ B ) / p P ( A ) P ( B ) .
Revised: (6) can be rewritten as P ( A ∩ B ) / (cid:112) P ( A ) P ( B ) .

Original: Since P ( A \ B ) has the upper bound that P ( A \ B )  min { P ( A ) , P ( B ) } , it is obvious that
Revised: Since P ( A ∩ B ) has the upper bound that P ( A ∩ B ) ≤ min { P ( A ) , P ( B ) } , it is obvious that

Original: [Equation]
Revised: [Equation]

Original: and the equality holds when event A happens with the same probability as event B , i.e. , P ( A )=
Revised: and the equality holds when event A happens with the same probability as event B , i.e. , P ( A )

Original:  P ( B ) = P ( A \ B ) ; alternatively, the equality can hold when P ( A c )
Revised: = 217 P ( B ) = P ( A ∩ B ) ; alternatively, the equality can hold when P ( A c )

Original: =
Revised: =

Original: P ( B c ) =
Revised: P ( B c ) =

Original: P ( A c [ B c ) .
Revised: P ( A c ∪ B c ) .

Original: Inother words, maximizing Eq.
Revised: Inother words, maximizing Eq.

Original: (6) encourages that the network activates the same set of neurons for xand ˜ x , or deactivates the same set of neurons for x and ˜ x .
Revised: (6) encourages that the network activates the same set of neurons for xand ˜ x , or deactivates the same set of neurons for x and ˜ x .

Original:  Experiments and results
Revised:  Experiments and results

Original: 6.1 Experimental conﬁgurations
Revised: 6.1 Experimental configurations

Original: We evaluate the performance of IGR on different datasets, including MNIST [19], Fashion-MNIST[34] and CIFAR-10 [17].
Revised: We evaluate the performance of IGR on different datasets, including MNIST [18], Fashion-MNIST[33] and CIFAR-10 [16].

Original: For MNIST and Fashion-MNIST, we use a network consisting of four
Revised: For MNIST and Fashion-MNIST, we use a network consisting of four

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: convolutional layers followed by three fully connected layers.
Revised: convolutional layers followed by three fully connected layers.

Original: The model is trained by Adam
Revised: The model is trained by Adam

Original: Optimizer [16] with learning rate 10  4 for 90 epochs.
Revised: Optimizer [15] with learning rate 10 − 4 for 90 epochs.

Original: For CIFAR-10, we train a ResNet-18 [11] for120 epochs using SGD [30] with initial learning rate 0.1, momentum 0.9 and weight decay 5 ⇥ 10  4 .
Revised: For CIFAR-10, we train a ResNet-18 [11] for120 epochs using SGD [29] with initial learning rate 0.1, momentum 0.9 and weight decay 5 × 10 − 4 .

Original: The learning rate decays by 0 .
Revised: The learning rate decays by 0 .

Original: 1 at the 75th and 90th epoch.
Revised: 1 at the 75th and 90th epoch.

Original: All the experiments are run on NVIDIA
Revised: All the experiments are run on NVIDIA

Original: GeForce RTX 3090 2 .
Revised: GeForce RTX 3090 1 .

Original: As discussed in Section 5, IGR is applied with state-of-the-art adversarial training methods: standardadversarial training (AT)[21], TRADES [36] and MART [32].
Revised: As discussed in Section 5, IGR is applied with state-of-the-art adversarial training methods: standardadversarial training (AT)[20], TRADES [35] and MART [31].

Original: L at , L trades and L mart in Table 1 arethe objective functions of these methods, and are regarded as L in Eq.
Revised: L at , L trades and L mart in Table 1 arethe objective functions of these methods, and are regarded as L in Eq.

Original: In Table 1, CE denotes thecross-entropy loss and KL denotes the KL-divergence.
Revised: In Table 1, CE denotes thecross-entropy loss and KL denotes the KL-divergence.

Original: BCE is a boosted cross-entropy (see details in
Revised: BCE is a boosted cross-entropy (see details in

Original: Wang et al.
Revised: Wang et al.

Original: Note that both AT and MART generate adversarial examples by maximizing the
Revised: Note that both AT and MART generate adversarial examples by maximizing the

Original: CE loss, while TRADES maximizes the KL-divergence regularizer.
Revised: CE loss, while TRADES maximizes the KL-divergence regularizer.

Original: Following the baseline methods,we directly leverage the perturbed examples generated by their original techniques to compute theintegrated gradients, as well as the IGR, instead of generating our own ones using Eq.
Revised: Following the baseline methods,we directly leverage the perturbed examples generated by their original techniques to compute theintegrated gradients, as well as the IGR, instead of generating our own ones using Eq.

Original: Moreover,to ensure fair comparisons, we keep the hyper-parameters the same for models with or without IGR.
Revised: Moreover,to ensure fair comparisons, we keep the hyper-parameters the same for models with or without IGR.

Original: 6.2 Evaluation on attribution robustness
Revised: 6.2 Evaluation on attribution robustness

Original: To evaluate our method under attribution attack, the iterative feature importance attacks (IFIA) usingtop-k intersection as dissimilarity function ( top-k ) [8] is adapted.
Revised: To evaluate our method under attribution attack, the iterative feature importance attacks (IFIA) usingtop-k intersection as dissimilarity function ( top-k ) [8] is adapted.

Original: IFIA generates perturbations byiteratively maximizing the dissimilarity function that measures the changes between attributions ofimages, while keeps the classiﬁcation results unchanged.
Revised: IFIA generates perturbations byiteratively maximizing the dissimilarity function that measures the changes between attributions ofimages, while keeps the classification results unchanged.

Original: In this experiment, we perform 200-step
Revised: In this experiment, we perform 200-step

Original: IFIA as in Chen et al.
Revised: IFIA as in Chen et al.

Original: For MNIST and Fashion-MNIST, we choose k = 100 and the perturbationsize " = 0 .
Revised: For MNIST and Fashion-MNIST, we choose k = 100 and the perturbationsize ε = 0 .

Original: For CIFAR-10, k
Revised: For CIFAR-10, k

Original: = 1000 and " = 8 / 255 .
Revised: = 1000 and ε = 8 / 255 .

Original: Two metrics are chosen to evaluate theperformance under attribution attack as in Chen et al.
Revised: Two metrics are chosen to evaluate theperformance under attribution attack as in Chen et al.

Original: [4]: top-k intersection and Kendall’s rankcorrelation, where top-k intersection counts the proportion of pixels that coincide in the k most
Revised: [4]: top-k intersection and Kendall’s rankcorrelation, where top-k intersection counts the proportion of pixels that coincide in the k most

Original: More visualization results are given in Appendix C
Revised: More visualization results are given in Appendix C

Original: [Table]
Revised: [Table]

Original: important features.
Revised: important features.

Original: Each sample is attacked ﬁve times and the mean metrics are reported.
Revised: Each sample is attacked five times and the mean metrics are reported.

Original: For bothmetrics, a higher number indicates that the model is more robust under the attack.
Revised: For bothmetrics, a higher number indicates that the model is more robust under the attack.

Original: To compare, attribution protection methods, IG-NORM and IG-SUM-NORM by Chen et al.
Revised: To compare, attribution protection methods, IG-NORM and IG-SUM-NORM by Chen et al.

Original: Smooth Surface Regularization (SSR) [33], Attributional Robustness Training (ART) [28] and Adversarial Attributional Training with robust training loss ( AdvAAT ), are implemented and evaluated onall the datasets.
Revised: Smooth Surface Regularization (SSR) [32], Attributional Robustness Training (ART) [27] and Adversarial Attributional Training with robust training loss ( AdvAAT ), are implemented and evaluated onall the datasets.

Original: A cross-entropy loss trained natural model ( standard ) is also included as a baseline.
Revised: A cross-entropy loss trained natural model ( standard ) is also included as a baseline.

Original: The details of these baseline methods are brieﬂy introduced in Appendix C.2.
Revised: The details of these baseline methods are briefly introduced in Appendix C.2.

Original: From the results in Table 2, we observed the following phenomenons.
Revised: From the results in Table 2, we observed the following phenomenons.

Original: (i)
Revised: (i)

Original: Compared with baselinemethods (AT, TRADES and MART), models trained with IGR outperform their correspondingcounterparts in terms of both top-k intersection and Kendall’s rank correlation.
Revised: Compared with baselinemethods (AT, TRADES and MART), models trained with IGR outperform their correspondingcounterparts in terms of both top-k intersection and Kendall’s rank correlation.

Original: (ii) Adversarial defensemethods themselves also help the attribution protection, especially improve on Fashion-MNIST and
Revised: (ii) Adversarial defensemethods themselves also help the attribution protection, especially improve on Fashion-MNIST and

Original: CIFAR-10 comparing with the standard cross-entropy training.
Revised: CIFAR-10 comparing with the standard cross-entropy training.

Original: (iii)
Revised: (iii)

Original: Compared with other attributionprotection methods, standard adversarial defense methods, including AT, TRADES and MART, areweaker in attribution robustness; however, they achieve comparable or even stronger attributionprotections when training with IGR.
Revised: Compared with other attributionprotection methods, standard adversarial defense methods, including AT, TRADES and MART, areweaker in attribution robustness; however, they achieve comparable or even stronger attributionprotections when training with IGR.

Original: (iv) TRADES itself has the best attribution protections amongmodels without IGR, and IGR provides the most signiﬁcant boost on TRADES.
Revised: (iv) TRADES itself has the best attribution protections amongmodels without IGR, and IGR provides the most significant boost on TRADES.

Original: (v) Since AdvAATuses Pearson’s correlation as a regularizer, which is close to IGR, and TRADES+IGR outperformsthe other baselines, we apply Pearson’s correlation on TRADES, i.e. , TRADES +AdvAAT, in the
Revised: (v) Since AdvAATuses Pearson’s correlation as a regularizer, which is close to IGR, and TRADES+IGR outperformsthe other baselines, we apply Pearson’s correlation on TRADES, i.e. , TRADES +AdvAAT, in the

Original: Table 2 for the comparison.
Revised: Table 2 for the comparison.

Original: Table 2 shows clearly that TRADES+AdvAAT does not always improveover TRADES and is consistently outperformed by TRADES+IGR.
Revised: Table 2 shows clearly that TRADES+AdvAAT does not always improveover TRADES and is consistently outperformed by TRADES+IGR.

Original: A visualization of attribution robustness is also presented in Figure 3.
Revised: A visualization of attribution robustness is also presented in Figure 3.

Original: It is observed that the attributionof the baseline model is easily corrupted.
Revised: It is observed that the attributionof the baseline model is easily corrupted.

Original: For model trained with IGR, although the magnitudes of
Revised: For model trained with IGR, although the magnitudes of

Original: IG are different from IG of the original images, the directions remain nearly identical, which is alsoaligned with human visual perceptions.
Revised: IG are different from IG of the original images, the directions remain nearly identical, which is alsoaligned with human visual perceptions.

Original: 6.3 Evaluation on white-box adversarial robustness
Revised: 6.3 Evaluation on white-box adversarial robustness

Original: To evaluate the performance of IGR on adversarial robustness, the trained defense models areevaluated under white-box adversarial attacks, including FGSM [9], PGD [21] and CW 1 [3], where
Revised: To evaluate the performance of IGR on adversarial robustness, the trained defense models areevaluated under white-box adversarial attacks, including FGSM [9], PGD [20] and CW ∞ [3], where

Original: [Figure]
Revised: [Figure]

Original: all the attacks have the information of the entire models, including architectures and parameters.
Revised: all the attacks have the information of the entire models, including architectures and parameters.

Original: Thenumbers are reported in both natural accuracy (Natural) and adversarial accuracies under FGSM,
Revised: Thenumbers are reported in both natural accuracy (Natural) and adversarial accuracies under FGSM,

Original: PGD with 20 steps (PGD20), and CW 1 attacks.
Revised: PGD with 20 steps (PGD20), and CW ∞ attacks.

Original: The maximum allowable perturbations are chosento be " = 0 .
Revised: The maximum allowable perturbations are chosento be ε = 0 .

Original: 3 for MNIST and Fashion-MNIST, and " = 8 / 255 for CIFAR-10 as previous studies.
Revised: 3 for MNIST and Fashion-MNIST, and ε = 8 / 255 for CIFAR-10 as previous studies.

Original: The white-box adversarial accuracy results, as well as natural accuracy are reported in Table 3.
Revised: The white-box adversarial accuracy results, as well as natural accuracy are reported in Table 3.

Original: As shown in Table 3, defense methods trained with IGR achieve higher accuracies under all threetypes of attacks upon their corresponding baseline methods, except TRADES+IGR under PGD attackin CIFAR-10.
Revised: As shown in Table 3, defense methods trained with IGR achieve higher accuracies under all threetypes of attacks upon their corresponding baseline methods, except TRADES+IGR under PGD attackin CIFAR-10.

Original: In the meantime, classiﬁcation accuracies of natural images are also improved in sevenout of nine evaluations.
Revised: In the meantime, classification accuracies of natural images are also improved in sevenout of nine evaluations.

Original: This suggests that training with IGR improves adversarial accuracies withoutlosing the generalization of natural accuracies.
Revised: This suggests that training with IGR improves adversarial accuracies withoutlosing the generalization of natural accuracies.

Original: Although IGR is designed for attribution protection,these improvements is considered as a side-effect and a rigorous study of the phenomenon can befuture work.
Revised: Although IGR is designed for attribution protection,these improvements is considered as a side-effect and a rigorous study of the phenomenon can befuture work.

Original: 6.4 Evaluation on activation consistency
Revised: 6.4 Evaluation on activation consistency

Original: This section reports the experimental results that verify the claim in Section 5.2 — IGR encouragesthat the network activates the same set of neurons for natural and perturbed samples x and ˜ x .
Revised: This section reports the experimental results that verify the claim in Section 5.2 — IGR encouragesthat the network activates the same set of neurons for natural and perturbed samples x and ˜ x .

Original: Duringthe experiments, all the pre-activation values are recorded and used to compute the proportion ofnonnegative values.
Revised: Duringthe experiments, all the pre-activation values are recorded and used to compute the proportion ofnonnegative values.

Original: Thus, the activation consistency deﬁned on the right-hand side of Eq.
Revised: Thus, the activation consistency defined on the right-hand side of Eq.

Original: (6) can benumerically computed.
Revised: (6) can benumerically computed.

Original: Fig.
Revised: Fig.

Original: 4 compares the activation consistency on the baselines and the corresponding models trainedwith IGR.
Revised: 4 compares the activation consistency on the baselines and the corresponding models trainedwith IGR.

Original: It is noticed that for all the datasets, the activation consistency of the models trainedwith IGR are consistently greater than the corresponding baseline models, which veriﬁes our theoryin Section 5.2.
Revised: It is noticed that for all the datasets, the activation consistency of the models trainedwith IGR are consistently greater than the corresponding baseline models, which verifies our theoryin Section 5.2.

Original: Moreover, as reported in Table 2, the improvements of AT+IGR in MNIST and
Revised: Moreover, as reported in Table 2, the improvements of AT+IGR in MNIST and

Original: MART+IGR in Fashion-MNIST are not as signiﬁcant as others.
Revised: MART+IGR in Fashion-MNIST are not as significant as others.

Original: The results are also reﬂected onactivation consistency, as the value of activation consistency slightly improves from 0.40 to 0.43 on
Revised: The results are also reflected onactivation consistency, as the value of activation consistency slightly improves from 0.40 to 0.43 on

Original: AT+IGR in MNIST and from 0.67 to 0.69 on MART+IGR in Fashion-MNIST, while TRADES+IGRthat boosts the most in attribution robustness also increases the most in activation consistency.
Revised: AT+IGR in MNIST and from 0.67 to 0.69 on MART+IGR in Fashion-MNIST, while TRADES+IGRthat boosts the most in attribution robustness also increases the most in activation consistency.

Original:  Conclusions
Revised:  Conclusions

Original: In order to leverage the non-differentiable Kendall’s rank correlation for attribution protection,this paper starts with a theorem indicating the positive correlation between cosine similarity and
Revised: In order to leverage the non-differentiable Kendall’s rank correlation for attribution protection,this paper starts with a theorem indicating the positive correlation between cosine similarity and

Original: Kendall’s rank correlation.
Revised: Kendall’s rank correlation.

Original: We then introduce a geometric perspective to explain the shortcomingsof ` p based attribution defense methods and propose the integrated gradients regularizer to improveattribution robustness.
Revised: We then introduce a geometric perspective to explain the shortcomingsof ℓ p based attribution defense methods and propose the integrated gradients regularizer to improveattribution robustness.

Original: It is discovered that IGR encourages networks activating the same set ofneurons for natural and perturbed samples.
Revised: It is discovered that IGR encourages networks activating the same set ofneurons for natural and perturbed samples.

Original: Finally, experiments show that IGR can be combined withadversarial objective functions, which simultaneously minimizes the angle between attribution vectorsfor attribution robustness and maximizes their magnitude to offer standard adversarial protection. 
Revised: Finally, experiments show that IGR can be combined withadversarial objective functions, which simultaneously minimizes the angle between attribution vectorsfor attribution robustness and maximizes their magnitude to offer standard adversarial protection. Checklist For all authors...

Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Revised: contributions and scope?

Revised: [Yes]

Revised: (b) Did you describe the limitations of your work?

Revised: [N/A]

Revised: (c) Did you discuss any potential negative societal impacts of your work?

Revised: [N/A](d)

Revised: Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Revised: [Yes]2.

Revised: If you are including theoretical results...

Revised: (a) Did you state the full set of assumptions of all theoretical results?

Revised: [Yes]

Revised: (b) Did you include complete proofs of all theoretical results?

Revised: [Yes]

Revised: If you ran experiments...

Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Revised: [Yes](b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?

Revised: [Yes]

Revised: (c) Did you report error bars (e

Revised: g., with respect to the random seed after running experiments multiple times)?

Revised: [N/A]

Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Revised: [Yes]

Revised: If you are using existing assets (e

Revised: g., code, data, models) or curating/releasing new assets...

Revised: (a)

Revised: If your work uses existing assets, did you cite the creators?

Revised: [Yes]

Revised: (b) Did you mention the license of the assets?

Revised: [N/A]

Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Revised: [N/A]

Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Revised: using/curating?

Revised: [N/A]

Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Revised: information or

Revised: offensive

Revised: content

Revised: [N/A]5.

Revised: If you crowdsourcing or conducted research with human subjects...

Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Revised: applicable?

Revised: [N/A]

Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Revised: Board (IRB) approvals, if applicable?

Revised: [N/A]

Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Revised: spent on participant compensation?

Revised: [N/A]