Original: M ODELING A DVERSARIAL N OISE FOR A DVERSARIAL D EFENSE
Revised: M ODELING A DVERSARIAL N OISE FOR A DVERSARIAL D EFENSE

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Deep neural networks have been demonstrated to be vulnerable to adversarial noise, promoting the development of defense against adversarial attacks.
Revised: Deep neural networks have been demonstrated to be vulnerable to adversarial noise, promoting the development of defense against adversarial attacks.

Original: Motivated by the fact that adversarial noise contains well-generalizing features and that the relationship between adversarial data and natural data can help infer natural data and make reliable predictions, in this paper, we study to model adversarial noise by learning the transition relationship between adversarial labels (i.e. the ﬂipped labels used to generate adversarial data) and natural labels (i.e. the ground truth labels of the natural data).
Revised: Motivated by the fact that adversarial noise contains well-generalizing features and that the relationship between adversarial data and natural data can help infer natural data and make reliable predictions, in this paper, we study to model adversarial noise by learning the transition relationship between adversarial labels (i.e. the ﬂipped labels used to generate adversarial data) and natural labels (i.e. the ground truth labels of the natural data).

Original: Speciﬁcally, we introduce an instance-dependent transition matrix to relate adversarial labels and natural labels, which can be seamlessly embedded with the target model (enabling us to model stronger adaptive adversarial noise).
Revised: Speciﬁcally, we introduce an instance-dependent transition matrix to relate adversarial labels and natural labels, which can be seamlessly embedded with the target model (enabling us to model stronger adaptive adversarial noise).

Original: Empirical evaluations demonstrate that our method could effectively improve adversarial accuracy.
Revised: Empirical evaluations demonstrate that our method could effectively improve adversarial accuracy.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep neural networks have been demonstrated to be vulnerable to adversarial noise (Goodfellow et al., 2015; Szegedy et al., 2014;
Revised: Deep neural networks have been demonstrated to be vulnerable to adversarial noise (Goodfellow et al., 2015; Szegedy et al., 2014;

Original: Jin et al., 2019; Liao et al., 2018; Ma et al., 2018; Wu et al., 2020a).
Revised: Jin et al., 2019; Liao et al., 2018; Ma et al., 2018; Wu et al., 2020a).

Original: The vulnerability of deep neural networks seriously threatens many decision-critical deep learning applications (LeCun et al., 1998;
Revised: The vulnerability of deep neural networks seriously threatens many decision-critical deep learning applications (LeCun et al., 1998;

Original: He et al., 2016; Zagoruyko & Komodakis, 2016; Simonyan &
Revised: He et al., 2016; Zagoruyko & Komodakis, 2016; Simonyan &

Original: Zisserman, 2015; Kaiming et al., 2017; Ma et al., 2021).
Revised: Zisserman, 2015; Kaiming et al., 2017; Ma et al., 2021).

Original: To alleviate the negative affects caused by adversarial noise, a number of adversarial defense methods have been proposed.
Revised: To alleviate the negative affects caused by adversarial noise, a number of adversarial defense methods have been proposed.

Original: A major class of adversarial defense methods focus on exploiting adversarial examples to help train the target model (Madry et al., 2018;
Revised: A major class of adversarial defense methods focus on exploiting adversarial examples to help train the target model (Madry et al., 2018;

Original: Ding et al., 2019; Zhang et al., 2019;
Revised: Ding et al., 2019; Zhang et al., 2019;

Original: Wang et al., 2019), which achieve the state-of-the-art performance.
Revised: Wang et al., 2019), which achieve the state-of-the-art performance.

Original: However, these methods do not explicitly model the adversarial noise.
Revised: However, these methods do not explicitly model the adversarial noise.

Original: The relationship between the adversarial data and natural data has not been well studied yet.
Revised: The relationship between the adversarial data and natural data has not been well studied yet.

Original: Studying (or modeling) the relationship between adversarial data and natural data is considered to be beneﬁcial.
Revised: Studying (or modeling) the relationship between adversarial data and natural data is considered to be beneﬁcial.

Original: If we can model the relationship, we can infer natural data information by exploiting the adversarial data and the relationship.
Revised: If we can model the relationship, we can infer natural data information by exploiting the adversarial data and the relationship.

Original: Previous researches have made some explorations on this idea.
Revised: Previous researches have made some explorations on this idea.

Original: Some data pre-processing based methods (Jin et al., 2019; Liao et al., 2018; Naseer et al., 2020; Zhou et al., 2021), which try to recover the natural data by removing the adversarial noise, share the same philosophy.
Revised: Some data pre-processing based methods (Jin et al., 2019; Liao et al., 2018; Naseer et al., 2020; Zhou et al., 2021), which try to recover the natural data by removing the adversarial noise, share the same philosophy.

Original: However, those methods suffer from the high dimensionality problem because both the adversarial data and natural data are high dimensional.
Revised: However, those methods suffer from the high dimensionality problem because both the adversarial data and natural data are high dimensional.

Original: The recovered data is likely to have human-observable loss (i.e., obvious inconsistency between processed examples and natural examples)
Revised: The recovered data is likely to have human-observable loss (i.e., obvious inconsistency between processed examples and natural examples)

Original: (Xu et al., 2017) or contain residual adversarial noise (Liao et al., 2018).
Revised: (Xu et al., 2017) or contain residual adversarial noise (Liao et al., 2018).

Original: To avoid the above problems, in this paper, we propose to model adversarial noise in the lowdimensional label space.
Revised: To avoid the above problems, in this paper, we propose to model adversarial noise in the lowdimensional label space.

Original: Speciﬁcally, instead of directly modeling the relationship between the adversarial data and the natural data, we model the adversarial noise by learning the label transition from the adversarial labels (i.e., the ﬂipped labels used to generate adversarial data) to the natural labels (i.e., the ground truth labels of the natural data).
Revised: Speciﬁcally, instead of directly modeling the relationship between the adversarial data and the natural data, we model the adversarial noise by learning the label transition from the adversarial labels (i.e., the ﬂipped labels used to generate adversarial data) to the natural labels (i.e., the ground truth labels of the natural data).

Original: Note that the adversarial labels have not been well exploited in the community of adversarial learning, which guide the generation of adversarial noise and thus contain valuable information for modeling the well-generalizing features of the adversarial noise (Ilyas et al., 2019).
Revised: Note that the adversarial labels have not been well exploited in the community of adversarial learning, which guide the generation of adversarial noise and thus contain valuable information for modeling the well-generalizing features of the adversarial noise (Ilyas et al., 2019).

Original: It is well-known that the adversarial noise depends on many factors, e.g., the data distribution, the adversarial attack strategy, and the target model.
Revised: It is well-known that the adversarial noise depends on many factors, e.g., the data distribution, the adversarial attack strategy, and the target model.

Original: The proposed adversarial noise modeling method is capable to take into account of the factors because that the label transition is dependent of the adversarial instance (enabling us to model how the patterns of data distribution and adversarial noise affect label ﬂipping) and that the transition can be seamlessly embedded with the target model (enabling us to model stronger adaptive adversarial noise).
Revised: The proposed adversarial noise modeling method is capable to take into account of the factors because that the label transition is dependent of the adversarial instance (enabling us to model how the patterns of data distribution and adversarial noise affect label ﬂipping) and that the transition can be seamlessly embedded with the target model (enabling us to model stronger adaptive adversarial noise).

Original: Speciﬁcally, we employ a deep neural network to learn the complex instance-dependent label transition.
Revised: Speciﬁcally, we employ a deep neural network to learn the complex instance-dependent label transition.

Original: By using the transition relationship, we propose a defense method based on Modeling Adversarial
Revised: By using the transition relationship, we propose a defense method based on Modeling Adversarial

Original: Noise (called MAN).
Revised: Noise (called MAN).

Original: Speciﬁcally, we embed a label transition network (i.e., a matrix-valued transition function parameterized by a deep neural network) into the target model, which denotes the transition relationship from mixture labels (mixture of adversarial labels and natural labels) to natural labels (i.e., the ground truth labels of natural instances and adversarial instances).
Revised: Speciﬁcally, we embed a label transition network (i.e., a matrix-valued transition function parameterized by a deep neural network) into the target model, which denotes the transition relationship from mixture labels (mixture of adversarial labels and natural labels) to natural labels (i.e., the ground truth labels of natural instances and adversarial instances).

Original: The transition matrix explicitly models adversarial noise and help us to infer natural labels.
Revised: The transition matrix explicitly models adversarial noise and help us to infer natural labels.

Original: Considering that adversarial data can be adaptively generated, we conduct joint adversarial training on the target model and the transition network to achieve an optimal adversarial accuracy.
Revised: Considering that adversarial data can be adaptively generated, we conduct joint adversarial training on the target model and the transition network to achieve an optimal adversarial accuracy.

Original: We empirically show that the proposed
Revised: We empirically show that the proposed

Original: MAN based defense method could provide signiﬁcant gains in the classiﬁcation accuracy against adversarial attacks in comparison to state-of-the-art defense methods.
Revised: MAN based defense method could provide signiﬁcant gains in the classiﬁcation accuracy against adversarial attacks in comparison to state-of-the-art defense methods.

Original: The rest of this paper is organized as follows.
Revised: The rest of this paper is organized as follows.

Original: In Section 2, we introduce some preliminary information and brieﬂy review related work on attacks and defenses.
Revised: In Section 2, we introduce some preliminary information and brieﬂy review related work on attacks and defenses.

Original: In Section 3, we discuss how to design an adversarial defense by modeling adversarial noise.
Revised: In Section 3, we discuss how to design an adversarial defense by modeling adversarial noise.

Original: Experimental results are provided in Section 4.
Revised: Experimental results are provided in Section 4.

Original: Finally, we conclude this paper in Section 5.
Revised: Finally, we conclude this paper in Section 5.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: In this section, we ﬁrst introduce some preliminary about notation and the problem setting.
Revised: In this section, we ﬁrst introduce some preliminary about notation and the problem setting.

Original: We then review the most relevant literature on adversarial attacks and adversarial defenses.
Revised: We then review the most relevant literature on adversarial attacks and adversarial defenses.

Original: Notation.
Revised: Notation.

Original: We use capital letters such as X and Y to represent random variables, and lower-case letters such as x and y to represent realizations of random variables X and Y , respectively.
Revised: We use capital letters such as X and Y to represent random variables, and lower-case letters such as x and y to represent realizations of random variables X and Y , respectively.

Original: For norms, we denote by (cid:107) x (cid:107) a generic norm.
Revised: For norms, we denote by (cid:107) x (cid:107) a generic norm.

Original: Speciﬁc examples of norms include (cid:107) x (cid:107) ∞ , the L ∞ norm of x , and (cid:107) x (cid:107) 2 , the L 2 norm of x .
Revised: Speciﬁc examples of norms include (cid:107) x (cid:107) ∞ , the L ∞ norm of x , and (cid:107) x (cid:107) 2 , the L 2 norm of x .

Original: Let B ( x, (cid:15) ) represent the neighborhood of x : { ˜ x : (cid:107) ˜ x − x (cid:107) ≤ (cid:15) } , where (cid:15) is the perturbation budget.
Revised: Let B ( x, (cid:15) ) represent the neighborhood of x : { ˜ x : (cid:107) ˜ x − x (cid:107) ≤ (cid:15) } , where (cid:15) is the perturbation budget.

Original: We deﬁne the classiﬁcation function as f : X → { 1 , 2 , . . .
Revised: We deﬁne the classiﬁcation function as f : X → { 1 , 2 , . . .

Original: , C } .
Revised: , C } .

Original: It can be parametrized, e.g., by deep neural networks.
Revised: It can be parametrized, e.g., by deep neural networks.

Original: Problem setting.
Revised: Problem setting.

Original: This paper focuses on a classiﬁcation task under the adversarial environment, where adversarial attacks are utilized to craft adversarial noise to mislead the predictions of the target classiﬁcation model.
Revised: This paper focuses on a classiﬁcation task under the adversarial environment, where adversarial attacks are utilized to craft adversarial noise to mislead the predictions of the target classiﬁcation model.

Original: Let X
Revised: Let X

Original: and Y be the variables for natural instances and natural labels (i.e., the ground truth labels of natural instances) respectively.
Revised: and Y be the variables for natural instances and natural labels (i.e., the ground truth labels of natural instances) respectively.

Original: We sample natural data { (˜ x i ,˜ y i ) } ni =
Revised: We sample natural data { ( x i , y i ) } ni =

Original: according to the distribution of the variables ( X, Y ) , where ( X, Y ) ∈ X × { 1 , 2 , . . . , C } .
Revised: according to the distribution of the variables ( X, Y ) , where ( X, Y ) ∈ X × { 1 , 2 , . . . , C } .

Original: Given a deep learning model based classiﬁer f and a pair of natural data ( x, y ) , the adversarial instance ˜ x satisﬁes the following constraint:
Revised: Given a deep learning model based classiﬁer f and a pair of natural data ( x, y ) , the adversarial instance ˜ x satisﬁes the following constraint:

Original: [Equation]
Revised: [Equation]

Original: The generation of the adversarial instance is guided by the adversarial label ˜
Revised: The generation of the adversarial instance is guided by the adversarial label ˜

Original: y which is different with the natural label y .
Revised: y which is different with the natural label y .

Original: Adversarial labels can be set by the attacker in target attacks, or be randomly initialized and then found by non-target attacks.
Revised: Adversarial labels can be set by the attacker in target attacks, or be randomly initialized and then found by non-target attacks.

Original: Let (cid:101) X and (cid:101)
Revised: Let (cid:101) X and (cid:101)

Original: Y be the variables for adversarial instances and adversarial labels respectively.
Revised: Y be the variables for adversarial instances and adversarial labels respectively.

Original: We denote by { (˜ x i , ˜ y i ) } ni =1 the adversarial data drawn according to a distribution of the variables ( (cid:101) X, (cid:101) Y ) .
Revised: We denote by { (˜ x i , ˜ y i ) } ni =1 the adversarial data drawn according to a distribution of the variables ( (cid:101) X, (cid:101) Y ) .

Original: Our aim is to design an adversarial defense to correct the adversarial label ˜
Revised: Our aim is to design an adversarial defense to correct the adversarial label ˜

Original: y into natural label y by modeling the relationship between adversarial data (i.e., ˜ x or ˜ y ) and natural data (i.e., x or y ).
Revised: y into natural label y by modeling the relationship between adversarial data (i.e., ˜ x or ˜ y ) and natural data (i.e., x or y ).

Original: Adversarial attacks.
Revised: Adversarial attacks.

Original: Adversarial examples are inputs maliciously designed by adversarial attacks to mislead deep neural networks (Szegedy et al., 2014).
Revised: Adversarial examples are inputs maliciously designed by adversarial attacks to mislead deep neural networks (Szegedy et al., 2014).

Original: They are generated by adding imperceptible but adversarial noise to natural examples.
Revised: They are generated by adding imperceptible but adversarial noise to natural examples.

Original: Adversarial noise can be crafted by multi-step attacks following the direction of adversarial gradients, such as PGD (Madry et al., 2018) and AA (Croce & Hein, 2020).
Revised: Adversarial noise can be crafted by multi-step attacks following the direction of adversarial gradients, such as PGD (Madry et al., 2018) and AA (Croce & Hein, 2020).

Original: These adversarial noise is bounded by a small norm-ball (cid:107) · (cid:107) p ≤ (cid:15) , so that their adversarial examples can be perceptually similar to natural examples.
Revised: These adversarial noise is bounded by a small norm-ball (cid:107) · (cid:107) p ≤ (cid:15) , so that their adversarial examples can be perceptually similar to natural examples.

Original: Optimization-based attacks such as CW (Carlini & Wagner, 2017b) and DDN (Rony et al., 2019) jointly minimize the perturbation L 2 and a differentiable loss based on the logit output of a classiﬁer.
Revised: Optimization-based attacks such as CW (Carlini & Wagner, 2017b) and DDN (Rony et al., 2019) jointly minimize the perturbation L 2 and a differentiable loss based on the logit output of a classiﬁer.

Original: Some attacks such as FWA (Wu et al., 2020b) and STA (Xiao et al., 2018) focus on mimicking non-suspicious vandalism by exploiting the geometry and spatial information.
Revised: Some attacks such as FWA (Wu et al., 2020b) and STA (Xiao et al., 2018) focus on mimicking non-suspicious vandalism by exploiting the geometry and spatial information.

Original: Adversarial defenses.
Revised: Adversarial defenses.

Original: The issue of adversarial examples promotes the development of adversarial defenses.
Revised: The issue of adversarial examples promotes the development of adversarial defenses.

Original: A major class of adversarial defense methods is devoted to enhance the adversarial robustness in an adversarial training manner (Madry et al., 2018;
Revised: A major class of adversarial defense methods is devoted to enhance the adversarial robustness in an adversarial training manner (Madry et al., 2018;

Original: Ding et al., 2019; Zhang et al., 2019; Wang et al., 2019).
Revised: Ding et al., 2019; Zhang et al., 2019; Wang et al., 2019).

Original: They augment training data with adversarial examples and use a min-max formulation to train the target model (Madry et al., 2018).
Revised: They augment training data with adversarial examples and use a min-max formulation to train the target model (Madry et al., 2018).

Original: However, these methods do not explicitly model the adversarial noise.
Revised: However, these methods do not explicitly model the adversarial noise.

Original: The relationship between the adversarial data and natural data has not been well studied yet.
Revised: The relationship between the adversarial data and natural data has not been well studied yet.

Original: In addition, some data pre-processing based methods try to remove adversarial noise by learning a denoising function or a feature-squeezing function.
Revised: In addition, some data pre-processing based methods try to remove adversarial noise by learning a denoising function or a feature-squeezing function.

Original: For example, denoising based defenses (Liao et al., 2018; Naseer et al., 2020) transfer adversarial examples into clean examples, and feature squeezing based defenses (Guo et al., 2018) aim to reduce redundant but adversarial information.
Revised: For example, denoising based defenses (Liao et al., 2018; Naseer et al., 2020) transfer adversarial examples into clean examples, and feature squeezing based defenses (Guo et al., 2018) aim to reduce redundant but adversarial information.

Original: However, these methods suffer from the high dimensionality problem because both the adversarial data and natural data are high dimensional.
Revised: However, these methods suffer from the high dimensionality problem because both the adversarial data and natural data are high dimensional.

Original: For examples, the recovered data is likely to have human-observable loss (Xu et al., 2017)
Revised: For examples, the recovered data is likely to have human-observable loss (Xu et al., 2017)

Original: (i.e., the obvious inconsistency between processed examples and natural examples), and the recovered data may contain residual adversarial noise (Liao et al., 2018), which would be ampliﬁed by the target model and mislead the ﬁnal prediction.
Revised: (i.e., the obvious inconsistency between processed examples and natural examples), and the recovered data may contain residual adversarial noise (Liao et al., 2018), which would be ampliﬁed by the target model and mislead the ﬁnal prediction.

Original: To avoid the above problems, we propose to model adversarial noise in the low-dimensional label space.
Revised: To avoid the above problems, we propose to model adversarial noise in the low-dimensional label space.

Original:  M ODELING ADVERSARIAL NOISE BASED DEFENSE
Revised:  M ODELING ADVERSARIAL NOISE BASED DEFENSE

Original: In this section, we present the proposed Modeling Adversarial Noise (MAN) based defense method, to improve the adversarial accuracy against adversarial attacks.
Revised: In this section, we present the proposed Modeling Adversarial Noise (MAN) based defense method, to improve the adversarial accuracy against adversarial attacks.

Original: We ﬁrst illustrate the motivation of the proposed defense method (Section 3.1).
Revised: We ﬁrst illustrate the motivation of the proposed defense method (Section 3.1).

Original: Next, we introduce how to model adversarial noise (Section 3.2).
Revised: Next, we introduce how to model adversarial noise (Section 3.2).

Original: Finally, we present the training process of the proposed adversarial defense (Section 3.3).
Revised: Finally, we present the training process of the proposed adversarial defense (Section 3.3).

Original: 3. M OTIVATION
Revised: 3. M OTIVATION

Original: Studying the relationship between adversarial data and natural data is considered to be beneﬁcial for adversarial defense.
Revised: Studying the relationship between adversarial data and natural data is considered to be beneﬁcial for adversarial defense.

Original: If we can model the relationship, we can infer clean data information by exploiting the adversarial data and the relationship.
Revised: If we can model the relationship, we can infer clean data information by exploiting the adversarial data and the relationship.

Original: Processing the input data to remove the adversarial noise is a representative strategy to estimate the relationship.
Revised: Processing the input data to remove the adversarial noise is a representative strategy to estimate the relationship.

Original: However, data pre-processing based methods may suffer from high dimensionality problems.
Revised: However, data pre-processing based methods may suffer from high dimensionality problems.

Original: To avoid the problem, we would like to model adversarial noise in the low-dimensional label space.
Revised: To avoid the problem, we would like to model adversarial noise in the low-dimensional label space.

Original: Adversarial noise can be modeled because it have imperceptible but well-generalizing features.
Revised: Adversarial noise can be modeled because it have imperceptible but well-generalizing features.

Original: Speciﬁcally, classiﬁers are usually trained to solely maximize accuracy for natural data.
Revised: Speciﬁcally, classiﬁers are usually trained to solely maximize accuracy for natural data.

Original: They tend to use any available signal including those that are well-generalizing, yet brittle.
Revised: They tend to use any available signal including those that are well-generalizing, yet brittle.

Original: Adversarial noise can arise as a result of perturbing these features (Ilyas et al., 2019), and it controls the ﬂip from natural labels to adversarial labels.
Revised: Adversarial noise can arise as a result of perturbing these features (Ilyas et al., 2019), and it controls the ﬂip from natural labels to adversarial labels.

Original: Thus, adversarial noise contains well-generalizing features which can be modeled by learning the label transition from the adversarial labels to natural labels .
Revised: Thus, adversarial noise contains well-generalizing features which can be modeled by learning the label transition from the adversarial labels to natural labels .

Original: Note that the adversarial noise depends on many factors, such as data distribution, the adversarial attack strategy, and the target model.
Revised: Note that the adversarial noise depends on many factors, such as data distribution, the adversarial attack strategy, and the target model.

Original: Modeling adversarial noise in label space is capable to take into account of these factors.
Revised: Modeling adversarial noise in label space is capable to take into account of these factors.

Original: Speciﬁcally, since that the label transition is dependent of the adversarial instance, we can model how the patterns of data distribution and adversarial noise affect label ﬂipping, and exploit the modeling to correct the ﬂipping of the adversarial label.
Revised: Speciﬁcally, since that the label transition is dependent of the adversarial instance, we can model how the patterns of data distribution and adversarial noise affect label ﬂipping, and exploit the modeling to correct the ﬂipping of the adversarial label.

Original: In addition, we can model adversarial noise crafted by the stronger white-box adaptive attack, because the label transition can be seamlessly embedded with the target model.
Revised: In addition, we can model adversarial noise crafted by the stronger white-box adaptive attack, because the label transition can be seamlessly embedded with the target model.

Original: By jointly training the transition matrix with the target model, it can also be adaptive to adversarial attacks.
Revised: By jointly training the transition matrix with the target model, it can also be adaptive to adversarial attacks.

Original: We employ a deep neural network to learn the complex label transition from the well-generalizing and misleadingly predictive features of instance-dependent adversarial noise.
Revised: We employ a deep neural network to learn the complex label transition from the well-generalizing and misleadingly predictive features of instance-dependent adversarial noise.

Original: Motivated by the value of modeling adversarial noise for adversarial defense, we propose a defense method based on Modeling Adversarial Noise (MAN) by exploiting label transition relationship.[Figure] × = [Figure]
Revised: Motivated by the value of modeling adversarial noise for adversarial defense, we propose a defense method based on Modeling Adversarial Noise (MAN) by exploiting label transition relationship.

Original: Transition network

Original: [Figure]
Revised: [Figure]

Original: 3. M ODELING ADVERSARIAL NOISE
Revised: 3. M ODELING ADVERSARIAL NOISE

Original: We exploit a transition network to learn the label transition for modeling adversarial noise.
Revised: We exploit a transition network to learn the label transition for modeling adversarial noise.

Original: The transition network can be regarded as a matrix-valued transition function parameterized by a deep neural network.
Revised: The transition network can be regarded as a matrix-valued transition function parameterized by a deep neural network.

Original: The transition matrix, estimated by the label transition network, explicitly models adversarial noise and help us to infer natural labels from adversarial labels.
Revised: The transition matrix, estimated by the label transition network, explicitly models adversarial noise and help us to infer natural labels from adversarial labels.

Original: The details are discussed below.
Revised: The details are discussed below.

Original: Transition matrix.
Revised: Transition matrix.

Original: To model the adversarial noise, we need to relate adversarial labels and natural labels in a explicit form (e.g., a matrix).
Revised: To model the adversarial noise, we need to relate adversarial labels and natural labels in a explicit form (e.g., a matrix).

Original: Inspired by recent researches in label-noise learning (Xia et al., 2020; Liu & Tao, 2015; Xia et al., 2019; Yang et al., 2021;
Revised: Inspired by recent researches in label-noise learning (Xia et al., 2020; Liu & Tao, 2015; Xia et al., 2019; Yang et al., 2021;

Original: Wu et al., 2021; Xia et al., 2021), we design a label transition matrix, which can encode the probabilities that adversarial labels ﬂip into natural labels.
Revised: Wu et al., 2021; Xia et al., 2021), we design a label transition matrix, which can encode the probabilities that adversarial labels ﬂip into natural labels.

Original: We then can infer natural labels by utilizing the transition matrix and adversarial data (see Figure 1(a)).
Revised: We then can infer natural labels by utilizing the transition matrix and adversarial data (see Figure 1(a)).

Revised: Note that the transition matrix used in our work is different from that used in label-noise learning.

Revised: The detailed illustration can be found in Appendix A

Original: Considering that the adversarial noise is instance-dependent, the transition matrix should be designed to depend on input instances, that is, we should ﬁnd a transition matrix speciﬁc to each input instance.
Revised: Considering that the adversarial noise is instance-dependent, the transition matrix should be designed to depend on input instances, that is, we should ﬁnd a transition matrix speciﬁc to each input instance.

Original: In addition, we not only need to focus on the accuracy of adversarial instances, but also the accuracy of natural instances.
Revised: In addition, we not only need to focus on the accuracy of adversarial instances, but also the accuracy of natural instances.

Original: An adversarially robust target model is often trained with both natural data and adversarial data.
Revised: An adversarially robust target model is often trained with both natural data and adversarial data.

Original: We therefore exploit the mixture of natural and adversarial instances (called mixture instances) as input instances, and design the transition matrix to model the relationship between the mixture of natural and adversarial labels (called mixture labels) to the ground-truth labels of mixture instances (called natural labels).
Revised: We therefore exploit the mixture of natural and adversarial instances (called mixture instances) as input instances, and design the transition matrix to model the relationship between the mixture of natural and adversarial labels (called mixture labels) to the ground-truth labels of mixture instances (called natural labels).

Original: Speciﬁcally, let X (cid:48) denote the variable for the mixture instances, Y (cid:48) denote the variable for the natural labels, and Y denote the variable for the natural labels.
Revised: Speciﬁcally, let X (cid:48) denote the variable for the mixture instances, Y (cid:48) denote the variable for the mixture labels, and Y denote the variable for the natural labels.

Original: We combine the natural data and adversarial data as the mixture data, i.e., { ( x (cid:48)
Revised: We combine the natural data and adversarial data as the mixture data, i.e., { ( x (cid:48)

Original: i , y (cid:48) i ) } 2 ni =

Revised: i , y (cid:48) i ) } 2 ni =

Original: = { ( x i , y i ) } ni =1 ∪ { ( ˜ x i , ˜ y i ) } ni =1 , where { ( x (cid:48)
Revised: = { ( x i , y i ) } ni =1 ∪ { ( ˜ x i , ˜ y i ) } ni =1 , where { ( x (cid:48)

Original: i , y (cid:48) i ) } 2 ni = is the data drawn according to a distribution of the random variables ( X (cid:48) , Y (cid:48) ) .
Revised: i , y (cid:48) i ) } 2 ni =1 is the data drawn according to a distribution of the random variables ( X (cid:48) , Y (cid:48) ) .

Original: Since we have the ground-truth labels of mixture instances (i.e., natural labels), we can extend the mixture data { ( x (cid:48)
Revised: Since we have the ground-truth labels of mixture instances (i.e., natural labels), we can extend the mixture data { ( x (cid:48)

Original: i , y (cid:48) i ) } 2 ni =1 into a triplet form { ( x (cid:48)
Revised: i , y (cid:48) i ) } 2 ni =1 into a triplet form { ( x (cid:48)

Original: i , y (cid:48)
Revised: i , y (cid:48)

Original: i , y i ) } 2 ni =1 .
Revised: i , y i ) } 2 ni =1 .

Original: We utilize the transition matrix T ∈ [0 , 1] C × C to model the relationship between mixture labels Y (cid:48) and natural labels Y .
Revised: We utilize the transition matrix T ∈ [0 , 1] C × C to model the relationship between mixture labels Y (cid:48) and natural labels Y .

Original: The transition matrix T is dependent of the mixture instances X (cid:48) .
Revised: The transition matrix T is dependent of the mixture instances X (cid:48) .

Original: It is deﬁned as:
Revised: It is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where T i,j denotes the ( i, j ) − th element of the matrix T ( X (cid:48)
Revised: where T i,j denotes the ( i, j ) − th element of the matrix T ( X (cid:48)

Original: =
Revised: =

Original: x (cid:48) ) .
Revised: x (cid:48) ) .

Original: It indicates the probability of the mixture label i ﬂipped to the natural label j for the input x (cid:48) .
Revised: It indicates the probability of the mixture label i ﬂipped to the natural label j for the input x (cid:48) .

Original: The basic idea is that given the mixture class posterior probability (i.e., the class posterior probability of the mixture labels Y (cid:48) )
Revised: The basic idea is that given the mixture class posterior probability (i.e., the class posterior probability of the mixture labels Y (cid:48) )

Original: P ( Y (cid:48)
Revised: P ( Y [Equation]

Original: | X (cid:48) =
Revised: | X [Equation]

Original: x (cid:48) )
Revised: x [Equation]

Original: =
Revised: =

Original: [ P ( Y (cid:48)
Revised: [ P ( Y [Equation]

Original: = 1 | X (cid:48)
Revised: 1 | X (cid:48)

Original: =
Revised: =

Original: x (cid:48) ) , . .
Revised: x (cid:48) ) , . .

Original: , P ( Y (cid:48) =
Revised: , P ( Y (cid:48) =

Original: C | X (cid:48)
Revised: C | X (cid:48)

Original: =
Revised: =

Original: x (cid:48) )] (cid:62) , the natural class posterior probability (i.e., the class posterior probability of natural labels Y )
Revised: x (cid:48) )] (cid:62) , the natural class posterior probability (i.e., the class posterior probability of natural labels Y )

Original: P ( Y | X (cid:48)
Revised: P ( Y | X (cid:48)

Original: =
Revised: =

Original: x (cid:48) ) could be inferred by exploiting P ( Y (cid:48) | X (cid:48)
Revised: x (cid:48) ) could be inferred by exploiting P ( Y (cid:48) | X (cid:48)

Original: =
Revised: =

Original: x (cid:48) ) and T ( X (cid:48)
Revised: x (cid:48) ) and T ( X (cid:48)

Original: =
Revised: =

Original: x (cid:48) ) :
Revised: x (cid:48) ) :

Original: [Equation]
Revised: [Equation]

Original: We then can obtain the natural labels by choose the class labels that maximize the robust class posterior probabilities.
Revised: We then can obtain the natural labels by choose the class labels that maximize the robust class posterior probabilities.

Original: Note that the mixture class posterior probability P ( Y (cid:48) | X (cid:48)
Revised: Note that the mixture class posterior probability P ( Y (cid:48) | X (cid:48)

Original: =
Revised: =

Original: x (cid:48) ) can be estimated by exploiting the mixture data.[Figure] ( 𝑥𝑥 ′ ; 𝜔𝜔 ) [Figure]
Revised: x (cid:48) ) can be estimated by exploiting the mixture data.

Original: 𝒈𝒈 𝝎𝝎

Original: Target model 𝒉𝒉 𝜽𝜽

Original: Transition network 𝒈𝒈 𝝎𝝎

Original: Target model 𝒇𝒇 𝜽𝜽 𝓛𝓛 𝑻𝑻 / 𝓛𝓛 𝒕𝒕𝒕𝒕𝒕𝒕

Original: Adversarial noise

Original: Maximize the cross- entropy loss

Original: Iterative adversarial training

Original: Natural instance 𝑥𝑥 + S o ft m ax 𝜹𝜹 ( � ) �𝒚𝒚 𝒚𝒚 �𝒚𝒚 ′

Original: Transition network.
Revised: Transition network.

Original: we employ a deep neural network (called transition network ) to estimate the label transition matrix by exploiting the mixture data { ( x (cid:48)
Revised: we employ a deep neural network (called transition network ) to estimate the label transition matrix by exploiting the mixture data { ( x (cid:48)

Original: i , y (cid:48)
Revised: i , y (cid:48)

Original: i , y i ) } 2 ni =1 .
Revised: i , y i ) } 2 n i =1 .

Original: Speciﬁcally, as shown in
Revised: Speciﬁcally, as shown in

Original:  Figure 1(b), the transition network g ω ( · ) parameterized by ω takes the mixture instance x (cid:48) as input, and output the label transition matrix (cid:98) T ( x (cid:48) ; ω ) =
Revised: [Figure] 𝜹𝜹 ( � ) �𝒚𝒚 𝒚𝒚 �𝒚𝒚 ′ Figure 1(b), the transition network g ω ( · ) parameterized by ω takes the mixture instance x (cid:48) as input, and output the label transition matrix (cid:98) T ( x (cid:48) ; ω ) =

Original: g ω ( x (cid:48) ) .
Revised: g ω ( x (cid:48) ) .

Original: We then can infer the natural class posterior probability P ( Y | X (cid:48)
Revised: We then can infer the natural class posterior probability P ( Y | X (cid:48)

Original: =
Revised: =

Original: x (cid:48) )
Revised: x (cid:48) )

Original: according to Eq. 3, and thus infer the natural label.
Revised: according to Eq. 3, and thus infer the natural label.

Original: To optimize the parameter ω of the transition network, we minimize the difference between the inferred natural labels and the ground-truth natural labels.
Revised: To optimize the parameter ω of the transition network, we minimize the difference between the inferred natural labels and the ground-truth natural labels.

Original: The loss function of the transition network is deﬁne as:
Revised: The loss function of the transition network is deﬁne as:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original:  where y (cid:48)
Revised: (4) where y (cid:48)

Original: i , y i are y (cid:48)
Revised: i , y i are y (cid:48)

Original: i , y i in the form of vectors.
Revised: i , y i in the form of vectors.

Original: (cid:96) ( · ) is the cross-entropy loss between the inferred natural labels and the ground-truth natural labels, i.e., (cid:96) (y (cid:48)
Revised: (cid:96) ( · ) is the cross-entropy loss between the inferred natural labels and the ground-truth natural labels, i.e., (cid:96) (

Original:  i · (cid:98)T ( x (cid:48)
Revised: [Equation] (cid:48) i · (cid:98)

Original: i ; ω ) , y i ) =
Revised: [Equation] [Equation]

Original: y i · log( y (cid:48) 
Revised: (cid:48) i [Equation]

Revised: [Equation]

Original:  i · (cid:98) T ( x (cid:48) 
Revised: ( y (cid:48) i · (cid:98) T ( x (cid:48) i

Revised: [Equation]

Original: i ; ω )) .
Revised: ω )) [Equation]

Original: 3. T RAINING
Revised: 3. T RAINING

Original: Considering that adversarial data can be adaptively generated, and the adversarial labels are also depended on the target model, we conduct joint adversarial training on the target model and the transition network to achieve the optimal adversarial accuracy.
Revised: Considering that adversarial data can be adaptively generated, and the adversarial labels are also depended on the target model, we conduct joint adversarial training on the target model and the transition network to achieve the optimal adversarial accuracy.

Original: We provide the overview of the training procedure in Figure 2.
Revised: We provide the overview of the training procedure in Figure 2.

Original: For the transition network, we optimize its model parameter ω according to Eq.
Revised: For the transition network, we optimize its model parameter ω according to Eq.

Original: For the target model, considering that the ﬁnal inferred natural class posterior probability is inﬂuenced by the target model, we also use the cross-entropy loss between the inferred natural labels and the ground-truth natural labels to optimize the parameter θ .
Revised: For the target model, considering that the ﬁnal inferred natural class posterior probability is inﬂuenced by the target model, we also use the cross-entropy loss between the inferred natural labels and the ground-truth natural labels to optimize the parameter θ .

Original: The loss function for the target model h θ is deﬁned as:
Revised: The loss function for the target model h θ is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: (5) where δ ( · ) denote the softmax function.
Revised: where δ ( · ) denote the softmax function.

Original: ˆ y (cid:48)
Revised: ˆ y (cid:48)

Original: i denote the mixture label in the form of a vector predicted by the target model.
Revised: i denote the mixture label in the form of a vector predicted by the target model.

Original: To demonstrate the effectiveness of joint adversarial training, we conduct an ablation study by independently training the transition network with a ﬁxed pre-trained target model (see Section 4.3).
Revised: To demonstrate the effectiveness of joint adversarial training, we conduct an ablation study by independently training the transition network with a ﬁxed pre-trained target model (see Section 4.3).

Original: In addition, to prove that using mixture instances can achieve better adversarial accuracy compared to only using adversarial instances, we conduct an ablation study by training the proposed defense using only adversarial data (see Section 4.3).
Revised: In addition, to prove that using mixture instances can achieve better adversarial accuracy compared to only using adversarial instances, we conduct an ablation study by training the proposed defense using only adversarial data (see Section 4.3).

Original: The details of the overall procedure are presented in Algorithm 1.
Revised: The details of the overall procedure are presented in Algorithm 1.

Original: Speciﬁcally, for each mini-batch
Revised: Speciﬁcally, for each mini-batch

Original: B = { x i } m i =1 sampled from natural training set, we ﬁrst generate adversarial instances (cid:101) B = { ˜ x i } m i = via a strong adversarial attack algorithm, and obtain the mixture mini-batch B (cid:48)
Revised: B = { x i } mi =1 sampled from natural training set, we ﬁrst generate adversarial instances (cid:101) B = { ˜ x i } mi = via a strong adversarial attack algorithm, and obtain the mixture mini-batch B (cid:48)

Original: = { x (cid:48) i } 2 mi =1 (i.e., mixture of B and (cid:101) B ) with mixture labels { y (cid:48) i } 2 mi =1 (i.e., mixture of adversarial labels { ˜ y i } mi =1 and natural labels { y i } mi =1 ).
Revised: = { x (cid:48) i } 2 mi =1 (i.e., mixture of B and (cid:101) B ) with mixture labels { y (cid:48) i } 2 mi =1 (i.e., mixture of adversarial labels { ˜ y i } mi =1 and natural labels { y i } mi =1 ).

Original: Then, we input the mixture instances { x (cid:48)
Revised: Then, we input the mixture instances { x (cid:48)

Original: i } 2 mi =1 into the target model
Revised: i } 2 mi =1 into the target model

Original: h θ ( · ) and output { ˆ y (cid:48) i } 2 mi =1 .
Revised: h θ ( · ) and output { ˆ y (cid:48) i } 2 m i =1 .

Original: We also input the mixture instances { x (cid:48) 
Revised: We also input the mixture instances { x (cid:48) i }

Original: i } 2 mi =1 into the transition network g ω ( · ) to estimate the label transition matrices { (cid:98) T ( x (cid:48)
Revised: 2 m i =1 into the transition network g ω ( · ) to estimate the label transition matrices { (cid:98) T ( x (cid:48)

Original: i ; ω ) } 2 mi =1 .
Revised: i ; ω ) } 2 mi =1 .

Original: We next infer the ﬁnal prediction labels by exploiting { ˆ y (cid:48) i } 2 mi =1 and { (cid:98) T ( x (cid:48)
Revised: We next infer the ﬁnal prediction labels by exploiting { ˆ y (cid:48) i } 2 mi =1 and { (cid:98) T ( x (cid:48)

Original: i ; ω ) } 2 mi =1 .
Revised: i ; ω ) } 2 mi =1 .

Original: Finally, we compute the loss functions L T ( ω ) and L tar ( θ ) , and update the parameters ω and θ .
Revised: Finally, we compute the loss functions L T ( ω ) and L tar ( θ ) , and update the parameters ω and θ .

Original: By iteratively conducting the procedures of adversarial example generation and defense training, ω and θ are expected to be adversarially optimized.
Revised: By iteratively conducting the procedures of adversarial example generation and defense training, ω and θ are expected to be adversarially optimized.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we ﬁrst introduce the experiment setup including datasets, attack settings and defense settings in Section 4.1.
Revised: In this section, we ﬁrst introduce the experiment setup including datasets, attack settings and defense settings in Section 4.1.

Original: Then, we evaluate the effectiveness of our defense against representative and commonly used L ∞ norm and L 2 norm adversarial attacks in Section 4.2.
Revised: Then, we evaluate the effectiveness of our defense against representative and commonly used L ∞ norm and L 2 norm adversarial attacks in Section 4.2.

Original: Finally, we conduct ablation studies in Section 4.3.
Revised: Finally, we conduct ablation studies in Section 4.3.

Original: 4. E XPERIMENT SETUP
Revised: 4. E XPERIMENT SETUP

Original: Datasets.
Revised: Datasets.

Original: We verify the effective of our defense method on two popular benchmark datasets, i.e.,
Revised: We verify the effective of our defense method on two popular benchmark datasets, i.e.,

Original: CIFAR-10 (Krizhevsky et al., 2009) and Tiny-ImageNet (Wu et al., 2017).
Revised: CIFAR-10 (Krizhevsky et al., 2009) and Tiny-ImageNet (Wu et al., 2017).

Original: CIFAR-10 has 10 classes of images including 50,000 training images and 10,000 test images.
Revised: CIFAR-10 has 10 classes of images including 50,000 training images and 10,000 test images.

Original: Tiny-ImageNet has 200 classes of images including 100,000 training images, 10,000 validation images and 10,000 test images.
Revised: Tiny-ImageNet has 200 classes of images including 100,000 training images, 10,000 validation images and 10,000 test images.

Original: Images in the three datasets are all regarded as natural instances.
Revised: Images in the two datasets are all regarded as natural instances.

Original: All images are normalized into [0,1], and are performed simple data augmentations in the training process, including random crop and random horizontal ﬂip.
Revised: All images are normalized into [0,1], and are performed simple data augmentations in the training process, including random crop and random horizontal ﬂip.

Original: For the target model, we use ResNet-18 (He et al., 2016) for CIFAR-10 and
Revised: For the target model, we use ResNet-18 (He et al., 2016) for CIFAR-10 and

Original: Tiny-ImageNet .
Revised: Tiny-ImageNet .

Original: Attack settings.
Revised: Attack settings.

Original: Adversarial examples for evaluating defense models are crafted by applying stateof-the-art attacks.
Revised: Adversarial examples for evaluating defense models are crafted by applying stateof-the-art attacks.

Original: These attacks include L ∞ norm PGD (Madry et al., 2018), L ∞ norm AA (Croce & Hein, 2020), L ∞ norm TI-DIM (Dong et al., 2019; Xie et al., 2019), L ∞ norm FWA (Wu et al., 2020b), L 2 norm CW (Carlini & Wagner, 2017b) and L 2 norm DDN (Rony et al., 2019).
Revised: These attacks include L ∞ norm PGD (Madry et al., 2018), L ∞ norm AA (Croce & Hein, 2020), L ∞ norm TI-DIM (Dong et al., 2019; Xie et al., 2019), L ∞ norm FWA (Wu et al., 2020b), L 2 norm CW (Carlini & Wagner, 2017b) and L 2 norm DDN (Rony et al., 2019).

Original: Among them, the AA attack algorithm integrates three non-target attacks and a target attack.
Revised: Among them, the AA attack algorithm integrates three non-target attacks and a target attack.

Original: Other attack algorithms belong to non-target attacks.
Revised: Other attack algorithms belong to non-target attacks.

Original: The iteration number of PGD is set to 40 with step size 0.007.
Revised: The iteration number of PGD is set to 40 with step size 0.007.

Original: The iteration number of FWA is set to 20 with step size 0.007.
Revised: The iteration number of FWA is set to 20 with step size 0.007.

Original: The iteration numbers of CW 2 and
Revised: The iteration numbers of CW 2 and

Original: DDN are set to 200 and 40 respectively with step size 0.01.
Revised: DDN are set to 200 and 40 respectively with step size 0.01.

Original: For CIFAR=10 and Tiny-ImageNet , the perturbation budgets for L 2 norm attacks and L ∞ norm attacks are (cid:15) = 0 .
Revised: For CIFAR=10 and Tiny-ImageNet , the perturbation budgets for L 2 norm attacks and L ∞ norm attacks are (cid:15) = 0 .

Original: 5 and 8 / 255 respectively.
Revised: 5 and 8 / 255 respectively.

Original: Defense settings.
Revised: Defense settings.

Original: We use  the  adversarial training (AT) (Madry et al., 2018) as the baseline.
Revised: We use three representative defense methods as the baselines: standard adversarial training AT (Madry et al., 2018), TRADES (Zhang et al., 2019) and MART (Wang et al., 2019).

Original: We use the ResNet-18 as the transition network for CIFAR-10 and Tiny-ImageNet .
Revised: We use the ResNet-18 as the transition network for CIFAR-10 and Tiny-ImageNet .

Original: The perturbation budget (cid:15) is set to 8 / 255 for CIFAR-10 and Tiny-ImageNet .
Revised: The perturbation budget (cid:15) is set to 8 / 255 for CIFAR-10 and Tiny-ImageNet .

Original: The epoch number is set to 150 for our defense, and 100 for baselines.
Revised: The epoch number is set to 150 for our defense, and 100 for baselines.

Original: For fair comparisons, all the methods are trained using SGD with momentum 0.9 and an initial learning rate of 0.1.
Revised: For fair comparisons, all the methods are trained using SGD with momentum 0.9 and an initial learning rate of 0.1.

Original: The weight decay is 2 × 10 − 4 for CIFAR-10 , and is 5 × 10 −  for Tiny-ImageNet .
Revised: The weight decay is 2 × 10 − 4 for CIFAR-10 , and is 5 × 10 − 4 for Tiny-ImageNet .

Original:  We use the L ∞ norm non-target PGD-10 (i.e., PGD with iteration number of 10) with random start and step size (cid:15)/ 4 to craft adversarial training data for all defensesaccording to the uniﬁed conﬁguration of baselines.
Revised: For TRADES and MART, we set λ = 6 . We use the L ∞ norm non-target PGD-10 (i.e., PGD with iteration number of 10) with random start and step size (cid:15)/ 4 to craft adversarial training data for all defenses

Original: 4. D EFENSE EFFECTIVENESS
Revised: 4. D EFENSE EFFECTIVENESS

Original: Defending against adaptive attacks.
Revised: Defending against adaptive attacks.

Original: A powerful adaptive attack strategy has been proposed to break defense methods (Athalye et al., 2018; Carlini & Wagner, 2017a).
Revised: A powerful adaptive attack strategy has been proposed to break defense methods (Athalye et al., 2018; Carlini & Wagner, 2017a).

Original: In this case, the attacker can access the architecture and model parameters of both the target model and the defense model (i.e., the transition network).
Revised: In this case, the attacker can access the architecture and model parameters of both the target model and the defense model (i.e., the transition network).

Original: We study the following attack scenarios for evaluating our defense.
Revised: We study the following attack scenarios for evaluating our defense.

Original: Scenario (i).
Revised: Scenario (i).

Original: Considering that the models in baselines are completely leaked to the attacker, for fair comparison, we utilize white-box adversarial attacks against the combination of the target model and the transition matrix.
Revised: Considering that the models in baselines are completely leaked to the attacker, for fair comparison, we utilize white-box adversarial attacks against the combination of the target model and the transition matrix.

Original: Similar to attacks against baselines, the goal of the non-target attack in this scenario is to maximize the distances between ﬁnal predictions of our defense and the ground-truth natural labels.
Revised: Similar to attacks against baselines, the goal of the non-target attack in this scenario is to maximize the distances between ﬁnal predictions of our defense and the ground-truth natural labels.

Original: The adversarial instance ˜ x is crafted by solving the following optimization problem:
Revised: The adversarial instance ˜ x is crafted by solving the following optimization problem:

Original: [Equation]
Revised: [Equation]

Original: where ˜ y
Revised: where ˜ y

Original: =
Revised: =

Original: δ ( h θ (˜ x )) and L ( · ) denote the speciﬁc loss function used by each attack.
Revised: δ ( h θ (˜ x )) and L ( · ) denote the speciﬁc loss function used by each attack.

Original: Similarly, we can craft adversarial examples via the target attack.
Revised: Similarly, we can craft adversarial examples via the target attack.

Original: The details are presented in Appendix A.
Revised: The details are presented in Appendix B.

Original: We combine this attack strategy with six representative adversarial attacks introduced in section attack settings to evaluate defenses.
Revised: We combine this attack strategy with six representative adversarial attacks introduced in section attack settings to evaluate defenses.

Original: For training our defense, We use adversarial examples crafted by adaptive PGD-10 attack as adversarial training data.
Revised: For training our defense, We use adversarial examples crafted by adaptive PGD-10 attack as adversarial training data.

Original: The average natural accuracy (i.e., the results in the second column) and the average adversarial accuracy of defenses are shown in Table 1.
Revised: The average natural accuracy (i.e., the results in the second column) and the average adversarial accuracy of defenses are shown in Table 1.

Original: The results show that our defense achieves superior adversarial accuracy compared withthe baseline AT.
Revised: The results show that our defense achieves superior adversarial accuracy compared with AT.

Original: This presents that our defense is effective.
Revised: This presents that our defense is effective.

Original: In addition, the proposed method achieves signiﬁcant improvements against more destructive FWA, CW 2 and DDN attacks (e.g., the adversarial accuracy is increased from 0.00% to 46.36% against CW 2 ), which represents thatour defense has better generalization compared with the baseline. This may be because that the L 2 norm adversarial noise and the L ∞ norm adversarial noise are obviously different, and the L 2 norm adversarial noise is more imperceptible in human vision.
Revised: In addition, our method achieves signiﬁcant improvements against more destructive FWA, CW 2 and DDN attacks (e.g., the adversarial accuracy is increased from 0.00% to 46.36% against CW 2 ). Note that the training mechanism in our method can be regarded as the standard adversarial training on the combination of the target network and the transition matrix, our method thus is applicable to different adversarial training methods.

Original: The baseline (trained on L ∞ norm adversarial data) thus lack generalization for defending against L 2 norm attacks. Our defense can learn the label transition relationship from imperceptible adversarial noise by the transition network, we thus can infer some natural labels via adversarial labels to improve adversarial accuracy.
Revised: To avoid the bias caused by different adversarial training methods, we apply our method to TRADES and MART. As shown in Table 1, the results show that our method improves the adversarial accuracy.

Original: Note that the training mechanism in our method can be regarded as the adversarial training on the combination of the target network and  thetransition matrix, we thus can apply other adversarial training algorithms to further improve the defense performance against adversarial attacks.
Revised: We ﬁnd that using TRADES and MART would affect the performance against L 2 -norm based attacks.

Original: We would conduct further research in future work.
Revised: We will further solve this issue in future work.

Original: Thedetails and the results on Tiny-ImageNet are shown in 
Revised: The results on Tiny-ImageNet are shown in Appendix B.1. In addition

Original:  Appendix A.1.
Revised: [Table] we evaluate the adversarial accuracy of defense methods against white-box adaptive attacks on CIFAR-10 by using the VggNet-19 as the target model and the transition network. The results are also shown in Appendix B.1.

Original: [Table]
Revised: [Table]

Original: Scenario (ii).
Revised: Scenario (ii).

Original: In this scenario, we design an adversarial attack to destroy the crucial transition matrix of our defense.
Revised: In this scenario, we design an adversarial attack to destroy the crucial transition matrix of our defense.

Original: If the transition matrix is destroyed, the defense would immediately become ineffective.
Revised: If the transition matrix is destroyed, the defense would immediately become ineffective.

Original: Since the ground-truth transition matrix is not given, we use the target attack strategy to craft adversarial examples.
Revised: Since the ground-truth transition matrix is not given, we use the target attack strategy to craft adversarial examples.

Original: We choose an anti-diagonal identity matrix (see Figure 4 in Appendix A.2) as an example of the target transition matrix in the target attack.
Revised: We choose an anti-diagonal identity matrix (see Figure 4 in Appendix B.2) as an example of the target transition matrix in the target attack.

Original: The optimization goal is designed as:
Revised: The optimization goal is designed as:

Original: [Equation]
Revised: [Equation]

Original: where L mse denote the mean square error loss.
Revised: where L mse denote the mean square error loss.

Original: We just showed two examples for non-targeted and targeted attacks designed to the transition network in the above two scenarios.
Revised: We just showed two examples for non-targeted and targeted attacks designed to the transition network in the above two scenarios.

Original: Note that different attacks can be designed, which however is beyond the scope of our work.
Revised: Note that different attacks can be designed, which however is beyond the scope of our work.

Original: We use L ∞ norm PGD-40 with this target attack strategy to evaluate the MAN based defense trained in scenario (i).
Revised: We use L ∞ norm PGD-40 with this target attack strategy to evaluate the MAN based defense trained in scenario (i).

Original: The adversarial accuracy is 70.26% on CIFAR-10 , which represents that our defense is effective against this adaptive target attack.
Revised: The adversarial accuracy is 70.26% on CIFAR-10 , which represents that our defense is effective against this adaptive target attack.

Original: This may be because that the attack in scenario (i) craft adversarial data for reducing the accuracy by destroying the transition matrix as well.
Revised: This may be because that the attack in scenario (i) craft adversarial data for reducing the accuracy by destroying the transition matrix as well.

Original: The transition network adversarially trained on such adversarial data thus have good robustness against the attack designed for the transition matrix.
Revised: The transition network adversarially trained on such adversarial data thus have good robustness against the attack designed for the transition matrix.

Original: Scenario (iii).
Revised: Scenario (iii).

Original: We explore another adaptive attack scenario.
Revised: We explore another adaptive attack scenario.

Original: In this scenario, the attack not only disturbs the ﬁnal prediction labels, but also disturbs the output of the target model.
Revised: In this scenario, the attack not only disturbs the ﬁnal prediction labels, but also disturbs the output of the target model.

Original: This attack is called dual adaptive attack .
Revised: This attack is called dual adaptive attack .

Original: The optimization goal of the non-target attack can be designed as:
Revised: The optimization goal of the non-target attack can be designed as:

Original: [Equation]
Revised: [Equation]

Original: where ˜ y
Revised: where ˜ y

Original: =
Revised: =

Original: δ ( h θ (˜ x )) , L ( · ) denote the speciﬁc loss function used by each adversarial attack and L ce ( · ) is the cross-entropy loss of the target model, i.e. L ce ( ˜ y , y )
Revised: δ ( h θ (˜ x )) , L ( · ) denote the speciﬁc loss function used by each adversarial attack and L ce ( · ) is the cross-entropy loss of the target model, i.e. L ce ( ˜ y , y )

Original: = − 1 n (cid:80)
Revised: = − 1 n (cid:80)

Original: ni
Revised: ni

Original: =

Revised: =

Original: y i · log( ˜ y ) .
Revised: y i · log( ˜ y ) .

Original: Similarly, we can craft adversarial examples via the target attack.
Revised: Similarly, we can craft adversarial examples via the target attack.

Original: The details are presented in Appendix A.3.
Revised: The details are presented in Appendix B.3.

Original: We combine this dual adaptive attack strategy with above six attacks to evaluate the proposed defense trained in scenario (i).
Revised: We combine this dual adaptive attack strategy with above six attacks to evaluate the proposed defense trained in scenario (i).

Original: As shown in Table 2, the results in the second row are the adversarial accuracy of our MAN based defense in this scenario, the results in the third row (Model T ) show that accuracy of the target model for adversarial instances.
Revised: As shown in Table 2, the results in the second row are the adversarial accuracy of our MAN based defense in this scenario, the results in the third row (Model T ) show that accuracy of the target model for adversarial instances.

Original: The results demonstrate that our defense method can infer natural labels by exploiting the instance-dependent transition matrix and adversarial labels, it provides effective protection against multiple attacks in this scenario.
Revised: The results demonstrate that our defense method can infer natural labels by exploiting the instance-dependent transition matrix and adversarial labels, it provides effective protection against multiple attacks in this scenario.

Original: We note that the adversarial accuracy of MAN in Table 2 is higher than that in Table 1.
Revised: We note that the adversarial accuracy of MAN in Table 2 is higher than that in Table 1.

Original: This may be because, for some input instances, the dual adaptive attack mainly uses the gradient information of the attack loss only against the target model, to craft adversarial noise for breaking the target model.
Revised: This may be because, for some input instances, the dual adaptive attack mainly uses the gradient information of the attack loss only against the target model, to craft adversarial noise for breaking the target model.

Original: Our MAN based defense can model the adversarial noise (i.e., learn the transition relationship) for the adversarial label predicted by the target model.
Revised: Our MAN based defense can model the adversarial noise (i.e., learn the transition relationship) for the adversarial label predicted by the target model.

Original: The ﬁnal natural label can be inferred, and the ﬁnal adversarial accuracy thus can be improved.
Revised: The ﬁnal natural label can be inferred, and the ﬁnal adversarial accuracy thus can be improved.

Original: [Table]
Revised: [Table]

Original: Defending against general attacks.
Revised: Defending against general attacks.

Original: We also evaluate the effectiveness of the proposed defense by using general attacks, i.e., the adversarial attacks against the target model.
Revised: We also evaluate the effectiveness of the proposed defense by using general attacks, i.e., the adversarial attacks against the target model.

Original: We utilize above six adversarial attacks to evaluate the proposed MAN based defense.
Revised: We utilize above six adversarial attacks to evaluate the proposed MAN based defense.

Original: We use the adversarial instances crafted by non-target L ∞ norm PGD-10 attack against the target model as the adversarial training data, to train our defense model.
Revised: We use the adversarial instances crafted by non-target L ∞ norm PGD-10 attack against the target model as the adversarial training data, to train our defense model.

Original: The adversarial accuracy of our MAN based defense is shown in Table 2 (MAN (cid:63) ).
Revised: The adversarial accuracy of our MAN based defense is shown in Table 2 (MAN (cid:63) ).

Original: The adversarial accuracy of the target model (Model ∗ T ) is also shown on the ﬁfth row in Table 2.
Revised: The adversarial accuracy of the target model (Model ∗ T ) is also shown on the ﬁfth row in Table 2.

Original: It can be seen that our defense method achieve a great defense effect against adversarial examples which are crafted against the target model.
Revised: It can be seen that our defense method achieve a great defense effect against adversarial examples which are crafted against the target model.

Original: Defense transferability.
Revised: Defense transferability.

Original: The work in Ilyas et al.
Revised: The work in Ilyas et al.

Original: (2019) shows that any two target models are likely to learn similar mixture pattern, so modeling the adversarial noise which manipulate such features would apply to both target models for improving adversarial accuracy.
Revised: (2019) shows that any two target models are likely to learn similar mixture pattern, so modeling the adversarial noise which manipulate such features would apply to both target models for improving adversarial accuracy.

Original: We apply the MAN based defense which is trained for the ResNet-18 (ResNet), to two naturally pre-trained target models
Revised: We apply the MAN based defense which is trained for the ResNet-18 (ResNet), to two naturally pre-trained target models

Original: VGG-19 (VGG) (Simonyan & Zisserman, 2015) and Wide-ResNet 28 × 10 (WRN) (Zagoruyko &
Revised: VGG-19 (VGG) (Simonyan & Zisserman, 2015) and Wide-ResNet 28 × 10 (WRN) (Zagoruyko &

Original: Komodakis, 2016) for evaluating the transferability of our defense.
Revised: Komodakis, 2016) for evaluating the transferability of our defense.

Original: The performance on CIFAR-10 is reported in Table 3.
Revised: The performance on CIFAR-10 is reported in Table 3.

Original: It can be seen that our defense method has effective transferability for providing cross-model protections against adversarial attacks. 
Revised: It can be seen that our defense method has effective transferability for providing cross-model protections against adversarial attacks. In addition, we transfer our defense model to

Revised: VGG-19 for defending against adaptive attacks.

Revised: The details can be found in Appendix B.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: A BLATION STUDY
Revised: A BLATION STUDY

Original: To demonstrate that the joint adversarial training on the target model and the transition network could achieve better defense compare to only training the transition network, we conduct an ablation study on our MAN based defense.
Revised: To demonstrate that the joint adversarial training on the target model and the transition network could achieve better defense compare to only training the transition network, we conduct an ablation study on our MAN based defense.

Original: Speciﬁcally, we use a naturally pre-trained ResNet-18 target model, and train the transition network independently.
Revised: Speciﬁcally, we use a naturally pre-trained ResNet-18 target model, and train the transition network independently.

Original: The model parameters of the target model are ﬁxed in the training procedure.
Revised: The model parameters of the target model are ﬁxed in the training procedure.

Original: The defense model is called as MAN-.
Revised: The defense model is called as MAN-.

Original: We use the adversarial examples crafted by L ∞ norm PGD-10 against the target model as adversarial training data.
Revised: We use the adversarial examples crafted by L ∞ norm PGD-10 against the target model as adversarial training data.

Original: Compared with the above MAN ∗ , the results shown in Figure 3(a) demonstrated the joint adversarial training manner can provide positive gain.
Revised: Compared with the above MAN ∗ , the results shown in Figure 3(a) demonstrated the joint adversarial training manner can provide positive gain.

Original: In addition, to show the effect of using mixture instances on defense performance, we only utilize the adversarial instances to retrain the defense model (called MAN (cid:48) ).
Revised: In addition, to show the effect of using mixture instances on defense performance, we only utilize the adversarial instances to retrain the defense model (called MAN (cid:48) ).

Original: The adversarial training data are crafted by L ∞ norm PGD-10 with adaptive attack strategy.
Revised: The adversarial training data are crafted by L ∞ norm PGD-10 with adaptive attack strategy.

Original: We use white-box adaptive attacks (i.e.,
Revised: We use white-box adaptive attacks (i.e.,

Original: Eq. 6) to evaluate the defenses.
Revised: Eq. 6) to evaluate the defenses.

Original: The results as shown in Figure 3(b) represent that the using mixture data can achieve better performance in defending adaptive attacks.
Revised: The results as shown in Figure 3(b) represent that the using mixture data can achieve better performance in defending adaptive attacks.

Revised: Moreover, to show that the improvement of our method has little relationship with the fact that the transition network introduces more model parameters, we conduct an experiment by using two parallel ResNet-18 as the target model.

Revised: The details can be found in Appendix C.

Original: [Figure]
Revised: [Figure]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: Traditional adversarial defense methods typically focus on directly exploiting adversarial examples to remove adversarial noise or train an adversarially robust target model.
Revised: Traditional adversarial defense methods typically focus on directly exploiting adversarial examples to remove adversarial noise or train an adversarially robust target model.

Original: In this paper, motivated by that the relationship between adversarial data and natural data can help infer clean data from adversarial data to obtain the ﬁnal correct prediction, we study to model adversarial noise by learning the label transition relationship for using adversarial labels to improve adversarial accuracy.
Revised: In this paper, motivated by that the relationship between adversarial data and natural data can help infer clean data from adversarial data to obtain the ﬁnal correct prediction, we study to model adversarial noise by learning the label transition relationship for using adversarial labels to improve adversarial accuracy.

Original: We propose a defense method based on Modeling Adversarial Noise (called MAN).
Revised: We propose a defense method based on Modeling Adversarial Noise (called MAN).

Original: Speciﬁcally, we embed a label transition network into the target model

Original: [Figure]

Original:  which denote the transition relationship from mixture labels (i [Figure] e., mixture of adversarial labels and natural labels) to robust labels (i
Revised: Speciﬁcally, we embed a label transition network into the target model, which denote the transition relationship from mixture labels (i.e., mixture of adversarial labels and natural labels) to robust labels (i

Original: [Figure]
Revised: [Figure]

Original: e., natural labels).
Revised: e., natural labels).

Original: The transition matrix explicitly models adversarial noise and help us to infer robust labels.
Revised: The transition matrix explicitly models adversarial noise and help us to infer robust labels.

Original: Considering that adversarial data can be adaptively generated, we conduct joint adversarial training on the target model and the transition network to achieve an optimal adversarial accuracy.
Revised: Considering that adversarial data can be adaptively generated, we conduct joint adversarial training on the target model and the transition network to achieve an optimal adversarial accuracy.

Original: The empirical results demonstrate that our defense method can provide effective protection against white-box general attacks and adaptive attacks.
Revised: The empirical results demonstrate that our defense method can provide effective protection against white-box general attacks and adaptive attacks.

Original: Our work provides a new adversarial defense strategy for the community of adversarial learning.
Revised: Our work provides a new adversarial defense strategy for the community of adversarial learning.

Original: In future, we will further optimize the MAN based defense method to improve its performance against white-box adaptive attacks.
Revised: In future, we will further optimize the MAN based defense method to improve its performance against white-box adaptive attacks.

Original: Ethics Statement This paper doesn’t raise any ethics concerns.
Revised: Ethics Statement This paper doesn’t raise any ethics concerns.

Original: This study doesn’t involve any human subjects, practices to data set releases, potentially harmful insights, methodologies and applications, potential conﬂicts of interest and sponsorship, discrimination/bias/fairness concerns, privacy and security issues, legal compliance, and research integrity issues.
Revised: This study doesn’t involve any human subjects, practices to data set releases, potentially harmful insights, methodologies and applications, potential conﬂicts of interest and sponsorship, discrimination/bias/fairness concerns, privacy and security issues, legal compliance, and research integrity issues.

Original: Reproducibility Statement To ensure the reproducibility of this work, we use the codes from the baseline method.
Revised: Reproducibility Statement To ensure the reproducibility of this work, we use the codes from the baseline method.

Original: We then conduct our experiments on this basis.
Revised: We then conduct our experiments on this basis.

Original: Note that we report the ∗ last ∗ adversarial accuracy
Revised: Note that we report the ∗ last ∗ adversarial accuracy

Original: (i e., the adversarial accuracy of the last epoch models obtained during training) in this work, instead of the ∗ best ∗ results.
Revised: (i e., the adversarial accuracy of the last epoch models obtained during training) in this work, instead of the ∗ best ∗ results.

Original: Appendices A D EFENDING AGAINST ADAPTIVE ATTACKS A
Revised: Appendices A T HE DIFFERENCE OF THE TRANSITION MATRIX BETWEEN OUR METHOD AND LABEL - NOISE LEARNING

Revised: In label-noise learning, the transition matrix is used to infer clean labels from given noisy labels. The transition matrix denotes the probabilities that clean labels ﬂip into noisy labels (Xia et al., 2020; Liu & Tao, 2015; Yang et al., 2021). The label-noise learning methods utilize the transition matrix to correct the training loss on noisy data (i.e., the clean instance with the noisy label ).

Original: S CENARIO ( I ) 
Revised: Given the noisy class posterior probability, the clean class posterior probability can be obtained, i.e., p ( y | x ) = ( T ( x ) (cid:62) ) − 1 p ( y ∗ | x ) , where y ∗ denotes the noisy label in the form of a vector. Differently, in our method, the transition matrix is used to infer natural labels from observed adversarial labels.

Original: In scenario (i)
Revised: The transition matrix denotes the probabilities that adversarial labels ﬂip into natural labels.

Original: [Table]
Revised: We utilize the transition matrix to help compute the training loss on the adversarial instance with the natural label . For the observed adversarial class posterior probability, the natural class posterior probability can be obtained, i.e., p ( y | ˜ x ) = T (˜ x ) (cid:62) p (˜ y | ˜ x ) . In addition, our method for the ﬁrst time utilizes the transition matrix to explicitly model adversarial noise for improving adversarial robustness. B D EFENDING AGAINST ADAPTIVE ATTACKS B. S CENARIO ( I )

Original:  the target attack solve the following optimization problem: 
Revised: In scenario (i), the target attack solve the following optimization problem: [Equation] [Equation] to:

Original: [Equation]
Revised: [Equation]

Original: where y ∗ is the target label in the form of a vector set by the attacker.
Revised: where y ∗ is the target label in the form of a vector set by the attacker.

Original: The adversarial accuracy of defense methods against white-box adaptive attacks on CIFAR-10 and Tiny-ImageNet are shown in
Revised: The adversarial accuracy of defense methods against white-box adaptive attacks on CIFAR-10 and Tiny-ImageNet are shown in

Original: Table 4 and Table 5 respectively. 
Revised: Table 4 and Table 5 respectively. [Table] [Table]

Revised: In addition

Original: [Table]
Revised: [Table]

Revised: we evaluate the adversarial accuracy of defense methods against white-box adaptive attacks on CIFAR-10 by using the VggNet-19 as the target model and the transition network.

Revised: As shown in Table

Original: [Table]
Revised: [Table]

Revised: our method still achieves better performance.

Original: [Figure]
Revised: [Table]

Original: A. S CENARIO ( II )
Revised: B. S CENARIO ( II )

Original: In scenario (ii), we design an adversarial attack to destroy the crucial transition matrix of our defense.
Revised: In scenario (ii), we design an adversarial attack to destroy the crucial transition matrix of our defense.

Original: Since the ground-truth transition matrix is not given, we use the target attack strategy to craft adversarial examples.
Revised: Since the ground-truth transition matrix is not given, we use the target attack strategy to craft adversarial examples.

Original: We choose an anti-diagonal identity matrix as an example of the target transition matrix in the target attack.
Revised: We choose an anti-diagonal identity matrix as an example of the target transition matrix in the target attack.

Original: The target transition matrix T ∗ is shown in Fig 4.
Revised: The target transition matrix T ∗ is shown in Fig 4.

Original: Note that there may be other attacks that can be designed, but this is beyond the scope of our work, and we would not explore further in this paper.
Revised: Note that there may be other attacks that can be designed, but this is beyond the scope of our work, and we would not explore further in this paper.

Original: [Figure]
Revised: [Figure]

Original: A. S CENARIO ( III )
Revised: B. S CENARIO ( III )

Original: In scenario (iii), the optimization goal of the target attack can be designed as: 
Revised: In scenario (iii), the optimization goal of the target attack can be designed as: [Equation] subject to

Original: [Equation]
Revised: [Equation]

Original: where y ∗ is the target label in the form of a vector set by the attacker.
Revised: where y ∗ is the target label in the form of a vector set by the attacker.

Original: Note that we report the ∗ last ∗ adversarial accuracy (i.e., the adversarial accuracy of the last epoch models obtained during training) in Table 1, 2, 3, 4 and 5 following the discussion in Carmon et al.
Revised: Note that we report the ∗ last ∗ adversarial accuracy (i.e., the adversarial accuracy of the last epoch models obtained during training) in Table 1, 2, 3, 4 and 5 following the discussion in Carmon et al.

Original: (2019), instead of the ∗ best ∗ results.
Revised: (2019), instead of the ∗ best ∗ results.

Revised: B. D EFENSE TRANSFERABILITY AGAINST ADAPTIVE ATTACKS

Revised: We transfer our defense model to VggNet-19 for defending against adaptive attacks.

Revised: The original target model is ResNet-18.

Revised: As shown in Table 7, the results show that our method has some transferability against adaptive attacks.

Revised: We ﬁnd that the adversarial accuracy of VggNet-19 is lower than that of ResNet-18.

Revised: However, this shows the importance and effectiveness of the joint adversarial training on the target model and the transition network in our method.

Revised: [Table]

Revised: C T HE INFLUENCE OF MORE MODEL PARAMETERS

Revised: We think that the improvement of the adversarial robustness has little relationship with the fact that the transition network introduces more model parameters.

Revised: The transition network is only used to learn the transition matrix, and it does not directly learn the logit output for the instance to predict the class label.

Revised: The main reason why our method can improve the robustness is that we explicitly model adversarial noise in the form of the transition matrix.

Revised: We can use this transition matrix to infer the natural label.

Revised: In addition, considering that the model parameters of our method are indeed larger than those of AT, we conduct the following experiment to compare AT and our method in the case of similar number of model parameters.

Revised: Considering that both the target model and the transition network in our method are mainly based on

Revised: ResNet-18 on CIFAR-10 , we use two parallel ResNet-18 as the target model to classify the input.

Revised: We use the average of their logit outputs as the ﬁnal logit output.

Revised: We use the training mechanism of AT to train the new target model.

Revised: In this way, The number of model parameters of our method is similar to that of AT.

Revised: As shown in Table 8, the results show that our method still has higher adversarial accuracy.

Revised: This demonstrates that the improvement of the adversarial robustness is not due to the increase of model parameters.

Revised: [Table]