Original: I NCREASING -M ARGIN A DVERSARIAL (IMA) TRAINING TO I MPROVE A DVERSARIAL R OBUSTNESS OF N EURAL N ETWORKS
Revised: I NCREASING -M ARGIN A DVERSARIAL (IMA) TRAINING TO I MPROVE A DVERSARIAL R OBUSTNESS OF N EURAL N ETWORKS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Deep neural network, including convolutional neural network, is known to be vulnerable to adversarial attacks , which may lead to disastrous consequences in med - ical applications
Revised: Deep neural network, including convolutional neural network, is known to be vulnerable to adversarial attacks, which may lead to disastrous consequences in medical applications.

Original: Adversarial samples are usually generated by attack algorithms and can also be induced by white noises , and therefore the threats are real
Revised: Adversarial samples are usually generated by attack algorithms and can also be induced by white noises, and therefore the threats are real.

Original: In this study we propose a novel training method, named Increasing Margin Adversarial (IMA) Training , to improve 
Revised: In this study, we propose a novel training method, named Increasing Margin Adversarial (IMA) Training, to improve DNN robustness against adversarial noises.

Original: DNN robustness against adversarial noises During training, the IMA method increases the margins of training samples by moving the decision boundaries of the DNN model far away from the training samples to improve robustness.
Revised: During training, the IMA method increases the margins of training samples by moving the decision boundaries of the DNN model far away from the training samples to improve robustness.

Original: The IMA method is evaluated on four publicly available datasets (including a COVID-

Revised: The IMA method is evaluated on four publicly available datasets (including a COVID-

Original: CT image dataset under strong 100-PGD whitebox adversarial attacks and the results show that the proposed method signiﬁcantly improved classiﬁcation accuracy on noisy data while keeping a relatively high accuracy on clean data.
Revised: CT image dataset) under strong 100-PGD whitebox adversarial attacks and the results show that the proposed method signiﬁcantly improved classiﬁcation accuracy on noisy data while keeping a relatively high accuracy on clean data.

Original: We hope our approach may facilitate the development of robust DNN applications, especially in the medical ﬁeld.
Revised: We hope our approach may facilitate the development of robust DNN applications, especially for COVID-10 diagnosis using CT images.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep neural network (DNN), especially convolutional neural network (CNN), has become the ﬁrst choice for automated image analysis due to its superior performance.
Revised: Deep neural network (DNN), especially convolutional neural network (CNN), has become the ﬁrst choice for automated image analysis due to its superior performance.

Original: However, recent studies have shown that DNNs are not robust to a special type of noise, called adversarial noise.
Revised: However, recent studies have shown that DNNs are not robust to a special type of noise, called adversarial noise.

Original: Adversarial noise was ﬁrst discovered by (Szegedy et al., 2013) and then explained by (Goodfellow et al., 2014).
Revised: Adversarial noise was ﬁrst discovered by (Szegedy et al., 2013) and then explained by (Goodfellow et al., 2014).

Original: Adversarial noises can signiﬁcantly affect robustness of DNNs for a wide range of image classiﬁcation applications (Akhtar & Mian, 2018), such as handwritten digits recognition (Graese et al., 2016), human face recognition (Mirjalili & Ross, 2017), and even trafﬁc sign detection (Eykholt et al., 2018).
Revised: Adversarial noises can signiﬁcantly affect robustness of DNNs for a wide range of image classiﬁcation applications (Akhtar & Mian, 2018), such as handwritten digits recognition (Graese et al., 2016), human face recognition (Mirjalili & Ross, 2017), and even trafﬁc sign detection (Eykholt et al., 2018).

Original: DNN-based image segmentation can also be affected by adversarial noises because image segmentation is often realized by pixel classiﬁcation.
Revised: DNN-based image segmentation can also be affected by adversarial noises because image segmentation is often realized by pixel classiﬁcation.

Original: The COVID-19 pandemic has infected millions of people and caused the death of about 1 million people as of today (WHO, 2020).
Revised: The COVID-19 pandemic has infected millions of people and caused the death of about 1 million people as of today (WHO, 2020).

Original: A large scale study in China shows that CT had higher sensitivity for the diagnosis of COVID-19 as compared with initial reverse-transcription polymerase chain reaction (RT-PCR) from swab samples (Ai et al., 2020).
Revised: A large scale study in China shows that CT had higher sensitivity for the diagnosis of COVID-19 as compared with initial reverse-transcription polymerase chain reaction (RT-PCR) from swab samples (Ai et al., 2020).

Original: As reviewed in (Shi et al., 2020), many
Revised: As reviewed in (Shi et al., 2020), many

Original: DNN models for COVID-19 diagnosis from CT images have been developed and achieved very high classiﬁcation accuracy.
Revised: DNN models for COVID-19 diagnosis from CT images have been developed and achieved very high classiﬁcation accuracy.

Original: However, none of these studies (Shi et al., 2020) considered DNN robustness against adversarial noises.
Revised: However, none of these studies (Shi et al., 2020) considered DNN robustness against adversarial noises.

Original: We modiﬁed a Resnet-18 model (He et al., 2016) and trained it on a public COVID-

Revised: We modiﬁed a Resnet-18 model (He et al., 2016) and trained it on a public COVID-

Original: CT image dataset (Soares et al., 2020), and then the model robustness is tested (details in section 3.3).
Revised: CT image dataset (Soares et al., 2020), and then the model robustness is tested (details in section 3.3).

Original: Fig.
Revised: Fig.

Original: 1 shows a CT image (denoted by x ) of a lung that was infected by
Revised: 1 shows a CT image (denoted by x ) of a lung that was infected by

Original: COVID-19 and correctly classiﬁed as infected.
Revised: COVID-19 and correctly classiﬁed as infected.

Original: After adding a small amount of noise δ to the image x , the noisy image x + δ is classiﬁed as uninfected.
Revised: After adding a small amount of noise δ to the image x , the noisy image x + δ is classiﬁed as uninfected.

Original: On the test set, although the model achieved ≥ 95% accuracy on original clean images, its accuracy dropped to zero on the noise level of 0 . 03 .
Revised: On the test set, although the model achieved ≥ 95% accuracy on original clean images, its accuracy dropped to zero on the noise level of 0 . 03 .

Original: This non-robust model clearly cannot be used in real clinical applications.
Revised: This non-robust model clearly cannot be used in real clinical applications.

Original: It may be argued that adversarial noises are created by algorithms and therefore may not exist in the real world unless some bad people want to hack the system to achieve personal gain at the expense of public health.
Revised: It may be argued that adversarial noises are created by algorithms and therefore may not exist in the real world unless some bad people want to hack the system to achieve personal gain at the expense of public health.

Original: However, on the COVID-19 dataset, we found that 2 .
Revised: However, on the COVID-19 dataset, we found that 2 .

Original: 75% of the noisy samples with uniform white noises on the noise level of 0 .
Revised: 75% of the noisy samples with uniform white noises on the noise level of 0 .

Original: 05 , can cause the DNN model to make wrong classiﬁcations, which shows that white-noise-induced adversarial samples can exist.
Revised: 05 , can cause the DNN model to make wrong classiﬁcations, which shows that white-noise-induced adversarial samples can exist.

Original: For medical applications, 2 .
Revised: For medical applications, 2 .

Original: 75% is not a negligible number, and it is worth developing methods to improve DNN adversarial robustness.
Revised: 75% is not a negligible number, and it is worth developing methods to improve DNN adversarial robustness.

Original: There are mainly two categories of adversarial attacks: white-box attack and black-box attack.
Revised: There are mainly two categories of adversarial attacks: white-box attack and black-box attack.

Original: For a white-box attack, the attacker knows everything about the DNN to be attacked.
Revised: For a white-box attack, the attacker knows everything about the DNN to be attacked.

Original: For a black-box attack, the attacker only can use the DNN as a black-box (i.e. send an input to the DNN and get an output from the DNN, not knowing anything else).
Revised: For a black-box attack, the attacker only can use the DNN as a black-box (i.e. send an input to the DNN and get an output from the DNN, not knowing anything else).

Original: From the perspective of defense, we should consider the worst-case scenario: white-box attack.
Revised: From the perspective of defense, we should consider the worst-case scenario: white-box attack.

Original: People have explored many ideas to improve robustness, but many of them have been shown to be ineffective (Uesato et al., 2018)
Revised: People have explored many ideas to improve robustness, but many of them have been shown to be ineffective (Uesato et al., 2018)

Original: (Tramer et al., 2020).
Revised: (Tramer et al., 2020).

Original: A general and effective strategy is adversarial training (Goodfellow et al., 2014)
Revised: A general and effective strategy is adversarial training (Goodfellow et al., 2014)

Original: (Madry et al., 2017)(Ding et al., 2019a), and the basic idea is to use adversarial attack algorithms to generate noisy samples and add those samples to the training set, which basically is a special data augmentation strategy.
Revised: (Madry et al., 2017)(Miyato et al., 2018)(Ding et al., 2019a), and the basic idea is to use adversarial attack algorithms to generate noisy samples and add those samples to the training set, which basically is a special data augmentation strategy.

Original: Through adversarial training, the DNN model can learn from the noisy samples and become robust to noises.
Revised: Through adversarial training, the DNN model can learn from the noisy samples and become robust to noises.

Original: Adversarial training is straightforward but computationally expensive.
Revised: Adversarial training is straightforward but computationally expensive.

Original: Thus, one needs to make sure that the generated noisy samples can indeed help to improve robustness: samples with too much noise can harm performance while samples with too little noise may have no effect at all.
Revised: Thus, one needs to make sure that the generated noisy samples can indeed help to improve robustness: samples with too much noise can harm performance while samples with too little noise may have no effect at all.

Original: Generative adversarial training has been directly applied to improve robustness; however, it can only defend against black-box attack (Wang & Yu, 2019).
Revised: Generative adversarial training has been directly applied to improve robustness; however, it can only defend against black-box attack (Wang & Yu, 2019).

Original: In this paper, we propose a novel method, Increasing-Margin Adversarial (IMA) Training, to improve robustness of deep neural networks for classiﬁcation tasks.
Revised: In this paper, we propose a novel method, Increasing-Margin Adversarial (IMA) Training, to improve robustness of deep neural networks for classiﬁcation tasks.

Original: Our method aims to increase margins of training samples by moving decision boundaries far away from the samples to improve robustness.
Revised: Our method aims to increase margins of training samples by moving decision boundaries far away from the samples to improve robustness.

Original: We evaluated our method on four datasets with 100-PGD white-box attack and the results show that our proposed method can achieve a signiﬁcant improvement in DNN robustness against adversarial noises.
Revised: We evaluated our method on four datasets with 100-PGD white-box attack and the results show that our proposed method can achieve a signiﬁcant improvement in DNN robustness against adversarial noises.

Original: L o ca l D e c
Revised: L o ca l D e c

Original: i s i o n B o und a r y 𝑥 ℇ ( 𝑥 )
Revised: i s i o n B o und a r y 𝑥 ℇ ( 𝑥 )

Original: L o ca l D e c
Revised: L o ca l D e c

Original: i s i o n B o und a r y 𝑥
Revised: i s i o n B o und a r y 𝑥

Original: 𝑛 𝑥 𝑛𝑥 𝑛𝑥 ℇ ( 𝑥 )
Revised: 𝑛 𝑥 𝑛𝑥 𝑛𝑥 ℇ ( 𝑥 )

Original:  M ETHODOLOGY 2. A DVERSARIAL A TTACK AND N EURAL N ETWORK R OBUSTNESS
Revised:  M ETHODOLOGY 2. A DVERSARIAL A TTACK AND N EURAL N ETWORK R OBUSTNESS

Original: To evaluate the robustness of different adversarial training methods, we use projected gradient descent (PGD) (Madry et al., 2017)(Kurakin et al., 2016) to generate adversarial noises, which is widely used for method evaluation (Uesato et al., 2018)
Revised: To evaluate the robustness of different adversarial training methods, we use projected gradient descent (PGD) (Madry et al., 2017)(Kurakin et al., 2016) to generate adversarial noises, which is widely used for method evaluation (Uesato et al., 2018)

Original: (Tramer et al., 2020).
Revised: (Tramer et al., 2020).

Original: For the convenience of the reader, we brieﬂy describe PGD:
Revised: For the convenience of the reader, we brieﬂy describe PGD:

Original: Let x denote an input sample and y be the true class label.
Revised: Let x denote an input sample and y be the true class label.

Original: Let J ( x ) denote the scalar objective function of PGD, which could be the cross-entropy loss function or other classiﬁcation loss functions.
Revised: Let J ( x ) denote the scalar objective function of PGD, which could be the cross-entropy loss function or other classiﬁcation loss functions.

Original: Let δ denote the adversarial noise, and its magnitude is ε which is measured by the vector Lp norm of δ , i.e., ε = ||
Revised: Let δ denote the adversarial noise, and its magnitude is ε which is measured by the vector Lp norm of δ , i.e., ε = ||

Original: δ || p , where p is inf or 2 usually.
Revised: δ || p , where p is inf or 2 usually.

Original: PGD adds noises to x iteratively:
Revised: PGD adds noises to x iteratively:

Original: [Equation]
Revised: [Equation]

Original: (1) where η is step size, J (cid:48)
Revised: (1) where η is step size, J (cid:48)

Original: ( x ) =
Revised: ( x ) =

Original: ∂J ∂x , and i is iteration index.
Revised: ∂J ∂x , and i is iteration index.

Original: x (0) =
Revised: x (0) =

Original: x + ξ , where ξ is random noise with || ξ || p ≤ ε .
Revised: x + ξ , where ξ is random noise with || ξ || p ≤ ε .

Original: ||
Revised: ||

Original: || p denotes the vector p-norm.
Revised: || p denotes the vector p-norm.

Original: The clip operation in Eq.(1) ensures that || x ( i ) − x || p ≤ ε (called ε -ball).
Revised: The clip operation in Eq.(1) ensures that || x ( i ) − x || p ≤ ε (called ε -ball).

Original: If x is an image, the clip operation also ensures that pixel values stay within the feasible range (e.g. 0 to 1).
Revised: If x is an image, the clip operation also ensures that pixel values stay within the feasible range (e.g. 0 to 1).

Original: If L-inf norm is used, h ( J (cid:48) ) is the sign function; and if L2 norm is used, then h ( J (cid:48) ) normalizes J (cid:48) by its L2 norm.
Revised: If L-inf norm is used, h ( J (cid:48) ) is the sign function; and if L2 norm is used, then h ( J (cid:48) ) normalizes J (cid:48) by its L2 norm.

Original: The total adversarial noise is δ = x ( N PGD ) − x , and N PGD is the number of iterations.
Revised: The total adversarial noise is δ = x ( N PGD ) − x , and N PGD is the number of iterations.

Original: η is usually set to α · ε/N PGD , and therefore the algorithm may sweep the ε -ball α times ( α ≥ 1 ) within N PGD iterations.
Revised: η is usually set to α · ε/N PGD , and therefore the algorithm may sweep the ε -ball α times ( α ≥ 1 ) within N PGD iterations.

Original: By adding the adversarial noise δ to the input x , the objective function J ( x + δ ) will be signiﬁcantly larger than J ( x ) , leading to wrong classiﬁcation of the noisy sample x + δ .
Revised: By adding the adversarial noise δ to the input x , the objective function J ( x + δ ) will be signiﬁcantly larger than J ( x ) , leading to wrong classiﬁcation of the noisy sample x + δ .

Original: It is easy to understand the robustness issue through Eq.(1) and Fig. 1.
Revised: It is easy to understand the robustness issue through Eq.(1) and Fig. 1.

Original: Given a trained classiﬁer, if a sample x is very close to the decision boundary of the classiﬁer, then one could add a small noise δ to x such that x + δ goes across the decision boundary and is wrongly-classiﬁed.
Revised: Given a trained classiﬁer, if a sample x is very close to the decision boundary of the classiﬁer, then one could add a small noise δ to x such that x + δ goes across the decision boundary and is wrongly-classiﬁed.

Original: As shown in section 3, using cross-entropy loss and clean data for training, the decision boundaries of the trained classiﬁer will often be very close to the samples; as a result, the accuracy can drop to zero upon very small noises.
Revised: As shown in section 3, using cross-entropy loss and clean data for training, the decision boundaries of the trained classiﬁer will often be very close to the samples; as a result, the accuracy can drop to zero upon very small noises.

Original: To improve robustness, vanilla adversarial training (Goodfellow et al., 2014)(Kurakin et al., 2016) based on PGD can be used: given the maximum noise level ε = ε max , generate a noisy sample x ε =
Revised: To improve robustness, vanilla adversarial training (Goodfellow et al., 2014)(Kurakin et al., 2016) based on PGD can be used: given the maximum noise level ε = ε max , generate a noisy sample x ε =

Original: PGD ( x, y, ε ) for every clean sample x in training set; use the clean samples and noisy samples together to train the model.
Revised: PGD ( x, y, ε ) for every clean sample x in training set; use the clean samples and noisy samples together to train the model.

Original: The loss function is given in (Goodfellow et al., 2014):
Revised: The loss function is given in (Goodfellow et al., 2014):

Original: [Equation]
Revised: [Equation]

Original: In practice, the loss is accumulated over each mini-batch.
Revised: In practice, the loss is accumulated over each mini-batch.

Original: L ce is the cross-entropy loss.
Revised: L ce is the cross-entropy loss.

Original: As shown in section 3, the vanilla adversarial training is very sensitive to the user-deﬁned noise level ε max .
Revised: As shown in section 3, the vanilla adversarial training is very sensitive to the user-deﬁned noise level ε max .

Original: 2. I NCREASING -M ARGIN A DVERSARIAL (IMA) T RAINING
Revised: 2. I NCREASING -M ARGIN A DVERSARIAL (IMA) T RAINING

Original: We developed a novel method, Increasing-Margin Adversarial (IMA) training, to enhance DNN classiﬁer robustness.
Revised: We developed a novel method, Increasing-Margin Adversarial (IMA) training, to enhance DNN classiﬁer robustness.

Original: As its name indicates, IMA is a type of adversarial training: it generates noisy samples and trains a DNN model on clean and noisy samples.
Revised: As its name indicates, IMA is a type of adversarial training: it generates noisy samples and trains a DNN model on clean and noisy samples.

Original: IMA algorithms are signiﬁcantly different from the other adversarial training algorithms.
Revised: IMA algorithms are signiﬁcantly different from the other adversarial training algorithms.

Original: IMA training includes two alternating sub-processes: Algorithm 1 to compute loss and update the
Revised: IMA training includes two alternating sub-processes: Algorithm 1 to compute loss and update the

Original: DNN model, and Algorithm 2 to update margin estimation.
Revised: DNN model, and Algorithm 2 to update margin estimation.

Original: In Algorithm 1, by minimizing the loss on clean and noisy data, the model will reach a balance between robustness and accuracy on clean data.
Revised: In Algorithm 1, by minimizing the loss on clean and noisy data, the model will reach a balance between robustness and accuracy on clean data.

Original: In Algorithm 2, the sample margins are updated after each epoch.
Revised: In Algorithm 2, the sample margins are updated after each epoch.

Original: These estimated margins are used to generate noisy samples for training the model in the next epoch.
Revised: These estimated margins are used to generate noisy samples for training the model in the next epoch.

Original: Algorithm 1 compute loss and update model in an epoch
Revised: Algorithm 1 compute loss and update model in an epoch

Original: Input: : S = ( x, y ) is the training set, containing pairs of clean sample x and true label y .
Revised: Input: : S = ( x, y ) is the training set, containing pairs of clean sample x and true label y .

Original: f is the
Revised: f is the

Original: DNN model and ˆ y = f
Revised: DNN model and ˆ y = f

Original: ( x ) .
Revised: ( x ) .

Original: E is an array containing the estimated margins of individual training samples.
Revised: E is an array containing the estimated margins of individual training samples.

Original: β is a scalar coefﬁcient.
Revised: β is a scalar coefﬁcient.

Original: In a batch, X contains samples and Y contains class labels.
Revised: In a batch, X contains samples and Y contains class labels.

Original: Output: updated model f after this training epoch
Revised: Output: updated model f after this training epoch

Original: Process: 1: for Batch data X and Y in S do 2:
Revised: Process: 1: for Batch data X and Y in S do 2:

Original: Run model f to classify the samples and divide the samples into two groups, wronglyclassiﬁed X 0 , Y 0 and correctly-classiﬁed X 1 , Y 1 where X =
Revised: Run model f to classify the samples and divide the samples into two groups, wronglyclassiﬁed X 0 , Y 0 and correctly-classiﬁed X 1 , Y 1 where X =

Original: X 0 ∪ X 1 and Y
Revised: X 0 ∪ X 1 and Y

Original: =
Revised: =

Original: Y 0 ∪ Y3:
Revised: Y 0 ∪ Y3:

Original: Get noisy samples X n = BPGD ( X 1 , Y 1 , E ( X 1 )) and classify them ˆ Y n = f ( X n ) · E ( X 1 ) is a sub-array of E , containing the estimated margins of samples in X 1 .
Revised: Get noisy samples X n = BPGD ( X 1 , Y 1 , E ( X 1 )) and classify them ˆ Y n = f ( X n ) · E ( X 1 ) is a sub-array of E , containing the estimated margins of samples in X 1 .

Original: BPGD is described in
Revised: BPGD is described in

Original: Algorithm 3.
Revised: Algorithm 3.

Original: Compute the loss L: 5:
Revised: Compute the loss L: 5:

Original: L 0 = cross entropy ( f ( X 0 ) , Y 0 ) (take sum) 6:
Revised: L 0 = cross entropy ( f ( X 0 ) , Y 0 ) (take sum) 6:

Original: L 1 = cross entropy ( f ( X 1 ) , Y 1 ) (take sum) 7:
Revised: L 1 = cross entropy ( f ( X 1 ) , Y 1 ) (take sum) 7:

Original: L 2 = cross entropy ( f ( X n ) , Y 1 ) (take sum) 8:
Revised: L 2 = cross entropy ( f ( X n ) , Y 1 ) (take sum) 8:

Original: L =
Revised: L =

Original: ( (1 − β ) ·
Revised: ( (1 − β ) ·

Original: ( L 0 + L 1 ) + β · L 2 )
Revised: ( L 0 + L 1 ) + β · L 2 )

Original: /batch size 9:
Revised: /batch size 9:

Original: Back-propagate from L and update the model f 10: end for
Revised: Back-propagate from L and update the model f 10: end for

Original: Our IMA method tries to generate noisy samples on decision boundaries as much as possible: adding too much noise to a training sample may lead to low accuracy on clean samples, but adding too little noise may have no effect on robustness.
Revised: Our IMA method tries to generate noisy samples on decision boundaries as much as possible: adding too much noise to a training sample may lead to low accuracy on clean samples, but adding too little noise may have no effect on robustness.

Original: Since the distance between a training sample and a decision boundary is different for different training samples, the noises added to the training samples have different magnitudes (i.e. vector norms).
Revised: Since the distance between a training sample and a decision boundary is different for different training samples, the noises added to the training samples have different magnitudes (i.e. vector norms).

Original: The rationale of doing so is discussed in section 2.3.
Revised: The rationale of doing so is discussed in section 2.3.

Original: Here, the margin of a sample is the minimum distance between the sample and the decision boundaries.
Revised: Here, the margin of a sample is the minimum distance between the sample and the decision boundaries.

Original: It is difﬁcult to compute the exact margin for a sample in a high dimension space, and therefore we can only estimate it.
Revised: It is difﬁcult to compute the exact margin for a sample in a high dimension space, and therefore we can only estimate it.

Original: Input: S =
Revised: Input: S =

Original: ( x, y ) is the training set.
Revised: ( x, y ) is the training set.

Original: f is the DNN model and ˆ y = f
Revised: f is the DNN model and ˆ y = f

Original: ( x ) .
Revised: ( x ) .

Original: E is an array of the estimated margins of training samples.
Revised: E is an array of the estimated margins of training samples.

Original: E ( x ) is the estimated margin of x .
Revised: E ( x ) is the estimated margin of x .

Original: ∆ ε is margin expansion step size.
Revised: ∆ ε is margin expansion step size.

Original: ε max is maximum noise level.
Revised: ε max is maximum noise level.

Original: Output: updated E
Revised: Output: updated E

Original: Process: 1: for each sample x and y in S do 2:
Revised: Process: 1: for each sample x and y in S do 2:

Original: Classify the sample ˆ y = f
Revised: Classify the sample ˆ y = f

Original: ( x ) 3: if x is wrongly-classiﬁed ( ˆ y is not y ) then 4:
Revised: ( x ) 3: if x is wrongly-classiﬁed ( ˆ y is not y ) then 4:

Original: Update E ( x ) =
Revised: Update E ( x ) =

Original: ∆ ε (re-initialize) 5: else if x is correctly-classiﬁed ( ˆ y is y )
Revised: ∆ ε (re-initialize) 5: else if x is correctly-classiﬁed ( ˆ y is y )

Original: then 6:
Revised: then 6:

Original: Get noisy sample x n = BPGD ( x, y, E ( x )) 7: if x n is on decision boundary (not None) then 8:
Revised: Get noisy sample x n = BPGD ( x, y, E ( x )) 7: if x n is on decision boundary (not None) then 8:

Original: Update E ( x ) =
Revised: Update E ( x ) =

Original: || x − x n || p (reﬁne/shrink) 9: else 10:
Revised: || x − x n || p (reﬁne/shrink) 9: else 10:

Original: Update E ( x ) =
Revised: Update E ( x ) =

Original: E ( x ) + ∆ ε (expand) 11: end if 12: end if 13: end for 14: Clip every element of E into the range of ∆ ε to ε max
Revised: E ( x ) + ∆ ε (expand) 11: end if 12: end if 13: end for 14: Clip every element of E into the range of ∆ ε to ε max

Original: Note: Every E ( x ) is initialized to be ∆ ε which can be set to ε max divided by the number of adversarial training epochs.
Revised: Note: Every E ( x ) is initialized to be ∆ ε which can be set to ε max divided by the number of adversarial training epochs.

Original: The algorithm runs in mini-batches. 
Revised: The algorithm runs in mini-batches. estimated margin

Original: Input
Revised: N PGD

Original: Output:
Revised: of Process:

Original: Process: samples f samples correctly classiﬁed (case-0 in Fig.2) then will be ignored) 5: else 6: (case-1 in Fig.2) Find x n 1 and x n 2 , two adjacent samples in the sequence.
Revised: 6: (case-1 in Fig.2) Find x n 1 and x n 2 , two adjacent samples in the sequence.

Original: x n 1 is correctlyclassiﬁed and x n 2 is wrongly-classiﬁed.
Revised: x n 1 is correctlyclassiﬁed and x n 2 is wrongly-classiﬁed.

Original: Thus, x n 1 and x n 2 are close to the decision boundary.
Revised: Thus, x n 1 and x n 2 are close to the decision boundary.

Original: Run binary search along the straight-line segment between x n 1 and x n 2 in order to ﬁnd x n on the decision boundary (Fig.3) 8: return x n 9: end if
Revised: Run binary search along the straight-line segment between x n 1 and x n 2 in order to ﬁnd x n on the decision boundary (Fig.3) 8: return x n 9: end if

Original: Note: it is unnecessary to store the whole sequence.
Revised: Note: it is unnecessary to store the whole sequence.

Original: The algorithm runs in mini-batches.
Revised: The algorithm runs in mini-batches.

Original: We developed Algorithm 3, named BPGD, to ﬁnd noisy samples on decision boundaries.
Revised: We developed Algorithm 3, named BPGD, to ﬁnd noisy samples on decision boundaries.

Original: At the very beginning of the IMA training process, the margins of the training samples are initialized to a small number equal to margin expansion step size in Algorithm 2.
Revised: At the very beginning of the IMA training process, the margins of the training samples are initialized to a small number equal to margin expansion step size in Algorithm 2.

Original: Intuitively, during IMA training, the margin of a training sample keeps increasing as if a ball is expanding (the ball center is the sample x , and the radius is the margin E ( x ) ; it is called ε -ball), until the ball of the sample x collides with the ball of another sample x (cid:48) in a different class.
Revised: Intuitively, during IMA training, the margin of a training sample keeps increasing as if a ball is expanding (the ball center is the sample x , and the radius is the margin E ( x ) ; it is called ε -ball), until the ball of the sample x collides with the ball of another sample x (cid:48) in a different class.

Original: When the two balls collide and therefore a local decision boundary is formed, Algorithm 2 will prevent them from further expansion by reﬁning/shrinking margins (Fig. 4).
Revised: When the two balls collide and therefore a local decision boundary is formed, Algorithm 2 will prevent them from further expansion by reﬁning/shrinking margins (Fig. 4).

Original: In section 2.3, we will show that an equilibrium state may exist (Fig. 5) under which the margins of the samples are maximized.
Revised: In section 2.3, we will show that an equilibrium state may exist (Fig. 5) under which the margins of the samples are maximized.

Original: 2. THE E QUILIBRIUM S TATE
Revised: 2. THE E QUILIBRIUM S TATE

Original: We can show that on certain conditions, an equilibrium state (Fig. 5-Left) exists under which the margins of the samples are maximized.
Revised: We can show that on certain conditions, an equilibrium state (Fig. 5-Left) exists under which the margins of the samples are maximized.

Original: To simplify the discussion, we assume there are three classes and three decision boundaries between classes (Fig.
Revised: To simplify the discussion, we assume there are three classes and three decision boundaries between classes (Fig.

Original: 5-Right).
Revised: 5-Right).

Original: The softmax output of the neural 
Revised: The softmax output of the neural Correctlyclassified

Revised: Wronglyclassified

Original:  [Figure]
Revised: D [Figure]

Revised: o n

Revised: [Figure]

Revised: r y ...

Revised: 𝑥 𝑛𝑥 𝑛𝑥 𝑛

Original: [Figure] 
Revised: [Figure] class- [Figure] class-j

Revised: T h

Revised: e f i n a l d ec i s i

Original: [Figure]
Revised: [Figure]

Revised: bound a r y a f t e r

Revised: I M

Revised: A t r a i n i ng

Revised: [Figure]

Revised: e po c h

Revised: [Figure]

Original:  network model f has three components: P 1 , P 2 and P 3 corresponding to the three classes.
Revised: e po c h [Figure] [Figure] network model f has three components: P 1 , P 2 and P 3 corresponding to the three classes.

Original: If a point (i.e. a sample) x is on the decision boundary B ij between class- i ( c i ) and class- j ( c j ) , then
Revised: If a point (i.e. a sample) x is on the decision boundary B ij between class- i ( c i ) and class- j ( c j ) , then

Original: P i ( x ) =
Revised: P i ( x ) =

Original: P j ( x ) .
Revised: P j ( x ) .

Original: The mathematical expectation of the cross-entropy loss of the noisy samples ( Lin Algorithm 1) is:
Revised: The mathematical expectation of the cross-entropy loss of the noisy samples ( Lin Algorithm 1) is:

Original: [Equation]
Revised: [Equation]

Original: The IMA method puts the noisy samples on the decision boundaries, thus:
Revised: The IMA method puts the noisy samples on the decision boundaries, thus:

Original: [Equation]
Revised: [Equation]

Original: E X n ∈

Original: [Equation]

Original: 2 ( − log ( P

Original: [Equation]

Original: [Equation]

Original: n )))

Original: =

Original: [Equation]

Original: n

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: n

Original: [Equation]

Original: E
Revised: E

Original: [Equation]
Revised: [Equation]

Original: If the noisy samples (random variables) X n ∈ c i and X n ∈ c j have the same spatial distribution on the decision boundary B ij between the two classes, Eq.(3) can be simpliﬁed to
Revised: If the noisy samples (random variables) X n ∈ c i and X n ∈ c j have the same spatial distribution on the decision boundary B ij between the two classes, Eq.(3) can be simpliﬁed to

Original: [Equation]
Revised: [Equation]

Original: (7) where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: E reaches the minimum when P i ( X n ) =
Revised: E reaches the minimum when P i ( X n ) =

Original: P j ( X n ) .
Revised: P j ( X n ) .

Original: The analysis shows that the loss of noisy samples will increase if the decision boundaries of the model f change a little bit from the current state.
Revised: The analysis shows that the loss of noisy samples will increase if the decision boundaries of the model f change a little bit from the current state.

Original: Thus, when the loss ( L 0 + L 1 ) on clean data is minimized and noisy samples are on the decision boundaries, an equilibrium is reached under the assumption that noisy samples have the same spatial distribution on the decision boundaries between classes.
Revised: Thus, when the loss ( L 0 + L 1 ) on clean data is minimized and noisy samples are on the decision boundaries, an equilibrium is reached under the assumption that noisy samples have the same spatial distribution on the decision boundaries between classes.

Original: This analysis provides the rationale that our IMA method puts the noisy samples on decision boundaries as much as possible, which is signiﬁcantly different from the theory of the MMA method (Ding et al., 2019a).
Revised: This analysis provides the rationale that our IMA method puts the noisy samples on decision boundaries as much as possible, which is signiﬁcantly different from the theory of the MMA method (Ding et al., 2019a).

Original: If the assumption does not hold, an equilibrium may still be reached (when L is minimized) and the decision boundaries will be determined by sample spatial distribution, which may not be desired solution.
Revised: If the assumption does not hold, an equilibrium may still be reached (when L is minimized) and the decision boundaries will be determined by sample spatial distribution, which may not be desired solution.

Original: Nevertheless, the experiment results show that sample margins can be increased signiﬁcantly, compared to those trained only on clean data. 
Revised: Nevertheless, the experiment results show that sample margins can be increased signiﬁcantly, compared to those trained only on clean data. Further discussion is provided in Appendix G.

Original:  E XPERIMENT
Revised:  E XPERIMENT

Original: We applied the IMA method for four DNNs on four datasets.
Revised: We applied the IMA method for four DNNs on four datasets.

Original: For every dataset, β in Algorithm 1 is 0.5 because robustness and accuracy are equally important.
Revised: For every dataset, β in Algorithm 1 is 0.5 because robustness and accuracy are equally important.

Original: In Algorithm 3, N PGD is 20,
Revised: In Algorithm 3, N PGD is 20,

Original: N binary is 10, and there is no repeat.
Revised: N binary is 10, and there is no repeat.

Original: α in PGD is 4.
Revised: α in PGD is 4.

Original: Larger N PGD and N binary may lead to better convergence of the algorithms but more computing-time.
Revised: Larger N PGD and N binary may lead to better convergence of the algorithms but more computing-time.

Original: Pytorch (Paszke et al., 2017) is used for model implementation.
Revised: Pytorch (Paszke et al., 2017) is used for model implementation.

Original: Nvidia V100 and Titan V GPUs are used for model training and testing.
Revised: Nvidia V100 and Titan V GPUs are used for model training and testing.

Original: To describe the DNNs, we use “COV (a, b, c, d, e)” to represent a convolution layer with “a” input channels, “b” output channels, kernel size of “c”, stride of “d” and padding of “e”; “Linear (f, g)” to denote a linear layer with input size of “f” and output size of “g”; “IN” to denote instance normalization; “LN” to denote layer normalization, and “LR” to denote leaky ReLU.
Revised: To describe the DNNs, we use “COV (a, b, c, d, e)” to represent a convolution layer with “a” input channels, “b” output channels, kernel size of “c”, stride of “d” and padding of “e”; “Linear (f, g)” to denote a linear layer with input size of “f” and output size of “g”; “IN” to denote instance normalization; “LN” to denote layer normalization, and “LR” to denote leaky ReLU.

Original: To obtain the baseline performance, each DNN was trained with cross-entropy loss on clean data, and the trained model is denoted by “ce”.
Revised: To obtain the baseline performance, each DNN was trained with cross-entropy loss on clean data, and the trained model is denoted by “ce”.

Original: To evaluate the performance, we compare our method with the other three adversarial training methods, including: (1) 20-PGD ( N PGD =20 and α =4) based vanilla adversarial training with noise level ε , denoted by “adv ε ”; (2) tradeoff-inspired adversarial defense via surrogate-loss minimization, named TRADES in (Zhang et al., 2019); (3) adversarial training for direct input space margin maximization, named MMA in (Ding et al., 2019a).
Revised: To evaluate the performance, we compare our method with the other three adversarial training methods, including: (1) 20-PGD ( N PGD =20 and α =4) based vanilla adversarial training with noise level ε , denoted by “adv ε ”; (2) tradeoff-inspired adversarial defense via surrogate-loss minimization, named TRADES in (Zhang et al., 2019); (3) adversarial training for direct input space margin maximization, named MMA in (Ding et al., 2019a).

Original: MMA is a state of art method, and TRADES was state of art at the time of its publication.
Revised: MMA is a state of art method, and TRADES was state of art at the time of its publication.

Original: To make the comparison as fair as possible, the number of iterations and step size of TRADES are the same as those of the 20-PGD; and the number of PGD-iterations in MMA is 20.
Revised: To make the comparison as fair as possible, the number of iterations and step size of TRADES are the same as those of the 20-PGD; and the number of PGD-iterations in MMA is 20.

Original: We used
Revised: We used

Original: PGD to evaluate the robustness of the methods, which is widely used for method evaluation (Liao et al., 2018)(Uesato et al., 2018).
Revised: PGD to evaluate the robustness of the methods, which is widely used for method evaluation (Liao et al., 2018)(Uesato et al., 2018).

Original: The number of PGD iterations is 100 to ensure a strong attack, and it is called 100-PGD.
Revised: The number of PGD iterations is 100 to ensure a strong attack, and it is called 100-PGD.

Original: To further enhance the attack during model testing, each 100-PGD rans twice, using MarginalLoss and CrossEntropyLoss respectively, which is implemented in (Ding et al., 2019b)https://github.com/BorealisAI/advertorch .
Revised: To further enhance the attack during model testing, each 100-PGD rans twice, using MarginalLoss and CrossEntropyLoss respectively, which is implemented in (Ding et al., 2019b)https://github.com/BorealisAI/advertorch. More evaluations are shown in Appendix D. Discussion about effect of coefﬁcient β is shown in Appendix E.

Revised: In Appendix F, we show how the allowed maximum margin (i.e. ε max ) affects the performance, and how to choose its value.

Original: 3. E VALUATION ON M OONS DATASET
Revised: 3. E VALUATION ON M OONS DATASET

Original: The Moons dataset is available in sk-learn (Pedregosa et al., 2011).
Revised: The Moons dataset is available in sk-learn (Pedregosa et al., 2011).

Original: There are two classes of 2D points in the dataset.
Revised: There are two classes of 2D points in the dataset.

Original: The training set has 20000 samples, the validation set has 2000 samples, and the test set has 2000 samples.
Revised: The training set has 20000 samples, the validation set has 2000 samples, and the test set has 2000 samples.

Original: The neural network structure is Linear(2, 32)-LR-Linear(32, 64)-LN-LR-Linear(64, 128)-LN-LR-Linear(128, 2).
Revised: The neural network structure is Linear(2, 32)-LR-Linear(32, 64)-LN-LR-Linear(64, 128)-LN-LR-Linear(128, 2).

Original: The purpose of this evaluation is to visualize the decision boundaries of the models trained by different adversarial training methods.
Revised: The purpose of this evaluation is to visualize the decision boundaries of the models trained by different adversarial training methods.

Original: For every method, we set the number of training epochs to 30, maximum noise level to 0.3, and batch-size to 128; Adam optimizer was used with default parameters.
Revised: For every method, we set the number of training epochs to 30, maximum noise level to 0.3, and batch-size to 128; Adam optimizer was used with default parameters.

Original: To measure robustness on the test set, adversarial samples on different noise levels (deﬁned by L-inf norm) are generated by the 100-PGD.
Revised: To measure robustness on the test set, adversarial samples on different noise levels (deﬁned by L-inf norm) are generated by the 100-PGD.

Original: The results of decision boundaries and accuracy scores are shown in Fig.
Revised: The results of decision boundaries and accuracy scores are shown in Fig.

Original: The accuracy scores
Revised: The accuracy scores

Original: [Figure]
Revised: [Figure]

Original: are also reported in Table 1 in the appendix.
Revised: are also reported in Table 1 in the appendix.

Original: Our method IMA is better than the other methods, and its decision boundary is close to the “middle line” of the two classes.
Revised: Our method IMA is better than the other methods, and its decision boundary is close to the “middle line” of the two classes.

Original: The results of TRADES and adv0.3 are very similar: the decision boundary is far away from most of the red dots.
Revised: The results of TRADES and adv0.3 are very similar: the decision boundary is far away from most of the red dots.

Original: MMA is better than TRADES and adv0.3 when the noise level is less than 0.15, and then it becomes worse with larger noises.
Revised: MMA is better than TRADES and adv0.3 when the noise level is less than 0.15, and then it becomes worse with larger noises.

Original: Surprisingly, “ce” (model trained with cross-entropy loss and clean data) has very good robustness when the noise level is less than 0.15.
Revised: Surprisingly, “ce” (model trained with cross-entropy loss and clean data) has very good robustness when the noise level is less than 0.15.

Original: 3. E VALUATION ON F ASHION -MNIST AND SVHN DATASETS
Revised: 3. E VALUATION ON F ASHION -MNIST AND SVHN DATASETS

Original: We used Fashion-MNIST dataset instead of MNIST which is too easy.
Revised: We used Fashion-MNIST dataset instead of MNIST which is too easy.

Original: The dataset contains 28× grayscale images in 10 classes (shirts, shoes, etc.).
Revised: The dataset contains 28× grayscale images in 10 classes (shirts, shoes, etc.).

Original: The network structure is COV(1, 64, 5, 2, 2)-
Revised: The network structure is COV(1, 64, 5, 2, 2)-

Original: LR-COV(64, 128, 5, 2, 2)-IN-LR-COV (128, 256, 5, 2, 2)-IN-LR-COV (256, 512, 4, 1, 0)-FlattenLN-LR-Linear(512, 10).
Revised: LR-COV(64, 128, 5, 2, 2)-IN-LR-COV (128, 256, 5, 2, 2)-IN-LR-COV (256, 512, 4, 1, 0)-Flatten- LN-LR-Linear(512, 10).

Original: For every method, we set the number of training epochs to 60, maximum noise level to 0.3 for IMA, MMA and TRADES, and batch-size to 128.
Revised: For every method, we set the number of training epochs to 60, maximum noise level to 0.3 for IMA, MMA and TRADES, and batch-size to 128.

Original: Adam optimizer was used with default parameters.
Revised: Adam optimizer was used with default parameters.

Original: Besides adversarial training, we did not use any other data-augmentation (e.g. crop, etc.).
Revised: Besides adversarial training, we did not use any other data-augmentation (e.g. crop, etc.).

Original: The results on the test set are shown in Fig.
Revised: The results on the test set are shown in Fig.

Original: 7 and reported in Table 2 and Tablein the appendix.
Revised: 7 and reported in Table 2 and Tablein the appendix.

Original: We also used SVHN dataset which contains 32×32 color images of 0 ∼ 9 digits.
Revised: We also used SVHN dataset which contains 32×32 color images of 0 ∼ 9 digits.

Original: The network structure is COV(3, 32, 3, 1, 1)-LR-COV(32, 32, 3, 2, 1)-IN-LR-COV(32, 64, 3, 1, 1)-IN-LRConv2d(64, 64, 3, 2, 1)-IN-LR-COV(64, 128, 3, 1, 1)-IN-LR-Conv2d(128, 128, 3, 2, 1)-IN-LRCOV(128, 256, 4, 1, 0)-Flatten-LN-LR-Linear(256, 10).
Revised: The network structure is COV(3, 32, 3, 1, 1)-LR-COV(32, 32, 3, 2, 1)-IN-LR-COV(32, 64, 3, 1, 1)-IN-LRConv2d(64, 64, 3, 2, 1)-IN-LR-COV(64, 128, 3, 1, 1)-IN-LR-Conv2d(128, 128, 3, 2, 1)-IN-LRCOV(128, 256, 4, 1, 0)-Flatten-LN-LR-Linear(256, 10).

Original: Besides adversarial training, we did not use any other data-augmentation.
Revised: Besides adversarial training, we did not use any other data-augmentation.

Original: Training parameters (e.g. epochs) are the same as those for FashionMNIST.
Revised: Training parameters (e.g. epochs) are the same as those for FashionMNIST.

Original: The results on the test set are shown in Fig.
Revised: The results on the test set are shown in Fig.

Original: 8 and reported in Table 4 and Table 5 in the appendix.
Revised: 8 and reported in Table 4 and Table 5 in the appendix.

Original: Form the results shown in Fig.
Revised: Form the results shown in Fig.

Original: 7 and Fig.
Revised: 7 and Fig.

Original: 8 on the two datasets, it can be seen that “ce” had the worst performance on noisy samples, “adv ε ” is very sensitive to the maximum noise level ε for training, and TRADES did not perform well on L-inf norm-based tests.
Revised: 8 on the two datasets, it can be seen that “ce” had the worst performance on noisy samples, “adv ε ” is very sensitive to the maximum noise level ε for training, and TRADES did not perform well on L-inf norm-based tests.

Original: Compared to “ce”, accuracy of every adversarial training method on clean data is slightly lower, which shows the sign of tradeoff between robustness and accuracy on clean data.
Revised: Compared to “ce”, accuracy of every adversarial training method on clean data is slightly lower, which shows the sign of tradeoff between robustness and accuracy on clean data.

Original: Recall that IMA signiﬁcantly outperformed the other methods on the Moons dataset, which demonstrates its strong potential.
Revised: Recall that IMA signiﬁcantly outperformed the other methods on the Moons dataset, which demonstrates its strong potential.

Original: On Fashion-MNIST and SVHN datasets, IMA also signiﬁcantly outperformed TRADES and “adv ε ”.
Revised: On Fashion-MNIST and SVHN datasets, IMA also signiﬁcantly outperformed TRADES and “adv ε ”.

Original: However, IMA is only slightly better than MMA on Fashion-MNIST and on-par with MMA on SVHN.
Revised: However, IMA is only slightly better than MMA on Fashion-MNIST and on-par with MMA on SVHN.

Original: The non-ideal performance of IMA is hightly likely caused by data scarcity in high dimensional space, so that the ideal equilibrium cannot be reached.
Revised: The non-ideal performance of IMA is hightly likely caused by data scarcity in high dimensional space, so that the ideal equilibrium cannot be reached.

Original: To further improve its performance, generative adversarial training could be used to generate more clean samples to ﬁll the “holes” of the space, which will be our future work.
Revised: To further improve its performance, generative adversarial training could be used to generate more clean samples to ﬁll the “holes” of the space, which will be our future work.

Original: Nevertheless, our method IMA is at least on-par with MMA method which is start-of-art.
Revised: Nevertheless, our method IMA is at least on-par with MMA method which is start-of-art.

Original: 3. A PPLICATION OF COVID-19 DETECTION FROM CT IMAGES
Revised: 3. A PPLICATION OF COVID-19 DETECTION FROM CT IMAGES

Original: We used a public COVID-

Revised: We used a public COVID-

Original: CT image dataset (Soares et al., 2020), which is collected from patients in hospitals from Sao Paulo, Brazil.
Revised: CT image dataset (Soares et al., 2020), which is collected from patients in hospitals from Sao Paulo, Brazil.

Original: It contains 1252 CT scans (2D images) that are positive for COVID-19 infection and 1230 CT scans (2D images) for patients non-infected by COVID-19, 2482 CT scans in total.
Revised: It contains 1252 CT scans (2D images) that are positive for COVID-19 infection and 1230 CT scans (2D images) for patients non-infected by COVID-19, 2482 CT scans in total.

Original: From infected cases, we randomly selected 200 samples for testing, 30 for validation, and 1022 for training.
Revised: From infected cases, we randomly selected 200 samples for testing, 30 for validation, and 1022 for training.

Original: From the uninfected cases, we randomly selected 200 for testing, 30 for validation and 1000 for training.
Revised: From the uninfected cases, we randomly selected 200 for testing, 30 for validation and 1000 for training.

Original: Each image is resized to 224 × 224 .
Revised: Each image is resized to 224 × 224 .

Original: Since there is no pixel segmentation mask in the dataset, we can only perform classiﬁcation for COVID-19 detection on the image level.
Revised: Since there is no pixel segmentation mask in the dataset, we can only perform classiﬁcation for COVID-19 detection on the image level.

Original: We modiﬁed the output layer of the Resnet-18 model (He et al., 2016) for the binary classiﬁcation task: uninfected (label 0) vs infected (label 1), and we also replaced batch normalization with instance normalization because it is known that batch normalization is not stable for small batch-size (Wu & He, 2018).
Revised: We modiﬁed the output layer of the Resnet-18 model (He et al., 2016) for the binary classiﬁcation task: uninfected (label 0) vs infected (label 1), and we also replaced batch normalization with instance normalization because it is known that batch normalization is not stable for small batch-size (Wu & He, 2018).

Original: As shown in the previous studies (Shi et al., 2020), infected regions in the images have a special pattern called ground-glass opacity.
Revised: As shown in the previous studies (Shi et al., 2020), infected regions in the images have a special pattern called ground-glass opacity.

Original: For every method, we set the number of training epochs to 100, and maximum noise level to 0.3 for
Revised: For every method, we set the number of training epochs to 100, and maximum noise level to 0.3 for

Original: IMA, MMA and TRADES, and batch-size to 32.
Revised: IMA, MMA and TRADES, and batch-size to 32.

Original: Adam optimizer was used with default parameters.
Revised: Adam optimizer was used with default parameters.

Original: For “ce”, weight decay of 0.01 is applied with Adam (i.e. AdamW with default parameters).
Revised: For “ce”, weight decay of 0.01 is applied with Adam (i.e. AdamW with default parameters).

Original: To measure robustness on the test set, adversarial samples on different noise levels (L-inf norm) are generated by the 100-PGD.
Revised: To measure robustness on the test set, adversarial samples on different noise levels (L-inf norm) are generated by the 100-PGD.

Original: We choose L-inf over L2 norm because L-inf norm can be used to control the maximum amplitude of noises on individual pixels, which resembles random white noises from a uniform distribution.
Revised: We choose L-inf over L2 norm because L-inf norm can be used to control the maximum amplitude of noises on individual pixels, which resembles random white noises from a uniform distribution.

Original: We also tested the effect of adding uniform white noises to the images, and the trained model is denoted by “rand ε ” where ε is the maximum noise level/amplitude.
Revised: We also tested the effect of adding uniform white noises to the images, and the trained model is denoted by “rand ε ” where ε is the maximum noise level/amplitude.

Original: “rand ε ” can be considered a weak form of adversarial training, analog to “adv ε ”.
Revised: “rand ε ” can be considered a weak form of adversarial training, analog to “adv ε ”.

Original: The results on the test set are shown in Fig.9 and reported in Table 6 in the appendix.
Revised: The results on the test set are shown in Fig.9 and reported in Table 6 in the appendix.

Original: Examples of clean and noisy images are shown in Fig.
Revised: Examples of clean and noisy images are shown in Fig.

Original: 10 (see in Appendix).
Revised: 10 (see in Appendix).

Original: The results show that IMA has the strongest overall performance, and MMA is slightly better than
Revised: The results show that IMA has the strongest overall performance, and MMA is slightly better than

Original: IMA when noise level is larger than 0.2.
Revised: IMA when noise level is larger than 0.2.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 10, when noise level is larger than 0.1, the noisy images look signiﬁcantly different from the clean images, and those noisy images cannot be from a well-functioning CT machine and may even look like images of infection.
Revised: 10, when noise level is larger than 0.1, the noisy images look signiﬁcantly different from the clean images, and those noisy images cannot be from a well-functioning CT machine and may even look like images of infection(Hamper et al., 2020).

Original: Thus, a model robust to noises less than the level of 0.2 is good enough for this application.
Revised: Thus, a model robust to noises less than the level of 0.2 is good enough for this application.

Original: Also, from Fig.
Revised: Also, from Fig.

Original: 9 (right), we can see that most of the estimated margins of MMA were larger than 0.3, which means most of the samples should have been free of inﬂuence of noises of magnitude less than 0.3.
Revised: 9 (right), we can see that most of the estimated margins of MMA were larger than 0.3, which means most of the samples should have been free of inﬂuence of noises of magnitude less than 0.3.

Original: However, if this were true, the accuracy of MMA in Fig.
Revised: However, if this were true, the accuracy of MMA in Fig.

Original: 9 (left) at noise of 0.3 should not have been signiﬁcantly different from that of “adv0.3”.
Revised: 9 (left) at noise of 0.3 should not have been signiﬁcantly different from that of “adv0.3”.

Original: The results show that MMA signiﬁcantly overestimated the margins of samples, because that MMA does not consider the equilibrium (Fig.5) between classes.
Revised: The results show that MMA signiﬁcantly overestimated the margins of samples, because that MMA does not consider the equilibrium (Fig.5) between classes.

Original: From section 1 and 2, it can be seen that the vanilla adversarial training (“adv ε ”) is quite good when the maximum noise level ε for training is chosen “properly”.
Revised: From section 1 and 2, it can be seen that the vanilla adversarial training (“adv ε ”) is quite good when the maximum noise level ε for training is chosen “properly”.

Original: The question arises: what is the proper ε of “adv ε ”?
Revised: The question arises: what is the proper ε of “adv ε ”?

Original: A straightforward way to ﬁnd a good (cid:15) would be running grid-research and evaluating the performance on the validation set.
Revised: A straightforward way to ﬁnd a good (cid:15) would be running grid-research and evaluating the performance on the validation set.

Original: However, the grid-research is impractical because adversarial training is computationally expensive and time-consuming, compared to training with cross-entropy loss on clean data.
Revised: However, the grid-research is impractical because adversarial training is computationally expensive and time-consuming, compared to training with cross-entropy loss on clean data.

Original: It turns out that the sample margin distribution ( E in Algorithm 1) estimated by our method IMA indicates a good ε = 0 .
Revised: It turns out that the sample margin distribution ( E in Algorithm 1) estimated by our method IMA indicates a good ε = 0 .

Original: 1 (Fig. 9).
Revised: 1 (Fig. 9).

Original: For this application, “adv0.1” is as competitive as other advanced methods and has a less computational cost.
Revised: For this application, “adv0.1” is as competitive as other advanced methods and has a less computational cost.

Original: Although adversarial training may reduce accuracy on clean data, for this application, one can tweak the classiﬁcation threshold to increase the sensitivity to infection, which is often desired.
Revised: Although adversarial training may reduce accuracy on clean data, for this application, one can tweak the classiﬁcation threshold to increase the sensitivity to infection, which is often desired.

Original: The results also show that adding random noises to the input can improve classiﬁcation accuracy on clean samples (from 0.9575 of “ce” to 0.9775 of “rand0.1”).
Revised: The results also show that adding random noises to the input can improve classiﬁcation accuracy on clean samples (from 0.9575 of “ce” to 0.9775 of “rand0.1”).

Original: However, random noise-based data augmentation (“rand0.1”, “rand0.2” and “rand0.3”) has little effect on adversarial robustness for this application, which is easy to explain: not every noise (vector) is in the direction that can drive the input across the decision boundary.
Revised: However, random noise-based data augmentation (“rand0.1”, “rand0.2” and “rand0.3”) has little effect on adversarial robustness for this application, which is easy to explain: not every noise (vector) is in the direction that can drive the input across the decision boundary.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this study, we proposed a novel Increasing-Margin Adversarial (IMA) training method to improve the robustness of DNN for classiﬁcation tasks and applied it for the application of COVID-19 detection from CT images.
Revised: In this study, we proposed a novel Increasing-Margin Adversarial (IMA) training method to improve the robustness of DNN for classiﬁcation tasks and applied it for the application of COVID-19 detection from CT images.

Original: Our method aims to increase the margins of the training samples to improve
Revised: Our method aims to increase the margins of the training samples to improve

Original: DNN robustness against adversarial attacks.
Revised: DNN robustness against adversarial attacks.

Original: The experiment results show that our method signiﬁcantly improved DNN classiﬁcation accuracy on noisy data while keeping a relatively high accuracy on clean data, and its performance is at least on-par with the start-of-art MMA method.
Revised: The experiment results show that our method signiﬁcantly improved DNN classiﬁcation accuracy on noisy data while keeping a relatively high accuracy on clean data, and its performance is at least on-par with the start-of-art MMA method.

Original: We also show that the vanilla adversarial training with a properly chosen ε is as competitive as other advanced methods for the COVID-19 detection application, and our method IMA can provide a good ε for it from the sample margin distribution.
Revised: We also show that the vanilla adversarial training with a properly chosen ε is as competitive as other advanced methods for the COVID-19 detection application, and our method IMA can provide a good ε for it from the sample margin distribution.

Original: We hope our approach may facilitate the development of robust DNNs, especially in the medical ﬁeld.
Revised: We hope our approach may facilitate the development of robust DNNs, especially in the medical ﬁeld.

Original: Note: (1) we will release the code when the paper is accepted.
Revised: Note: (1) we will release the code when the paper is accepted.

Original: (2) all ﬁgures are in high resolution, please zoom in.
Revised: (2) all ﬁgures are in high resolution, please zoom in.

Original: A A PPENDIX
Revised: A A PPENDIX

Revised: It seems that adversarial noises are created by algorithms (e

Revised: [Table]

Revised: g. PGD)

Original: We encountered a problem when we were trying to evaluate IMA and MMA on cifar10.
Revised: and therefore it is only a security issue caused by hackers. In fact, random imaging noises could also be ”adversarial” noises leading to wrong classiﬁcations. We did an additional test and found out that 2 . 75% of the noisy samples with uniform white noises on the level of 0.05, can cause the model ”ce” to make wrong classiﬁcations.

Revised: 75% is not a negligible number for this application.

Revised: We note that CT imaging noises can be better described by Poisson distribution (Wang et al., 2008).

Revised: However, without the hardware parameters of the CT machine, it is impossible to simulate Poisson noises.

Revised: Nevertheless, adversarial robustness should be the built-in property of a model for this application

Revised: [Table]

Revised: and all of the

Revised: DNN models in the previous COVID-19 studies (Shi et al., 2020) should be checked and enhanced for adversarial robustness before deploying those models in clinics and hospitals.

Revised: B A PPENDIX

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Figure]

Revised: C A PPENDIX

Revised: D A PPENDIX D. E VALUATION ON MNIST DATASET

Revised: We evaluated IMA on MNIST dataset.

Revised: The maximum noise level (i.e. ε max ) matches the corresponding parameter in MMA ( 4 . 0 for L2-norm deﬁned noise level and 0 .

Revised: 45 for Linf-deﬁned noise level)

Revised: ; Adamax optimizer was used with default parameters, and other settings are the same as those for Fashion-MNIST.

Revised: The CNN model is the same as that in the MMA paper.

Revised: We run the IMA method twice with two different random seeds and name the resulting models as IMA s0 and IMA s1.

Revised: The scores of MMA are directly copied from the MMA paper (Ding et al., 2019a).

Revised: The settings of PGD attack are the same as those in the MMA paper.

Revised: The results are reported in Table 7 and Table 8, which show that the performance of the two methods is similar.

Revised: [Table]

Revised: [Table]

Revised: To show that IMA does not do gradient obfuscation, we applied a black-box attack to the trained model using the SPSA method (Uesato et al., 2018).

Revised: For the SPSA attack, the number of SPSA samples is 2018, the maximum number of iterations is 100, delta is 0.01 and learning rate is 0.01.

Revised: Since SPSA is extremely time-consuming, we only applied it for the ﬁrst 100 samples in the test set, which is the same approach used in the MMA paper.

Revised: We also applied 100-PGD attack for the ﬁrst 100 samples in the test set.

Revised: The results are reported in Table 9 which shows that SPSA attack is not stronger than PGD attack, and therefore there is no gradient obfuscation.

Original:  MMA(Ding et al., 2019a) was evaluated on cifar10 with standard data augmentation (e g. crop).
Revised: [Table] D. E VALUATION ON CIFAR10 DATASET We encountered a problem when we were trying to evaluate IMA and MMA on cifar10. MMA(Ding et al., 2019a) was evaluated on cifar10 with standard data augmentation (e.g. crop).

Original: Without standard data augmentation it is impossible to achieve a classiﬁcation accuracy of > 90% on clean data.
Revised: Without standard data augmentation, it is impossible to achieve a classiﬁcation accuracy of > 90% on clean data.

Original: However, standard data augmentation imposes two challenges:
Revised: However, standard data augmentation imposes two challenges:

Original: (1) standard data augmentation produces a random set of augmented images, and the randomness may affect robustness (2) both IMA and
Revised: (1) standard data augmentation produces a random set of augmented images, and the randomness may affect robustness (2) both IMA and

Original: MMA need to record the margin of each sample.
Revised: MMA need to record the margin of each sample.

Original: Since the augmented images are generated online, then it is impossible to assign an ID to each augmented image.
Revised: Since the augmented images are generated online, then it is impossible to assign an ID to each augmented image.

Original: The way of MMA handling this issue is not ideal: it assigns the same ID to image x and its augmented version x aug .
Revised: The way of MMA handling this issue is not ideal: it assigns the same ID to image x and its augmented version x aug .

Original: In fact, x and x aug 
Revised: In fact, x and x aug have

Original: have different margins and thus need
Revised: different margins and thus need

Original: different
Revised: different

Original: IDs in the margin 
Revised: IDs in the margin data array

Original: data array. The solution could be to generate a large number of images from standard data augmentation and save those images to hard-drive before training.
Revised: The solution could be to generate a large number of images from standard data augmentation and save those images to hard-drive before training.

Original: This solution needs a huge amount of storage and computing power (the margin of every sample needs to be expanded), which our humble research lab currently does not have.
Revised: This solution needs a huge amount of storage and computing power (the margin of every sample needs to be expanded), which our humble research lab currently does not have.

Original: Thus, we leave the evaluate on cifar10 (and other datasets that need standard data augmentation) to our future investigation. Our current experiments have empirically veriﬁed our theory and demonstrated the effectiveness of IMA on four datasets including a COVID-19 CT dataset. B A PPENDIX
Revised: Per the request of the reviewer, we evaluated the IMA method by assuming x and x aug have the same margin.

Original: It seems that adversarial noises are created by algorithms (e.g. PGD) and therefore it is only a security issue caused by hackers. In fact, random imaging noises could also be ”adversarial” noises leading to wrong classiﬁcations. We did an additional test and found out that 2 . 75% of the noisy samples with uniform white noises on the level of 0.05, can cause the model ”ce” to make wrong classiﬁcations.
Revised: The maximum noise level (i.e. ε max ) matches the corresponding parameter in MMA (3.0 for L2-norm deﬁned noise level and 32 / 255 for Linf-deﬁned noise level); SGD optimizer was used with momentum 0.9 and initial learning rate 0.1.

Original: 75% is not a negligible number for this application.
Revised: The learning rate is reduced by half after every 25 epochs, and the total number of training epochs is 200.

Original: We note that CT imaging noises can be better described by Poisson distribution (Wang et al., 2008). However, without the hardware parameters of theCT machine, it is impossible to simulate Poisson noises.
Revised: Since MMA used 10-PGD for adversarial training, IMA used 10 PGD-iterations in algorithm 3. The CNN model is the same as that in the

Original: Nevertheless, adversarial robustness should be the built-in property of a model for this application, and all of the
Revised: MMA paper.

Original: DNN models in the previous COVID-19 studies (Shi et al., 2020) should be checked and enhanced for adversarial robustness before deploying those models in clinics and hospitals.
Revised: We run the IMA method twice with two different random seeds and name the resulting models as IMA s0 and IMA s1. The scores of MMA are directly copied from the MMA paper.

Original: C A PPENDIX
Revised: The settings of PGD attack are the same as those in the MMA paper.

Original: [Table] [Table] [Table] [Table]
Revised: The results are reported in Table 10 and Table 11, which show that the performance of the two methods are similar.

Original:  MMA 0. TRADES 0. adv0.
Revised: In general, IMA has higher accuracy on clean data, while MMA has higher accuracy on noisy data.

Original: adv0. adv0.
Revised: [Table] [Table]

Original: ce 0.
Revised: To show that IMA does not do gradient obfuscation

Original: [Table]
Revised: [Table]

Revised: we applied the SPSA black-box attack to the trained model, using the same settings.

Revised: The results are reported in Table 12 which shows that SPSA attack is not stronger than PGD attack

Original: [Table]
Revised: [Table]

Revised: and therefore there is no gradient obfuscation

Original: [Table] 
Revised: [Table] [Table]

Revised: We note that the user could tweak the parameters of IMA on the validation set to further improve the performance.

Original: [Table]
Revised: In the next two appendices, Appendix E and Appendix F, we show how each of the two parameters β and ε max can be used to make a trade-off between robustness and accuracy on clean data. E A PPENDIX We evaluated the effect of β in the IMA method on the Fashion-MNIST dataset. The results are reported in the Table 13 and Table 14.

Original: [Table] 
Revised: [Table] [Table]

Revised: It can be clearly seen that smaller β leads to higher accuracy on clean data (noise level = 0) and larger β leads to higher accuracy on noisy data.

Revised: The trade-off between robustness and accuracy is highly nonlinear , as shown in the Table 15: a small decrease in accuracy on clean data can result in a large increase in accuracy on noisy data.

Original: [Table]
Revised: [Table]

Revised: The nonlinear trade-off between robustness and accuracy makes it difﬁcult to directly compare two methods (e.g. IMA and MMA), as different methods make different trade-off between robustness and accuracy.

Revised: The average accuracy is clearly not a good measure of performance on robustness.

Revised: F A PPENDIX

Revised: In this appendix, we show how the allowed maximum margin (i.e. ε max ) affects the performance, and how to choose its value.

Revised: Noisy samples with adversarial noises can be correctly-classiﬁed by humans but may be incorrectlyclassiﬁed by neural networks, which is basically the deﬁnition of adversarial noises.

Revised: By following this deﬁnition, we choose the allowed maximum margin by looking at the noisy samples: if the noise magnitude is too large such that we barely can recognize the objects in the images, then this magnitude is chosen as the allowed maximum margin.

Revised: This is how we choose the allowed maximum margin in the experiments on the datasets, except for MNIST and CIFAR, for which the allowed maximum margin matches the corresponding parameter of the MMA method.

Revised: Next, we provide another strategy for the user to choose the allowed maximum margin by making a trade-off between robustness and accuracy on clean data.

Revised: As shown in Figure 11, the trade-off between clean-accuracy (i.e. accuracy on clean data) and robustness (i.e. accuracy on noisy data) can be observed during training by using our IMA method.

Revised: During IMA training, the (estimated) margin of a sample x 1 in class-1 is initialized to be close toand it keeps increasing as if the radius of a ball increases during the training process.

Revised: The sample xis at the center of the ball, and the radius of the ball is the current margin of the sample.

Original: [Table]
Revised: When the ball of x 1 collides with the ball of another sample x 2 from a different class-2, then a local decision boundary is formed, and the two balls stop expanding. If there are enough training data samples, a sample x in some class will eventually meet with its counterpart in another class somewhere in the middle of the two classes, which forms a robust decision boundary. In practice, the amount of training data is never enough to cover the input space, and therefore the margin of a sample could be overestimated because of missing the counterparts that are needed to stop the expansion of the margin. If the margins of many samples are overestimated, then the balls of these samples may penetrate the optimal decision boundary and cause lower classiﬁcation accuracy on the validation set of clean samples, which explains the existence of the trade-off between robustness and clean-accuracy from the perspective of sample margins.

Revised: The above analysis is conﬁrmed by the trend of the training and validation accuracy curves in the some accuracy becomes stable, and the validation accu racy starts to decrease, which margin overestimation method the cause of margin overestimation lack high dimensional space moons dataset there are enough training samples, and therefore, the decision boundary of our IMA method is right in the middle between the two classes, i.e., no margin overestimation

Revised: Using the validation accuracy curve, we can choose the allowed maximum margin such that validation accuracy is above a pre-deﬁned threshold that is set by the user to meet some application requirements.

Revised: Next, we plot the curves from the COVID-19 dataset.

Revised: The validation accuracy decreased only slightly after 20 epochs, which means the value of the allowed maximum margin is reasonable and the risk of margin overestimation is low.

Original: [Table]
Revised: We note that using the margin distribution estimated by IMA, we can obtain a good perturbation mangnitude ε (0.1 or 0.15) in the vanilla adversarial training. When ε =0.15, vanilla adversarial training achieved the best performance on the COVID-19 CT dataset, compared to other methods. Please see Figure 14 which contains the margin distributions estimated by IMA and MMA, and Table 6 which contains accuracy scores in a large range of noise level. In practice, we only need robustness against noises within a certain level, because signiﬁcantly-noisy images can only be produced from a malfunctioning CT machine.

Revised: Thus, vanilla adversarial training should be good enough as long as its parameter ε is appropriate.

Revised: In this application, good values of ε are 0.1 and 0.15, as revealed by the margin distribution estimated by IMA.

Original: [Table]
Revised: For the convenience of the reader, we re-plot the results in the new Figure 14. For vanilla adversarial training, a straightforward way to ﬁnd a good ε would be running gridresearch and evaluating the performance on the validation set. However, the grid-research in a large range (e.g. 0 to 0.3) is impractical because adversarial training is computationally expensive and time-consuming, compared to standard training with cross-entropy loss and clean data. The margin distribution estimated by IMA can reveal a good range of the ε , and then grid-research in this small range can be performed to ﬁnd the optimal ε .

Revised: After the optimal ε is found for vanilla adversarial training, we could could combine other techniques to further improve robustness.

Original: TRADES
Revised: If there are only two classes and the data samples are linearly-sparable, then SVM (support vector machine) will produce a linear decision boundary in the ”middle” between the two classes. The decision boundary of SVM is robust against noises: the classiﬁcation output will not change if a small amount of noise δ is added to x as long as the vector norm of δ is smaller than the margin of x . Here, the margin of x is the (minimum) distance between x and the decision boundary.

Revised: In general, the data samples in multiple classes are nonlinearly-separable, and the robust decision boundary should be somewhere in the middle between classes, which is the goal that IMA pursues.

Original: [Table]
Revised: The result on the moons dataset shows that IMA can indeed produce a nonlinear decision boundary in the ”middle” between classes. We use 2D moons dataset because it is impossible to directly visualize a nonlinear decision boundary in a high dimensional space. Also, we would like to ask the reader to think about this question: if a method cannot produce a nonlinear decision boundary in the ”middle” between classes for this 2D dataset, can the method produce a nonlinear decision boundary in the ”middle” between classes for a higher dimensional dataset? G. T HE EQUILIBRIUM STATE

Revised: Our IMA method is a heuristic-based method that is not derived from any theory.

Revised: We use the equilibrium state analysis to provide a theoretical explanation of the method.

Revised: We have shown that an equilibrium state can be achieved when the noisy samples have the same spatial distribution on the decision boundary.

Original: [Table]
Revised: Here, we will analyze what will happen if the spatial distributions of the noisy samples in different classes are not the same on the current decision boundary. We note that our IMA method will actively generate and put noisy samples on (”close to” due to numerical precision) the current decision boundary of the neural network model, and the training is a dynamic process to adjust the decision boundary of the model.

Original: [Table]
Revised: Let’s focus on the following two terms: [Equation] where q i ( x ) and q j ( x ) are the distributions (i.e. densities) of the noisy samples on the current decision boundary between the two classes, and q i ( x ) and q j ( x ) may not be equal to each other. In fact, F i and F j can be interpreted as two forces that try to expand the margins of the samples in the two classes against each other.

Original: [Table]
Revised: By dividing the decision boundary into small regions (i.e. linear segments), the two integrals can be evaluated in the individual regions. In a region, if q i ( x ) > q j ( x ) (i.e. more samples in class-i) then the current state is not in equilibrium: after updating the model using these noisy samples, the noisy samples in class-i will be correctly classiﬁed and the noisy samples in class-j will be incorrectly-classiﬁed (this is a simple result of classiﬁcation with imbalanced data in the region), which means the decision boundary will shift towards the samples in class-j, and therefore the margins of the corresponding samples in class-i will expand and the margins of the corresponding samples in class-j will shrink. Thus, the decision boundary may shift locally towards the samples in one of the classes.

Revised: Obviously, the decision boundary will stop shifting when the local densities of noisy samples in different classes are the same along the decision boundary, i.e. q i ( x ) becomes equal to q j ( x ) , which means an equilibrium state is reached.

Revised: This analysis is a sketch of the dynamic process during training, and we defer a math-rigor analysis to our future work.

Revised: Two questions arise from the above analysis:

Revised: (1) Will there exist a decision boundary in the middle between classes, along which the local densities of noisy samples in different classes are roughly the same?

Original: [Figure]
Revised: Clearly, the existence of such a decision boundary is determined by the dataset itself. If the samples in different classes are well separated (e.g. the datasets used in our experiments), we hypothesize that such a decision boundary exists, just like the decision boundary of IMA on the 2D moons dataset. (2) Assuming such an optimal robust decision boundary exists on a dataset, will the IMA method ﬁnd it? Although it cannot give a theoretical guarantee, the experimental results show that our IMA method can indeed increase the margins of the samples on different datasets: increasing margin means increasing robustness.