Original: I MPLICIT B IAS OF G RADIENT D ESCENT BASED A DVERSARIAL T RAINING ON S EPARABLE D ATA
Revised: I MPLICIT B IAS OF G RADIENT D ESCENT BASED A DVERSARIAL T RAINING ON S EPARABLE D ATA

Original: A BSTRACT
Revised: A BSTRACT

Original: Adversarial training is a principled approach for training robust neural networks.
Revised: Adversarial training is a principled approach for training robust neural networks.

Original: Despite of tremendous successes in practice, its theoretical properties still remain largely unexplored.
Revised: Despite of tremendous successes in practice, its theoretical properties still remain largely unexplored.

Original: In this paper, we provide new theoretical insights of gradient descent based adversarial training by studying its computational properties, speciﬁcally on its implicit bias.
Revised: In this paper, we provide new theoretical insights of gradient descent based adversarial training by studying its computational properties, speciﬁcally on its implicit bias.

Original: We take the binary classiﬁcation task on linearly separable data as an illustrative example, where the loss asymptotically attains its inﬁmum as the parameter diverges to inﬁnity along certain directions.
Revised: We take the binary classiﬁcation task on linearly separable data as an illustrative example, where the loss asymptotically attains its inﬁmum as the parameter diverges to inﬁnity along certain directions.

Original: Speciﬁcally, we show that for any ﬁxed iteration T , when the adversarial perturbation during training has proper bounded (cid:96) 2 -norm, the classiﬁer learned by gradient descent based adversarial training converges in direction to the maximum (cid:96) 2 -norm margin classiﬁer at the rate of (cid:101)
Revised: Speciﬁcally, we show that for any ﬁxed iteration T , when the adversarial perturbation during training has proper bounded (cid:96) 2 -norm, the classiﬁer learned by gradient descent based adversarial training converges in direction to the maximum (cid:96) 2 -norm margin classiﬁer at the rate of (cid:101)

Original: O
Revised: O

Original: (1 / √ T ) , signiﬁcantly faster than the rate O
Revised: (1 / √ T ) , signiﬁcantly faster than the rate O

Original: (1 / log T ) of training with clean data.
Revised: (1 / log T ) of training with clean data.

Original: In addition, when the adversarial perturbation during training has bounded (cid:96) q -norm with q ≥ 1 , the resulting classiﬁer converges in direction to a maximum mixed-norm margin classiﬁer, which has a natural interpretation of robustness, as being the maximum (cid:96) 2 - norm margin classiﬁer under worst-case (cid:96) q -norm perturbation to the data.
Revised: In addition, when the adversarial perturbation during training has bounded (cid:96) q -norm with q ≥ 1 , the resulting classiﬁer converges in direction to a maximum mixed-norm margin classiﬁer, which has a natural interpretation of robustness, as being the maximum (cid:96) 2 -norm margin classiﬁer under worst-case (cid:96) q -norm perturbation to the data.

Original: Our ﬁndings provide theoretical backups for adversarial training that it indeed promotes robustness against adversarial perturbation
Revised: Our ﬁndings provide theoretical backups for adversarial training that it indeed promotes robustness against adversarial perturbation

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep neural networks have achieved remarkable success on various tasks, including visual and speech recognitions, with intriguing generalization abilities to unseen data (Krizhevsky et al., 2012; Hinton et al., 2012).
Revised: Deep neural networks have achieved remarkable success on various tasks, including visual and speech recognitions, with intriguing generalization abilities to unseen data (Krizhevsky et al., 2012; Hinton et al., 2012).

Original: One salient feature of deep models is its overparameterization, with the number of parameters several orders of magnitude larger than the training sample size.
Revised: One salient feature of deep models is its overparameterization, with the number of parameters several orders of magnitude larger than the training sample size.

Original: As a consequence of such overparameterization, it is likely that the empirical loss function, in addition to being non-convex, can have substantial amount of global minimizers (Choromanska et al., 2015), while only a small subset of global minimizers have the desired generalization properties (Brutzkus et al., 2018).
Revised: As a consequence of such overparameterization, it is likely that the empirical loss function, in addition to being non-convex, can have substantial amount of global minimizers (Choromanska et al., 2015), while only a small subset of global minimizers have the desired generalization properties (Brutzkus et al., 2018).

Original: Contrary to the worst-case reasoning above, researchers have observed that simple ﬁrst-order algorithm such as Stochastic Gradient Descent (SGD) 1 , performs surprisingly well in practice, even without any explicit regularization terms in the objective function (Zhang et al., 2017).
Revised: Contrary to the worst-case reasoning above, researchers have observed that simple ﬁrst-order algorithm such as Stochastic Gradient Descent (SGD) 1 , performs surprisingly well in practice, even without any explicit regularization terms in the objective function (Zhang et al., 2017).

Original: Inspired by classical computational learning theories, one plausible explanation of such a remarkable phenomenon is that the training algorithm enjoys some implicit bias.
Revised: Inspired by classical computational learning theories, one plausible explanation of such a remarkable phenomenon is that the training algorithm enjoys some implicit bias.

Original: That is, the training algorithm tends to converge to certain kinds of solutions (Neyshabur et al., 2015b;c), and SGD converges to low-capacity solutions with the desired generalization property (Brutzkus et al., 2018).
Revised: That is, the training algorithm tends to converge to certain kinds of solutions (Neyshabur et al., 2015b;c), and SGD converges to low-capacity solutions with the desired generalization property (Brutzkus et al., 2018).

Original: Recently, some exciting works have related the implicit bias to speciﬁc ﬁrst-order algorithms (Wilson et al., 2017), stopping time (Hoffer et al., 2017), and optimization geometry (Gunasekar et al., 2018a;
Revised: Recently, some exciting works have related the implicit bias to speciﬁc ﬁrst-order algorithms (Wilson et al., , stopping time (Hoffer et al., 2017), and optimization geometry (Gunasekar et al., 2018a;

Original: Keskar et al., 2017).
Revised: Keskar et al., 2017).

Original: Some practical suggestions based on these ﬁndings have also been proposed to further improve the generalization ability of deep networks (Neyshabur et al., 2015a).
Revised: Some practical suggestions based on these ﬁndings have also been proposed to further improve the generalization ability of deep networks (Neyshabur et al., 2015a).

Original: Despite the aforementioned phenomenal success achieved by deep neural networks, it is observed that adversarially constructed small perturbation to the input can potentially fool the network into making wrong predictions with high conﬁdence (Szegedy et al., 2014; Goodfellow et al., 2015).
Revised: Despite the aforementioned phenomenal success achieved by deep neural networks, it is observed that adversarially constructed small perturbation to the input can potentially fool the network into making wrong predictions with high conﬁdence (Szegedy et al., 2014; Goodfellow et al., 2015).

Original: This issue raises serious concerns about using neural network for some security-sensitive tasks (Papernot et al., 2017).
Revised: This issue raises serious concerns about using neural network for some security-sensitive tasks (Papernot et al., 2017).

Original: Researchers have devised various mechanisms to generate and defend against adversarial perturbations (Goodfellow et al., 2015; Moosavi-Dezfooli et al., 2016; Carlini and Wagner, 2017;
Revised: Researchers have devised various mechanisms to generate and defend against adversarial perturbations (Goodfellow et al., 2015; Moosavi-Dezfooli et al., 2016; Carlini and Wagner, 2017;

Original: Athalye et al., 2018; Xie et al., 2018; Papernot et al., 2016).
Revised: Athalye et al., 2018; Xie et al., 2018; Papernot et al., 2016).

Original: However, most of the defense mechanisms are heuristic or ad-hoc, which lack principled theoretical justiﬁcation (Carlini and Wagner, 2016;
Revised: However, most of the defense mechanisms are heuristic or ad-hoc, which lack principled theoretical justiﬁcation (Carlini and Wagner, 2016;

Original: He et al., 2017).
Revised: He et al., 2017).

Original: Inspired by literatures in robust optimization (Wald, 1939;
Revised: Inspired by literatures in robust optimization (Wald, 1939;

Original: Ben-Tal et al., 2009),
Revised: Ben-Tal et al., 2009),

Original: Feige et al.
Revised: Feige et al.

Original: ; Madry et al.
Revised: ; Madry et al.

Original: (2018) formalize the notion of achieving adversarial robustness (i.e., having small adversarial risk) as solving the following minimax optimization problem
Revised: (2018) formalize the notion of achieving adversarial robustness (i.e., having small adversarial risk) as solving the following minimax optimization problem

Original: [Equation]
Revised: [Equation]

Original:  where ∆ is the set that each sample could be contaminated by arbitrary perturbation chosen within this set.
Revised: (1) where ∆ is the set that each sample could be contaminated by arbitrary perturbation chosen within this set.

Original: As a common practice, adversarial training refers to the ﬁnite-sample empirical version of (1) without access to the underlying distribution D that
Revised: As a common practice, adversarial training refers to the ﬁnite-sample empirical version of (1) without access to the underlying distribution D that

Original: [Equation]
Revised: [Equation]

Original: A commonly adopted approach to solving (2) is the the Gradient Descent based Adversarial Training (GDAT) method.
Revised: A commonly adopted approach to solving (2) is the the Gradient Descent based Adversarial Training (GDAT) method.

Original: At each iteration, GDAT ﬁrst solves the inner maximization problem (approximately) for adversarial perturbations, and then uses the gradient of the loss function evaluated at the perturbed samples to perform a gradient descent step on the parameter θ .
Revised: At each iteration, GDAT ﬁrst solves the inner maximization problem (approximately) for adversarial perturbations, and then uses the gradient of the loss function evaluated at the perturbed samples to perform a gradient descent step on the parameter θ .

Original: A natural question is then how adversarial training helps the trained model in achieving adversarial robustness.
Revised: A natural question is then how adversarial training helps the trained model in achieving adversarial robustness.

Original: Some recent theoretical results partially answer this question, such as deriving adversarial risk bound (Athalye et al., 2018), relating it to the distributionally robust optimization (Sinha et al., 2018), and characterizing trade-offs between robustness and accuracy via regularization (Zhang et al., 2019).
Revised: Some recent theoretical results partially answer this question, such as deriving adversarial risk bound (Athalye et al., 2018), relating it to the distributionally robust optimization (Sinha et al., 2018), and characterizing trade-offs between robustness and accuracy via regularization (Zhang et al., 2019).

Original: Yet, all existing results neglect the algorithmic effect during the training process in promoting adversarial robustness.
Revised: Yet, all existing results neglect the algorithmic effect during the training process in promoting adversarial robustness.

Original: Inspired by the signiﬁcant role of algorithmic bias in the generalization of neural networks, it is natural to ask
Revised: Inspired by the signiﬁcant role of algorithmic bias in the generalization of neural networks, it is natural to ask

Original: Does gradient descent based adversarial training enjoy any implicit bias property?
Revised: Does gradient descent based adversarial training enjoy any implicit bias property?

Original: If so, does the implicit bias provide insights on how adversarial training promotes robustness?
Revised: If so, does the implicit bias provide insights on how adversarial training promotes robustness?

Original: Motivated by these questions, in this paper, we study the algorithmic effect of adversarial training by investigating the implicit bias of GDAT.
Revised: Motivated by these questions, in this paper, we study the algorithmic effect of adversarial training by investigating the implicit bias of GDAT.

Original: Due to current technical limits in directly analyzing deep neural networks, we analyze a simpler model, with the key characteristics that the model overﬁts the training data while being able to generalize well.
Revised: Due to current technical limits in directly analyzing deep neural networks, we analyze a simpler model, with the key characteristics that the model overﬁts the training data while being able to generalize well.

Original: Speciﬁcally, we take the binary classiﬁcation with linearly separable data as an example.
Revised: Speciﬁcally, we take the binary classiﬁcation with linearly separable data as an example.

Original: This helps us focus on the effect of implicit bias without dealing with complicated structures of neural networks.
Revised: This helps us focus on the effect of implicit bias without dealing with complicated structures of neural networks.

Original: Main Contributions.
Revised: Main Contributions.

Original: We summarize our main theoretical ﬁndings below
Revised: We summarize our main theoretical ﬁndings below.

Original: • Our ﬁrst part of result shows an interesting interplay between adversarial perturbation and implicit bias of the gradient descent (GD).
Revised: • Our ﬁrst part of result shows an interesting interplay between adversarial perturbation and implicit bias of the gradient descent (GD).

Original: By exploiting this interplay, we show a property of adversarial training that is not known in the literature before: adversarial training accelerates convergence.
Revised: By exploiting this interplay, we show a property of adversarial training that is not known in the literature before: adversarial training accelerates convergence.

Original: Speciﬁcally, when the perturbation is bounded by (cid:96) 2 -norm, i.e., ∆ = { δ ∈ R d : ||
Revised: Speciﬁcally, when the perturbation is bounded by (cid:96) 2 -norm, i.e., ∆ = { δ ∈ R d : ||

Original: δ || 2 ≤ c } , with proper choice of c , the gradient descent based adversarial training is directionally convergent that lim t →∞ θ t ||
Revised: δ || 2 ≤ c } , with proper choice of c , the gradient descent based adversarial training is directionally convergent that lim t →∞ θ t ||

Original: θ t
Revised: θ t

Original: || 

Revised: ||

Original: =
Revised: =

Original: u 2 , where u 2 is the maximum (cid:96) 2 -norm margin hyperplane (i.e., standard SVM) of the training data.
Revised: u 2 , where u 2 is the maximum (cid:96) 2 -norm margin hyperplane (i.e., standard SVM) of the training data.

Original: In addition, when the perturbation level c is set according to T appropriately, the rate of convergence is (cid:101) O
Revised: In addition, when the perturbation level c is set according to T appropriately, the rate of convergence is (cid:101) O

Original: (1 / √ T ) 2 , which is exponentially faster than the rate O (1 / log T ) when we use standard clean training, i e., training with clean data using gradient descent.
Revised: (1 / √ T ) 2 , which is exponentially faster than the rate O (1 / log T ) when we use standard clean training, i.e., training with clean data using gradient descent.

Original: Based on this, we establish that the convergence of training loss on clean data using GDAT is almost exponentially faster than standard clean training using GD.
Revised: Based on this, we establish that the convergence of training loss on clean data using GDAT is almost exponentially faster than standard clean training using GD.

Revised: (cid:101) O hides logarithmic factor.

Original: • Our second part of result shows that adversarial training adapts the implicit bias of gradient descent for different adversarial perturbation geometry.
Revised: • Our second part of result shows that adversarial training adapts the implicit bias of gradient descent for different adversarial perturbation geometry.

Original: Speciﬁcally, when the perturbation is bounded by (cid:96) q -norm for q ≥ 1 , i e., ∆ = { δ ∈ R d : ||
Revised: Speciﬁcally, when the perturbation is bounded by (cid:96) q -norm for q ≥ 1 , i.e., ∆ = { δ ∈ R d : ||

Original: δ || q ≤ c } , with proper choice of c , the gradient descent based adversarial training is directionally convergent that lim t →∞ 
Revised: δ || q ≤ c } , with proper choice of c , the gradient descent based adversarial training is directionally convergent that lim t →∞ θ t ||

Original: [Equation]{+θ t || 
+}
Revised: {+θ t || 
+}

Original: 2 = u q , where u 2 ,q is the maximum mixed-norm margin hyperplane of the training data.
Revised: = u 2 ,q , where u 2 ,q is the maximum mixed-norm margin hyperplane of the training data.

Original: We further reveal natural interpretation of robustness that we obtain the maximum (cid:96) 2 -norm margin classiﬁer under worst-case (cid:96) q -norm perturbation.
Revised: We further reveal natural interpretation of robustness that we obtain the maximum (cid:96) 2 -norm margin classiﬁer under worst-case (cid:96) q -norm perturbation.

Original: Notations.
Revised: Notations.

Original: For two vectors x, y ∈ R d , (cid:104) x, y (cid:105) =
Revised: For two vectors x, y ∈ R d , (cid:104) x, y (cid:105) =

Original: (cid:80) dj
Revised: (cid:80) dj

Original: =1 x j y j denotes their Euclidean inner product.
Revised: =1 x j y j denotes their Euclidean inner product.

Original: For a vector θ ∈ R d , || θ || p deﬁned by ||
Revised: For a vector θ ∈ R d , || θ || p deﬁned by ||

Original: θ ||
Revised: θ ||

Original: pp =
Revised: pp =

Original: (cid:80) dj
Revised: (cid:80) dj

Original: =1 | θ j | p denotes its p -norm for p ∈ [1 , ∞ ) , and || θ || ∞ =
Revised: =1 | θ j | p denotes its p -norm for p ∈ [1 , ∞ ) , and || θ || ∞ =

Original: max j ∈ [ d ] | θ j | , where [ d ] = { 1 , . . .
Revised: max j ∈ [ d ] | θ j | , where [ d ] = { 1 , . . .

Original: , d } .
Revised: , d } .

Original: For any general norm || · || , we denote its dual norm by || x || ∗ = max
Revised: For any general norm || · || , we denote its dual norm by || x || ∗ = max

Original: || y ||≤ 1 (cid:104) x, y (cid:105) .
Revised: || y ||≤ 1 (cid:104) x, y (cid:105) .

Original: The sign function is sign( v ) = 1 ( v ≥ 0) − 1 ( v< 0) .
Revised: The sign function is sign( v ) = 1 ( v ≥ 0) − 1 ( v< 0) .

Original: For a linear subspace
Revised: For a linear subspace

Original: L ∈ R d , we denote its orthogonal subspace by L ⊥ .
Revised: L ∈ R d , we denote its orthogonal subspace by L ⊥ .

Original:  B ACKGROUND
Revised:  B ACKGROUND

Original: We consider a binary classiﬁcation problem using a dataset S
Revised: We consider a binary classiﬁcation problem using a dataset S

Original: = { ( x i , y i ) } ni =1 ⊂ R d ×{− 1 , +1 } .
Revised: = { ( x i , y i ) } ni =1 ⊂ R d ×{− 1 , +1 } .

Original: We aim to learn a linear decision boundary f ( x ) =
Revised: We aim to learn a linear decision boundary f ( x ) =

Original: (cid:104) θ, x (cid:105) and its associated classiﬁer (cid:98)y
Revised: (cid:104) θ, x (cid:105) and its associated classiﬁer (cid:98)

Original: ( x ) = sign ( f ( x )) , by solving the empirical risk minimization problem:
Revised: [Equation] sign [Equation] ) , by solving the empirical risk minimization problem:

Original: [Equation]
Revised: [Equation]

Original: where (cid:96) ( · ) is some loss function.
Revised: where [Equation] · ) is some loss function.

Original: [Equation]
Revised: [Equation]

Original: In what follows, we suppress the explicit presentation of S when the context is clear, and we focus on the exponential loss (cid:96) ( r ) =
Revised: In what follows, we suppress the explicit presentation of S when the context is clear, and we focus on the exponential loss (cid:96) ( r ) =

Original: exp( − r ) .
Revised: exp( − r ) .

Original: We point out that our analysis can be further extended to other smooth loss functions with tight exponential tail such as logistic loss.
Revised: We point out that our analysis can be further extended to other smooth loss functions with tight exponential tail such as logistic loss.

Original: We assume the dataset S is linearly separable, i.e., there exists u such that min i ∈ [ n ] y i x (cid:62) i u > 0 .
Revised: We assume the dataset S is linearly separable, i.e., there exists u such that min i ∈ [ n ] y i x (cid:62) i u > 0 .

Original: Under this assumption, one notable feature of problem (3) is that there is no ﬁnite minimizer, and
Revised: Under this assumption, one notable feature of problem (3) is that there is no ﬁnite minimizer, and

Original: L ( θ ) → 0 only if || θ || 2 → ∞ along certain directions.
Revised: L ( θ ) → 0 only if || θ || 2 → ∞ along certain directions.

Original: In fact, there is a polyhedral cone C , such that for any u ∈ C , we have lim a →∞ L ( au ) = 0 .
Revised: In fact, there is a polyhedral cone C , such that for any u ∈ C , we have lim a →∞ L ( au ) = 0 .

Original: Several recent results have studied the implicit bias of gradient descent algorithm on separable dataset.
Revised: Several recent results have studied the implicit bias of gradient descent algorithm on separable dataset.

Original: Soudry et al.
Revised: Soudry et al.

Original: (2018) study the implicit bias of the gradient descent algorithm (GD) on (3), and show that lim t →∞ || θ t || 2 = ∞ , while θ t converges in direction to the maximum (cid:96) 2 -norm margin classiﬁer (i.e., the standard SVM).
Revised: (2018) study the implicit bias of the gradient descent algorithm (GD) on (3), and show that lim t →∞ || θ t || 2 = ∞ , while θ t converges in direction to the maximum (cid:96) 2 -norm margin classiﬁer (i.e., the standard SVM).

Original: Ji and Telgarsky (2018) further study the convergence of risk and parameter without separability condition.
Revised: Ji and Telgarsky (2018) further study the convergence of risk and parameter without separability condition.

Original: (Ji and Telgarsky, 2019) and (Gunasekar et al., 2018b) study the implicit bias for training deep linear network and linear convolutional networks, respectively.
Revised: (Ji and Telgarsky, 2019) and (Gunasekar et al., 2018b) study the implicit bias for training deep linear network and linear convolutional networks, respectively.

Original: Gunasekar et al.
Revised: Gunasekar et al.

Original: (2018a) also analyze the implicit bias of steepest descent in general norm || · || , and show that θ t converges in direction to the maximum [Equation]
Revised: (2018a) also analyze the implicit bias of steepest descent in general norm || · || , and show that θ t converges in direction to the maximum || · || ∗ -norm margin hyperplane.

Original: Throughout this paper, we assume the perturbation set is an (cid:96) q -norm ball with radius c , i.e., ∆ = { δ ∈ R d : ||
Revised: Throughout this paper, we assume the perturbation set is an (cid:96) q -norm ball with radius c , i.e., ∆ = { δ ∈ R d : ||

Original: δ || q (cid:54) c } .
Revised: δ || q (cid:54) c } .

Original: Under the general framework of adversarial training in (2), we aim to minimize the empirical adversarial risk
Revised: Under the general framework of adversarial training in (2), we aim to minimize the empirical adversarial risk

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: Note that, given any θ , the inner maximization problem in (4) admits a closed form solution.
Revised: Note that, given any θ , the inner maximization problem in (4) admits a closed form solution.

Original: Then the gradient descent based adversarial training (GDAT)
Revised: Then the gradient descent based adversarial training (GDAT)

Original: algorithm runs iteratively that at the t -th iteration, we ﬁrst solve the inner maximization problem by deriving the worst adversarial perturbation of each sample.
Revised: algorithm runs iteratively that at the t -th iteration, we ﬁrst solve the inner maximization problem by deriving the worst adversarial perturbation of each sample.

Original: It is not difﬁcult to see that for each sample, the worst perturbation is δ ti =
Revised: It is not difﬁcult to see that for each sample, the worst perturbation is δ ti =

Original: cy i δ ∗ t , where δ ∗ t = argmin
Revised: cy i δ ∗ t , where δ ∗ t = argmin

Original: δ :
Revised: δ :

Original: ||
Revised: ||

Original: δ || q ≤ 1 (cid:104) δ, θ t (cid:105) .
Revised: δ || q ≤ 1 (cid:104) δ, θ t (cid:105) .

Original: Then, letting each sample’s perturbed counterpart be ( (cid:101) x ti , y i ) =
Revised: Then, letting each sample’s perturbed counterpart be ( (cid:101) x ti , y i ) =

Original: ( x i + δ ti , y i ) , we take gradient of the loss function evaluated at the perturbed samples and perform a gradient descent step, i.e., θ t +1 =
Revised: ( x i + δ ti , y i ) , we take gradient of the loss function evaluated at the perturbed samples and perform a gradient descent step, i.e., θ t +1 =

Original: θ t − η t ∇ θ L ( θ t ; { ( (cid:101) x ti , y i ) } ni =1 )) , where η t > 0 is some prespeciﬁed stepsize.
Revised: θ t − η t ∇ θ L ( θ t ; { ( (cid:101) x ti , y i ) } ni =1 )) , where η t > 0 is some prespeciﬁed stepsize.

Original: We present the outline of GDAT in Algorithm 1.
Revised: We present the outline of GDAT in Algorithm 1.

Original: [Equation]

Original: O hides logarithmic factor.

Original:  T HEORETICAL R ESULTS
Revised:  T HEORETICAL R ESULTS

Revised: In this section, we show that the GDAT algorithm possesses implicit bias, which depends on the perturbation set during training.

Revised: We provide explicit characterization of the implicit bias, and further conclude that such implicit bias indeed promotes robustness against adversarial perturbation.

Original: (cid:96)
Revised: (cid:96)

Original: Input: Number of iterations T , perturbation level c , stepsizes { η t } Tt =0 , samples { x i , y i } ni =1 .
Revised: Input: Number of iterations T , perturbation level c , stepsizes { η t } Tt =0 , samples { x i , y i } ni =1 .

Original: Initialize: θ← 0 .
Revised: Initialize: θ← 0 .

Original: for t = 0 , . . .
Revised: for t = 0 , . . .

Original: , T − 1 do for i = 1 , . . .
Revised: , T − 1 do for i = 1 , . . .

Original: , n do
Revised: , n do

Original: Compute δ ti =
Revised: Compute δ ti =

Original: cy i argmin ||
Revised: cy i argmin ||

Original: δ || q ≤ 1 (cid:104) δ, θ t (cid:105) Let ( (cid:101) x ti , y i )
Revised: δ || q ≤ 1 (cid:104) δ, θ t (cid:105) Let ( (cid:101) x ti , y i )

Original: ← ( x i + δ ti , y i ) .
Revised: ← ( x i + δ ti , y i ) .

Original: end for θ t +1 ← θ t − η t n (cid:80) ni =1 exp (cid:0) − y i (cid:101) x
Revised: end for θ t +1 ← θ t − η t n (cid:80) ni =1 exp (cid:0) − y i (cid:101) x

Original: (cid:62) i θ t (cid:1) ( − y i (cid:101) x i ) .
Revised: (cid:62) i θ t (cid:1) ( − y i (cid:101) x i ) .

Original: end for
Revised: end for

Original: In this section, we show that the GDAT algorithm possesses implicit bias, which depends on the perturbation set during training.

Original: We provide explicit characterization of the implicit bias, and further conclude that such implicit bias indeed promotes robustness against adversarial perturbation.

Original: Let us start with some deﬁnitions.
Revised: Let us start with some deﬁnitions.

Original: Consider a dataset S
Revised: Consider a dataset S

Original: = { ( x i , y i ) } ni =1 ⊂ R d × {− 1 , +1 } .
Revised: = { ( x i , y i ) } ni =1 ⊂ R d × {− 1 , +1 } .

Original: Given p, q > 0 such that 1 /p
Revised: Given p, q > 0 such that 1 /p

Original: + 1 /q
Revised: + 1 /q

Original: = 1 , the (cid:96) q -norm margin of H θ on
Revised: = 1 , the (cid:96) q -norm margin of H θ on

Original: S is deﬁned as γ q ( θ ) =
Revised: S is deﬁned as γ q ( θ ) =

Original: min i ∈ [ n ] y i x (cid:62) i θ/ ||
Revised: min i ∈ [ n ] y i x (cid:62) i θ/ ||

Original: θ || p .
Revised: θ || p .

Original: Note that for x i ∈ R d , | θ (cid:62) x | / ||
Revised: Note that for x i ∈ R d , | θ (cid:62) x | / ||

Original: θ
Revised: θ

Original: || p mea- sures the (cid:96) q distance between x i and the hyperplane H θ = { x ∈ R d : θ (cid:62) x = 0 } .
Revised: || p mea- sures the (cid:96) q distance between x i and the hyperplane H θ = { x ∈ R d : θ (cid:62) x = 0 } .

Original: Since y i ∈ {− 1 , +1 } , when H θ correctly classiﬁes all samples, γ q ( θ ) measures the minimal (cid:96) q distance between the samples in S and H θ .
Revised: Since y i ∈ {− 1 , +1 } , when H θ correctly classiﬁes all samples, γ q ( θ ) measures the minimal (cid:96) q distance between the samples in S and H θ .

Original: Given that γ q ( θ ) is scale-invariant with respect to θ , without loss of generality, we restrict ||
Revised: Given that γ q ( θ ) is scaleinvariant with respect to θ , without loss of generality, we restrict ||

Original: θ || p
Revised: θ || p

Original: = 1 .
Revised: = 1 .

Original: We also identify the hyperplane H θ by its normal vector θ .
Revised: We also identify the hyperplane H θ by its normal vector θ .

Original: Deﬁnition 3.1.
Revised: Deﬁnition 3.1.

Original: For p, q > 0 with 1 /p + 1 /q
Revised: For p, q > 0 with 1 /p + 1 /q

Original: =
Revised: =

Original: 1 , the maximum (cid:96) q -norm margin hyperplane
Revised: 1 , the maximum (cid:96) q -norm margin hyperplane

Original: u q of
Revised: u q of

Original: S = { ( x i , y i ) } ni =1 ⊂ R d × {− 1 , +1 } and its associated (cid:96) q -norm margin γ q are deﬁned as
Revised: S = { ( x i , y i ) } n i =1 ⊂ R d × {− 1 , +1 } and its associated (cid:96) q -norm margin γ q are deﬁned as

Original: [Equation]
Revised: [Equation]

Original: We denote SV( S ) as the support vectors of S , i.e., SV( S ) =
Revised: We denote SV( S ) as the support vectors of S , i.e., SV( S ) =

Original: argmin
Revised: argmin

Original: ( x
Revised: ( x

Original: [Equation]
Revised: [Equation]

Original: y ) ∈S [Equation] u q [Equation]
Revised: (cid:104) u q , yx (cid:105) .

Original: yx [Equation] By the separability assumption, u q is an optimal hyperplane that correctly classiﬁes all samples with the maximal margin γ q > 0 .
Revised: By the separability assumption, u q is an optimal hyperplane that correctly classiﬁes all samples with the maximal margin γ q > 0 .

Original: Next, by the notion of margin deﬁned above, we characterize the landscape of empirical adversarial risk in (4) based on the perturbation level c .
Revised: Next, by the notion of margin deﬁned above, we characterize the landscape of empirical adversarial risk in (4) based on the perturbation level c .

Original: Proposition 3.1.
Revised: Proposition 3.1.

Original: Let p, q > 0 satisfy 1 /p
Revised: Let p, q > 0 satisfy 1 /p

Original: + 1 /q
Revised: + 1 /q

Original: = 1 .
Revised: = 1 .

Original: Given a nonnegative scalar c , where 0 ≤ c < γ q = max || θ || p ≤ 1 min i ∈ [ n ] y i x (cid:62) i θ , problem (4) has inﬁmum 0 but does not admit a ﬁnite minimizer.
Revised: Given a nonnegative scalar c , where [Equation] [Equation] ] y i x (cid:62) i θ , problem (4) has inﬁmum 0 but does not admit a ﬁnite minimizer.

Original: When c > γ q , problem (4) has a unique ﬁnite minimizer (cid:98) θ ( c ) , and is equivalent to the standard clean training with explicit (cid:96) p -norm regularization.
Revised: When c > γ q , problem (4) has a unique ﬁnite minimizer (cid:98) θ ( c ) , and is equivalent to the standard clean training with explicit (cid:96) p -norm regularization.

Original: That is, there exists λ ( c ) > 0 such that
Revised: That is, there exists λ ( c ) > 0 such that

Original: [Equation]
Revised: [Equation]

Original: It is not difﬁcult to see that for c < γ q , any perturbed dataset (
Revised: It is not difﬁcult to see that for c < γ q , any perturbed dataset (

Original: cid:101) S
Revised: cid:101) S

Original: = { ( (cid:101) x i , y i ) } ni =1 , with || x i − (cid:101) x
Revised: = { ( (cid:101) x i , y i ) } ni =1 , with || x i − (cid:101) x

Original: i || q (cid:54) c for all i , is still linearly separable, which directly follows from the deﬁnition of γ q above.
Revised: i || q (cid:54) c for all i , is still linearly separable, which directly follows from the deﬁnition of γ q above.

Original: On the other hand, when c > γ q , by the deﬁnition of γ q , there exists some perturbed dataset (cid:101) S
Revised: On the other hand, when c > γ q , by the deﬁnition of γ q , there exists some perturbed dataset (cid:101) S

Original: = { ( (cid:101) x i , y i ) } ni =1 , with || x i − (cid:101) x
Revised: = { ( (cid:101) x i , y i ) } ni =1 , with || x i − (cid:101) x

Original: i || q (cid:54) c for all i , such that (cid:101) S is no longer linearly separable.
Revised: i || q (cid:54) c for all i , such that (cid:101) S is no longer linearly separable.

Original: 3. A DVERSARIAL P ERTURBATION WITH B OUNDED (cid:96) 2 -N ORM
Revised: 3. A DVERSARIAL P ERTURBATION WITH B OUNDED (cid:96) 2 -N ORM

Original: In this subsection, we analyze both the empirical adversarial risk convergence and the parameter convergence of the case when the perturbation set ∆ in (4) is an (cid:96) 2 -norm ball with radius c .
Revised: In this subsection, we analyze both the empirical adversarial risk convergence and the parameter convergence of the case when the perturbation set ∆ in (4) is an (cid:96) 2 -norm ball with radius c .

Original: Adversarial Risk Convergence.
Revised: Adversarial Risk Convergence.

Original: We ﬁrst analyze the convergence of empirical adversarial risk (4) using GDAT.
Revised: We ﬁrst analyze the convergence of empirical adversarial risk (4) using GDAT.

Original: One substantial roadblock of minimizing (4) is its non-smoothness, in the sense that
Revised: One substantial roadblock of minimizing (4) is its non-smoothness, in the sense that

Original: L adv ( θ ) is not differentiable at the origin, and its Hessian ∇ 2 L adv ( θ ) explodes around the origin.
Revised: L adv ( θ ) is not differentiable at the origin, and its Hessian ∇ 2 L adv ( θ ) explodes around the origin.

Original: To address the challenge, our key observation is that, by the next lemma, at each iteration, there exists an acute angle between the update on θ t and the maximum (cid:96) 2 -norm margin hyperplane u 2 .
Revised: To address the challenge, our key observation is that, by the next lemma, at each iteration, there exists an acute angle between the update on θ t and the maximum (cid:96) 2 -norm margin hyperplane u 2 .

Original: This gives a lower bound on ||
Revised: This gives a lower bound on ||

Original: θ t
Revised: θ t

Original: || 2 .
Revised: || 2 .

Original: Lemma 3.1.
Revised: Lemma 3.1.

Original: Take ∆ =
Revised: Take ∆ =

Original: { δ ∈ R d : ||
Revised: { δ ∈ R d : ||

Original: δ || 2 ≤ c } in problem (4) .
Revised: δ || 2 ≤ c } in problem (4) .

Original: Given c < γ 2 , we have that (cid:104)−∇L adv ( θ ) , u 2 (cid:105) ≥
Revised: Given c < γ 2 , we have that (cid:104)−∇L adv ( θ ) , u 2 (cid:105) ≥

Original: L adv ( θ )( γ 2 − c ) > 0 for any θ ∈ R d .
Revised: L adv ( θ )( γ 2 − c ) > 0 for any θ ∈ R d .

Original: We highlight that despite its simple proof, Lemma 3.1 and its generalization to (cid:96) q -perturbation is a crucial step for analyzing both adversarial risk and implicit bias.
Revised: We highlight that despite its simple proof, Lemma 3.1 and its generalization to (cid:96) q -perturbation is a crucial step for analyzing both adversarial risk and implicit bias.

Original: In addition, our techniques here can also be adapted to simplify the proof of Lemma 10 in (Gunasekar et al., 2018a), which, in comparison, is more technically involved.
Revised: In addition, our techniques here can also be adapted to simplify the proof of Lemma 10 in (Gunasekar et al., 2018a), which, in comparison, is more technically involved.

Original: Since we initialize GDAT (Alg. 1) using θ 0 = 0 , any perturbation inside ∆ will have no effect on the adversarial loss.
Revised: Since we initialize GDAT (Alg. 1) using θ 0 = 0 , any perturbation inside ∆ will have no effect on the adversarial loss.

Original: Hence we take clean samples as adversarial examples at the ﬁrst iteration of
Revised: Hence we take clean samples as adversarial examples at the ﬁrst iteration of

Original: GDAT.
Revised: GDAT.

Original: From Lemma 3.1, we have the following simple corollary showing that our whole solution path { θ t } Tt =1 is bounded away from the origin.
Revised: From Lemma 3.1, we have the following simple corollary showing that our whole solution path { θ t } Tt =1 is bounded away from the origin.

Original: Corollary 3.1.
Revised: Corollary 3.1.

Original: Let θ
Revised: Let θ

Original: = 0 in Algorithm 1 with q = 2 , we have:
Revised: = 0 in Algorithm 1 with q = 2 , we have:

Original: || θ t || 2 ≥ η 0 γ 2 for all t ≥ 1 .
Revised: || θ t || 2 ≥ η 0 γ 2 for all t ≥ 1 .

Original: By Corollary 3.1, we bypass the non-differentiability issue at the origin and also control the Hessian ∇ 2 L adv ( θ ) throughout the entire training process.
Revised: By Corollary 3.1, we bypass the non-differentiability issue at the origin and also control the Hessian ∇ 2 L adv ( θ ) throughout the entire training process.

Original: Similar to (Ji and Telgarsky, 2018), in the next theorem, we show that the loss L adv ( θ ) , although not uniformly smooth, is locally L adv ( θ ) -smooth.
Revised: Similar to (Ji and Telgarsky, 2018), in the next theorem, we show that the loss L adv ( θ ) , although not uniformly smooth, is locally L adv ( θ ) -smooth.

Original: Consequently, by the smoothness based analysis of the gradient descent algorithm, we establish the convergence of the empirical adversarial risk.
Revised: Consequently, by the smoothness based analysis of the gradient descent algorithm, we establish the convergence of the empirical adversarial risk.

Original: Theorem 3.1.
Revised: Theorem 3.1.

Original: Suppose || x
Revised: Suppose || x

Original: i || 2 ≤ 1 for all i = 1 . . .
Revised: i || 2 ≤ 1 for all i = 1 . . .

Original: n .
Revised: n .

Original: For GDAT (Alg. 1) with (cid:96) 2 -norm perturbation, i.e., ∆ = { δ ∈ R d : ||
Revised: For GDAT (Alg. 1) with (cid:96) 2 -norm perturbation, i.e., ∆ = { δ ∈ R d : ||

Original: δ || 

Revised: δ ||

Original: (cid:54) c } , we set c < γ 2 , η 0 = 1 and η t = η ≤ min { γ 2 /e (1+ c [Equation] 3 γ 2 +2 c ( [Equation]
Revised: (cid:54) c } , we set c < γ 2 , η 0 = 1 and η t = η ≤ min { γ 2 /e (1+ c ) 3 γ 2 +2 c (1+ c ) , 1 } for t ≥ 1 , then we have

Original: 1 } for t ≥ 1 , then we have

Original: [Equation]
Revised: [Equation]

Original: In comparison with the standard clean training using GD (Ji and Telgarsky, 2018), this theorem states that we pay an extra ( γ 2 − c ) − 2 factor in the risk convergence of adversarial training.
Revised: In comparison with the standard clean training using GD (Ji and Telgarsky, 2018), this theorem states that we pay an extra ( γ 2 − c ) − 2 factor in the risk convergence of adversarial training.

Original: However, this direct comparison is too pessimistic since we compare the adversarial risk with the standard risk (corresponding to ∆ = { 0 } ).
Revised: However, this direct comparison is too pessimistic since we compare the adversarial risk with the standard risk (corresponding to ∆ = { 0 } ).

Original: Interestingly, as seen later in Corollary 3.2, we prove that the convergence of standard risk in GDAT is signiﬁcantly faster than its counterpart in the standard clean training using GD.
Revised: Interestingly, as seen later in Corollary 3.2, we prove that the convergence of standard risk in GDAT is signiﬁcantly faster than its counterpart in the standard clean training using GD.

Original: Parameter Convergence.
Revised: Parameter Convergence.

Original: We then show that if we set the perturbation level c < γ 2 in the GDAT algorithm, GDAT with (cid:96) 2 -norm perturbation possesses the same implicit bias as the standard clean training using GD, i.e., we have lim t →∞ [Equation]
Revised: We then show that if we set the perturbation level c < γ 2 in the GDAT algorithm, GDAT with (cid:96) 2 -norm perturbation possesses the same implicit bias as the standard clean training using GD, i.e., we have lim t →∞ θ t ||

Original: u
Revised: θ t

Original: [Equation]{+|| 
 = u 2 .+}
Revised: {+|| 
 = u 2 .+}

Original: Intuitively, GDAT with (cid:96) 2 -norm perturbation searches for a decision hyperplane that is robust to (cid:96) 2 -norm perturbation.
Revised: Intuitively, GDAT with (cid:96) 2 -norm perturbation searches for a decision hyperplane that is robust to (cid:96) 2 -norm perturbation.

Original: Since the learned decision hyperplane in the standard clean using GD converges to u 2 , which is already the most robust decision hyperplane against (cid:96) 2 -norm perturbation to the data, GDAT retains the implicit bias of standard clean training using GD.
Revised: Since the learned decision hyperplane in the standard clean using GD converges to u 2 , which is already the most robust decision hyperplane against (cid:96) 2 -norm perturbation to the data, GDAT retains the implicit bias of standard clean training using GD.

Original: Surprisingly, even though both GDAT in the adversarial training and GD in the standard clean training converge in directions to u 2 , their rates of directional convergence are signiﬁcantly different as shown later.
Revised: Surprisingly, even though both GDAT in the adversarial training and GD in the standard clean training converge in directions to u 2 , their rates of directional convergence are signiﬁcantly different as shown later.

Original: Speciﬁcally, letting the perturbation level c depend on the total number of iterations T in the
Revised: Speciﬁcally, letting the perturbation level c depend on the total number of iterations T in the

Original: GDAT algorithm, the directional error after T iterations in GDAT algorithm can be signiﬁcantly smaller than the error of GD in the standard clean training.
Revised: GDAT algorithm, the directional error after T iterations in GDAT algorithm can be signiﬁcantly smaller than the error of GD in the standard clean training.

Original: We ﬁrst show that the projection of θ t onto the orthogonal subspace of span( u 2 ) is bounded.
Revised: We ﬁrst show that the projection of θ t onto the orthogonal subspace of span( u 2 ) is bounded.

Original: Lemma 3.2.
Revised: Lemma 3.2.

Original: Deﬁne α ( S ) =
Revised: Deﬁne α ( S ) =

Original: min
Revised: min

Original: || ξ || 2 =1 ,ξ ∈ span( u 2 ) ⊥ max ( x,y ) ∈ SV( S ) (cid:104) ξ, yx (cid:105) , where we assume
Revised: || ξ || 2 =1 ,ξ ∈ span( u 2 ) ⊥ max ( x,y ) ∈ SV( S ) (cid:104) ξ, yx (cid:105) , where we assume

Original: SV( S ) spans R d .
Revised: SV( S ) spans R d .

Original: Let θ ⊥ be the projection of vector θ onto span( u 2 ) ⊥ .
Revised: Let θ ⊥ be the projection of vector θ onto span( u 2 ) ⊥ .

Original: Then there exists a constant K that only depends on α ( S ) and log n , such that || θ t ⊥ || 2 ≤ K for any t ≥ 0 in the GDAT algorithm.
Revised: Then there exists a constant K that only depends on α ( S ) and log n , such that || θ t ⊥ || 2 ≤ K for any t ≥ 0 in the GDAT algorithm.

Original: Note that the same α ( S ) is deﬁned in (Ji and Telgarsky, 2019) and proved to be positive with probability 1 if the data is sampled from absolutely continuous distribution.
Revised: Note that the same α ( S ) is deﬁned in (Ji and Telgarsky, 2019) and proved to be positive with probability 1 if the data is sampled from absolutely continuous distribution.

Original: We then show in the next lemma that ||
Revised: We then show in the next lemma that ||

Original: θ t || 2 goes to inﬁnity, where we provide a reﬁned analysis to establish the acceleration of the directional convergence in comparison with the standard clean training.
Revised: θ t || 2 goes to inﬁnity, where we provide a reﬁned analysis to establish the acceleration of the directional convergence in comparison with the standard clean training.

Original: Lemma 3.3.
Revised: Lemma 3.3.

Original: Under the same conditions in Theorem 3.1, and let α = α ( S ) deﬁned in Lemma 3.2.
Revised: Under the same conditions in Theorem 3.1, and let α = α ( S ) deﬁned in Lemma 3.2.

Original: Then for all t ≥ 0 , we have
Revised: Then for all t ≥ 0 , we have

Original: [Equation]
Revised: [Equation]

Original: Lemma 3.3 provides the key insight to establish the acceleration of directional convergence.
Revised: Lemma 3.3 provides the key insight to establish the acceleration of directional convergence.

Original: Speciﬁcally, it allows us to set c depending on the total number of iterations T , so that || θ T
Revised: Speciﬁcally, it allows us to set c depending on the total number of iterations T , so that || θ T

Original: || 2 is sublinear in
Revised: || 2 is sublinear in

Original: T , in comparison with being logarithmic in T in standard clean training as in Ji  Telgarsky (2018).
Revised: T , in comparison with being logarithmic in T in standard clean training as in Ji and Telgarsky (2018).

Original: We are now ready to present the main theorem for parameter convergence.
Revised: We are now ready to present the main theorem for parameter convergence.

Original: Theorem 3.2 (Speed-up of Parameter Convergence) .
Revised: Theorem 3.2 (Speed-up of Parameter Convergence) .

Original: Under same conditions in Theorem 3.1, and let α = α ( S ) and K be deﬁned in Lemma 3.2.
Revised: Under same conditions in Theorem 3.1, and let α = α ( S ) and K be deﬁned in Lemma 3.2.

Original: In GDAT with (cid:96) 2 -norm perturbation, let c and total number of iterations T satisfy γ 2 − c
Revised: In GDAT with (cid:96) 2 -norm perturbation, let c and total number of iterations T satisfy γ 2 − c

Original: = (cid:16) n 1+1 /α log T ηT [Equation] 1 / 2 , and deﬁne θ T = 
Revised: = (cid:16) n 1+1 /α log T ηT (cid:17) 1 / 2 , and deﬁne θ T = θ

Original: [Equation]
Revised: T

Original: || θ T
Revised: || θ T

Original: || 2 .
Revised: || 2 .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: One might argue that the polynomial dependence on sample size n in (7) is too pessimistic, making the GDAT unfavorable in comparison with the standard clean training.
Revised: One might argue that the polynomial dependence on sample size n in (7) is too pessimistic, making the GDAT unfavorable in comparison with the standard clean training.

Original: We show that this is not an issue by a direct comparision of iteration complexity to achieve || θ T − u 2 || 2 ≤ (cid:15) for a given precision (cid:15) > 0 .
Revised: We show that this is not an issue by a direct comparision of iteration complexity to achieve || θ T − u 2 || 2 ≤ (cid:15) for a given precision (cid:15) > 0 .

Original: Speciﬁcally, given (cid:15) > 0 , to achieve || θ T − u 2 || 2 ≤ (cid:15) , GDAT needs (cid:101) O (cid:0) n (1+1 /α )
Revised: Speciﬁcally, given (cid:15) > 0 , to achieve || θ T − u 2 || 2 ≤ (cid:15) , GDAT needs (cid:101) O (cid:0) n (1+1 /α )

Original: (cid:15) − 2 (cid:1) number of iterations.
Revised: (cid:15) − 2 (cid:1) number of iterations.

Original: In comparison, the standard clean training by GD needs (cid:101) O (cid:0) n exp (cid:0) (cid:15) − 1 (cid:1)(cid:1) number of iterations (Ji and Telgarsky, 2018), which has exponential dependence on precision (cid:15) .
Revised: In comparison, the standard clean training by GD needs (cid:101) O (cid:0) n exp (cid:0) (cid:15) − 1 (cid:1)(cid:1) number of iterations (Ji and Telgarsky, 2018), which has exponential dependence on precision (cid:15) .

Original: Finally, by Theorem 3.1 and Lemma 3.3, we show that the empirical clean risk after T iterations of
Revised: Finally, by Theorem 3.1 and Lemma 3.3, we show that the empirical clean risk after T iterations of

Original: GDAT is almost exponentially smaller than its counterpart in the standard clean training.
Revised: GDAT is almost exponentially smaller than its counterpart in the standard clean training.

Original: Corollary 3.2 (Speed-up of Clean Risk Convergence) .
Revised: Corollary 3.2 (Speed-up of Clean Risk Convergence) .

Original: Under the same conditions in Theorem 3.2, we have
Revised: Under the same conditions in Theorem 3.2, we have

Original: [Equation]
Revised: [Equation]

Original: where µ is a constant dependent on η, α, n .
Revised: where µ is a constant dependent on η, α, n .

Original: Note that the empirical clean risk decreases at the rate of O (cid:16) exp( −√ T ) (cid:17) up to a logarithmic factor in the exponent.
Revised: Note that the empirical clean risk decreases at the rate of O (cid:16) exp( −√ T ) (cid:17) up to a logarithmic factor in the exponent.

Original: In comparison, using standard clean training with GD, we only have L ( θ T ) =
Revised: In comparison, using standard clean training with GD, we only have L ( θ T ) =

Original: O (1 /T )
Revised: O (1 /T )

Original: (Soudry et al., 2018).
Revised: (Soudry et al., 2018).

Original: 3. A DVERSARIAL P ERTURBATION WITH B OUNDED (cid:96) q -N ORM
Revised: 3. A DVERSARIAL P ERTURBATION WITH B OUNDED (cid:96) q -N ORM

Original: In this subsection, we generalize our results to the case where the perturbation set is some bounded (cid:96) q -norm ball.
Revised: In this subsection, we generalize our results to the case where the perturbation set is some bounded (cid:96) q -norm ball.

Original: To facilitate our discussion, we ﬁrst deﬁne a robust version of SVM.
Revised: To facilitate our discussion, we ﬁrst deﬁne a robust version of SVM.

Original: Deﬁnition 3.2.
Revised: Deﬁnition 3.2.

Original: For a given separable dataset S with (cid:96) q -norm margin γ q and c < γ q , letting 1 /p + 1 /q
Revised: For a given separable dataset S with (cid:96) q -norm margin γ q and c < γ q , letting 1 /p + 1 /q

Original: = 1 , the robust SVM against (cid:96) q -norm perturbation parameterized by c is
Revised: = 1 , the robust SVM against (cid:96) q -norm perturbation parameterized by c is

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: Remark 3.1 (Maximum Mixed-norm Margin) .
Revised: Remark 3.1 (Maximum Mixed-norm Margin) .

Original: Note that problem (8) is equivalent to solving for a maximum mixed-norm margin hyperplane.
Revised: Note that problem (8) is equivalent to solving for a maximum mixed-norm margin hyperplane.

Original: Speciﬁcally, by the KKT condition of (8) , there exists η ( c ) > 0 , such that (8) is equivalent to the following problem:
Revised: Speciﬁcally, by the KKT condition of (8) , there exists η ( c ) > 0 , such that (8) is equivalent to the following problem:

Original: [Equation]
Revised: [Equation]

Original: Now deﬁne || · || = || · || 2 + η ( c ) || · || p , it is clear that || · || deﬁnes a norm which is a mixture of (cid:96) 2 and (cid:96) p norm.
Revised: Now deﬁne || · || = || · || 2 + η ( c ) || · || p , it is clear that || · || deﬁnes a norm which is a mixture of (cid:96) 2 and (cid:96) p norm.

Original: Let || · || ∗ be its dual norm.
Revised: Let || · || ∗ be its dual norm.

Original: Then we have that the solution to (9) is the maximum || · || ∗ -norm margin hyperplane.
Revised: Then we have that the solution to (9) is the maximum || · || ∗ -norm margin hyperplane.

Original: Note that the constraint in (8) is equivalent to min ||
Revised: Note that the constraint in (8) is equivalent to min ||

Original: δ i ||
Revised: δ i ||

Revised: q

Revised: [Equation]

Revised: y i

Original: q ≤ c y i ( x i + δ i ) (cid:62) θ ≥ 1 , ∀ i = 1 , . . . , n .
Revised: [Equation] x i + [Equation] i ) (cid:62) θ ≥ 1 , ∀ i = 1 , . . . , n .

Original: By a simple scaling argument, in the following lemma, we see the robust nature of (8).
Revised: By a simple scaling argument, in the following lemma, we see the robust nature of (8).

Original: Lemma 3.4.
Revised: Lemma 3.4.

Original: Under the same notations in Deﬁnition 3.2, problem (8) is equivalent to:
Revised: Under the same notations in Deﬁnition 3.2, problem (8) is equivalent to:

Original: [Equation]
Revised: [Equation]

Original: We denote the (unique) solution to problem (10) as u 2 ,q ( c ) .
Revised: We denote the (unique) solution to problem (10) as u 2 ,q ( c ) .

Original: In what follows, we surpress explicit presentation of c when the context is clear.
Revised: In what follows, we surpress explicit presentation of c when the context is clear.

Original: The equivalent formulation (10) provides a clear interpretation on the robustness of (10).
Revised: The equivalent formulation (10) provides a clear interpretation on the robustness of (10).

Original: In particular, the robust SVM against (cid:96) q -norm perturbation parameterized by c is in fact the SVM problem on the the dataset S ( c, q ) , which is generated from S by placing a (cid:96) q -norm ball with radius c around each samples, i.e., S ( c, q ) = { ( x, y ) :
Revised: In particular, the robust SVM against (cid:96) q -norm perturbation parameterized by c is in fact the SVM problem on the the dataset S ( c, q ) , which is generated from S by placing a (cid:96) q -norm ball with radius c around each samples, i.e., S ( c, q ) = { ( x, y ) :

Original: ∃ i ∈ [ n ] , s .
Revised: ∃ i ∈ [ n ] , s .

Original: t ., || x − x i
Revised: t ., || x − x i

Original: || p (cid:54) c, y =
Revised: || p (cid:54) c, y =

Original: y i } .
Revised: y i } .

Original: In other words, u 2 ,q is the maximum (cid:96) 2 -norm margin classiﬁer under worst case (cid:96) q -norm perturbation bounded by c .
Revised: In other words, u 2 ,q is the maximum (cid:96) 2 -norm margin classiﬁer under worst case (cid:96) q -norm perturbation bounded by c .

Original: In the remaining part of this section, we ﬁrst analyze the convergence of the empirical adversarial risk, and then establish the implicit bias of GDAT with (cid:96) q perturbation for q ∈ [1 , ∞ ] .
Revised: In the remaining part of this section, we ﬁrst analyze the convergence of the empirical adversarial risk, and then establish the implicit bias of GDAT with (cid:96) q perturbation for q ∈ [1 , ∞ ] .

Original: Our analysis for q ∈ { 1 , ∞} is based on approximation argument.
Revised: Our analysis for q ∈ { 1 , ∞} is based on approximation argument.

Original: For ease of presentation, we only discuss when q ∈ (1 , ∞ ) in the main text, and defer the discussion for q ∈ { 1 , ∞} in Appendix D.
Revised: For ease of presentation, we only discuss when q ∈ (1 , ∞ ) in the main text, and defer the discussion for q ∈ { 1 , ∞} in Appendix D.

Original: Adversarial Risk Convergence.
Revised: Adversarial Risk Convergence.

Original: Our analysis is similar to the analysis for GDAT with (cid:96) 2 perturbation, where we use similar techniques to address issues such as non-differentiability at the origin and
Revised: Our analysis is similar to the analysis for GDAT with (cid:96) 2 perturbation, where we use similar techniques to address issues such as non-differentiability at the origin and

Original: Hessian explosion of L adv ( θ ) around the origin.
Revised: Hessian explosion of L adv ( θ ) around the origin.

Original: Theorem 3.3.
Revised: Theorem 3.3.

Original: Suppose || x
Revised: Suppose || x

Original: i || 2 ≤ 1 for i = 1 , . . . , n , and let 1 p + 1 q = 1 .
Revised: i || 2 ≤ 1 for i = 1 , . . . , n , and let 1 p + 1 q = 1 .

Original: In the GDAT with (cid:96) q -norm perturbation, setting c < γ q and letting M p = (cid:104)(1 + c √ d ) 2 + c (p − 1)
Revised: In the GDAT with (cid:96) q -norm perturbation, setting c < γ q and letting M p = (cid:104) (

Original: [Equation]
Revised: [Equation]

Revised: +

Original: [Equation]
Revised: [Equation]

Original: q d
Revised: √ d

Original: [Equation]
Revised: [Equation]

Original: − 2 (cid:105) exp (cid:16) − γ 22 ,q + c √ d (cid:17) , set η 0 = 1 and η t = η ≤ min { 1 M p , 1 } for t ≥ 1 .
Revised: + [Equation] [Equation] (cid:17) , set η 0 = 1 and η t = η ≤ min { 1 M p , 1 } for t ≥ 1 .

Original: We have that
Revised: We have that

Original: [Equation]
Revised: [Equation]

Original: We point out here that (6) is a special case of (11).
Revised: We point out here that (6) is a special case of (11).

Original: In particular, by the deﬁnition of γ 2 ,q ( c ) , we have that γ 2 , 2 ( c ) =
Revised: In particular, by the deﬁnition of γ 2 ,q ( c ) , we have that γ 2 , 2 ( c ) =

Original: γ 2 − c , which recovers bound (6) from (11).
Revised: γ 2 − c , which recovers bound (6) from (11).

Original: Parameter Convergence.
Revised: Parameter Convergence.

Original: We show that if we set c < γ q in the GDAT algorithm with stepsizes speciﬁed in Theorem 3.3, with (cid:96) q perturbation, the algorithm still possesses implicit bias property, i.e., θ t still has directional convergence, and the limiting direction depends on the perturbation set ∆ .
Revised: We show that if we set c < γ q in the GDAT algorithm with stepsizes speciﬁed in Theorem 3.3, with (cid:96) q perturbation, the algorithm still possesses implicit bias property, i.e., θ t still has directional convergence, and the limiting direction depends on the perturbation set ∆ .

Original: Theorem 3.4 (Implicit Bias of GDAT with (cid:96) q -norm Perturbation) .
Revised: Theorem 3.4 (Implicit Bias of GDAT with (cid:96) q -norm Perturbation) .

Original: Under the same conditions in
Revised: Under the same conditions in

Original: Theorem 3.3, deﬁne θ t = θ
Revised: Theorem 3.3, deﬁne θ t = θ

Original: t
Revised: t

Original: ||
Revised: ||

Original: θ t || 2 , then we have:
Revised: θ t || 2 , then we have:

Original: [Equation]
Revised: [Equation]

Original: Combining Theorem 3.4 and Lemma 3.4, we conclude that GDAT with (cid:96) q -norm perturbation indeed promotes robustness against (cid:96) q perturbation.
Revised: Combining Theorem 3.4 and Lemma 3.4, we conclude that GDAT with (cid:96) q -norm perturbation indeed promotes robustness against (cid:96) q perturbation.

Original: Using GDAT with (cid:96) q -norm perturbation will result in a classiﬁer which is the maximum (cid:96) 2 -norm margin classiﬁer under worst case (cid:96) q -norm perturbations to the samples bounded by c .
Revised: Using GDAT with (cid:96) q -norm perturbation will result in a classiﬁer which is the maximum (cid:96) 2 -norm margin classiﬁer under worst case (cid:96) q -norm perturbations to the samples bounded by c .

Original: The learned classiﬁer will have (cid:96) q -norm margin at least c .
Revised: The learned classiﬁer will have (cid:96) q -norm margin at least c .

Original: As we increase perturbation level c to γ q , the learned classiﬁer will converge to maximum (cid:96) q -norm margin classiﬁer.
Revised: As we increase perturbation level c to γ q , the learned classiﬁer will converge to maximum (cid:96) q -norm margin classiﬁer.

Original:  N UMERICAL E XPERIMENT
Revised:  N UMERICAL E XPERIMENT

Original: In this section, we ﬁrst conduct numerical experiments on linear classiﬁers to backup our theoretical ﬁndings.
Revised: In this section, we ﬁrst conduct numerical experiments on linear classiﬁers to backup our theoretical ﬁndings.

Original: We further empirically extend our method to neural networks, where our numerical results demonstrate that our theoretical results can be potentially generalized.
Revised: We further empirically extend our method to neural networks, where our numerical results demonstrate that our theoretical results can be potentially generalized.

Original: Linear Classiﬁers.
Revised: Linear Classiﬁers.

Original: We investigate the empirical performance of the GDAT algorithm on linear classiﬁers, with training set S =
Revised: We investigate the empirical performance of the GDAT algorithm on linear classiﬁers, with training set S =

Original: { (( − 0 .
Revised: { (( − 0 .

Original: 5 , 1) , +1) , (( − 0 .
Revised: 5 , 1) , +1) , (( − 0 .

Original: 5 , − 1) , − 1) , (( − 0 .
Revised: 5 , − 1) , − 1) , (( − 0 .

Original: 75 , − 1) , − 1) , ((2 , 1) , +1) } .
Revised: 75 , − 1) , − 1) , ((2 , 1) , +1) } .

Original: It is straightforward to verify that the maximum (cid:96) 2 -norm margin classiﬁer is u 2 = (0 , 1) .
Revised: It is straightforward to verify that the maximum (cid:96) 2 -norm margin classiﬁer is u 2 = (0 , 1) .

Original: Considering (cid:96) 2 -norm perturbations, we ﬁrst run standard clean training with GD, and GDAT with (cid:96) 2 -norm perturbation ( c = 0 .
Revised: Considering (cid:96) 2 -norm perturbations, we ﬁrst run standard clean training with GD, and GDAT with (cid:96) 2 -norm perturbation ( c = 0 .

Original: 95 γ 2 ), for 2 . 5 × 10 4 number of iterations.
Revised: 95 γ 2 ), for 2 . 5 × 10 4 number of iterations.

Original: In both GD and GDAT we take constant stepsizes, with η = 1 and η = 0 . 1 , respectively.
Revised: In both GD and GDAT we take constant stepsizes, with η = 1 and η = 0 . 1 , respectively.

Original: By Figure 1(a), we see that the convergence rate of adversarial loss using GDAT is similar to the convergence rate of clean loss using GD.
Revised: By Figure 1(a), we see that the convergence rate of adversarial loss using GDAT is similar to the convergence rate of clean loss using GD.

Original: However, when we directly compare the clean losses of GDAT and GD, GDAT clearly demonstrates an exponential speed-up in comparison with GD, which is consistent with Corollary 3.2.
Revised: However, when we directly compare the clean losses of GDAT and GD, GDAT clearly demonstrates an exponential speed-up in comparison with GD, which is consistent with Corollary 3.2.

Original: Additionally, as pointed out by Theorem 3.2, GDAT also enjoys signiﬁcant speed-up in terms of the directional convergence of θ t to u 2 .
Revised: Additionally, as pointed out by Theorem 3.2, GDAT also enjoys signiﬁcant speed-up in terms of the directional convergence of θ t to u 2 .

Original: We also compare the norm growth ||
Revised: We also compare the norm growth ||

Original: θ t || 2 , and observe that the norm generated by GDAT grows much faster than the norm generated by GD, which is also in alignment with our discussions in Section 3.1.
Revised: θ t || 2 , and observe that the norm generated by GDAT grows much faster than the norm generated by GD, which is also in alignment with our discussions in Section 3.1.

Original: We further run GDAT with (cid:96) ∞ -norm perturbation ( c = 0 . 5 ).
Revised: We further run GDAT with (cid:96) ∞ -norm perturbation ( c = 0 . 5 ).

Original: By Lemma 3.4, we have that u 2 , ∞ =
Revised: By Lemma 3.4, we have that u 2 , ∞ =

Original: Note that the Hausdorff distance between (cid:96) q -norm ball and (cid:96) ∞ -norm ball distance goes to zero as q goes to inﬁnity.
Revised: Note that the Hausdorff distance between (cid:96) q -norm ball and (cid:96) ∞ -norm ball distance goes to zero as q goes to inﬁnity.

Original: Thus, we have that (10) for q = 1000 is a close approximation of (10) for 
Revised: Thus, we have that (10) for q = 1000 is a close approximation of (10) for q = ∞ .

Original: L o ss : c l e a n d a t a{+We run two versions of GDAT, where one uses (cid:96) q -norm perturbation with q = 1000 , and the other uses (cid:96) ∞ -norm perturbation. We run both algorithms with stepsize η = 0 . 1 for 5 . 
 × 10number of iterations, and we present the results in Figure 1(b).+}
Revised: {+We run two versions of GDAT, where one uses (cid:96) q -norm perturbation with q = 1000 , and the other uses (cid:96) ∞ -norm perturbation. We run both algorithms with stepsize η = 0 . 1 for 5 . 
 × 10number of iterations, and we present the results in Figure 1(b).+}

Original: L o ss : c l e a n d a t a
Revised: We ﬁnd that the two training methods behave similarly. In addition, the empirical directional convergence rates of θ t just differ slightly. Neural Networks. It is seen above that GDAT with (cid:96) 2 -norm perturbations converges signiﬁcantly faster than GD for linear classiﬁers in adversarial training.

Original:  L o ss : a d v e r s a r i
Revised: A natural question is whether this is still the L o ss : c l e a n d a t a L o ss : c l e a n d a t a L o ss : a d v e r s a r i

Original: a l d a t a
Revised: a l d a t a

Original: L o ss : a d v e r s a r i
Revised: L o ss : a d v e r s a r i

Original: a l d a t a 1  h ✓ t , u 

Revised: a l d a t a 1  h ✓ t , u

Original: i k ✓ t kk ✓ t k
Revised: i k ✓ t kk ✓ t k

Original: Clean Training ` 2 Perturbation ` 1 Perturbation ` q Perturbation
Revised: Clean Training ` 2 Perturbation ` 1 Perturbation ` q Perturbation

Original: [Figure]
Revised: [Figure]

Original: 1 i q = ∞ .

Original: We run two versions of GDAT, where one uses (cid:96) q -norm perturbation with q = 1000 , and the other uses (cid:96) ∞ -norm perturbation.

Original: We run both algorithms with stepsize η = 0 .

Original: [-1 for 5 . 
-]
Revised: [-1 for 5 . 
-]

Original: × 10number of iterations, and we present the results in Figure 1(b).

Original: We ﬁnd that the two training methods behave similarly.

Original: In addition, the empirical directional convergence rates of θ t just differ slightly. Neural Networks. It is seen above that GDAT with (cid:96) 2 -norm perturbations converges signiﬁcantly faster than GD for linear classiﬁers in adversarial training. A natural question is whether this is still the case on adversarial training of more complicated neural networks.
Revised: , 1 i case on adversarial training of more complicated neural networks.

Original: We conduct experiments on neural network with one hidden layer.
Revised: We conduct experiments on neural network with one hidden layer.

Original: We take the two classes from MNIST dataset with label “ 2 " and “ 9 " to form our training set S .
Revised: We take the two classes from MNIST dataset with label “ 2 " and “ 9 " to form our training set S .

Original: We also vary the width of the hidden layer in { 64 × 64 , 128 × 128 , 256 × 256 } .
Revised: We also vary the width of the hidden layer in { 64 × 64 , 128 × 128 , 256 × 256 } .

Original: One major difference from the case of linear classiﬁers is that we cannot solve the inner maximization problem of (2) exactly as it does not admits a closed-form solution.
Revised: One major difference from the case of linear classiﬁers is that we cannot solve the inner maximization problem of (2) exactly as it does not admits a closed-form solution.

Original: Instead, we solve the inner problem approximately using projected gradient descent with 20 iterations and stepsize 0 .
Revised: Instead, we solve the inner problem approximately using projected gradient descent with 20 iterations and stepsize 0 .

Original: We test two versions of GDAT, where one adopts (cid:96) 2 -norm perturbations ( c = 2 . 8 ), and the other uses (cid:96) ∞ - norm perturbations ( c = 0 . 1 ).
Revised: We test two versions of GDAT, where one adopts (cid:96) 2 -norm perturbations ( c = 2 . 8 ), and the other uses (cid:96) ∞ - norm perturbations ( c = 0 . 1 ).

Original: For standard clean training and the outer minimization problem in (2), we use the stochastic gradient descent algorithm with batch size 128 and constant stepsize 10 − 5 .
Revised: For standard clean training and the outer minimization problem in (2), we use the stochastic gradient descent algorithm with batch size 128 and constant stepsize 10 − 5 .

Original: We compare the loss and classiﬁcation accuracy, which are evaluated using the clean training samples, of standard clean training and GDAT.
Revised: We compare the loss and classiﬁcation accuracy, which are evaluated using the clean training samples, of standard clean training and GDAT.

Original: By Figure 2, we see that GDAT indeed accelerates the convergence of both loss and classiﬁcation accuracy on clean training samples.
Revised: By Figure 2, we see that GDAT indeed accelerates the convergence of both loss and classiﬁcation accuracy on clean training samples.

Original: The performance gap is most obvious when the width of the hidden layer is small, and reduces gradually as we increase the width of the hidden layer.
Revised: The performance gap is most obvious when the width of the hidden layer is small, and reduces gradually as we increase the width of the hidden layer.

Original: We argue that such reduction comes from the fact that as network width increases, the margin on the samples outputted by the hidden layer also increases.
Revised: We argue that such reduction comes from the fact that as network width increases, the margin on the samples outputted by the hidden layer also increases.

Original: As suggested by Theorem 3.2, in this case, a larger perturbation level c should be used.
Revised: As suggested by Theorem 3.2, in this case, a larger perturbation level c should be used.

Original: We conduct additional experiments with various perturbation level in Appendix E to empirically verify our argument. 
Revised: We conduct additional experiments with various perturbation level in Appendix E to empirically verify our argument. [Figure] u r a c y : c l e a n d a t a

Revised: [Figure]

Revised: L o ss : c l e a n d a t a

Original:   D ISCUSSIONS
Revised: [Figure]  D ISCUSSIONS

Original: We investigate the implicit bias of GDAT for linear classiﬁer.
Revised: We investigate the implicit bias of GDAT for linear classiﬁer.

Original: There are several plausible natural extensions.
Revised: There are several plausible natural extensions.

Original: For example, we can represent a linear classiﬁer using a deep linear network , which is signiﬁcantly overparameterized.
Revised: For example, we can represent a linear classiﬁer using a deep linear network , which is signiﬁcantly overparameterized.

Original: Some recent results characterize the implicit bias of gradient descent for training deep linear networks (Ji and Telgarsky, 2019) and linear convolutional networks (Gunasekar et al., 2018b).
Revised: Some recent results characterize the implicit bias of gradient descent for training deep linear networks (Ji and Telgarsky, 2019) and linear convolutional networks (Gunasekar et al., 2018b).

Original: Motivated by these results, investigating the implicit bias of GDAT in training deep linear networks worths future investigations.
Revised: Motivated by these results, investigating the implicit bias of GDAT in training deep linear networks worths future investigations.

Original: Meanwhile, investigating implicit bias in deep nonlinear networks is a more important and challenging direction: (1) For linear classiﬁers, adding adversarial perturbations during training can be understood as a form of regularization, which explains the faster convergence in training.
Revised: Meanwhile, investigating implicit bias in deep nonlinear networks is a more important and challenging direction: (1) For linear classiﬁers, adding adversarial perturbations during training can be understood as a form of regularization, which explains the faster convergence in training.

Original: Although observed empirically, the potential acceleration of adversarial training is not yet understood in the current literature, to the best of our knowledge.
Revised: Although observed empirically, the potential acceleration of adversarial training is not yet understood in the current literature, to the best of our knowledge.

Original: The notion of margin for neural networks still lacks proper deﬁnition, which we need to deﬁne to facilitate investigations on the effect of adversarial training in promoting robustness.(3) Ultrawide nonlinear networks have been shown to evolve A cc u r a c y : c l e a n d a t a Iterations
Revised: The notion of margin for neural networks still lacks proper deﬁnition, which we need to deﬁne to facilitate investigations on the effect of adversarial training in promoting robustness.

Original: Iterations

Original: Iterations

Original: Iterations

Original: Iterations

Original: Iterations

Original: L o ss : c l e a n d a t a (

Original: [Figure]

Original: Width

Original: [Figure]

Original: 64 ⇥

Original: [Figure]

Original: Width

Original: [Figure]

Original: Width

Original: [Figure]

Original: [Figure]

Original: Clean Training ` 2 Perturbation ` 1 Perturbation Iterations [Figure] Iterations similarly to linear networks using gradient descent (Ghorbani et al., 2019; Lee et al., 2019).
Revised: (3) Ultrawide nonlinear networks have been shown to evolve similarly to linear networks using gradient descent (Ghorbani et al., 2019; Lee et al., 2019).

Original: We shall further investigate if our results on linear classiﬁers can be extended to wide nonlinear networks.
Revised: We shall further investigate if our results on linear classiﬁers can be extended to wide nonlinear networks.

Original:  A CKNOWLEDGEMENTS
Revised:  A CKNOWLEDGEMENTS

Original: Fang is partially supported by NSF DMS-1820702 and NSF DMS-
Revised: Fang is partially supported by NSF DMS-1820702 and NSF DMS-1953196.

Original: A P ROOF OF P ROPOSITION 3. 
Revised: A P ROOF OF P ROPOSITION 3.

Original: Proof.
Revised: Proof.

Original: Suppose c < γ q .
Revised: Suppose c < γ q .

Original: Letting θ α
Revised: Letting θ α

Original: = αu q for α > 0 , we have
Revised: = αu q for α > 0 , we have

Original: [Equation]
Revised: [Equation]

Original: Letting α → ∞ , we obtain lim α →∞ L adv ( θ α )
Revised: Letting α → ∞ , we obtain lim α →∞ L adv ( θ α )

Original: = 0 , which implies inf θ ∈ R d L adv ( θ ) = 0 .
Revised: = 0 , which implies inf θ ∈ R d L adv ( θ ) = 0 .

Original: Note that
Revised: Note that

Original: L ( θ ) does not admit any ﬁnite minimizer since L adv ( θ ) > 0 for any θ ∈ R d .
Revised: L ( θ ) does not admit any ﬁnite minimizer since L adv ( θ ) > 0 for any θ ∈ R d .

Original: If c > γ q , by the deﬁnition of maximum (cid:96) q -norm margin, for any θ ∈ R d , there exists ( y i , x i ) ∈ S for some i ∈ [ n ] such that y i x (cid:62) i θ (cid:54) γ q || θ || p .
Revised: If c > γ q , by the deﬁnition of maximum (cid:96) q -norm margin, for any θ ∈ R d , there exists ( y i , x i ) ∈ S for some i ∈ [ n ] such that y i x (cid:62) i θ (cid:54) γ q || θ || p .

Original: Hence, L adv ( θ ) (cid:62) exp (cid:0) n − 1 ( c − γ q ) ||
Revised: Hence, L adv ( θ ) (cid:62) exp (cid:0) n − 1 ( c − γ q ) ||

Original: θ || p (cid:1) .
Revised: θ || p (cid:1) .

Original: Then it is easy to see that L adv ( θ ) has bounded sublevel set and hence a ﬁnite minimizer (cid:98) θ .
Revised: Then it is easy to see that L adv ( θ ) has bounded sublevel set and hence a ﬁnite minimizer (cid:98) θ .

Original: Since L adv ( θ ) is convex, we examine its ﬁrst-order KKT condition, given by
Revised: Since L adv ( θ ) is convex, we examine its ﬁrst-order KKT condition, given by

Original: [Equation]
Revised: [Equation]

Original: Consider the regularized problem with regularization parameter η :
Revised: Consider the regularized problem with regularization parameter η :

Original: [Equation]
Revised: [Equation]

Original: Its ﬁrst-order KKT condition is
Revised: Its ﬁrst-order KKT condition is

Original: [Equation]
Revised: [Equation]

Original: Looking at (12) and (13) together, by taking η = cn (cid:80) ni =1 exp (cid:16) − y i x (cid:62) i (cid:98) θ + c || (cid:98) θ || p (cid:17) , we have that the solution to the adversarial training problem (cid:98) θ is also the solution to the regularized problem.
Revised: Looking at (12) and (13) together, by taking η = cn (cid:80) ni =1 exp (cid:16) − y i x (cid:62) i (cid:98) θ + c || (cid:98) θ || p (cid:17) , we have that the solution to the adversarial training problem (cid:98) θ is also the solution to the regularized problem.

Original: To facilitate our later discussions, we point out that by the conjugacy of (cid:96) p -norm and (cid:96) q -norm, (4) has the following equivalent form that
Revised: To facilitate our later discussions, we point out that by the conjugacy of (cid:96) p -norm and (cid:96) q -norm, (4) has the following equivalent form that

Original: [Equation]
Revised: [Equation]

Original: In fact, one can verify that the GDAT algorithm is equivalent to gradient descent algorithm on (14).
Revised: In fact, one can verify that the GDAT algorithm is equivalent to gradient descent algorithm on (14).

Original: B P ROOFS FOR S ECTION 3. 
Revised: B P ROOFS FOR S ECTION 3.

Original: Proof of Lemma 3.1.
Revised: Proof of Lemma 3.1.

Original: Recall we have L adv ( θ) = 1 n (cid:80) ni =1 max ||
Revised: Recall we have L [Equation] θ

Original: δ || 2 ≤ c exp (cid:0) − y i ( x i + δ i ) (cid:62) θ (cid:1) .
Revised: [Equation]

Original: For each sample ( x i , y i ) ∈ S , given a classiﬁer θ , the worse case perturbation is (cid:101) δ i= argmax
Revised: For each sample ( x i , y i ) ∈ S , given a classiﬁer θ , the worse case perturbation is (cid:101) δ i

Original: [Equation]
Revised: [Equation]

Original: = argmin ||

Original: δ || 2 ≤ c y i δ (cid:62) θ .

Original: The corresponding loss is Ladv ( θ ) =
Revised: The corresponding loss is L

Original: [Equation]
Revised: [Equation]

Original: Since for a ﬁxed δ i , the function exp (cid:0) − y i ( x i + δ i ) (cid:62) θ (cid:1) is convex in θ , hence the gradient of L adv ( θ ) is
Revised: Since for a ﬁxed δ i , the function exp (cid:0) − y i ( x i + δ i ) (cid:62) θ (cid:1) is convex in θ , hence the gradient of L adv ( θ ) is

Original: [Equation]
Revised: [Equation]

Original: Then from the deﬁnition of u 2 (5), we have
Revised: Then from the deﬁnition of u 2 (5), we have

Original: [Equation]
Revised: [Equation]

Original: where in the second inequality holds since || (cid:101)
Revised: where in the second inequality holds since || (cid:101)

Original: δ
Revised: δ

Original: i || 2 (cid:54) c and || u 2 || 2 = 1 .
Revised: i || 2 (cid:54) c and || u 2 || 2 = 1 .

Original: Proof of Corollary C.1.
Revised: Proof of Corollary C.1.

Original: Since L adv ( θ ) is not differentiable at θ 0 = 0 , we use subgradient (note that
Revised: Since L adv ( θ ) is not differentiable at θ 0 = 0 , we use subgradient (note that

Original: L adv ( θ ) is convex) at 0 .
Revised: L adv ( θ ) is convex) at 0 .

Original: Speciﬁcally, we take ∇L adv ( θ 0 )
Revised: Speciﬁcally, we take ∇L adv ( θ 0 )

Original: = 1 n (cid:80) ni =

Revised: = 1 n (cid:80) ni =

Original: z i ∈ ∂
Revised: z i ∈ ∂

Original: L adv ( θ 0 ) .
Revised: L adv ( θ 0 ) .

Original: Then we have (cid:10) θ 1 , u 2 (cid:11) =
Revised: Then we have (cid:10) θ 1 , u 2 (cid:11) =

Original: η 0 n (cid:80) i (cid:104)
Revised: η 0 n (cid:80) i (cid:104)

Original: z i , u 2 (cid:105) ≥ η 0 γ 2 , where the last inequality uses the deﬁnition of γ 2 .
Revised: z i , u 2 (cid:105) ≥ η 0 γ 2 , where the last inequality uses the deﬁnition of γ 2 .

Original: By Lemma 3.1, we have (cid:104) θ t , u 2 (cid:105) ≥ η 0 γ 2 for all t ≥ 1 , which also implies (cid:104) v, u 2 (cid:105) ≥ η 0 γ 

Revised: By Lemma 3.1, we have (cid:104) θ t , u 2 (cid:105) ≥ η 0 γ 2 for all t ≥ 1 , which also implies (cid:104) v, u 2 (cid:105) ≥ η 0 γ

Original: and hence ||
Revised: and hence ||

Original: v t
Revised: v t

Original: || 2 ≥ η 

Revised: || 2 ≥ η

Original: γ 2 for v ∈ (cid:2) θ t , θ t +1 (cid:3) .
Revised: γ 2 for v ∈ (cid:2) θ t , θ t +1 (cid:3) .

Original: Proof of Theorem 3.1.
Revised: Proof of Theorem 3.1.

Original: For simplicity, we let z i
Revised: For simplicity, we let z i

Original: =
Revised: =

Original: y
Revised: y

Original: i x i , where we have ||
Revised: i x i , where we have ||

Original: z i || 2 ≤ 1 as we assume (cid:107) x i (cid:107) 2 ≤ 1 .
Revised: z i || 2 ≤ 1 as we assume (cid:107) x i (cid:107) 2 ≤ 1 .

Original: We have
Revised: We have

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Note that the Hessian expression indicates that the objective is highly non-smooth around origin, and the loss is not even differentiable at origin.
Revised: Note that the Hessian expression indicates that the objective is highly non-smooth around origin, and the loss is not even differentiable at origin.

Original: However, we shall prove that starting from origin, every iteration generated by GADT stays away from the origin with distance bounded below.
Revised: However, we shall prove that starting from origin, every iteration generated by GADT stays away from the origin with distance bounded below.

Original: Using Taylor’s expansion, and by deﬁnition θ t +1 =
Revised: Using Taylor’s expansion, and by deﬁnition θ t +1 =

Original: θ t − η t ∇L adv ( θ t ) , we have
Revised: θ t − η t ∇L adv ( θ t ) , we have

Original: [Equation]
Revised: [Equation]

Original: where λ ( H ( v )) max denotes the largest eigenvalue of H ( v ) , where 
Revised: where λ ( H ( v )) max denotes the largest eigenvalue of H ( v ) , where H ( v ) = 1 n I (cid:88) [Equation]

Original: [Equation]
Revised: [Equation]

Original: To upper bound H ( v ) , we need a lower bound on || v || , which is readily given by Corollary C.1.
Revised: To upper bound H ( v ) , we need a lower bound on || v || , which is readily given by Corollary C.1.

Original: That is, 
Revised: That is, ||

Original: [Equation]
Revised: v || 2 ≥ η 0 γ 2 .

Original: We now analyze (18) for t ≥ 1 , where we show that L adv ( θ t ) is locally smooth with parameter proportional to L adv ( θ t ) , and with proper stepsize, the risk is monotonely decreasing.
Revised: We now analyze (18) for t ≥ 1 , where we show that L adv ( θ t ) is locally smooth with parameter proportional to L adv ( θ t ) , and with proper stepsize, the risk is monotonely decreasing.

Original: Note that z
Revised: Note that z

Original: i z ti ≤
Revised: i z ti ≤

Original: I , − 2 c z [Equation] i v ||
Revised: I , − 2 c z (cid:62) i v ||

Original: v || 2 ≤ 2 cI , c 2 vv (cid:62) /
Revised: v || 2 ≤ 2 cI , c 2 vv (cid:62) /

Original: || v || 22 ≤ c 2 I .
Revised: || v || 22 ≤ c 2 I .

Original: Now since || v t || 2 ≥ η 0 γ 2 , we have cI/
Revised: Now since || v t || 2 ≥ η 0 γ 2 , we have cI/

Original: ||
Revised: ||

Original: v || 2 − cvv (cid:62) /
Revised: v || 2 − cvv (cid:62) /

Original: || v || 32 ≤ 2 c η 0 γ 

Revised: || v || 32 ≤ 2 c η 0 γ

Original: I .
Revised: I .

Original: Plugging them in, we have
Revised: Plugging them in, we have

Original: [Equation]
Revised: [Equation]

Original: and (18) reduces to
Revised: and (18) reduces to

Original: [Equation]
Revised: [Equation]

Original: Suppose L adv ( θ t +1 ) >
Revised: Suppose L adv ( θ t +1 ) >

Original: L adv ( θ t ) , and let M = (cid:104) (1 + c ) 2 + 2 c η 0 γ 2 (cid:105) .
Revised: L adv ( θ t ) , and let M = (cid:104) (1 + c ) 2 + 2 c η 0 γ 2 (cid:105) .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: which implies
Revised: which implies

Original: [Equation]
Revised: [Equation]

Original: Meanwhile, if we choose η t satisfying
Revised: Meanwhile, if we choose η t satisfying

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: then we have the right hand side of (20) is upper bounded by L adv ( θ t ) 
Revised: then we have the right hand side of (20) is upper bounded by L adv ( θ t ) , and we have L adv

Original: [Equation]
Revised: [Equation]

Original: and we have
Revised: ≤

Original: [Equation]
Revised: [Equation]

Original: which is clearly a contradiction.
Revised: which is clearly a contradiction.

Original: Hence, if η t satisﬁes (21), by (19) we have
Revised: Hence, if η t satisﬁes (21), by (19) we have

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where the last inequality holds by the choice of η t in (21).
Revised: where the last inequality holds by the choice of η t in (21).

Original: Note that if (21) holds for t = 1 for η 1 = η , by induction it is easy to see that with constant stepsize η t = η for t ≥ 1 , (21) holds for all t ≥ 1 .
Revised: Note that if (21) holds for t = 1 for η 1 = η , by induction it is easy to see that with constant stepsize η t = η for t ≥ 1 , (21) holds for all t ≥ 1 .

Original: Hence for t ≥ 1 , we choose stepsize η such that η L
Revised: Hence for t ≥ 1 , we choose stepsize η such that η L

Original: adv ( θ 1 ) (cid:104) (1 + c ) 2 + 2 c η 0 γ 2 (cid:105) ≤ 1 .
Revised: adv ( θ 1 ) (cid:104) (1 + c ) 2 + 2 c η 0 γ 2 (cid:105) ≤ 1 .

Original: Note that L adv ( θ 1 )
Revised: Note that L adv ( θ 1 )

Original: =
Revised: =

Original: 1 n (cid:80) ni =1 exp (cid:0) − z (cid:62)
Revised: 1 n (cid:80) ni =1 exp (cid:0) − z (cid:62)

Original: i θ 

Revised: i θ

Original: + c ||
Revised: + c ||

Original: θ
Revised: θ

Original: || 2 (cid:1) ≤ exp (cid:0) (1 + c ) η 0 (cid:1) since || θ 1 || ≤ η 0 .
Revised: || 2 (cid:1) ≤ exp (cid:0) (1 + c ) η 0 (cid:1) since || θ 1 || ≤ η 0 .

Original: Then we only require
Revised: Then we only require

Original: [Equation]
Revised: [Equation]

Original: where in the last inequality we take η 0 = 1 and use basic inequality exp( − x ) x ≤ e − 1 for x ≥ 1 .
Revised: where in the last inequality we take η 0 = 1 and use basic inequality exp( − x ) x ≤ e − 1 for x ≥ 1 .

Original: In summary, we choose η 0 = 1 and η t = η = min {
Revised: In summary, we choose η 0 = 1 and η t = η = min {

Original: [Equation]
Revised: [Equation]

Original: +
Revised: 3 γ 2 +2 c

Original: [Equation]
Revised: [Equation]

Original:  , 1 } for t ≥ 1 , then by previous argument, we have (22) holds for all t ≥ 1 .
Revised: 1+ c ) , 1 } for t ≥ 1 , then by previous argument, we have (22) holds for all t ≥ 1 .

Original: Now we are ready to apply the standard smoothness-based analysis of gradient descent using (22), take any θ ∈ R d , we have
Revised: Now we are ready to apply the standard smoothness-based analysis of gradient descent using (22), take any θ ∈ R d , we have

Original: [Equation]
Revised: [Equation]

Original: where the ﬁrst inequality holds by the convexity of L adv ( θ ) , and the second inequality holds by (22).
Revised: where the ﬁrst inequality holds by the convexity of L adv ( θ ) , and the second inequality holds by (22).

Original: Now sum up the above inequality from s = 1 to t − 1 .
Revised: Now sum up the above inequality from s = 1 to t − 1 .

Original: By η t = η ≤ 1 = η 0 and L adv ( θ s +1 ) ≤
Revised: By η t = η ≤ 1 = η 0 and L adv ( θ s +1 ) ≤

Original: L adv ( θ s ) , we have
Revised: L adv ( θ s ) , we have

Original: [Equation]
Revised: [Equation]

Original: Now since θ is arbitrary, letting θ = log( t )
Revised: Now since θ is arbitrary, letting θ = log [Equation]

Original: γ 2 − c · u 2 , we have
Revised: − c · u 2 , we have

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: which yields
Revised: which yields

Original: [Equation]
Revised: [Equation]

Original: Proof of Lemma 3.2.
Revised: Proof of Lemma 3.2.

Original: For simplicity, we let z i =
Revised: For simplicity, we let z i =

Original: y
Revised: y

Original: i x i and (cid:96) i ( θ ) = exp (cid:0)
Revised: i x i and (cid:96) i ( θ ) = exp (cid:0)

Original: − z (cid:62) i θ + c
Revised: − z (cid:62) i θ + c

Original: ||
Revised: ||

Original: θ
Revised: θ

Original: || 2 (cid:1) .
Revised: || 2 (cid:1) .

Original: Deﬁne 
Revised: Deﬁne α =

Original: [Equation]
Revised: [Equation]

Original: where SV( S ) denotes the set of support vectors.
Revised: where SV( S ) denotes the set of support vectors.

Original: It has been shown in Ji and Telgarsky (2019)
Revised: It has been shown in Ji and Telgarsky (2019)

Original: (Lemma 2.10) that α > 0 with probability 1 if the data is sampled from absolutely continuous distribution.
Revised: (Lemma 2.10) that α > 0 with probability 1 if the data is sampled from absolutely continuous distribution.

Original: We have
Revised: We have

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where z (cid:48)
Revised: where z (cid:48)

Original: j ∈ S is arbitrary
Revised: j ∈ S is arbitrary

Original: [Equation]
Revised: [Equation]

Original: by deﬁnition of α : (cid:10) − z(cid:48)
Revised: by deﬁnition of α : (cid:10) − z

Revised: [Equation]

Original: j , θ t ⊥ (cid:11) ≥ α || θ t ⊥ || 2 .
Revised: , θ t ⊥ (cid:11) ≥ α || θ t ⊥ || [Equation]

Original: We bound the ﬁrst term as
Revised: We bound the ﬁrst term as

Original: [Equation]
Revised: [Equation]

Original: where the second inequality uses (cid:10) z (cid:48)
Revised: where the second inequality uses (cid:10) z (cid:48)

Original: j , u 2 (cid:11) ≥
Revised: j , u [Equation]

Original:  γ2 .
Revised: ≥ γ

Original: On the other hand, we can bound the second term in (23) as n

Original: [Equation]
Revised: [Equation]

Original: where in the last inequality holds since (cid:104) θ t , u 2 (cid:105) ≥ 0 , (cid:10) z i
Revised: On the other hand

Original: [Equation]
Revised: [Equation]

Original: θ tu
Revised: we can bound the second term in (23) as n

Original: [Equation]
Revised: [Equation]

Original: = z
Revised: where in the last inequality holds since (cid:104) θ t , u 2 (cid:105) ≥ 0 , (cid:10)

Original: [Equation]
Revised: [Equation]

Original: i
Revised: u

Original: [Equation]
Revised: [Equation]

Original: u
Revised: u

Original: [Equation]
Revised: [Equation]

Original: 2 θ t (cid:1) u 2 ≥ γ
Revised: ≥ [Equation] 2 (cid:104)

Original: [Equation]
Revised: [Equation]

Original: θ t , u 2 (cid:105) and − x exp( − x ) ≥ − 1 e for x ≥ 
Revised: , u 2 (cid:105) and − x exp( − x ) ≥ − 1 e for x ≥ 0 .

Revised: Plugging the two bounds above into (23), we have

Original: [Equation]
Revised: [Equation]

Original: Plugging the two bounds above into (23), we have
Revised: e

Original: [Equation]
Revised: [Equation]

Original: which is non-negative when ||
Revised: which is non-negative when ||

Original: θ
Revised: θ

Original: t ⊥ || 2 ≥ K (cid:48)
Revised: t ⊥ || 2 ≥ K (cid:48)

Original: =
Revised: =

Original: 1+log n α .
Revised: 1+log n α .

Original: Supposing ||
Revised: Supposing ||

Original: θ t ⊥ || 2 ≥ K (cid:48) , by gradient descent update, we have,
Revised: θ t ⊥ || 2 ≥ K (cid:48) , by gradient descent update, we have,

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: 24) where the last inequality uses (22).
Revised: where the last inequality uses (22).

Original: Now let t 0 satisfy || θ t 0 − 1 ⊥ ||
Revised: Now let t 0 satisfy || θ t 0 − 1 ⊥ ||

Original: <
Revised: <

Original: K (cid:48) and || θ t 0 − 1 ⊥ || 2 ≥ K (cid:48) . 
Revised: K (cid:48) and || θ t 0 − 1 ⊥ || 2 ≥ K (cid:48) . Deﬁne t

Original: Deﬁne t 1 = min { s ≥ t 0 :
Revised: [Equation] = min { s [Equation] t 0 :

Original: ||
Revised: ||

Original: θ s ⊥
Revised: θ s ⊥

Original: || 2 < K (cid:48) } , when ||
Revised: || 2 < K (cid:48) } , when ||

Original: θ s ⊥ || 2 ≥ K (cid:48) for all s ≥ t 0 we deﬁne t 1 = ∞ .
Revised: θ s ⊥ || 2 ≥ K (cid:48) for all s ≥ t 0 we deﬁne t 1 = ∞ .

Original: That is for any t ∈ { t 0 , . . .
Revised: That is for any t ∈ { t 0 , . . .

Original: , t 1 − 1 } , we have ||
Revised: , t 1 − 1 } , we have ||

Original: θ t ⊥ || 2 ≥ K (cid:48) .
Revised: θ t ⊥ || 2 ≥ K (cid:48) .

Original: then for any s such that t 0 ≤ s < t 1 , summing (24) up from t 0 to s − 1 yields:
Revised: then for any s such that t 0 ≤ s < t 1 , summing (24) up from t 0 to s − 1 yields:

Original: [Equation]
Revised: [Equation]

Revised: + 18 , where we use

Revised: L

Original: where we use L adv ( θ t ) ≤ L adv ( θ 1 ) ≤ exp(1 + c ) and c < 1 .
Revised: [Equation] ) ≤ L adv [Equation] θ 1 ) ≤ exp(1 + c ) and c < 1 .

Original: This inequality shows that for θ t ∈ { θ t 0 , . .
Revised: This inequality shows that for θ t ∈ { θ t 0 , . .

Original: . , θ t 1 − 1 } ⊂ { θ : || θ ⊥ || 2 ≥ K (cid:48) } , {+|| θ t ⊥ || 2 ≤ || θ t 0 ⊥ || 
+}
Revised: . , θ t 1 − 1 } ⊂ { θ : || θ ⊥ || 2 ≥ K (cid:48) } , {+|| θ t ⊥ || 2 ≤ || θ t 0 ⊥ || 
+}

Original: [Equation]
Revised: + 18 .

Original: Then, we only need to bound || θ t 0 ⊥ || 2 to conclude the proof, where t 0 is the ﬁrst time θ t enters { θ : || θ ⊥
Revised: Then, we only need to bound || θ t 0 ⊥ || 2 to conclude the proof, where t 0 is the ﬁrst time θ t enters { θ : || θ ⊥

Original: ||
Revised: ||

Original: 2 ≥ K (cid:48) } .
Revised: 2 ≥ K (cid:48) } .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: where P ⊥ ( · ) denotes the projection onto span ( u 2 ) ⊥ .
Revised: where P ⊥ ( · ) denotes the projection onto span ( u 2 ) ⊥ .

Original: Note that t 0 is the ﬁrst time θ t (re)-enters the region { θ : || θ ⊥ || 2 ≥ K (cid:48) } , and thus || θ t 0 − 1 ⊥ || [Equation] K
Revised: Note that t 0 is the ﬁrst time θ t (re)-enters the region { θ : || θ ⊥ || 2 ≥ K (cid:48) } , and thus || θ t 0 − 1 ⊥ || 2 < K

Original: [Equation]
Revised: [Equation]

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: where the last inequality we use c < γ 2 ≤ 1 .
Revised: where the last inequality we use c < γ≤ 1 .

Original: In summary, we have shown that for any t such that || θ t ⊥
Revised: In summary, we have shown that for any t such that || θ t ⊥

Original: ||
Revised: ||

Original: 2 ≥ K (cid:48) , we have ||
Revised: 2 ≥ K (cid:48) , we have ||

Original: θ t ⊥ || 2 ≤ K (cid:48)
Revised: θ t ⊥ || 2 ≤ K (cid:48)

Original: + 20 , and we conclude that ||
Revised: + 20 , and we conclude that ||

Original: θ t ⊥ ||
Revised: θ t ⊥ ||

Original: 2 = K (cid:48)
Revised: 2 = K (cid:48)

Original: + 20 = K for all t ≥ 0 .
Revised: + 20 = K for all t ≥ 0 .

Original: Note that K only depends α ( S ) and sample size n .
Revised: Note that K only depends α ( S ) and sample size n .

Original: Proof of Lemma 3.3.
Revised: Proof of Lemma 3.3.

Original: To obtain a lower bound on || θ t || 2 , we ﬁrst denote θ t = θ tu + θ t ⊥ , where θ tu denotes the projection of θ onto span ( u 2 ) , and θ t ⊥ denotes the projection of θ onto span ( u 2 ) ⊥ .
Revised: To obtain a lower bound on || θ t || 2 , we ﬁrst denote θ t = θ tu + θ t ⊥ , where θ tu denotes the projection of θ onto span ( u 2 ) , and θ t ⊥ denotes the projection of θ onto span ( u 2 ) ⊥ .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: Let us assume that || θ t ⊥ || is bounded so that exp( || θ t ⊥ || ) ≤ M , which will be veriﬁed immediately.
Revised: Let us assume that || θ t ⊥ || is bounded so that exp( || θ t ⊥ || ) ≤ M , which will be veriﬁed immediately.

Original: Choosing an arbitrary support
Revised: Choosing an arbitrary support

Original: vector z i , we have 0 <
Revised: vector z i , we have 0 <

Original: [Equation]
Revised: [Equation]

Original: γ
Revised: =

Original: [Equation]
Revised: [Equation]

Revised: (cid:104)

Original: θ [Equation]{+t , u 2 (cid:105) = γ 
+}
Revised: θ {+t , u 2 (cid:105) = γ 
+}

Original: || [Equation] ≤ γ [Equation]
Revised: || θ tu || 2 ≤ γ 2 ||

Original: θ t || 2 , hence the previous inequality becomes:
Revised: θ t || 2 , hence the previous inequality becomes:

Original: [Equation]
Revised: [Equation]

Original: which is equivalent to
Revised: which is equivalent to

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Now we only need to show that ||
Revised: Now we only need to show that ||

Original: θ t ⊥ || ≤ M for all t for some M .
Revised: θ t ⊥ || ≤ M for all t for some M .

Original: Since we have shown in Lemma 3.
Revised: Since we have shown in Lemma 3.

Original: that ||
Revised: that ||

Original: θ t
Revised: θ t

Original: || 2 ≤ K , we choose M =
Revised: || 2 ≤ K , we choose M =

Original: e K ≤ exp (cid:16) [Equation] n α [Equation]
Revised: e K ≤ exp (cid:16) 20+log n α (cid:17) =

Original: O [Equation] n 
Revised: O ( n 1 α ) , and the lower bound (25) becomes

Original: [Equation]
Revised: [Equation]

Original: ) , and the lower bound (25) becomes

Original: [Equation]
Revised: [Equation]

Original: which concludes our proof.
Revised: which concludes our proof.

Original: Proof of Theorem 3.2.
Revised: Proof of Theorem 3.2.

Original: We denote θ t = θ tu + θ t ⊥ , where θ tu denotes the projection of θ onto span ( u 2 ) , and θ t ⊥ denotes the projection of θ onto span ( u 2 ) ⊥ .
Revised: We denote θ t = θ tu + θ t ⊥ , where θ tu denotes the projection of θ onto span ( u 2 ) , and θ t ⊥ denotes the projection of θ onto span ( u 2 ) ⊥ .

Original: Combine Lemma 3.2 and Lemma 3.3, we have
Revised: Combine Lemma 3.2 and Lemma 3.3, we have

Original: [Equation]
Revised: [Equation]

Revised: |

Revised: [Equation]

Original: By our choice of c and T that γ 2 − c
Revised: By our choice of c and T that γ 2 − c

Original: =
Revised: =

Original: (cid:16)
Revised: (cid:16)

Original: n 1+1 /α log 2 T ηT [Equation] 1 / 2 , together Lemma 3.3, the Theorem holds as desired.
Revised: n 1+1 /α log 2 T ηT (cid:17) 1 / 2 , together Lemma 3.3, the Theorem holds as desired.

Original: Proof of Corollary 3.2.
Revised: Proof of Corollary 3.2.

Original: By Lemma 3.3 and the the choice of parameters that γ 2 − c
Revised: By Lemma 3.3 and the the choice of parameters that γ 2 − c

Original: =
Revised: =

Original: (cid:16)
Revised: (cid:16)

Original: n1+1 /α log 2 T ηT
Revised: n

Original: [Equation]
Revised: [Equation]

Original: 1  / 2 , we have:
Revised: 1 /α log 2 T ηT (cid:17) 1 / 2 , we have:

Original: [Equation]
Revised: [Equation]

Original: Together with Theorem 3.1, we have
Revised: Together with Theorem 3.1, we have

Original: [Equation]
Revised: [Equation]

Original: / 2 (cid:33)(cid:33) .

Original: where the last equality holds by the parameter choice γ 2 − c
Revised: where the last equality holds by the parameter choice γ 2 − c

Original: = (cid:16) n 1+1 /α log 2 T ηT (cid:17) 1 / 2 .
Revised: = (cid:16) n 1+1 /α log 2 T ηT (cid:17) 1 / 2 .

Original: Finally, letting µ = c (cid:0) η n 1+1 /α ) (cid:1) 1 / 2 , the claim follows immediately.
Revised: Finally, letting µ = c (cid:0) η n 1+ [Equation] [Equation] ) (cid:1) 1 / 2 , the claim follows immediately.

Original: C P ROOFS FOR S ECTION 3. 
Revised: C P ROOFS FOR S ECTION 3.

Original: In this section, we consider general (cid:96) q -norm perturbations.
Revised: In this section, we consider general (cid:96) q -norm perturbations.

Original: In short, we show that no matter how small the perturbation is, adversarial training changes the implicit bias of standard clean training using gradient descent, and adapt it to speciﬁc norm we choose for adversarial training.
Revised: In short, we show that no matter how small the perturbation is, adversarial training changes the implicit bias of standard clean training using gradient descent, and adapt it to speciﬁc norm we choose for adversarial training.

Original: Intuitively, we might expect that under the (cid:96) q -norm perturbation the implicit bias of gradient descent algorithm changes to converging in direction to (cid:96) q -norm max margin solution u q .
Revised: Intuitively, we might expect that under the (cid:96) q -norm perturbation the implicit bias of gradient descent algorithm changes to converging in direction to (cid:96) q -norm max margin solution u q .

Original: We provide a counter example here.
Revised: We provide a counter example here.

Original: Consider S
Revised: Consider S

Original: = { z 1 = ( x 1 , y 1 ) , z 2 = ( x 2 , y 2 ) } with x 1 = (10 , 1) , x 2 = ( − 10 , − 1) and y 1 = 1 , y 2 = − 1 .
Revised: = { z 1 = ( x 1 , y 1 ) , z 2 = ( x 2 , y 2 ) } with x 1 = (10 , 1) , x 2 = ( − 10 , − 1) and y 1 = 1 , y 2 = − 1 .

Original: It is easy to see that the (cid:96) ∞ -norm max margin solution is u ∞ =
Revised: It is easy to see that the (cid:96) ∞ -norm max margin solution is u ∞ =

Original: (1 , 0) with γ ∞ = 10 , and the (cid:96) 2 -norm max margin solution is u 2 = ( 10 √ 101 , [Equation] with γ 2 = √ 101 .
Revised: (1 , 0) with γ ∞ = 10 , and the (cid:96) 2 -norm max margin solution is u 2 = ( 10 √ 101 , 1 √ 101 ) with γ 2 = √ 101 .

Original: Without perturbation, we have that the gradient descent initialized at the origin converges in direction to (cid:96) 2 -norm max margin solution u 2 with one step.
Revised: Without perturbation, we have that the gradient descent initialized at the origin converges in direction to (cid:96) 2 -norm max margin solution u 2 with one step.

Original: Now we take l ∞ -norm perturbation with c = 0 .
Revised: Now we take l ∞ -norm perturbation with c = 0 .

Original: 5 , the negative gradient is given by:−∇L [Equation] sign
Revised: 5 , the negative gradient is given by:

Original: [Equation]
Revised: [Equation]

Original: We initialize gradient descent at the origin with any constant step size.
Revised: We initialize gradient descent at the origin with any constant step size.

Original: By the symmetry of the training data, we have that θ t always stays always inside quadrant I, and converges in direction to u = ( √ 361 √ 362 , 1 √ 362 ) , which is neither u ∞ or u 2 , but inside the interior of convex hull of u ∞ and u 2 .
Revised: By the symmetry of the training data, we have that θ t always stays always inside quadrant I, and converges in direction to u = ( √ 361 √ 362 , 1 √ 362 ) , which is neither u ∞ or u 2 , but inside the interior of convex hull of u ∞ and u 2 .

Original: In fact, u exactly equals to the u 2 , ∞ deﬁned in (10).
Revised: In fact, u exactly equals to the u 2 , ∞ deﬁned in (10).

Original: Proof of Lemma 3.4.
Revised: Proof of Lemma 3.4.

Original: We prove that solutions to (10) and the robust SVM against (cid:96) q -norm perturbation parameterized by c (8) are equal up to a constant factor.
Revised: We prove that solutions to (10) and the robust SVM against (cid:96) q -norm perturbation parameterized by c (8) are equal up to a constant factor.

Original: We ﬁrst have that γ 2 ,q ( c ) in (10) is equivalent to
Revised: We ﬁrst have that γ 2 ,q ( c ) in (10) is equivalent to

Original: [Equation]
Revised: [Equation]

Original: We denote the unique solution to (27) as u 2 ,q .
Revised: We denote the unique solution to (27) as u 2 ,q .

Original: It is not difﬁculty to see that
Revised: It is not difﬁculty to see that

Original: [Equation]
Revised: [Equation]

Original: We deﬁne u 2 ,q = u 2 ,q γ 2 ,q , then:
Revised: We deﬁne u 2 ,q = u 2 ,q γ 2 ,q , then:

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Revised: = 1 , . .

Original:  It is now clear that u 2 ,q is a feasible solution to (8).
Revised: , n. It is now clear that u 2 ,q is a feasible solution to (8).

Original: We denote the optimal solution to (8) as u , then we have by the optimality of u that ||
Revised: We denote the optimal solution to (8) as u , then we have by the optimality of u that ||

Original: u
Revised: u

Original: ||

Original: [Equation]

Original:  ≤ | [Equation] u
Revised: || 2 ≤ || u

Original: [Equation]
Revised: [Equation]

Original: |

Original: [Equation] ≤ [Equation] ,q , and feasibility of u that
Revised: q || 2 ≤ || u 2 ,q || 2 γ 2 ,q , and feasibility of u that

Original: [Equation]
Revised: [Equation]

Original: Then from previous two inequalities we have γ 2 ,q u is a feasible solution to (27) with objective value equal to the optimal objective value of (27).
Revised: Then from previous two inequalities we have γ 2 ,q u is a feasible solution to (27) with objective value equal to the optimal objective value of (27).

Original: Since the optimal solution to (27) is unique, this implies that u
Revised: Since the optimal solution to (27) is unique, this implies that u

Original: = u 2 ,q γ 2 ,q , which concludes our proof.
Revised: = u 2 ,q γ 2 ,q , which concludes our proof.

Original: We extend Lemma 3.1 to bounded (cid:96) q -norm perturbation set.
Revised: We extend Lemma 3.1 to bounded (cid:96) q -norm perturbation set.

Original: Lemma C.1.
Revised: Lemma C.1.

Original: Recall the deﬁnition of γ 2 ,q in (10) .
Revised: Recall the deﬁnition of γ 2 ,q in (10) .

Original: For any c < γ q , we have that (cid:104)−∇L adv ( θ ) , u 2 ,q (cid:105) ≥
Revised: For any c < γ q , we have that (cid:104)−∇L adv ( θ ) , u 2 ,q (cid:105) ≥

Original: L adv ( θ ) γ 2 ,q for all θ ∈ R d .
Revised: L adv ( θ ) γ 2 ,q for all θ ∈ R d .

Original: Proof.
Revised: Proof.

Original: Recall that we have L adv ( θ )
Revised: Recall that we have L adv ( θ )

Original: [Equation]
Revised: [Equation]

Original: For each sample ( x i , y i ) ∈ S , given a classiﬁer θ , the worst case perturbation is (cid:101) δ i = argmax||
Revised: For each sample ( x i , y i ) ∈ S , given a classiﬁer θ , the worst case perturbation is (cid:101) δ i = argmax

Original: δ || q ≤ c exp (cid:0) − y i 
Revised: [Equation] (cid:0) [Equation] y i ( x i

Original: ( x i + δ ) (cid:62) θ (cid:1) = argmin ||
Revised: [Equation] ) (cid:62) θ (cid:1) = argmin ||

Original: δ || q ≤ c y
Revised: δ || [Equation]

Original: i
Revised: i

Original: δ (cid:62) θ .
Revised: [Equation] θ .

Original: The corresponding loss is then
Revised: The corresponding loss is then

Original: [Equation]
Revised: [Equation]

Original: Since for a ﬁxed δ i , the function exp (cid:0) − y i ( x i + δ i ) (cid:62) θ (cid:1) is convex in θ , hence the gradient of L adv ( θ ) is−∇L
Revised: Since for a ﬁxed δ i , the function exp (cid:0) − y i ( x i + δ i ) [Equation] θ (cid:1) is convex in θ , hence the gradient of L adv ( θ ) is

Original: [Equation]
Revised: [Equation]

Original: Then by the deﬁnition of u 2 ,q , we have
Revised: Then by the deﬁnition of u 2 ,q , we have

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where the second inequality holds by || (cid:101)
Revised: where the second inequality holds by || (cid:101)

Original: δ i ||
Revised: δ i ||

Original: q ≤ c , and the deﬁnitions of u 2 ,q and γ 2 ,q in Lemma 3.4.
Revised: q ≤ c , and the deﬁnitions of u 2 ,q and γ 2 ,q in Lemma 3.4.

Original: Note that for q = 2 , by the fact that γ 2 , 2 ( c ) = γ 2 − c , we immediately have Lemma 3.1 holds.
Revised: Note that for q = 2 , by the fact that γ 2 , 2 ( c ) = γ 2 − c , we immediately have Lemma 3.1 holds.

Original: As a direct corollary of Lemma C.1, we have || θ t || 2 is bounded away from 0 for all t ≥ 1 .
Revised: As a direct corollary of Lemma C.1, we have || θ t || 2 is bounded away from 0 for all t ≥ 1 .

Original: Corollary C.1.
Revised: Corollary C.1.

Original: Let θ
Revised: Let θ

Original: = 0 in Algorithm 1, we have:
Revised: = 0 in Algorithm 1, we have:

Original: || θ t || 2 ≥ η 0 γ 2 ,q for al t ≥ 1 .
Revised: || θ t || 2 ≥ η 0 γ 2 ,q for al t ≥ 1 .

Original: Proof.
Revised: Proof.

Original: The proof is similar to Corollary 3.1, we omit the details here.
Revised: The proof is similar to Corollary 3.1, we omit the details here.

Original: Proof of Theorem 3.3.
Revised: Proof of Theorem 3.3.

Original: For simplicity, we deﬁne z
Revised: For simplicity, we deﬁne z

Original: i
Revised: i

Original: =
Revised: =

Original: y
Revised: y

Original: i x i and have ||
Revised: i x i and have ||

Original: z i || 2 ≤ 1 since (cid:107) x i (cid:107) 2 ≤ 1 .
Revised: z i || 2 ≤ 1 since (cid:107) x i (cid:107) 2 ≤ 1 .

Original: We have for θ
Revised: We have for θ

Original: [Equation]
Revised: [Equation]

Original:  where (cid:12) p − 1 θ denotes taking element-wise ( p − 1) -th power of θ .
Revised: ∇ [Equation] where (cid:12) p − 1 θ denotes taking element-wise ( p − 1) -th power of θ .

Original: Note that we have ||
Revised: Note that we have ||

Original: ∂
Revised: ∂

Original: ||
Revised: ||

Original: θ
Revised: θ

Original: || p
Revised: || p

Original: || q = 1 .
Revised: || q = 1 .

Original: By the conjugacy of (cid:96) p -norm and (cid:96) q -norm with 1 p + 1 q = 1 , we have ||
Revised: By the conjugacy of (cid:96) p -norm and (cid:96) q -norm with 1 p + 1 q = 1 , we have ||

Original: θ || p = max
Revised: θ || p = max

Original: || s || q ≤ 1 (cid:104) θ, s (cid:105) .
Revised: || s || q ≤ 1 (cid:104) θ, s (cid:105) .

Original: Hence we upper bound the ﬁrst term in Hessian ∇ 2 L adv ( θ ) above byn
Revised: Hence we upper bound the ﬁrst term in Hessian ∇ 2 L adv ( θ ) above by

Original: [Equation]
Revised: [Equation]

Original: We further have:
Revised: We further have:

Original: [Equation]
Revised: [Equation]

Original: Together with the fact that p ≥ 1 , we bound the Hessian ∇ 2 L adv ( θ ) as:
Revised: Together with the fact that p ≥ 1 , we bound the Hessian ∇ 2 L adv ( θ ) as:

Original: [Equation]
Revised: [Equation]

Original: Note that the Hessian expression indicates that the objective is highly non-smooth around origin.
Revised: Note that the Hessian expression indicates that the objective is highly non-smooth around origin.

Original: However, as shown in Corollary C.1, starting from origin, θ t always stays away from the origin with distance bounded below.
Revised: However, as shown in Corollary C.1, starting from origin, θ t always stays away from the origin with distance bounded below.

Original: Using Taylor expansion, and by θ t +1 =
Revised: Using Taylor expansion, and by θ t +1 =

Original: θ t − η t ∇L adv ( θ t ) , we have
Revised: θ t − η t ∇L adv ( θ t ) , we have

Original: [Equation]
Revised: [Equation]

Original: where λ ( H ( v )) max denotes the largest eigenvalue of H ( v ) , and
Revised: where λ ( H ( v )) max denotes the largest eigenvalue of H ( v ) , and

Original: [Equation]
Revised: [Equation]

Original: Since η 0 = 1 , by Corollary C.1, for any t ≥ 1 , we have ||
Revised: Since η 0 = 1 , by Corollary C.1, for any t ≥ 1 , we have ||

Original: θ t
Revised: θ t

Original: || 2 (cid:62) γ 2 ,q .
Revised: || 2 (cid:62) γ 2 ,q .

Original: Letting m p =
Revised: Letting m p =

Original: (1 + c √ d ) 2 + c
Revised: (1 + c √ d ) 2 + c

Original: ( p − 1)
Revised: ( p − 1)

Original: d 3 p − 2 2 p − 2 1 γ 2 ,q , and since that L adv ( θ ) is a convex function, we obtain that
Revised: d 3 p − 2 2 p − 2 1 γ 2 ,q , and since that L adv ( θ ) is a convex function, we obtain that

Original: [Equation]
Revised: [Equation]

Original: We then show by contradiction that we have L adv ( θ t +1 ) <
Revised: We then show by contradiction that we have L adv ( θ t +1 ) <

Original: L adv ( θ t ) .
Revised: L adv ( θ t ) .

Original: Assume this is not the case, then we have:
Revised: Assume this is not the case, then we have:

Original: [Equation]
Revised: [Equation]

Original: However, if we choose η t satisfying η t ≤ [Equation] m q L adv ( θ t ) , we have the right hand side of previous inequality strictly smaller than L adv ( θ t ) , which is clearly a constradiction.
Revised: However, if we choose η t satisfying η t ≤ 2 m q L adv ( θ t ) , we have the right hand side of previous inequality strictly smaller than L adv ( θ t ) , which is clearly a constradiction.

Original: Hence when we choose η t ≤ 2 m q L adv ( θ t ) , we have L adv ( θ t +1 ) <
Revised: Hence when we choose η t ≤ 2 m q L adv ( θ t ) , we have L adv ( θ t + [Equation]

Original: L adv ( θ t ) andL adv
Revised: L adv ( θ t ) and

Original: [Equation]
Revised: [Equation]

Original: +
Revised: Now by induction, if we choose η t = η ≤

Original: [Equation]
Revised: [Equation]

Original: Now by induction, if we choose η t = η ≤ 1 m q L adv ( θ 1 ) for t ≥ 1 , then we have (33) holds for all t ≥ 1 .
Revised: adv ( θ 1 ) for t ≥ 1 , then we have (33) holds for all t ≥ 1 .

Original: Note that we have an upper bound of L adv ( θ 1 ) , which is
Revised: Note that we have an upper bound of L adv ( θ 1 ) , which is

Original: [Equation]
Revised: [Equation]

Original: ) where (cid:101) δ i denotes the worst case perturbation to x i , and θ 1 u denotes projection of θ 1 onto span( u 2 ,q ) , and θ ⊥ denotes projection of θ 1 onto span( u 2 ,q ) ⊥ .
Revised: [Equation] where (cid:101) δ i denotes the worst case perturbation to x i , and θ 1 u denotes projection of θ 1 onto span( u 2 ,q ) , and θ ⊥ denotes projection of θ 1 onto span( u 2 ,q ) ⊥ .

Original: In summary, we have that if η t = η ≤ min { 1 M p , 1 } for all t ≥ 1 , where M p =
Revised: In summary

Original: [Equation]
Revised: [Equation]

Original: we have 
Revised: we have that if

Original: [Equation]
Revised: [Equation]

Original:  where the last inequality holds since ηm p L adv ( θ t ) ≤ ηm p L adv ( θ 1 ) ≤ 1 . Now for any θ ∈ R d
Revised: , where [Equation]

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: where the ﬁrst inequality holds by the convexity of L adv ( θ  ) 
Revised: where the last inequality holds since ηm p L adv ( θ t ) ≤ ηm p L adv ( θ 1 ) ≤ 1 . Now for any θ ∈ R d , we have

Original: [Equation]
Revised: [Equation]

Revised: where the ﬁrst inequality holds by the convexity of

Original:  and the second inequality holds by (37) [Equation]
Revised: L adv ( θ ) , and the second inequality holds by (37).

Original: Summing up the above inequality from s = 1 to t − 1 and by η t = η ≤ 1 = η 0 together with
Revised: Summing up the above inequality from s = 1 to t − 1 and by η t = η ≤ 1 = η 0 together with

Original: L adv ( θ s +1 ) ≤ L adv ( θ s ) , we have
Revised: L adv ( θ s +1 ) ≤ L adv ( θ s ) , we have

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Since θ is arbitrary, by choosing θ = log( t ) γ 2 ,q · u 2 ,q , we have
Revised: Since θ is arbitrary, by choosing θ = log( t ) γ 2 ,q · u 2 ,q , we have

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: which yields
Revised: which yields

Original: [Equation]
Revised: [Equation]

Original: Parameter Convergence: Intuition .
Revised: Parameter Convergence: Intuition .

Original: Before we formally prove the implicit bias of GDAT, we provide some intuitions here for better understanding.
Revised: Before we formally prove the implicit bias of GDAT, we provide some intuitions here for better understanding.

Original: We claim that u ∞ =
Revised: We claim that u ∞ =

Original: lim t →∞ [Equation] || θ | | 2 is in the same direction as the solution to
Revised: lim t →∞ θ t || θ | | 2 is in the same direction as the solution to

Original: [Equation]
Revised: [Equation]

Original: Note that θ t is a conic combination of { z i − ca ||
Revised: Note that θ t is a conic combination of { z i − ca ||

Original: θ t || p } i ∈
Revised: θ t || p } i ∈

Original: [ n ] , and ∂ ||
Revised: [ n ] , and ∂ ||

Original: θ t || p only depends on the direction of θ t .
Revised: θ t || p only depends on the direction of θ t .

Original: Hence by normalizing the norm of θ t and using lim t →∞ || θ t || 2 = ∞ , if the limit u ∞ =
Revised: Hence by normalizing the norm of θ t and using lim t →∞ || θ [Equation] || 2 = ∞ , if the limit u ∞ =

Original: lim t →∞ θ t || θ t || 2 exists, it satisﬁes the following condition under proper scaling that
Revised: lim t →∞ θ t || θ t || 2 exists, it satisﬁes the following condition under proper scaling that

Original: [Equation]
Revised: [Equation]

Original: Deﬁning a
Revised: Deﬁning a

Original: =
Revised: =

Original: ( a 1 , . .
Revised: ( a 1 , . .

Original: . , a n ) and ( (cid:98) θ, a ) = (cid:0) (
Revised: . , a n ) and ( (cid:98) θ, a ) = (cid:0) (

Original: || θ || p c + 1) θ, ( || θ
Revised: || θ || p c + 1) θ, ( || θ

Original: || p c
Revised: || p c

Original: + 1) a (cid:1) , it is easy to see that ( (cid:98) θ, a ) is a solution to the following system
Revised: + 1) a (cid:1) , it is easy to see that ( (cid:98) θ, a ) is a solution to the following system

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Notice that the above set of equations (41)-(43) is exactly the ﬁrst-order KKT condition of the following optimization problem
Revised: Notice that the above set of equations (41)-(43) is exactly the ﬁrst-order KKT condition of the following optimization problem

Original: [Equation]
Revised: [Equation]

Original: (44) has a robust reformulation as maximizing the (cid:96) 2 -norm margin under the worse case (cid:96) q -norm perturbation bounded by c that
Revised: (44) has a robust reformulation as maximizing the (cid:96) 2 - [Equation] margin under the worse case (cid:96) q -norm perturbation bounded by c that

Original: [Equation]
Revised: [Equation]

Original: or equivalently
Revised: or equivalently

Original: [Equation]
Revised: [Equation]

Original: We note that (45) is a Support Vector Machine problem over an uncoutable data set that is generated by norm-bounded perturbation S ( c, q ) = { ( x, y ) : where ∃ i ∈ [ n ] , || x − x i || q ≤ c, y = y i } .
Revised: We note that (45) is a Support Vector Machine problem over an uncoutable data set that is generated by norm-bounded perturbation S ( c, q ) = { ( x, y ) : where ∃ i ∈ [ n ] , || x − x i || q ≤ c, y = y i } .

Original: By the separability and c < γ q , we have that S ( c, q ) is well deﬁned.
Revised: By the separability and c < γ q , we have that S ( c, q ) is well deﬁned.

Original: By the ﬁrst-order KKT condition we have that (44) is equivalent to
Revised: By the ﬁrst-order KKT condition we have that (44) is equivalent to

Original: [Equation]
Revised: [Equation]

Original: for some proper η ( c ) that depends on c .
Revised: for some proper η ( c ) that depends on c .

Original: Hence in summary, if u ∞ =
Revised: Hence in summary, if u ∞ =

Original: lim t →∞θ
Revised: lim t →∞

Original: [Equation]
Revised: [Equation]

Original: || θ t || 2 exists, it is in the same direction as the solution to the mixed ( (cid:96) 2 , (cid:96) 1 ) -norm max margin solution of (40).
Revised: t || 2 exists, it is in the same direction as the solution to the mixed ( (cid:96) 2 , (cid:96) 1 ) -norm max margin solution of (40).

Original: Claim: In general, for (cid:96) q -norm perturbation bounded by c , θ t converges in direction to the solution tomin θ
Revised: Claim: In general, for (cid:96) q -norm perturbation bounded by c , θ t converges in direction to the solution to

Original: [Equation]
Revised: [Equation]

Original: or
Revised: or

Original: [Equation]
Revised: [Equation]

Original: for some proper η ( c ) that depends on c .
Revised: for some proper η ( c ) that depends on c .

Original: Proof of Theorem 3.4.
Revised: Proof of Theorem 3.4.

Original: Recall that in Theorem 3.3 we showed in (36) the following recursion
Revised: Recall that in Theorem 3.3 we showed in (36) the following recursion

Original: [Equation]
Revised: [Equation]

Original: where the last inequality holds by Lemma C.1.
Revised: where the last inequality holds by Lemma C

Revised: [Equation]

Revised: [Equation]

Original: Applying the previous inequality recursively from s = 1 to t − 1 , we have
Revised: Applying the previous inequality recursively from s = 1 to t − 1 , we have

Original: [Equation]
Revised: [Equation]

Original: Now since in the proof of Theorem 3.3 we showed that ηm p < 1 (35), combining the above inequality this with (37), we havet − 1 (cid:88)
Revised: Now since in the proof of Theorem 3.3 we showed that ηm p < 1 (35), combining the above inequality this with (37), we have

Original: [Equation]
Revised: [Equation]

Original: Combining this inequality with the upper bound on L adv ( θ 1 ) in (34), we have
Revised: Combining this inequality with the upper bound on L adv ( θ 1 ) in (34) [Equation] we have

Original: [Equation]
Revised: [Equation]

Original: Now for all i ∈ [ n ]
Revised: Now for all i ∈ [ n ]

Original: [Equation]
Revised: [Equation]

Original: we have:
Revised: we have:

Original: [Equation]
Revised: [Equation]

Original: which yields 
Revised: which yields min

Original: [Equation]
Revised: [Equation]

Original: Dividing both sides by || θ || [Equation] and since lim t →∞ L adv ( θ t ) = 
Revised: Dividing both sides by || θ || 2 , and since lim t →∞ L adv ( θ t ) = 0 , we have lim t →∞ ||

Original: [Equation]

Original: we have lim t →∞ ||

Original: θ t
Revised: θ t

Original: || 2 = ∞ .
Revised: || 2 = ∞ .

Original: Hence,
Revised: Hence,

Original: [Equation]
Revised: [Equation]

Original: where the last inequality holds by||
Revised: where the last inequality holds by

Original: θ t

Original: || 2 ≤ η (cid:80) t − 1 s =0 ||∇L adv

Original: ( θ s )

Original: ||

Original: [Equation]
Revised: [Equation]

Original: Hence in summary [Equation] we havemin [Equation]
Revised: Hence in summary, we have

Original: γ

Original: [Equation]
Revised: [Equation]

Original: Hence, we have lim t →∞ θ t / ||
Revised: Hence, we have lim t →∞ θ t / ||

Original: θ t
Revised: θ t

Original: || 2 is a solution to (10), but notice that the solution to (10) is unique since a multiple of its optimal solution would be the solution to (8)
Revised: || 2 is a solution to (10), but notice that the solution to (10) is unique since a multiple of its optimal solution would be the solution to (8)

Original: that
Revised: that

Original: [Equation]
Revised: [Equation]

Original: which is a convex program with strongly convex objective.
Revised: which is a convex program with strongly convex objective.

Original: By this fact, we conclude that lim t →∞ θ t || θ t || [-
-]
Revised: By this fact, we conclude that lim t [Equation] || [-
-][Equation]

Revised: ||

Original:  = u 2 ,q .
Revised: 2 = u 2 ,q .

Original: To further get the rate of convergence, we use the convergence of adversarial risk in (39), and establish the lower bound on || θ t || 2 :
Revised: To further get the rate of convergence, we use the convergence of adversarial risk in (39), and establish the lower bound on || θ t || 2 :

Original: || θ t || 2 =
Revised: || θ t || 2 =

Original: Ω (log t ) .
Revised: Ω (log t ) .

Original: Combining this with (46), the claim follows immediately.
Revised: Combining this with (46), the claim follows immediately.

Original: D 
Revised: D (cid:96) ∞ -N ORM P ERTURBATION

Original: [Equation] ERTURBATION

Original: Recall that the robust SVM against (cid:96) ∞ -norm perturbation parameterized by c is formulated as
Revised: Recall that the robust SVM against (cid:96) ∞ -norm perturbation parameterized by c is formulated as

Original: [Equation]
Revised: [Equation]

Original: and its associated max-margin classiﬁer is
Revised: and its associated max-margin classiﬁer is

Original: [Equation]
Revised: [Equation]

Original: It is easy to see that for c < γ ∞ , both γ 2 , ∞ and u 2 , ∞ are well deﬁned, and γ 2 , ∞ > 0 .
Revised: It is easy to see that for c < γ ∞ , both γ 2 , ∞ and u 2 , ∞ are well deﬁned, and γ 2 , ∞ > 0 .

Original: Before showing parameter convergence, we ﬁrst prove that the adversarial risk goes to zero.
Revised: Before showing parameter convergence, we ﬁrst prove that the adversarial risk goes to zero.

Original: To avoid analyzing (cid:96) ∞ -perturbation directly, which can go messy.
Revised: To avoid analyzing (cid:96) ∞ -perturbation directly, which can go messy.

Original: For λ > 0 , we deﬁne a smooth approximation of (cid:96) 1 -norm that
Revised: For λ > 0 , we deﬁne a smooth approximation of (cid:96) 1 -norm that

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: Note that as λ → 0 , H λ ( θ ) → || θ
Revised: Note that as λ → 0 , H λ ( θ ) → || θ

Original: || 1 uniformly.
Revised: || 1 uniformly.

Original: We then deﬁne a smoothiﬁed version of (47) that we let perturbation set be ∆ i ( λ ) =
Revised: We then deﬁne a smoothiﬁed version of (47) that we let perturbation set be ∆ i ( λ ) =

Original: { δ : ∀ j ∈ [ d ] , | δ j | ≤ c h λ ( θ j )
Revised: { δ : ∀ j ∈ [ d ] , | δ j | ≤ c h λ ( θ j )

Original: | θ j | } , and the corresponding γ 2 , ∞ and u 2 , ∞ become
Revised: | θ j | } , and the corresponding γ 2 , ∞ and u 2 , ∞ become

Original: [Equation]
Revised: [Equation]

Original: Note that the Hausdorff distance between ∆ i ( λ ) and { δ : || δ || ∞ ≤ c } converges to 0 as λ goes to 0 .
Revised: Note that the Hausdorff distance between ∆ i ( λ ) and { δ : || δ || ∞ ≤ c } converges to 0 as λ goes to 0 .

Original: It can be seen that when λ → 0 , the smoothiﬁed problem (48) reduces to (47).
Revised: It can be seen that when λ → 0 , the smoothiﬁed problem (48) reduces to (47).

Original: That is, lim λ
Revised: That is, lim λ

Original: → 0 γ 2 ,λ = γ 2 , ∞ and lim λ → 0 u 2 ,λ
Revised: → 0 γ 2 ,λ = γ 2 , ∞ and lim λ → 0 u 2 ,λ

Original: =
Revised: =

Original: u 2 , ∞ .
Revised: u 2 , ∞ .

Original: Theorem D.1.
Revised: Theorem D.1.

Original: Let perturbation set be ∆ i ( λ ) =
Revised: Let perturbation set be ∆ i ( λ ) =

Original: { δ : ∀ j ∈ [ d ] , | δ j | ≤ c h λ ( θ j )
Revised: { δ : ∀ j ∈ [ d ] , | δ j | ≤ c h λ ( θ j )

Original: | θ j | } , and let its associated adversarial risk be
Revised: | θ j | } , and let its associated adversarial risk be

Original: [Equation]
Revised: [Equation]

Original: For c < γ 2 ,λ , letting η = [Equation] + [Equation]
Revised: For c < γ 2 ,λ , letting η = 1 (1+2 cλ − 1 / 2 ) 2 , we have

Original: [Equation]

Original: 2 ) 2 , we have

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: By the deﬁnition of perturbation set that ∆ i = { δ : ∀ j ∈ [ d ] , | δ j | ≤ c h λ ( θ j ) | θ j | } , we have
Revised: By the deﬁnition of perturbation set that ∆ i = { δ : ∀ j ∈ [ d ] , | δ j | ≤ c [Equation] | [Equation]

Revised: j | }

Original: [Equation]
Revised: [Equation]

Revised: we have

Revised: [Equation]

Original: By some simple calculation, we have
Revised: By some simple calculation [Equation] we have

Original: [Equation]
Revised: [Equation]

Original: Then, it holds that
Revised: Then [Equation] it holds that

Original: [Equation]
Revised: [Equation]

Original: It can be veriﬁed that ∇ 2 L adv ( θ ) ≤ (1 + 2 c √ λ ) 2 L adv ( θ ) I .
Revised: It can be veriﬁed that ∇ 2 L adv ( θ ) ≤ (1 + 2 c √ λ ) 2 L adv ( θ ) I .

Original: By Talyer expansion, we have
Revised: By Talyer expansion, we have

Original: [Equation]
Revised: [Equation]

Original: Now we show that L adv ( θ t +1 ) ≥ L adv ( θ t ) does not hold when η ≤
Revised: Now we show that L adv ( θ t +1 ) ≥ L adv ( θ t ) does not hold when η ≤

Original: [Equation]
Revised: [Equation]

Original: L
Revised: L

Original: [Equation]
Revised: [Equation]

Original:  t ) .
Revised: ( θ t ) .

Original: Suppose the contrary holds.
Revised: Suppose the contrary holds.

Original: By (50), we have
Revised: By (50), we have

Original: L adv

Original: [Equation]
Revised: [Equation]

Original: where the last inequality holds by η =
Revised: where the last inequality holds by η =

Original: [Equation]
Revised: [Equation]

Original: adv ( θ t ) .

Original: Hence we obtain a contradiction.
Revised: Hence we obtain a contradiction.

Original: Note that L adv ( θ 0 )
Revised: Note that L adv ( θ 0 )

Original: = 1 , and if η ≤
Revised: = 1 , and if η ≤

Original: [Equation]
Revised: [Equation]

Original: holds for t = 0 , and L adv ( θ 1 ) ≤ 1 .
Revised: holds for t = 0 , and L adv ( θ 1 ) ≤ 1 .

Original: Consequently, we can inductively show that L adv ( θ t ) ≤ 1 for all t , and η ≤
Revised: Consequently, we can inductively show that L adv ( θ t ) ≤ 1 for all t , and η ≤

Original: [Equation]
Revised: [Equation]

Original:  always holds if we let η =
Revised: ) always holds if we let η =

Original: [Equation]
Revised: [Equation]

Original: By the choice of η , we obtain the following recursion taht
Revised: By the choice of η , we obtain the following recursion taht

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Using the previous recursion we have that for any θ ∈ R d ,
Revised: Using the previous recursion we have that for any θ ∈ R d ,

Original: [Equation]
Revised: [Equation]

Original: where the second inequality holds by convexity and (51).
Revised: where the second inequality holds by convexity and (51).

Original: Summing up the previous inequality from s = 0 to s = t − 1 and by L adv ( θ s +1 ) ≤ L adv ( θ s ) , we have
Revised: Summing up the previous inequality from s = 0 to s = t − 1 and by L adv ( θ s +1 ) ≤ L adv ( θ s ) [Equation] we have

Original: [Equation]
Revised: [Equation]

Original: Taking θ = log t γ 2 ,λ u 2 ,λ , we have
Revised: Taking θ = log t γ 2 ,λ u 2 ,λ [Equation] we have

Original: [Equation]
Revised: [Equation]

Original: where the last inequality holds by max δ
Revised: where the last inequality holds by max δ

Original: i ∈ ∆
Revised: i ∈ ∆

Original: i y i ( x i + δ i ) (cid:62) u 2 ,λ ≥ γ 2 ,λ .
Revised: i y i ( x i + δ i ) (cid:62) u 2 ,λ ≥ γ 2 ,λ .

Original: Hence we obtain
Revised: Hence we obtain

Original: [Equation]
Revised: [Equation]

Original: Before showing parameter convergence, we need the following lemma which is a generalization of
Revised: Before showing parameter convergence, we need the following lemma which is a generalization of

Original: Lemma 10 in Gunasekar et al.
Revised: Lemma 10 in Gunasekar et al.

Original: (2018a), but with much simpler proof.
Revised: (2018a), but with much simpler proof.

Original: Lemma D.1.
Revised: Lemma D.1.

Original: Fix c < γ 2 ,λ , for any θ ∈ RR d , we have
Revised: Fix c < γ 2 ,λ , for any θ ∈ RR d , we have

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: [Equation]
Revised: [Equation]

Original: where (cid:101) x i = argmin x (cid:48)
Revised: where (cid:101) x i = argmin x (cid:48)

Original: i − x i ∈ ∆ i ( λ )
Revised: i − x i ∈ ∆ i ( λ )

Original: y i ( x (cid:48) i ) (cid:62) θ .
Revised: y i ( x (cid:48) i ) (cid:62) θ .

Original: Then by the deﬁnition of γ 2 ,λ and u 2 ,λ (48), we have
Revised: Then by the deﬁnition of γ 2 ,λ and u 2 ,λ (48), we have

Original: [Equation]
Revised: [Equation]

Original: From which we obtain (cid:104)−∇L adv ( θ ) , u 2 ,λ (cid:105) ≥ L adv ( θ ) γ 2 ,λ , the claim follows by Cauchy-Schwarz inequality.
Revised: From which we obtain (cid:104)−∇L adv ( θ ) , u 2 ,λ (cid:105) [Equation] L adv ( θ ) γ 2 ,λ , the claim follows by Cauchy-Schwarz inequality.

Original: Theorem D.2.
Revised: Theorem D.2.

Original: Under the same setting as in Theorem D.1, we have
Revised: Under the same setting as in Theorem D.1, we have

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Recall that in Theorem D.1 we showed in (51)
Revised: Recall that in Theorem D.1 we showed in (51)

Original: that
Revised: that

Original: [Equation]
Revised: [Equation]

Original: where the last inequality holds by Lemma D
Revised: where the last inequality holds by Lemma D

Original: [Equation]
Revised: [Equation]

Original: Applying the previous inequality recursively from s = 0 to t − 1 , we have
Revised: Applying the previous inequality recursively from s = 0 to t − 1 , we have

Original: [Equation]
Revised: [Equation]

Original: Now by (51)
Revised: Now by (51)

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: which yields
Revised: which yields

Original: [Equation]
Revised: [Equation]

Original: 2 +
Revised: Next for all i ∈ [ n ]

Original: [Equation]
Revised: [Equation]

Original: Next for all i ∈ [ n ] , we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: which implies
Revised: which implies

Original: [Equation]
Revised: [Equation]

Original: Dividing both sides by || θ || 2 , and since lim t →∞ L adv ( θ t ) = 0 , we have lim t →∞ ||
Revised: Dividing both sides by || θ || 2 , and since lim t →∞ L adv ( θ t ) = 0 , we have lim t →∞ ||

Original: θ t
Revised: θ t

Original: || 2 = ∞ .
Revised: || 2 = ∞ .

Original: Hence,
Revised: Hence,

Original: [Equation]
Revised: [Equation]

Original: where the last inequality holds by ||
Revised: where the last inequality holds by ||

Original: θ t
Revised: θ t

Original: || 2 ≤ η (cid:80) t − 1 s =0 ||∇L adv
Revised: || 2 ≤ η (cid:80) t − 1 s =0 ||∇L adv

Original: ( θ s )
Revised: ( θ s )

Original: || 2 .
Revised: || 2 .

Original: In summary, we have
Revised: In summary, we have

Original: [Equation]
Revised: [Equation]

Original: Hence lim θ t
Revised: Hence lim θ t

Original: || θ t
Revised: || θ t

Original: || 2 is a solution to (48).
Revised: || 2 is a solution to (48).

Original: Note that the solution to (48) is unique since it is equivalent to
Revised: Note that the solution to (48) is unique since it is equivalent to

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: We thus conclude that lim t →∞
Revised: We thus conclude that lim t →∞

Original: [Equation]
Revised: [Equation]

Original: To summarize, we have shown that for all λ > 0 , lim t →∞ θ t || θ t
Revised: To summarize, we have shown that for all λ > 0 , lim t →∞ [Equation]

Original: [-|| 
-]
Revised: [-|| 
-]The

Original: = u 2 ,λ .
Revised: [Equation]

Original: The (cid:96) ∞ -norm perturbation corresponds to the case when λ → 0 , it is natural to conclude that for (cid:96) ∞ perturbation, we have lim t →∞ θ t ||
Revised: ∞ -norm perturbation corresponds to the case when λ → 0 , it is natural to conclude that for (cid:96) ∞ perturbation, we have lim t →∞ [Equation] ||

Original: θ t
Revised: [Equation]

Original: ||[-
-]
Revised: ||[-
-]

Original:  = u 2 , ∞ .
Revised: 2 = u 2 , ∞ .

Original: The discussion for q = 1 follows similar argument, hence we omit the details here.
Revised: The discussion for q = 1 follows similar argument, hence we omit the details here.

Original: E A DDITIONAL E XPERIMENTS ON P ERTURBATION L EVEL AND S PEED - UP
Revised: E A DDITIONAL E XPERIMENTS ON P ERTURBATION L EVEL AND S PEED - UP

Original: We provide additional experiments on the connection of perturbation level c and the speed-up effect of adversarial training for neural networkds.
Revised: We provide additional experiments on the connection of perturbation level c and the speed-up effect of adversarial training for neural networkds.

Original: We run GDAT with (cid:96) ∞ -norm perturbation.
Revised: We run GDAT with (cid:96) ∞ -norm perturbation.

Original: The setup of the experiments is exactly the same as the setup in Section 4.
Revised: The setup of the experiments is exactly the same as the setup in Section 4.

Original: We will vary the perturbation level c used in GDAT algorithm in { 0 . 1 , 0 . 15 , 0 . 2 } .
Revised: We will vary the perturbation level c used in GDAT algorithm in { 0 . 1 , 0 . 15 , 0 . 2 } .

Original: From Figure 3 we could see that GDAT indeed accelerates convergence of loss and accuracy on clean training samples.
Revised: From Figure 3 we could see that GDAT indeed accelerates convergence of loss and accuracy on clean training samples.

Original: Moreoever, the acceleration effect is stronger when we use larger perturbation level, and this relationship is consistent across different width of hidden layer.
Revised: Moreoever, the acceleration effect is stronger when we use larger perturbation level, and this relationship is consistent across different width of hidden layer.

Original: Similar speed-up effects on the test loss and test accuracy evaluated on clean test samples are also observed for GDAT.
Revised: Similar speed-up effects on the test loss and test accuracy evaluated on clean test samples are also observed for GDAT.

Original: From Figure 4, we see that the speed-up effects become stronger when we use larger perturbation level, and this relationship is consistent across different width of hidden layer.
Revised: From Figure 4, we see that the speed-up effects become stronger when we use larger perturbation level, and this relationship is consistent across different width of hidden layer.

Original: Traditionally, the beneﬁt of adversarial training is understood as two fold: 1.
Revised: Traditionally, the beneﬁt of adversarial training is understood as two fold: 1.

Original: it improves the robustness of the learning algorithm, i.e., the solution has better loss toward adversarilly perturbed seample; 2.
Revised: it improves the robustness of the learning algorithm, i.e., the solution has better loss toward adversarilly perturbed seample; 2.

Original: it has better generalization ability.
Revised: it has better generalization ability.

Original: Our experiments demonstrate a third property of adverserial training that is not known in literature before, i.e., adversarial training accelerates convergence.
Revised: Our experiments demonstrate a third property of adverserial training that is not known in literature before, i.e., adversarial training accelerates convergence.

Original: [Figure] l [Figure] n 
Revised: A cc u r a c y : c l e a n d a t a

Original: [Figure]
Revised: [Figure]

Original:  ss : c l 
Revised: L o ss : c l e a n

Original: [Figure]
Revised: [Figure]

Original: a n
Revised: a t a

Original: [Figure]
Revised: [Figure]

Original: a

Original: [Figure][Figure]
Revised: [Figure]

Original: u
Revised: u

Original: [Figure]
Revised: [Figure]

Original: a c y : c l e a n
Revised: a c y : c l e a n

Original: [Figure]
Revised: [Figure]

Original: a t a
Revised: a t a

Original: [Figure]
Revised: [Figure]

Original: : c l
Revised: : c l

Original: [Figure]
Revised: [Figure]

Original: a n d a t a
Revised: a n d a t a

Original: [Figure]
Revised: [Figure]