E XPLORING AND E XPLOITING D ECISION B OUNDARY D YNAMICS FOR A DVERSARIAL R OBUSTNESS
Anonymous authors
Paper under double-blind review
A BSTRACT
The robustness of a deep classifier can be characterized by its margins : the decision boundary’s distances to natural data points.
However, it is unclear whether existing robust training methods effectively increase the margin for each vulnerable point during training.
To understand this, we propose a continuous-time framework for quantifying the relative speed of the decision boundary with respect to each individual point.
Through visualizing the moving speed of the decision boundary under Adversarial Training, one of the most effective robust training algorithms, a surprising moving-behavior is revealed: the decision boundary moves away from some vulnerable points but simultaneously moves closer to others, decreasing their margins.
To alleviate these conflicting dynamics of the decision boundary, we propose Dynamics-Aware Robust Training ( DyART ), which encourages the decision boundary to engage in movement that prioritizes increasing smaller margins.
In contrast to prior works, DyART directly operates on the margins rather than their indirect approximations, allowing for more targeted and effective robustness improvement.
Experiments on the CIFAR-10 and Tiny-ImageNet datasets verify that DyART alleviates the conflicting dynamics of the decision boundary and obtains improved robustness under various perturbation sizes compared to the state-of-the-art defenses
I NTRODUCTION
Deep neural networks have exhibited impressive performance in a wide range of applications (Krizhevsky et al., 2012;
Goodfellow et al., 2014;
He et al., 2016a).
However, they have also been shown to be susceptible to adversarial examples, leading to issues in security-critical applications such as autonomous driving and medicine (Szegedy et al., 2013; Nguyen et al., 2015).
To alleviate this problem, adversarial training (AT) (Madry et al., 2017; Shafahi et al., 2019; Zhang et al., 2019; Gowal et al., 2020) was proposed and is one of the most prevalent methods against adversarial attacks.
Specifically, AT aims to find the worst-case adversarial examples based on some surrogate loss and adds them to the training dataset in order to improve robustness.
Despite the success of AT, it has been shown that over-parameterized neural networks still have insufficient model capacity for fitting adversarial training data, partly because AT does not consider the vulnerability difference among data points (Zhang et al., 2021).
The vulnerability of a data point can be measured by its margin: its distance to the decision boundary.
As depicted in Figure 1a, some data points have smaller margins and are thus more vulnerable to attacks.
Since AT does not directly operate on the margins and it uses a pre-defined perturbation bound for all data points regardless of their vulnerability difference, it is unclear whether the learning algorithm can effectively increase the margin for each vulnerable point.
Geometrically, we would like to know if the decision boundary moves away from the data points, especially the vulnerable ones.
As illustrated in Figure 1b, there can exist conflicting dynamics of the decision boundary: it moves away from some vulnerable points but simultaneously moves closer to other vulnerable ones during training.
This motivates us to ask:
Question
Given a training algorithm, how can we analyze the dynamics of the decision boundary with respect to the data points?
To answer the above question, we propose a continuous-time framework that quantifies the instantaneous movement of the decision boundary as shown in Figure 1c.
Specifically, we define the relative speed of the decision boundary w.r.t.
a point to be the time derivative of its margin, which can be
[Figure]
interpreted as the speed of its closest adversarial example moving away from it.
We show that the speed can be derived from the training algorithm using a closed-form expression.
Using the proposed framework, we empirically compute the speed of the decision boundary w.r.t.
data points for AT.
As will be shown in Figure 3, the aforementioned conflicting dynamics of the decision boundary (Figure 1b,1c) is revealed: the decision boundary moves towards many vulnerable points during training and decrease their margins, directly counteracting the objective of robust training.
The desirable dynamics of the decision boundary, on the other hand, should increase the margins of all vulnerable points.
This leads to another question:
Question 2 How to design algorithms that encourage the decision boundary to engage in movements that increase margins for vulnerable points, and not decrease them?
To this end, we propose Dynamics-Aware Robust Training ( DyART ) , which prioritizes moving the decision boundary away from more vulnerable points and increasing their margins.
Specifically,
DyART directly operates on margins of training data and carefully designs its cost function on margins for more desirable dynamics.
Note that directly optimizing margins in the input space is technically challenging since it was previously unclear how to compute the gradient of the margin.
In this work, we derive the closed-form expression for the gradient of the margin and present an efficient algorithm to compute it, making gradient descent viable for DyART .
In addition, since
DyART directly operates on margins instead of using a pre-defined uniform perturbation bound for training as in AT, DyART is naturally robust for a wide range of perturbation sizes ϵ .
Experimentally, we demonstrate that DyART mitigates the conflicting dynamics of the decision boundary and achieves improved robustness performance on diverse attacking budgets.
Summary of contributions.
(1) We propose a continuous-time framework to study the relative speed of the decision boundary w.r.t.
each individual data point and provide a closed-form expression for the speed.
(2) We visualize the speed of the decision boundary for AT and identify the conflicting dynamics of the decision boundary.
(3) We present a close-form expression for the gradient of the margin, allowing for direct manipulation of the margin.
(4) We introduce an efficient alternative to compute the margin gradient by replacing the margin with our proposed soft margin , a lower bound of the margin whose approximation gap is controllable.
(5) We propose Dynamics-Aware Robust
Training ( DyART ), which alleviates the conflicting dynamics by carefully designing a cost function on soft margins to prioritize increasing smaller margins.
Experiments show that DyART obtains improved robustness over state-of-the-art defenses on various perturbation sizes.
R ELATED W ORK
Decision boundary analysis.
Prior works on decision boundary of deep classifiers have studied the small margins in adversarial directions (Karimi et al., 2019), the topology of classification regions (Fawzi et al., 2018), the relationship between dataset features and margins (Ortiz-Jimenez et al., 2020) and improved robust training by decreasing the unwarranted increase in the margin along adversarial directions (Rade & Moosavi-Dezfooli, 2022).
While these works study the static decision boundary of trained models, our work focuses on the decision boundary dynamics during training.
Weighted adversarial training.
Adversarial training and its variants (Madry et al., 2017; Zhang et al., 2019; Wang et al., 2019; Zhang et al., 2020b) have been proposed to alleviate the adversarial vulnerability of deep learning models.
To better utilize the model capacity, weighted adversarial training methods are proposed (Zeng et al., 2020; Liu et al., 2021; Zhang et al., 2021) aiming to assign larger weights to more vulnerable points closer to the decision boundary.
However, these methods rely on indirect approximations of margins that are not optimal.
For example, GAIRAT (Zhang et al., 2021) uses the least number of iterations needed to flip the label of an clean example as an surrogate to its margin, which is shown to be likely to make wrong judgement on the robustness (Liu et al., 2021).
As another approximation, the logit margin (Liu et al., 2021; Zeng et al., 2020) is used but larger logit margin values do not necessarily correspond to larger margins.
In contrast, our proposed DyART directly uses margins to characterize the vulnerability of data points.
Margin maximization.
Increasing the distance between the decision boundary and data points has been discussed in prior works.
The authors of Elsayed et al.
(2018) propose to maximize the first order Taylor’s expansion approximation of the margin at the clean data point, which is inaccurate and computationally prohibitive since it requires computing the Hessian of the classifier.
The authors of
Atzmon et al.
(2019) propose to maximize the distance between each data point and some point on the decision boundary, which is not the closest one and thus does not increase the margin directly.
MMA (Ding et al., 2020) uses the uniform average of cross-entropy loss on the closest adversarial examples as the objective function, indirectly increasing the average margins.
All of these methods maximize the average margin indirectly and do not consider the vulnerability differences among points.
In contrast, our proposed DyART will utilize our derived closed-form expression for margin gradient to directly operate on margins and moreover, prioritize increasing smaller margins.
P RELIMINARIES AND N OTATIONS
Data and model.
We consider a classification task with inputs x ∈ X and corresponding labels y ∈
Y = { 1 , 2 , ..., K } .
A deep classifier parameterized by θ is denoted by f θ ( x ) =
arg max c ∈Y z cθ ( x ) where z cθ ( x ) is the logit for class c .
Decision boundary.
Denote the logit margin for class y as follows:
[Equation]
In this paper, we will use ϕ yθ ( x ) and ϕ y ( x, θ ) interchangeably.
Observe that x is classified as y if and only if ϕ yθ ( x ) > 0 .
Therefore, the zero level set of ϕ yθ ( x ) , denoted by Γ y
= { x : ϕ yθ ( x ) = 0 } , is the decision boundary for class y .
For a correctly classified point ( x, y ) , its margin R θ ( x ) is defined to be the distance from x to the decision boundary for class y .
That is,
[Equation]
where ∥ · ∥ p is the ℓ p norm with 1 ≤ p ≤ ∞ .
Difference between logit margin and margin.
The logit margin ϕ yθ ( x ) is the gap between the logits values that are output by the neural network.
On the other hand, the margin R θ ( x ) is the distance from the data point to the decision boundary, and is measured in the input space X .
Continuous-time formulation.
To study the instantaneous movement of the decision boundary in Section 4, we will use the continuous-time formulation for the optimization on the parameters θ , denoted as θ ( t ) .
Let θ ′ ( t ) be the continuous-time description of the update rule of the model parameters.
When using gradient descent on a loss function L , we have θ ′ ( t )
=
−∇ θ
L ( θ ( t )) .
D YNAMICS OF THE DECISION BOUNDARY
In this section, we will study the dynamics of the decision boundary in continuous time.
We first define its speed w.r.t.
each data point, and then provide a closed-form expression for it.
Finally, we visualize the speed of the decision boundary under Adversarial Training and analyze its dynamics.
4. S PEED OF THE DECISION BOUNDARY
Consider a correctly classified clean example ( x i , y i ) .
Our goal is to capture the movement of the decision boundary Γ
y
i ( t ) =
{ x : ϕ y i ( x, θ ( t ))
= 0 } w.r.t.
x i as t varies continuously.
To this end, we consider the curve of the closest boundary point ˆ x i ( t ) on Γ
y i ( t ) to x i :
Definition 1 (Curve of the closest boundary point ˆ x i ( · ) ) .
Suppose that ( x i , y i ) is correctly classified by f θ ( t ) in some time interval I .
Define the curve of the closest boundary point ˆ x i ( · ) : I → X as
[Equation]
Define the margin of x i at time t to be R ( x i , t ) =
∥
ˆ x i ( t ) − x i ∥ p .
An example of the curve of the closest boundary point is depicted in Figure 2.
To understand how the distance between the decision boundary Γ y i ( t ) and x i changes, it suffices to focus on the curve of the closest boundary point
ˆ x i ( t ) .
We define the speed of the decision boundary to be the time derivative of the margin as follows:
Definition 2 (Speed of the decision boundary s ( x i , t ) ) .
Under the setting of definition 1, define the speed of the decision boundary w.r.t.
x i as
[Equation]
i
[Equation]
Note that the speed s ( x i , t ) > 0 means that the robustness is improving for x i at time t , which is desirable during robust training.
The following proposition gives a closed-form expression for the speed, given a training algorithm θ ′ ( t ) .
Proposition 3 (Closed-form expression of the speed s ( x i , t ) ) .
Let ˆ x i ( t ) be the curve of the closest boundary point w.r.t.
x i .
For 1 ≤ p ≤ ∞ , the speed of decision boundary w.r.t.
x i under ℓ p norm is
[Equation]
where q satisfies that 1 /q + 1 /p = 1 .
In particular, q = 1 when p = ∞ .
Remark.
Note that equation 4 is still valid when ˆ x i ( t ) is just a locally closest boundary point (i.e. a local optimum of equation 3).
In this case, s ( x i , t ) is interpreted as the moving speed of decision boundary around the locally closest boundary point
ˆ x i ( t ) .
The main assumption for equation 4 is the local differentiability of ϕ y i ( · , θ ( t )) at ˆ x i ( t ) .
The full assumptions, proof and discussions are left to
Appendix B. Special care has been taken for p = ∞ since ℓ ∞ norm is not differentiable.
According to equation 4, the speed s ( x i , t 0 ) is positive at time t 0 when ∇ θ ϕ
y i (ˆ x i ( t 0 ) , θ ( t 0 )) · θ ′ ( t 0 ) > 0 , i.e., ϕ y
i
(ˆ x i ( t ) , θ ( t ))
increases at time t 0 , meaning that the boundary point
ˆ x i ( t 0 ) will be correctly classified after the update.
Also, the magnitude of the speed tends to be larger if ∥∇ x ϕ
y
i
(ˆ x i ( t ) , θ ( t ))
∥ q is smaller, i.e., the margin function ϕ
y i is flatter around ˆ x i ( t ) .
In the remaining part of the paper, we will denote s ( x i , t ) and R ( x i , t ) as s ( x i ) and R ( x i ) when the indication is clear.
Computing the closest boundary point.
We use the Fast Adaptive Boundary Attack (FAB) (Croce & Hein, 2020a) to compute ˆ x i ( t ) in equation 4.
Specifically, FAB iteratively projects onto the linearly approximated decision boundary with a bias towards the original data point, so that the resulting boundary point is close to the original point x i .
Note that FAB only serves as an algorithm to find ˆ x i ( t ) , and can be decoupled from the remaining part of the framework.
In our experiments we find that FAB can reliably find locally closest boundary points given enough iterations, where the speed expression in equation 4 is still valid.
We give more details of how to check the local optimality condition of equation 3 and the performance of FAB in Appendix C.1.
Note that in Section 5.2, we will see that directly using FAB is computationally prohibitive for robust training and we will propose a more efficient solution.
In the next section, we will still use FAB to find closest boundary points for more accurate visualization of the dynamics during adversarial training.
4. D YNAMICS OF ADVERSARIAL TRAINING
In this section, we numerically investigate the dynamics of the decision boundary during adversarial training.
We visualize the speed and identify the conflicting dynamics of the decision boundary.
Experiment setting.
To study the dynamics of AT in different stages of training where models have different levels of robustness, we train a ResNet-18 (He et al., 2016a) with group normalization (GN) (Wu & He, 2018) on CIFAR-10 using 10-step PGD under ℓ ∞ perturbation with ϵ = 8255 from two pretrained models: (1) a partially trained model using natural training with 85% clean accuracy and 0% robust accuracy; (2) a partially trained model using AT with 75% clean accuracy and 42% robust accuracy under 20-step PGD attack.
Note that we replace the batch normalization (BN) layers with
GN layers since the decision boundaries are not the same during training and evaluation when BN is used, which can cause confusion when studying the dynamics of the decision boundary.
On both pretrained models, we run one iteration of AT on a batch of training data.
For correctly classified points in the batch of data, we compute the margins as well as the speed of the decision boundary.
[Figure]
Conflicting dynamics.
The dynamics of the decision boundary on both pretrained models under AT is shown in Figure 3.
The speed values are normalized so that the maximum absolute value is 1 for better visualization of their relative magnitude.
We can observe that on both pretrained models, the decision boundary has negative speed w.r.t.
a significant proportion of non-robust points with R ( x i ) < 8 255 .
That is, the margins of many vulnerable points decrease during adversarial training even though the current update of the model is computed on these points, which counteracts the objective of robust training.
In the next section, we will design a dynamics-aware robust training method to mitigate such conflicting dynamics issue.
D Y ART : D YNAMICS - AWARE R OBUST T RAINING
In this section, we propose Dynamics-Aware Robust Training ( DyART ) to mitigate the conflicting dynamics issue.
In Section 5.1, we show how to design an objective function to prioritize improving smaller margins and how to compute the gradient of such objective.
In Section 5.2, we overcome the expensive cost of finding the closest boundary points and present the full DyART algorithm.
5. O BJECTIVE FOR DESIRABLE DYNAMICS
We aim to design a loss function L R ( θ ) to directly increase the overall margins for effective robustness improvement.
We propose to use the robustness loss L R ( θ ) :
=
E x
[ h ( R θ ( x ))] , where h : R → R is a carefully selected cost function that assigns a cost value
h
( R ) to a margin R .
When designing h ( · ) , it is crucial that minimizing L R ( θ ) =
E x [ h ( R θ ( x ))] encourages the desirable dynamics of the decision boundary: the decision boundary has positive speed w.r.t.
vulnerable points with small margins.
Dynamics-aware loss function.
To design such a dynamics-aware loss function, the following two properties of the cost function h are desired.
(1) Decreasing (i.e., h ′ ( · ) < 0 ): a point with a smaller margin should be assigned a higher cost value since it is more vulnerable.
(2) Convex (i.e., h ′′ ( · ) > 0 ): the convexity condition helps prioritize improving smaller margins.
To see this, consider minimizing the loss function L R ( θ ) on m points { x i , y i } mi =1 with margins { R i } mi =1 , where the objective becomes 1 m (cid:80) mi =1 h ( R θ ( x i )) .
At each iteration, the optimizer should update the model to decrease the objective value.
Therefore, in continuous-time we have that ddt (cid:80) mi =1 h ( R ( x i , t )) < 0 .
Using the chain rule and the definition that the speed s ( x i , t ) = ddt R ( x i , t ) , we obtain that (cid:80) mi =1 h ′
( R i ) s ( x i , t ) < 0 .
Given that h ′ ( · ) < 0 , the ideal case is that s ( x i , t ) > 0 for all x i and thus the sum (cid:80) mi =1 h ′ ( R i ) s ( x i , t ) < 0 .
In this case, the margins of all data points increase.
However, due to the existence of conflicting dynamics as described in Section 4.2, some points may have negative speed s ( x i , t ) < 0 while (cid:80) mi =1 h
( R i ) s ( x i , t ) stays negative.
In the presence of such conflicting dynamics, if | h ′ ( R i ) | is large (i.e., h ′ ( R i ) is small since h ′ ( · ) < 0 ), it is more likely that s ( x i , t ) >since otherwise it is harder to make (cid:80) m i
=1 h ′
( R i ) s ( x i , t ) negative.
When h ′′ ( · ) > 0 , a smaller margin R i has smaller h ′
( R i ) and thus s ( x i , t ) tends to be positive.
Therefore, requiring h ′′ ( · ) >incentivizes the decision boundary to have positive speed w.r.t.
points with smaller margins.
How to design the optimal h ( · ) is still an open problem.
In this paper, we propose to use
[Equation]
where the hyperparameters α > 0 and r 0 > 0 .
Larger α values prioritize improving smaller margins.
The threshold r 0 is used to avoid training on points that are too far away from the clean data points.
Difficulties of computing margin gradient.
Directly minimizing E x [ h ( R θ ( x ))] through gradientbased optimization methods requires computing the gradient ∇ θ h ( R θ ( x i ))
w.r.t.
the model parameters.
However, it was previously unclear how to compute ∇ θ h
( R θ ( x i )) , which partly explains why previous works did not directly operate on the margins.
The difficulty of computing ∇ θ h ( R θ ( x i )) lies in the fact that R θ ( x i ) , as defined in equation 2, involves a constrained optimization problem and thus its gradient ∇ θ
R
θ
( x i ) cannot be computed straightforwardly.
An additional challenge is dealing with the non-smoothness of the ℓ ∞ norm, which is widely used in the robust training literature.
Our solution.
We overcome the above challenges and provide the following close-form expression for the gradient of any smooth function of the margin.
The proof is provided in Appendix B.
Theorem 4 (The gradient ∇ θ h
( R θ ( x i )) of any smooth function of the margin) .
For 1 ≤ p ≤ ∞ ,
[Equation]
where q satisfies that 1 /q + 1 /p = 1 .
In particular, q = 1 when p = ∞ .
Note that another expression for the margin gradient (i.e., h is the identity function in equation 6) was derived in MMA (Ding et al., 2020), with the following distinctions from us: (a) The expression in MMA does not apply to the ℓ ∞ norm while ours does.
(b) The coefficient
[Equation]
q in our expression is more informative and simpler to compute.
(c) MMA treats the aforementioned coefficient as a constant during training, and therefore does not properly follow the margin gradient.
Computing ∇ θ h ( R θ ( x i )) requires computing the closest boundary points ˆ x i , which can be computationally prohibitive for robust training.
In the next section, we propose to use the closest point ˆ x soft i on the soft decision boundary instead, whose quality of approximation to the exact decision boundary is controllable and computational cost is tractable.
We will then present the full DyART algorithm.
5. E FFICIENT R OBUST T RAINING
Directly finding the closest boundary points is expensive.
Since the closest boundary point ˆ x i can be on the decision boundary between the true class and any other class, FAB needs to form a linear approximation of the decision boundary between the true class and every other class at each iteration.
This requires computing the Jacobian of the classifier, and the computational cost scales linearly with the number of classes K (Croce & Hein, 2020b).
Therefore, finding the closest points on the exact decision boundary is computationally prohibitive for robust training in multi-class classification settings, especially when K is large.
To remedy this, we propose to instead use the closest points on the soft decision boundary as elaborated below.
Soft decision boundary.
We replace the maximum operator in logit margin (equation 1) with a smoothed maximum controlled by the temperature β > 0 .
Specifically, we define the soft logit margin of the class y as
[Equation]
The soft decision boundary is defined as the zero level set of the soft logit margin: Γ soft y
[Equation]
{ x
[Equation]
Φ yθ ( x ; β ) = 0 } .
For x i with Φ y i θ
( x i ; β ) > 0 , the closest soft boundary point is defined as
[Equation] [Equation]
and the soft margin is defined as R soft θ ( x i ) =
∥
ˆ x soft i − x i ∥ p .
Note that we do not define R soft ( x i ) when Φ y i θ ( x i ; β ) < 0 .
The relationship between the exact and soft decision boundary is characterized by the following proposition:
Proposition 5.
If x is on the soft decision boundary Γ soft y , i.e. Φ yθ ( x ; β ) = 0 , then log( K − 1)
β ≥ ϕ yθ ( x ) ≥ 0 .
Moreover, when Φ y i θ ( x i ; β ) > 0 , then R soft θ ( x i ) ≤ R θ ( x i ) .
In other words, the soft decision boundary is always closer to x i than the exact decision boundary as shown in Figure 4.
Moreover, the quality of approximation to the exact decision boundary is controllable: the gap between the two decreases as β increases and vanishes when β → ∞ .
Therefore, increasing the soft margins will increase the exact margins as well.
Benefits of the soft decision boundary.
(1) Computational efficiency.
Using the smoothed max operator, finding the closest soft boundary point does not require forming linear approximations for the decision boundary between the true class and all the other classes anymore.
We adapt the FAB method to solve for ˆ x soft i (see details in Appendix C.2).
Its computational cost for each iteration is independent of the number of classes K , which is the same as the
PGD training.
(2) Effective information usage.
Another benefit of using the smoothed max operator in soft logit margin is that, unlike the logit margin ϕ y i θ ( x i ) , the soft logit margin Φ y i θ ( x i ; β ) contains information of logit values of all classes.
Therefore, the information of all classes is used at each iteration when finding ˆ x soft i .
Loss function and its gradient.
The overall objective of DyART is to increase the soft margins and also achieve high clean accuracy.
Denote a training data batch B of size n and B + θ of size m to be { i ∈ B : Φ y i θ ( x i ; β ) > 0 } .
Our proposed method DyART uses the following loss function
[Equation]
where the first term is the average cross-entropy loss on natural data points and the second term is for increasing the soft margins.
The hyperparameter λ balances the trade-off between clean and robust accuracy.
By applying equation 6, the gradient of the objective can be computed as
[Equation]
Since the soft margin R soft θ ( x i ) is only defined for x i with Φ y i θ ( x i ; β ) > 0 , DyART requires training on a pretrained model with a relatively high proportion of points with positive Φ
y i θ values.
In practice, we find that a burn-in period of several epochs of natural training is enough for such pretrained model.
Novelty compared with prior works.
(1) Direct and efficient manipulation of the margin.
(1a)
In contrast to prior works that depend on indirect approximations of margins, DyART directly operates on margins by utilizing the closed-from expression for the margin gradient in equationwhose computation was previously unclear.
(1b)
We significantly reduce the computational cost of computing margins and its gradients by introducing the soft margin , a lower bound of the margin whose approximation gap is controllable.
(2) Prioritizing the growth of smaller margins by carefully designing the cost function h ( · ) to mitigate the conflicting dynamics.
Therefore, DyART achieves more targeted and effective robustness improvement by directly and efficiently operating on margins as well as prioritizing the growth of smaller margins.
E XPERIMENTS
In this section, we empirically evaluate the effectiveness and performance of the proposed DyART on the CIFAR-10 (Krizhevsky et al., 2009) and Tiny-ImageNet (Deng et al., 2009) datasets.
In
Section 6.1, we evaluate the adversarial robustness of DyART and compare it with several state-ofthe-art baselines.
In Section 6.2, we visualize the dynamics of the decision boundary under DyART and analyze how it alleviates the conflicting dynamics.
6. R OBUSTNESS E VALUATION
Architectures and training parameters.
In all experiments on the CIFAR-10 dataset, we use the Wide Residual Network (Zagoruyko & Komodakis, 2016) with depth 28 and width factor
(WRN-28-10).
On the Tiny-ImageNet dataset, we use pre-activation ResNet-18 (He et al., 2016b).
For both architectures, we replace the batch normalization layers with group normalization layers so that the decision boundaries are the same during training and evaluation.
All models are trained using stochastic gradient descent with momentum 0.9 and weight decay 0.0005 for 100 epochs with batch size 256 .
We use a cosine learning rate schedule (Loshchilov & Hutter, 2016) without restarts where the initial learning rate is set to 0 .
1 for baselines in all experiments.
For DyART on Tiny-ImageNet, we use the cosine learning rate schedule with the initial learning rate 0 .
and on CIFAR-10, the learning rate begins at 0.1 and is decayed by a factor of 10 at the 50th and 75th epoch.
To alleviate robust overfitting (Rice et al., 2020), we perform early stopping on a validation set of size 1024 using projected gradient descent (PGD) attacks with 20 steps.
Baselines.
On CIFAR-10, the baselines include: (1) standard adversarial training (AT) (Madry et al., 2017) which trains on the worst case adversarial examples; (2) TRADES (Zhang et al., 2019) which trades off between the clean and robust accuracy; (3) MMA (Ding et al., 2020) which uses cross-entropy loss on the closest boundary points; (4) GAIRAT (Zhang et al., 2021) which reweights adversarial examples based on the least perturbation iterations.
(5) MAIL
(Liu et al., 2021) which reweights adversarial examples based on their logit margins.
(6) FAT (Zhang et al., 2020b) which utilizes friendly adversarial data; (7) MART (Wang et al., 2019) which explicitly differentiates between the correctly and incorrectly classified examples.
For MAIL and FAT, we report their combinations with TRADES, which have higher robust accuracy.
On Tiny-ImageNet, we compare with AT, TRADES, and MART whose hyperparameter settings are available for this dataset.
The hyperparameters of baselines are found in Appendix D.1.
Evaluation details.
We evaluate DyART and the baselines under ℓ ∞ norm constrained perturbations.
The final robust accuracy is reported on AutoAttack (AA) (Croce & Hein, 2020b).
For all methods, we choose the hyperparameters to achieve the best robust accuracy under the commonly used perturbation bound ϵ = 8255 .
To fully compare the robustness performance among different methods, we report the robust accuracy under four additional perturbation bounds: 2255 , 4255 , 12255 and 16255 .
Hyperparameters of DyART .
We use the cost function h ( · ) in equation 5.
On CIFAR-10, we use α = 8 , r 0 = 16255 and λ = 400 .
On Tiny-ImageNet, we use α = 3 , r 0 = 20255 and λ = 500 .
The temperature β is set to 5 .
We use 20 iterations to find the closest soft boundary points using the adapted version of FAB.
We use 10 epochs of natural training as the burn-in period.
[Table]
[Table] [Table]
[Table] [Table]
[Table]
Performance.
The evaluation results on CIFAR-10 and Tiny-ImageNet are shown in Table 1 and
Table 2, respectively.
Overall on both datasets, under four out of five perturbation bounds, DyART achieves the best robustness performance.
This indicates the superiority of DyART in increasing margins.
(1) Specifically, on CIFAR-10, DyART achieves the highest clean accuracy as well as robust accuracy under all perturbation bounds among all baselines except FAT-TRADES.
(1a)
Since
FAT-TRADES prevents the model from learning on highly adversarial data in order to keep clean accuracy high, it achieves the best clean accuracy and robustness under a very small perturbation bound 2255 .
However, its performance on larger perturbation bounds is inadequate.
(1b)
Thanks to directly operating on margins in the input space and encourage robustness improvement on points with smaller margins, DyART performs better than GAIRAT and MAIL-TRADES which use indirect approximations of the margins.
(2) On Tiny-ImageNet, DyART achieves the best clean accuracy and the best robust accuracy under all perturbation bounds except the largest 16255 .
Although MART is the most robust under 16255 , it has much lower clean accuracy ( 8 . 93% lower than DyART ) and worse robustness under smaller perturbation bounds.
Further experimental results on both datasets using various hyperparameter settings are left to Appendix D.2.
6. D YNAMICS OF D Y ART
In this section we provide further insights into how DyART encourages the desirable dynamics by comparing it with adversarial training.
Experimental setting.
To compare the dynamics of the decision boundary during training using
DyART and AT, we empirically compute the margins and speed values for both methods.
For fair comparison, we run DyART and AT on the same pretrained models for one iteration on the same batch of training data points.
The pretrained models include a partially trained model using natural training and a partially trained model using AT, which are the same as in Section 4.2.
For all the correctly classified points in this batch, we compute the margins and speed values under both methods.
Note that the speed and margins correspond to the exact decision boundary, instead of the soft decision boundary used by DyART for robust training.
Since both methods train on the same model and the same batch of data at this iteration, the margins are the same and only the speed values differ, which corresponds to the difference in dynamics of the decision boundary.
[Figure]
DyART mitigates the conflicting dynamics.
We visualize the dynamics on both pretrained models under
DyART and AT in Figure 5.
Specifically, we divide the range of margins into multiple intervals and compute the proportion of positive and negative speed within all the correctly classified points.
On the naturally pretrained model, most of the points have margins less than 4255 (the first bin) and are considered more vulnerable.
Among these points, DyART reduces the proportion of the negative speed from 29 .
2% to 15 .
3% when comparing with AT.
Therefore, a higher percentage of the margins of vulnerable points will increase using DyART .
On the adversarially pretrained model, DyART reduces the proportion of negative speed values in the first three margin intervals and therefore has better dynamics of the decision boundary.
We conclude that compared with AT, DyART leads to better dynamics of the decision boundary where increasing smaller margins is prioritized.
C ONCLUSIONS AND D ISCUSSIONS
This paper takes one more step towards understanding adversarial training by proposing a framework for studying the dynamics of the decision boundary.
The phenomenon of conflicting dynamics is revealed, where the movement of decision boundary causes the margins of many vulnerable points to decrease and harms their robustness.
To alleviate the conflicting dynamics, we propose DynamicsAware Robust Training ( DyART ) which prioritizes moving the decision boundary away from more vulnerable points and increasing their margins.
Experiments on CIFAR-10 and Tiny-ImageNet demonstrate that DyART achieves improved robustness under various perturbation bounds.
Future work includes (a) theoretical understanding of the dynamics of adversarial training; (b) developing more efficient numerical methods to find the closest boundary points for robust training.  R EPRODUCIBILITY S TATEMENT
We are working towards providing an open-sourced implementation of our method as well as the trained models in the near future.
We will make the links available for the camera-ready version of this work. Supplementary Material A A DDITIONAL R ELATED W ORK Decision boundary analysis In this paper
, we mathematically characterizes the dynamics of decision boundaries and provide methods to directly compute and control the dynamics.
Prior to this work, there are also some interesting studies on the dynamics of margins, though from different perspectives.
Rade & Moosavi-Dezfooli (2022) point out that adversarial training leads to a superfluous increase in the margin along the adversarial directions, which can be a reason behind the trade-off between accuracy and robustness.
Ortiz-Jimenez et al.
(2020) investigate the relationship between data features and decision boundaries, and reveal several properties of CNNs and adversarial training.
Their results show that adversarial training exploits the sensitivity and invariance of models to improve the robustness.
Tramèr et al.
(2020) studies invariance-based adversarial examples and expose a fundamental trade-off between commonly used sesitivity-based adversarial examples and the invariance-based ones, where the behaviors of decision boundaries are identified.
Other Approaches to Improve Adversarial Training.
Recent works (Najafi et al., 2019; Rebuffi et al., 2021; Gowal et al., 2021; 2020) have shown that the robust accuracy of adversarial training can be improved significantly with additional data from unlabeled datasets, data augmentation techniques and generative models.
These approaches enhance the robustness of models by augmenting the dataset, which is orthogonal to our proposed algorithm that focus on how to optimize the model with the original dataset.
Wu et al.
(2020) show that model robustness is related to the flatness of weight loss landscape, which is implicitly achieved by commonly used adversarial learning techniques.
Based on this insight, the authors propose to explicitly regularize the flatness of the weight loss landscape, which can improve the robust accuracy of existing adversarial training methods.
Cui et al.
(2021) propose to use logits from a clean model to guide the learning of a robust model, which leads to both high natural accuracy and strong robustness.
We note that our method focuses on a different perspective of adversarial training, i.e., dynamics of decision boundary, and can be combined with these techniques to further improve the robust accuracy of the model.
The investigation of such combination is out of the scope of this paper, and will be addressed in our future work.
Certifiable Robustness.
There is an important line of work studying guaranteed robustness of neural networks.
For example, convex relaxation of neural networks (Gowal et al., 2019;
Zhang et al., 2018; Wong & Kolter, 2018;
Zhang et al., 2020a; Gowal et al., 2018) bounds the output of a network while the input data is perturbed within an ℓ p norm ball.
Randomized smoothing (Cohen et al., 2019) is another certifiable defense which adds Gaussian noise to the input during test time.
Croce et al.
(2019) propose a provably robust regularization for ReLU networks that maximizes the linear regions of the classifier and the distance to the decision boundary.
Note that certifiable robust radius is a strict lower bound of the margin, which is the focus of our work.
B P ROOF OF THE CLOSED - FORM EXPRESSION FOR THE SPEED IN EQUATIONAND THE MARGIN GRADIENT IN EQUATION
In this section, our goal is to prove the closed-form expression equation 4 as well as the margin gradient in equation 6 and provide further discussions.
We first provide two preliminary lemmas and present the mathematical assumptions.
Then we rigorously derive the closed-form expressions.
Finally, we discuss more about the expression and its assumptions.
Lemma 6.
For 1 ≤ p ≤ ∞ and let q satisfies 1 /q + 1 /p = 1 .
Let a be any fixed vector.
Then
[Equation]
Proof.
Without loss of generality, assume a is the zero vector.
Write the k -th component of x as x k .
Case 1: 1 ≤ p < ∞
By calculation, ∂ ∥ x ∥ p ∂x k = ( | x k | ∥ x ∥ p ) p − 1 · sign( x k ) .
Since q = pp − 1 , we have that
[Equation]
Therefore,
[Equation]
Case 2: p = ∞
In this case, ∇ x ∥ x ∥ ∞ is a one-hot vector (with the one being the position of the element of x with the largest absolute value).
Therefore, ∥∇ x ∥ x ∥ ∞ ∥ 1 = 1.
The following lemma deals with the optimality condition for p = ∞ .
Special care needs to be taken since L ∞ norm is not a differentiable function.
Lemma 7.
Let ˆ x be a local optimum of the constrained optimization problem:
[Equation]
where a is any fixed vector with ϕ ( a ) > 0 .
Assume that ϕ is differentiable at point ˆ x .
Denote the coordinates set J =
{ j : | ˆ x j − a j | =
∥ ˆ x − a ∥ ∞ } .
Denote the k -th component of ∇ x ϕ (ˆ x ) as ∇ x ϕ (ˆ x ) k .
Then (a) for j ∈ J , ∇ x ϕ (ˆ x ) j and ˆ x j − a j have opposite signs; (b) for k / ∈ J , ∇ x ϕ (ˆ x ) k = 0 .
Remark.
If ϕ ( a ) < 0 , then (a) for j ∈ J , ∇ x ϕ (ˆ x ) j and ˆ x j − a j have the same sign; (b) for k / ∈ J , ∇ x ϕ (ˆ x ) k = 0 .
Proof.
(a) Consider the perturbation ˆ x ( ϵ )
=
ˆ x + (0 , · · · , ϵ j 1 , · · · , ϵ j m , · · · , 0) where
J = { j 1 , · · · , j m } and ϵ is a m dimensional vector with j -th component ϵ j .
Since ϕ ( a ) >
and ˆ x is a local optimum, ∥ x − a ∥ ∞ <
∥ ˆ x − a ∥ ∞ imply ϕ ( x ) > 0 if x is sufficiently close to ˆ x .
Therefore, if every ϵ j i is chosen so that | ˆ x j i + ϵ j i − a j i | < | ˆ x j i − a j i | (that is, ϵ j i has different sign from ˆ x j i − a j i ) and ∥ ϵ ∥ sufficiently small, then
∥ ˆ x ( ϵ ) − a ∥ <
∥ ˆ x − a ∥ and thus ϕ (ˆ x ( ϵ )) > 0 .
On the other hand, by Taylor expansion and the fact that ϕ (ˆ x ) = 0 , we have that
[Equation]
Therefore, (cid:80) j ∈J ∇ ϕ (ˆ x ) j ϵ j > 0 for any such ϵ .
By taking other ϵ k → 0 if necessary, we obtain that ∀ j ∈ J , ∇ ϕ
(ˆ x ) j ϵ j ≥ 0 , where ϵ j has different sign from ˆ x j − a j .
Therefore, ∇ ϕ (ˆ x ) j and ˆ x j − a j have different signs.
(b) Take any k / ∈ J and consider the perturbation ˆ x ( ϵ )
= ˆ x + (0 , · · · , ϵ j 1 , · · · , ϵ k , · · · , ϵ j k , · · · , 0) where
ϵ =
( ϵ j 1 , · · · , ϵ k , · · · , ϵ j k ) .
Choose any ϵ so that ∥ ϵ ∥ is sufficiently small, each ϵ j i has the opposite sign of ˆ x j i − a j i and ϵ k small enough (which can be positive or negative), we have that ϕ
(ˆ x ( ϵ )) > 0 since ∥ ˆ x ( ϵ ) − a ∥ ∞ <
∥ ˆ x − a ∥ ∞ .
By Taylor expansion, (cid:80) j ∈J ∇ ϕ (ˆ x ) j ϵ j + ϵ k ∇ x ϕ (ˆ x ) k > 0 for any such ϵ .
By taking ϵ j → 0 and using the fact that ϵ k can be positive or negative, we conclude that ∇ x ϕ (ˆ x ) k
= 0 .
Now we are ready to derive the closed-form expression of the speed.
We first provide the full assumptions, then derive the expression, and finally we will discuss more about the assumptions.
We will write ˆ x i ( t ) as ˆ x i when the indication is clear.
Assumption 8.
Suppose that ( x i , y i ) is correctly classified by f θ ( t ) in some time interval t ∈
I and ˆ x i ( t ) is a locally closest boundary point in the sense that for any t ∈ I , it is the local optimum of the following:
[Equation]
Assume that in the time interval I : (a) ˆ x i ( t ) is differentiable in t ; (b) ϕ
y i is differentiable at point ˆ x i ( t ) and at the current parameter θ ( t ) .
Proposition (Closed-form expression of the speed s ( x i , t ) ) .
For 1 ≤ p ≤ ∞ and under Assumption 8, define the (local) speed according to ˆ x i ( t ) in Assumption 8 as s ( x i , t ) = ddt ∥ ˆ x i ( t ) − x i ∥ p , we have the following:
[Equation]
where q satisfies that 1 /q + 1 /p = 1 .
In particular, q = 2 when p
= 2 and q = 1 when p = ∞ .
Proof.
Case 1: 1 ≤ p < ∞
To compute s ( x i , t ) = ddt ∥ ˆ x i ( t ) − x i ∥ 2 , we need to characterize the curve of the closest boundary point ˆ x i ( t ) , where two key points stand out.
First, ˆ x i ( t ) is on the decision boundary Γ y ( t ) and thus ϕ y (ˆ x i ( t ) , θ ( t ))
=
for all t ∈ I .
By taking the time derivative on both sides, we obtain the level set equation (Osher et al., 2004; Aghasi et al., 2011)
[Equation]
Second, ˆ x i ( t ) is the optimal solution of constrained optimization equation 3.
Therefore, we have the following optimality condition:
[Equation]
Since x i is correctly classified, ϕ y i ( x i ) > 0 .
Since ˆ x i ( t ) is the closest point to x i whose ϕ
y i value is zero, λ ( t ) > 0 .
By taking the L q norm in Equation equation 11 and using Lemma 6, we obtain that λ ( t ) =
∥∇ x ϕ
y i
(ˆ x i ( t ) , θ ( t ))
∥ q .
Now, we derive s ( x i , t ) as follows:
[Equation]
By the optimality condition equation 11) [Equation] (By the level set equation equation 10)
[Equation]
Case 2: p = ∞
Note that since L ∞ is not differentiable, the optimality condition in Equation equation 11 does not hold anymore.
Denote the j -th component of ˆ x i ( t ) and x i as ˆ x ij ( t ) and x ij .
Let J =
{ j : | ˆ x ij ( t ) − x ij | =
∥ ˆ x i ( t ) − x i ∥ ∞ } .
By Lemma 7, s ( x i , t ) = ddt | ˆ x ij ( t ) − x ij | =
ˆ x ′ ij ( t ) sign(ˆ x ij ( t ) − x ij ) =
− ˆ x ′ ij ( t ) sign( ∇ x ϕ y
i
(ˆ x i ) j ) for all j ∈ J .
Therefore, by Equation equation 10 and Lemma
[Equation]
Therefore,
[Equation]
, where the last equality follows from Lemma 7 that the components of ∇ x ϕ y
i
(ˆ x i ) are zeros if they are not in J .
As an corollary of the proposition we prove above, we can obtain the closed-form expression for the gradient of margin (or the gradient of any smooth function of the margin) as follows:
Theorem (Closed-form expression of ∇ θ h ( R θ ( x i )) ) .
For 1 ≤ p ≤ ∞ ,
[Equation]
where q satisfies that 1 /q + 1 /p = 1 .
Proof.
In continuous time we consider h ( R ( x i , t ))
(or more rigorously, h ( R ( x i , θ ( t ))) ) and its time derivative.
We use the following relationship between the gradient and the time derivative, where θ ′ ( t ) can be any update rule:
[Equation]
On the other hand:
[Equation]
where the last equality uses the closed-form expression for the speed s ( x i , t ) .
Therefore we have that for any θ ′ ( t ) , ∇ θ h
( R ( x i , t [Equation] ) · θ ′ ( t [Equation]
=
[Equation]
· θ ′ ( t ) .
We conclude that ∇ θ h ( R θ ( x i
[Equation]
=
[Equation]
D ISCUSSIONS ON THE ASSUMPTIONS
Assumption 8 has several points that need to be explained further.
First, we only require that ˆ x i ( t ) is a local closest boundary point.
This is important because in practice when an algorithm for searching the closest boundary point is used (e.g. FAB), a local solution is the best one can hope for due to the non-convex nature of the optimization problem.
When ˆ x i ( t ) is a local solution, the speed should be interpreted as how fast the distance changes around that local solution.
In this case, although ˆ x i is not the globally closest adversarial example, the local speed around ˆ x i still has much information on the relative movement of the decision boundary w.r.t.
x i , especially when the distance ∥
ˆ x i − x i ∥ is relatively small and the input space is a high-dimensional space (e.g. pixel space).
Second, we require that ˆ x i ( t ) is a differentiable curve in time interval I .
Note that if we constrain ˆ x i ( t ) to be the global closest boundary point, ˆ x i is unlikely to be differential for a large time interval
I , especially in high dimensional space.
This is because as the decision boundary moves.
the closest point might switch from one point to another point that is very far away abruptly.
Fortunately, this problem is alleviated because our derived closed-form expression still works when ˆ x i ( t ) is a locally closest boundary point.
Note that however, due to the topological change of the decision boundary, it still can happen that ˆ x i ( t ) stops existing (and thus stops being differentiable) at some time point, when typically the speed will go to infinity.
From a mathematical point of view, this is related to shock in partial differential equation (PDE) theories.
More exploration on this is left to future work.
In this work, we only consider the speed of the decision boundary at each discrete time step.
C C OMPUTATION OF THE EXACT AND SOFT CLOSEST BOUNDARY POINT
Either computing the speed of the decision boundary or using DyART to directly optimize a function of margins requires the computation of the closest boundary point
ˆ x (or the closest soft boundary point
ˆ x soft ), where we omit the subscript i in this section.
As discussed in Appendix B, it suffices to find the locally closest (soft) boundary point in order for the closed-form expression 4 and expressionfor the speed and the gradient of margin to be valid.
C. C LOSEST BOUNDARY POINT
In this section, we will explain how to check the quality of the found ˆ x for the constrained optimization problem 3 in practice.
We will also give a simple analysis on how FAB(Croce & Hein, 2020a), the algorithm we use in our implementation, solves the problem 3 in practice.
We include both p =and p = ∞ although in our work, only p = ∞ is used.
We discuss both of them in order to highlight the difference in checking optimality conditions for smooth ( p = 2 ) and non-smooth norm ( p = ∞ ).
The key points of analyzing ˆ x are that ϕ y (ˆ x )
=
and the KKT conditions of problem 3.
Case 1: p =
In this case, the KKT condition is given by ∇ x ϕ y (ˆ x ) + λ (ˆ x − x ) = 0 for some λ > 0 (since ϕ y ( x ) > 0 ).
In other words, ∇ x ϕ y (ˆ x ) ∥∇ x ϕ y (ˆ x ) ∥ · x − ˆ x ∥ x − ˆ x ∥ = 1 .
In practice, we check the following two conditions (a) | ϕ
(ˆ x ) | ≤ 0 .
1 ; (b) ∇ x ϕ y (ˆ x ) ∥∇ x ϕ y
(ˆ x ) ∥ · x − ˆ x ∥ x − ˆ x ∥ > 0 . 8 .
We observe in our experiments that FAB can find high-quality closest boundary points for over 90% of the correctly classified data points.
Case 2: p = ∞
In this case, we consider the optimality condition given in Lemma 7 of Appendix B. Denote # B the number of points in a set B .
Using the notation J =
{ j : | ˆ x j − x j | =
∥ ˆ x − x ∥ ∞ } and J C the complement set of J , we check the following conditions in practice: (a) | ϕ
(ˆ x ) | ≤ 0 .
1 ; (b) # { j ∈J : ∇ x ϕ y (ˆ x ) j
(ˆ x j − x j ) ≤ 0 } # J > 0 .
9 ; (c) # { k/ ∈ J :
|∇ x ϕ y
(ˆ x ) k | < 0 . 1 } # J C > 0 . 8 .
Note the unlike p = 2 , the optimality conditions for p = ∞ are on each coordinate of ˆ x , which is more difficult to satisfy in practice.
We observe in our experiments that FAB with 100 iterations can find high-quality closest boundary points for about 85% of the correctly classified points.
However, when only 20 iterations are used, condition (3) is barely satisfied for all of the found boundary points (the first two conditions are still satisfied).
In our visualizations of dynamics of the decision boundary for AT in Section 4.2, we useiterations for FAB and only use high-quality closest boundary points, so that the visualization results are relatively accurate.
C. C LOSEST SOFT BOUNDARY POINT
Adapt FAB for soft decision boundary.
In DyART , the closest point ˆ x soft on the soft decision boundary is used.
To find ˆ x soft , we adapt the FAB method.
The original FAB method aims to find the closest point on the exact decision boundary.
In particular, FAB forms linear approximations for decision boundary between the ground truth class and every other classes.
The only adaptation we do on the FAB method is that now FAB only forms one linear approximation for the soft decision boundary of the ground truth class.
This is because we use the smoothed max operator in the soft logit margin, and there is no concept of the ’decision boundary between the ground truth class and another class’ anymore.
Computational efficiency.
By using the soft decision boundary, every iteration of FAB only requires one linear approximation of the soft decision boundary, which cost one back-propagation.
In contrast, the original FAB which aims to find the closest boundary point on the exact decision boundary costs K back-propagation at each iteration, where K is the number of classes.
Therefore, using the soft decision boundary is more efficient and is used in our proposed robust training method
DyART .
Local optimality condition.
The procedure of checking optimality condition is similar to the one in the last section.
Denote # B the number of points in a set B .
Using the notation J =
{ j : | ˆ x j − x j | =
∥ ˆ x − x ∥ ∞ } and J C the complement set of J , we check the following conditions in practice: (a) | ϕ
(ˆ x ) | ≤ 0 .
1 ; (b) # { j ∈J : ∇ x ϕ y (ˆ x ) j
(ˆ x j − x j ) ≤ 0 } # J > 0 .
9 ; We find that when the temperature β is relatively large (we use β = 5 in all of our experiments) and 20 iterations is used, 95% of the soft boundary point found for the correctly classified points satisfy these two conditions.
During training, we only use these higher quality points and discard the rest of the boundary points that do not satisfy these two conditions.
Note that we do not consider the third condition (c) # { k/ ∈ J : |∇ x ϕ y
(ˆ x ) k | < 0 . 1 } # J C > 0 . 8 .
This is because condition (c) cannot be satisfied unless a very large iteration number is used, which is computationally prohibitive for robust training.
Experimentally DyART achieves improved robustness over baseline methods, indicating that the closest soft boundary points used by DyART are indeed useful for robust training.
Designing faster and more reliable methods to solve the constrained optimization problem 3 is left for future work.
D E XPERIMENTS
In this section, we provide the details of experimental settings and further results of DyART using various choices of hyperparameters.
We also provide further analysis on the decision boundary dynamics.
D. D ETAILED EXPERIMENTAL SETTINGS
Architectures and training settings.
In all experiments on the CIFAR-10 dataset, we use the Wide
Residual Network (Zagoruyko & Komodakis, 2016) with depth 28 and width factor 10 (WRN-2810) with Swish activation function (Ramachandran et al., 2017).
On the Tiny-ImageNet dataset, we use pre-activation ResNet-18 (He et al., 2016b).
For both architectures, we replace the batch normalization layers with group normalization layers so that the decision boundaries are the same during training and evaluation.
In all experiments, we use stochastic weight averaging (Izmailov et al., 2018) with a decay rate of 0 .
995 as in prior work (Gowal et al., 2020;
Chen et al., 2020).
All models are trained using stochastic gradient descent with momentum 0.9 and weight decay 0.
for 100 epochs with batch size 256 .
We use a cosine learning rate schedule (Loshchilov & Hutter, 2016) without restarts where the initial learning rate is set to 0 .
1 for baselines in all experiments.
For DyART on Tiny-ImageNet, we use the cosine learning rate schedule with initial learning rate 0 .and on CIFAR-10, the learning rate begins at 0.1 and is decayed by a factor of 10 at the 50th and 75th epoch.
To alleviate robust overfitting (Rice et al., 2020), we compute the robust and clean accuracy at every epoch on a validation set of size 1024 using projected gradient descent (PGD) attacks withsteps using margin loss function.
All experiments are run on two NVIDIA GeForce RTX {+
+} Ti GPU.
Compared baselines and their hyperparameters.
In all experiments we consider the ℓ ∞ perturbation setting.
On CIFAR-10, the baseline defense methods include: (1) standard adversarial training (AT) (Madry et al., 2017) which trains on the worst case adversarial examples generated by 10-step
PGD (PGD-10) on the cross-entropy loss.
The perturbation bound is 8 255 and the step size of PGD is 2255 ; the training setting follows Rice et al.
1 (Rice et al., 2020).
(2) TRADES 2 (Zhang et al., 2019) which trades off between the clean and robust accuracy.
The perturbation bound is 8255 with the step size of PGD-10 0 .
The regularization constant beta (or 1/lambda) is set to 6.
(3) MMA(Ding et al., 2020) which trains on the closest adversarial examples (closest boundary points) with uniform weights.
The MaxEps is set to 32255 .
(4) GAIRAT 4 (Zhang et al., 2021) which reweights adversarial examples using the least perturbation steps.
The perturbation bound is 8255 with step size 0 .
007 using PGD-10 and the ’tanh’ weight assignment function is used.
(5) MAIL 5 (Liu et al., 2021) which reweights adversarial examples using margin value.
We choose its combination with
TRADES (MAIL-TRADES) which provides better robustness performance than combining with
AT (MAIL-AT).
Its hyperparamters beta, bias and slope are set to 5 . 0 , − 1 . 5 and 1 . 0 , respectively.
(6) FAT 6 (Zhang et al., 2020b) that exploits friendly adversarial data, where the perturbation bound is set to 8255 .
(7) MART (Wang et al., 2019) which explicitly differentiates the mis-classified and correctly classified examples.
On Tiny-ImageNet, we compare with AT, TRADES, and MART whose hyperparameter settings are available for this dataset.
We follows the PyTorch implementation of (Gowal et al., 2020) 7 for the Tiny-ImageNet dataset.
Evaluation details.
We evaluate DyART and the baselines under ℓ ∞ norm constrained perturbations.
The final robust accuracy is reported on AutoAttack (AA) (Croce & Hein, 2020b), which uses an ensemble of selected strong attacks.
For all methods, we choose the hyperparameters to achieve the best robust accuracy under the commonly used perturbation bound ϵ = 8255 .
To fully compare the robustness performance among different methods, we report the robust accuracy under four additional perturbation bounds: 2255 , 4255 , 12255 and 16255 .
Per-sample gradient For computing the speed of the decision boundary in Section 4.2 and Section 6.2, we need to compute the per-sample gradient ∇ θ ϕ
y i (ˆ x i , θ ) for every correctly classified point x i .
We use the Opacus package (Yousefpour et al., 2021) for computing per-sample gradients in parallel.
Also, another reason why we replace BN with GN is because Opacus does not support BN for computing per-sample gradients.
Although using this package will increase the memory usage, it is worth mentioning that during robust training, DyART does not need to compute the per-sample gradient and thus does not have the excessive memory issue.
Per-sample gradients are only collected for computing speed, which is for interpretation of dynamics of different methods and not for robust training.[-1 Robust Overfitting’s Github 2 TRADES’s Github 
 MMA’s Github 4 GAIRAT’s-]
D. H YPERPARAMETER S ENSITIVITY E XPERIMENTS
Hyperparameters of DyART In this paper, we use the cost function of the form h
( R ) =
α exp( − αR ) when R < r 0 and h ( R ) = 0 otherwise.
We present results under different decay strengh α > 0 , margin threshold r 0 as well as regularization constant λ for the robustness loss.
Performance results.
The evaluation results on CIFAR-10 and Tiny-ImageNet are shown in Table 3 and Table 4, respectively.
We analyze the effects of hyperparameters as follows.
(1) Effect of α : Larger α corresponds to a cost function h ( · ) that decays faster, and therefore prioritize improvement on even smaller margins.
Therefore, it should be expected that larger α leads to higher clean accuracy and higher robust accuracy under smaller perturbation sizes, and results in lower robust accuracy under larger perturbation sizes.
For example, on CIFAR-10, when α =is increased to α = 8 when r 0 = 16255 , λ = 400 , the clean accuracy as well as the robust accuracy under ϵ = 2255 , 4255 and 8255 improves, while the robust accuracy under larger ϵ gets lower.
The same patterns can also be observed on Tiny-ImageNet, for example, when α = 8 is increased to α =when r 0 = 20255 , λ = 1000 . {+1 Robust Overfitting’s Github 2 TRADES’s Github 
 MMA’s+}
Github 4 GAIRAT’s
Github 5 MAIL’s github 6 FAT’s github 7 UncoveringATLimits’s
Github (2) Effect of r 0 : r 0 is from preventing DyART from training boundary points that are too far away from clean data points.
Therefore, it should be expected that training on smaller r 0 tends to increase the clean accuracy and the robust accuracy under relatively small perturbation sizes.
Indeed, on
Tiny-ImageNet, when r 0 = 24255 is decreased to r 0 = 20255 when α = 10 and λ = 1000 , we can observe that the clean accuracy as well as robust accuracy under ϵ = 2255 increases but the robust accuracy under larger perturbation sizes ϵ = 12255 and 16255 decreases.
(3) : Effect of robust loss constant λ : A larger λ tends to increase the robustness of the model (in particular, the robust accuracy under relatively larger perturbation sizes) while decrease the clean accuracy and the robust accuracy under relatively small perturbation sizes.
For example, on TinyImageNet, when λ = 800 is increased to λ = 1000 when α = 10 and r 0 = 20255 , the clean accuracy and robust accuracy under relatively small ϵ = 2255 , 4255 drops but the robust accuracy under larger perturbation sizes increase. [Table]
[Table]
(4) Effect of the burn-in period: A burn-in period of natural training is necessary for DyART since its robust loss function depends on the closest boundary points, which can only be found on correctly classified points.
That is, DyART requires a descent initial clean accuracy.
In our experiments, we find that the learning rate of the burn-in period is important: DyART will train successfully if the learning rate of the burn-in period is relatively large (e.g. 0.1 for CIFAR-10 and Tiny-ImageNet).
However, when the learning rate is small (such as 0 . 001 ), DyART sometimes drives the clean accuracy to be very low at first, and fails to train.
Our suggestion is to use a larger learning rate to obtain a naturally pretrained model.
D. F URTHER ANALYSIS ON D ECISION BOUNDARY DYNAMICS
In Section 4.2 and Section 6.2 we have presented the dynamics of both AT and DyART on the same pretrained models using the same batch of data at one iteration.
In this section, we demonstrate the dynamics of the decision boundary throughout the whole training process.
Experiment setting.
{+To study the decision boundary dynamics throughout the training process, we train a ResNet-
+}
(He et al., 2016a) with group normalization (GN) (Wu & He, 2018) on CIFARusing (1) Adversarial Training with 10-step PGD under ℓ ∞ perturbation with ϵ = 8255 from scratch; (2) DyART with α = 8 , λ = 400 , r 0 = 16255 from a naturally pretrained model.
The models are trained with a initial learning rate of 0 .
01 and the learning rate decays to 0 .
001 at 20000 iteration.
At each iteration, we compute the proportion of negative speed among points with margins smaller than 8255 that are regarded as vulnerable.
Conflicting dynamics throughout training In Figure 6, the clean and robust accuracy of both methods are shown, along with the proportion of negative speed among vulnerable points.
We apply curve smoothing for negative speed proportion plot for better visualization.
Note that we omit the initial part of training (first 10 epochs) since at this initial stage, there are not enough correctly classified data points but speed and margin are only defined for these points.
We can see that both methods exhibit some degree of robust overfitting, where the training robust accuracy becomes larger than the test robust accuracy.
In addition, the conflicting dynamics exists throughout the whole training process, since the proportion of negative speed is never zero.
We can see that DyART consistently has less conflicting dynamics than AT.
Interestingly, the proportion of negative speed decreases over time during training for both methods.
The connection between the decreasing degree of conflicting dynamics on the training data and the robust overfitting phenomenon is left for future research.
[Figure]