Year: 2021

Abstract:
While adversarial training has become the de facto approach for training robust classifiers, it leads to a drop in accuracy. This has led to prior works postulating that accuracy is inherently at odds with robustness. Yet, the phenomenon remains inexplicable. In this paper, we closely examine the changes induced in the decision boundary of a deep network during adversarial training. We find that adversarial training leads to unwarranted increase in the margin along certain adversarial directions, thereby hurting accuracy. Motivated by this observation, we present a novel algorithm, called Helper-based Adversarial Training (HAT), to reduce this effect by incorporating additional wrongly labelled examples during training. Our proposed method provides a notable improvement in accuracy without compromising robustness. It achieves a better trade-off between accuracy and robustness in comparison to existing defenses.

text: 1 INTRODUCTION
Despite the remarkable success of modern deep networks on computer vision tasks, they are notoriously fragile to small, imperceptible changes in the input data (Szegedy et al., 2014). Ever since the introduction of `p-norm based adversaries (Goodfellow et al., 2015; Moosavi-Dezfooli et al., 2016), a growing body of researchers has focused on developing solutions for improving the robustness of DNNs. At present, adversarial training (Madry et al., 2018) remains the most effective approach for robustification. Nevertheless, it is known to cause an undesirable reduction in accuracy, thus leading to the much-debated trade-off between accuracy and robustness (Tsipras et al., 2019; Yang et al., 2020). Subsequently, in an endeavour to alleviate the accuracy vs. robustness trade-off, several refinements have been proposed to the vanilla adversarial training, e.g., Zhang et al. (2019); Wang et al. (2020); Rice et al. (2020); Wu et al. (2020); Zhang et al. (2021). However, as shown in Fig. 1, the progress has been rather incremental over the
last few years. Notwithstanding these recent advances, closing the large gap between accuracy and robustness still remains an open challenge and every modest improvement in this regard remains significant for the practitioners.
In this paper, we take a step towards demystifying and improving the aforementioned trade-off. To this end, we take a closer look at the effect of training with adversarial examples on the geometry of decision boundaries learnt by deep networks. Specifically, relying on the perspective proposed by Ortiz-Jimenez et al. (2020) which relates the directional margin with the discriminative features used by a network, we uncover an unwanted consequence of adversarial training. That is, we identify
that adversarial training leads to a superfluous increase in the margin along the adversarial directions of the input space computed for a regularly trained network. We refer to these directions as initial
adversarial directions1. In other words, the network becomes excessively robust (compared to the attack radius ε) along the directions which are otherwise pivotal for regular networks, and hence the accuracy drops. Further, we establish an empirical connection between the margin along initial adversarial directions and the classification performance of the network. In particular, a smaller margin along these directions corresponds to a higher accuracy. Thus, while the excessive margin is not essential for robustness, it severely derails the performance on clean samples. This naturally raises the following question:
Can we foresee an improvement in accuracy by reducing the margin along initial adversarial directions while preserving the same level of robustness?
To address this question, we propose a heuristic adversarial training scheme to reduce the directional margin and thus, achieve a better accuracy without sacrificing robustness. We call this algorithm, Helper-based Adversarial Training (HAT), the name derived from the fact that we incorporate additional training examples comprising of overly perturbed adversarial images (possibly) wrongly labelled using a standard trained network to help impede excessive directional robustness and hence, improve clean accuracy (see Fig. 2). We also provide an extensive analysis of HAT and compare it with state-of-the-art AT methods. The main contributions of this paper include:
• We demonstrate that adversarial training leads to a large margin along initial adversarial directions. In addition, we identify a direct connection between the excessive margin and the reduction in clean accuracy caused by adversarial training.
• We propose Helper-based Adversarial Training (HAT) to explicitly reduce the excessive margin by augmenting the training data with properly-labelled helper examples.
• Experimentally, we show that HAT consistently improves accuracy without sacrificing robustness. Notably, in combination with recent improvements from Rebuffi et al. (2021), HAT sets a new state-of-the-art and improves clean accuracy by 3.33% on CIFAR-10 and 4.63% on CIFAR-100 using PreAct ResNet-18 against `∞ perturbations of size 8/255.

heading: 1.1 RELATED WORK
text: Improving adversarial robustness. To mitigate the adversarial susceptibility of DNNs, a myriad of defense methodologies (Papernot et al., 2016; Tramèr et al., 2018; Kannan et al., 2018; Wang et al., 2019; Xiao et al., 2020) have been proposed. Yet, many of them have been shown to provide superficial robustness by introducing gradient obfuscation (Athalye et al., 2018), hence calling for a cautious evaluation of robustness (Tramer et al., 2020; Croce et al., 2020). Nonetheless, adversarial training (AT) (Madry et al., 2018) or training with worst-case inputs has been the most popular approach for improving the robustness of DNNs albeit at the cost of a sizeable drop in accuracy. Notably, AT has been able to withstand newer sophisticated adversaries (Gowal et al., 2019; Croce & Hein, 2020). Another prominent defense is TRADES (Zhang et al., 2019), which provides a systematic way to control the trade-off between accuracy and robustness by solving:
argmin θ n∑ i=1 CE(yi, fθ(xi)) + β max δi:||δi||p≤ε KL(fθ(xi), fθ(xi + δi)), (1)
1The reason for such terminology will become clear in the later sections.
where CE is the cross-entropy loss, KL is the Kullback-Leibler divergence and β controls the trade-off between accuracy and robustness. More recent attempts to improve the accuracy vs. robustness trade-off have predominantly dealt with refining adversarial training by (i) modifying the objective function (Wang et al., 2020; Zhang et al., 2020; 2021), (ii) early-stopping (Rice et al., 2020), (iii) heuristics and hyperparameter selection (Pang et al., 2021; Gowal et al., 2021), (iv) extra data (Hendrycks et al., 2019; Alayrac et al., 2019; Carmon et al., 2019; Rebuffi et al., 2021; Sehwag et al., 2021), and (v) weight initialization for training with large perturbations (Shaeiri et al., 2020). Despite these advancements, the problem is far from being solved with a huge gap between accuracy and robustness on practical image recognition benchmarks. We revisit this issue in our paper and build upon TRADES to further ameliorate the trade-off.
Understanding adversarial vulnerability. A concurrent stream of prior works has provided plausible reasons for the existence of adversarial examples. Adversarial vulnerability of DNNs has been accredited to the existence of highly generalizable, but non-robust features in the data which are exploited by DNNs to achieve state-of-the-art performance (Jetley et al., 2018; Ilyas et al., 2019). Besides, Ortiz-Jimenez et al. (2020) devised a novel experimental framework which relates the distance of data points to the decision boundary of a DNN with the discriminative features used by the DNN in order to understand the mechanism behind AT.

heading: 2 PRELIMINARIES
text: Consider the input space X ⊆ Rd. Let fθ : X → RC represent a deep neural network classifier parameterized by θ, where C is the number of output classes. Let Fθ(x) = argmaxk fθ(x)k be the class label predicted by fθ for any x ∈ X , where fθ(x)k is the kth component of fθ(x). Margin. Given a classifier Fθ, input x and an unit vector r̂ ∈ Sd−1, we define margin µ at x along the direction r̂ as:
µ(x, r̂) = argmin α
|α| s.t. Fθ(x+ αr̂) 6= Fθ(x) (2)
Additionally, note that we refer to a deep network trained only on clean samples as standard network and a network trained via adversarial training as robust network. Besides, we reuse the definitions of clean (natural) and robust (adversarial) accuracy as stated by Zhang et al. (2019).
Initial adversarial directions. Given a standard network fθ, input dataset {(xi, yi)}ni=1, we define the set of initial adversarial directions asRinit = {ri/||ri||2}ni=1 where ri is obtained by solving:
ri = max δi:||δi||p≤ε `(yi, fθ(xi + δi)). (3)
where, `(·, ·) is an arbitrary loss function e.g. cross-entropy (CE). This optimization is usually solved via a multi-step procedure called projected gradient descent (PGD) (Madry et al., 2018).

heading: 3 ADVERSARIAL TRAINING INTRODUCES EXCESSIVE INVARIANCE
text: We begin our analysis by examining the effect of adversarial training (AT) on the decision boundary of DNNs. Via novel analysis on a synthetic dataset and CIFAR-10 (Krizhevsky, 2009), we show that AT triggers a superfluous increase in the margin along the initial adversarial directions as compared to the nominal increase required to attain robustness. In addition, we provide empirical evidence which signifies a connection between the increase in margin and reduction in clean accuracy.
Toy problem. First, we study a toy setting to shed some light on the phenomenon of excessive directional margin caused by AT. We construct a 3-d binary classification dataset drawn from two distributions which live on two noisy concentric circles of different radii in the x1-x2 plane and being linearly separable along the third dimension x3. In particular, x1 = ρi cos(z) + 1, x2 = ρi sin(z) + 2 and x3 ∼ U(αi, βi) where z ∼ U(0, 2π) and 1, 2 ∼ N (0, σ2) where i = 1, 2 for class 1 and 2 respectively. We train a single hidden-layer MLP via both standard training and adversarial training. Fig. 3 visualizes the decision regions with both the training procedures. It is
evident that the network primarily uses x3 to achieve zero classification error when trained using standard training, but the resulting model performs poorly in terms of robustness. In contrast, when we use AT, the learned decision boundary is completely different from that in the standard case. Here, the network becomes reasonably invariant along x3 (Fig. 3b), thus causing the directional margin along x3 to tend to∞. This enables the network to attain robustness at the cost of a small increase in classification error.
Evidence on CIFAR-10. Next, we illustrate that a similar phenomenon occurs in the case of stateof-the-art deep networks trained on CIFAR-10. To this end, we measure directional margin (as defined by Eq. 2) to the classification boundary which allows us to partly capture the geometry of decision boundary. We restrict ourselves to the following setting. We take a ResNet-18 (He et al., 2016) trained until convergence on CIFAR-10 (achieving 94.6% accuracy and 0% robustness to `∞- PGD perturbations on the test set) and then fine-tune it using AT with adversarial examples. This framework allows us to study the evolution of the decision boundary caused by AT in comparison to that learnt by a standard network. We use `∞-PGD with norm ε = 8/255 for training. The network attains 83.3% accuracy and 51.6% robustness on the test set after adversarial fine-tuning. Please refer to App. B.1 for more details of the experimental setup.
Meanwhile, during adversarial fine-tuning, we track the margins along the adversarial directions found by PGD to shed some light on the learning dynamics. SupposeRk = {rki /||rki ||2}, where rki denotes adversarial perturbation found by PGD at kth epoch of adversarial fine-tuning for the input sample xi. Thus, Rinit = R0 represents the set of initial adversarial directions. We hypothesize that during adversarial training, the network becomes excessively robust to these initial adversarial directions R0 while slightly shifting the decision boundaries along other adversarial directions Rk≥1. Fig. 4 illustrates the margins along R0, R1 and R5 before and after adversarial fine-tuning respectively. The dashed red line indicates the value of ε used for training, i.e., any allowable pertur-
bation r has ||r||∞ ≤ 8/255 or equivalently ||r||2 ≤ 1.74. Intuitively, one might expect adversarial fine-tuning to cause small shifts in the decision boundary so that the margin becomes greater than 1.74, and attain robustness. However, this is not the case in practice. Intriguingly, the classifier instead resorts to becoming largely insensitive along R0 for some data points as evident in Fig. 4 while undergoing small shifts along other directionsRk≥1. We also observe a decrease of 11.3% in clean accuracy after fine-tuning.
Connection between margin alongRinit and clean accuracy. We now provide a two-fold argument which justifies the following hypothesis: ”The drastic rise in the margin alongRinit is directly correlated to the observed reduction in accuracy”. In fact, a larger margin contributes to a larger drop in accuracy. (i) Firstly, we complement our hypothesis with the following observation by Ortiz-Jimenez et al. (2020). The directions of input space with small margins and in turn, the initial adversarial directions in the case of standard network, are associated with discriminative features learnt by the network. We believe that these directions are crucial for the performance of the network. Thus, a drastic directional margin along these
directions contributes to the drop in classification accuracy. (ii) We train a robust network on CIFAR10 using TRADES with different values for the trade-off parameter β. (Please refer to Sec. 5.1 for the exact experimental setup.) As β increases, we observe an increase in the margin along Rinit (computed on a subset of 1024 examples from the CIFAR-10 test set) and a corresponding reduction in clean accuracy (see Table 1). This further corroborates our hypothesis.

heading: 4 HELPER-BASED ADVERSARIAL TRAINING
text: As demonstrated in Sec. 3, AT triggers an unwarranted increase in the margin along initial adversarial directions, thus hindering the network from using highly discriminative features in those directions. Note that a large margin is not a sufficient condition for `p-based ε-robustness. A trivial example of this is a constant classifier which has an infinite margin, but has both poor robustness and poor accuracy. Instead, the excessive rise in margin translates to large changes in the geometry of decision boundary compared to a standard network and causes a corresponding drop in clean accuracy. We here base our algorithm on the intuition that slightly pushing the decision boundary away from data should suffice for attaining robustness as opposed to the large shifts observed. Our results in Sec. 5 suggest that HAT, by reducing margin, can indeed lead to improved performance.
Now, we introduce our proposed algorithm, helper-based adversarial training (HAT), to reduce the excessive directional margin observed in practice. We aim to preserve certain geometric properties of a standard trained network viz. the predictive power along adversarial directions while learning a robust model. To this end, we add additional training examples, called helper examples, which are generated on-the-fly during the adversarial training procedure. In particular, a helper example is constructed by extrapolating the adversarial perturbation found during training and is (possibly wrongly) labelled by a standard network (see Fig. 2). Formally, we define a helper example as:
Helper examples. Given an input sample (xi, yi), a standard network fθstd , a robust network iterate fθkrob at k
th training iteration and adversarial example x′i computed by the adversary ϕ for fθkrob , the corresponding helper example is given by (x̃i, ỹi) where
x̃i = xi + 2ri, ri = x′i − xi and ỹi = argmaxk fθstd(x′i)k
The motivation behind this definition is illustrated in Fig. 2 where it is evident that we can stimulate a slight push to the decision boundary along any adversarial direction by making the network predict the correct label yi at adversarial example x′i and have it predict ỹi (often ỹi 6= yi) at helper example x̃i to preserve the discriminative characteristics as modelled by a standard network. Defining the helper examples as x+2ri is a heuristic choice that provides a good compromise between the helper example being sufficiently dissimilar (from x) for the model to assign a different label (see Fig. 2 for a visual example), and not too dissimilar to affect the performance on other clean samples. In
Algorithm 1 Helper-based Adversarial Training Input: Training dataset D = {(xi, yi)}ni=1 Parameter: Batch size m, learning rate η, weight of robust loss β, weight of helper loss γ, attack radius ε, attack step size α and number of attack iterations K
1: Train a network fθstd via standard training on D i.e., θstd ← argminθ ∑n i=1 CE(yi, fθ(xi)) 2: Randomly initialize the network parameters θHAT 3: repeat . Train a robust classifier 4: Sample a mini-batch {(xij , yij )}mj=1 from D 5: for j = 1, 2, . . . ,m do 6: x′ij ← xij + 0.001 · N (0, I) . Construct adversarial example 7: for k = 1, 2, ...,K do 8: x′ij ← ∏ B(xij ,ε) (x′ij + α · sign(∇x′ij KL(fθHAT(xij ), fθHAT(x ′ ij ))))
9: end for 10: Compute helper example: x̃ij ← xij + 2 (x′ij − xij ) 11: Set helper label: ỹij ← argmaxk fθstd(x′ij )k 12: end for 13: θHAT ← θHAT − ηm · ∑m j=1∇θHAT ( CE ( yi, fθHAT(xij ) ) + β · KL(fθHAT(xij ), fθHAT(x′ij ))
+ γ · CE(ỹij , fθHAT(x̃ij )) )
14: until training completed
contrast to AT, this allows preventing the undesirable excessive rise in the margin to some extent, thus making it possible to achieve significantly better performance on clean samples. Put differently, HAT can also be framed as performing geometric self-distillation (Hinton et al., 2015) to mimic certain geometric properties of a standard trained network.
HAT algorithm. We choose to instantiate HAT by extending TRADES which alike TRADES allows us to balance the accuracy-robustness trade-off. Algorithm 1 summarizes the pseudo-code for HAT. The training objective for HAT comprises of three terms: standard loss, robust loss, and an additional helper loss. Thus, in comparison to TRADES, we have an additional parameter γ which controls the weight of helper loss and thus, the extent of resistance to excessive directional margin. Finally, note that our extension can be easily plugged into other recent techniques e.g. Wu et al. (2020); Gowal et al. (2021); Rebuffi et al. (2021). In fact, in our experiments, we indeed combine HAT with the approach in Rebuffi et al. (2021) to advance the prior art by a significant margin (Sec. 5.3).

heading: 5 EXPERIMENTS
text: This section constitutes an extensive evaluation of HAT. Initially, to test the universality of our approach, we study the performance of HAT with ResNets on different datasets and attack configurations. Next, we leverage extra data and wider networks to obtain state-of-the-art performance on conventional robustness benchmarks. Towards the end, we conduct experiments to analyze HAT.

heading: 5.1 PERFORMANCE EVALUATION WITH RESNETS
text: We here empirically validate the performance of HAT. We report results using ResNet-18 (He et al., 2016) on three datasets: CIFAR-10, CIFAR-100 (Krizhevsky, 2009) and SVHN (Netzer et al., 2011). We compare HAT with three prominent adversarial defenses: (i) AT (Madry et al., 2018), (ii) TRADES (Zhang et al., 2019) and (iii) MART (Wang et al., 2020).
Training setup. We borrow the set of training hyperparameters from DAWNBench (Coleman et al., 2017). Precisely, we use SGD optimizer with Nesterov momentum (Nesterov, 1983); cyclic learning rates (Smith & Topin, 2018) with cosine annealing and a maximum learning rate of 0.21 for CIFAR10, CIFAR-100, and 0.05 for SVHN. We train each model for 50 epochs on CIFAR-10 and CIFAR100 whereas we apply 15 epochs on SVHN. For `∞ training, we use PGD attack with maximum perturbation ε = 8/255 and run the attack for K = 10 iterations for all datasets. The PGD step size is set to α = ε/4 = 2/255 for CIFAR-10, CIFAR-100; α = 1/255 for SVHN. For HAT, we fix γ to
0.5 and use β = 2.5 for CIFAR-10 and SVHN; β = 3.5 for CIFAR-100. Whereas the regularization parameter β for TRADES is set to 5.0 for CIFAR-10, SVHN and 6.0 for CIFAR-100. For MART, we choose β = 5.0. More details can be found in App. C.1.
Evaluating robustness. During training, we perform early stopping (Rice et al., 2020) and select the model that has the highest robustness against PGD (K = 20) for further evaluation. Further, to evaluate the robust accuracy of our models, we use AutoAttack (Croce & Hein, 2020) which comprises an ensemble of four diverse attacks (including a black-box attack). AutoAttack has been consistently shown to provide a reliable estimation of robustness. Nevertheless, we also conduct additional evaluations and checks with HAT to eliminate the possibility of gradient obfuscation in App. C.4.
Results. Table 2 reports the performance of HAT and other prominent defenses in the literature. It is evident that HAT can notably increase the clean accuracy of the models with little to no degradation of robustness. In other words, HAT consistently shrinks the gap between accuracy and robustness by around 2% compared to prior adversarial training schemes. For example, in the case of CIFAR-10, HAT boosts clean accuracy by 2.17% whilst achieving similar robustness as TRADES. In addition, we find that the 2.17% gain in clean accuracy on CIFAR-10 equates to a 2.08% jump on common corruptions (Hendrycks & Dietterich, 2019) (see App. C.3) which is highly desirable given the practical relevance of common corruptions. Finally, we note that our models do not show any signs of gradient obfuscation as evident from the analysis in App. C.4.

heading: 5.2 OTHER THREAT MODELS AND LARGER DATASETS
text: To verify the generality of our results, we now experiment with different attack configurations and larger benchmarks.
Threat models. We investigate two different threat configurations on CIFAR-10: (i) `∞ with norm 12/255 and (ii) `2 with norm 128/255 and follow the same setup as Sec. 5.1. Here, we compare HAT only with TRADES since TRADES achieves superior results than other techniques on CIFAR10. As shown in Table 3, HAT surpasses TRADES by a large margin. Specifically, with `∞ perturbations of size 12/255, HAT clearly surpasses TRADES, effecting 5.95%
improvement in clean accuracy while simultaneously bettering robustness by 0.70%. Moreover, HAT achieves a higher clean accuracy (↑ 1.46%) against `2 perturbations as well. However, the improvement here is rather limited owing to the fact that when training with smaller ε’s, the phenomenon of excessive directional margin might not be very severe.
Larger datasets. The efficacy of HAT indeed holds for large-scale datasets such as TinyImageNet200 and ImageNet-100 (Deng et al., 2009), where AT is known to cause a huge degradation in accuracy. We observe that HAT improves the accuracy of AT from 47.76% to 52.60% and robustness from 17.92% to 18.14% on TinyImageNet-200 against `∞ distortions of size 8/255. With
ImageNet-100 against `∞ perturbations of size 4/255, HAT outperforms TRADES by 4.20% on clean accuracy and 1.92% on robustness; surpasses AT by 1.36% on clean accuracy and 0.98% on robustness. More details on the experiment can be found in App. C.3.

heading: 5.3 EFFECT OF ADDITIONAL DATA AND WIDER NETWORKS
text: It has been observed that robust generalization benefits remarkably from the use of extra training data (Schmidt et al., 2018) and increased model capacity (Madry et al., 2018). We investigate the impact of these two factors on HAT training. Specifically, we build upon the recent approaches from Gowal et al. (2021) and Rebuffi et al. (2021) which provide a carefully designed experimental suite composed of model weight averaging, SiLU activation function, larger models and additional training data to considerably progress the state-of-the-art performance on multiple robustness benchmarks. While Gowal et al. (2021) uses an additional subset of 500k natural images extracted from 80 Million Tiny Images (80M TI) dataset (Torralba et al., 2008), Rebuffi et al. (2021) leverages 1M synthetic images generated by DDPM (Ho et al., 2020). Following the exact experimental setup from these papers2, we train several models on CIFAR-10 and CIFAR-100 using three different model architectures namely, PreAct ResNet-18 (PRN-18) (He et al., 2016), WideResNet-28-10 (WRN28-10) and WideResNet-34-10 (WRN-34-10) (Zagoruyko & Komodakis, 2016). However, since the additional data from Gowal et al. (2021) is not publicly available3, we resort to using a similar dataset provided by Carmon et al. (2019). Note that we always pick β = 3.5 and γ = 0.5 for HAT.
In Table 4, we benchmark the performance of HAT along with that of state-of-the-art approaches. Using PRN-18 and synthetic DDPM-generated images, HAT improves clean and robust accuracy by (i) 3.33% and 0.43% respectively on CIFAR-10 and (ii) 4.63% and 0.38% respectively on CIFAR100 against `∞ perturbations with norm 8/255. Notably, HAT sets a new state-of-the-art on CIFAR10 obtaining 89.02% clean accuracy and 57.67% robust accuracy. More importantly, these sizeable improvements with PRN-18 suggest that smaller networks have not yet exhausted their limits and we must invent techniques which better exploit the capacity of these networks. Moreover, with WRN-28-10, HAT also surpasses the state-of-the-art accuracy in the respective categories by 2-3%.

heading: 5.4 ANALYSIS
text: Accuracy vs. robustness trade-off. We investigate the sensitivity of HAT to its hyperparameters namely, the weight of robust loss β and the weight of helper loss γ. Fig. 5 includes a plot which
2Note that we do not use CutMix augmentation (Yun et al., 2019) with the setup from Rebuffi et al. (2021). 3Gowal et al. (2021) use a custom regenerated dataset which contributes to around 0.7% improvement in
robustness in comparison to the data from Carmon et al. (2019). See Sec. 4.3 from Gowal et al. (2021).
demonstrates the performance of HAT at different values of β ∈ [1.5, 4.0] and γ fixed to 0.25. We also show the curve obtained by varying the regularization parameter β of TRADES in Fig. 5. We do not include MART here since it achieves a relatively poor trade-off. We can clearly see that for HAT, β controls the trade-off between accuracy and robustness where increasing β improves robustness. Next, we study the impact of γ. First, as expected, HAT with γ = 0 performs identically to TRADES. Second, γ ∈ {0.25, 0.5} achieves the best trade-off and within this range, the choice of γ has negligible influence on the resulting trade-off exhibited by HAT. Third, increasing γ beyond 0.75 causes the robust accuracy to deteriorate due to the dominance of helper loss over robust loss. The hyperparameter sweeps for γ are presented in App. C.5.
How does HAT improve clean accuracy. The improvement in the performance on clean samples provided by HAT can be accredited to the following two observations on CIFAR-10: (i) In contrast to TRADES, HAT marginally compromises robustness to `∞ perturbations with a larger norm. We take a robust network and evaluate it using PGD attack (K = 40) for different values of `∞-norm ε ∈ [0, 20/255]. Fig. 6 plots the difference between robustness of HAT and TRADES vs. ε. We observe that while HAT outperforms TRADES at smaller ε’s, it performs slightly worse after the ε exceeds the value used during training, i.e., ε > 8/255. This implies that we have traded robustness to high ε’s for an improvement in accuracy. (ii) HAT exhibits a slightly lower directional margin along initial adversarial directions Rinit in comparison to AT and TRADES. The margin along Rinit for different algorithms (as evaluated on a random subset of 1024 samples from the CIFAR-10 test set) is illustrated in Table 5. Note that in Table 5, we list the models that achieve the same robust accuracy (∼ 47.9% to AutoAttack). Yet, the median margin for HAT is slightly lower than that for AT and TRADES. Additionally, in comparison to TRADES, HAT reduces the number of samples with margin greater than 10ε from 40% to 27%.

heading: 6 CONCLUSION
text: We presented experimental evidence to highlight that state-of-the-art adversarial defenses foster a superfluous increase in the margin along certain adversarial directions of the input space. This largely destroys the discriminative characteristics along these directions and partly contributes to the much-debated accuracy vs. robustness trade-off. Further, inspired by our analysis, we introduced a novel algorithm, Helper-based Adversarial Training (HAT), to alleviate the problem of excessive directional margin. HAT attempts to mimic the discriminative features learnt by standard trained networks to improve the accuracy on clean samples, hence achieving a superior accuracy vs. robustness trade-off compared to existing defenses. Moreover, HAT surpasses the known state-of-the-art clean accuracy by 3-5% without sacrificing robustness. Finally, we verify that HAT slightly reduces the directional margin, thus directly benefiting the accuracy. We believe that our experiments and the proposed HAT algorithm can open the door for further research on relieving the accuracy vs. robustness trade-off under limited model capacity. While HAT is only one possible candidate, future works can come up with more effective techniques to alleviate the problem of excessive margin.

heading: A TOY PROBLEM

heading: A.1 EXPERIMENTAL SETUP
text: Fig. 7 illustrates the toy dataset used for our experiment from two different viewpoints. To be precise, we draw the 3 features x1, x2 and x3 as follows. x1 = ρi cos(z) + 1, x2 = ρi sin(z) + 2 and x3 ∼ U(αi, βi) where z ∼ U(0, 2π) and 1, 2 ∼ N (0, σ2) where i = 1, 2 for class 1 and 2 respectively. We set σ = 0.2, ρ1 = 0.35, ρ2 = 1, α1 = 0.65, β1 = 0.70, α2 = 0.80 and β2 = 0.85.
We use a single hidden layer MLP with 25 hidden units and ReLU activation. For training, we use SGD with momentum 0.9, weight decay 0.0005 and set the learning rate to 0.1. We train the model via standard training and adversarial training respectively for 100 epochs. We use `∞ PGD with step size α = 0.05, maximum perturbation radius ε = 0.1 and run K = 5 iterations for AT.

heading: A.2 HAT ALLEVIATES EXCESSIVE MARGIN
text: In this part, we check whether HAT can correct the failure mode of adversarial training for the toy problem studied in Sec. 3. Fig. 8 shows the classification boundary learnt with HAT. Clearly, the decision boundary in the x1-x3 plane is tilted and not completely parallel to the x3 axis, indicating some dependence on the feature x3. Thus, HAT indeed attenuates the problem of excessive margin which is also reflected in a 4% improvement in accuracy over adversarial training. Additionally, it improves robustness from 73.8% to 74.6%.

heading: B SEC. 3 CONTINUED: EVIDENCE ON CIFAR-10

heading: B.1 EXPERIMENTAL SETUP
text: For the margin experiment, we train ResNet-18 (He et al., 2016) on CIFAR-10 (Krizhevsky, 2009) training set. We simply adopt the set of hyperparameters and some improvements from DAWNBench (Coleman et al., 2017) submissions. We use SGD optimizer with Nesterov momentum 0.9 (Nesterov, 1983) and weight decay 0.0005. We further use cyclic learning rates (Smith & Topin, 2018) with cosine annealing and a maximum learning rate of 0.21. We train the model for 50 epochs via standard training. Then, we perform adversarial fine-tuning for 25 epochs with the same scheduler and learning rate settings. Further, we evaluate margins on a set of 512 samples drawn uniformly at random from CIFAR10 test set for the visualizations in Fig. 4 (and Fig. 9).

heading: B.2 ADDITIONAL MARGIN PLOTS
text: The margins along other adversarial directions R10,R15,R20 before and after adversarial finetuning are displayed in Fig. 9. Here, Rk corresponds to the adversarial directions computed at the kth iteration of adversarial fine-tuning. We see an increase in the margin along other adversarial directions as expected. Nevertheless, the relative increase along these directions is not as large as compared to that along initial onesR0 (see Fig. 4).

heading: C FURTHER PERFORMANCE EVALUATION

heading: C.1 EXPERIMENTAL SETUP FOR SEC. 5.1
text: In this section, we list all the details of our training and evaluation setup. We run all our experiments thrice and report the average scores obtained unless stated otherwise.
Training setup. We use ResNet-18 (He et al., 2016) for CIFAR-10 and CIFAR-100 (Krizhevsky, 2009); and PreAct ResNet-18 (He et al., 2016) for SVHN (Netzer et al., 2011). For all our experiments, we use SGD optimizer with Nesterov momentum (Nesterov, 1983), momentum factor 0.9 and weight decay 0.0005. We further employ cyclic learning rates (Smith & Topin, 2018) with cosine annealing and a maximum learning rate of 0.21 for CIFAR-10 and CIFAR-100; 0.05 for SVHN. For CIFAR-10 and CIFAR-100, we train the models for 50 epochs with a batch size of 128. In the case of SVHN, we only train for 15 epochs.
For computing adversarial examples during training, we apply `∞-PGD with the following hyperparameters: `∞ norm ε = 8/255, step size α = 2/255 and run the attack for K = 10 iterations. Note that we re-implement AT, TRADES and MART and train existing methods and HAT according to the aforementioned settings. The hyperparameters of TRADES, MART and HAT are as follows. For HAT, we fix γ to 0.5 and use β = 2.5 for CIFAR-10 and SVHN; β = 3.5 for CIFAR-100. The regularization parameter β for TRADES is set to 5.0 for CIFAR-10, SVHN and 6.0 for CIFAR-100. For MART, we choose β = 5.0. We also use the same setup to train a standard model for computing helper labels during HAT training.
Evaluation protocol. During training, we perform early stopping (Rice et al., 2020) i.e., we track the robustness of the model to PGD (K = 20) on the test set and pick the model that performs the best for further evaluation. In order to benchmark the `∞ robustness, we always test our models against AutoAttack (AA) (Croce & Hein, 2020) using the default code available at https:// github.com/fra31/auto-attack.
Setup for other attack configurations. We reuse the exact aforementioned setup when investigating different threat models on CIFAR-10. In the case of `2 perturbations, we use the following training adversary: ε = 128/255, α = 15/255 and K = 10. We choose β = 2.5 and γ = 0.5 for HAT and β = 5.0 for TRADES. Whereas we set α = 4/255,K = 10 for `∞ perturbations with norm ε = 12/255. Additionally, we pick β = 3.5 and γ = 0.5 for HAT and β = 6.0 for TRADES.

heading: C.2 EXPERIMENTAL SETUP FOR SEC. 5.3
text: For training the models enumerated in Table 4 of Sec. 5.3, we follow the setting designed by Gowal et al. (2021) and Rebuffi et al. (2021) respectively. For the sake of completeness, we briefly recap the training setup here. We use SiLU activation function (Hendrycks & Gimpel, 2016) with PreAct ResNet (He et al., 2016)/WideResNet (Zagoruyko & Komodakis, 2016) backbone. The optimizer used is SGD with Nesterov momentum (Nesterov, 1983), momentum factor 0.9 and weight decay 0.0005. We further use cyclic learning rates (Smith & Topin, 2018) with cosine annealing, a maximum learning rate of 0.4 and a warmup of 10 epochs. The training batch size is set to 1024 with 70% of the batch comprising of extra/synthetic data. We also use model weight averaging (Izmailov et al., 2018) with decay τ = 0.995 and train the models for 400 CIFAR-10-equivalent epochs with the extra data from Carmon et al. (2019). In the setting with synthetic DDPM-generated data, we apply 800 epochs. The training attack used is PGD (K = 10) with step size α = 2/255 and norm ε = 8/255. Finally, we perform early-stopping by tracking the performance on a disjoint validation set using PGD (K=40) with margin loss (Carlini & Wagner, 2017). We separate first 1024 samples from the training set for validation. Note that we do not employ CutMix (Yun et al., 2019).

heading: C.3 ADDITIONAL RESULTS FOR SEC. 5.1
text: Detailed version of Table 2. The complete performance evaluation with ResNet-18 across different datasets appears in Table 6. Here, we also include the standard deviation over 3 runs for completeness. It is clear that HAT outperforms other existing methods, hence narrowing the gap between accuracy and robustness.
Evaluation with other adversaries. We additionally scrutinize the performance of robust models trained on CIFAR-10 against conventional adversaries. In order to benchmark the `∞ robustness, we apply PGD with ε = 8/255, α = 0.01, K = 40 and r = 5 restarts following the evaluation protocol used in Carmon et al. (2019), we denote this adversary as PGD+. We also evaluate the vulnerability of our models to PGD with CW loss (Carlini & Wagner, 2017), ε = 8/255, α = 0.01 and K = 40 steps. Besides, we also test the vulnerability against naturally occurring benign corruptions. Specifically, we use the CIFAR-10-C dataset from Hendrycks & Dietterich (2019). The results presented in Table 7 show that HAT does not break down against other adversaries. Crucially, HAT leads to 2.08% improvement against common corruptions. This suggests that one needs to improve both accuracy as well as robustness in order to advance the performance under real-world distortions.
Combination with other adversarial defenses. To study the utility of HAT in concert with recent adversarial defenses, we examine the performance of HAT when integrated with two schemes that improve over TRADES namely, (i) FAT (Zhang et al., 2020) which utilizes earlystopping when constructing adversarial data during training and (ii) AWP (Wu et al., 2020) which encourages flatness in weight loss landscape. Note that although HAT is based on TRADES, it can be easily integrated into these defenses. In our experiments, we follow the same training and evaluation setup as employed by Zhang et al. (2020) and Wu et al. (2020) respectively. The resulting performance on CIFAR-10 is summarized in Table 8. As shown in Table 8, incorporating HAT pushes the clean accuracy of FAT by 2% and also benefits robust accuracy by 0.9%. In combination with AWP, HAT leads to a 2.6% gain in
clean accuracy relative to AWP. These results underline that newer methods also suffer from the phenomenon of excessive margin and HAT indeed provides a general fix to alleviate this problem.
Larger datasets. We also examine the performance of HAT on large-scale datasets such as TinyImageNet-200 and ImageNet-100 (Deng et al., 2009). ImageNet-100 (Laidlaw et al., 2021) is a 100-class subset of ImageNet. We use a similar setup as Sec. 5.1 with PreAct ResNet-18 except the following changes. We train for 30 epochs on TinyImageNet-200 against `∞ perturbations of size 8/255. Whereas on ImageNet-100, we train for 50 epochs with batch size 256, weight decay 0.0001, use input normalization and `∞ perturbations of size 4/255. On these datasets, we observed that TRADES performed significantly worse than AT. This might be due to the use of KL-divergence loss instead of cross-entropy (CE) loss for crafting adversarial examples. So, we use AT-based formulation of HAT which utilizes CE loss as robust loss as well as for computing adversarial perturbations. For TRADES, we use β = 8.0 for TinyImageNet-200; β = 6.0 for ImageNet-100 while we pick β = 1.75 and γ = 1.0 for HAT on both the datasets. The results are presented in Table 9. Note that we measure robust accuracy using AutoAttack.

heading: C.4 CHECKING FOR GRADIENT OBFUSCATION
text: We evaluate our models with AutoAttack (Croce & Hein, 2020) which has been consistently shown to provide a reliable evaluation of robustness. Nevertheless, we also include additional sanity checks
to eliminate the possibility of gradient masking. First, following the guidelines in Athalye et al. (2018), we examine the impact of following changes on the robustness of a model trained via HAT on CIFAR-10 against `∞ perturbations of size 8/255:
• Attack radius ε: Fig. 10 plots the robustness of HAT vs. the radius ε of PGD (K = 40) attack. As expected, increasing the attack budget causes the robust accuracy to monotonically drop to 0%. In particular, unbounded PGD adversary reduces the robustness to 0%.
• Attack iterations K and random restarts r: Increasing the number of attack iterations K or number of random restarts r only marginally lowers the robust accuracy (see Table 10 and Table 11). In order words, the attack has converged and does not suffer from gradient obfuscation.
• As shown in Tables 12 and 13, transfer and non-transfer black box evaluations do not show any signs of gradient obfuscation. We use square attack (Andriushchenko et al., 2020) for non-transfer black box evaluation. Importantly, black box attacks have a lower success rate than white box attacks.
Moreover, as a sanity check, we also visualize the loss landscapes of our trained models in Fig. 11. The loss landscapes are smooth and exhibit low curvature which are the typical characteristics of a robust network (Moosavi-Dezfooli et al., 2019). In summary, these observations indicate that HAT does not lead to gradient obfuscation.
Evaluation with GAMA-PGD. To further confirm that our method does not suffer from gradient masking, we investigate the robustness of models reported in Table 4 against GAMA-PGD (Sriramanan et al., 2020), a stronger adversary than vanilla PGD attack. The evaluation results are provided in Table 14. As evident from Table 14, the performance of HAT does not break down even against GAMA attack which signifies that HAT does not cause obfuscated gradients.

heading: C.5 HAT: ANALYSIS (CONTINUED)
text: Accuracy vs. robustness trade-off. Fig. 12 compares the trade-off obtained by HAT with that of AT and MART (Wang et al., 2020). For HAT, we fix γ = 0.25 and vary β ∈ [1.5, 4.0]; for MART, we vary β ∈ [1.0, 5.0]. HAT surpasses AT and MART by a large margin. With MART, we observe notable improvements in robustness to weaker adversaries such as PGD (see Table 7), but the gains diminish when evaluated with a stronger adversary such as AutoAttack (Croce & Hein, 2020).
Sensitivity of HAT to γ. We here examine the impact of the weight of helper loss γ on the accuracy vs. robustness trade-off exhibited by HAT and conduct additional hyper-parameter sweeps. We show the corresponding trade-off curves for HAT with γ ∈ {0.0, 0.25, 0.5, 0.75} in Fig 13. As expected, the trade-off curve for γ = 0 almost coincides with that of TRADES (note that there is a minor difference due to randomness). Further, γ ∈ {0.25, 0.5} achieve the best performance and identical trade-off curves. With γ = 0.75, the robust accuracy starts to deteriorate due to the dominance of helper loss over robust loss with the effect being more severe for lower values of β. We believe that increasing γ beyond 0.75 would further negatively affect the performance.
Next, we conduct ablation studies on CIFAR-10 (`∞, 8/255) to better understand HAT algorithm. While we elaborate on the HAT design choices below for the sake of completeness, note that we do not conduct any hyperparameter tuning concerning these choices and naively resort to the setting mentioned in Algorithm 1 throughout this work. In the following analysis, we use the same experimental setup as in Sec. 5.1.
Impact of helper example definition. We analyze the influence of different choices for defining helper examples for HAT (refer Algorithm 1). We consider the following choices: x + αr where r is the adversarial perturbation computed at x and α ∈ {1.5, 2.0, 2.5, 3.0, 3.5}. Note that the helper label is always queried at x + r as in Algorihtm 1. Selecting α = 2.0 means that the helper examples are given by x + 2r which represents the setup used in this paper. Fig. 14 illustrates the performance obtained by the resulting HAT-trained models. A lower value for α e.g., α = 1.5, hinders the model from increasing the margin and hence, the resulting model achieves high clean accuracy but a slightly poor robust accuracy. α = 2.0 and α = 2.5 achieve identical results, and the model does not compromise robustness yet achieves a gain in clean accuracy. Further increasing α beyond 3.0 leads to a drop in both clean as well as robust accuracy. This is because the resulting training samples might not be well separated and their target labels may conflict with each other.
Ablation study for obtaining helper labels. We perform an ablation study with different choices for obtaining the labels for helper examples during HAT training (refer Algorithm 1). Specifically, we query the helper label ỹ, via a regularly trained model fθstd , at x+ αr where r is the adversarial perturbation computed for input x and vary α ∈ {0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5}. In other words, the helper label is defined as the prediction of the standard model fθstd at x + αr; whereas the helper example is always defined to be x + 2r. Setting α = 1.0 recovers the setup used in this work. The results are shown in Fig. 15 (solid lines). For α ∈ {0.5, 1.0, 1.5}, we observe a similar performance relative to that reported in Table 2. With α ≥ 2.0, the clean accuracy of the robust model slightly increases whilst its robust accuracy starts to degrade. This is because helper labels queried at farther points with α ≥ 2.0 might be much more pessimistic (worst-case) and hence, present more opposition to the robust loss. Thus, we observe a drop in robust accuracy which can be in fact addressed by simply increasing the weight of robust loss. This is indicated by the dashed lines in Fig. 15 where we increase the weight of robust loss β in proportion to the rise in α. Taking this adjustment into account leads to the same performance as that with α = 1.0, indicating that the underlying trade-off still remains the same.

heading: C.6 COMPARISON WITH CHEN ET AL. (2021)
text: The work by Chen et al. (2021) is closest our approach in the sense that it also uses knowledge distillation or self-training. However, the standpoint and the way of using it is significantly different from
our work. We use distillation with a motivation to prevent excessive margin and benefit clean accuracy, whilst Chen et al. (2021) uses it from the usual perspective of using self-learning to improve generalization and thereby alleviate robust overfitting. Moreover, it applies self-training at the adversarial inputs while we apply it for overly perturbed helper data. From a computational perspective, the approach from Chen et al. (2021) uses two self-teachers, one of which is adversarially trained, which presents a significant overhead compared to HAT which requires only a regularly trained one. In addition, Chen et al. (2021) use model weight averaging which begs a larger number of training iterations to obtain improvements. Empirically, the distillation setup from Chen et al. (2021) which uses two self-teachers attains 83.67% accuracy and 48.03% robustness against AutoAttack. Clearly, HAT outperforms these results by achieving 84.90% accuracy and 49.08% robustness. Moreover, we also replicate their setup with model weight averaging and perform HAT to further improve performance. In contrast to their scores of 84.65% accuracy and 49.35% robustness, our proposed HAT obtains 85.69% accuracy and 49.34% robustness.

Review summaries:
The authors aim to reduce the gap between clean accuracy without adversarial training and with adversarial training. To improve the robustness-accuracy trade-off, the authors introduce Helper-based Adversarial Training. The main idea is to use adversarial examples $\mathbf{x}_{\text{adv}} = \mathbf{x} + 2 \mathbf{r}$, where $\mathbf{r}$ is standard PGD adversarial perturbation, as helper adversarial examples. The model is trained to classify these helper adversarial examples as the adversarial label predicted by the model trained without adversarial training. In the experiments, the authors show that HAT improves clean accuracy and robust accuracy on CIFAR-10 and CIFAR-100 datasets when compared with TRADES defense.

The paper highlights the presence of excessive invariance in the prediction of robust models along the initial adversarial directions. Initial adversarial directions refers to the directions in which adversarial images generated using a standard trained model are present. Based on this hypothesis the authors propose a training method where the excessive invariance is minimized using the cross entropy loss between the prediction(made using the standard trained model) of larger epsilon adversarial image and the prediction of the adversarial image, in addition to the TRADES loss formulation. This additional loss term indeed improves the accuracy-robustness trade-off by giving a significant boost in clean accuracy along with a slight boost in adversarial robustness as compared to the existing methods. Overall the paper is well written and easy to follow.

**Few sentences summary**: the paper proposes a new training loss for adversarial training in the Lp-norm setting. Based on the observation that adversarial training increases the classification margin in a disproportionate manner compared to the nominal training setting, the authors introduce additional samples called "helpers" to reduce the classification margin. The helper samples, which are samples translated further away in the worst adversarial direction, can change labels compared to the adversarial samples. Helper samples get assigned labels from a standardly trained model, thus acting as a constraint coming from this standardly trained network.  Regarding the **results** and **contributions**: * Novel method using helpers to define a new training loss for adversarial training. * On par or better results in robust accuracy compared to the SOTA training loss TRADES on CIFAR-10/100, SVHN, TinyImageNet and a subset of ImageNet. * Much improved results in clean accuracy compared to TRADES, thus reducing the gap between clean and robust accuracy which is primordial for the application of Lp norm models to practical uses. * Clear analytical tools based on the margin analysis to investigate the proposed method and how/why it works.