Squeeze Training for Adversarial Robustness
Authors: Qizhang Li
Yiwen Guo
Wangmeng Zuo
Hao Chen
Venue: ICLR 2023 poster (2023)
Keywords: Adversarial Training
Adversarial Examples
Model Robustness

Abstract: The vulnerability of deep neural networks (DNNs) to adversarial examples has attracted great attention in the machine learning community. The problem is related to non-flatness and non-smoothness of normally obtained loss landscapes. Training augmented with adversarial examples (a.k.a., adversarial training) is considered as an effective remedy. In this paper, we highlight that some collaborative examples, nearly perceptually indistinguishable from both adversarial and benign examples yet show extremely lower prediction loss, can be utilized to enhance adversarial training. A novel method is therefore proposed to achieve new state-of-the-arts in adversarial robustness. Code: https://github.com/qizhangli/ST-AT.

Published as a conference paper at ICLR 2023
SQUEEZE TRAINING FOR ADVERSARIAL ROBUSTNESS
Qizhang Li Yiwen Guo Wangmeng Zuo Hao Chen
Harbin Institute of Technology. Tenneet Security Big Data Lab, 'Independent Researcher, *UC Davis fliqi Aigizhangg99 quoyiwent 9 9 @egail..coom wmzuc@hit.. edu. chen@ucdavis.edd
ABSTRACT
The vulnerability of deep neural networks (DNNs) to adversarial examples has attracted great attention in the machine learning community. The problem is related to non-flatness and non-smoothness of normally obtained loss landscapes. Training augmented with adversarial examples (a.ka., adversarial training) is considered an effective remedy. In this paper, we highlight that some collaborative examples, nearly perceptually indistinguishable from both adversarial and benign examples yet show extremely lower prediction loss, can be utilized to enhance adversarial training. A novel method is therefore proposed achieve new state-of-the-ans adversarial robustness. Code: https:/gtthhb.com////aaag//////
INTRODUCTION
Adversarial examples (Szegedy et al. 2013 Biggio al. 2013) crafted by adding imperceptible perturbations to benign examples are capable of fooling DNNs to make incorrect predictions. The existence of such adversarial examples has raised security concerns and attracted great attention.
Much endeavour has been devoted to improve the adversarial robustness of DNNs. As one of the most effective methods, adversarial training (Madry et al., 2018) introduces powerful and adaptive adversarial examples during model training and encourages the model to classify them correctly.
In this paper, to gain deeper understanding of DNNs, robust or not, we examine the valley of their loss landscapes and explore the existence of collaborative examples in the e-bounded neighborhood of benign examples, which demonstrate extremely lower prediction loss in comparison to that of their neighbors. Somewhat unsurprisingly, the existence of such examples can be related to the adversarial robustness of DNNs. particular, given a model that was trained to be adversarially more robust, then it is less likely to discover collaborative example. Moreover, incorporating such collaborative examples into model training seemingly also improves the obtained adversarial robustness. On this point, advocate squeeze training (ST), in which adversarial examples and collaborative examples of each benign example are jointly and equally optimized in a novel procedure, such that their maximum possible prediction discrepancy is constrained. Extensive experimental results verify the effectiveness of our method. We demonstrate that ST outperforms state-of-the-art remarkably on several benchmark datasets, achieving absolute robust accuracy gain of >+1000 % without utilizing additional data on 10. It can also be readily combined with variety of recent efforts, e.g., RST (Carmon al. 2019) and RWP (Wu et al. 2020b). to further improve the performance.
BACKGROUND AND RELATED WORK
2.1
ADVERSARIAL EXAMPLES
Let x; and yi denote benign example (e.g., natural image) and its label from S = {Xx, V:)) Hi where x; E X' and y ) (O...., C. 1}. We use B, xx] (x' ||xx x; |loc S c) to represent the e-bounded Lo neighborhood of x,. A DNN parameterized by θ can be defined as function
fe(+) X e Without ambiguity, we will drop the subscript in fe(-) and write as f(-).
In general, adversarial examples are almost perceptually indistinguishable to benign examples, yet they lead to arbitrary predictions on the victim models. One typical formulation of generating an
Work was done under co-supervision of Yiwen Guo and Wangmeng Zuo who in correspondence.

---

Published as a conference paper at ICLR 2023
adversarial example is to maximize the prediction loss in constrained neighborhood of a benign example.
Projected gradient descent (PGD) (Mdryy et al. 2018) (or the iterative fast gradient sign method, i.e. I-FGSM (Kurakin et al. 2017) is commonly chosen for achieving the aim. It seeks possible adversarial examples by leveraging the gradient of g Eo f w.r.t. its inputs, where l is loss function (e.g., the cross-entropy loss CE((, y)). Given a starting point x", an iterative update is performed with:
x Bx (x" + a sign CEX((((
[x] (+) projects its input onto where x" is temporary result obtained at the t-th step and function [x
the c-bounded neighborhood of the benign example. The starting point can be the benign example (for I-FGSM) or its randomly neighbor (for PGD).
Besides I-FGSM and PGD, the single-step FGSM (Goodfellow et al. 2015), C&W's attack (Carlini & Wagner 2017). DeepFool (Moosavi-Deztooli al. 2019)), and the momentum iterative FGSM (Dong et 2018) are also popular and effective for generating adversarial examples. Some work also investigates the way of generating adversarial examples without any knowledge of the victim model.
which are known as -bx attacks (Papernot et al.] 2017 Chen et al. 2017, Ilyas et al. 2018 Cheng et at. 2019 al. 2019 Guo et al. (2020) and no- no-box attacks (Papernot al. 2017 Li et al. 2020). Recently, the ensemble of variety of attacks becomes popular for performing adversarial attack and evaluating adversarial robustness. Such a strong adversarial benchmark, called AutoAttack (A.A) (Croce& & Hein| 2020), consists of three white-box attacks, i.e., APGD-CE, APGD-DLR, and
FAB Croce& & Hein 2019). and one black-box attack, i.e., the Square Attack (Andriushchenko et al. 2019
We adopt it in experimental evaluations.
In this paper, we explore valley of the loss landscape of DNNs and study the benefit of incorporating collaborative examples into adversarial training. In an independent paper (Tao et al.) 2022), hypocriti- cal examples were explored for concealing mistakes of a model, as an attack. These examples also lied in the valley. Yet, due to the difference in aim, studies of hypocritical examples in (Tao et al. 2022) were mainly performed based on mis-classified benign examples according to their formal definition, while our work concerns local landscapes around all benign examples. Other related work include unadversarial examples (Salman et al.) 202T) and assistive signals (Pestana et al. 2021 that designed 3D textures customize objects for better classifying them.
2.2 ADVERSARIAL TRAINING (AT)
Among the numerous methods for defending against adversarial examples, adversarial training that incorporates such examples into model training probably one of the most effective ones. We will revisit some representative adversarial training methods in this subsection.
Vanilla AT (Madry et al. 2018) formulates the training objective a simple min-max game. Adversarial examples are first generated using for instance PGD to maximize some loss (e.g.. the cross-entropy loss) in the objective, and then the model parameters are optimized to minimize the same loss with the obtained adversarial examples:
min max CF(S(x)) v))
x/EB, [x.]
Although effective in improving adversarial robustness, the vanilla AT method inevitably leads to decrease in the prediction accuracy of benign examples, therefore several follow-up methods discuss improved and more principled ways to better trade off' clean and robust accuracy (Zhang et al. 2019 Kannan al. 2018 Wang et al. 2020 Wu al. 2020b). Such methods advocate regularizing the output of benign example and its ad adresaaiial neighbors. With remarkable empirical performance, they are regarded as strong baselines, and we will introduce a representative one, i.e., TRADES (Zhang
et al. 2019).
TRADES (Zhang et al. 2019) advocates a learning objective comprising two loss terms. Its first term penalizes the cross-entropy loss of benign training samples, and the second term regularizes the difference between benign output and the output of possibly malicious data points. Specifically, the worst-case Kullback-Leible (KL) divergence between the output of each benign example and that of any suspicious data point in its e-bounded lor neighborhood is minimized in the regularization term:
min Σ (CE(f(x;), yi) + + max f θ
(3)
[x]
2

---

Published as a conference paper at ICLR 2023
Normal
Vanilla AT
TRADES
MART
Normal
Vanilla AT
TRADES
MART
1.0
1.0
1.0
1.0
Cross ortooy
0.8
0.0
Cross entropy
Cross ortoogy
0.0
0.0
Closs Cros-entropy
0.8
0.0
0.0
Coss-entrgg
0.6
0.6
66
0.6
0.6
0.6
0.6
0.5
0.4
0.4
Closs
0.
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0.2
0.2
0.0
0.0
0.0
0.0
0.0
32
16
8
32
16
(2255)
(2253)
(/255)
(/255)
((255)
(1255
(a) CIFAR-I0, ResNett-1
(b) CIFAR-10, WRN-34-10
Normal
Vanilla AT
TRADES
MART
Normal
Vanilla AT
TRADES
MART
2.5
2.5
Choss 3
2.5
2.5
Cress 2222922
2.5
2.5
2.5
2.5
Closs entrayy
2.0
2.00
2.00
Cross-end Tereoo
2.00
2.0
Cross endropy
2.0
2.00
2.0
2.5
1.5
1.5
1.5
1.5
1.5
Crosseen
1.5
2 1.5
2.0
Closs
1.0
1.0
1.0
1.0
1.0
1.0
0.5
8 16
8
8
32
8 16
8 16
8
(1225))
(U255)
(1/2551
(1/255)
(225)
(1255)
(1225))
(2255)
(c) CIFAR- 100, ResNet-18
(d) CIFAR-ION WRN-34-10
Figure I: The average cross-entropy loss value of benign examples (blue) and collaborative examples (red) on (a) ResNet-18 trained using CIFAR-10, 10, (b) wide ResNet-34-10 10 trained using CIFAR-10, (c) ResNet- 18 trained on CIFAR- 100, and (d) wide ResNet-344 10 trained on CIFAR- 100. Shaded areas indicate scaled standard deviations. The collaborative examples are crafted with a fixed step size of 1/255 and various perturbation budgets.
Other efforts have also been devoted in the family of adversarial training research, egg. MART (Wang et al. 2020). robust self training (RST) (Carmon et al. [019)) and adversarial weight perturba- tion (AWP) (Wu et al. 20205). More specifically, after investigating the influence of mis-classified samples on model robustness, MART advocates giving specific focus to these samples for robustness. AWP identifies that flatter loss changing with respect to parameter perturbation leads to improved generalization of adversarial training, and provides a novel double perturbation mechanism. RST proposes to boost adversarial training by using unlabeled data and incorporating semi-supervised learning. Rebuffi et al. (202T) focus on data augmentation and study the performance of using genera- tive models. There are also insightful papers that focuses on model architectures (Huang et al. [021 Wu et al. 2020a; Bai et al., [021] Mao et al., 202T Paul & Chen 2021), batch normalization (Xie et al. 2020a). and activation functions (Xie et al.) 202 Dai et al. 2021 Distillation from robust
models has also been studied (Zi al. 202 Shao et al. 2021 Awais et al., 202T).
Our ST for improving adversarial robustness is partially inspired by recent adversarial training effort, and we will discuss and compare to TRADES and other state-of-the-arts in Section 5. and 5.2
Besides, our method can be naturally combined with variety of other prior effort introduced in this section, to achieve further improvements, as will be demonstrated in Section
3 COLLABORATIVE EXAMPLES
With the surge of interest in adversarial examples, we have achieved some understandings of plateau regions on the loss landscape of DNNs. However, valleys of the landscape seem less explored. In this section, we examine the valleys and explore the existence of collaborative examples, i.e., data points are capable of achieving extremely lower classification loss, in the e-bounded neighborhood of benign examples. In particular, we discuss how adversarial robustness of DNNs and the collaborative examples affect each other, by providing several intriguing observations.
3.1 VALLEY OF THE LOSS LANDSCAPE
Unlike adversarial examples that are data points with higher or even maximal prediction loss, we pay attention to local minimum around benign examples in this subsection.
To achieve this, we here simply adapt the PGD method to instead minimize the prediction loss with:
++1
1g [x] (x* - a sign (xx CE(((x)), (x))))))).
(4)
Comparing Eq. (4) to D. it can be seen that their main difference is that, in Eq. gradient descent is performed rather than gradient ascent. Similar to the I-FGSM and PGD attack, we clip the result in Eq. (4) after each update iteration to guarantee that the perturbation is within a presumed budget, e.g., 4/255, 8/255. 16/ /2555, or 32/255. 255. We perform such update with step size of = 1/255. ResNet (He et al. 2016b) and wide ResNet (Zagoruyko & Komodakis 2016) models trained on CIFAR- 10 and CIFAR-100 are tested.
3

---

Published as a conference paper at ICLR 2023
Teeergan Sarm
Teat Santo
collaborative collaborative "mountain" (rohust)
Diigke Pret Max Prtt
Original Prbt: hoo. eat
(normall
(a) normal
(b) robust
Figure 2: Visualization of benign example (left), collaborative example crafted on normally trained ResNet- 18 model (middle), and collaborative ex- ample crafted on robust ResNet-18 model (right).
Figure 3: The loss landscapes in (a) and (b) show a normal ResNet-18 18 and an adversari- ally trained ResNet-18 using TRADES, respec- tively, all on CIFAR-100.
After multiple update steps (preciously 100 steps for this experiment), evaluate the eross-entropy loss the obtained examples. We compare it to the benign loss in the left most panels in Figure || can be that, though benign data shows relatively low cross-entropy loss (i.e., 0.2 on average for ResNet-18 on the CIFAR-10 10 test set) already, there always exists neighboring data that easily achieve extremely lower loss values (i.e., almost 0). Such data points showing lower prediction loss are collaborative examples of our interest. Figure 2 visualize the collaborative examples.
The existence of collaborative examples implies large local Lipschitz constants or non-flat regions of Lo f. from a somehow different perspective against the conventional adversarial phenomenon. To shed more light on this, we further test with DNN models that were trained to robust adversarial examples. using the vanilla AT, TRADES, and MART The results can be found in the right panels Figure [ We see that it more difficult achieve zero prediction loss with these models, probably owing to flatter loss landscapes (I.i et al., 2018). See Figure in which the perturbation direction obtained utilizing Eq. 44. and is random chosen in a hyperplane orthogonal to u. We analyze the angle between collaborative perturbations and PGD adversarial perturbations in Appendix B and the results show that, on less robust model, the collaborative perturbations and the adversarial perturbations are closer to be orthogonal.
Figure 2 also illustrate an collaborative example generated on the robust model, and we see that the collaborative perturbations for robust and non-robust models perceptually different. In particular, a bluer sky leads to more confident prediction the robust model. For readers who are interested, more visualization results provided in Appendix
3.2 How CAN COLLABORATIVE EXAMPLES AID?
Given the results that the collaborative examples are less "destructive" on more adversarially robust model, we raise question:
How can collaborative examples in return benefit adversarial robustness?
Towards answering the question, one may first try to incorporate the collaborative examples into the training phase to see whether adversarial robustness of the obtained DNN model can be improved. To this end, we resort to the following learning objective:
min Σ ((CE(f(x))) v.) + 3 fx
(5)
where xtal is collaborative example crafted using the method introduced in Section 3.1 and Eq.
Such an optimization problem minimizes output discrepancy between the collaborative examples and their corresponding benign examples, in addition the loss term that encourages correct prediction on benign examples. This simple and straightforward method has been similarly adopted in Tao er al.'s 's work (Tao et al. 2022) for resisting hypocritical examples. A quick experiment is performed here to test its benefit to adversarial robustness in our settings. The inner update for obtaining collaborative examples is performed over 10 steps, with step size of 2/255 and perturbation budget of = 8/2 8/255. We evaluate prediction accuracy on the PGD adversarial examples and benign examples for comparison. Figure shows the test-set performance of ResNet-18 trained as normal and trained using Eq. (5) on CIFAR-10 10 and CIFAR-100, respectively. It can be seen that the robust accuracy is improved remarkably by solely incorporating collaborative examples into training. In the meanwhile, those normally trained models consistently show ~ 0% robust accuracy.
All models trained via adversarial training show quite good robust accuracy under adversarial attacks on CIFAR-10 and CIFAR-100. Clean and robust accuracy of these models provided in Appendix
4

---

Published as a conference paper at ICLR 2023
100
100
Nona (ciaa)
En (cleen)
80
80
Nermal (rabue))
Es (5) Orebuet)
Accuracy (90)
Accuracy (1%)
60
60
In w feel
Normal (clear)
Eq (clean)
Normal (robust)
(5) crooust)
20
40
100
120
40
1000
120
Epach
Epoch
(a) CIFAR-10
(b) CIFAR-100
Figure 4: Changes in clean and robust accuracy during training ResNet- on (a) CIFAR-IO and (b) CIFAR-100, using Eq. (5). Best viewed in color.
However, there still exists a considerable gap between the obtained performance of Eq. 5 and that of TRADES (see Figure Appendix). Robust overfitting (Rice et al. (0200) can be observed on the red curves
after the 80-th epoch), even though training with collaborative examples and testing with adversarial examples, and it seems more severe in comparison to that occurs with existing adversarial training methods. We also test other DNN architectures, e.g., VGG (Simonyan & Zisserman 2015) and wide ResNet (Zagoruyko & Komodakis, 2016), and the results are similar.
4 SQUEEZE TRAINING (ST)
In the above section, we have experimentally shown that collaborative examples exist and they can be used to improve the adversarial robustness of DNNs, by simply enforcing their output probabilities to be close to the output of their corresponding benign neighbors. In this section, we consider utilizing collaborative examples and adversarial examples jointly during model training, aiming at regularizing non-flat regions (including valleys and plateaus) of the loss landscape altogether.
4.1 METHOD
The adversarial examples can be utilized during training in a variety of ways, leading to various adversarial training methods. In this paper, considering that the adversarial and collaborative examples are both caused by non-flatness of the loss landscapes, we propose to penalize the maximum possible output discrepancy of any two data points within the e-bounded neighborhood of each benign example. Inspired by the adversarial regularization in, e.g., TRADES, we adopt benign prediction loss term in combination with a term which possible adversarial examples and possible collaborative examples are jointly regularized. The benign example itself is not necessarily involved in error accumulation from the regularization term, since, in this regard, the output of a benign example is neither explicitly encouraged to be "adversarial" nor to be "collaborative". To achieve this, we advocate squeeze training (ST) whose learning objective is:
min Σ [(CE(((x)::x) v6) + B
max
Eveg [x)),
EB, [.]
x" EB x.
(6)
s.t. P3(3 xx) 2 py (3: xx)= 2 py py (y: x"), Vi,
where freg() (.) is a regularization function which evaluates the discrepancy between two probability vectors, and 3 is scaling factor which balances clean and robust accuracy. ST squeezes possible prediction gaps within the whole e-bounded neighborhood of each benign example, and it jointly regularizes the two sorts of non-flat regions of the loss landscape. The constraint in Eq. (6) is of the essence and is introduced to ensure that the two examples obtained in the inner optimization include one adversarial example and one collaborative example, considering makes little sense to minimize the gap between two adversarial examples or between two collaborative examples. There are several different choices for the regularization function Eregs e.g., the Jensen-Shannon (JS) divergence, the squared le distance, and the symmetric KL divergence, which are formulated follows:
(I) JS:
Ereg (KL( f(x') + f(x") ,((')) + KL fxx') + f(x")
f(x")),,
2
2
(2) (Squared) 1z:
frog = [1((xx)) (xx")
5

---

Published as a conference paper at ICLR 2023
Algorithm Squeeze Training (ST)
Input: A set of benign example and their labels S. number of training iterations 7, learning rate number of inner optimization steps K, perturbation budget e, step size a, and a choice of
regularization function Eegg.
Initialization: Perform random initialization for f.
fort 1, 1..... do
Sample a mini-batch of training data {x;; y) J
for = 1 ...., m (in parallel) do x + 0.001 (O, x f + 0.001 N
while 2 0 do
x. + argmax CE (E,)) Mi) x
argmin CE(f(%)) (x;).
_,(xx,x(x"))
Jinner = Creg (f (x(x)do), f(x(o))
X
+ [x (xqdu +a-sign( V xqde x" + II, (x) (x +crsigg( x(m
K K-1
end while
9i CE(f(x;;; yk) + 3. (f((dd)) f(x((a))
end for
θ +- θ 1/ D Vegi end for Output: A robust classifier / parameterized by O.
(3) Symmetric KL:
Erg 1 (KL(((()))X")))))))))) +LLC))
Among these choices, the (squared) and JS divergence already symmetric, and also adapt the original KL divergence to make satisfy the symmetry axiom of desirable metrics, such that the adversarial examples and collaborative examples are treated equally.
With the formulation in Eq. (6) pairs of collaborative and adversarial examples are obtained simulta- neously. The inner optimization are different from Eq. (T) and (4) The procedure is implemented as in Algorithm [ each step, the two examples are re-initialized by selecting from triplet based on their cross-entropy loss and updated using sign of gradients. The comparison of cross-entropy loss for re-initialization from the triplet is carried out just to guarantee the chained inequality in Eq. (6).
At each training iteration, we perform K update steps for the inner optimization, and
4.2 DISCUSSIONS
Some discussions about ST are given. We would like to first mention that, although Eq, (6) is par- tially inspired by TRADES, the collaborative ex- amples can generally and naturally be introduced to other adversarial training formulations.
Benign examples
2.0
Adversarial examples
Cross-entrops entropt
Collaborative examples
2.5
Then, we compare ST (which optimizes Eq. (6) to TRADES carefully. We know that the latter regularizes KI. divergence between benign exam- ples and their neighboring data points. neigh- boring data point whose output shows maximal KL divergence from the benign output, can be an
Cross-
1.0
0.5
Collaborattve Rate
20
60
80
100
120
50%
adversarial example or a collaborative example actually. In Figure we demonstrate the ratio of collaborative examples and the prediction loss of different examples along with the training using TRADES proceeds. It can be seen that the ratio of collaborative examples used for model train- ing decreases consistently, and being always less than 50%. Our ST aims to use 100% of the col- laborative and adversarial examples, if possible.
30%
60
80
100
120
Epoch
Figure 5: Changes in average cross-entropy loss of three types of examples along with TRADES training (upper), and changes in the ratio of col- laborative examples it utilized (lower). The exper- iment is performed with ResNet- 18 on CIFAR-10.
Comparing with TRADES, the Creg term in our ST imposes a stricter regularization, since the maxi- mum possible output discrepancy between any two data points within the e-bounded neighborhood

---

Published as a conference paper at ICLR 2023
51
51
51
50
Robusi Acturicy
-0
Robtust Accurscy
50
Rolwst Accuracy
50
49-
49
49
40
Robrst
48-
48
TRAGES
48
TAAE5
48
TINEES
TAADES Ub!
TIACES Byyreci
UBI
47
47
79
81
85
70
81
83
85
87
79
83
85
87
79
63
65
67
Clean Accuracy
Clean Accuracy
Clean Accuracy
Clean Accuracy
(a) Squared l2
(b) JS
(c) Symmetric KL
(d) Compare altogether
Figure 6: Performance of TRADES and ST with different choices of the regularization functions Ereg and the scaling factor 3. The results are obtain on ResNet-1 18 and CIFAR-10. The robust accuracy is evaluated using AutoAttack with e 8/255. For TRADES, we also try using the JS divergence, the squared le distance, and the symmetric KL divergence for regularizing suspicious and benign outputs, as reported in (a), (b), and (c). Top right indicates better trade-off. Best viewed in color.
is penalized, which bounds the TRADES regularization from above. In this regard, the worst-case outputs (i.e., adversarial outputs) and the best-case outputs (collaborative outputs) are expected to be optimized jointly and equally. Moreover, has been mentioned, since ST does not explicitly enforce the henign output probabilities to match the output of adversarial examples, we expect improved trade-off between robust and clean errors. Extensive empirical comparison to TRADES in Section will verify that our ST indeed outperforms it significantly, probably benefits from these merits.
Our ST adopts the same regularization loss for inner and outer optimization, and we also observed that, if not, moderate gradient masking occurs, i.e., higher PGD-20 accuracy but lower classification accuracy under more powerful attacks, just like in Gowal et al. (2020)'s. s.
EXPERIMENTS
In this section, we compare the proposed ST to state-of-the-ar methods. We mainly compare to vanilla AT (Madry et al. 2018). TRADES (Zhang et al. (2019), and MART (Wang et al., 2020).
Performance of other outstanding methods (Zhang al. [2020] Wu et al. 2020bb Kim al. 2021 Yu et al. 2022) will be compared in Section[5.3] as additional data may be used. Experiments are conducted on popular benchmark datasets, including CIFAR-10, CIFAR- 100 (Krizhevsky & Hinton 2009). and SVHN (Netzer et al. 2011). Table [able|[[smmmrrzze our main results, with ResNet (He et al. 2016b). We also test with wide ResNet (Zagoruyko & Komodakis, (016) to show that our method works as well on large-scale classification models, whose results can be found in Table Jand
worth noting that our ST can be readily combined with many recent advances, e.g., AWP (Wu et al. 2020b) and RST (Carmon et al. 2019). AWP utilizes weight perturbation in addition to input perturbation. We can combine ST with it by replacing its learning objective with ours. RST uses unlabeled data to boost the performance of adversarial training. It first produces pseudo labels for unlabeled data, and then minimizes a regularization loss on both labeled and unlabeled data.
Training settings. In most experiments in this section, perturbation budget of e = 8/255 and an inner step size a = 2/255, except for the SVHN dataset, where we
perform adversarial training with
1/255. In the training phase, we always of 0.9, a weight decay of 0.0005, and a batch size of 128. We train ResNet- (He et al.) (016a) for 120 epochs on CIFAR-I0 and CIFAR-I00 and we adopt an initial learning rate 0.1 and cut by 10x at the 80-th and 100-th epoch. For SVHN, we train ResNet-18 for 80 epochs with an initial learning rate of 0.01, and we cut by 10x at the 50-th and 65-th epoch. We adopt 9 6 for TRADES and / = 5 for MART by following their original papers. The final choice for the regularization function Ereg and the scaling factor in our ST will be given in Section 5.1 All models are trained on an NVIDIA Tesla 1100 GPU.
SGD optimizer with momentum
Evaluation details. We evaluate the performance of adversarially trained models by computing their clean and robust accuracy. For robust accuracy, we perform various white-box attack methods including FGSM (Goodfellow et al.) 2015), PGD (Madry et al. 2018), C&W's attack (Carlini & Wagner 2017), and AutoAttack (AA) (Croce & Hein 2020). Specifically, we perform PGD-20, PGD- 100, and C& W oo (i.e., the loo version of C& 's loss optimized using PGD-100) under = 8/255 and 2/255. Since adversarial training generally shows overfitting (Rice et al. 2020), select the model with the best PGD-20 performance from all checkpoints, as suggested in many recent
papers (Zhang et al. 2020 Wu et al. 2020b Wang et al. 2020 Gowal al. 2021
7

---

Published as a conference paper at ICLR 2023
Table Clean and robust accuracies of robust ResNet- 18 models trained using different adversarial training methods. The robust accuracy is evaluated under an loc. threat model with = 8/255. We perform seven runs and report the average performance with 95% confidence intervals.
Dataset
Method
Clean
FGSM
PGD-20
PGD-100
C&W 0c
AA
Vanilla AT
82.78%
$6.94 66.94%
51.30%
50.88%
49.72%
47.63%
1u.12%
10.17%
10.18%
10.26%
10.24%
1.0.8%
TRADES
82.41%
58.47%
52.76%
52.47%
50,43%
49.37%
MART
80.70%
58.91
54.02%
53.58%
49,35%
47.49%
CIFAR-10
ST (ours)
83.10%
+0.17%
59.51 %
+0.24%
54.62%
+0.29%
%
+0.30%
51.43%
=0.2716
50.50% %
+30
+0.10%
+0 220.
+0.14%%
+0.10%
#0.00%%
+0000%
TRADES+AWP
81.16%
57.86%
54.56%
54,45%
50.95%
50.31%
10.125
10.14%
10.006
10.14%
12%
10%
ST (ours)+AWP
82.53%
59.73%
55,56 %
55.47 %
52.05% 20 1e%6
51.23%
10.144%
1.0.14%
10-1335
13.133%
2.01 120%
Vanilla AT
57.27%
31.81%
28.66%
28.49%
26.89%
24.60%
TRADES
57.94%
32.37%
29.25%
29.10%
25.88%
24.71%
10.15%
1o. 1.9.18%
1.-110%
10.20%
1o 16%
1.000%
MART
55.03%
33.12%
30.32%
30.20%
26.60%
25.13%
CIFAR-100
ST (ours)
58.44%
+0.10%
33.35%
+- 2695
30.53%
+00.15%
30.39 %
+3.17%
26.70%
+0.119%
25.61 %
+0.16%
+0.12%
2:0 +50.25%
+00.15%
+-0.17%
2:0 20%6
+0007%
TRADES+AWP
58.76%
33.82%
31,53%
31.42%
27. 27.03%
26.06%
+0.00%
+0.15%
+0.14%%
+0..1%%
+0, 16.1
ST (ours)+AWP
59.06%
34,50 %
32.22%
32.16%
27.83%
26.86 %
20.00%
2:0 14%5
1:0-1495
1..115%
20 1176
2.0007
Vanilla AT
89.21%
59.81%
11.18%
50.35%
48.39%
45.96%
10.35%
1.0.29%
10.21%
1o 18%
10.01%
TRADES
90.20%
66.40%
54.49%
54.18%
52.09%
49.51
10.00%
2.0 1875
10013355
15.15%
2.0 10%
1.0.
MART
88.70%
64.16%
54.70%
54.13%
46.95%
44.98%
SVHN
ST (ours)
90.68%
200.00%
66.68% %
20 2402245
56.35% %
20.28%
56.00%
2.0.29%
52.57%
20. 24%
50.54%
10.17%
32%
TRADES+AWP
89.80%
66. 30%
59.01%
58.63%
54.72%
52.54%
+-0.21%
+0 10%
+00.20%
+0.22%
=0.119%
+0.12%
(ours)+AWP
90.77%
67.77% 20.90%%
59.95% 20.17%
59.76%
55.26
53.37% %
10.100
20.30%
1976
10.100%
5.1 COMPARISON OF DIFFERENT DISCREPANCY METRICS
To get started, we compare the three choices of discrepancy metric for freg. e.g., the JS divergence, the squared la distance, and the symmetric KL divergence. In Figure we summarize the performance of ST with these choices. We vary the value of scaling factor ,3 to demonstrate the trade-off between clean and robust accuracy, and the robust accuracy is evaluated using AutoAttack (Croce & Hcin 2020) which provides reliable evaluations. For fair comparison, original KL. divergence function being replaced with these newly introduced discrepancy functions and illustrate the results in the same plots (i.e., Figure 6(a) 6(b) and ((C) correspondingly. The performance curve of the original TRADES is shown in every sub-figure (in grey) in Figure 6 See Table 2 for all values in the figure. With the same freg. regularizations imposed by TRADES are
also evaluate TRADES with the
less significant thus we use 3 sets with larger elements.
From Figure 6(a) to b(c). one can that considerably improved trade-off between the clean and robust accuracy is achieved by using our ST, in comparison TRADES using the same discrep- ancy metric for measuring the gap between probability vectors. Moreover, in Figure ((dd] it can be seen that our ST with different choices for the discrepancy functions always outperforms the original TRADES by a large margin, and using the symmetric KL divergence for freg leads to the best performance overall. We will stick with the symmetric KL divergence for ST in our following comparison, and use B 6 for CIFAR-10, 3 4 for CIFAR-100, and 3 = 8 for SVHN.
Table 2: The scaling factor 3 for different discrepancy functions reported in Figure
frop
Method
sct
KL.
TRADES
(2,4,6,8 8. 10)
4
TRADES
24)
ST (ours)
6,
TRADES
[16. 32, 48)
IS
ST (ours)
(6. 12, 16)
Symmetric KI.
TRADES
[4.
16]
ST (ours)
(, 10)
5.2 COMPARISON TO THE--ARTSS
Table I reports the performance of our adversarial training method ST and its competitors. Intensive results demonstrate that ST outperforms the vanilla AT (Madry et al. 2018). TRADES (Zhang et al., 2019), and MART (Wang et al., (020) significantly, gaining consistently higher clean and robust accuracy on CIFAR- 10, CIFAR-TOM and SVHN. In other words, our ST significantly enhances
8

---

Published as a conference paper at ICLR 2023
adversarial robustness with less degradation of clean accuracy, indicating better trade-off between clean and robust performance. Specifically, on -IFAR-10, the best prior method shows classification accuracy of 49.37% and 82.41 %on the adversarial and clean test sets, respectively, while our ST with symmetric KI. achieves 50.50% (+1.13%) and 83. 10% (+0.69% (6%,,,,.. ). Combining with AWP (Wu et al. 2020b), we further gain absolute improvement of +0.92 % and +1.37% in robust and clean accuracy, respectively, compared to TRADES+AWP, on CIF Similar observations can be made on CIFAR-I00 and SVHN. Complexity analyses of our method is deferred to Appendix C and training curves are given in Figure 9to to demonstrate less overfitting than that in Figure
In addition to the experiments on ResNet-18. we also employ larger-scale DNNs, i.e. wide ResNet (WRN) (Zagoruyko & Komodakis (0166).
We train robust WRN models, including
robust WRN-34-5 and robust WRN-34-10 on CIFAR- 10, and we report experimental results
Table 3: Evaluation using WRNs on 10.AR-10. It can be seen that the effectiveness of our method holds when the size of DNN scales. PGD TRADES is the default evaluation in (Zhang et al. 2019).
in Table 3 Obviously, the WRN models lead to higher clean and robust accuracy than that of the ResNet-18. Importantly, our ST still outper- these networks, showing
forms competitors that the effectiveness of our method holds when the size of DNN model scales. Another WRN, Le., WRN-28-10, is also tested and the same observations can be made. We will test with it carefully in Section (5.3] in which additional unlabeled data is utilized during training.
Model
Method
Clean
PGPPRADDS
AA
WRN-34-5
TRADES
83.11%
55.78% 57.10%
51.33%
SI (ours)
83.14%
52.21%
TRADES
84.80%
56.65%
52.94%
WRN-34-10
MART
84.17%
51.10%
ST (ours)
84.92%
57.73%
53.54%
5.3 ADVERSARIAL TRAINING WITH ADDITIONAL DATA
2019) is recent work that confirms unlabeled data could also be properly incorporated into training for enhancing adversarial robustness. Here we consider a simple and direct combination with it. Recall that, in the RST paper, extracted 500K unlabeled data from 80 Million Tiny Images (Torralba et al. 2008). To utilize these unlabeled images, it generates pseudo labels for them, then performs adversarial training on a set including all CIFAR-10 training images and the originally unlabeled data. Our ST can easily be incorporated after obtaining the pseudo labels.
RST (Carmon et al.
We implement RST and our combination with it (called ST-RST) by following settings in the orig- inal paper of RST. Table 4 report empirical results. We then compare the performance of ST-RST to re- cent work that utilizes the same set of unlabeled data,
Table 4: Training WRN-28-11 10 with additional unlabeled data. All methods in the table use the same unlabeled data from RST's official GitHub repository. PGDRST the evaluation method in the RST paper.
i.e., GAIR-RST (Zhang et al. 2020), AWP-RST (Wu)
et al. 2020b), BAT-RST (Kim et al. (02T). and RWP. RST Yuu et al. 2002)) Their results are collected from official implementations. For RWP-RST, which is in
Method
Clean
PGDRST
AA
RST
89.66%
62.09%
59.54%
BAT-RST
89.61% %
59.54%
fact one of the best solutions in the same setting on RobustBench (Croce et al., 2020), it achieves robust accuracy of 60.36% by sacrificing clean accuracy, while our ST-RST gains +0.39% and +1.53% in ro-
GAIR-RS RST
89.36%
59.64%
AWP-RST
88.25%
60. 60.04%
RWP-RST
88.87%
60.36%
ST-RST (ours)
90,40%
63.55%
60.75%
bust and clean accuracy comparing to it.
6 CONCLUSION
In this paper, we have studied the loss landscape of DNN models (robust or not) and specifically paid more attention to the valley region of the landscapes where collaborative examples widely exist. We have verified that collaborative examples can be utilized to benefit adversarial robustness. In particular, we have proposed ST, squeeze training method, to take both adversarial examples and collaborative examples into accounts, jointly and equally, for regularizing the loss landscape during DNN training, forming a novel regularization regime. Extensive experiments have shown that our ST outperforms current state-of-the tteh-artss across different benchmark datasets and network architectures, and it can be combined with recent advances (including RST and AWP) to gain further progress in improving adversarial robustness.

---

Published as a conference paper at ICLR 2023
REFERENCES
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: query-efficient black-box adversarial attack via random search. arXiv preprint arXiv:1912. 000099 2019.
Muhammad Awais, Fengwei Zhou, Chuanlong Xie, Jiawei Li, Sung-Ho Bae, and Zhenguo Li. Mixacm: Mixup-based robustness transfer via distillation of activated channel maps. arXiv
preprint arXiv:2111.05077 2021.
Yutong Bai, Jieru Mei, Alan I. Yuille, and Cihang Xie. Are transformers more robust than cnns? Advances in Neural Information Processing Systems, 34, 2021.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šındić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases, pp. 387-402. Springer, 2013.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In S&P, 2017.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy Liang. Unlabeled data improves adversarial robustness. In NeurlPS, 2019.
Pin- Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 15-26. ACM, 2017.
Shuyu Cheng, Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Improving black-box adversarial attacks with transfer-based prior. In NeurIPS, 2019.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. arXiv preprint arXiv:1907.02044 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-fiee free attacks. In ICML, 2020.
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetri, Nicolas Flam- marion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670, 2020.
Sihui Dai, Saeed Mahloujifar, and Prateek Mittal, Parameterizing activation functions for adversarial robustness. arXiv preprint 1 10.05626, 2021.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In CVPR, 2018
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In /CLR, 2015.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03593, 2020.
Sven Gowal, Allvee Rebuffi, Olivia Wiles, Florian Stimberg, Dan Andrei Calian, and Timothy A Mann. Improving robustness using generated data, Advances in Neural Information Processing Systems, 34, 2021.
Yiwen Guo, Qizhang Li, and Hao Chen. Backpropagating linearly improves transferability of adversarial examples. arXiv preprint arXiv:2012.03528 2020.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image
recognition. In CVPR, 2016a.
10

---

Published as a conference paper at ICLR 2023
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In ECCV, 2016b.
Hanxun Huang, Yisen Wang, Sarah Erfani, Quanquan Gu, James Bailey, and Xingjun Ma. Explor- ing architectural ingredients of adversarially robust deep neural networks. Advances in Neural Information Processing Systems, 34, 2021.
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In JCML, 2018.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing, arXiv preprint arXiv:1803.06373, 2018.
Hoki Kim, Woojin Lee, Sungyoon Lee, and Jaewook Lee. Bridged adversarial training. arXiv preprint (txXiv:2108.11133. 2021.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical Report, University of Toronto, 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In ICLR, 2017.
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. In NeurlPS, 2018.
Qizhang Li, Yiwen Guo, and Hao Chen. Practical no-box adversarial attacks against dnns. Advances in Neural Information Processing Systems, 33, 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICML, 2018.
Xiaofeng Mao, Gege Qi, Yuefeng Chen, Xiaodan Li, Ranjie Duan, Shaokai Ye, Yuan He, and Hui Xue. Towards robust vision transformer. arXiv preprint arXiv:2105.07926, 2021.
Seyed-Mohsen Moosavi-
Alhussein Fawzi, Jonathan Uesato, and Pascal Frossard. Robust- via curvature regularization, and vice versa. In CVPR, 2019.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011, 2011.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, 2 Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Asia CCS, 2017.
Sayak Paul and Pin-Yu Chen. arXiv:2105.07581, 2021.
Vision transformers are robust learners.
arXiv preprint
Camilo Pestana, Wei Liu, David Glance, Robyn Owens, and Ajmal Mian. Assistive signals for deep neural network classifiers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1221-1225, 2021.
Sylvestre-Alvise Rebuffi, Sven Gowal, Dan A Calian, Florian Stimberg, Olivia Wiles, and Tim- othy Mann. Fixing data augmentation to improve adversarial robustness. arXiv preprint arXiv:2103.0190 2021.
Leslie Rice, Eric Wong, and J Zico Kolter. Overfitting adversarially robust deep learning. In ICML,, 2020.
Hadi Salman, Andrew Ilyas, Logan Engstrom. Sai Vemprala, Aleksander Madry, and Ashish Kapoor. Unadversarial examples: Designing objects for robust vision. Advances in Neural Information Processing Systems, 34, 2021.
Rulin Shao, Jinfeng Yi, Pin- Yu Chen, and Cho-Jui Hsich. How and when adversarial robustness transfers in knowledge distillation? arXiv preprint arXiv:2/ 10. /2072, 2021.
11

---

Published as a conference paper at ICLR 2023
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2013.
Christian Szegedy, Wojciech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Lue Tao, Lei Feng, Jinfeng Yi, and Songcan Chen. With false friends like these, who can notice mistakes? AAAI, 2022.
Antonio Torralba, Rob Fergus, and William T Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE transactions on pattern analysis and machine intelligence, 30(1 19:1198--9990 2008
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR, 2020.
Boxi Wu, Jinghui Chen, Deng Cai, Xiaofei He, and Quanquan Gu. Do wider neural networks really help adversarial robustness? arXiv preprint arXiv:2010.01279, 2020a.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust general- ization. Advances in Neural Information Processing Systems, 33, 2020b.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L. Yuille. Improving transferability of adversarial examples with input diversity. In CVPR, 2019.
Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan L Yuille. and Quoc V Le. Adversarial examples improve image recognition. Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 1998888, 2020a.
Cihang Xie, Mingxing Tan, Boqing Gong. Alan Yuille, and Quoc V Le. Smooth adversarial training. arXiv preprint arXiv:2006. 12166.145555 14536, 2020b.
Chaojian Yu, Bo Han, Mingming Gong, Li Shen, Shiming Ge, Bo Du, and Tongliang Liu. Robust weight perturbation for adversarial training. arXiv preprint arXiv:2205. 14826, 2022.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.07146 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing. Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019.
Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. Geaaetry-aware instance-reweighheed adversarial training. arXiv preprint arXiv:2010.01736, 2020.
Bojia Zi, Shihao Zhao, Xingjun Ma, and Yu- Yu-Gang Jiang. Revisiting adversarial robustness distillation: Robust soft labels make student better. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16443-16452, 2021.
12

---

Published as a conference paper at ICLR 2023
A PERFORMANCE OF MODELS IN FIGURE 1
Table 5: The clean and robust accuracy of models used in Figure 1, and the robust accuracy is evaluated by AutoAttack.
CIFAR-10 10, ResNet-18
CIFAR-I0, WRN-34-10
CIFAR-100, ResNet-18
CIFAR-100, WRN-34-10
Clean
Clean
Clean
Clean
Normal
95.09%
0.00%
95.28%
0.00%
76.33%
79.42%
0.00%
Vanilla AT
82.66%
47.62%
86.90%
48.31%
57.72%
24.67%
61.82%
25.39%
TRADES
82.51%
49.18%
84.80%
52.94%
57.99%
24.58%
57.10%
26.76%
MART
81.15%
47.28%
83.62%
50.93%
55.28%
25.15%
57.99%
27.12%
B COLLABORATIVE AND ADVERSARIAL. DIRECTIONS
We analyze the angle between collaborative perturbations and PGD adversarial perturbations. We summarize the experiments in Figure 7 After bunch of update steps, we observe that lies in limited range around 90", which is unsurprising in the high dimensional input space. However, for more robust models, we see that the angle deviate more from 90", indicating that powerful collaborative and adversarial perturbations become more correlated the robust landscapes.
Normal
Normal
Normal
400
200
200
200
200
50
10
00"
50
70
40'
50
50
Angle
Angle
Ardle
Angle
50
Angle
Ancle
Andle
Andle
(a) CIFAR-10, R8sNes-18
CIFAR-10. WRN-34-10
(c) CIFAR-IOD, ResNet-18
(d) CIFAR-100, WRN-34-10
Figure 7: The distributions of angles between the PGD adversarial perturbations and collaborative perturbations. At the first update iteration, collaborative example and its corresponding adversarial examples show opposite directions. However, after more and more update steps, since the gradient is computed w.r.t. different inputs, the adversarial and collaborative directions become less and less correlated and their angle finally lies in range around 90º. Interestingly. on a more robust model (TRADES). their correlation is more obvious and less concentrated around 90".
C COMPUTATIONAL COMPLEXITY OF ST
Since the adversarial examples and collaborative examples are both required in ST, the computational complexity in its inner optimization increases. Yet, we note that the two sorts of examples can be computed parallel, thus the run time of our ST can be similar or only slightly higher than that of the baseline. In fact, even if we cut the number of inner iteration steps of our ST by 2x to reduce its run time by roughly 2x, the performance of our method still satisfactory. shows robust accuracy (evaluated by AutoAttack) of 49.52% and clean accuracy of 84.37%, which already surpasses the performance of Vanilla AT, TRADES, and MART.
Furthermore, the performance of previous state-of-the-arts does not always improve with higher computational capacity (e.g., more inner optimization steps). For instance, TRADES show slightly better robust accuracy (AA: 19.76% + 0.09%) but decreased clean accuracy (81.57% + 0.14%) on CIFAR- with 2x more inner steps, which are both not better than our ST (AA: 50.50% + 0.07%, and clean accuracy: 83. 10% + 0.10%).
13

---

Published as a conference paper at ICLR 2023
We also consider generating two neighboring examples for each benign example and compute the mean/max of the regularization loss for out optimization for existing state-of-the-arts. Experimental results show that the mean of regularization is more effective, and T the ResNet results of all compared methods on CIFAR-10, EIFAR-100, and SVHN, with such an innovative formulation. We see that none of the methods (including the vanilla ATT, TRADES", and MARTT, where the symbol "I" indicates the utilization of two examples) improves consistently, comparing to the results in Table I Among all these competitors of ST, TRADES' performs the best. Table 6 demonstrates that its performance advances on CIFAR- 10 and CIFAR-100 but drops on SVHN in comparison to the TRADES results in Table With WRN architectures, improves on WRN-28- 10 (clean: +0.34%, AA: +0.42%) with additional unlabeled data, while drops on WRN-34-5 (clean: -0.02% AA: -..1%% and WRN-34-10 10 (clean: -0.71%, %,, AA: - 181% without.
In fact, such an innovative formulation involving two examples is more likely to incorporate collabo- rative examples into TRADEST, and this might be the reason why it works better with TRADESt than with the vanilla AT and MARTÍ, After all, combined with AWP or not, such innovative formulation involving two examples is still obviously inferior to our ST.
Table 6: Two neighboring examples are generated for each benign example in the inner optimization of the vanilla AT TRADES and MARTI, and the mean of regularization is adopted for the outer optimization. The robust accuracy is still evaluated under an Loc threat model with = 8/ 255.. We perform seven runs and report the average performance with 95% confidence intervals.
Dataset
Method
Clean
FGSM
PGD-20
PGD-100
C&W
AA
Vanilla ATI
82.72%
57.26%
51.51%
50.93%
49.91%
47.56%
TRADES
82.59%
58.77%
53.25%
52.96%
51.03%
49.47%
20.00
10.315
2016
20.30%
0000
MART
80.27%
58, 86%
54.36%
54.16%
49.48%
47.61%
CIFAR-10
10.139%
1u.25%
-0.27%
1.11311%
100.18%
240.0K
ST (ours)
83.10%
59.51
54.62% %
54.39%
51.43%
50.50% %
#0.10%
+:0 22%
=0 14%
10.110%
+0.00%
07%
TRADES+AWP!
81.43%
58.59%
55.13%
54.99%
51.64%
50.50%
ST AWP
82.53%
59.73%
55.56%
55,47 %
52.05%
51.23%
$0.1455
+0.14%
=0 18%
10.188
10.
20, 12%
Vanilla
56.57%
31.71%
28.52%
28.27%
27.11%
24.64%
10.11%
1.0 1.0.20%
=0 19%6
1.0.91%
10.18%
20 19%
TRADES
$7.99%
32.18%
29.99%
29.42%
25.90%
24.77%
+0.15%
+0.10%
=0
+00.10%
+0.31%
+0.17%
MARTI
55.06%
32.40%
30.52%
30.17%
26.42%
24.92%
CIF
1:0.925
=0
10.17%
ST (ours)
58.44%
33.35%
30.53%
30.39%
26.70%
25.61 %
19.12%
10.25%
33%
1.0.17%
10.200
1.0.7%
TWA
58.86%
34.11%
31.55%
31.64%
27.07%
26.17%
10.12%
10.17%
=0 13%
1.0.300
10.10%
20
ST (WP
59.06 %
34.50%
32.22%
32.16%
27.83%
26.86% %
Vanilla ATT
87.39%
58.66%
50.35%
49.44%
46.45%
44.26%
+0.15%
+0.17%
=0.26%
+-0.20%
+0.31%%
+0,
TRADES!
88.16%
63 63.23%
55.06%
54.54%
51.00%
48.81%
+0.15%
+0..25%
-0 20%
+-0.20%
+00.24%
#0.11
MART!
86.72%
62.23%
55.08%
54.43%
46.77 46.77%
44.78%
SVHN
17.177%
100
20 91%
1.0.95%
10-18%
20
ST (ours)
90.68% %
66.68 %
56.35%
56.00
52.57%
50.54%
TRADES+AWPI
90.32%
67.55%
58.67%
58.35%
$4.72%
52.67%
+0.20%
+0.27%
-0 -7.27%
+00.44%
+0.27%
+0,
ST (WP
90.77%
67.77%
59,95 %
59.76%
55.26%
53,37 %
19.119%
10.255%
20 17%
2.0.00%
10.100
20. 10%
14

---

Published as a conference paper at ICLR 2023
D TRAINING CURVES OF ST
Benign examples
Adversarial examples
examples
Cross
1.0
0.5
0.0
Epoch
Figure 8: How the average cross-entropy loss changes during training using our ST. The experiment performed with ResNet-18 on CIFAR-10. Shaded areas represent sealed standard deviations. can be seen that the gap between the collaborative examples and the benign examples and that between the adversarial examples and the benign examples are both obviously reduced, comparing to Figure 5. Best viewed in color.
85
35
60
80
2
30
3
65
15
Roblus
30
50
10
STorsum
20
40
60 Spoch
80
100
120
20
40
600
80
100
120
20
40
60 Epoch
80
300
120
20
40
60
80
100
120
Epoch
Esoch
(a) CIFAR-10
(b) CIFAR-100
Figure 9: Changes in clean and robust accuracy during training ResNet- on (a) CIFAR-10 and (b) CIFAR- 100, using TRADES and our ST. The robust accuracy is evaluated using PGD adversarial examples generated over 20 steps with = 8/255 and a step size a = 2/255. Best viewed in color.
50
20
63
35
120
150
200
120
Epoch
Epoch
Figure 10: Changes in clean and robust accuracy during training WRN-28-10 CIFAR-10, using RST and ST-RST. By following the original paper of RST, the robust accuracy is evaluated using PGD adversarial examples generated over 20 steps with 0.031 and step size of 0.007 on the first 500 images in the test set. Although we only train 200 epochs as suggested in the RST paper for comparison in Table 4 in our main paper, beneficial to our method. Best viewed color.
be that more training epochs still be
E TO MAKE "ADVERSARIAL" AND "COLLABORATIVE" CLEAR
In this paper and especially Section 4 we sometimes generalize the definition of adversarial examples to include all data points (in the bounded neighborhood benign examples) showing considerably higher prediction loss than that of the benign loss, in contrast to the definition in a narrower sense saying that different label prediction ought to be made. The collaborative examples are similarly "defined",, somewhat non-rigorously. Likewise, throughout the paper, abuse the word "adversarial" and "collaborative" to describe the spirit of achieving higher and lower loss than the benign loss, respectively.
15

---

Published as a conference paper at ICLR 2023
F
VISUALIZATIONS
Here we visualize some collaborative examples and collaborative perturbations in CIFAR-10, CIFAR- 100, and SVHN.
beaign examples
collaborative patuebation (normal)
collaborative example (normal)
collaborative perturtation (rohust)
collaborative examples (robast)
(a) CIFAR-10
henign examples
collahorative perturbation (normal)
collahorative examples (normal)
collahorative perturhation (robust)
collahorazive examples (robust)
(b) CIFAR-100
benign examples
collaborative perturbation (normal)
collaborative examples (normal)
collaborative perturbation (rubust)
exumples (robust)
(c)
CIFAR-100
Figure 11: Visualization benign examples, collaborative perturbations, and collaborative examples in (a) CIFAR-10, (b) CIFAR-I00 100, and (c) SVHN. The collaborative examples are crafted on normally trained ResNet-18 18 models and robust ResNet-18 models. Apparently, collaborative examples on robust and non-robust models strikingly different.
16


Reviews:
[{"review_id": "y61jwZPCgqq", "reviewer": "Reviewer_Yw6r", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The vulnerability of DNNs to adversarial examples is related to local non-smoothness and the steepness of loss landscapes. To solve the above problem, the main contribution of this paper is that they explore the existence of collaborative examples by simply adapting the PGD method by gradient descent rather than gradient ascent in the original PGD. They propose squeeze training (ST) by considering utilizing collaborative examples and adversarial examples jointly during training to regularize non-smooth regions of the whole loss landscape. Empirically, their method can outperform some SOTA methods under various attacks and datasets.", "summary_of_the_review": "Pls see the above comments.", "strength_and_weaknesses": "Strength:\n\n1. Using collaborative samples in adversarial training is novel and effective, and it can be used as a plugin for some data augmentation-based adversarial training methods.\n\n2. The proposed ST method is simple and effective, and the theoretical analysis is reasonable and correct.\n\n3. They conduct comprehensive experiments to evaluate the performance of the proposed ST.\n\n\nWeaknesses:\n\n1. P4, Sec. 3.2, Paragraph 3: Line 1 ‘using the method introduced in Section 3.1. Eq. (5)’ should be Eq. (4).\n\n2. One question is the ratio and sample size of adversarial examples and collaborative examples in ST. Is the total sample size for ST twice the sample size of regular adversarial training or is the sample size the same and adversarial examples and collaborative examples each account for 50%?\n\n3. P8, Sec. 5.1, Table 2: In the comparison between ST and TRADES, the scaling factor values are inconsistent. Can you explain why not adopt the same values of scaling factor, since ST and TRADES are based on similar ideas?\n\n4. The idea of ST in this paper is based on TRADES, and the generation of collaborative examples is based on the reverse operation of PGD. The idea is novel, but the innovation is slightly weak.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "gMIuxALaD53", "reviewer": "Reviewer_Q4iB", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a simple modification to existing adversarial training methods - rather than merely using adversarial examples that are meant to maximize classification loss, they propose to use collaborative examples which minimize the training loss. The training formulation minimizes a distance such as the symmetric KL divergence between the adversarial and collaborative examples to impose a stricter smoothness constraint when compared to existing methods. The method shows consistent gains over existing methods on CIFAR-10, CIFAR-100 and SVHN, and scales to WideResNet models as well. ", "summary_of_the_review": "The work presents a simple and intuitive change to existing adversarial training algorithms and shows improvements over existing methods. Some more ablations would ascertain the importance of collaborative examples further. Hence I recommend a borderline accept for now. \n\n--- Post Rebuttal update ---\n\nI thank the authors for the rebuttal and I would like to maintain my score. \n", "strength_and_weaknesses": "\nStrengths - \n- This work builds upon existing methods to propose a simple and intuitive change that shows consistent performance gains. \n- The observation that merely using collaborative examples can improve adversarial robustness is interesting. \n- The observation that TRADES training indeed results in the generation of collaborative examples in some cases is surprising and interesting. \n- The paper is well written and clear. \n\nWeaknesses - \n- The performance gains are marginal - hence it is important to perform more ablation experiments to ensure that the gains are indeed due to the use of collaborative examples. (some suggestions mentioned below)\n- Attack generation part of Algorithm-1 can be discussed in more detail. Specifically, how does this attack ensure max and min loss, rather than two examples which are far from each other? \n\nQuestions for rebuttal - \n- It is possible that collaborative example based training causes gradient masking and hence shows improved robustness. Could the authors share AutoAttack accuracy of the best epoch of each case in Fig.4? \n- Could the authors also compare with TRADES defense where CE loss is used in the inner maximization step, as discussed in [1]?\n- It is not clear how well the inner maximization can generate collaborative and adversarial examples given a common generation step. Could the authors compare the loss of the resulting examples with the loss obtained when CE loss is maximized or minimized independently?\n- It would be interesting to see what happens when adversarial examples are first generated by maximizing CE loss, and in a next step, collaborative examples are generated to maximize the KL divergence w.r.t. the adversarial examples. \n\n[1] Gowal et al., Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "RKLIqKnyhZ", "reviewer": "Reviewer_84B3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors study adversarially robust learning where the goal is to train a model that predicts correctly, even under small, worst-case perturbations of its inputs. The authors focus on “collaborative examples”: inputs close to natural ones that have been perturbed to induce very low loss. These examples are then incorporated into an adversarial training method which the authors find to slightly improve over existing ones. Essentially, the idea is: instead of minimizing the difference in prediction between the original example and its adversarial counterpart (e.g., TRADES), minimize the difference in prediction between any pair of points within the L-p ball around the original example.", "summary_of_the_review": "The main idea proposed is interesting and does provide non-trivial improvements on a number of datasets. However, the justification and analysis of the method could be significantly improved. \n\n---\nGiven the additional experimental results provided by the authors, I believe that the analysis is sufficient to support the claims of the paper. I thus increase my score.", "strength_and_weaknesses": "Strenghts:\n\n- The idea of minimizing the maximum prediction difference across arbitrary pairs of points within the Lp ball is interesting and, to the best of my knowledge, novel.\n- The experimental results do show a consistent improvement in the robustness and accuracy of the resulting model, even against strong attacks such as Auto-Attack.\n\nWeaknesses:\n\n- I did not find the discussion around “collaborative examples” particularly insightful. The existence of such examples is more or less implied by the existence of adversarial examples in the first place (if increasing the loss with small perturbations is easy, one would expect that decreasing the loss would be easy as well).\n- The motivation of the proposed method is not entirely clear. If the goal is to reduce the worst-case error, why does the best-case error have an effect? Is there a fundamental reason or does it happen to help make the loss smoother in practice?\n- Relatedly, the proposed method cannot be directly compared with existing approaches since (as the authors note in Appendix C) their method includes additional examples computed from the inner optimization loop. Comparing with other methods that use more inner steps is not a direct comparison. Instead, it would be useful to compare to variants of these methods that, say, compute two adversarial examples in the inner loop and train on the worst one or on both.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "PLkgYCw0H2", "reviewer": "Reviewer_46B8", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors tackle adversarial training. They pay more attention to the valley region of the loss landscape of DNNs. This is where collaborative examples exists. Collaborative examples fall within a bounded region of a benign example while having very small loss values. The authors first show that such examples can be directly used to improve the adversarial robustness of DNNs. Next they show that such examples in conjunction with adversarial examples (where they are both treated equally) can give an even higher boost to the robustness of DNNs. This is known as squeeze training as it \"squeezes possible prediction gaps and jointly regularizes the two sorts of non-smooth and steep regions in the loss landscape\". The authors compare their proposed method against multiple state of the art adversarial training methods. They also show how the choice of the regularization function impacts the robustness. Furthermore, they show that proposed method can also be extended to other adversarial training methods like RST.", "summary_of_the_review": "In this empirical paper, the proposed squeeze method shows impressive performance across multiple datasets. However, as mentioned above some further analysis could elevate this paper further.", "strength_and_weaknesses": "Strengths\n1. The paper is well written. The main idea of the paper is presented in an effective manner. This is followed by initial results with just collaborative examples, then squeeze training and others.\n2. The idea of using collaborative examples is an interesting one. The authors show that such examples alongside adversarial examples, not only improves robustness but also shows less degradation on the clean dataset. The results seem to hold across a large variety of datasets as well. \n3. Another strength of the proposed method is its extendability to other adversarial training methods. The authors show that this can be done easily and that it still yields better results.\n\nWeakness\n1. While I like the idea of collaborative examples presented in this paper, the paper can still benefit from an explanation regarding why such examples would be useful in adversarial training. From the perspective of the model these examples are \"easy\". So, in principle the model should not learn many useful information from such examples. However, we see that when used in conjunction with adversarial examples and the regularization function the results are impressive.\n2. In this work, the authors use the whole dataset when doing collaborative training. This should incur increased cost. I wonder if the whole dataset is necessary or not. Some analysis on this regard will also help shed further light to the nature of the collaborative examples.\n3. Extending on the previous point, I would have liked to see a further analysis on the cost of doing squeeze training. I noticed some analysis in the appendix. However, I would have liked to seen a more in depth analysis regarding cost vs gain.\n4. Similarly I would have liked to see more details about the experiments. I might have missed it but I did not see a lot of discussion about the split. In particular I want to know how the adversarial examples for the test cases were sampled.\n5. The authors can also consider adding imagenet dataset in their experiments.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster