Self-ensemble Adversarial Training for Improved Robustness
Authors: Hongjun Wang
Yisen Wang
Venue: ICLR 2022 Poster (2022)
Keywords: Adversarial Example
Adversarial Training

Abstract: Due to numerous breakthroughs in real-world applications brought by machine intelligence, deep neural networks (DNNs) are widely employed in critical applications. However, predictions of DNNs are easily manipulated with imperceptible adversarial perturbations, which impedes the further deployment of DNNs and may result in profound security and privacy implications. By incorporating adversarial samples into the training data pool, adversarial training is the strongest principled strategy against various adversarial attacks among all sorts of defense methods. Recent works mainly focus on developing new loss functions or regularizers, attempting to find the unique optimal point in the weight space. But none of them taps the potentials of classifiers obtained from standard adversarial training, especially states on the searching trajectory of training. In this work, we are dedicated to the weight states of models through the training process and devise a simple but powerful \emph{Self-Ensemble Adversarial Training} (SEAT) method for yielding a robust classifier by averaging weights of history models. This considerably improves the robustness of the target model against several well known adversarial attacks, even merely utilizing the naive cross-entropy loss to supervise. We also discuss the relationship between the ensemble of predictions from different adversarially trained models and the prediction of weight-ensembled models, as well as provide theoretical and empirical evidence that the proposed self-ensemble method provides a smoother loss landscape and better robustness than both individual models and the ensemble of predictions from different classifiers. We further analyze a subtle but fatal issue in the general settings for the self-ensemble model, which causes the deterioration of the weight-ensembled method in the late phases. 

Published as a conference paper at ICLR 2022
SELF-ENSEMBLA ADVERSARIAL TRAINING FOR IM- PROVED ROBUSTNESS
Hongjun Wang! Yisen Wang!a
Key Lab. of Machine Perception (MoE), School of Artificial Intelligence, Peking University 2 Institute for Artificial Intelligence, Peking University
ABSTRACT
Due to numerous breakthroughs in real-world applications brought by machine intelligence, deep neural networks (DNNs) are widely employed in critical applica- tions. However, predictions of DNNs are easily manipulated with imperceptible adversarial perturbations, which impedes the further deployment of DNNs and may result in profound security and privacy implications. By incorporating adversarial samples into the training data pool, adversarial training is the strongest principled strategy against various ad adversarial attacks among all sorts of defense methods. Recent works mainly focus developing new loss functions tempting to find the unique optimal point in the weight space. But none of them taps the potentials of classifiers obtained from standard adversarial training. especially states on the searching trajectory of training. In this work, we are dedicated the weight states of models through the training process and devise a simple but powerful Self-Ennemble Adversarial Training (SEAT) method for yielding robust classifier by averaging weights of history models. This considerably improves the robustness of the target model against several well known adversarial attacks, even merely utilizing the naive cross-entropy loss to supervise. We also discuss the relationship between the ensemble of predictions from different adversarially trained models and the prediction of weight-ensembled models, as well as provide theoretical and empirical evidence that the proposed self-ensemble method pro- vides a smoother loss landscape and better robustness than both individual models and the ensemble of predictions from different classifiers. We further analyze a subtle but fatal issue in the general settings for the self-ensemble model, which causes the deterioration of the weight-ensembled method in the late phases"
regularizers, at-
INTRODUCTION
Deep learning techniques have showed promise in disciplines such computer vision (Krizhevsky
ctt al. 2012 He et al. 2016), natural language processing (Vaswani et al.) 2017] Devlin et al. 2019).
speech recognition (Sak et al. 2015 Wang al.. (017) and even in protein structural modeling (Si &
Yan 2021 Jumper et al. 2022 D. However, even for those efforts surpassing human-level performance
in several fields, deep learning based methods are vulnerable to adversarial examples generated by adding small perturbations to natural examples (Szegedy al. 2014
Goodfellow et al.
2015).
Following the discovery of this adversarial vulnerability (Ma et al.
2020 Wang et al. 2020b
Niu
et al.] 2021), numerous defense approaches for protecting DNNS from adversarial attacks
have been
proposed (Papernot et al., 2016 Xu et al. 2018 Liao et al.
2018
Ross & Doshi- Velez
2018 Bai
al., 2019; Qin et al. 2020 Wang et al. 2020a Bai et al.
2021 Huang et al.) (021). Because
is a straightfrwwrrd and effective data-driven strategy, among these methods, adversarial training
(Madr et al. 2018) has become the de-facto way for strengthening the robustness of neural networks.
The core of adversarial training is proactively inducing a robust model by forcing DNNs to learn adversarial examples generated from the current model, which helps DNNs gradually make up for their deficiencies. Therefore, is a practical learning framework to alleviate the impact of adversarial attacks.
As a saying goes, a person may go faster but a team can go farther. An ensemble of classifiers fused by output aggregates the collective wisdom of all participated classifiers, which outperform the
"Correspondence Yisen Wang wwngen.wwnn@@eneeeee "ooee is available at atfhttss://gg / thub. www.w//o/wwww..ec.ccc....eeooc
Kin sembl Adver -laainning

---

Published as a conference paper at ICLR 2022
decision made by single classifier. Many ensemble methods Tramer et al. 2018 Pang al. 2019) have shown their abilities boost adversarial robustmess. Despite averaged predictions of different models enables the aggregation against a large number of attacks, it is still heavy computing burden to adversarially train all the member models. What is worse, the architectures and parameters of every member model, as well as their related contribution factors, should be safely saved and need performing forward propagation for every member repeatedly, which is a nightmare when dealing with massive components. Despite the fact that the ensemble technique adopts variety of architectures and parameters, these models may have similar decision decision-makinn making boundaries. Therefore, it is natural to wonder:
Could we develop the potential of the classifier itself to build robust model rather than rely on multiple seemingly disparate classifiers? Could we make use of states on the searching trajectory of optimization when performing adversarial training?
Motivated by these questions, discuss the relationship between an ensemble of different classifiers fused by predictions and by parameters, and present a imputation-friendll self-ensemble method named SEAT. In summary, this paper has the following contributions:
By rethinking the standard adversarial training and its variants, instead of developing well-designed loss to find single optimal in the weight space, we design an efficient algorithm called Self- Self-Ensemble Adversarial Training (SEAT) by uniting states of every history model on the optimization trajectory through the process of adversarial training. Unlike the traditional ensemble technique, which is extremely consuminn and repeating adversarial training procedures for individuals, SEAT simply requires training once.
We present theoretical explanation of the differences between the two ensemble approaches discussed above and visualize the change of loss landscape caused by SEAT. Furthermore, while using data augmentation of external data can experimentally fix the deterioration during the late stages (Gowal et al. 2020 Rebulfi al.) (02T), we further investigate subtle but fatal intrinsic issue in the general settings for the self-ensemble model, which causes the deterioration of the weight-ensembled method.
We thoroughly compare adversarial robustness of SEAT with other models trained by the state-of-the-ut techniques against several attack methods on CIFAR-10 10 and CIFAR-100 datasets. Results have shown that the SEAT method itself can efficiently search in the weight space and significantly improve adversarial robustness.
2 REVIEW: STANDARD ADVERSARIAL TRAINING AND ITS SUCCESSORS
In this section, we first introduce basic concepts of adversarial training. Then we briefly review some variants and enhancements of standard adversarial training.
2.1 PRELIMINARIES
Consider standard image task which involves classifying an input image into one of C' classes, let x E [0, 11"h denote the input natural image and E 21,2..... C} be the ground truth label, sampled from a distribution D. For adversarial attack, the goal of the adversary is to seek malignant example in the vicinity of x, by adding a human imperceptible perturbation E E R". To exactly describe the meaning of "imperceptible" this neighbor region IB. (x)) anchored at x with apothem can be defined
as 1B = ((x', v) E D || I'1l < }). We also define a loss function: { def θ x ID + 10, oc)
which is usually positive. bounded, and upper-semi continuous (Blanchet & Murthy 2019 Villani 2003 Bartlett& & Mendelson, 2001) for all E O Instead of the original formulation of adversarial training (Szegedy et al., [0144) Goodfellow et al.] 2015). which mixes natural examples and adversarial examples x' together to train models, the adversarial risk minimization problem can be formulated as a two-player game that writes as below:
inf E (E.a))D
sup l (0:x', (x)
Outer minimization Inner maximization
In general, the adversary player computes more and more harmful perturbations step by step, where the PGD method (Madry al. 2018) is commonly applied:
(2)(2) (x' + ssign (Va((s,))) v)))
(2)
2

---

Published as a conference paper at ICLR 2022
where is the step size and the generated samples each iteration will be projected back to the (x). When finding x' at their current optimal in the inner maximization, the model parameters will be updated according to these fixed samples only in the outer minimization.
2.2 RELATED WORK
Emerging adversarial training methods are primarily concerned with strengthening their robustness through empirical defense. Here we roughly separate these improvements into two categories: (1) sample-oriented (i.e. for x' in the inner maximization) and (2) loss-oriented (i.e. for & in the outer minimization) improvement.
From the side of the former, BBluja Fischer (2017) utilize a GAN framework to create adversarial
examples via the generative model. Wang et al. (2019) proposes a criterion to control the strength of
the generated adversarial examples while FXT Zhang et al. 2020 control the length of searching
trajectories of PGD through the lens of geometry.
For the latter, a stream of works have been carried out with the goal of establishing a new super- vised loss for better optimization. Usually, the loss in the both inner maximization and outer m are cross-entropy defined as: fee -y log when given the probability vector (efo(=");
efo(4")= ////// eJo(z')s, The addition of regularizer to the vanilla loss is one branch of the modification. Representative works like ALP (Kannan et al. 2018), TRADES (Zhang et al. [ [2019), and VAT (Miyato et al.] 2019) introduce a regularization term to smooth the gap between the probability output of natural examples and adversarial ones, which can be denoted as:
Preg = lee +
Σ (21.14))
R(p(O; 2/ 91). v:). p(0; x0; 3)),
(3)
where is the regularization hyper-parameter and R(:,.) :) is l2 distance for ALP and VAT and Kullback-Lese divergence for TRADES. B stands for the sampled minibatch. Another branch of the loss modification is to reweight the contributions of each instance in the minibatch based on their intrinsic characteristics. This can be formulated by:
Crew
w(O: Di, thy Lce 20; v1)
(Z..8))
where w(-) is a real-valued function mapping the input instances to an importance score. MMA (Ding eet al. 2020) separately considers the correct and incorrect classification natural cases and switches between them by learning a hard importance weight:
MMMA
Σ
wi w0 Fi, (0; Eoe (0; (0; yi) + w2 Wi, Me Eoe (0; 2/e y:).
(5)
(17-34))~5
where Wy and W2 the indicator function 1 (fo (x,) * and (fo (z,) y:). respectively. Inspired by the geometry concern of FAT, GAIRAT (Zhang et al., 2021 evaluates w(0; xr, y) (1 + tanh(c; 10c₂ + 5))/2 to adaptively control the contribution of different instances by their corresponding geometric distance of PGD, where e₁ is tlype--paaaeete and c2 is the ratio of the minimum successful iteration numbers PGD to the maximum iteration step. Embracing both two schools of improvement, MART (Wang et al., 2020c) adds a new regularization term apart from KI. divergence and explicitly assigns weights within regularizers:
ÉMART = lee + Σ
w(O; F, U4 Y,) R (RRL(p((;; v.), (0;;,,,))) + Rmag (p 2 W:)),
(6)
(B..
where RILE is Kullback-Leibler divergence same as TRADES and w, = (1 p(0: æi 11)) is softer scheme when compared with MMA. The margin term Rmeg = log (1 max Ph (0: 2/, y:)) aims at improving the decision boundary.
The above methods endeavor to solve the highly non-convex and non-concave optimization in Eqn However, they assume that the optimal weights of classifier appear its prime and abandon all the states along the route of optimization, which are beneficial to approach the optimal.
Forr simplification, here we ignore the implemented difference of loss in the inner maximization and the outer minimization (c.g. TRADES and MMA). The reference of can he contextually inferred easily. Forr VAT, in the first term and the second term different.
3

---

Published as a conference paper at ICLR 2022
Algorithm 1 Self-Ensemble Adversarial Training (SEAT)
Input: A DNN classifier fo(.) with initial learnable parameters Ou and loss function é data distribution D; number of iterations N; number of adversarial attack steps K; magnitude of perturbation step size learning rate exponential decay rates for ensembling og constant factor c.
Initialize 0 <- 0u. 0 4- 0.
for t < 1, ......... N....... do Sample minibatch (x, y) from data distribution TD> 26 f x +
Uniform(-e,e),
for k <- 1,2,
do
z' 1 cb, (nsign (ok-11 V (()
2(0:
end for
go. + E( Ve. (2%, e)]] in every minibatch
Calculate according to current iterations 0: 6 0, T.go,
a' + min a,
+- d'ó + (1 a')o. end for
Return A self-ensemble adversarial training model fa(-)
METHODOLOGY
In this section, we first discuss the traditional ensemble methods based on predictions of several individual classifiers. Then, we propose a Self-ensemble Adversarial Training (SEAT) strategy fusing weights of individual models during different periods, which both intrinsically and computationally convenient. Following that, we further provide theoretical and empirical analysis on why the prediction of such an algorithm can make great difference from that of simple ensembling classifiers.
3.1 PREDICTION-ORIENTE ENSEMBLE
Ensemble methods are usually effective to enhance the performance CCrruana et al. 2004 Garipov ct al. 2018) and improve the robustness (Tramèr et al. 2018 Pang et al. 2019). For clarity, we denote aa pool of candidate models where f= Uoo: ... Jo. J. To represent the scalar output of the averaged prediction of candidate classifiers over C' categories, we define the averaged prediction f involving all candidates in the pool F as:
fz(2.0) = BrSo, (2.
1=1
n
8.. Σ
3,
where 3 represents the normalized contribution score allotted the candidates. it logical to assume that v33 > O since each candidate model must well-trained, otherwise it will be expelled from the list. Note that here we only discuss whether the participated model performs well on its own, not whether has a positive impact on the collective decision making, which is a belated action.
3.2 SELF-ENSEMBLE ADVERSARIAL TRAINING
In order to average weights of a model, it natural choose an analogous form as Eqn 7 to obtain the predicted weights: θ = E 3,0,, s.t. Σ 31 1. However, such a simple moving average cannot keenly capture the latest change, lagging behind the latest states by half the sample width. To address this problem, we calculate the optimal weights by characterizing the trajectory of the weights state as an exponential moving average (EMA) for measuring trend directions over period of time. Intuitively, EMA not only utilizes recent proposals from the current SGD optimization but also maintains some influence from previous weight states:
0- = aθT- + OT
T
(8)
= Σ (1 a) 1--((11) 2-6(1 T- T- 1=1
4

---

Published as a conference paper at ICLR 2022
where 8(.) is the unit impulse function (i.e., 5(0) 1, otherwise it is 0). Algorithm summarizes the full algorithm.
In consideration of both using the moving average technique, we wonder whether the averaged prediction has a link the prediction of a weight-averagee model. In fact, they are indeed related to each other to some extent:
Proposition 1. (Proof in Appendix B) Let So (-) denote the predictions of neural network parametrized by weights 9. Assuming that ve E ④. fo (-) is contimuous and (Iz E D. fo(x. is at least twice differentiable. Consider two points 0. E è in the weight space and let Ç = 02 0 for t (1,2, ... ., the difference between fz(x, y) and fg (z, y) is of the second order of smallness if and only if Et
Based on Proposition 1 we could immediately obtain the below conclusions:
Remark 1. Note that it always constructs ensembles of the well-matche enhanced networks to obtain stronger defense, so it evenly assigns 31 = =B
3n, 1/n. However, things change in the self- ensemble setting since models obtained at relatively late stages will be much robust than the beginning.
Based this assumption, it has non-decreasing sequence B1 S 33 S .... 8.,..l. The inequality tight only when the initial weight reaches its fixed point.
8.l. E B:
In this case, the averaged prediction will predispose to the models obtained at the end of optimization and such a predisposition loses the superiority of ensemble due to the phenomenon of homogenization in the late phases. To provide an empirical evidence of homogenization of models at latter periods, visualize the effect of homogenization along the training process in Figure Ia We define the homogenization of model as: A. = Tpi Σ min:e(1.mm] lff, (I, y) fo. (z. v)l. used for calculat- ing the difference of the output of history models over period of time m. Note that Ae becomes smaller and smaller along with epochs passing, which proves the existence of homogenization.
100
SPAT
AI
0.8
Accuracy Rate (%)
0.6
0.4
0.2
Nita to 2011
20
40
60
0
20
40
60
S0
100
120
Epochs
Epochs
(a) The effect of homogenization of models.
(b) Robust accuracy of paraaeder-averaaed ResNet18 against various attacks CIFAR-10. Figure 1: The overall analyse of AT and SEAT.
Here we formally state the gap between such averaged prediction and the prediction given by SEAT in the following theorem:
Theorem 1. (Proof in Appendix B) Assuming that for j, E (1..... ,T) 0, 0, if and only if The difference between the averaged prediction of multiple networks and the prediction of SEAT is of the second order of smallness if and only if 3, (1- - T for (1.2.....TT
Theorem tells us that it is much harder for SEAT to approximate the averaged prediction of history networks than the EMA method. So the prediction of SEAT keeps away from the averaged prediction of models, which suffers from homogenization. To ensure whether the difference in Theorem is benign, provide empirical evidence of the improvement of the self-ensemble technique the loss landscape.
is difficult for the traditional 1D linear interpolation method to visualize non-convexities (Goodfel low & Vinyals 2015) and biases caused by invariance symmetries (e.g. batch normalization) the DNN. et al. (2018): Wu al. (2020) address these challenges, providing a scheme called filter normalization to explore the Starpness/llttees of DNNs on the loss landscape. Let Uj and Vy are two random direction vectors sampled from random Gaussian distribution. So we plot the value of
5

---

Published as a conference paper at ICLR 2022
(a) The best individual
(b) SEAT
(c) Deteriorated SEAT
Figure 2: The loss surfaces of R on CIFAR-10 10 test data. The loss surface of the standard adversarially trained models transitions up and down, which means the loss of such a model changes dramatically moving along some directions. In contrast, SEAT has a fairly smooth landscape.
loss function around when inputting data samples:
(9) and -|I|| denotes the Frobenius norm. Specifically, we apply the method to plot the surface of DNN within the 3D projected space. We visualize the loss values ResNet18 on the testing dataset of CIFAR-10 in Figure [ Each grid point in the x-axis represents a sampled gradient direction tuple (w,, U2). It is clear that SEAT has smooth and slowly varying loss function while the standard adversarially trained model visually has a sharper minimum with higher test error.
L(O; v1 02) = e {O + me1 + nu2; v)
where m
e
| II Teeg
Safeguard for the Initial Stage: During the initial training period, the history models are less instrumental since they are not adequately learned. Therefore, is not wise to directly make use of the ensembled weights, which may accumulate some naive weight states and do harm to approach the oracle. Therefore, we apply the protection mechanism for novice models at the beginning, denoted
as a' = min a, s+c) where e is constant factor controlling the length of warm-up periods for history models.
3.3 DETERIORATION: THE DEVII. IN THE LEARNING RATE
Having provided both theoretical and experimental support as well as a clear procedure of SEAT algorithm, we may take for granted that this self-ensemble method could be integrated into the routine working flow as an off-shelf technique. But it is not the case.
As shown in Figure 2b and 2c] while the two self-ensemble models have almost nearly identical settings, their loss landscapes are completely differrntt (quantitative results can be referred to Appendix A.4). Despite the fact that the landscape of deteriorated SEAT little mild than the standard adversarially trained model, it is still extremely steep when compared with SEAT. This phenomenon about deterioration is also discovered by Rebuffi al. (2021). They combine model weight averaging with data augmentation and external data to handle the deterioration. The intention behind such a solution is robust overfitting (Rice et al. 2020) of individuals. However, it is unclear why an ensemble model assembled from overlitting individuals performs poorly, while inferior models can still be aggregated to produce a superior one in the middle of training stages. the rest of this section, we provide another solution from quite new perspective: the strategy of learning rate.
Proposition 2. (Proof in Appendix B) Assuming that every candidate classifier is updated by SGD- like strategy, meaning 0++1 0, 7. Vo, Jo, (x', ()' with 2 2 ... 2 TT > 0, the performance of self-ensemble model depends on learning rate schedules.
As illustrated in Figure tipping point (e.g. Epoch 76, 91 and 111) when using the staircase strategy of learning rate. Combined with the trend of robust accuracy in Figure [b] we find that the self-ensemble model cannot gain any benefits when the learning rate decays below certain value. More results on different learning rate schedules can be found in Figure Bal
we note that the phenomenon of homogenization exacerbates after each
4 EXPERIMENTAL RESULTS
In this section, we first conduct set of experiments to verify the advantages of the proposed method. Then, investigate how each component of SEAT functions and how it affects robust accuracy.

---

Published as a conference paper at ICLR 2022
4.1 EXPERIMENTAL SETUP
We mainly use R8 and WRN-32-10 (Zagoruyko & Komodakis 2016) for the experiments on CIFAR-IOCIFAR-- 100 and all images are normalized into [0, II. For simplicity, we only report the results based on Loo norm for the non-targeted attack. We train ResNet18 using SGD with 0.9 momentum for 120 epochs and the weight decay factor is set to 3.5e 3 for ResNet18 and 7e for WRN-32-10. For SEAT, we use the piecewise linear learning rate schedule instead of the staircase one based on Proposition 2 The initial learning rate for ResNet18 is set to 0.01 and 0.1 for WRN-32-10 till Epoch 40 and then linearly reduced to 0.001, 0.0001 and 0.01, 0.001 at Epoch 60 and 120, respectively. The magnitude of maximum perturbation at each pixel is = 8/255 with step size = 2/255 and the PGD steps number in the inner maximization is 10. To evaluate adversarial defense methods, we apply several adversarial attacks including PGD (Madry et al., 2018), MIM (Dong al. 2018), CW (Carlini & Wagner (017)) and Auto (AA) (Croce & Hem 2020). We mainly compare the following defense methods in our experiments:
TRADES, MART, FAT and GAIRAT:
For TRADES and MART, we follow the official implementation of MART to train both two models for better robustness. The hyper- parameter of both TRADES and MART is set to 6.0. The learning rate is divided by 10 at Epoch 75, 90, 100 for ResNet18 and at 75, 90, 110 for WRN-32-10, respectively. For FAT and GAIRAT, we completely comply with the official implementation to report results.
We also perform the experiments on the Prediction-ories oriented Ensemble method (PoE). We select the above four advanced models as the candidate models. When averaging the output of these models, we assign different weights to individuals according to their robust accuracy. Specifically we respectively fix 31,2,,4 = 0.1, 0.2, 0.3, 0.4 for FAT, GAIRAT, TRADES, and MART, named PoE. Considering the poor robust performance of FAT and GAIRAT, we also provide the PoE result by averaging the outputs of TRADES trained with x= 1, 2, 4, 6, 8 and assign the same 3 to them, named POE (TRADES).
PoE:
CutMix (with WA): Based on the experimental results Rebutii et al. (222)), Cutmix with a fixed window size achieves the best robust accuracy. We tollow this setting and set the window size to 20. Since we do not have matched computational resources with Deepmind, we only train models for 120 epochs with batch size of 128 rather than 400 epochs with batch size of 512. Besides, we neither use Swish/SiLU activation functions (Hendrycks & Gimpel 2016) nor introduce external data and samples crafted by generative models (Brock et 2019 Ho et al. 2020 Child 202I for fair comparison.
4.2 ROBUSTNESS EVALUATION OF SEAT
In this section, we fully verify the effectiveness of the SEAT method. We report both average accuracy rates and standard deviations. All results in Tables | and 2 are computed with 5 individual trials. Results on R are summarized in Table Here. we mainly report the results on the CIFAR- 10 dataset due to the space limitation. For the results on CIFAR-100, please refer to Appendix A.2. From the table, candidate models supervised by the traditional cross-entropy loss without external data. Compared with two advanced loss-oriented methods (TRADES and MART), the results of SEAT are at least 2% better than TRADES and MART against CW and AA. We further emphasize that although the boost of SEAT regarding PGD attacks is slight when compared with GAIRAT, SEAT achieves startling robust accuracy results when facing CW and AA attacks. Note that AA is an ensemble of various advanced attacks (the performance under each component of AA can be found in Appendix A. and thus the robust accuracy of AA reliably reflects the adversarial robustness, which indicates that the robustness improvement of SEAT is not biased. It's worth noting that PoE is unexpectedly much weaker in defending AA attack even than its members. We guess the poor performance of FAT and GAIRAT encumber the collective though we appropriately lower the contribution score of them. If we switch to ensemble of the outputs of TRADES with different As, the performance will be slightly improved. That demonstrates that the robustness of every candidate model will affect the performance of the ensemble. The negligible computational burden estimated in Appendix A.5 also demonstrates the superiority of our on-the-fly -ly parameter-averaged method of SEAT.
see that the superiority of SEAT is apparent especially considering
only use
We also evaluate the SEAT method on WRN-32-10. 10. Results are shown in Table 2 We notice that the superiority of SEAT appears to be enhanced when using WRN-32-10, The gap between SEAT
hhtp://ggiihbbboom//iisaaaaaaaa htwpyy//.thnooooooooooooooooomm.
7

---

Published as a conference paper at ICLR 2022
Table I: Comparison of algorithm with different defense methods using ResNet18 CIFAR-10. The maximum perturbation E = 8/255. Average accuracy rates (in %) and standard deviations have shown that the proposed SEAT method greatly improves the robustness of the model.
Method
NAT
PGD
PGD 100
MIM
CW
AA
AT
14.32
10.19
0..1
4.04
10.5
0.337
TRADES
83.91+0.33
54.25+0.11
52.21+0.09
55.65+0.1
52.22+0.055
48.2+0.2
FAT
87.72.1..14
46.6911.31
46.8110.3
47.03.10.17
49.66.10.88
43.14_1.433
MART
83.12-10.23
55.4310.16
53.46:10.24
57.06-10.22
51.45.1.299
48.13-10.31
GAIRAT
83.4+-0.11
54.76-t0.42
54.81+0.63
53.57-00.11
38.71+0.26
31.25-0044
PoE
85.41+0.29
55.2+0.37
55.07=00224
54.33-0032
19.25=0.16
46.17=0.35
POE (TRADES)
83.57#0.31
53.88+0045
53,82=0.27
55.01=0.188
52.72=0.65
49.2=0.24
CutMix (with WA)
81.261:0.44
52.7710.33
52.55:0.225
53.01=0.255
50.01
47.3820.36
SEAT
83.
56,02:1011
55.97=0007
57.13=0.12
5..38
51.3=0266
SEAT+CutMix
81.53+0.31
55.3+0.27
54.82+00.18
56.41=00117
53.83+0.31
49.1
Table 2: Comparison of algorithm with different defense methods using WRN-32-10 CIFAR-
10. The maximum perturbation is 8/255. Average accuracy rates (in %) and standard deviations have shown that SEAT also shows a great improvement on robustness.
Method
NAT
PGD
PGD 100
MIM
CW
AA
AT
87.32.0.21
49.01=:0.33
48,8310.27
08.25=0.17
52.8=0.25
48.17=0.48
TRADES
85.1110.77
54.5810.49
54.82-10.38
55.67-0.31
54.9111.21
52.19-0.44
FAT
89.6510.04
48.7411.23
48.69-10.18
48.24.10.16
52.1110.711
46.7.104
MART
84.26++0.28
54.1110..88
54.13+10.3
55.2+-0.22
53.41+0.17
50.2-0036
GAIRAT
85.92++0.69
58.51+0.42
58.48-0.34
58.37-0.277
44.31=0.22
39.64=1.01
PoE
87.1=0.25
55.75+002
55.47=0.19
56.04=0.31
53,66=0.18
49.44=0.35
PoE (TRADES)
86.03-10.37
54.26-10.47
54.73--0.21
55.01=0.222
55.52+00118
53.2+0.4
CutMix (with WA)
82.79+H0.44
58.43+1.21
58.2+-0.33
58.95+0.557
58.32+0043
54.1+0.82
SEAT
H0.42
59.84+0022
59.8+0.16
60.87+0.11
58.95+0034
55.67=0.22
SEAT+CutMix
84.8110.118
60.2.10.16
60.31
60.53
59.46.10.24
56.03
and advanced defense methods enlarges to at least 4%. In the setting with data augmentation, when compared with the results ResNet18, SEAT combined with CutMix gains much higher performance on robust accuracy against several attacks as the model becomes larger. The observation given by RRebufi et al. (2021) shows that additional generated data would overload the model with low capacity. When comes to ours, we can further infer from a marginal and even a negative effect shown in Table that the model with low capacity cannot benefit from both external and internal generated data. This phenomenon springs from the fact that CutMix techniques tend produce augmented views that are far away from the original image they augment, which means that they are little hard for small neural networks to learn.
4.3 ABLATION STUDY
In this section, perform several ablation experiments investigate how different aspects of SEAT influence its effectiveness. If not specified otherwise, the experiments are conducted on CIFAR-I0 using ResNet18.
110
(4))
20
80
500
Epoche
Epocits
(a) Different learning rate strategies
(b) Epoch Iteration
(c) Length of initial safeguard
Figure 3: The overall ablation study of SEAT. The dashed line represents the natural accuracy while the solid line represents the robust accuracy.
Different Learning Rate Strategies. The staircase learning rate technique is always applied to update weights in commonly used neural networks, which remains unchanged within the specific
8

---

Published as a conference paper at ICLR 2022
epochs and suddenly decreases to certain value at some points. We also have the cosine, cyclie, and warming up learning rate schedules compare with, addition to the staircase and piecewise linear one deployed to SEAT. Figure [ depicts the results of this experiment using various learning rate methodologies. The natural aceuracy under various learning rate schedules is reported by the top dashed lines, which all grow gradually over time. After reaching their maxima, the relative bottom solid lines fall more or less. Quantitative Results of above five strategies from best checkpoints are shown in Appendix A.3. However, it can be observed that the red line (piecewise linear) and the blue line (cosine) decrease more slowly than the orange line (staircase) and they achieve higher robust accuracy at much later epochs than the staircase. Apparently, the piecewise linear strategy makes approach the threshold Ty at a slow pace while the cosine one increases the valid candidate models that can make contributions. Likewise, the cyclic strategy gets better robust accuracy and further improves the performance in the late stage. But the effect of warming up learning rate is marginal since the homogenization of the candidate models we mentioned Sec. 3.2 cannot be fixed by the warmup strategy. These facts bear out Proposition 2 from the side.
Epoch- based v.s. Iteration-basee ensembling has great effect on the robustness of the collective. Specifically, we select two scenarios to compare with. One is an epoch-based ensemble that only gathers the updated models after traversing the whole training dataset. The other is an iteration-based ensemble collecting models whenever SGD functions. Comparison between the epoch-based is shown in Figure 3b It can be seen that the curves of both two schemes are nearly the same in their prime except that the ivcaationbbase method has slightly greater natural accuracy and degrades more slowly.
Ensembling. Next, we investigate whether the frequency of
and the iteration-bar -based ensembling
Influence of the Initial Safeguard. We also study the effect of the length of the initial safeguard. We plot all results against different kinds of attacks with e ranging from 0 10000 (i.e. Epoch 32). The curves in Figure [c show that the model ensembled with large c, which ignores more history models, results in weaker memorization and poor robustness. Actually, from the perspective of Proposition 2 fixed if the scheme of learning rate updating and the total training epochs are unchanged. So using a large is equivalent to squeeze the number of benign candidates before t'. Similarly, a small can manually filter the early models without being well trained. However, such a drop is minor because EMA has a built-in mechanism for adjusting.
Mixed Strategies with SEAT for Improved Robustness. Finally, we report the results under differ- mixed strategies CIFAR-10 10 dataset with ResNet18 in Table Overall, SEAT+TRADES and TEATHTRADES+CUMMI do not perform well. We believe the reason that TRADES cannot well adapt to learning strategies other than the staircase one for its history snapshots of TRADES trained under other strategies are much inferior than the ones trained under the staircase learning rate strategy. However, MART is in harmony with other strategies. The combination of JEAT-MART+CumMIs achieves the best or the second-best robustness against almost all types of attacks.
Table 3: Average robust accuracy (%) and standard deviation under different mixed strategies on CIFAR-10 dataset with ResNet18.
Method
NAT
PGD20
PGD100
MIM
CW
APGICE
APGDDLR
APGLT
FABT
Square
AA
SEAT
#0.13
83.7
36,02
=0.11
33.97
10.07
10.12
57.13
54.38
16.1
10.17
53.87
33.35
5038
51.41
57.77
s0.36
$1.3
81.21
57.05
50.75
49.01
+TRADES
&MART
60.14
40144
SEAT
78.22
57.14
50.91
50.01
STRADES-CuMix
40.33
-0.21
10.45
10.39
10.41
10.82
20.17
20.29
:0446
20.44
SEAT
75.87
57.3
$7.29
57.47
53.13
54.33
$3.98
51.3
51.42
56,11
52.1
AMART+CaMi
+0.24
10.16
+0.43
+0.18
+0.2
20.33
#0.61
#1.73
+0.17
=008
s0.22
5 CONCLUSION
In this paper, we propose a simple but powerful method called Seff-Ensemble Adversarial Training (SEAT). which unites states of every history model on the optimization trajectory through the process of adversarial training. Compared with the standard ensemble method, SEAT only needs training once and has a better reusability. Besides, we give theoretical explanation of the difference between the above two ensemble methods and visualize the change of loss landscape caused by SEAT. Furthermore, we analyze subtle but fatal intrinsic issue in the learning rate strategy for the self-ensemble model, which causes the deterioration of the weight-ensembled method. Extensive experiments validate the effectiveness of the proposed SEAT method.

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGEMENT
Yisen Wang is partially supported by the National Natural Science Foundation China under Grant 62006153, Project 2020BD006 supported by PKU-Baidu Fund, and Open Research Projects of Zhejiang Lab (No. 2022RC0AB05)
REFERENCES
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: query-efficient black-box adversarial attack via random search. ECCV, 2020.
Yang Bai, Yan Feng, Yisen Wang, Tao Dai, Shu-Tao Xia, and Yong Jiang. Hilbert-based generative defense for adversarial examples. In ICCV, 2019.
Yang Bai, Yuyuan Zeng, Yong Jiang, Shu-Tao Xia, Xingjun Ma, and Yisen Wang. Improving adversarial robustness via channel-wise activation suppressing. In ICLR, 2021.
Shumeet Baluja and Ian Fischer. Adversarial transformation networks: Learning to generate adver- sarial examples. arXiv preprint arXiv:1703.0993 2017.
Peter L. Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. In COLT, 2001.
Jose H. Blanchet and Karthyek R. Murthy. Quantifying distributional model risk via optimal transport. Math. Oper. Res., 2019.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. In ICLR, 2019.
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness neural networks. In S&P, 2017.
Rich Caruana, Alexandru Nieulescu-Mizi,, Geoff Crew, and Alex Ksikes. Ensemble selection from libraries of models. In ICML, 2004.
Rewon Child, Very deep vaes generalize autoregressive models and can outperform them on images. In ICLR, 2021.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML, 2020.
Jacob Devlin, Ming- Weig Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL, 2019.
Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. MMA training: Direct input space margin maximization through adversarial training. In ICLR, 2020.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In CVPR, 2018
Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P. Vetrov, and Andrew Gordon Wilson. Loss surfaces, mode connectivity, and fast ensembling of dnns. In NeurIPS, 2018.
Ian J. Goodfellow and Oriol Vinyals. Qualitatively characterizing neural network optimization problems. In ICI.R, 2015.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against boun-dounded adversarial examples. arXiv preprint arXiv:2010.03593. 2020.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
10

---

Published as a conference paper at ICLR 2022
Dan Hendryeks and Kevin Gimpel. Bridging nonlinearities and stochastic regularizers with gaussian error linear units. arXiv preprint arXiv: 08415, 2016.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurlPS, 2020.
Hanxun Huang, Yisen Wang, Sarah Monazam Erfani, Quanquan Gu, James Bailey, and Xingjun Ma. Exploring architectural ingredients of adversarially robust deep neural networks. In NeurIPS, 2021.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool. Russ Bates, Augustin Zidek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A A Kohl, Andrew Ballard, Andrew Cowie, Bernardino Romera-Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman, Ellen Clancy, Michal Zielinski, Martin Steinegger, Michalina Pacholska, Tamas Berghammer, Sebastian Bodenstein, David Silver. Oriol Vinyals, Andrew W Semior, Koray Kavukcuoglu, Pushmeet Kohli, and Demís Hassabis. Highly accurate protein structure prediction with AlphaFold. Nature, 2021.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing, arXiv preprint /80300630.. 2018.
Alex Krizhevsky, Ily Sutskever, and Geoffrey Hinton. Imagenet classification with deep convolu- tional neural networks. In NeurlPS, 2012.
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. NeurlPS, 2018.
Fangzhou Liao, Ming Liang. Yinpeng Dong, Tianyu Pang. Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. CVPR, 2018.
Xingjun Ma, Yuhao Niu, Lin Gu, Yisen Wang, Yitian Zhao, James Bailey, and Feng Lu. Understanding adversarial attacks on deep learning based medical image analysis systems. Pattern Recognition, 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii Virtual adversarial training: A regularization method for supervised and semi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell., 2019.
Dantong Niu, Ruohao Guo, and Yisen Wang. Moiré attack (ma): new potential risk of screen photos. In NeurlPS, 2021.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In ICML, 2019.
Nicolas Papernot, Patrick D. McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation defense to adversarial perturbations against deep neural networks. In S&P, 2016.
Yao Qin, Nicholas Frosst, Sara Sabour, Colin Raffel, Garrison W. Cottrell, and Geoffrey E. Hinton. Detecting and diagnosing adversarial images with class-conditional capsule reconstructions. In ICLR, 2020.
Sylvestre-Alvise Rebuffi, Sven Gowal, Dan A Calian, Florian Stimberg, Olivia Wiles, and Tim- othy Mann. Fixing data augmentation to improve adversarial robustness. arXiv preprint arXiv:2103.01946, 2021.
Leslie Rice. Eric Wong, and J. Zico Kolter. Overfitting in adversarially robust deep learning. In ICML, 2020.
Andrew Slavin Ross and Finale Doshi- Velez. Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients. In AAAI, 2018.
Hasim Sak, Andrew W. Senior, Kanishka Rao, and Françoise Beaufays. Fast and accurate recurrent neural network acoustic models for speech recognition. In INTERSPEECH. 2015.
11

---

Published as a conference paper at ICLR 2022
Yunda Si and Chengfei Yan. Improved protein contact prediction using dimensional hybrid residual networks and singularity enhanced loss function. bioRxiv, 2021.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, lan Goodfellow, Dan Boneh, and Patrick D. McDaniel. Ensemble adversarial training: Attacks and defenses. In ICLR, 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In NeurIPS, 2017.
Cedric Villani. Topics in optimal cansportation.(bookk) OR/MS Today, 2003.
Hongjun Wang, Guanbin Li, Xiaobai Liu, and Liang Lin. A hamiltonian monte carlo method for probabilistic adversarial attack and learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020a.
Hongjun Wang. Guangrun Wang, Ya Li, Dongyu Zhang, and Liang Lin. Transferable, controllable, and inconspicuous adversarial attacks person re-identification with deep mis-ranking. In CVPR, 2020b.
Yisen Wang, Xuejiao Deng, Songbai Pu, and Zhiheng Huang. Residual convolutional ctc networks for automatic speech recognition. arXiv preprint arXiv: arXiv:1702.07799 2017.
Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICML., 2019.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. ICLR, 2020c.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust general- ization. In NeurIPS, 2020.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In NDSS, 2018.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing. Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In ICMI., 2019.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankanhalli. Attacks which do not kill training make adversarial learning stronger. In ICML, 2020.
Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan S. Kankanhalli. Geometry-aware instance-reweighteed adversarial training. In ICLR, 2021.
12

---

Published as a conference paper at ICLR 2022
A FURTHER EXPERIMENTS
Here we adopt & and / or WRN-32-10 as the backbone model with the same experimental setup in Sec. 4.1, where we reported the natural accuracy (NAT), PGD-20 and PGD-100 attack (PGD), MIM (PGD with momentum term), CW attack and each component of AutoAttack. All the experiments are conducted for individual trials and we also report their standard deviations. All the methods were realized by Pytorch 1.5, where we used single NVIDIA GeForce RTX 3090 GPU.
A.1 ROBUSTNESS AGAINST COMPONENTS OF AUTOATTACK
To broadly demonstrate the robustness of our proposal, conducted experiments against each component of AutoAttack. We perform each component of AA on CIFAR-10 dataset with both ResNet18 and WRN-32. M-NN-3--0. including three parameter-free versions of PGD with the CE, DLR, targeted-CE loss with 9 target classes loss (APGD APGD APGDT), the targeted version of FAB (F and an existing complementary Square (Andriushchenko et al. (020). Results are shown in the following Table And it is obvious that our SEAT outperforms other methods against all components of AA.
Table 4: Average robust accuracy (%) and standard deviation against each component of AA on CIFAR-10 dataset with ResNet18 and WRN-32-10.
ResNet18
WRN-32-10
APGD&R
APGDDLR
APGDT
FAB
Square
APGDGE
APGDDLR
APGDy
FAB
Square
47.47
48.57
45.14
26.17
47.34
48.00
56.3
+0.15
+0.26
+0043
TRADES
20.21
53.47
1:0.66
30.89
1:0.56
47.93
10.43
38.53
10.21
55.75
1:043
55.38
10.42
55.55
20.13
52.2
4:0.72
33.11
1:0.17
39,47
MART
20.13
52.8
50.30
#-03
+-0.22
48.17
+0.28
29.99
40.51
55.73
+032
35.2
35.41
+0.4
$1.99
-0033
+-063
52.88
+-0.38
59.01
SEAT
20.17
5387
2:034
53.35
1:0.7
10037
51.11
1:0.2
57.77
1:..8
57.57
20029
57.74
0.227
55.06
2:0.36
55.53
2:0.3
62.26
A.2 PERFORMANCE ON CIFAR-100
To further demonstrate the robustness of our proposal against adversarial attacks, we benchmark the state-of-the-arr robustness with ResNet18 on CIFAR-100. We widely investigate the performance of SEAT against the PGD methods (PGD and PGD 100), MIM, CW, AA and its all components. Results shown in Table demonstrate the effectiveness of SEAT for building robust classifier.
Table 5: Comparison of our algorithm with different defense methods using ResNet18 on CIFAR1O. The maximum perturbation have shown that the proposed SEAT method greatly improves the robustness of the model.
8/255. Average accuracy rates (in %) and standard deviations
Mathed
NAT
20
PCD
MIM
CW
APGD
APGDpL2
APGDy
FABT
Square
AAA
60.1
26:22
25:21
25.31
21.87
20.53
25.13
21355
23.37
27.93
2331
10.12
1044
1.0.51
10.29
10.20
10.3
-0.21
20.12
1:0.4
TRADES
4:1.37
24.72
29.01
SEAT
+3.33
36.28
14.17
32.15
1-0.66
32.13
+0.15
32.62
2.026
29.08
20.18
30.97
+0.2
2062
+4.23
26.88
-0.24
27.71
20.34
32.35
1.0.4
27.87
A.3 DIFFERENT LEARNING RATE STRATEGIES
Apart from showing the curve of different learning rate schedule in Figure 3 (a) Sec. 4.3, we also report final results in Table The effect of warming up learning rate is marginal. When compared with the staircase one, the warmup strategy cannot generate diverse models in the later stages so the homogenization of the candidate models we mentioned in Sec. 3.2 cannot be fixed by the warmup strategy. On the contrary, those methods like cosine linear cyclic that provide relatively diverse models in the later stages can mitigate the issue, accounting for more robust ensemble models.
A.4 DETERIORATION VANILLA EMA
As shown in Sec. 3.3, the deteriorated SEAT underperforms SEAT lot from the perspective of optimization. We also report quantitative results on both ResNer18 and WRN-32-10 shown in the
13

---

Published as a conference paper at ICLR 2022
Table 6: Average robust accuracy (%) under different learning strategies CIFAR-10 dataset with ResNet18.
Method
NAT
PGD
PGD400
MIM
CW
APGDDE
APGD
APGD
FABr
Square
AAA.
SEAT (Saincss))
80.91
51287
09:71
52.39
85.01
45.83
15:11
33.51
15.85
(Cosine)
50.15
(Lincur)
57-13
54.38
51.41
57.77
(Warmup)
(Cyclic)
Tables 7 and The deteriorated one does not bring too much boost when compared to the vanilla adversarial training (except for PGD methods). A plausible explanation for the exception of PGD that the SEAT technique produces an ensemble of individuals that are adversarially trained by PGD with the cross-entropy loss, which means that they are intrinsically good at defending the PGD attack with the eoosspy loss and its variants even though they suffer from the deterioration. Considering results have greatly improved after using the piecewise linear learning rate strategy. is fair to say that adjusting learning rate is effective. As we claimed in Proposition 2 and its proof, the staircase will inevitably make the self-ensemble model worsen since Σ (3,g will gradually approach to zero, meaning the difference between F y) and fa y) achieves the second order of smallness.
Table 7: Average robust accuracy (%) and standard deviation CIFAR-10 dataset with ResNet18. 18,
PGD 20
PGD 100
APGDCE
APGDDLR
FAB
Squrre
84.32
48.29
48.12 20.1
47.95
49.57
47,47
48,57
45.14
46.17
54311
44.37
+1.23
+:.11
20004
20015
2:035
+0.18
40.11
20.11
20.25
20.37
SEAL
B0.91
34.58
55.30
$4,47
45.71
32.30
45301
25.24
45.11
3304
45.85
(ettrrorratd)
=0.4
1:.19
SEAT
$1.3
Table 8: Average robust accuracy (%) and standard deviation on CIFAR-10 dataset with WRN-32-10.
Method
NAT
PGD
PGD100
MIM
CW
APGDCE
APGDpLR
APGDy
FAB
Square
AA
34.17
19.00
57,5
48.17
14.21
10.17
20.25
2:0.6
10.36
20.43
20.18
1:048
SEAT
85.33
S301
54.12
5002
40.36
(eteriurated)
1:0.2
1.042
10.19
1.0.3
20041
1.0.54
4:0.28
10067
100.5
20033
1.0.11
SEAT
120.2
86.44
59.84
2:0.2
110.66
59.8
60.87
10.1
2.034
58.95
2.018
57.57
1.0.9
$7.74
10.27
55.06
20036
55.53
2.0.3
62.26
1.0.2
55.67
A.5 COMPUTATIONAL COMPLEXITY FOR SEAT
To demonstrate the efficiency of the SEAT method, we use the number operations (MACs) in Giga (G) to compute the theoretical amount of multiply-add operations in DNNs, roughly GMACs 0.5 GFLOPs. Besides, we also provide the actual running time. As shown in Table 9 the SEAT method takes negligible MACs and training time when compared with standard adversarial training.
Table Evaluation of time complexity of SEAT. Here we use the number of Multiply-Accumile operations (MACs) Giga (G) to measure the running time complexity. And we also compute the actual training time with without the SEAT method using ResNet18 and WRN-32- 10 on single
NVIDIA GeForce RTX 3090 GPU.
Method
MACs (G)
Training Time (mins)
ResNett8 (AT)
0.56
272
ResNet (SEAT)
0,59
273
PRNN32-10 (AT)
6.67
1534
WRN-32-10 (SEAT)
6.81
1544
14

---

Published as a conference paper at ICLR 2022
B PROOFS OF THEORETICAL RESULTS
B.1 PROOF OF PROPOSITION
Proposition 1. (Restated) Let So (.) denote the predictions of neural network parametrized by weights 0. Assuming that v0 6, (-)-) is continuous and V(z, y) E D. fe(x.y) is at least twice differentiable. Consider two points 0., 0 ④ the weight space and & 0, 0. for t & {1, ... ,, the difference between fI y) and folæ, y) is of the second order of smallness and only if E ( (
Proof. For the sake of the twice differentiability of (z, y), based on the Taylor expansion, fit quadratic polynomial of Jo(e (z. y) to approximate the value So, (xr, y):
fo, (z. v) fa v) Vefo(x. + 2 Vefoa, vic + O
(10)
where O (A") represents the higher-order remainder term. Note that the subscript & here stands for neighborhood where the Taylor expansion approximates a function by polynomials of any point (i.e. 0) in terms of its value and derivatives. So the difference between the averaged prediction of candidate classifiers and the prediction the ensembled weight classifier can formulated as:
[r(#,8) fo(2,
Σ BeSo, (x. (x
T
T Σ (A0 (11)
(a +
Σ 3E Velo(z,))
(B,E Velo(z, 0(22))
Therefore, we can claim that the difference between fo, (x, y) and (z(2,)) g) "almost" least the first order of smallness except for some special cases. And we will immediately declare under which condition this difference can achieve the second order of smallness in the following proof of Theorem
B.2 PROOF OF THEOREM 1
Theorem 1. (Restated) Assuming that for i.j E f1,....,11... 0; =0; if and only if = j. The
difference between the averaged prediction multiple networks and the prediction of SEAT is the second order of smallness if and only if B,
a)) 1-6(1)1) T- for
...,,..... ,T))
Proof. According Eqn 11 we know that the second order of smallness will achieve when
(B, = 0. Thus, we continue deducing from Eqn
as:
r
Σ (BEg )) =0
1=1
Σ B, 0) =
(12)
Σ 3,0, -
T
T
Σ
3,0, = (1--)1-0((-1 T-i0.
To get further conclusion, we next use Mathematical Induction (MI) to prove only when 8;
- (1-j)-6(--1) for E (1,2,... T) will lead Σi= 3,0; -'= o))
15

---

Published as a conference paper at ICLR 2022
Base case: Let i 1, it is clearly true that B1 a= if and only if 3101 a hence the base case holds.
Inductive step: Assume the induction hypothesis that for a particular k, the single case T = holds, 8(0) a 7- (1
meaning the sequence of (31, 33. ... ........... . .Bx) is equal to the sequence of ((1 - a) -
a)' a))=6())a) T-2 (1 a) a T-k) if Σ i== 3,0: = Ei a) 1-)- 1) T 0.
For T = k + 1, it follows that:
Σ
3,0 (1-a)) (1--))16((-1)) TTo
1=1
3 Bk+ Ox+1 Σ (1 a) 1(1-1) a --0 (1 a) 1(((((+))))))) r-(+11 Ou+1 (13)
Bk+10 = 1--a))(((()))-----111 - 0k+1
BE+1 = x) -(((++))-))) -- T-(++1)
The sequence of normalized scores at the (k+1) 1) ensembling at left hand is 3,, BR Bx+1)
after adding the new term 3k+1- Likewise, the sequence of the right hand (3,, 32, 38x equal
the sequence of ((1 a)" a= (1 (01=61))a=22 .... (1 every Jo, E F is different from others and the sequence is ordered, we have (B1, B2, ... 3u. Bx+1) =
aa'= 10)=((k--)) 1) k)). Because
((1 (1-a) --- (1 - a
(1
Conclusion: Since both the base case and the inductive step have been proved as true, by mathematical induction the statement 33 integer T. Following Eqn [1] the difference between SEAT and the averaged prediction of history
a0==-()- for i {1, 2, ... . holds for every positive
networks is controlled by the first order term which reaches 0 only 8 (1 (1-a)² -a)-=((="aa""
for 21, T}. Thus, SEAT hardly be approximate to the averaged prediction of history networks indeed.
16

---

Published as a conference paper at ICLR 2022
B.3 PROOF OF PROPOSITION 2
Proposition 2. (Restated) Assuming that every candidate classifier is updated by SGD-like strategy,
meaning 0++1 0t Vo, Jo. (x', y) with Ti 2 T2
2 TI 0, the performance of self-
ensemble model depends on learning rate schedules.
Proof. First we discuss a special case the change the t-th iteration. Reconsidering the first order term in Eqn I1 we have:
BB Vefa(z,))
Σ [3,(0: V)
Σ [3(((1 (1 -000)))00-11)2--- (
T
> [(1 (1 a)) 1- 7 00 - (1 (1 -a) (1-0) --((1)) a1 ---100--1
00-1
-(1- - a) - T 1 0 V
E [Bi(11 (1 a) 2 0 (1 (1 a) 1-56(111) aT -1000-11111 -
-(1 a)) 8(t 1)
(1 () - 6(t T ++| 01-1 07(4-1))V
2 [3((1 (1 a) - 6(t- T- T0 (1 (1 a) - ((t-1) ---00-11
+ - (1 a) - a)) 8(t 1) T- 0 ( v):
(14)
so we can deduce by combining:
r
Σ 13.(11 (1 -000--0,1)) -,-1)+ (
(15)
where C (1 (1-(1 (1-(1- -000- (1 (0) 0 By using SGD
to update O,, we have:
T
Σ [a((1 - (1 (1)) 1880)(1))) ( (T( (Vo, (00. 8)) + (())))
(16)
Considering C' is a constant for the f-th update. without changing samples in the f-th minibatch, we conclude that the output of SEAT depends the learning rate Tt-
17

---

Published as a conference paper at ICLR 2022
To further analyse the whole training process, construct 0 + E 02 and & = 0 0 to unpack y for the averaged prediction history networks, and then reformulate Eqn
(BE V Vefo(z.y)
T
[3:(0- 0)[ {fo
T
[Bell T - (1 - -1--) 1-01)- T- - -
T
0 4 (1 - -1- 0.-1 T
[8a(( T - (1
a)) 10)-
T
1
(1 a))--((-11) T -0,00-1-+ 00-1 T
13 T - (1 a)
01
T
))) 1 1-
| 5( 0, 1-1 0, - [
T
13 T - (1 a)
0,
(1 1 5(c T )0:-1 + 1
1
T
- a)2 2-
00_1 0
T
Σ 1=1 T (1 a) 1 8(1 T 0, 01-1) (f
13
(17)
where C' ( (1- a) 1-8 8(- a - 1-t - 0 Likewise, the above equation can be further deduced:
Σ 13,01 T - a)
(Vo,e(Os (4h u)) + (fo))))
(18)
which means the difference between fx(8,8) y) and (a y) depends on learning rate schedules.
18


Reviews:
[{"review_id": "SNoGaE7IQVZ", "reviewer": "Reviewer_Ztw4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The core algorithm SEAT proposed is to apply self-ensemble to model parameters during adversarial training. The paper demonstrates the effectiveness of the method with extensive experiments as well as with ablation studies.\n\nFor the main results in Section 4.2, it might be better to elaborate on how these numbers are obtained from experiments: e.g. are these all trained to the same step; are the numbers reported average over runs; what are the variances, etc, to give a better sense of how statistically significant the improvements are.\n\nAnd comparing Table 1 and Table 2, it seems to be inconsistent around incorporating CutMix. \n\nSince Section 3.3 tries to explain why the self-ensemble method could not be integrated into the routine working flow and argues the deterioration comes from the learning rate. It would also be interesting to know how it works with other learning rate schedules, such as with a warm-up stage at the beginning.", "summary_of_the_paper": "This paper proposes the Self-Ensemble Adversarial Training (SEAT) which averages weights of history models for robustness. Compared to individual models or an ensemble of predictions from different classifiers, they argue that the proposed self-ensemble method provides a smoother loss landscape and better robustness. They also shed light on the deterioration of general self-ensemble models because of learning rates in the late phases.", "summary_of_the_review": "The paper is well written with theoretical propositions and extensive experimental results.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "BKzTl9nhXP2", "reviewer": "Reviewer_E4Bu", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n* The proposed method is well-motived, and authors provided multiple experiments to support their claim and justify the performances of their proposed method. \n* This paper is overall clear, well-organized and easy to follow. \n\nWeaknesses / discussion questions:\n* Deterioration is a very interesting topic and desired more studies. The paper didn't explain in detail what caused deteriorated SEAT (in Figure 2). And does deteriorated SEAT underperform SEAT a lot? If the answer is yes, does adjusting learning rate sufficiently enough to prevent this decreasing? I believe the paper could be improved if a clear explanation can be studied and explored.\n\n* The experimental results are not convincing to me. My specific concerns are the following:\n  * The authors are suggested to report more details for experimental settings, e.g., whether report **best** or the **last** epoch results? what loss function used for SEAT? if xentropy used, then can the performance be further boosted if using TRADES? The authors are suggested to report more ablation study discussing different combinations (e.g., SEAT by TRADES, SEAT + CutMix by TRADES) as well. \n  * It does not make sense to me that PoE involves with FAT and GAIRAT, especially when TABLE 1 already provides evidence that FAT and GAIRAT fails on robustness. It would be more interesting and meaningful when including stronger components, e.g., TRADES trained with different \\lambda. \n  * On Table 1, I found at least TRADES is not sufficiently trained, its performance over NAT and AA should be at least better than 84% and 49.5% respectively. The authors are suggested to re-train and re-evaluate the baselines.\n  * On Table 2, authors claim that \"when compared with the results of ResNet18, CutMix performs much better when the model becomes larger.\" However, SEAT+CutMix didn't achieve the best against all the attacks (e.g., MIM), and we can also observe SEAT (86.44) performs better than SEAT+CutMix (84.81) over NAT, so CutMix's better performance over AA could be due to the trade-off between Natural accuracy and robustness. It is well possible that CutMix may benefit the model with more capacity, but this conclusion cannot be directly drawn based on the current observations on Table 2. \n  * For a thorough evaluation, it would be better to report larger datasets, e.g., CIFAR-100.\n  * Given the results reported for the proposed method show a great boost in robust performance, it would be nice to have the ability to view the training and evaluation code.\n\n* Although the paper is overall easy to follow, it seems to be written in a rush. There are several typos. The writing of the paper needs further polishing.\n  * Page 5, Figure 1(b): Resnet -> ResNet\n  * Page 7, first row under section 4.1: wrong citation format\n  * Page 8, Paragraph 1: robusta -> robust\n  \n", "summary_of_the_paper": "This paper proposes an adversarial training scheme called SEAT, where the final robust classifier is united by averaging states of every history models through the adversarial training process. Authors compares SEAT with prediction-ensembled methods and analyze its effectiveness both theoretically and experimentally.", "summary_of_the_review": "This paper proposes an adversarial training scheme called SEAT, where the final robust classifier is united by averaging states of every history models through the adversarial training process. Although the idea of self-weight-ensembled method is intriguing, I am not that convinced by the current experimental settings and found it is lack of several competing baselines and performance on larger dataset.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2tl4x5o1QuB", "reviewer": "Reviewer_np74", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Strength:\n\nThis work proposes a novel ensemble method. Instead of training several individuals, it reuses the historical states during training, which saves computation costs. The method is novel and meaningful. It also performs theoretical analysis, which compares the difference between this new self-ensemble method and existing ensemble methods. The analysis and proof are easy to understand and meaningful when following the authors' logic. Experiments are performed with various network structures under different adversarial attacks.\n\n# Weakness:\n\n1. The exponential moving average (EMA) is used in this paper, which is also widely used to improve the test performance of models. Could the authors explain what improvements do they make on EMA?\n2. The experiments include results under AutoAttack (AA), but there are only overall results reported. Could the authors also report the results of each component of AA?\n3. The authors claim the method is efficient. Could they report the FLOPs or the throughput of the model?\n\n=========\n\n# Post rebuttal responses:\n\nThanks for the response. My concerns are well addressed. Thus, I raise my score to 8. The added experiments compared to vanilla EMA further demonstrate the effectiveness and novelty of the proposed SEAT. Additional experiments on each component of AA (one type of adaptive attack) present the reliability of the robustness. The FLOPs and running time convince me the efficiency of SEAT. Overall, I think the method is novel and meaningful. The theoretical analysis and proof are easy to understand and thoughtful.", "summary_of_the_paper": "This work proposes Self-Ensemble Adversarial Training (SEAT), which utilizes the states of history models during training to efficiently improve the adversarial robustness. The reusing of historical states in the self-ensemble can save computation costs in contrast to previous ensemble methods that usually train several individuals separately. Theoretical analysis is also performed to explain the new self-ensemble method.", "summary_of_the_review": "The idea of ensemble historical states is novel. The theoretical analysis is meaningful. The experiments show this method has good potential.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "2YMO5XkuOyU", "reviewer": "Reviewer_K6Do", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Please find my review of the paper *\"Self-Ensemble Adversarial Training for Improved Robustness\"* below. Unfortunately, I do not find the current version of the manuscript to be ready for publication at ICLR. As outlined below, the main contributor to this decision are the clarifications required to understand the proposed method, the formality of the theory sections, and the somewhat lacking experiment section. \n\nIn addition to these three points, lack of clarity in exposition also played an important role in my final decision. The current version of the manuscript contains a number of overly verbose or oddly formalised sentences that make the paper hard to follow. In the section \"Small Notes\" below, I've listed a number of these issues, but this is certainly not an exhaustive list. \n\n### Proposed method is hard to understand\n\nAs I also said in the summary, if I understand correctly, the authors proposed method for improving the adversarial robustness of a model is to keep a running average of the models weights throughout training. Based on the initial discussion in the abstract and introduction this is very far from the ensembling approach that I was expecting. \n\nBased on the presented algorithm box, I would argue that the main contribution of the paper is showing that using the SGD optimiser augmented with a momentum term improves the adversarial robustness of a model compared to when only plain SGD is used. The smoothness argument discussed in the paper also supports the view of the contribution being an augmented optimiser. \n\n### Formality theoretical contributions\n\nThe paper proposes a number of theorems and propositions that I personally found to be written in a slightly weird form. This made it specifically hard to judge the correctness and usefulness of these propose theorems. \n\nA clear example of this is found in Theorem 1, which ends with: *\"SEAT will hardly be approximate to the average prediction of history networks\".* In this case it is unclear to me what is meant with *hardly be approximate.* \n\nIn general this ties in with my earlier comment concerning the clarity of the exposition. \n\n### Experimental results are lacking\n\nWhile the authors did an admirable job by providing a exhaustive list of baseline methods for improving the adversarial robustness of a classifier, I believe the experimental evaluation still to be somewhat lacking. Specifically, as suggested in numerous papers discussing the most effective way of evaluating the effectiveness of a new defence strategy, the authors unfortunately do not include an adaptive attack in their evaluation. Specifically, I believe it to be important to study adaptive attacks that would be able to deal with the smoothness of the loss landscape. \n\nIn addition to this, I also believe that the experimental evaluation would benefit from doing multiple runs and reporting the standard deviation on the accuracy of these runs. This is especially important considering that the paper is not accompanied by code. \n\n### Additional notes:\n\n- What is meant by extremely imperceptible?\n- Depicting adversarial training as \"memorising\" adversarial examples seems a very rough over simplification.\n- First line of second paragraph of introduction is not grammatically correct.\n- Sec 3.2, what is meant by: *\"However, such a simple moving average cannot keenly capture the latest change, lagging behind the latest states by half the sample width.\"?*\n- What is meant by this part of proposition 1:  \"is ”almost” at least of the first order of smallness.\"?\n- Remark 1: If the later settings are always more robust, why would we need the earlier models?\n- Paragraph after remark 1, \"uesd\" → \"used\"\n\n# Update after rebuttal.\nThe theoretical contributions of this paper are still not clear to me. However, based on the discussion with the authors and other reviewers, I will increase my score.", "summary_of_the_paper": "The authors of the paper propose a new method for improving the robustness of a model against adversarial attacks. If I understood correctly, the proposed method is a combination of both adversarial training and model ensembling, where in this instance the ensembling is performed by maintaining a moving average of the weights of the model at past time steps. \n\nIn addition to some providing some theorems concerning their proposed method, the authors empirically show that the trained SEAT models are competitive with current methods for improving the adversarial robustness of the method.", "summary_of_the_review": "The paper lacks clarity which makes it specifically hard to judge the contributions of the paper. In addition to this, the paper experimental evaluations requires more work.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes Self-Ensemble Adversarial Training (SEAT) for yielding a robust classifier by averaging weights of history models. The solution is different from an ensemble of predictions of different adversarially trained models. The authors also provided theoretical and empirical evidence that the proposed self-ensemble method yields a smoother loss landscape and better robustness than both individual models and an ensemble of predictions from different classifiers.

The paper receives a mixed rating of 8-6-6-5 (after private discussion; initially it was 8-6-5-3), and all reviewers actively engaged in discussion. From the three positive reviewers, it is in general consensus that this paper has a clear motivation, is easy to follow, and owns reasonable (not exceptional) novelty. The negative reviewer poses a number of concerns, citing the absence of adaptive attack evaluation, the unclear difference between vanilla EMA and SEAT, and the proof of Proposition 1. The authors provided detailed responses and the negative reviewer was partially convinced (not fully) after viewing other comments. 

AC carefully reads all discussions and feels this fall into a borderline case. The authors did solid work and there is no fatal concern as AC can see. The majority sentiment is that this is a good paper, just not an exciting one. Hence, the current recommendation is a borderline acceptance.
Decision: Accept (Poster)