Adversarial Training using Contrastive Divergence
Authors: Hongjun Wang
Guanbin Li
Liang Lin
Venue:  (2021)
Keywords: Adversarial Training
Contrastive Divergence

Abstract: To protect the security of machine learning models against adversarial examples, adversarial training becomes the most popular and powerful strategy against various adversarial attacks by injecting adversarial examples into training data. However, it is time-consuming and requires high computation complexity to generate suitable adversarial examples for ensuring the robustness of models, which impedes the spread and application of adversarial training. In this work, we reformulate adversarial training as a combination of stationary distribution exploring, sampling, and training. Each updating of parameters of DNN is based on several transitions from the data samples as the initial states in a Hamiltonian system. Inspired by our new paradigm, we design a new generative method for adversarial training by using Contrastive Divergence (ATCD), which approaches the equilibrium distribution of adversarial examples with only few iterations by building from small modifications of the standard Contrastive Divergence (CD). Our adversarial training algorithm achieves much higher robustness than any other state-of-the-art adversarial training acceleration method on the ImageNet, CIFAR-10, and MNIST datasets and reaches a balance between performance and efficiency.

Under review as a conference paper at ICLR 2021
ADVERSARIAL TRAINING USING CONTRASTIVE DI- VERGENCE
Anonymous authors Paper under double-blind review
ABSTRACT
To protect the security of machine learning models against adversarial examples, adversarial training becomes the most popular and powerful strategy against various adversarial attacks by injecting adversarial examples into training data. However, it is time-consune mine-consuming and requires high computation complexity to generate suitable adversarial examples for ensuring the robustness of models, which impedes the spread and application of adversarial training. In this work, we reformulate adver- sarial training as a combination of stationary distribution exploring, sampling, and training. Each updating of parameters of DNN is based on several transitions from the data samples as the initial states in a Hamiltonian system. Inspired by our new paradigm, we design a generative method for adversarial training by using Contrastive Divergence (ATCD), which approaches the equilibrium distribution adversarial examples with only few iterations by building from small modifications of the standard Contrastive Divergence (CD). Our adversarial training algorithm achieves much higher robustness than any other state-of-the-art adversarial training acceleration method on the ImageNet, CIFAR-10, and MNIST datasets and reaches a balance between performance and efficiency.
INTRODUCTION
Although deep neural networks have become increasingly popular and successful in many machine
learning tasks (e.g., image recognition He et al.] (2016b), speech recognition Hinton et al. (2012);
van den Oord et al. (2016) and natural language processing Hochreiter & Schmidhuber (997);
Vaswani et al. (2017). ) the discovery of adversarial examples Szegedy et al. (2014); Goodfellow et al.
(2015) has attracted great attention to strengthening the robustness of deep neural network (DNN)
under such subtle but malicious perturbations. These crafted samples pose potential security threats
in various safety-critical tasks such as autonomous vehicles Evtimov al. (2017) or face recognition
Sharif et al. (016): : Dong et al. 2019), which are required to be highly stable and reliable.
Unfortunately, it is considered to be unresolved since final conclusion has yet been reached on the root of the adversarial examples. Many defense methods Papernot et al. (2016); Na et al. (2018); Buckman et al. (2018) motivated by different interpretabilityy of adversarial examples Goodfellow et al. (2015); Fawzi et al. (2018): 8): Ma et al. (2018) were broken within a short time, indicating that there still no thorough solution to settle this matter once and away. Nonetheless, adversarial training Szegedy et al. (2014); Goodfellow et al. (2015) has shown its ability to make classifiers more robust against sorts attacks than any other defenses in Madry et al. yp; Athalyy al. (2018) offers an intuitive approach to handle the problem, which first obtains suitable adversaria examples by solving the inner maximization problem and then update the parameters of ML model from these examples by outer minimization. More and more advanced defenses Kannan et al. (2018): Lin et al. (2019): Xie et al. (2019); Zhang et al. (2019c) are developed based on adversarial training.
However, a major issue of the current adversarial training methods is their significantly higher computational cost than regular training. It often needs multiple days and hundreds of GPUs for ImageNet-like datasets achieve better convergence Xie et al. (2019), which makes it nearly intractable and impractical for large models on tons of data. Even for small-sized datasets like CIFAR10, 10, adversarial training takes much longer time than regular training.
To address this issue, we formulate the problem of generating adversarial examples in a Hamiltonian Monte Carlo framework (HMC) Neal et al. (2011), which can be considered as exploring the

---

Under review as a conference paper at ICLR 2021
stationary distribution of adversarial examples for current parameters. The high computational cost of adversarial training can be easily attributed to the long trajectory of HMC producing. Therefore, we propose a new adversarial training algorithm called ATCD for strengthening the robustness of target models, enlightened by the Contrastive Divergence (CD) Hinton (2002). We minimize the difference of Kullback-Leebllrr divergence between two adjacent sampling steps to avoid running long Monte-Carie Markov Chains (MCMC). Instead of running the chain to achieve equilibrium, we can simply run the chain for fewer or even only one full step and then update the parameters to reduce the tendency of the chain to wander away from the initial distribution on the first step. Our approach is advantageous over existing ones three folds:
We offer new perspective on adversarial examples generation in HMC framework. From the view of HMC, we bridge the relationship between several adversarial examples generating methods and MCMC sampling. which effectively draw multiple fair samples from the underlying distribution of adversarial examples.
By analyzing the trajectory shift of different lengths of MCMC simulating, we speed up the adversarial training by proposing a contrastive adversarial training (ATCD) method, which accelerates the process of achieving distribution equilibrium.
We thoroughly compare the effectiveness of our algorithm in various settings and different architectures on ImageNet, CIFARIO and MNIST, Models trained by our proposed algorithm achieve robust accuracies markedly exceeding the ones trained by regular adversarial training and the state-of the-art art speedup methods when defending against several attacks.
2 BACKGROUND AND RELATED WORK
Adversarial Defense. To deal with the threat of adversarial examples, different strategies have been studied to find countermeasures protect MI models. These approaches he roughly categorized into two main types: (a) detection only and (b) complete defense. The former approaches) Bhagoji al. (2018); Ma et al. (2018); 8); Lee al. (2018) Tao et al. (2018); Zhang et al. (2018) is to reject the potential malignant samples before feeding them to the ML models. The latter defenses obfuscate the
gradient information of the classifiers to confuse the attack mechanisms including gradient masking
Papernot& & McDaniel (2017); [Athalye et al. (2018) or randomized models Liu et al. (2018); Xie et al.
(2018a): : Lecuyer et al. (2019); Liu et al. (2019). There are also some add-ons modules Xie et al.
(0019) Svoboda et al. (2019); Akhtar et al. (2018) Liao et al. (2018) being appended to the targeted
network or adversarial interpolation schemes /Zhang & Xu (2020) Lee et al. (2020) to protect deep
networks against the adversarial attacks.
Fast Adversarial Training. Besides all the above methods, adversarial training Goodfellow et al. (2015); Kurakin al. (2017): Kannan et al. (2018); Madry et al. (2018); Tramèr et al. (2018); Liu & Hsieh (2019); Wang et al. (2020) 2019) is the most effective way to ensure better robustness, which has been widely verified in many works and competitions. However, limited works focus on boosting robust accuracy with reasonable training speed. (Seefhih et al. (2019) recycle the gradient information computed reduce the overhead cost of adversarial training. YOPO Zhang et al.) (2019b) recast the ad adversarial training as a discrete time differential game and derive Pontryagin's 's Maximum Principle (PMP) for it. Fast-FGSM Wong al. (2020) combines FGSM with random initialization to accelcrate the whole process.
Markoy Chain Monte Carlo Methods. Markov chain Monte Carlo (MCMC) Neal (9933 provides
a powerful framework for exploring the complex solution space and achieves a nearly global optimal solution independent of the initial state. But the slow convergence rate of MCMC hinders its wide use in time critical fields. By utilizing the gradient information in the target solution space, Hamiltonian (or Hybrid) Monte Carlo method (HMC) Duane et al. ) Neal et al.) (20TT) achieves tremendous speed-up in comparison to previous MCMC algorithms. Multiple variants of HMC [Pasarica & Gelman (2010); Salimans et al. (2015); Hoffman & Gelman (2014) were yet to be developed for adaptively tuning step size iterations of leapfrog integrator. The fusion of MCMC and machine learning Tu & Zhu (2002); Chen et al. (2014); Song et al. (2017) Xie et al. (2018b) also shows great potential of MCMC.
Contrastive Divergence. Contrastive Divergence (CD) has achieved notable success in training energy-based models including Restricted Boltzmann Machines (RBMs) as efficient training
2

---

Under review as a conference paper at ICLR 2021
method. The standard approach to estimating the derivative of the log- log-likelihood function is using the Markov chain Monte Carlo Gilks et al. (1995). which can be expressed as the difference of two expectations. It runs &o MCMC transition steps at each iteration T and iteratively generates sequence of parameter estimates (0r)r given an i.i.d. data sample {X+] Pg. where Pa is the distribution of target samples for the true parameter 0. To reduce the computational complexity, the traditional Contrastive Divergence algorithm computes approximate RBM oggliikllhhod gradient setting A = 1. Various works are devoted to addressing the problem of the vanilla CD afterwards, such as uncontrolled biases and divergence Carreira-Perpinaa & Hinton (2005); Yuille (2003); Mac (200T); (Fichher& & Igel [2011] 2014). Persistent CD (PCD) and its relevant works Tieleman (2008); Desjardins et al. (210)) show a steady decrease of the log-likelhhoo in many numerical analysis while some works Schulz et al.] (2010); ;): Fischer & Igel (2010) also
Tieleman & Hinton
give examples in which PCD failed to converge. Although none of these works provide a solid convergence guarantee since the major problems of CD family stem from the fact that the stochastic approximation to the true gradient is a biased estimator, our work do not need the exact values of the derivatives. Actually, we just borrow the idea from the vanilla CD to accelerate the process of distribution equilibrium over the visible variables instead of discovering the unknown distribution Pang et al. [Alayyre et al. (2019).
3 PRELIMINARIES
Considering a target DNN model f E F, where F is the solution function space for classification task. We assume softmax is employed for the output layer of the model f()) and let (xx) denote the softmax output of given imput I E R' i.e., f(z) Rd - RC, where C is the number of categories. We also assume that there exists an oracle mapping function fr E F:: x + y², which pinpoints the belonging of the input x to all the categories by accurate confidence scores y² E RC. The common training is to minimize the cross-entropy (CE) loss Jee. which is defined as:
arg min E(x,y)~D [Jee v)].
fEF
where y is the manual one-hot annotation of the input since y" is invisible, The goal of Eq. T is to update the parameters of f for better approaching f", which leads to f() R y R y = (x)). Suppose the target DNN model correctly classifies most of the input after hundreds of iterations, it will still be badly misclassified by adversarial examples (i.e., arg max f......))((. * y(c)).
f())
In adversarial training, these constructed adversarial examples are used to updates the model using minibatch SGD, The objective of this minmax game can be formulated as a robust optimization following Madry et al. (2018):
arg min E JEF (R
max Joe (f (z) (z) (EN(2)
where the inner maximization problem attempts to generate the most easily misclassified samples while the outer minimization problem is to search a mapping function f' which is the closest one to the oracle f
HAMILTONIAN MONTE CARLO FOR ADVERSARIAL LEARNING
4.1 AN OVERVIEW OF MCMC AND HAMILTONIAN MONTE CARLO
The crux of this work relies on offering a fundamentally different view of adversarial example generation, which simulates the inner maximization in Eq. (2) as proposing dynamics by HMC. We now give the overall description of Sitopoolis-Hstting based MCMC algorithm. Suppose is our target distribution over a space D. MCMC methods construct a Markov Chain that has the desired distribution p as its stationary distribution. At the first step, MCMC chooses an arbitrary point 2o as the initial state. Then it repeatedly performs the dynamic process consisting of the following steps: (1) Generate candidate sample
"proposed" value for state I++1 from the candidate generating ((
density (ā). (2) Compute the acceptance probability & = min(1, piz, (()))) ). which used to decide whether to accept or reject the candidate.
(3) Accept the candidate sample as the next state with probability & by setting Fz+1 G. Otherwise reject the proposal and remain 3++1 24-
3

---

Under review as a conference paper at ICLR 2021
Although MCMC makes it possible sample from any desired distributions, its random-wall nature makes the Markov chain converge slowly to the stationary distribution p(æ)).
In contrast, HMC employs physics-driven dynamics to explore the target distribution, which is much more efficient than the alternative MCMC methods. Before introducing HMC, we start out from an analogy of Hamiltonian systems in Neal al. (2011) follows. Suppose hockey puck sliding over a surface of varying height and both the puck and the surface are frictionless. The state of the puck is determined by potential energy (0) and kinetic energy K(v),, where and are the position and the momentum of the puck. The evolution equation is given by the Hamilton's 's equations:
a0
ah (())
ol 89
(3)
VoU(O.
81
Due to the reversibility of Hamiltonian dynamics, the total energy of the system remains constant:
H(0, v) (0)) + K(v)-
As for HMC, contains three major parts: (1) Hamiltonian system construction; (2) Leapfrog integration; (3) Metropolis-Hasting correction. Firstly, the Hamiltonian is an energy function for the joint density of the variables of interest and auxiliary momentum variable e, so HMC defines a joint distribution via the concept of canonical distribution:
-((0.
p(0, v) x exp
(5)
where 1 for the common setting. Then, HMC discretizes the system and approximately simulates Eq. 3 over time via the leapfrog integrator. Finally, because of inaccuracies caused by
the discretization, HMC performs Detropolis-Hastinne Metropolis et al. (153)) correction without reducing the acceptance rate.
According to Eq.
(4) and (5) the joint distribution can be divided into two parts:
v(0,v) x exp ( (U(())
K((u)
exp
(6)
Since K(()) an auxiliary term and always setting K(v) = v ITI-1//2 with identity matrix I for standard HMC, our aim is that the potential energy U (0) can be defined as (()) log p(0) to explore the target density p more efficiently than using a proposal probability distribution. we can calculate (O) lng((())) 00
then we can simulate Hamiltonian dynamics that can be used in an
MCMC technique.
4.2 SIMULATING ADVERSARIAL EXAMPLES GENERATING BY HMC
Assume that the adversarial examples for with label y are distributed over the solution space Q. Given any input pair (x, y). for a specified model f(-) E F with fixed parameters, the adversary aims to find such examples 7 that can mislead the model:
( y)
arg max (())
where (ax)) is the neighboring regions of and defined x' EN(x)) == { |1 2111,,,0000 se.
From the perspective of Bayesian statistics, we can make inference about adversarial examples over a solution space Ω from the posterior distribution of ă given the natural inputs z and labels y.
fh ~ p (öla: y) x p(v))) f E2
(8)
In Hamiltonian system, it becomes generate samples from the joint distribution p(0, v). Let 0 = ã. according to Eq. (8) and (6), we can express the posterior distribution as a canonical distribution (with 1) using a potential energy function defined as:
U
log (0 (1)00) - log (()
N
(9)
1=1
J (z, y) log (()l).
4

---

Under review as a conference paper at ICLR 2021
1000-01
L2-diifeemnce
1AMEAG
Cosine Similarity
9.
1,00-44
Layer5 Pyyod0
Layer5
Layer20
1006-00
21
33
13
23
37
Epoches
Epoches
Measurement of TS (as defined in Definition 5 in different layers of ResNet34 on CIFARIO. For layer, measure the 1.2-difference and cosine similarity of the gradients running different lengths of trajectory.
Figure 1:
Since J (æ, y) is the usual classification likelihood measure, the question remains how to define p()(xr) A sensible choice is uniform distribution over the L, ball around x, which means we can directly use a DNN classifier to construct a Hamiltonian system for adversarial examples generation the base step of HMC.
Recall that the development of adversarial attacks is mainly based on the improvement of the vanilla fast gradient sign method, which derives I-FGSM. PGD and MI-FGSM. For clarity, we omit some details about the correction due to the constraint of adversarial examples. The core policy of the family of fast gradient sign methods is:
I = _x-1 + siign():),
(10)
where gr is the gradient of J at the t-th iteration, i.c., (=.((Ee-1,)) is clear that the above methods are the specialization of HMC by setting:
0. Zt, St.
(11)
(0, v) (0) + |0|.
More specifically, I-FGSM can be considered as the degeneration of HMC, which explicitly updates the position item 0 but implicitly changes the momentum item u at every iteration. One of the derivation of I-FGSM, MI-FGSM, has explicitly updated both 0 and by introducing St pgg-1 19) J (2, y) after Eq. (0) at each step with the decay factor = 1. The other
derivative PGD runs Eq. (10) set of initial points 5o { ,,0(),, ,.... (15) adding different
(1)
noises, which can be treated as parallel HMC but the results are mutually independent.
5 ADVERSARIAL TRAINING USING CONTRASTIVE DIVERGENCE
As mentioned in Section the inner maximization problem can be reformulated as the process of HMC. It is obvious that the high computational cost of adversarial training be easily attributed to the long trajectory of MCMC searching for the stationary distribution of adversarial examples. Nevertheless, does training robust model really need such long trajectory?
To answer this question, we consider studying the gradient of the loss since the training procedure (obtaining V. V.K()) and (D) and updating parameters of DNN) is first-order method. To quantify the extent to which the parameters in layer would change in reaction to the length of the trajectory, we measure the difference between the gradients of each layer running different lengths of trajectory. This leads to the following definition.
Definition 5.1. Let w(Kk) .-., (K be the parameters of each of the layers and (z) :y) be the batch of inbut-label pairs used to adversarially train the network. We define trajectory shift (TS) of activation along different lengths of trajectoriry & and k' to be the difference
5

---

Under review as a conference paper at ICLR 2021
|oh. - g Ilaiss where
Ik.i = V want (w,(t) ...., with;; 12(4)
(12)
O2.: V waner (w](k) w(()) 2(4) 11
The difference between Ok.i and Ie,; thus reflects the change in the optimization landscape of parameters Wi caused by the changes to its input, which captures the shift of different lengths of trajectory that could have an influence on adversarial training. Equipped with this definition, we measure TS on ResNet34 trained with adversarial examples simulating by different lengths of trajectory (A: = 2, k' = 10) throughout the training. Results are shown in Fig. 1 Although the situation in the bottom layer (e.g. layer5) is rather different than that in the top layer (e.g. layer20), both the direction and the magnitude of the gradients are quite close when simulating different lengths of trajectory. These evidences suggest that running a full trajectory for many steps too inefficient since the model changes very slightly between parameter updates.
Thus, we might take advantage of that by initializing a HMC at the state in which it ended for the previous model. This initialization is often fairly close the model distribution, even though the model has changed a bit in the parameter update. Besides, the high acceptance rate of HMC indicates that it is not necessary the chain for small (or even one) full step and then update the parameters to reduce the tendency of the chain to wander away from the initial distribution on the first step instead of running the full trajectory to equilibrium. We take small number K of transitions from the data sample {x:} 1a = 1 as the initial values of the MCMC chains and then use these K-step MCMC samples to approximate the gradient for updating the parameters of the model. Algorithm summarizes the full algorithm.
long trajectory from the initial point. Therefore, can simply run
Moreover, we also present a new training objective function Jed. which minimizes the difference of KI. divergence between two adjacent sampling steps to substitute the common KI. loss:
Jot p(Q") || - x(Q*|| )
(13)
where || denotes Kullback-Leible divergence and p and À the balanced factors. Qo and Q are the output vector of DNN given input images x + to and x + UK- The intuitive motivation for using this Jed is that we would like every state in HMC exploring
leave the initial distribution Qo and Q" 1 would never exceed Q 2'1 until Q1 achieves the equilibrium distribution. We set p. = and analyze how this objective function influences the partial derivative of the output probability vector with respect to the input. Due to the fact that the equilibrium distribution Q is considered as a fixed distribution and the chain rule, we only need to focus on the derivative of the softmax output vector with respect to its input vector in the last layer as follows:
i alog ( )
Olog fa (8)
Vusst Σ Uc
Σ ye
OE
di
= 2f )c E Vee-3 - fo (3),
Σ yc -
(14)
f (w--),,,
where Aʃ fa (x) ) Sa (ā). Based on this abbreviation, we can easily get the relationship between
Eq. 14h and 0J dØ" = (xx y. For each adversarial example generation, Eq. (44 makes an
amendment of which is determined by the difference of current and the last K-step HMC samples output probability. Since fa and (Ir) are more closer to f" and y" than fa and (x), each update of ã would be better corrected.
EXPERIMENTAL RESULTS
In this section, we focus on the ImageNet Deng et al. (2009). CIFAR10 10 Krizhevsky & Hinton (2009) and MNIST LeCun (1998) datasets with extensive experiments to validate the effectiveness of the proposed methods. For most part of experiments, we compare three standard adversarial training MaselineMMadry et al. (2018); Zhang et al.) (2019d); Rice et al. (2020) and three advanced acceleration methods Shafahi method&SSaffhhi et al. (2019); Zhang et al. (2019b) Wong et al. (2020) with our ATCD, More details
about experiment setup can refer to Appendix A.I Extensive ablation studies on CIFAR10 10 can also be found in Appendix A.3

---

Under review as a conference paper at ICLR 2021
Algorithm I Adversarial Training using Contrastive Divergence (ATCD)
Input: DNN classifier (-) with initial learnable parameters wo; training data x with visible label ys number of epochs N; length trajectory K; repeat time T; magnitude of perturbation e; learning rate n; step size o.
SSaage--:: Construct Hamiltonian system*/ (0,w,, w, W.
Jed (0 ), ).)). =|o]
Initialize ④ wo. 8 for epoch= 1............ (T..) do
8
8 f- x no. R0 Uniform(-ey for t = to do
/SStge-1: Generate adversarial examples by K-step contrastive divergence*/ for k = 1 to K do
0² -- 00111_000000 + VK(vt-1)
Vt + 1/-1 - aV w.
+ clip(v,, -6
end for
(Ok v2) (0t, v.), M-H step decides whether it should be accepted rejected.
Stagee-:: Update parameters of DNN by generated adversarial examplex
< [(,.v) [V VuwJee (f=(k)) ),
W +
f kg.
end for
end
6.1 IMAGENET
For ImageNet, we fix the total loop times T x same as Free-4 Shafahi et al. (2019) for fair comparison. We report average over the final 3 evaluation. Comparison between free adversarial training and ours are shown in Table Although the 2-PGD trained ResNet-50 model still maintains its leading role in the best robust accuracy, takes three times longer than our ATCD method. Actually, when compared with its high computational cost of ImageNet training, this performance gain can be considered inefficient or even impractical for resource limited entities. Besides, ATCD is an anytime algorithm expected to find better and better solutions the longer it keeps running. We also compare ResNet-50 model trained by our ATCD method with the Free-4 trained, model trained by ATCD produces much more robust models than Free-4 against different attacks in almost the same order of time. Though Fast-FGSM achieves a sterling acceleration, both its clean accuracy and robust accuracy satisfactory enough.
Methods
Clean Data
PGD-10
PGD-20
PGD-50
MI-FGSM-20
Speed (mins)
Natural train
75.34%8
0.14%
0.06%
0.03%
0.03%
1437
PGD Madry et al 2018)
63.95
36.89%
36.44%
36.17%
35.29%
8928
Shataaii
31.12%
30.07%
2745
Rast-FOSM Wong
2020 with apex
55.6898
30.23%
29.07%
28.91.%
27.88%
718
ATCD-2-1
apex
39.23%
35.91%
35.72%
35.76%
34.67%
1229
2992
Table 1: Validation accuracy and robustness ResNet50 on ImageNet. We report average over the final 3 runs. The maximum perturbation of all the attackers is 4/255. The best results are in red while the second best results are in blue. Our ATCD achieves trade-off between efficiency and accuracy.
6.2 CIFAR10
For CIFAR10, we fix the total loop times T K 8 same as Free-8 SSafahi et al. (2019) for fair comparison and show the training time of all methods. calculate the deviation value of final 5 evaluation and report average over 5 runs with different restarts. Results Wide ResNet34
Zagoruyko Komodakis (2016) are summarized in Table
From the table, vulnerable to all the attacks, while the baseline adversarial training methods (PGD, TRADES, Robust- Overfitting) produce robust models that effective to defend PGD attacks and goodish other type of attacks. Three advanced acceleration methods (Free, YOPO/TRADES+ YOPO, Fast-FGSM) and ours can be at least 4~5 times faster than previous adversarial training methods. Although
see that the naturally trained model (without any adversarial examples) is
7

---

Under review as a conference paper at ICLR 2021
Methods
Clean
PGD-20
MI-FGSM-20
CW
Speed (mins)
Natural
94.58%
0.00P%
0.00%
PG Podd-
2013
87.11 -2.377
23%
44.37%
45.919:00118
43.88%
40.919:40.37%
TRADES-TOZhBn
53.218-4b0 59%g
52.22%%
52.7%
26.919:--.236
Rohax--verr Oaee--vviiiin miling Rio d al
1170
85.21%% 66%
57.46% 1:0771
54.38% 11.5%%
54.81 0.51
10.34%
58.13% 4.037
4500
Free-8 Shufihi el
(201
84.29% 1:1.::44490 1.4%%
47.8% 1.1.::1330000 133%
47.01%:001195
46.71%:20222
42.55% 37%
51.379:1:0855
646
YOPO33- Ziinng A0
20
84.72%%:2 1.23%
46.4% 16.48-12111 49%
47 2.225
47. 47.5% 17%8
44,44% 29%
50.815-2:115 1.55%
457
TRADES+ TONN Juin
86%-
2o%
51.199.1
FiutFOSM WWnn ATA with apex
83.218::00665 66%
46 469-4-9-07111
0.555
45.51::0.515
43.01% 17%
48. 280-1005510
ATCD-41 willywichott opes
86.09% 1..27%
54.2% 1.0.4%
52.73% 0.21%
52.2%% 10.15%
30.2% 10.1%
57.11% 10.4%%
1677672
Table 2: Validation accuracy and robustness of Wide ResNet34 on CIFARIO. The maximum perturbation all the attackers is 8/255. We report average over 5 runs with different restarts. The "oo" error bars mean the method sometimes cannot converge during the training process. The best results are in red while the second best results are in blue.
Fast-FGSM achieves the best speed improvement (using the apex library), our ATCD the only method that even greatly boost the robust accuracy in reasonable training speed. Similar success also appear in different architectures (see in Appendix(A.3.I] We also perform the evaluation among different methods on both clean accuracy (i.e. accuracy on natural images) and robust accuracy (i.e. accuracy on adversarial examples) after every training epoch and show the remarkable reliability We further emphasize that although YOPO may be computationally cheaper when compared to conventional approaches and other methods, it is clear that the curve of YOPO vibrates greatly and frequently, which implies the training scheme of YOPO should be carefully designed to achieve stable results. This is also reflected in Table when compared with the error bars of YOPO (TRADES+ YOPO) and ours.
about our ATCD in Fig. 2
100
Accuracy Rate
60
50
40
0
Epochs
Figure 2:
Comparison with different advanced fast adversarial training methods. We use PGD-20 as the attacker and report their clean accuracy and robust accuracy. Solid lines represent the robust accuracy and dashed lines represent the clean accuracy.
6.3 MNIST
We also investigate our ATCD method on MNIST. PGD-40 still has comparable clean accuracy and robust accuracy among all the methods, but its computational cost significantly higher than other training methods. Our method still achieves good trade-off between efficiency and robust accuracy.
Data
PGD-40
CW
Speed (sees)
Natural train
99.98%
0.00%
0.00%
196
PGD-40 Mudry el al.
2018
99.50%
97.17%
93.27%
1877
Free- 10 Shatahi et (119))
98.29%
95.33%
92.66%
415
YOFO-S-TU Zhang 20
99.98%
94.79%
92.58%
312
AICD-2-1
99.36%
97.48% %
94.77%
441
Table 3: Validation accuracy and robustness of a small CNN on MNIST. The maximum perturbation of all the attackers is 0.3. The best results are red while the second best results are in blue.
7 CONCLUSION
In this paper, reformulate the generation of adversarial examples MCMC process and present a new adversarial learning method called ATCD, which approaches equilibrium distribution of adversarial examples with only few iterations by building from small modifications of the standard Contrastive Divergence. Extensive results with comparisons on various datasets show that ATCD achieves trade-off between efficiency and accuracy in adversarial training.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-fre freee attacks, In ICML, 2020.
Naveed Akhtar, Jian Liu, and Ajmal Mian. Defense against universal adversarial perturbations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3389-3398, 2018.
Jean- San--Bapistte Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In Advances in Neural Information Processing Systems, pp. 12192-12202, 2019.
Anish Athalye, Nicholas Carlini, and David A. Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In ICML, pp. 274-283, 2018.
Arjun Nitin Bhagoji, Warren He, Bo Li, and Dawn Song. Exploring the space of black-box attacks on deep neural networks. arXiv preprint arXiv: [tti::1711 1712.0941,, 2017.
Arjun Nitin Bhagoji, Daniel Cullina, Chawin Sitawarin, and Prateck Mittal. Enhancing robustness of machine learning systems via data transformations. In 2018 52nd Annual Conference on Information Sciences and Systems (CISS), pp. 1-5. IEEE, 2018.
Jacob Buckman, Aurko Roy, Colin Raffel, and Ian J. Goodfellow. Thermometer encoding: One hot way to resist adversarial examples. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 2018, Conference Track Proceedings, 2018. URL htRL:ttps:// httpss///peeeeveeieeeeee net. fttee/ooummmm ? i d= BSu--CW
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 39-57. IEEE, 2017.
Miguel A Carreira-Perpint and Geoffrey E Hinton. On contrastive divergence learning. In Aistats, volume 10, pp. 33-40. Citeseer, 2005.
Jinghui Chen and Quanquan Gu. Rays: A ray searching method for hard-label adversarial attack. In Proceedings of the 26rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2020.
Tianqi Chen, Emily Fox, and Carlos Guestrin. Stochastic gradient hamiltonian monte carlo. In International conference machine learning, pp. 1683-1691, 2014.
Jia Deng, Wei Dong. Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pp. 248-255. Ieee, 2009.
Guillaume Desjardins, Aaron Courville, Yoshua Bengio, Pascal Vincent, and Olivier Delalleau. Parallel tempering for training of restricted boltzmann machines. In Proceedings of the thir- teenth international conference on artificial intelligence and statistics, pp. 145-152. MIT Press Cambridge, MA, 2010.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
2.18.
Yinpeng Dong, Hang Su, Baoyuan Wu. Zhifeng Li, Wei Liu, Tong Zhang, and Jun Zhu. Efficient decision- based black-box adversarial attacks on face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7714-7722 2019.
Simon Duane, Anthony Kennedy, Brian Pendleton, and Duncan Roweth. Hybrid monte carlo. Physics letters B, 195(2):216-22)) 1987.
Ivan Evtimov, Kevin Eykholt, Earlence Fernandes, Tadayoshi Kohno, Bo Li, Atul Prakash, Amir Rahmati, and Dawn Song. Robust physical- world attacks on deep learning models. arXiv preprint arXiv: 1707.08900 2017.

---

Under review as a conference paper at ICLR 2021
Alhussein Fawzi, Seyed-Moie Moyee-Mohsenn Moosavi-Dezfoolii Pascal Frossard, and Stefano Soatto. Empirical study of the topology and geometry of deep networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3762-3770, 2018.
Asja Fischer and Christian lgel. Empirical analysis of the divergence gibbs sampling based learning algorithms for restricted boltzmann machines. In International Conference on Artificial Neural Networks, pp. 208-217. Springer, 2010. 2010.
Asja Fischer and Christian Igel. Bounding the bias of contrastive divergence learning. Neural computation, 23(3):664-6) 600,-673,21110 2011.
Asja Fischer and Christian lgel. Training restricted boltzmann machines: An introduction. Pattern Recognition, 47(1):25-39, 2014.
Walter R Gilks, Sylvia Richardson, and David Spiegelhalter. Markov chain Monte Carlo in practice. Chapman and Hal//CRC, 1995.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (eds.), Computer Vision ECCV 2016 14th European Conference, Amstendam, The Netherlands, October 11-14, 2016, Proceedings, Part IV, volume 9908 of Lecture Notes in Computer Science, pp. 630-645. Springer, 2016a.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016b.
Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine, 296):82-97, 2012.
Geoffrey E Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 14(8):1771-1800 2002.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1780, 1997.
Matthew D Hoffman and Andrew Gelman. The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo. Journal of Machine Learning Research, 15()):1993-1633, 2014.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing, arXiv preprint 18X3::8803.06333 2018
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In ICLR, 2017.
Yann LeCun. The mnist database of handwritten digits. htlp://yann. lecun. com/exdb/nmist// 1998.
Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness to adversarial examples with differential privacy. In 2019 IEEE Symposium Security and Privacy (SP), pp. 656-672. IEEE, 2019.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting outtoo--iistriuttinn samples and adversarial attacks. In Advances in Neural Information Processing Systems, pp- 7177-7177, 2018.
10

---

Under review as a conference paper at ICLR 2021
Sachyung Lee, Hyungyu Lee, and Sungroh Yoon. Adversarial vertex mixup: Toward better adver- sarially robust generalization. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pp. 269-278. IEEE, 2020.
Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1778-1787, 2018.
Lin, Chuang Gan, and Song Han. Defensive quantization: When efficiency meets robustness. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
Xuanqing Liu and Cho-Jui Hsich. Rob-gan: Generator, discriminator, and adversarial attacker, In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp- 1 11234- 11243, 2019.
Xuanqing Liu, Minhao Cheng, Huan Zhang, and Cho-Jui Hsieh. Towards robust neural networks via random self-ensemble. Proceedings of the European Conference on Computer Vision (ECCV), pp. 369-385, 2018.
Xuanqing Liu, Yao Li, Chongruo Wu, and Cho-Jui Hsieh. Adv-bnn: Improved adversarial defense through robust bayesian neural network. In ICLR, 2019.
Xingjun Ma, Bo Li, Yisen Wang, Sarah M. Erfani, Sudanthi
Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E. Houle, and James Bailey. Characterizing adversarial subspaces using local intrinsic dimensionality.
6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings, 2018
David MacKay. Failures of the one-step learning algorithm. In Available electronically htt:://ww.. inference. phy. cam. ac. uk/muckay/ubstrrtt//ggmm himl, 2001.
Aleksander Madry. Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. ICLR, 2018.
Nicholas Metropolis, Arianna Rosenbluth, Marshall N Rosenbluth, Augusta Teller, and Edward Teller. Equation of state calculations by fast computing machines. The journal of chemical physics, 21(6):088-1102. 1953.
Taesik Na, Jong Hwan Ko, and Saibal Mukhopadhyay. Cascade adversarial machine learning regularized with unified embedding. In 6th International Conference Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30- May 3, 2018, Conference Track Proceedings, 2018. URL htttps://ppeerreeeettee
net. forum? i HyRVyZaz
Radford M Neal. Probabilistic inference using Markov chain Monte Carlo methods. Department of Computer Science, University of Toronto Toronto, Ontario, Canada, 1993.
Radford M Neal et al. Meme using hamiltonian dynamics. Handbook of markov chain monte carlo, 2(11):2, 2011.
Tianyu Pang, Chao Du, Yinpeng Dong, and Jun Zhu. Towards robust detection of adversarial examples. In Advances in Neural Information Processing Systems, pp. 4579 4589, 2018.
Nicolas Papernot and Patrick McDaniel. Extending defensive distillation. arXiv:1705.05264, 2017.
arXiv preprint
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as defense to adversarial perturbations against deep neural networks, In 2016 IEEE Symposium on Security and Privacy (SP), pp. 582-597. IEEE, 2016.
Cristian Pasarica and Andrew Gelman. Adaptively scaling the metropolis algorithm using expected squared jumped distance. Statistica Sinica, pp. 343-364, 2010.
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. arXiv: Learning, 2020.
11

---

Under review as a conference paper at ICLR 2021
Jérôme Rony, Luiz G Hafemann, Luiz S Oliveira, Ismail Ben Ayed, Robert Sabourin, and Eric Granger. Decoupling direction and norm for efficient gradient- based 12 adversarial attacks and defenses. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2322-4330, 2019.
Tim Salimans. Diederik Kingma, and Max Welling. Markov chain monte carlo and variational inference: Bridging the gap. In International Conference on Machine Learning, pp. 1218-1226, 2015.
Hannes Schulz, Andreas Müller, and Sven Behnke. Investigating convergence of restricted boltzmann machine learning. In NIPS 2010 Workshop Deep Learning and Unsupervised Feature Learning. 2010.
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Advances in Neural Information Processing Systems, pp. 3353-3364, 2019.
Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K Reiter. Accessorize to a crime: Real and stealthy attacks sate--of-the-att face recognition. In Proceedings of the 2016 acm sigsac conference on computer and communications security, pp. 1528-1540, 2016.
Jiaming Song, Shengjia Zhao, and Stefano Ermon. A-nice-me Adversarial training for meme. In Advances in Neural Information Processing Systems, pp. 5140-5150, 2017.
Jan Svoboda, Jonathan Masci, Federico Monti, Michael M. Bronstein, and Leonidas J. Guibas. Peernets: Exploiting peer wisdom against adversarial attacks. In ICLR, 2019.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.
Guanhong Tao. Shiqing Ma. Yingqi Liu, and Xiangyu Zhang. Attacks meet interpretabilityy Attribute- steered detection adversarial samples. In Advances in Neural Information Processing Systems, pp. 7717-7728, 2018.
Tijmen Ticleman. Training restricted boltzmann machines using approximations to the likelihood gradient. In Proceedings of the 25th international conference on Machine learning, pp. 1064-1071, 2008.
Tijmen Ticleman and Geoffrey Hinton. Using fast weights to improve persistent contrastive diver- gence. In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 1033-1044 2009.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick D. McDaniel. Ensemble adversarial training: Attacks and defenses. In ICLR, 2018.
Zhuowen Tu and Song-Chun Zhu, Image segmentation by data-driven markov chain monte carlo. IEEE Transactions pattern analysis and machine intelligence, 24(5):657-673, 2002.
Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew W. Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. In The 9th ISCA Speech Synthesis Workshop, Sunnyvale, CA, USA, 13-15 September 2016, pp. 125, 2016.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5198-5998999 6008, 2017.
Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In Kamalika Chaudhuri and Ruslan Salakhut- dinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 6586-6595. PMLR, 2019.
12

---

Under review as a conference paper at ICLR 2021
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma. and Quanquan Gu. Improving adver- sarial robustness requires revisiting misclassified examples. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia. April 26-30, 2020. OpenReview.net, 2020.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan L. Yuille. Mitigating adversarial effects through randomization. ICLR, 2018a.
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan Yuille. and Kaiming He. Feature denoising for improving adversarial robustness. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 501-509, 2019.
Jianwen Xie, Yang Lu, Ruiqi Gao, and Ying Nian Wu. Cooperative learning of energy- model and latent variable model via meme teaching. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018b.
Alan L Yuille. The convergence contrastive divergences. Advances in neural information processing systems, pp. 1593-1600, 2005.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. CoRR, abs/1605.07144 2016.
Chiliang Zhang, Zuochang Ye, Yan Wang, and Zhimou Yang. Detecting adversarial perturbations with saliency. In 2018 IEEE 3rd International Conference on Signal and Image Processing (ICSIP), pp. 271-275. IEEE, 2018
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. In Advances in Neural Information Processing Systems, pp. 227-238, 2019a.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. In Advances in Neural Information Processing Systems, pp. 227-238, 2019b.
Haichao Zhang and Wei Xu. Adversarial interpolation training: A simple approach for improving model robustness, 2020. URL
net/Eorum?1dd-yeee
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing. Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, pp. 7472-7482, 2019c.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. arXiv: Learning, 2019d.
Tianhang Zheng, Changyou Chen, and Kui Ren. Distributionally adversarial attack. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 2253-2260, 2019.
A APPENDIX
A.1 EXPERIMENT SETUP
All the experiments taken single NVIDIA GeForce GTX Xp GPU. Results of Free training Shafahi et al. (2019), TRADES- 10 Zhang et al. (2019d), YOPO Zhang et al. (2019b), Fast-FGST
Wong et al. 2020) and GabuustOOvefitttig Rice et al. (2020) reproduced our local machines
13

---

Under review as a conference paper at ICLR 2021
but completely follow their official code Note that in our experiments, we do nor any label-smoothing or other tricks to boost the robustness accuracy since we would like to fairly compare PGD, Free and YOPO with our ATCD method. These extra tricks and adversarial interpolation schemes can be added to improve results for both approaches but not the focus of our paper.
ImageNet setup. The ILSVRC 2012 classification dataset contains "1.28M training images and 50k validation images labeled with 1,000 classes. Following the common practice, we perform horizontal flip, scale, and aspect ratio augmentation for training images and apply a single center crop of 224 x 224 pixels during evaluation. We choose the standard Resnet50 as the target model. For all methods, we use a batch size of 256, and SGD optimizer with momentum 0.9 and a weight decay of lc-4. The initial learning rate is 0.1 and the learning rate is decayed by 10 every 30/TK epochs. We also set step size 4/255 and magnitude of perturbation = 4/255 based on Tooe norm. We compare to the best performing configuration of Free adversarial training which uses 4 minibatch replays over 96 epochs of training. That means we only run N/TK 24 passes over the whole dataset for both Free training and ATCD.
CIFARIO setup. The CIFARIO dataset is a widely used dataset consisting of 60,000 colour images of 10 categories. Each category has 6,000 images. We choose the standard Wide ResNet-34 and Preact-ResNet18 following previous works Madry et al. 2018); Zhang et al. (2019)). We use a similar scheme that was used in YOPO Zhang et al. (2119):
train models for 40 epochs and the initial learning rate is set to 0.2, reduced by 10 times at epoch 30 and 36. For PGD adversarial training, we set the total epoch number N 105 and 10 steps of PGD with step size magnitude of perturbation & = 8/255 as a common practice. The initial learning rate is set to Se-2, reduced by 10 times at epoch 79, 90 and 100. We use batch size of 256, a weight decay of Se-4 and a momentum of 0.9 for both algorithm. For evaluating, we test our model's robustness under CW Carlini & Wagner 2017). . MI-FGSM Dong al. (2018), AutoAttck (AA)ero (2020), RaySChen & Gu (220)) and 20 steps of PGD with step size c = 2/255 and magnitude of perturbation 8/255 based on To norm.
2/255 and
MNIST setup. MNIST is a database for handwritten digit classification. It consists of 60,000 training images and 10,000 test images, which are all 28 28 greyscale images, representing the digits 0-9. We choose simple ConvNet with four convolutional layers followed by three fully connected layers, which is same as Zhang et al. (0119)). For PGD adversarial training, we train the models for 55 epochs. The initial learning rate set to 0.1, reduced by 10 times at epoch 45. We use a batch size of 256, a weight decay of 5e-4 and a momentum of 0.9. For evaluating, we perform a PGD-40 and CW attack against our model and set the size of perturbation practice Madry et al. (2018) Zhang et al. c2019a]c).
0.3 based on T as common
A.2 DETAILS ABOUT TRAJECTORY SHIFT
Trajectory Shift (TS) is our proposed metric to measure the difference of running various lengths of trajectory. To reflect the change in the optimization landscape of parameters of a network during the course of training, we essentially take different lengths in MCMC simulating and measure the gradient obtained by each backward.
Specifically, suppose we would like to investigate the TS of the i-th layer and the lengths of different trajectories are K and K', what we need to do is to generate adversarial inputs according to the K and K" -testep trajectory of MCMC simulating. When the trajectory simulation is finished, we perform gradient backward based on the potential energy function and adversarial examples at current state, and then gather all the gradients in the mini- batch Finally, all gradients will be added and averaged throughout the dataset when both trajectory simulating and parameters updating are completed. So the trajectory shift is the difference of these two averaged results. The whole process can be summarized
'Code from htts://gthubbcommaasssafiieeeeer
i "Code from http://siil...o...oooo/22/2/0/02
'Code from http://gttabccom//oooooggyggggg
*Code from htpp://guhhaaccom/luuussaaaaaatt adversarial
CCode from http://gtthaa.com///eessaaaaaaa overfittingg
14

---

Under review as a conference paper at ICLR 2021
as the following formualtion:
T
W J (w(1) s.... w ;
TS = D
(15)
V Waees (Wr(t), ..... W!! ;000, llaist, k=1
Σ
where B is the size of mini- batch. Note that in order to eliminate unnecessary effects, the same random seed and share initialization parameters for two sets of trajectory with different lengths, and the input samples of each mini-batch are exactly the same.
A.3 ABLATION STUDIES
A.3.1 DIFFERENT ARCHITECTURES
Besides the success Wide ResNet34 Zagoruyko & Komodakis (2016), also achieves similar acceleration against PGD-10 in Preact-ResNetll He et al. (a), ) as shown in Table [ ATCD achieve more aggressive robust accuracy boost with only slight drop in clean accuracy when compared with Free and YOPO. The word "natural train" in the table means the models are trained by using only natural (clean) images the tradition classification task always does.
Methods
Clean Data
PGD-20
MI-FGSM-20
CW
Speed (mins)
Natural
0.00%
0.00%
0.00%
PGD-10 Madry et all 2018)
14.9%%00.
10. 12%
41.58%%11..1
19.47%
08.88.00.3330 33%
132
TRADES- Zaaang 2019d)
85.05% 00.8%%
43.67%-+0..64%
42.33%
57.21%%
607
Robust-Overrttung Kice ctat 2020
85.15% + 1.03%
16.73%-1:0088
45.38% 0.52%
60.81% 11.1%%%20,5111 %
1033
Free-8 Shafahi et
2019
82.44% 10.37%
42.07%--10.44%
41.88%
57.02%:1102201
110
YOPO-3-3/haap Zhang of
2019
82.65% 20.76-00.756 75%
42.56%%-t00885 83%%
41.85%-20.445
56.93% 0.71%%
66
TRAADES-YPPD-5 Liang
2019
84.73%
o3.56%
oo.
oo
Fast FGSM Wong et al 2021 with
80.91% 0.77%
45.74% 1.0663%
43.27%
$6.57% 0.228
ATCD-2-1 witlywithour apex
81.54% =0.3%
49.27%
18.56%
10.29
28/114
Table 4: Validation accuracy and robustness of Preact-ResNet18 on CIFAR10. The maximum perturbation of all the attackers is 8/255. We report average over 5 runs with different restarts. The best results are in red while the second best results are in blue.
A.3.2 INFLUENCE OF UPDATING FREQUENCY AND TRAJECTORY LENGTH
We study the effect of updating iteration T and the length of trajectory K on the accuracy rate for our ATČD. We fix * K = 8 and apply several configurations. Results are shown in 5 Figure. 3 It can be observed that (1) The best accuracy rate of ATCD both clean images and PGD ad versariia examples appears at T = 4, K = 2. This may result from the balance it achieves between updating frequency and the length of trajectory when T K is limited. (2) We would like to point out Free training is special case of ATCD, except that ATCD has new objective function for approaching equilibrium distribution and can freely control the updating frequency and the length of MCMC trajectory. We compare our ATCD under several configurations with Free-8 (at the last row, and that's the reason why we fix T = =). Note that our ATCD method surpass Free-8 all cases, which shows that it is necessary to seperate parameters updating and adversarial examples generating instead of treating the service as one like Free model and Jed is helpful for the robustness.
A.3.3 INFLUENCE OF M-H RESAMPLING
The proposed method has a new adversarial example generation objective and the M-H resampling. In order to find out what actually causes the performance improvement, we ablate the behavior of the generation objective and the M-H resampling. This ablation is in Table 6 We investigate the above two components for training Wide ResNet34 on CIFAR10. When adding the CD generation objective in the procedure of training. the accuracy increases greatly e.g.. increasing from 47.8% to 53. 1% under 20-iteration PGD attacks. On the other hand, adding the M-H resampling strategy makes slight
15

---

Under review as a conference paper at ICLR 2021
(T.K)
Clean Data
PGD-20
(1,8)
84.68%=0.57% 85.09%+0.279%
48.8%=0.72%%
(2,4)
51.44%:+00.88%
(4,2)
86.39 % --2.7% %
54.2% 0.44%
(8.1)
44.88%
49.29%10.443%
(8,1) w/o Jed
84.29%=1.44
Table 5: Validation accuracy and robustness of Wide ResNet34 on CIFARIO. We report average over 5 runs. The best result under different attack methods is in bold.
(T, K)
100
Accuracy Rate (%)
80
40
20
Natural
PGD-20 Attack
(1.8)
(2,4)
(4.2)
(8.1)
Figure 3: The accuracy rates of Wide ResNet34 trained by ATCD with different configurations (T, K) on CIFAR10.
improvement the robust accuracy, and has a considerable increase the clean accuracy. These results suggest that both of them bring benefit for the clean and the robust accuracy. As samples in proposed method stay in (and return large numbers of samples from) high-density regions of the candidate distribution while only occasionally visiting low-density regions by the acceptance probability, which can can decipt the underlying distribution of adversarial examples. appears essential to properly combine the CD loss with the M-H resampling in the training procedure.
Method
Clean Data
PGD-20
Baseline
84.29%
47.8%
+ CD loss
85.05%
53.11% %
+ MH resampling
86.39%
54.2%
Table 6: Effectiveness verification of the CD loss and the M-H resampling CIFARIO. We use Wide ResNet34 as the target model and the baseline method is Free-8
A.3.4 DIFFERENT NUMBERS OF ATTACK ITERATIONS
To analyze the influence of the number of attack iterations on different adversarially trained models, we range the number of iterations PGD attacker from 10 to 200. 4shows the robust accuracy of different udversarially trained models. We fix æ = 8/255 and ε = 2/255, and change the iteration number ranging (10, 20, 50, 100, 200}. From the plots, we see that no matter how larger the attack iteration is, the checkpoint which achieves the high robust accuracy under one certain configuration still has higher accuracy than other models different iteration configurations.
A.3.5 DIFFERENT VALUES OF EPSILON
Figure 5 shows the robust accuracy models using various state-of-the-art adversarial training methods. We performed thorough experiments epsilon parameters {8/255, 16/255, 32/255, 64/255, 128/222 128/255). and found that the robust accuracy drop dramatically when e 2 22/255 especially all of the adversarial training acceleration methods. In spite of this,
16

---

Under review as a conference paper at ICLR 2021
63
50
Accuracy Rate (%)
40
35
25
TEA065 18
10
100043
FCOM
PGD iterations
Figure 4: The accuracy rates of Wide ResNet34 trained by different adversarial training methods with different epsilon on CIFARIO. We use PGD with 8/255 and experiments over the number of attack iterations E = {10, 20, 50, 100, 200).
2/ 2/255 performe thorough
our ATCD method still aces any other acceleration method, even outperforming the standard PGD training.
60
TRADES
Accuracy Rate (%)
45
35
30
20
15
8/255
16/255
32/255
64/255
128/255
Epsilon
Figure 5: The accuracy rates of Wide ResNet34 trained by different adversarial train- ing methods with different epsilon on CIFARIO. We use PGD with 100 iterations and well-tuned steps attacker to perform thorough experiments over epsilon parameters (8/255, 16/255, 32/255, 64/255, 128/255).
A.3.6 SENSITIVE ANALYSIS OF AND P
In the ATCD algorithm, the main parameters include the balance factors À and Figure/6|poor the results of the parameter sensitivity analysis. We broadly investigate different ratios of A/p p ranging from 1/32 to 32. It can be found that when the parameters are changed, the algorithm performs stably. Models can achieve rather better accuracy when the ratio of X approaches and the best performance appears when A/p 4. It is worth mentioning that our proposed objective function will degenerate into the original generation objective widely used in adversarial examples generation stage when p & X. Therefore, ATCD will be practically equal Free with resampling when A/p 32, which outperforms Free itself and shows the benefits brought from the Metropolis Haatiiss correction.
17

---

Under review as a conference paper at ICLR 2021
95
85
75
70
Accuracy Rate (%)
65
55
50
45
40
35
30
25
20
15
10
AA AZaC2
(1,32) (1.16) (1.8) (1.4) (1.2) (1.1) (2.1) (4.1) (8.1) (16.1) (32,1)
(2, p)
Figure 6: The accuracy rates of Wide ResNet34 trained by our method with different ratios of A/P ranging from 1/32 to 32 on CIFAR10 10. We use several attackers including PGD-20, MI-FGSM-20. CW and AA validate the stability of parameters.
A.4 COMPLEXITY ANALYSIS
A typical adversarial training method contains three major steps: forward propagation, backprop- agation, and adversarial examples generation. Suppose that deep neural network contains L layers (including input and output layer). d, denotes the number of nodes of in each layer, where I E (1,.... L.). Assume the dataset has Z training examples and the number of training epochs N. Here we ignore the run-time of batch normalization for simplification.
Forward propagation. We see that for each layer a matrix multiplication, and an activation function is computed. It is easy to know that naive matrix multiplication has asymptotic run-time of (d*). (d").am and the activation operator g is an elementwise function, we know that has a run-time of O (d). So the total forward run-time therefore becomes Oforward = NZ (Omul + Ouctivation) = NZ (Eh, (d ((-))t-)aaa (( (1)))) + (d ()) E (d = NdL + d).
Backpropagation. The run-time complexity of backpropagation can be computed in similar manner. Suppose 2(" the raw output signal the i-th neuron in layer before the activation function has been applied. We introduce new variable 65²) which is the error-sum of neuron i in layer
1. So the total backpropagation run-time can be calculated as Obackward = NZ (Ouiddle
O Ilstt + O update)) = -N
(( (ws(tt))) ((tt)) o (x")) +0
(v/))
(1.) o oogg (2(1)))+
DS. (d()) NZL d3 d dd.
Adversarial Training. Recall that adversarial training is used adversarial examples to train DNN models to improve the robustness. So the major difference between adversarial training and normal training is that adversarial training contains the process of adversarial examples generation. Here we show several methods of adversarial examples generation compare their time complexity.
FGSM: Since FGSM finds the adversarial perturbation in the direction of the loss gradient only once, FGSM includes two full forward propagation, one backpropagation without weight updating, and one backpropagation with weight updating. So the total run-time of FGSM is: OFGSM = 2N (dd d) L-1 df jdl.
PGD: PGD can be considered as an iterative version of FGSM repeating times updating procedure of adversarial examples generation with several random restarts. That means PGD includes one full forward propagation, forward propagation and backpropagation without weight updating. Thus, the total run-time
backpropagation with weight updating. times
of PGD can be computed by: OPGD NZI(r [d d) 1-1 d + +dd2 + 44d
18

---

Under review as a conference paper at ICLR 2021
The major time cost of PGD results from conducting T sweeps of forward and backward propagation. YOPO restricts most of the forward and back propagation within the first layer of DNN during adversary updates, which effectively reduces the total number of full forward and backward propagation. The total number of full forward propagation is only m apart from times forward propagation at the first layer. The backward process is divided into m times backpropagation except the first layer and n times backpropagation at the first layer. Therefore, the total run-time of YOPO can be formulated as: OYOPO
YOPO:
Nm + [d + d) + (L - 1)( -T d + I. d. + d) + ra + + dR)].
ATCD: Our ATCD method relies on the replay of current samples and the searching trajectory of HMC. So the total run-time of ATCD can be easily expressed by: OATCD N'ZLK(T + [) + d) + ا d' + 7, + 1
K(T41) d). Because of N' TK. our ATCD
is r+1 times faster than PGD but K(T+1) slower than FGSM.
K(741]
A.5 LONG OR SHORT-RUN TRAJECTORY?
Both long and short-run trajectory of HMC simulating have been able to fit with contrastive adversarial training. In our experiments we use K = 2 to achieve the trade-off between performance and efficiency. But we have found that if condition allows (without computation time being limited) and just pursue the best performance, long-run HMC with Cestropolis-Hsstinn correction are preferable terms of equilibrium distribution.
As shown in Figure 7 we fix the number of parameters updating as 7" = 4 and sweep over the length of trajectory K from 1 to 8 (almost in line with PGD adversarial training). In all configurations, ATCD has much higher robust accuracy than Free. This shows evidence that ATCD is a more effective adversarial training method to improve the robustness of models compared Free. We also find that by using contrastive divergence, we could reduce the lengths of MCMC trajectory per iteration when compared to PGD-10 and still allow for relatively high robustness. This gives a 3x speedup over our fastest short-run ATCD (K 1). Further, robust accuracies against different adversarial attacks share the similar trend and ATCD using longest trajectory has the best result. So if time allows, we recommend using a long-run trajectory of HMC with Metropolis-Hasiinn approach equilibrium distribution. But even so, the clean accuracy of adversarially trained model still decreases with the growth of the length of simulating trajectory. Besides, some generated adversarial examples will be recalled by cetropolis-Hastinne longer, which would increase the additional time cost. Overall, the short-run trajectory is slightly less robust than the long-run trajectory, but in total, the short-run trajectory increases in speed and maintains the suitable clean accuracy to be a worthwhile trade-off.
correction to better
correction when the length of trajectory becomes
Another interesting observation we found while using long-run trajectory of HMC is that the model would use the length of the trajectory to perform like "clustering". We discovered that when generating adversarial examples (the inner loop in Algorithm 2) on CIFAR10, these samples are always identified as one specific category (e.g. trunk) at very beginning of trajectory. When the length of trajectory grows, they could almost always be identified as another category (e.g. deer). However, if we use short-run trajectory, this phenomenon will not happen. This behavior is likely some compromise of ATCD since contrastive adversarial training with short-run trajectory accelerates the process of cquilibrium distribution by discarding the solution space exploring and Metropolis-Hasting correction.
A.6 WHAT IS THE ADVANTAGE OF HMC FRAMEWORK FOR ADVERSARIAL LEARNING?
In this section we describe the advantage of reformulating adversarial examples generation from the perspective of HMC. In addition to transforming the process of adversarial training into the trajectory of MCMC simulating, thereby introducing more well studied solutions of high computational cost of MCMC, HMC also brings the following advantages:
First, we reinterpret the adversarial examples generation as the solution space searching through the conservation transformation of two kinds of energy. If we further consider a standard discriminative classifier of p(yla (z) as an energy based model for the joint distribution p(æ, y), we can apply our framework in both discriminative and generative modeling. That means we enable this framework
19

---

Under review as a conference paper at ICLR 2021
90
80
Accuracy Rate (%)
70
60
50
Natuml
40
PGD-20 Attek
MI-FGSM-20 Atacck
30
2
K-trajectory
Figure 7: The accuracy rates of Wide ResNet34 trained by ATCD with different lengths of trajectory on CIFARIO, Basically, we could find that the robust accuracy of model increases with the length of the trajectory.
not only to improve the robustness of modern classifier, but also to generate samples as generative models.
Second, when simulating adversarial examples generation in HMC framework, different types of adversarial attacks can be considered as special cases of HMC. Actually, there will be more powerful searching algorithm if new potential or kinetic (or both) energy is designed (Baigoo et al. (2017); Carlini Wagner (2017); Zheng et al. (2019); Rony et al. (2019), e.g. mapping the clipped gradient descent into tanh space adding KL--diveggecce term.
Third, the unique advantage brought by HMC itself is that it can better explore the distribution of adversarial examples, which is different from heuristic algorithm (e.g. PGD with random restarts). That means we can design a more powerful adversarial attack from the view of HMC. To further verify this point, we apply generating method for adversarial examples based HMC to attack the real-world celebrity recognition APIs in Clarifa These celebrity recognition APIs allow users to upload any face images and recognize the identity of them with confidence score. The users have knowledge about the dataset and types of models used behind these online systems. We choose 10 pairs of images from the LFW dataset and learn perturbations from local facenet model launch targeted attack, whose goal is to mislead the API to recognize the adversarial images as our selected identity. We randomly pick up 10 celebrities
victims from Google and 10 existing celebrities as targets from LFW, ensuring that all colors and genders are taken into account. Then we apply the same strategy as Geekpwn CAAD 2018 method that pulls victims towards their corresponding targets by the inner product of their feature vectors and generates noise to them. Finally, we examine their categories and confidence scores by uploading these adversarial examples to the online systems API. We fix 16 and total iteration number N 100. It is worth mentioning that we generate a sequence of adversarial examples to show how well HMC-based method explores the space of adversarial examples. Results shown in Fig. 9 and I0
Figure presents presee&presentss three celebrity pairs where both Geekpwn CAAD 2018 method and our HMC-based method successfully fool the face recognition system to recognize them the target celebrities. But the confidence score of each pair generated by Geekpwn CAAD 2018 method is lower than any in generated sequence. Figure presents three pairs where the generated adversarial examples by our HMC-based method all successfully fool the system as the target celebrities while Geekpwn CAAD 2018 method fails. Figure shows a case that any one in the generated sequence by bMC-based method can fool the system with very high confidence while the adversarial examples
20

---

Under review as a conference paper at ICLR 2021
generated by Geekpwn CAAD 2018 method being "caught" which is recognized by the online systems and inferred to the true category.
We also list two failure cases in Figure IT Note that the former one is slightly harder because the source and target are different races, but HMC-based method still generates few success samples. When looking at the big picture, these failure might be caused by our HMC-based method (or all the white-box adversarial attack methods) which are dependent of the targeted classifier. The decision boundary of different models may be slightly different from the global perspective, but the targeted attack forces the attacker to focus on the local decision boundary, which greatly magnifies this difference. is worth noting that the phenomenon when using long-run trajectory with Metropolis- Matstnglss-Hastings correction which we mentioned in Appendix A.5 occurs again.
Our generated sequence
Pairs
Score: 98.2
Score: 95.4
Score: 95.3
Score: 94,7
Score: 94.4
Geekpwn CAAD 2018
Original
Target
Score: 86.1 Our generated sequence
Pairs
Score:
Score:
Score: 78.1
Geekpwn CAAD 2018
Original
Tarpet
Our generated sequence
Pairs
Score: $2.0
Score: 27.9
Score: 30.4
Score: 44,6
Score: 38.4
Original
Tanget
Geekpwn CAAD
Score: 22.6
Figure 8: Success cases for both HMC-based method and Geekpwn CAAD 2018 method. The confidence of adversarial examples generated by Geekpwn CAAD 2018 for each pair is lower than any one in our generated sequence.
21

---

Under review as a conference paper at ICLR 2021
Our generated sequence
Pairs
Scorc: 56.3
Score: 62.2
Scorc: 66.6
Soare: 66.7
Score: 69,
Gookpwn CAAD 2018
Original
Target
Failed Our generated sequence
Pairs
Score: 23.6
Score: 17.8
Score: 18.4
Score: 18.6
Score: 22.2
Original
Target
Geekpwn CAAD 2018
Falled Our generated sequence
Pairs
Score: 17.9
Score: 12.1
Score: 11.8
Score: 6.4
Score: 5,3
Geekpwn CAAD 2018
Original
Target
Failed
Figure 9: Three cases where our method succeeds but Geekpwn CAAD 2018 method fails
Our generated sequence
Pairs
Score: 96.6
Score: 98.8
Score: 98.9
Score: 99.2
Score: 99.4
Geckpwn CAAD 2018
Original
Target
Catched, 34.3
Figure 10: A case shows that an adversarial example generated by Geekpwn CAAD 2018 method is classified correctly by the online systems but any one in our generated sequence still fool the systems with high confidence score.
22

---

Under review as a conference paper at ICLR 2021
Our generated sequence
Pairs
Score: 6.7
Score:
Score: 18.7
Scorc: 45.6
Score: 7.9
Original
Target
(Don Chendie)
Peter Olbooe
Carson
Geekpwn CAAD 2018
Catched,
Our generated sequence
Pairs
Score: 4.3
Score: 22.0
Score: 15.5
Score: 16.3
Score: 4.3
Original
Targot
(Bill Gates)
Cnson Kidd)
Rob Schneider
Michael Emersum
Geekpwn CAAD 2018
Cutched, 13.2
Figure 11: Typical failure cases of our method. In the task of attacking Don Cheadle as Peter O'Toole (the first row), HMC-based method only has a partial success due their different colors. HMC-based method also fails attacking Bill Gates as Jason Kidd (the second row). We believe that Bill Gates an easy case for mutiple classifiers since there exist numerous related pictures about him and the recognition system will give priority to the correctness of him.
23


Reviews:
[{"review_id": "iZc7pGkPuZJ", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: \nIn this paper, the authors reformulated the generation of adversarial examples as an MCMC process and present a new adversarial learning method called ATCD, which approaches the equilibrium distribution of adversarial examples with only a few iterations by building from small modifications of the standard Contrastive Divergence. Extensive results with comparisons on various datasets show that ATCD achieves a trade-off between efficiency and accuracy in adversarial training.\n\nComments:\n\n1 . The proposed algorithm ATCD views the generation of adversarial examples as a sampling procedure. Specifically, it can be seen as performing HMC sampling. Also, it modifies the adversarial example generation objectives using a modified contrastive divergence objective. Compared with free-adversarial-training, which also utilize mini-batch replay and use the last iterate’s result as initialization, the difference only lies in the adversarial example generation objective and the extra noise brought by the HMC sampling procedure. However, it is still not clear to me what actually causes the performance improvement. The different objectives or the sampling noise? I would suggest the authors conduct ablation studies to find out this answer and it would be clearer for the readers to understand the true driving force for the proposed method.\n\n2 . In Eq (13) what is Q0 and Q1? It is better to formally define them. The intuition for this objective is a bit confusing to me. For standard contrastive divergence, it is the same as measuring the difference between the output probability between init point and K-step updated point. Here since the equilibrium distribution is unknown (fixed but unknown right?), how to compute the objective here with rho and lambda parameters? And why using different rho and lambda helps?\n\n3 . Notice that even adversarial training based algorithms could cause obfuscated gradient problem, therefore, it might be a good idea to further evaluate model robustness via totally gradient-free methods, such as hard-label attacks. I would suggest the authors to also evaluate using the following method: \n\n“RayS: A Ray Searching Method for Hard-label Adversarial Attack” KDD (2020)\n\nIn order to make the experimental results more convincing.\n\n4 . There are some other recent adversarial training methods that the authors might want to comment on\n\n\"Improving adversarial robustness requires revisiting misclassified examples.\" ICLR (2019).\n\"On the Convergence and Robustness of Adversarial Training.\" ICML (2019).\n\n\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "IuLdKSKM0eA", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper proposed a new adversarial attack method by using Markov chain Monte Carlo.  Based on this attack method, a new adversarial learning method called adversarial training by using Contrastive Divergence (ATCD) which approaches equilibrium distribution of adversarial examples with only a few iterations is performed. The experimental results demonstrated the effectiveness of ATCD.\n\nPros:\n1. The method of generating adversarial examples is new and efficient.\n2. Experimental results with comparisons on ImageNet, CIFAR and MNIST datasets show that ATCD achieves a good trade-off between efficiency and accuracy in adversarial training.\n\nCons:\n1. There is a notable natural accuracy gap between ATCD and Madry's PGD method and Free-m.\nMany papers were aware of overfitting in adversarial training such as 'Overfitting in adversarially robust deep learning'. Nature accuracy and robust accuracy are sometimes conflicted in the latter epochs. It seems like some plots like Figure 3 may answer the question, but Figure 3 only plots the first 40 iterations (I guess it means 'epochs' here), while the total epoch is 105.\nSo only observe the results of the last epoch may not fair since maybe different methods have different convergence rates.\n2. Some hyperparameter settings and sensitive analyses like $\\rho$ and $\\lambda$ are missing.\n3. Although, the experimental results show the efficiency of ATCD, the formal time complexity analysis of ATCD should be performed.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "FNdhzvGM49o", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Adversarial examples are time-consuming to generate. In this paper, the adversarial training is reformulated as a combination of stationary distribution exploring, sampling, and training. A Hamiltonian system is proposed to model data samples from their initial states, and is shown as the general form of FGSM. The sample generation method is proposed via contrastive divergence with few training iterations. Experiments have been validated on datasets. \n\nThe formulation of HMC sounds interesting and its relationship towards FGSM/PGD is discussed. There are few issues towards the theoretical discovery and experimental validations:\n\n1. The FGSM/PGD based attacks are formulated as the degeneration of HMC, as explained in Sec. 4.2. However, the experiments on the benchmarks consistently indicate PGD performs better than the proposed method ATCD. As ATCD utilizes a more advanced HMC, there lacks explanations of why the results do not correspond to the theory.  \n\n2. The proposed ATCD is claimed to efficiently generate adversarial examples while the performance seems to suffer from limited iterations. Is it possible to increase the iterations of ATCD for performance improvement?\n\n3. As the efficiency is claimed as a major contribution of ATCD, the computational complexity and time cost compared to other methods (e.g., FGSM, PGD) shall be reported as well.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose adversarial training using contrastive divergence based on ideas from Hybrid Monte Carlo methods.
On the positive side the shown experimental results are promising both in terms of robustness and efficiency. On the negative side the paper seems to be written in a hurry. At several places terms are not defined, not explained or important details (e.g. parameters of the attack algorithms) are missing. 

Thus the paper is not ready for publication yet and below the bar for ICLR but I encourage the authors to submit a significantly revised version to another conference.

Details:
- in (7) N(x) is nowhere explained or defined, here also several threat models are introduced but later on only l_infty seems
to be used e.g in Algorithm 1 (should be clarified)
- as noted in the reviews the definition in (13) is based on quantities nowhere introduced
- the potential U is only defined in the Algorithm (but then the arguments do not match with the RHS)
- the kinetic energy in the algorithm is different from what has been used before
- the parameters for the attacks used are not reported
- why is AutoAttack not used for all datasets? They report 28% robust accuracy for the model trained by the Madry group
 (https://github.com/MadryLab/robustness) whereas in the present paper it is 35%.
- the scales of the plots should be chosen such that the curves can be distinguished
Decision: Reject