A DVERSARIAL T RAINING USING C ONTRASTIVE D IVERGENCE
Anonymous authors
Paper under double-blind review
A BSTRACT
To protect the security of machine learning models against adversarial examples, adversarial training becomes the most popular and powerful strategy against various adversarial attacks by injecting adversarial examples into training data.
However, it is time-consuming and requires high computation complexity to generate suitable adversarial examples for ensuring the robustness of models, which impedes the spread and application of adversarial training.
In this work, we reformulate adversarial training as a combination of stationary distribution exploring, sampling, and training.
Each updating of parameters of DNN is based on several transitions from the data samples as the initial states in a Hamiltonian system.
Inspired by our new paradigm, we design a new generative method for adversarial training by using
Contrastive Divergence (ATCD), which approaches the equilibrium distribution of adversarial examples with only few iterations by building from small modiﬁcations of the standard Contrastive Divergence (CD).
Our adversarial training algorithm achieves much higher robustness than any other state-of-the-art adversarial training acceleration method on the ImageNet, CIFAR-10, and MNIST datasets and reaches a balance between performance and efﬁciency.
I NTRODUCTION
Although deep neural networks have become increasingly popular and successful in many machine learning tasks (e.g., image recognition He et al.
(2016b), speech recognition Hinton et al.
(2012); van den Oord et al.
and natural language processing Hochreiter & Schmidhuber (1997);
Vaswani et al.
(2017)), the discovery of adversarial examples Szegedy et al.
Goodfellow et al.
(2015) has attracted great attention to strengthening the robustness of deep neural network (DNN) under such subtle but malicious perturbations.
These crafted samples pose potential security threats in various safety-critical tasks such as autonomous vehicles Evtimov et al.
or face recognition
Sharif et al.
; Dong et al.
(2019), which are required to be highly stable and reliable.
Unfortunately, it is considered to be unresolved since no ﬁnal conclusion has yet been reached on the root of the adversarial examples.
Many defense methods Papernot et al.
Na et al.
Buckman et al.
(2018) motivated by different interpretability of adversarial examples Goodfellow et al.
; Fawzi et al.
; Ma et al.
(2018) were broken within a short time, indicating that there is still no thorough solution to settle this matter once and away.
Nonetheless, adversarial training
Szegedy et al.
; Goodfellow et al.
(2015) has shown its ability to make classiﬁers more robust against sorts of attacks than any other defenses in Madry et al.
; Athalye et al.
It offers an intuitive approach to handle the problem, which ﬁrst obtains suitable adversarial examples by solving the inner maximization problem and then update the parameters of ML model from these examples by outer minimization.
More and more advanced defenses Kannan et al.
Lin et al.
; Xie et al.
; Zhang et al.
(2019c) are developed based on adversarial training.
However, a major issue of the current adversarial training methods is their signiﬁcantly higher computational cost than regular training.
It often needs multiple days and hundreds of GPUs for
ImageNet-like datasets to achieve better convergence Xie et al.
(2019), which makes it nearly intractable and impractical for large models on tons of data.
Even for small-sized datasets like
CIFAR10, adversarial training takes much longer time than regular training.
To address this issue, we formulate the problem of generating adversarial examples in a Hamiltonian
Monte Carlo framework (HMC) Neal et al.
(2011), which can be considered as exploring the stationary distribution of adversarial examples for current parameters.
The high computational cost of adversarial training can be easily attributed to the long trajectory of HMC producing.
Therefore, we propose a new adversarial training algorithm called ATCD for strengthening the robustness of target models, enlightened by the Contrastive Divergence (CD) Hinton (2002).
We minimize the difference of Kullback-Leibler divergence between two adjacent sampling steps to avoid running long Monte-Carlo Markov Chains (MCMC).
Instead of running the chain to achieve equilibrium, we can simply run the chain for fewer or even only one full step and then update the parameters to reduce the tendency of the chain to wander away from the initial distribution on the ﬁrst step.
Our approach is advantageous over existing ones in three folds:
• We offer a new perspective on adversarial examples generation in a HMC framework.
From the view of HMC, we bridge the relationship between several adversarial examples generating methods and MCMC sampling, which effectively draw multiple fair samples from the underlying distribution of adversarial examples.
• By analyzing the trajectory shift of different lengths of MCMC simulating, we speed up the adversarial training by proposing a contrastive adversarial training (ATCD) method, which accelerates the process of achieving distribution equilibrium.
• We thoroughly compare the effectiveness of our algorithm in various settings and different architectures on ImageNet, CIFAR10 and MNIST.
Models trained by our proposed algorithm achieve robust accuracies markedly exceeding the ones trained by regular adversarial training and the state-of-the-art speedup methods when defending against several attacks.
B ACKGROUND AND R ELATED W ORK
Adversarial Defense.
To deal with the threat of adversarial examples, different strategies have been studied to ﬁnd countermeasures to protect ML models.
These approaches can be roughly categorized into two main types: (a) detection only and (b) complete defense.
The former approaches Bhagoji et al.
; Ma et al.
; Lee et al.
; Tao et al.
; Zhang et al.
(2018) is to reject the potential malignant samples before feeding them to the ML models.
The latter defenses obfuscate the gradient information of the classiﬁers to confuse the attack mechanisms including gradient masking
Papernot & McDaniel (2017); Athalye et al.
(2018) or randomized models Liu et al.
; Xie et al.
(2018a); Lecuyer et al.
Liu et al.
There are also some add-ons modules Xie et al.
; Svoboda et al.
; Akhtar et al.
; Liao et al.
(2018) being appended to the targeted network or adversarial interpolation schemes Zhang & Xu (2020); Lee et al.
(2020) to protect deep networks against the adversarial attacks.
Fast Adversarial Training.
Besides all the above methods, adversarial training Goodfellow et al.
; Kurakin et al.
; Kannan et al.
; Madry et al.
; Tram ` er et al.
(2018); Liu & Hsieh (2019); Wang et al. (2020; 2019) is the most effective way to ensure better robustness, which has been widely veriﬁed in many works and competitions.
However, limited works focus on boosting robust accuracy with reasonable training speed.
Free Shafahi et al.
(2019) recycle the gradient information computed to reduce the overhead cost of adversarial training.
YOPO Zhang et al.
(2019b) recast the adversarial training as a discrete time differential game and derive a Pontryagin’s Maximum Principle (PMP) for it.
Fast-FGSM Wong et al.
(2020) combines FGSM with random initialization to accelerate the whole process.
Markov Chain Monte Carlo Methods.
Markov chain Monte Carlo (MCMC) Neal (1993) provides a powerful framework for exploring the complex solution space and achieves a nearly global optimal solution independent of the initial state.
But the slow convergence rate of MCMC hinders its wide use in time critical ﬁelds.
By utilizing the gradient information in the target solution space, Hamiltonian (or Hybrid) Monte Carlo method (HMC) Duane et al.
; Neal et al.
(2011) achieves tremendous speed-up in comparison to previous MCMC algorithms.
Multiple variants of HMC Pasarica &
Gelman (2010); Salimans et al. (2015); Hoffman & Gelman (2014) were yet to be developed for adaptively tuning step size or iterations of leapfrog integrator.
The fusion of MCMC and machine learning Tu & Zhu (2002); Chen et al.
; Song et al.
; Xie et al. (2018b) also shows great potential of MCMC.
Contrastive Divergence.
Contrastive Divergence (CD) has achieved notable success in training energy-based models including Restricted Boltzmann Machines (RBMs) as an efﬁcient training method.
The standard approach to estimating the derivative of the log-likelihood function is using the Markov chain Monte Carlo Gilks et al.
(1995), which can be expressed as the difference of two expectations.
It runs k MCMC transition steps at each iteration T and iteratively generates a sequence of parameter estimates { θ T } T ≥ 0 given an i.i.d.
data sample { X i } Ni =1 ∼ p ¯ θ , where p ¯ θ is the distribution of target samples for the true parameter ¯ θ .
To reduce the computational complexity, the traditional Contrastive Divergence algorithm computes approximate RBM log-likelihood gradient setting k = 1 .
Various works are devoted to addressing the problem of the vanilla CD afterwards, such as uncontrolled biases and divergence Carreira-Perpinan & Hinton (2005);
Yuille (2005); MacKay (2001); Fischer & Igel (2011; 2014).
Persistent CD (PCD) and its relevant works Tieleman (2008);
Tieleman & Hinton (2009); Desjardins et al.
(2010) show a steady decrease of the log-likelihood in many numerical analysis while some works Schulz et al.
; Fischer & Igel (2010) also give examples in which PCD failed to converge.
Although none of these works provide a solid convergence guarantee since the major problems of CD family stem from the fact that the stochastic approximation to the true gradient is a biased estimator, our work do not need the exact values of the derivatives.
Actually, we just borrow the idea from the vanilla CD to accelerate the process of distribution equilibrium over the visible variables instead of discovering the unknown distribution
Pang et al.
; Alayrac et al.
P RELIMINARIES
Considering a target DNN model ˆ f ∈ F , where F is the solution function space for classiﬁcation task.
We assume softmax is employed for the output layer of the model f ( · ) and let f ( x ) denote the softmax output of a given input x ∈ R d , i.e., f ( x ) : R d → R C , where C is the number of categories.
We also assume that there exists an oracle mapping function f ∗ ∈ F : x (cid:55)→ y ∗ , which pinpoints the belonging of the input x to all the categories by accurate conﬁdence scores y ∗ ∈ R C .
The common training is to minimize the cross-entropy (CE) loss J ce , which is deﬁned as:
[Equation]
where y is the manual one-hot annotation of the input x since y ∗ is invisible.
The goal of Eq.
(1) is to update the parameters of f for better approaching f ∗ , which leads to f ( x ) ≈ y ≈ y ∗ =
f ∗ ( x ) .
Suppose the target DNN model correctly classiﬁes most of the input after hundreds of iterations, it will still be badly misclassiﬁed by adversarial examples (i.e., arg max c ∈{ 1 , ··· ,C } f (˜ x ) c (cid:54) =
y [ c ] ).
In adversarial training, these constructed adversarial examples are used to updates the model using minibatch SGD.
The objective of this minmax game can be formulated as a robust optimization following Madry et al.
[Equation]
where the inner maximization problem attempts to generate the most easily misclassiﬁed samples while the outer minimization problem is to search a mapping function f (cid:48) which is the closest one to the oracle f ∗ .
H AMILTONIAN M ONTE C ARLO FOR A DVERSARIAL L EARNING 4. A N O VERVIEW OF MCMC AND H AMILTONIAN M ONTE C ARLO
The crux of this work relies on offering a fundamentally different view of adversarial example generation, which simulates the inner maximization in Eq.
(2) as proposing dynamics by HMC.
We now give the overall description of Metropolis-Hasting based MCMC algorithm.
Suppose p is our target distribution over a space D , MCMC methods construct a Markov Chain that has the desired distribution p as its stationary distribution.
At the ﬁrst step, MCMC chooses an arbitrary point x 0 as the initial state.
Then it repeatedly performs the dynamic process consisting of the following steps:
(1) Generate a candidate sample ˜ x as a “proposed” value for state x t +1 from the candidate-generating density Q ( x t | ˜ x ) .
(2) Compute the acceptance probability ξ = min(1 , p (˜ x ) Q ( x t | ˜ x ) p ( x t ) Q (˜ x | x t ) ) , which is used
to decide whether to accept or reject the candidate.
) Accept
the
candidate sample as the next state with probability
ξ by setting x t +1 =
˜ x .
Otherwise reject the proposal and remain
x t +1 = x t
Although MCMC makes it possible to sample from any desired distributions, its random-walk nature makes the Markov chain converge slowly to the stationary distribution p ( x ) .
In contrast, HMC employs physics-driven dynamics to explore the target distribution, which is much more efﬁcient than the alternative MCMC methods.
Before introducing HMC, we start out from an analogy of Hamiltonian systems in Neal et al.
(2011) as follows.
Suppose a hockey puck sliding over a surface of varying height and both the puck and the surface are frictionless.
The state of the puck is determined by potential energy U ( θ ) and kinetic energy K ( v ) , where θ and v are the position and the momentum of the puck.
The evolution equation is given by the Hamilton’s equations:
[Equation]
Due to the reversibility of Hamiltonian dynamics, the total energy of the system remains constant:
[Equation]
As for HMC, it contains three major parts: (1) Hamiltonian system construction; (2) Leapfrog integration; (3) Metropolis-Hastings correction.
Firstly, the Hamiltonian is an energy function for the joint density of the variables of interest θ and auxiliary momentum variable v , so HMC deﬁnes a joint distribution via the concept of a canonical distribution:
[Equation]
where τ = 1 for the common setting.
Then, HMC discretizes the system and approximately simulates Eq.
(3) over time via the leapfrog integrator.
Finally, because of inaccuracies caused by the discretization, HMC performs Metropolis-Hastings Metropolis et al.
(1953) correction without reducing the acceptance rate.
According to Eq.
(4) and (5), the joint distribution can be divided into two parts:
[Equation]
Since K ( v ) is an auxiliary term and always setting K ( v )
= v T
I − 1 v/ 2 with identity matrix I for standard HMC, our aim is that the potential energy U ( θ ) can be deﬁned as U ( θ )
= − log p ( θ ) to explore the target density p more efﬁciently than using a proposal probability distribution.
If we can calculate ∇ θ U ( θ ) = − ∂ log( p ( θ ))
∂θ , then we can simulate Hamiltonian dynamics that can be used in an
MCMC technique.
4. S IMULATING A DVERSARIAL E XAMPLES G ENERATING BY HMC
Assume that the adversarial examples for x with label y are distributed over the solution space Ω .
Given any input pair ( x, y ) , for a speciﬁed model f ( · ) ∈ F with ﬁxed parameters, the adversary aims to ﬁnd such examples ˜ x that can mislead the model:
[Equation]
where N ( x ) is the neighboring regions of x and deﬁned as x (cid:48) ∈
N ( x ) :=
[Equation]
From the perspective of Bayesian statistics, we can make inference about adversarial examples over a solution space Ω from the posterior distribution of ˜ x given the natural inputs x and labels y .
[Equation]
In Hamiltonian system, it becomes to generate samples from the joint distribution p ( θ, v ) .
Let θ = ˜ x , according to Eq.
(8) and (6), we can express the posterior distribution as a canonical distribution (with τ = 1 ) using a potential energy function deﬁned as:
[Equation]
1.00E1.00E1.00E1.00E1.00EL 2 - d i ff e r e n ce
Epoches
Layer5 Layer -0.
C
o s i n e
S i m il a r it y
Epoches
Layer5Layer
For a layer, we measure the L difference and cosine similarity of the running
Since J (˜ x, y ) is the usual classiﬁcation likelihood measure, the question remains how to deﬁne p (˜ x | x ) .
A sensible choice is a uniform distribution over the L p ball around x , which means we can directly use a DNN classiﬁer to construct a Hamiltonian system for adversarial examples generation as the base step of HMC.
Recall that the development of adversarial attacks is mainly based on the improvement of the vanilla fast gradient sign method, which derives I-FGSM, PGD and MI-FGSM.
For clarity, we omit some details about the correction due to the constraint of adversarial examples.
The core policy of the family of fast gradient sign methods is:
[Equation]
(10) where g t is the gradient of J at the t -th iteration, i.e., ∇ x J (˜ x t − 1 , y ) .
It is clear that the above methods are the specialization of HMC by setting:
[Equation] [Equation]
More speciﬁcally, I-FGSM can be considered as the degeneration of HMC, which explicitly updates the position item θ but implicitly changes the momentum item v at every iteration.
One of the derivation of I-FGSM, MI-FGSM, has explicitly updated both θ and v by introducing g t = µg t − 1 +
||∇ J
(˜ x t − 1 ,y )
|| 1 ∇ J (˜ x t − 1 , y ) after Eq.
(10) at each step with the decay factor µ = 1 .
The other derivative PGD runs Eq.
(10) on a set of initial points ˜ x 0 ∈ (cid:110) ˜
x (1)0 , ˜ x (2)0 , · · · , ˜ x ( S ) 0 (cid:111) adding different noises, which can be treated as a parallel HMC but the results are mutually independent.
A DVERSARIAL T RAINING USING C ONTRASTIVE D IVERGENCE
As mentioned in Section 4, the inner maximization problem can be reformulated as the process of
HMC.
It is obvious that the high computational cost of adversarial training can be easily attributed to the long trajectory of MCMC searching for the stationary distribution of adversarial examples.
Nevertheless, does training a robust model really need such a long trajectory?
To answer this question, we consider studying the gradient of the loss since the training procedure (obtaining ∇ v K ( v ) and ∇ θ
U
θ ) and updating parameters of DNN) is a ﬁrst-order method.
To quantify the extent to which the parameters in a layer would change in reaction to the length of the trajectory, we measure the difference between the gradients of each layer running different lengths of trajectory.
This leads to the following deﬁnition.
Deﬁnition 5.1.
Let W ( K ), ..., W ( K ) n be the parameters of each of the n layers and (cid:0)
˜ x ( k ) ;
y (cid:1) , (cid:16) ˜
x ( k (cid:48) ) ; y (cid:17) be the batch of input-label pairs used to adversarially train the network.
We deﬁne trajectory shift (TS) of activation i along different lengths of trajectoriry k and k (cid:48) to be the difference
[Equation]
dist , where
[Equation]
The difference between g k,i and g (cid:48) k,i thus reﬂects the change in the optimization landscape of parameters W i caused by the changes to its input, which captures the shift of different lengths of trajectory that could have an inﬂuence on adversarial training.
Equipped with this deﬁnition, we measure TS on ResNet34 trained with adversarial examples simulating by different lengths of trajectory ( k = 2 , k (cid:48)
= 10 ) throughout the training.
Results are shown in Fig.
Although the situation in the bottom layer (e.g. layer5) is rather different than that in the top layer (e.g. layer20), both the direction and the magnitude of the gradients are quite close when simulating different lengths of trajectory.
These evidences suggest that running a full trajectory for many steps is too inefﬁcient since the model changes very slightly between parameter updates.
Thus, we might take advantage of that by initializing a HMC at the state in which it ended for the previous model.
This initialization is often fairly close to the model distribution, even though the model has changed a bit in the parameter update.
Besides, the high acceptance rate of HMC indicates that it is not necessary to run a long trajectory from the initial point.
Therefore, we can simply run the chain for small (or even one) full step and then update the parameters to reduce the tendency of the chain to wander away from the initial distribution on the ﬁrst step instead of running the full trajectory to equilibrium.
We take small number K of transitions from the data sample { x i } ni = 1 as the initial values of the MCMC chains and then use these K -step MCMC samples to approximate the gradient for updating the parameters of the model.
Algorithm1 summarizes the full algorithm.
Moreover, we also present a new training objective function J cd , which minimizes the difference of
KL divergence between two adjacent sampling steps to substitute the common KL loss:
[Equation]
where || denotes a Kullback-Leibler divergence and ρ and λ are the balanced factors.
The intuitive motivation for using this J cd is that we would like every state in HMC exploring to leave the initial distribution Q 0 and Q 0 || Q ∞ would never exceed Q 1 || Q ∞ until Q 1 achieves the equilibrium distribution.
We set λ = 2 , ρ = 1 and analyze how this objective function inﬂuences the partial derivative of the output probability vector with respect to the input.
Due to the fact that the equilibrium distribution Q ∞ is considered as a ﬁxed distribution and the chain rule, we only need to focus on the derivative of the softmax output vector with respect to its input vector in the last layer as follows:
[Equation]
[Equation]
where ∆ f = f ω ( x K ) − f ˜ ω (˜ x ) .
Based on this abbreviation, we can easily get the relationship between
Eq.
(14) and ∂J ce ∂
˜ x (cid:48) =
f ω ( x K ) − y .
For each adversarial example generation, Eq.
(14) makes an amendment of y which is determined by the difference of current and the last K -step HMC samples output probability.
Since f ω and f ω ( x ) are more closer to f ∗ and y ∗ than f ˜ ω and f ˜ ω ( x ) , each update of ˜ x would be better corrected.
E XPERIMENTAL R ESULTS
In this section, we focus on the ImageNet Deng et al.
(2009), CIFAR10 Krizhevsky & Hinton (2009) and MNIST LeCun (1998) datasets with extensive experiments to validate the effectiveness of the proposed methods.
For most part of experiments, we compare three standard adversarial training baselinesMadry et al.
; Zhang et al.
(2019d);
Rice et al.
(2020) and three advanced acceleration methodsShafahi et al.
; Zhang et al.
(2019b)
; Wong et al.
(2020) with our ATCD.
More details about experiment setup can refer to Appendix A.1.
Extensive ablation studies on CIFAR10 can also be found in Appendix A.3.
Input: A DNN classiﬁer f ω ( · ) with initial learnable parameters ω 0 ; training data x with visible label y ; number of epochs N ; length of trajectory K ; repeat time T ; magnitude of perturbation ε ; learning rate κ ; step size α . /*Stage-0:
Construct Hamiltonian system*/
U ( θ, ω, ˜ ω, y, k ) =
− J cd (cid:0) f ω ( θ k − 1 ) , f ˜ ω ( θ K ) , y (cid:1) , K ( v ) = | v |
Initialize ω =
˜ ω
= ω 0 , θ K
= θ 0 .
for epoch = 1 · · · N/ ( TK ) do θ
← x + v 0 , v 0 ∼ Uniform( − ε, ε ) .
for t = 1 to T do /*Stage-1:
Generate adversarial examples by K-step contrastive divergence*/ for k = 1 to K do θ k ← θ k − 1 + ε · ∇K ( v t − 1 ) v t ← v t − 1 − α ∇ U
( θ, ω, ˜ ω, y, k ) v t ← clip( v t , − ε, ε ) end for (cid:0) θ K , v t (cid:1)
= (cid:0) θ k , v t (cid:1) , M-H step decides whether it should be accepted or rejected.
/*Stage-2: Update parameters of DNN by generated adversarial examples*/ g ω ← E ( θ,y ) (cid:2) ∇ ω J ce ( f ω ( θ K ) , y ) (cid:3) ˜ ω ← ω ω ← ω − κ g ω end for end for
6. I MAGE N ET
For ImageNet, we ﬁx the total loop times T ∗ K = 4 same as Free-4 Shafahi et al.
for fair comparison.
We report average over the ﬁnal 3 evaluation.
Comparison between free adversarial training and ours are shown in Table 1.
Although the 2-PGD trained ResNet-50 model still maintains its leading role in the best robust accuracy, it takes three times longer than our ATCD method.
Actually, when compared with its high computational cost of ImageNet training, this performance gain can be considered inefﬁcient or even impractical for resource limited entities.
We also compare
ResNet-50 model trained by our ATCD method with the Free-4 trained, model trained by ATCD produces much more robust models than Free-4 against different attacks in almost the same order of time.
Though Fast-FGSM achieves a sterling acceleration, both its clean accuracy and robust accuracy are not satisfactory enough.
[Table]
6. CIFAR
For CIFAR10, we ﬁx the total loop times T ∗ K = 8 same as Free-8 Shafahi et al.
for fair comparison and show the training time of all methods.
We calculate the deviation value of ﬁnal 5 evaluation and report average over 5 runs with different restarts
Zagoruyko & Komodakis (2016) are summarized in Table 2.
From the table, we can see that the naturally trained model (without any adversarial examples) is vulnerable to all the attacks, while the baseline adversarial training methods (PGD, TRADES, RobustOverﬁtting) produce robust models that are effective to defend PGD attacks and goodish to other type of attacks.
Three advanced acceleration methods (Free, YOPO/TRADES+YOPO, Fast-FGSM) and ours can be at least 4 ∼ 5 times faster than previous adversarial training methods.
Although
Fast-FGSM achieves the best speed improvement (using the apex library), our ATCD is the only
[Table]
method that even greatly boost the robust accuracy in a reasonable training speed.
Similar success also appear in different architectures (see in Appendix A.3.1).
We also perform the evaluation among different methods on both clean accuracy (i.e. accuracy on natural images) and robust accuracy (i.e. accuracy on adversarial examples) after every training epoch and show the remarkable reliability about our ATCD in Fig.
We further emphasize that although YOPO may be computationally cheaper when compared to conventional approaches and other methods, it is clear that the curve of YOPO vibrates greatly and frequently, which implies the training scheme of YOPO should be carefully designed to achieve stable results.
This is also reﬂected in Table 2 when compared with the error bars of YOPO (TRADES+YOPO) and ours.
[Figure]
different advanced fast adversarial training methods.
We use PGDas the attacker and report their clean accuracy and robust accuracy.
Solid lines represent the robust accuracy and dashed lines represent the clean accuracy.
6. MNIST
We also investigate our ATCD method on MNIST.
PGD-40 still has comparable clean accuracy and robust accuracy among all the methods, but its computational cost is signiﬁcantly higher than other training methods.
Our method still achieves a good trade-off between efﬁciency and robust accuracy.
[Table]
C ONCLUSION
In this paper, we reformulate the generation of adversarial examples as a MCMC process and present a new adversarial learning method called ATCD, which approaches equilibrium distribution of adversarial examples with only few iterations by building from small modiﬁcations of the standard
Contrastive Divergence.
Extensive results with comparisons on various datasets show that ATCD achieves a trade-off between efﬁciency and accuracy in adversarial training.
A A PPENDIX A. E XPERIMENT S ETUP
All the experiments are taken on a single NVIDIA GeForce GTX Xp GPU.
Results of Free training
Shafahi et al.
(2019), TRADES-10 Zhang et al.
(2019d), YOPO Zhang et al.
(2019b)
, Fast-FGSM
Wong et al.
and Robust-Overﬁtting Rice et al.
(2020) are reproduced on our local machines but completely follow their ofﬁcial codes 12345 .
Note that in our experiments, we do not use any label-smoothing or other tricks to boost the robustness accuracy since we would like to fairly compare
PGD, Free and YOPO with our ATCD method.
These extra tricks and adversarial interpolation schemes can be added to improve results for both approaches but not the focus of our paper.
ImageNet setup.
The ILSVRC 2012 classiﬁcation dataset contains ˜1.28M training images and 50k validation images labeled with 1,000 classes.
Following the common practice, we perform horizontal ﬂip, scale, and aspect ratio augmentation for training images and apply a single center crop of 224 × 224 pixels during evaluation.
We choose the standard Resnet50 as the target model.
For all methods, we use a batch size of 256, and SGD optimizer with momentum 0.9 and a weight decay of 1e-4.
The initial learning rate is 0.1 and the learning rate is decayed by 10 every 30 /TK epochs.
We also set step size (cid:15)
= 4 / 255 and magnitude of perturbation ε = 4 / 255 based on L ∞ norm.
We compare to the best performing conﬁguration of Free adversarial training which usesminibatch replays over 96 epochs of training.
That means we only run N/TK = 24 passes over the whole dataset for both Free training and ATCD.
CIFAR10 setup.
The CIFAR10 dataset is a widely used dataset consisting of 60,000 colour images of 10 categories.
Each category has 6,000 images.
We choose the standard Wide ResNet-34 and
Preact-ResNet18 following previous works Madry et al.
; Zhang et al.
(2019a).
We use a similar scheme that was used in YOPO Zhang et al.
(2019b): we train models for 40 epochs and the initial learning rate is set to 0.2, reduced by 10 times at epoch 30 and 36.
For PGD adversarial training, we set the total epoch number N = 105 and 10 steps of PGD with step size (cid:15) = 2 / 255 and magnitude of perturbation ε = 8 / 255 as a common practice.
The initial learning rate is set to 5e-2, reduced by 10 times at epoch 79, 90 and 100.
We use a batch size of 256, a weight decay of 5e-4 and a momentum of 0.9 for both algorithm.
For evaluating, we test our model’s robustness under CW
Carlini & Wagner (2017), MI-FGSM Dong et al.
(2018), AutoAttck (AA)cro (2020), RaySChen &
Gu (2020) and 20 steps of PGD with step size (cid:15) = 2 / 255 and magnitude of perturbation ε = 8 /based on L ∞ norm.
MNIST setup.
MNIST is a database for handwritten digit classiﬁcation.
It consists of 60,000 training images and 10,000 test images, which are all 28 × 28 greyscale images, representing the digits 0-9.
We choose a simple ConvNet with four convolutional layers followed by three fully connected layers, which is same as Zhang et al. (2019a).
For PGD adversarial training, we train the models forepochs.
The initial learning rate is set to 0.1, reduced by 10 times at epoch 45.
We use a batch size of 256, a weight decay of 5e-4 and a momentum of 0.9.
For evaluating, we perform a PGD-40 and CW attack against our model and set the size of perturbation as ε = 0 .
3 based on L ∞ norm as a common practice Madry et al.
; Zhang et al.
(2019a;c).
A. D ETAILS ABOUT T RAJECTORY S HIFT
Trajectory Shift (TS) is our proposed metric to measure the difference of running various lengths of trajectory.
To reﬂect the change in the optimization landscape of parameters of a network during the course of training, we essentially take different lengths in MCMC simulating and measure the gradient obtained by each backward.
Speciﬁcally, suppose we would like to investigate the TS of the i-th layer and the lengths of different trajectories are K and K (cid:48) , what we need to do is to generate adversarial inputs according to the K and K (cid:48)
-step trajectory of MCMC simulating.
When the trajectory simulation is ﬁnished, we perform gradient backward based on the potential energy function and adversarial examples at current state, and then gather all the gradients in the mini-batch.
Finally, all gradients will be added and averaged throughout the dataset when both trajectory simulating and parameters updating are completed.
So the trajectory shift is the difference of these two averaged results.
The whole process can be summarized as the following formualtion:
[Equation]
where B is the size of mini-batch.
Note that in order to eliminate some unnecessary effects, we use the same random seed and share initialization parameters for two sets of trajectory with different lengths, and the input samples of each mini-batch are exactly the same.
A. A BLATION S TUDIES A.3. D IFFERENT A RCHITECTURES
Besides the success on Wide ResNet34 Zagoruyko & Komodakis (2016), we also achieves similar acceleration against PGD-10 in Preact-ResNet18 He et al.
(2016a), as shown in Table 4.
ATCD can achieve more aggressive robust accuracy boost with only a slight drop in clean accuracy when compared with Free and YOPO.
The word “natural train” in the table means the models are trained by using only natural (clean) images as the tradition classiﬁcation task always does.
[Table]
A.3. I NFLUENCE OF U PDATING F REQUENCY AND T RAJECTORY L ENGTH
We study the effect of updating iteration T and the length of trajectory K on the accuracy rate for our ATCD.
We ﬁx T ∗ K = 8 and apply several conﬁgurations.
Results are shown in Table 5 and
Figure.
It can be observed that (1) The best accuracy rate of ATCD on both clean images and
PGD adversarial examples appears at T = 4 , K = 2 .
This may result from the balance it achieves between updating frequency and the length of trajectory when T ∗ K is limited.
(2) We would like to point out Free training is a special case of ATCD, except that ATCD has a new objective function for approaching equilibrium distribution and can freely control the updating frequency and the length of
MCMC trajectory.
We compare our ATCD under several conﬁgurations with Free-8 (at the last row, and that’s the reason why we ﬁx T ∗ K = 8 ).
Note that our ATCD method surpass Free-8 in all cases, which shows that it is necessary to seperate parameters updating and adversarial examples generating instead of treating the service as one like Free model and J cd is helpful for the robustness. A.3. I NFLUENCE OF M-H R ESAMPLING
The proposed method has a new adversarial example generation objective and the M-H resampling.
In order to ﬁnd out what actually causes the performance improvement, we ablate the behavior of the generation objective and the M-H resampling. This ablation is in Table 6. We investigate the above two components for training Wide ResNet34 on CIFAR10. When adding the CD generation objective in the procedure of training, the accuracy increases greatly — e.g., increasing from 47.8% to 53.11% under 20-iteration PGD attacks.
On the other hand, adding the M-H resampling strategy makes slight
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
improvement on the robust accuracy, and it has a considerable increase on the clean accuracy. These results suggest that both of them bring beneﬁt for the clean and the robust accuracy. As samples in our proposed method stay in (and return large numbers of samples from) high-density regions of the candidate distribution while only occasionally visiting low-density regions by the acceptance probability, which can can decipt the underlying distribution of adversarial examples. It appears essential to properly combine the CD loss with the M-H resampling in the training procedure
[Figure]
[Table] method is Free- A.3. D IFFERENT NUMBERS OF A TTACK I TERATIONS To analyze the inﬂuence of the number of attack iterations on different adversarially trained models, we range the number of iterations of PGD attacker from 10 to 200.
Figure 4 shows the robust accuracy of different adversarially trained models.
We ﬁx ε = 8 / 255 and ε = 2 / 255 , and change the iteration number ranging in { 10 , 20 , 50 , 100 , 200 } .
From the plots, we see that no matter how larger the attack iteration is, the checkpoint which achieves the high robust accuracy under one certain conﬁguration still has higher accuracy than other models in different iteration conﬁgurations.
A.3. D IFFERENT V ALUES OF E PSILON
Figure 5 shows the robust accuracy of models using various state-of-the-art adversarial training methods.
We performed thorough experiments over epsilon parameters ε = { 8 / 255 , 16 / 255 , 32 / 255 , 64 / 255 , 128 / 255 } , and found that the robust accuracy drop dramatically when ε ≥ 32 / 255 especially all of the adversarial training acceleration methods.
In spite of this, [Figure] our ATCD method still aces any other acceleration method, even outperforming standard PGD training. [Figure]
We use PGD with 100 iterations and well-tuned steps as attacker to perform thorough experiments over epsilon parameters ε = { 8 / 255 , 16 / 255 , 32 / 255 , 64 / 255 , 128 / 255 } .
A.3. S ENSITIVE A NALYSIS OF λ AND ρ
In the ATCD algorithm, the main parameters include the balance factors λ and ρ . Figure 6 plots the results of the parameter sensitivity analysis. We broadly investigate different ratios of λ/ρ ranging from 1/32 to 32.
It can be found that when the parameters are changed, the algorithm performs stably.
Models can achieve rather better accuracy when the ratio of λ/ρ approaches 1 and the best performance appears when λ/ρ = 4 .
It is worth mentioning that our proposed objective function will degenerate into the original generation objective widely used in adversarial examples generation stage when ρ (cid:28) λ .
Therefore, our ATCD will be practically equal to Free with resampling when λ/ρ = 32 , which outperforms Free itself and shows the beneﬁts brought from the Metropolis-Hastings correction.
[Figure]
[Figure]
A. C OMPLEXITY A NALYSIS
A typical adversarial training method contains three major steps: forward propagation, backpropagation, and adversarial examples generation.
Suppose that a deep neural network contains L layers (including input and output layer).
d l denotes the number of nodes of in each layer, where l ∈ { 1 , · · · , L } .
Assume the dataset has Z training examples and the number of training epochs is N .
Here we ignore the run-time of batch normalization for simpliﬁcation.
Forward propagation.
We see that for each layer a matrix multiplication, and an activation function is computed.
It is easy to know that naive matrix multiplication has a asymptotic run-time of
O (cid:0) d 3 (cid:1) , and the activation operator g is an elementwise function, we know that it has a run-time of
O ( d ) .
So the total forward run-time therefore becomes O forward =
NZ ∗ ( O mul + O activation ) =
NZ
[Equation]
=
NZL ( d 3 + d ) .
Backpropagation.
The run-time complexity of backpropagation can be computed in a similar manner.
Suppose z ( l ) i is the raw output signal of the i -th neuron in layer l before the activation function has been applied.
We introduce a new variable δ ( l ) i which is the error-sum of neuron i in layer l .
So the total backpropagation run-time can be calculated as O backward = NZ ∗ ( O middle layers +
[Equation]
Adversarial Training.
Recall that adversarial training is used adversarial examples to train DNN models to improve the robustness.
So the major difference between adversarial training and normal training is that adversarial training contains the process of adversarial examples generation.
Here we show several methods of adversarial examples generation to compare their time complexity.
• FGSM :
Since FGSM ﬁnds the adversarial perturbation in the direction of the loss gradient only once, FGSM includes two full forward propagation, one backpropagation without weight updating, and one backpropagation with weight updating.
So the total run-time of FGSM is: O FGSM =
2 NZL [( d 3 + d ) + L − 1 L d 3 + 1 L d 2 + 1 2 d ] .
• PGD : PGD can be considered as an iterative version of FGSM repeating r times updating procedure of adversarial examples generation with several random restarts.
That
means PGD includes one full forward propagation, one backpropagation with weight updating, r times forward propagation and backpropagation without weight updating.
Thus, the total run-time of PGD can be computed by: O PGD =
NZL ( r + 1)[( d 3 + d ) + L − 1 L d 3 + 1 L d 2 + 1 r +1 d ] .
• YOPO :
The major time cost of PGD results from conducting T sweeps of forward and backward propagation.
YOPO restricts most of the forward and back propagation within the ﬁrst layer of DNN during adversary updates, which effectively reduces the total number of full forward and backward propagation.
The total number of full forward propagation is only m apart from n times forward propagation at the ﬁrst layer.
The backward process is divided into m times backpropagation except the ﬁrst layer and n times backpropagation at the ﬁrst layer.
Therefore, the total run-time of YOPO can be formulated as: O YOPO =
NZ ( m + 1)[ L ( d 3 + d ) +
( L − 1)
( L − 1 L d 3 + 1 L d 2 + d ) + n ( d 3 + d + d 2 )] .
• ATCD :
Our ATCD method relies on the replay of current samples and the searching trajectory of HMC.
So the total run-time of ATCD can be easily expressed by: O ATCD = N (cid:48)
ZLK ( T + 1)[( d 3 + d ) + L − 1 L d 3 + 1 L d 2 + 1 K ( T +1) d ] .
Because of N (cid:48)
= NTK , our ATCD is r +1 K
( T +1) times faster than PGD but K ( T +1) slower than FGSM.
A. L ONG OR S HORT - RUN T RAJECTORY ?
Both long and short-run trajectory of HMC simulating have been able to ﬁt with contrastive adversarial training.
In our experiments we use K = 2 to achieve the trade-off between performance and efﬁciency.
But we have found that if condition allows (without computation or time being limited) and just pursue the best performance, long-run HMC with Metropolis-Hastings correction are preferable in terms of equilibrium distribution.
As shown in Figure we ﬁx the number of parameters updating as T = 4 and sweep over the length of trajectory K from 1 to 8 (almost in line with PGD adversarial training).
In all conﬁgurations,
ATCD has much higher robust accuracy than Free.
This shows evidence that ATCD is a more effective adversarial training method to improve the robustness of models compared to Free.
We also ﬁnd that by using contrastive divergence, we could reduce the lengths of MCMC trajectory per iteration when compared to PGD-10 and still allow for relatively high robustness.
This gives a 3x speedup over our fastest short-run ATCD ( K = 1 ).
Further, robust accuracies against different adversarial attacks share the similar trend and ATCD using longest trajectory has the best result.
So if time allows, we recommend using a long-run trajectory of HMC with Metropolis-Hastings correction to better approach equilibrium distribution.
But even so, the clean accuracy of adversarially trained model still decreases with the growth of the length of simulating trajectory.
Besides, some generated adversarial examples will be recalled by Metropolis-Hastings correction when the length of trajectory becomes longer, which would increase the additional time cost.
Overall, the short-run trajectory is slightly less robust than the long-run trajectory, but in total, the short-run trajectory increases in speed and maintains the suitable clean accuracy to be a worthwhile trade-off.
Another interesting observation we found while using long-run trajectory of HMC is that the model would use the length of the trajectory to perform like “clustering”.
We discovered that when generating adversarial examples (the inner loop in Algorithm 2) on CIFAR10, these samples are always identiﬁed as one speciﬁc category (e.g. trunk) at very beginning of trajectory.
When the length of trajectory grows, they could almost always be identiﬁed as another category (e.g. deer).
However, if we use short-run trajectory, this phenomenon will not happen.
This behavior is likely some compromise of ATCD since contrastive adversarial training with short-run trajectory accelerates the process of equilibrium distribution by discarding the solution space exploring and Metropolis-Hastings correction.
A. W HAT IS THE A DVANTAGE OF HMC F RAMEWORK FOR A DVERSARIAL L EARNING ?
In this section we describe the advantage of reformulating adversarial examples generation from the perspective of HMC.
In addition to transforming the process of adversarial training into the trajectory of MCMC simulating, thereby introducing more well studied solutions of high computational cost of
MCMC, HMC also brings the following advantages:
First, we reinterpret the adversarial examples generation as the solution space searching through the conservation transformation of two kinds of energy.
If we further consider a standard discriminative classiﬁer of p ( y | x ) as an energy based model for the joint distribution p ( x, y ) , we can apply our framework in both discriminative and generative modeling.
That means we enable this framework [Figure] not only to improve the robustness of modern classiﬁer, but also to generate samples as generative models.
Second, when simulating adversarial examples generation in HMC framework, different types of adversarial attacks can be considered as special cases of HMC.
Actually, there will be a more powerful searching algorithm if a new potential or kinetic (or both) energy is designed Bhagoji et al.
Carlini & Wagner (2017); Zheng et al.
Rony et al.
(2019), e.g. mapping the clipped gradient descent into tanh space or adding KL-divergence term.
Third, the unique advantage brought by HMC itself is that it can better explore the distribution of adversarial examples, which is different from heuristic algorithm (e.g. PGD with random restarts).
That means we can design a more powerful adversarial attack from the view of HMC.
To further verify this point, we apply a generating method for adversarial examples based on HMC to attack the real-world celebrity recognition APIs in Clarifai 6 .
These celebrity recognition APIs allow users to upload any face images and recognize the identity of them with conﬁdence score.
The users have no knowledge about the dataset and types of models used behind these online systems.
We choosepairs of images from the LFW dataset and learn perturbations from local facenet model to launch targeted attack, whose goal is to mislead the API to recognize the adversarial images as our selected identity.
We randomly pick up 10 celebrities as victims from Google and 10 existing celebrities as targets from LFW, ensuring that all colors and genders are taken into account.
Then we apply the same strategy as Geekpwn CAAD 2018 method that pulls victims towards their corresponding targets by the inner product of their feature vectors and generates noise to them.
Finally, we examine their categories and conﬁdence scores by uploading these adversarial examples to the online systems
API.
We ﬁx ε = 16 and total iteration number N = 100 .
It is worth mentioning that we generate a sequence of adversarial examples to show how well HMC-based method explores the space of adversarial examples.
Results are shown in Fig.
8, 9 and 10.
Figure 8 presents three celebrity pairs where both Geekpwn CAAD 2018 method and our HMC-based method successfully fool the face recognition system to recognize them as the target celebrities.
But the conﬁdence score of each pair generated by Geekpwn CAAD 2018 method is lower than any one in our generated sequence.
Figure 9 presents three pairs where the generated adversarial examples by our HMC-based method all successfully fool the system as the target celebrities while
Geekpwn CAAD 2018 method fails.
Figure 10 shows a case that any one in the generated sequence by HMC-based method can fool the system with very high conﬁdence while the adversarial examples generated by Geekpwn CAAD 2018 method being “caught” – which is recognized by the online systems and inferred to the true category.
We also list two failure cases in Figure 11.
Note that the former one is slightly harder because the source and target are different races, but HMC-based method still generates few success samples.
When looking at the big picture, these failure might be caused by our HMC-based method (or all the white-box adversarial attack methods) which are dependent of the targeted classiﬁer.
The decision boundary of different models may be slightly different from the global perspective, but the targeted attack forces the attacker to focus on the local decision boundary, which greatly magniﬁes this difference.
It is worth noting that the phenomenon when using a long-run trajectory with
Metropolis-Hastings correction which we mentioned in Appendix A.5 occurs again.
[Figure]
Original
Target
Pairs
Our generated sequence
Score: 62.
Failed
Score: 66.
Score: 66.7 Score: 56.
Geekpwn CAAD
Score: 69.
Original
Target
Pairs
Our generated sequence
Score: 17.
Failed
Score: 18.
Score: 18.6 Score: 23.
Geekpwn CAAD
Score: 22.
Original
Target
Pairs
Our generated sequence
Score: 12.
Score: 11.
Score: 6.4 Score: 17.
Geekpwn CAAD
Score: 5.
Failed
[Figure]
Our
[Figure]
Score: [Figure] Score:
[Figure]
Score: 99.
[Figure]
Score: [Figure]
Geekpwn [Figure]
Score: 99.
Catched , 34.
Original
Target
Pairs
Our generated sequence
Score: 55.
Score: 7.9 Score: 18.7 Score: 6.
Geekpwn CAAD
Score: 45.
Catched , 6.
Peter OToole
Ben Carson (Don Cheadle)
Original
Target
Pairs
Our generated sequence
Score: 22.
Score: 15.
Score: 16.3 Score: 4.
Geekpwn CAAD
Score: 4.
Catched , 13.
Michael Emerson Rob Schneider
[Figure]