S TOCHASTIC S ECURITY : A DVERSARIAL D EFENSE U SING L ONG -R UN D YNAMICS OF E NERGY -B ASED M ODELS
A BSTRACT
The vulnerability of deep networks to adversarial attacks is a central problem for deep learning from the perspective of both cognition and security.
The current most successful defense method is to train a classiﬁer using adversarial images created during learning.
Another defense approach involves transformation or puriﬁcation of the original input to remove adversarial signals before the image is classiﬁed.
We focus on defending naturally-trained classiﬁers using Markov Chain Monte
Carlo (MCMC) sampling with an Energy-Based Model (EBM) for adversarial puriﬁcation.
In contrast to adversarial training, our approach is intended to secure highly vulnerable pre-existing classiﬁers.
To our knowledge, no prior defensive transformation is capable of securing naturally-trained classiﬁers, and our method is the ﬁrst to validate a post-training defense approach that is distinct from current successful defenses which modify classiﬁer training.
The memoryless behavior of long-run MCMC sampling will eventually remove adversarial signals, while metastable behavior preserves consistent appearance of MCMC samples after many steps to allow accurate long-run prediction.
Balancing these factors can lead to effective puriﬁcation and robust classiﬁcation.
We evaluate adversarial defense with an EBM using the strongest known attacks against puriﬁcation.
Our contributions are 1) an improved method for training EBM’s with realistic long-run MCMC samples for effective puriﬁcation, 2) an Expectation-Over-Transformation (EOT) defense that resolves ambiguities for evaluating stochastic defenses and from which the
EOT attack naturally follows, and 3) state-of-the-art adversarial defense for naturally-trained classiﬁers and competitive defense compared to adversarial training on CIFAR10, SVHN, and CIFAR-100. Our code and pre-trained models are available at https://github.com/point0bar1/ebm-defense .
M OTIVATION AND C ONTRIBUTIONS
Deep neural networks are highly sensitive to small input perturbations.
This sensitivity can be exploited to create adversarial examples that undermine robustness by causing trained networks to produce defective results from input changes that are imperceptible to the human eye (Goodfellow et al., 2015).
The adversarial scenarios studied in this paper are primarily untargeted white-box attacks on image classiﬁcation networks.
White-box attacks have full access to the classiﬁer (in particular, to classiﬁer gradients) and are the strongest attacks against the majority of defenses.
Many whitebox methods have been introduced to create adversarial examples.
Strong iterative attacks such as Projected Gradient Descent (PGD) (Madry et al., 2018) can reduce the accuracy of a naturally-trained classiﬁer to virtually 0.
Currently the most robust form of adversarial defense is to train a classiﬁer on adversarial samples in a procedure known as adversarial training (AT) (Madry et al., 2018).
Another defense strategy, which we will refer to as adversarial preprocessing (AP), uses defensive transformations to purify an image and remove or nullify adversarial signals before classiﬁcation (Song et al.
; Guo et al.
; Yang et al.
(2019), and others).
AP is an attractive strategy compared to AT because it has the potential to secure vulnerable pre-existing classiﬁers.
Defending naturally-trained classiﬁers is the central focus of this work.
Athalye et al.
(2018) revealed that many preprocessing defenses can be overcome with minor adjustments to the standard PGD attack.
Both stochastic behavior from preprocessing and the computational difﬁculty of end-to-end backpropagation can be circumvented to attack the classiﬁer through the defensive transformation.
In this paper we carefully address Athalye et al.
(2018) to evaluate AP with an EBM using attacks with the greatest known effectiveness and efﬁciency.
a method for AP (Du & Mordatch, 2019; Grathwohl et al., 2020).
However, the proposed defenses are not competitive with AT (see Table 1 and Croce & Hein (2020)).
In the present work we demonstrate that EBM defense of a naturally-trained classiﬁer can be stronger than standard AT (Madry et al., 2018) and competitive with state-of-the-art AT (Zhang et al., 2019; Carmon et al., 2019).
Our defense tools are a classiﬁer trained with labeled natural images and an EBM trained with unlabeled natural images.
For prediction, we perform Langevin sampling with the EBM and send the sampled images to the naturally-trained classiﬁer.
An intuitive visualization of our defense method is shown in Figure 1.
Langevin chains constitute a memoryless trajectory that removes adversarial signals, while metastable sampling behaviors preserve image classes over long-run trajectories.
Balancing these two factors leads to effective adversarial defense.
Our main contributions are:
• A simple but effective adjustment to improve the convergent learning procedure from Nijkamp et al.
Our adjustment enables realistic long-run sampling with EBMs learned from complex datasets such as CIFAR-10.
• An Expectation-Over-Transformation (EOT) defense that prevents the possibility of a stochastic defense breaking due to random variation in prediction instead of an adversarial signal.
The EOT attack (Athalye et al., 2018) naturally follows from the EOT defense.
• Experiments showing state-of-the-art defense for naturally-trained classiﬁers and competitive defense compared to state-of-the-art AT.
I MPROVED C ONVERGENT L EARNING OF E NERGY -B ASED M ODELS
The Energy-Based Model introduced in (Xie et al., 2016) is a Gibbs-Boltzmann density
[Equation]
where x ∈ R D is an image signal, U ( x ; θ ) is a ConvNet with weights θ and scalar output, and
Z ( θ ) =
(cid:82) X exp {− U ( x ; θ ) } dx is the intractable normalizing constant.
Given i.i.d.
samples from a data distribution q ( x ) , one can learn a parameter θ ∗ such that p ( x ; θ ∗ )
≈ q ( x ) by minimizing the expected negative log-likelihood L ( θ ) =
E q [ − log p ( X ; θ )] of the data samples.
Network weights θ are updated using the loss gradient
[Equation]
where { X + i } ni =1 are a batch of training images and { X − i } mi =1 are i.i.d.
samples from p ( x ; θ ) obtained via MCMC.
Iterative application of the Langevin update
[Equation]
where Z k ∼ N (0 , I D ) and τ > 0 is the step size parameter, is used to obtain the samples { X − i } mi =1 .
[Figure]
distribution.
Nijkamp et al.
(2020) reveal that EBM learning heavily gravitates towards an unexpected outcome where short-run MCMC samples have a realistic appearance and long-run MCMC samples have an unrealistic appearance.
The work uses the term convergent learning to refer to the expected outcome where short-run and long-run MCMC samples have similar appearance, and the term non-convergent learning to refer to the unexpected but prevalent outcome where models have realistic short-run samples and oversaturated long-run samples.
Convergent learning is essential for our defense strategy because long-run samples must be realistic so that classiﬁers can maintain high accuracy after Langevin transformation (see Section 4.1 and Appendix D).
As observed by Nijkamp et al.
(2020), we were unable to learn a convergent model when updating θ with the Adam optimizer (Kingma & Ba, 2015).
Despite the drawbacks of Adam for convergent learning, it is a very effective tool for obtaining realistic short-run synthesis.
Drawing inspiration from classiﬁer training from Keskar & Socher (2017), we learn a convergent EBM in two phases.
The ﬁrst phase uses Adam to update θ to achieve realistic short-run synthesis.
The second phase uses SGD to update θ to align short-run and long-run MCMC samples to correct the degenerate steady-state from the Adam phase.
This modiﬁcation allows us to learn the convergent EBMs for complex datasets such as CIFAR-10 using 20% of the computational budget of Nijkamp et al.
See Figure 2.
We use the lightweight EBM from Nijkamp et al.
as our network architecture.
We use long run chains for our EBM defense to remove adversarial signals while maintaining image features needed for accurate classiﬁcation.
The steady-state convergence property ensures adversarial signals will eventually vanish, while metastable behaviors preserve features of the initial state.
Theoretical perspectives on our defense can be found in Appendix C and a comparison of convergent and non-convergent EBM defenses can be found in Section 4.1 and
Appendix D.
A TTACK AND D EFENSE F ORMULATION 3. C LASSIFICATION WITH S TOCHASTIC T RANSFORMATIONS
Let T ( x ) be a stochastic pre-processing transformation for a state x ∈ R D .
Given a ﬁxed input x , the transformed state T ( x ) is a random variable over R D .
In this work, T ( x ) is obtained from K steps of the Langevin update (3) starting from X
= x .
One can compose T ( x ) with a deterministic classiﬁer f ( x ) ∈ R J (for us, a naturally-trained classiﬁer) to deﬁne a new classiﬁer F ( x ) ∈ R J as
F [Equation]
T [Equation] [ f [Equation]
T [Equation] F ( x ) is a deterministic classiﬁer even though T ( x ) is stochastic.
The predicted label for x is then c ( x ) =
arg max j F ( x ) j .
In this work, f ( x ) denotes logits and F ( x ) denotes expected logits, although other choices are possible (e.g. softmax outputs).
We refer to (4) as an Expectation-OverTransformation (EOT) defense.
The classiﬁer F ( x ) in (4) is simply the target of the EOT attack (Athalye et al., 2018).
The importance of the EOT formulation is well-established for adversarial attacks, but its importance for adversarial defense has not yet been established.
Although direct evaluation of F ( x ) is generally impossible, the law of large numbers ensures that the ﬁnite-sample approximation of F ( x ) given by
[Equation]
where,
[Equation]
can approximate F ( x ) to any degree of accuracy for a sufﬁciently large sample size H .
In other words, F ( x ) is intractable but trivial to accurately approximate via ˆ F H ( x ) given enough computation.
[Figure]
In the literature both attackers (Athalye et al., 2018; Tramer et al., 2020) and defenders (Dhillon et al., 2018; Yang et al., 2019; Grathwohl et al., 2020) evaluate stochastic classiﬁers of the form f ( T ( x )) using either ˆ F 1 ( x ) or ˆ F H adv ( x ) where H adv is the number of EOT attack samples, typically aroundto 30.
This evaluation is not sound when ˆ F H ( x ) has a small but plausible chance of misclassiﬁcation because randomness alone could cause x to be identiﬁed as an adversarial image even though ˆ F H ( x ) gives the correct prediction on average (see Figure 3).
In experiments with EBM defense, we identify many images x that exhibit variation in the predicted label
[Equation]
for smaller H ≈ 10 but which have consistently correct prediction for larger H ≈ 150 .
Fair evaluation of stochastic defenses must be based on the deterministic EOT defense F in (4) and attackers must use sufﬁciently large H to ensure that ˆ F H ( x ) accurately approximates F ( x ) before declaring that an attack using adversarial image x is successful.
In practice, we observe that ˆ F 150 is sufﬁciently stable to evaluate F over several hundred attacks with Algorithm 1 for our EBM defense.
3. A TTACKING S TOCHASTIC C LASSIFIERS
Let L ( F ( x ) , y ) ∈
R be the loss (e.g. cross-entropy) between a label y ∈ { 1 , . . .
, J } the outputs
F ( x ) ∈ R J of a classiﬁer (e.g. the logits) for an image x ∈ R D .
For a given pair of observed data ( x + , y ) , an untargeted white-box adversarial attack searches for the state
[Equation]
that maximizes loss for predicting y in a set S ⊂ R D centered around x + .
In this work, a natural image x
+ will have pixels intensities from 0 to 1 (i.e. x + ∈ [0 , 1] D ).
One choice of S is the intersection of the image hypercube [0 , 1] D and the l ∞ -norm ε -ball around x + for suitably small ε > 0 .
Another option is the intersection of the hypercube [0 , 1] D with the l 2 -norm ε -ball.
The Projected Gradient Descent (PGD) attack (Madry et al., 2018) is the standard benchmark when
S is the ε -ball in the l p norm.
PGD begins at a random x 0 ∈ S and maximizes (7) by iteratively updating x i with
[Equation]
where Π S denotes projection onto S , ∆( x, y ) is the attack gradient, and α > 0 is the attack step size.
Standard PGD uses the gradient ∆ PGD ( x, y )
= ∇ x L ( F ( x ) , y ) .
The EOT attack (Athalye et al., 2018) circumvents the intractability of F by attacking ﬁnite sample logits
ˆ
F H adv , where H adv is the number of EOT attack samples, with the gradient
[Equation]
The EOT attack is the natural adaptive attack for our EOT defense formulation.
Another challenge when attacking a preprocessing defense is the computational infeasibility or theoretical impossibility of differentiating T ( x ) .
The Backward Pass Differentiable Approximation (BPDA) technique (Athalye et al., 2018) uses an easily differentiable function g ( x ) such that g ( x ) ≈ T ( x ) to attack
F ( x ) =
f ( T ( x )) .
One calculates the attack loss using L ( f ( T ( x )) , y ) on the forward pass but calculates the attack gradient using ∇ x L ( f ( g ( x )) , y ) on the backward pass.
A simple but effective form of BPDA is the identity approximation g ( x ) = x .
This approximation is reasonable for preprocessing defenses that seek to remove adversarial signals while preserving the main features of the original image.
When g ( x ) = x , the BPDA attack gradient is ∆ BPDA ( x, y ) = ∇ z L ( f ( z ) , y ) where z = T ( x ) .
Intuitively, this attack obtains an attack gradient with respect to the puriﬁed image and applies it to the original image.
Combining the EOT attack and BPDA attack with identity g ( x ) = x gives the attack gradient
[Equation]
The BPDA+EOT attack represents the strongest known attack against preprocessing defenses, as shown by its effective use in recent works (Tramer et al., 2020).
We use ∆ BPDA+EOT ( x, y ) in (8) as our primary attack to evaluate the EOT defense (4).
Pseudo-code for our adaptive attack can be found in Algorithm 1 in Appendix A.
E XPERIMENTS
We use two different network architectures in our experiments.
The ﬁrst network is the lightweight
EBM from Nijkamp et al.
The second network is a 28 × 10 Wide ResNet classiﬁer (Zagoruyko
Komodakis, 2016).
The EBM and classiﬁer are trained independently on the same dataset.
No adversarial training or other training modiﬁcations are used for either model.
The EBM defense in our work is given by Equation (4) and approximated by Equation (5), where T ( x ) is K steps of the Langevin iterations in Equation (3).We use the parameters from Algorithm 1 for all evaluations unless otherwise noted.
In Section 4.1, we examine the effect of the number of Langevin steps K and the stability of Langevin sampling paths on defense.
Section 4.2 examines our defense against a PGD attack from the base classiﬁer, and Section 4.3 examines our defense against the adaptive BPDA+EOT attack.
4. C OMPARISON OF C ONVERGENT AND N ON -C ONVERGENT D EFENSES
[Figure]
non-convergent EBM defense from achieving high natural or robust accuracy.
We examine the effect the number of Langevin steps has on defense accuracy for the CIFARdataset (see Figure 4).
Each point displays either the baseline accuracy of our stochastic classiﬁer (blue) or the results of a BPDA+EOT attack (orange) on 1000 test images.
The attacks used to make this diagram use a reduced load of H adv
= 7 replicates for EOT attacks so these defense accuracies are slightly higher than the full attack results presented in Figure 5.
Short-run Langevin with K ≤ 100 steps yields almost no adversarial robustness.
Increasing the number of steps gradually increases robustness until the defense saturates at around K = 2000 .
We chose K = 1500 steps in our experiments as a good tradeoff between robustness, natural accuracy, and computational cost.
For comparison, we run the same experiment using a non-convergent EBM.
The network structure and training are identical to our convergent model, except that we use Adam instead of SGD throughout training.
The non-convergent EBM defense cannot achieve high natural accuracy with long-run sampling because of the oversaturated features that emerge.
Without a high natural accuracy, it is impossible to obtain good defense results.
Thus convergent EBMs that can produce realistic long-run samples are a key ingredient for the success of our method.
In the context of EBM defense, short-run sampling refers to trajectories of up to about 100 steps where no defensive properties emerge, and long-run sampling refers to trajectories of about 1000 or more steps where defense becomes possible.
4. PGD A TTACK FROM B ASE C LASSIFIER FOR CIFAR-10 D ATASET
We ﬁrst evaluate our defense using adversarial images created from a PGD attack on the classiﬁer f ( x ) .
Since this attack does not incorporate the Langevin sampling from T ( x ) , the adversarial images in this section are relatively easy to secure with Langevin transformations.
This attack serves as a benchmark for comparing our defense to the IGEBM (Du & Mordatch, 2019) and JEM (Grathwohl et al., 2020) models that also evaluate adversarial defense with a ConvNet EBM (1).
For all methods, we evaluate the base classiﬁer and the EBM defense for K = 10 Langevin steps (as in prior defenses) and K = 1500 steps (as in our defense).
The results are displayed in Table 1.
[Table]
[Table]
[Table]
Our natural classiﬁer f ( x ) has a high base accuracy but no robustness.
The JEM base classiﬁer has high natural accuracy and minor robustness, while the IGEBM base classiﬁer has signiﬁcant robustness but very low natural accuracy.
Short-run sampling with K = 10 Langevin steps does not signiﬁcantly increase robustness for any model.
Long-run sampling with K = 1500 steps provides a dramatic increase in defense for our method but only minor increases for the prior methods.
Further discussion of the IGEBM and JEM defenses can be found in Appendix G.
4. BPDA+EOT A TTACK
In this section, we evaluate our EBM defense using the adaptive BPDA+EOT attack (10) designed speciﬁcally for our defense approach.
This attack is recently used by Tramer et al.
(2020) to evaluate the preprocessing defense from Yang et al.
(2019) that is similar to our method.
[Table]
CIFAR-10 .
We ran 5 random restarts of the BPDA+EOT attack in Algorithm 1 with the the listed parameters on the entire CIFAR-10 test set.
In particular, the attacks use adversarial perturbation ε = 8 / 255 and attack step size α = 2 / 255 in the l ∞ norm.
One evaluation of the entire test set took approximately 2.5 days using 4x RTX 2070 Super GPUs.
We compare our results to a representative selection of AT and AP defenses in Table 2.
We include the training method for the classiﬁer, preprocessing transformation (if any), and the strongest attack for each defense.
Our EBM defense is stronger than standard AT (Madry et al., 2018) and comparable to modiﬁed AT from Zhang et al.
Although our results are not on par with state-of-the-art AT (Carmon et al., 2019), our defense is the ﬁrst method that can effectively secure naturally-trained classiﬁers.
We now examine the effect of the perturbation ε , number of attacks N , and number of EOT attack replicates H adv on the strength of the BPDA+EOT attack.
To reduce the computational cost of the diagnostics, we use a ﬁxed set of 1000 randomly selected test images for diagnostic attacks.
Figure 5 displays the robustness of our model compared against two AT models, standard AT (Madry et al., 2018) and Semi-supervised AT (Carmon et al., 2019) for l ∞ and l 2 attacks across perturbation size ε .
Our model is attacked with BPDA+EOT while AT models are attacked with PGD.
Our defense is more robust than standard AT for a range of medium-size perturbations in the l ∞ norm and much [Figure] [Figure] more robust than standard AT for medium and large ε in the l 2 norm.
Our model is less robust than
Semi-supervised AT for the l ∞ norm but more robust for the l 2 norm.
AT-based models are trained to defend against a speciﬁc attack while our method does not use any attacks during training.
This is likely a reason why our defense outperforms l ∞ -trained AT against l 2 attacks
[Figure]
Figure 6 visualizes the effect of increasing the computational power of the attacker and defender in
Algorithm 1.
The left ﬁgure compares our defense with AT over 1000 attacks using an increased number H adv = 20 of attack replicates.
The majority of breaks happen within the ﬁrst 50 attacks as used in the CIFAR-10 experiment in Table 2, while a small number of breaks occur within a few hundred attack steps.
It is likely that some breaks from long-run attacks are the result of lingering stochastic behavior from ˆ F H def ( x ) rather than the attack itself.
The right ﬁgure shows the effect of the number of EOT replicates.
The strength of the EOT attack saturates when using 20 to 30 replicates.
A small gap in attack strength remains between the 15 replicates used in our attacks and the strongest possible attack.
Some of this effect is likely mitigated by our use of 5 random restarts.
Defense with
H def = 150 is close to the maximum achievable defense for our method, although more replicates would slightly strengthen our results.
The evaluation in Table 2 pushes the limits of what is computationally feasible with widely available resources.
The diagnostics indicate that our defense report is an accurate approximation of the defense of our ideal classiﬁer F ( x ) in (4) against the BPDA+EOT attack (10), although more computation would yield a slightly more accurate estimate.
SVHN and CIFAR-100 .
The attack and defense parameters for our method are identical to those used in the CIFAR-10 experiments.
We compare our results with standard AT.
Overall, our defense performs well for datasets that are both simpler and more complex than CIFAR-10.
In future work, further stabilization of image appearance after Langevin sampling could yield signiﬁcant beneﬁts for settings where precise details need to be preserved for accurate classiﬁcation.
The AT results for CIFAR-100 are from Balaji et al.
(2019) and the results for SVHN are from our implementation.
[Table]
R ELATED W ORK
Adversarial training learns a robust classiﬁer using adversarial images created during each weight update.
The method is introduced by Madry et al.
Many variations of AT have been explored, some of which are related to our defense.
He et al.
(2019) apply noise injection to each network layer to increase robustness via stochastic effects.
Similarly, Langevin updates with our EBM can be interpreted as a ResNet (He et al., 2016) with noise injected layers as discussed by Nijkamp et al.
Semi-supervised AT methods (Alayrac et al., 2019; Carmon et al., 2019) use unlabeled images to improve robustness.
Our EBM also leverages unlabeled data for defense.
Adversarial preprocessing is a strategy where auxiliary transformations are applied to adversarial inputs before they are given to the classiﬁer.
Prior forms of pre-processing defenses include rescaling (Xie et al., 2018a), thermometer encoding (Buckman et al., 2018), feature squeezing (Xu et al., 2018), activation pruning (Dhillon et al., 2018), reconstruction (Samangouei et al., 2018), ensemble transformations (Raff et al., 2019), addition of Gaussian noise (Cohen et al., 2019), and reconstruction of masked images (Yang et al., 2019).
Prior preprocessing methods also include energy-based models such as Pixel-Defend (Song et al., 2018) and MALADE (Srinivasan et al., 2019) that draw samples from a density that differs from (1).
It was shown by Athalye et al.
; Tramer et al.
(2020) that many preprocessing defenses can be totally broken or dramatically weakened by simple adjustments to the standard PGD attack, namely the EOT and BPDA techniques.
Furthermore, many preprocessing defenses such as Cohen et al.
; Raff et al.
; Yang et al.
(2019) also modify classiﬁer training so that the resulting defenses are analogous to AT, as discussed in Appendix F. A concurrent work called Denoised Smoothing (Salman et al., 2020) prepends a Denoising Autoencoder to a natural image classiﬁer.
The joint classiﬁer ﬁxes the natural classiﬁer weights and the denoiser is trained by minimizing the cross-entropy loss of the joint model using noisy training inputs. Our defensive transformation provides security without information from any classiﬁer.
No prior preprocessing defense is competitive with AT when applied to independent naturally-trained classiﬁers.
Energy-based models are a probabilistic method for unsupervised modeling.
Early energy-based image models include the FRAME (Zhu et al., 1998) and RBM (Hinton, 2002).
The EBM is a descriptive model (Guo et al., 2003; Zhu, 2003).
Our EBM (1) is introduced as the DeepFRAME model by Xie et al.
and important observations about the learning process are presented by Nijkamp et al.
Other studies of EBM learning include Gao et al.
; Xie et al.
(2018b) ; Gao et al. Preliminary investigations for using the EBM (1) for adversarial defense are presented by Du & Mordatch (2019); Grathwohl et al.
but the results are not competitive with AT (see Section 4.2).
Our experiments show that the instability of sampling paths for these non-convergent models prevents Langevin trajectories from manifesting defensive properties.
We build on the convergent learning methodology from Nijkamp et al.
(2020) to apply long-run Langevin sampling as a defense technique.
C ONCLUSION
This work demonstrates that Langevin sampling with a convergent EBM is an effective defense for naturally-trained classiﬁers.
Our defense is founded on an improvement to EBM training that enables efﬁcient learning of stable long-run sampling for complex datasets.
We evaluate our defense using non-adaptive and adaptive whitebox attacks for the CIFAR-10, CIFAR-100, and SVHN datasets.
Our defense is competitive with adversarial training.
Securing naturally-trained classiﬁers with post-training defense is a long-standing open problem that this work resolves.
B I MPROVED L EARNING OF C ONVERGENT EBM S
Algorithm 2 provides pseudo-code for our improvement of the convergent learning method from
Nijkamp et al.
This implementation allows efﬁcient learning of convergent EBMs for complex datasets.
C E RASING A DVERSARIAL S IGNALS WITH MCMC S AMPLING
This section discusses two theoretical perspectives that justify the use of an EBM for purifying adversarial signals: memoryless and chaotic behaviors from sampling dynamics.
We emphasize that the discussion applies primarily to long-run behavior of a Langevin image trajectory.
Memoryless and chaotic properties do not appear to emerge from short-run sampling.
Throughout our experiments, we never observe signiﬁcant defense beneﬁts from short-run Langevin sampling.
Require: ConvNet potential U ( x ; θ ) , number of training steps J = 150000 , step to switch from
SGD to Adam J SGD = 50000 , initial weight θ 1 , training images { x + i } N data
i =1 , data perturbation τ data = 0 .
02 , step size τ = 0 .
01 , Langevin steps K = 100 , Adam learning rate γ Adam = 0 . 0001 ,
SGD learning rate γ SGD = 0 .
Ensure:
Weights θ J +1 for energy U ( x ; θ ) .
Set optimizer g ← Adam ( γ Adam ) .
Initialize persistent image bank as N data uniform noise images.
for j =1:( J +1) do if j = J SGD then
Set optimizer g ← SGD ( γ SGD ) .
end if
Draw batch images { x +( i ) } mi =1 from training set, where ( i ) indicates a randomly selected index for sample i , and get samples X + i
= x ( i ) + τ data Z i , where Z i ∼ N (0 , I D ) i.i.d.
Draw initial negative samples { Y (0)
i
1 from persistent image bank.
Update
) i }
1 with the Langevin equation
[Equation]
where Z i,k ∼ N (0 , I D ) i.i.d., for K steps to obtain samples { X − i } mi =1 = { Y ( K ) i } mi =1 .
Update persistent image bank with images { Y ( K ) i }
Update the weights by θ j +
=
θ j − g (∆ θ j ) , where g is the optimizer and ∆ θ [Equation] =
[Equation]
n [Equation] i = U ( X
[Equation]
θ j ) − 1 m m (cid:88) i =
U ( X − i ; θ j ) (cid:33) is the ML gradient approximation.
end for
[Figure]
Energy of Natural, Adversarial, and Noise Images
[Figure]
C. M EMORYLESS D YNAMICS
The ﬁrst justiﬁcation of EBM defense is that iterative probabilistic updates will move an image from a low-probability adversarial region to a high-probability natural region, as discussed in prior works (Song et al., 2018;
Srinivasan et al., 2019; Grathwohl et al., 2020).
Comparing the energy of adversarial and natural images shows that adversarial images tend to have marginally higher energy, which is evidence that adversarial images are improbable deviations from the steady-state manifold of the EBM that models the distribution of natural images (see Figure 7).
The theoretical foundation for removing adversarial signals with MCMC sampling comes from the well-known steady-state convergence property of Markov chains.
The Langevin update (3) is designed to converge to the distribution p ( x ; θ ) learned from unlabeled data after an inﬁnite number of
Langevin steps.
The steady-state property guarantees that adversarial signals will be erased from the sampled state as long as enough steps are used because the sampled state will have no dependence on the initial state.
This property is actually too extreme because full MCMC mixing would completely undermine classiﬁcation by causing samples to jump between class modes.
The quasi-equilibrium and metastable behaviors of MCMC sampling (Bovier & den Hollander, 2006) can be as useful as its equilibrium properties.
Metastable behavior refers to intramodal mixing that can occur for MCMC trajectories over intermediate time-scales, in contrast to the limiting behavior of full intermodal mixing that occurs for trajectories over arbitrarily large time-scales.
Although slow-mixing and high autocorrelation of MCMC chains are often viewed as major shortcomings, these properties enable EBM defense by preserving class-speciﬁc features while sampling erases adversarial signals.
Our EBM samples always exhibit some dependence on the initial state for computationally feasible
Langevin trajectories.
Mixing within a metastable region can greatly reduce the inﬂuence of an initial adversarial signal even when full mixing is not occurring.
Successful classiﬁcation of long-run
MCMC samples occurs when the metastable regions of the EBM p ( x ; θ ) are aligned with the class boundaries learned by the classiﬁer network f ( x ) .
Our experiments show that this alignment naturally occurs for convergent EBMs and naturally-trained classiﬁers.
No training modiﬁcation for f ( x ) is needed to correctly classify long-run EBM samples.
Our defense relies on a balance between the memoryless properties of MCMC sampling that erase noise and the metastable properties of MCMC sampling that preserve the initial state.
C. C HAOTIC D YNAMICS
Chaos theory provides another perspective for justifying the erasure of adversarial signals with longrun iterative transformations.
Intuitively, a deterministic system is chaotic if an initial inﬁnitesimal perturbation grows exponentially in time so that paths of nearby points become distant as the system evolves (i.e. the butterﬂy effect).
The same concept can be extended to stochastic systems.
The SDE
[Equation]
(11) where ξ ( t ) is Brownian motion and η ≥ 0 , that encompasses the Langevin equation is known to exhibit chaotic behavior in many contexts for sufﬁciently large η (Lai et al., 2003).
One can determine whether a dynamical system is chaotic or ordered by measuring the maximal
Lyapunov exponent λ given by
[Equation]
[Equation]
where δX η ( t ) is an inﬁnitesimal perturbation between system state at time t after evolution according to (11) from an initial perturbation δX η (0) .
For ergodic dynamics, λ does not depend on the initial perturbation δX η (0) .
Ordered systems have a maximal Lyapunov exponent that is either negative or 0, while chaotic systems have positive Lyapunov exponents.
The SDE (11) will have a maximal exponent of at least 0 since dynamics in the direction of gradient ﬂow are neither expanding nor contracting.
One can therefore detect whether a Langevin equation yields ordered or chaotic dynamics by examining whether its corresponding maximal Lyapunov exponent is 0 or positive.
We use the classical method of Benettin et al.
(1976) to calculate the maximal Lyapunov exponent of the altered form Langevin transformation (3) given by
[Equation]
(13) for a variety of noise strengths η .
Our results exhibit the predicted transition from noise to chaos.
The value η = 1 which corresponds to our training and defense algorithms is just beyond the transition
[Figure]
Langevin
[Figure]
η 0.
[Figure]
[Figure]
s a m p l e s from the ordered region to the chaotic region.
Our defense dynamics occur in a critical interval where ordered gradient forces that promote pattern formation and chaotic noise forces that disrupt pattern formation are balanced.
Oversaturation occurs when the gradient dominates and noisy images occur when the noise dominates.
The results are shown in Figure 8.
The unpredictability of paths under T η is an effective defense against BPDA because informative attack gradients cannot be generated through chaotic transformation.
Changes in adversarial perturbation from one BPDA attack to the next attack are magniﬁed by the Langevin transformation and it becomes difﬁcult to climb the loss landscape L ( F ( x ) , y ) to create adversarial samples.
Other chaotic transformations, either stochastic or deterministic, might be an interesting line of research as a class of defense methods.
D E FFECT OF N UMBER OF L ANGEVIN S TEPS ON FID S CORE
We examine the difference between convergent and non-convergent EBM sampling paths by measuring the FID score (Heusel et al., 2017) of samples initialized from the data (see Figure 9).
The convergent EBM maintains a reasonably low FID score across many steps so that long-run samples can be accurately classiﬁed.
The non-convergent EBM experiences a quick increase in FID as oversaturated samples appear.
Labels for the oversaturated samples cannot be accurately predicted by a naturally-trained classiﬁer, preventing successful defense.
E D ISCUSSION OF D EFENSE R UNTIME
Robustness does not depend on the computational resources of the attacker or defender (Athalye et al., 2018).
Nonetheless, we took efforts to reduce the computational cost of our defense to make it as practical as possible.
Our defense requires less computation for classiﬁer training but more computation for evaluation compared to AT due to the use of 1500 Langevin steps as a preprocessing procedure.
Running 1500 Langevin steps on a batch of 100 images with our lightweight EBM takes about 13 seconds on a RTX 2070 Super GPU.
While this is somewhat costly, it is still possible to evaluate our model on large test sets in a reasonable amount of time.
The cost of our defense also poses a computational obstacle to attackers, since the BPDA+EOT attack involves iterative application of the defense dynamics.
Thoroughly evaluating our defense on CIFAR-10 took about 2.5 days for the entire testing set with 4 RTX 2070 Super GPUs.
Our EBM is signiﬁcantly smaller and faster than previous EBMs used for adversarial defense.
Our
EBM has less than 700K parameters and sampling is about 20 × to 30 × faster than IGEBM and JEM, as shown in Appendix G. Generating adversarial images using PGD against a large base classiﬁer is also expensive (about 30 × slower for a PGD step compared to a Langevin step because our base
[Figure]
I [Figure] s c
o r e ( l o w e r
i s b e tt e r )
FID Score vs. Langevin Steps for
Data-Initialized Samples (CIFAR-10 dataset)
[Figure]
classiﬁer uses the same Wide ResNet backbone as the JEM model).
Therefore 50 PGD steps against the base classiﬁer takes about the same time as 1500 Langevin steps with our EBM, so our model can defend images at approximately the same rate as an attacker who generates adversarial samples against the base classiﬁer.
Generating adversarial examples using BPDA+EOT is much slower because defense dynamics must be incorporated.
Further reducing the cost of our defense is an important direction for future work.
Our approach is amenable toward massive parallelization.
One strategy is to distribute GPU batches temporally and run our algorithm in high FPS with a time delay.
Another strategy is to distribute the workload of computing ˆ F H across parallel GPU batches.
F A N OTE ON M ODIFIED C LASSIFIER T RAINING FOR P REPROCESSING D EFENSES
Many works that report signiﬁcant robustness via defensive transformations (Cohen et al.
Raff et al.
; Yang et al.
(2019) and others) also modify classiﬁer learning by training with transformed images rather than natural images.
Prior defensive transformations that are strong enough to remove adversarial signals have the side effect of greatly reducing the accuracy of a naturallytrained classiﬁer.
Therefore, signals introduced by these defensive transformations (high Gaussian noise, ensemble transformations, ME reconstruction, etc.) can be also considered "adversarial" signals (albeit perceptible ones) because they heavily degrade natural classiﬁer accuracy much like the attack signals.
From this perspective, modifying training to classify transformed images is a direct analog of AT where the classiﬁer is trained to be robust to "adversarial" signals from the defensive transformation itself rather than a PGD attack.
Across many adversarial defense works, identifying a defensive transformation that removes attack signals while preserving high accuracy of a natural classiﬁer has universally proven elusive.
Our experiments train the classiﬁer with natural images alone and not with images generated from our defensive transformation (Langevin sampling).
Our approach represents the ﬁrst successful defense based purely on transformation and validates an entirely different approach compared to defenses which modify training of the base classiﬁer (Madry et al.
; Cohen et al.
; Raff et al.
; Yang et al.
(2019) etc.).
To our knowledge, showing that natural classiﬁers can be secured with post-training defensive transformation is a contribution that is unique in the literature.
Our task independent approach has the potential of securing images for many applications using a single defense model, while AT and relatives must learn a robust model for each application.
G D ISCUSSION OF IGEBM AND JEM D EFENSES
We hypothesize that the non-convergent behavior of the IGEBM (Du & Mordatch, 2019) and JEM (Grathwohl et al., 2020) models limits their use as an EBM defense method.
Long-run samples from both models have oversaturated and unrealistic appearance (see Figure 10).
Non-convergent learning behavior is a consequence of training implementation rather than model formulation.
Convergent learning may be a path to robustness using the IGEBM and JEM defense methods.
[Figure]
use very large networks to maximize scores on generative modeling metrics from be times lightweight
EBM structure from Nijkamp et al.
(see Figure 10).
The computational feasibility of our method currently relies on the the small scale of our EBM.
Given the effectiveness of the weaker and less expensive PGD attack in Section 4.2 and the extreme computational cost of sampling with large
EBM models, we do not to apply BPDA+EOT to the IGEBM or JEM defense.
The original evaluations of the IGEBM and JEM model use end-to-end backpropagation through the Langevin dynamics when generating adversarial examples.
On the other hand, the relatively weak attack in Section 4.2 is as strong or much stronger than the theoretically ideal end-to-end attack.
Gradient obfuscation from complex second-order differentiation might hinder the strength of end-to-end PGD when attacking Langevin defenses.
The IGEBM defense overcomes oversaturation by restricting sampling to a ball around the input image, but this likely prevents sampling from being able to manifest its defensive properties.
An adversarial signal will be partially preserved by the boundaries of the ball regardless of how many sampling steps are used.
Unrestricted sampling, as performed in our work and the JEM defense, is essential for removing adversarial signals.