Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Models
Authors: Mitch Hill
Jonathan Craig Mitchell
Song-Chun Zhu
Venue: ICLR 2021 Poster (2021)
Keywords: adversarial defense
adversarial robustness
energy-based model
Markov chain Monte Carlo
Langevin sampling
adversarial attack

Abstract: The vulnerability of deep networks to adversarial attacks is a central problem for deep learning from the perspective of both cognition and security. The current most successful defense method is to train a classifier using adversarial images created during learning. Another defense approach involves transformation or purification of the original input to remove adversarial signals before the image is classified. We focus on defending naturally-trained classifiers using Markov Chain Monte Carlo (MCMC) sampling with an Energy-Based Model (EBM) for adversarial purification. In contrast to adversarial training, our approach is intended to secure highly vulnerable pre-existing classifiers. To our knowledge, no prior defensive transformation is capable of securing naturally-trained classifiers, and our method is the first to validate a post-training defense approach that is distinct from current successful defenses which modify classifier training.

The memoryless behavior of long-run MCMC sampling will eventually remove adversarial signals, while metastable behavior preserves consistent appearance of MCMC samples after many steps to allow accurate long-run prediction. Balancing these factors can lead to effective purification and robust classification. We evaluate adversarial defense with an EBM using the strongest known attacks against purification. Our contributions are 1) an improved method for training EBM's with realistic long-run MCMC samples for effective purification, 2) an Expectation-Over-Transformation (EOT) defense that resolves ambiguities for evaluating stochastic defenses and from which the EOT attack naturally follows, and 3) state-of-the-art adversarial defense for naturally-trained classifiers and competitive defense compared to adversarial training on CIFAR-10, SVHN, and CIFAR-100. Our code and pre-trained models are available at https://github.com/point0bar1/ebm-defense.

Published as a conference paper at ICLR 2021
STOCHASTIC SECURITY: ADVERSARIAL DEFENSE USING LONG-RUN DYNAMICS OF ENERGY-BASE
MODELS
Mitch Hill
Jonathan Mitchell & Song-Chun Zhu 1
Department of Statistics and Data Science University of Central Florida
Department of Computer Science
Department of Statisties
mi mitchell 1el...iilll www.hhilll....... edu
University of California, Los Angeles jemi t wwwwhheeeeaae.eeeeeee edu sczhuu stat. ucla.edu
ABSTRACT
The vulnerability of deep networks to adversarial attacks a central problem for deep learning from the perspective of both cognition and security. The current most successful defense method is to train a classifier using adversarial images created during learning. Another defense approach involves transformation or purification of the original input to remove adversarial signals before the image is classified.
We focus on defending naturally-trained classifiers using Markov Chain Monte Carlo (MCMC) sampling with an Energy-Basee Model (EBM) for adversarial purification. In contrast to adversarial training, our approach intended secure highly vulnerable pre- pre-existing classifiers. To our knowledge, prior defensive transformation is capable securing naturally-traing is the first to validate post-training defense approach that is distinct from current successful defenses which modify classifier training.
classifiers, and our method
The memoryless behavior of long-run MCMC sampling will eventually remove adversarial signals, while metastable behavior preserves consistent appearance of MCMC samples after many steps to allow accurate long-run prediction. Bal- ancing these factors can lead to effective purification and robust classification. We evaluate adversarial defense with EBM using the strongest known at- tacks against purification. Our contributions are 1) an improved method for training EBM's with realistic long-run MCMC samples for effective purifica- Transformation tion, 2) an E (EOT) defense that resolves ambi- guities for evaluating stochastic defenses and from which the EOT attack nat- urally follows, and 3) state-of-the-art adversarial defense for naturally-trained classifiers and competitive defense compared to adversarial training CIFAR- 10, SVHN, and CIFAR- 100. Our code and pre-trained models are available at attps://githus cow www.pooot......occococ..o...t..t 0barl ebm defense
MOTIVATION AND CONTRIBUTIONS
Deep neural networks are highly sensitive to small input perturbations. This sensitivity can be exploited create adversarial examples that undermine robustness by causing trained networks to produce defective results from input changes that are imperceptible to the human eye (Goodfellow al., 2015). The adversarial scenarios studied in this paper are primarily untargeted white-bot box attacks on image classification networks. White-box attacks have full access to the classifier (in particular, classifier gradients) and are the strongest attacks against the majority of defenses.
Many whitebox methods have been introduced to create adversarial examples. Strong iterative attacks such as Projected Gradient Descent (PGD) (Madry et al.) (018)) can reduce the accuracy of naturally-traines classifier to virtually 0. Currently the most robust form of adversarial defense is to train a classifier on adversarial samples a procedure known as adversarial training (AT) (Madry
Equul Contribution

---

Published as a conference paper at ICLR 2021
Another defense strategy, which will refer as adversarial preprocessing (AP), uses defensive transformations to purify an image and remove or nullify adversarial signals before classification (Song et al. (2018); Guo et al. 2018); Yang et al. (2019), and others). AP is attractive strategy compared to AT because it has the potential to secure vulnerable pre-existing classifiers. Defending naturally trrineed classifiers is the central focus of this work.
al. 2018
Athalye et al. (2018) revealed that many preprocessing defenses can be overcome with minor adjustments the standard PGD attack. Both stochastic behavior from preprocessing and the computational difficulty of end-to-end backpropagation can be circumvented to attack the classifier through the defensive transformation. In this paper we carefully address Athalye et (2018) to evaluate AP with an EBM using attacks with the greatest known effectiveness and efficiency.
Clossiliet
T(x)
7(6))
X
EBM
Fn(x)
r(Aw)
H
h-1
Update
X
Backeward
Vx, (Xx)))
Figure 1: Left: Visualization of calculating our stochastic logits (x) from 5 The input image replicated H times and parallel Langevin updates with ConvNet EBM are performed on each replicate to generate { } Purified samples are sent in parallel to our naturally-trainee classifier network fzx and the resulting logits {f(,,) ) are averaged to produce Fn (x). The logits Fi (x) give an approximation of our true classifier logits (xx) in (4) that can be made arbitrarily precise by increasing H. Right: Graphical diagram of the Langevin dynamics (3) that
for T(x). Images
are iteratively updated with a gradient from a naturally-trained EBM (I) and Gaussian noise Za.
Langevin sampling using an EBM with a ConvNet potential (Xie et al. 2016) has recently emerged as method for AP (Du & Mordatch| 2019 Grathwohl al. 2020). However, the proposed defenses are not competitive with AT (see Table I and Croce & Hein (2020)) In the present work we demonstrate that EBM defense
naturally-trained classifier can stronger than standard AT (Madry et al.)
2018) and competitive with state-of-the-an thetth-artttttte AT (Zhang al. 2019 Carmon et al. 2019).
Our defense tools are a classifier trained with labeled natural images and an EBM trained with unlabeled natural images. For prediction, we perform Langevin sampling with the EBM and send the sampled images to the naturally-trained classifier. An intuitive visualization of our defense method shown in Figure [ Langevin chains constitute memoryless trajectory that removes adversarial signals, while metastable sampling behaviors preserve image classes over long-run trajectories. Balancing these two factors leads effective adversarial defense. Our main contributions are:
A simple but effective adjustment to improve the convergent learning procedure from Nijkamp et (2020). Our adjustment enables realistic long-run sampling with EBMs learned from complex datasets such as CIFAR-10.
An Expectation Exppetttioo-OveeeT.rannnaaaaoto (EOT) defense that prevents the possibility of a stochastic defense breaking due to random variation in prediction instead of an adversarial signal. The EOT attack (Athalye et al. 2018)) naturally follows from the EOT defense.
Experiments showing state-of-the- thrt defense for ceetturaall-trriinn classifiers and competitive defense compared to state-of-the-ant AT.
IMPROVED CONVERGENT LEARNING OF ENERGY-BASED MODELS
The Energy-Band -Based Model introduced in Xie et al. 2016) is a Sobbs-Boltzzann density
0)=;0)
exp 1 (x; 0))
Z
2

---

Published as a conference paper at ICLR 2021
where RD is image signal, (2; 8) is ConvNet with weights 0 and scalar output, and ( - exp(-U(<; 0) dr is the intractable normalizing constant. Given i.i.d. samples from a data distribution g(x), one can learn a parameter 0* such that p(2; 0*) = g(x) by minimizing the expected negative log-likelihood [()) Eg logp(X; 0)] of the data samples. Network weights are updated using the loss gradient
m
V2(0)
VXU 0)
(xU(X;:0)
(2)
where {X;>)_ via MCMC. Iterative application of the Langevin update
Vi are batch of training images and {X;]]" 3 are i.i.d. samples from p(æ; 0) obtained
Xk+1 X
V xxxx UX 0x -
(3)
where Z ~ N(O, ID) and r > 0 is the step size parameter, is used to obtain the samples (X; }
Adam
SGD
short-ru
long- min--m
10k batches
25k batches
50k batches
75k batches
100k batches
150k batches
Figure 2: Comparison of long-run and short-run samples over model updates for our improved
method of convergent learning. The model is updated in non-convergent learning phase with the Adam optimizer for the first 50,000 batches. The majority of short-run synthesis realism is learned during this phase, but the long-run samples are very unrealistic. The second learning phase uses SGD with a low learning rate. Short-run synthesis changes very little, but the long-run distribution gradually aligns with the short-run distribution.
Nijkamp et al. (2020) reveal that EBM learning heavily gravitates towards an unexpected outcome where short-run MCMC samples have a realistic appearance and long-run MCMC samples have an unrealistic appearance. The work uses the term convergent learning to refer to the expected outcome where short-run and long-run MCMC samples have similar appearance, and the term non-come non-convergent learning to refer the unexpected but prevalent outcome where models have realistic short-run samples and oversaturated long-run samples. Convergent learning is essential for our defense strategy because long-run samples must be realistic that classifiers maintain high accuracy after Langevin transformation (see Section and Appendix D
As observed by Nijkamp al. (2020), we were unable learn convergent model when updating 0 with the Adam optimizer (Kingma & Ba 2015). Despite the drawbacks of Adam for convergent learning, it is very effective tool for obtaining realistic short-run synthesis. Drawing inspiration from classifier training from Keskar & Socher (2017), we learn convergent EBM in two phases. The first phase uses Adam update uses SGD to update 0 to align short-run and long-run MCMC samples to correct the degenerate steady-state from the Adam phase. This modification allows us to learn the convergent MB for complex datasets such as CIFAR- 10 using 20% of the computational budget of Nijkamp et al. (2020). We use the lightweight EBM from Nijkamp et al. (2019) as our network architecture.
achieve realistic short-run synthesis. The second phase
See Figure 2
We use long run chains for our EBM defense to remove adversarial signals while maintaining image features needed for accurate classification. The steady-state convergence property ensures adversarial signals will eventually vanish, while metastable behaviors preserve features of the initial state. Theoretical perspectives on our defense can found in Appendix C and a comparison of convergent and non-convergent EBM defenses be found in Section 4.1 and Appendix D
3

---

Published as a conference paper at ICLR 2021
3 ATTACK AND DEFENSE FORMULATION
3.1 CLASSIFICATION WITH STOCHASTIC TRANSFORMATIONS
Let T(x) be a stochastic pre- pre-prrcsssing transformation for a state I RDD Given fixed input æ, the transformed state (x)) is random variable over RD_ In this work, T(x) is obtained from K steps of the Langevin update
3 starting from X, = z. One can compose T(x) with a deterministic classifier fx) E R (for us, ttturallyy-taiieed classifier) to define new classifier Fxx) E R as
F(Ø) (T(O)
F(x)) a deterministic classifier even though T(x) stochastic. The predicted label for x is then c(x) arg max; F(z),. In this work, f(z) denotes logits and F(z) denotes expected logits, although other choices are possible (e.g. softmax outputs). We refer to (4) as an Expectation -verr- Transformation (EOT) defense. The classifier F(x) in is simply the target of the EOT attack (Athalye et al.] 2018). The importance the EOT formulation is well-established for adversarial attacks, but its importance for adversarial defense has not yet been established. Although direct evaluation of F(x) is generally impossible, the law of large numbers ensures that the finite-sample approximation of F(x) given by
(x) Σ f(x)) where, By ~ T(x) i.d.
(5)
can approximate F(x) to any degree of accuracy for a sufficiently large sample size H. In other words, F(x) is intractable but trivial to accurately approximate via Fu (x) given enough computation.
Stochastic Logits Stable Classification
Stochastic Logits for Unstable Classification
3.0
25
nd shaice coneet
2d dhdche Ni 252
250
2d chaice Me 150
2.0
3-
density
density
2-
logit value Fr(x)) for class
-12
-15
5.0
05
20
logit value Fa(x), for class j
Figure 3: The histograms display different realizations of the logits (x) for the correct class and the second most probable class for images æ (left) and 22 (right) over different choices of H. In both cases, F(x) (dashed vertical lines) gives correct classification. However, the overlap between the logit histograms of Fnz (x1) indicate a high probability of misclassification even for large H, while (1()2) gives correct prediction even for small H because the histograms well- The EOT defense formulation is essential for securing borderline images such as 21-
In the literature both attackers (Athalye et al. 2018 Tramer al. 2020) and defenders (Dhillon et al.
2018 Yang et al. 2019 Grathwohl et al.) 2020) evaluate stochastic classifiers of the form f( (x)) using either Fi
(x) or FH... (4)) where Had is the number of EOT attack samples, typically around 10 to 30. This evaluation is not sound when (+) has a small but plausible chance of misclassification
because randomness alone could cause be identified an adversarial image even though Fyr (a)
gives the correct prediction on average (see Figure 33. In experiments with EBM defense, we identify
many images x that exhibit variation in the predicted label
čh (x) = arg max FH (z);
(6)
4

---

Published as a conference paper at ICLR 2021
for smaller H x 10 but which have consistently correct prediction for larger H 150. Fair evaluation of stochastic defenses must be based on the deterministic EOT defense F in and attackers must use sufficiently large H to ensure that FHr (z) accurately approximates (z) before declaring that an attack using adversarial image и is successful. In practice, we observe that Fi50 is sufficiently stable to evaluate over several hundred attacks with Algorithm [ for our EBM defense.
3.2 ATTACKING STOCHASTIC CLASSIFIERS
Let L(F()),)) E IR be the loss (e.g. cross-entropy) between label J} the outputs F(x) E I of classifier (e.g. the logits) for an image æ E RD. For a given pair of observed data (x y), untargeted white-box adversarial attack searches for the state
Fa .1) arg max L(F()),))
aES
that maximizes loss for predicting y in a S DR centered around x+ In this work, a natural image 2 will have pixels intensities from 0 to (i.e. xt E [0, I) One choice of S is the intersection of the image hypercube [o, 1] and the -nomm g-ball around x+ for suitably small > 0. Another option is the intersection of the hypercube [0, 1]_ with the ly-norm e-ball.
The Projected Gradient Descent (PGD) attack (Madry et al. 2018) is the standard benchmark when S is the E-ball in the ip norm. PGD begins at a random zo E S and maximizes (7) by iteratively updating Z; with
T++1 (c ag(Zt, g(x, v) arg max v. A(x, y):
11
where IIs denotes projection onto S, (ar, v) the attack gradient, and a > the attack step size. Standard PGD uses the gradient ArGD( y) V (FFx))
The EOT attack (Athalye et al. 2018) cireumvents the intractability of F by attacking finite sample logits Ha where Hadv the number of EOT attack samples, with the gradient
Δ (2, 3) L.
The EOT attack is the natural adaptive attack for our EOT defense formulation. Another challenge when attacking preprocessing defense is the computational infeasibility theoretical impossibil- ity of differentiating Txx). The Backward Pass Differentiable Approximation (BPDA) technique (Athalye et al. 2018) uses an easily differentiable function g(x) such that g(x) R T(x) to attack F(z) [([(z)) (). One calculates the attack loss using ((f(T(z)), y) on the forward pass but calcu- lates the attack gradient using V=L(f(g(z)). v) on the backward pass. A simple but effective form of BPDA is the identity approximation g(x)) = This approximation is reasonable for preprocessing defenses that seek to remove adversarial signals while preserving the main features of the original image. When g(z) xr the BPDA attack gradient is ABPDA (x, y) = V.L(f(2), y) where T(z). Intuitively, this attack obtains an attack gradient with respect to the purified image and applies it to the original image.
Combining the EOT attack and BPDA attack with identity g(x) = x gives the attack gradient
Han V
Hate
ABPDAHBOT (A
L
f(an).
in T(x) iid.
(10)
Hady
H
k=1
The BPDA+EOT attack represents the strongest known attack against preprocessing defenses, as shown by its effective use in recent works (Tramer et al. 2020). We use ABPDA-EOT (2, y) in (8) as primary attack to evaluate the EOT defense (4). Pseudo-com code for adaptive attack can be found in Algorithmm]]p Appendix A
4 EXPERIMENTS
We use two different network architectures in our experiments. The first network is the lightweight EBM from Nijkamp al. (019). The second network is 28 x 10 Wide ResNet classifier (Zagoruyko
5

---

Published as a conference paper at ICLR 2021
& Komodakis 2016). The EBM and classifier trained independently the same dataset. No adversarial training or other training modifications are used for either model. The EBM defense in our work is given by Equation 4 and approximated by Equation (5), where T(x) is K steps of the Langevin iterations in Equation We use the parameters from Algorithm I for all evaluations unless otherwise noted. In Section 4.
examine the effect of the number of Langevin steps K and the stability of Langevin sampling paths on defense. Section 4.2 examines our defense against
PGD attack from the base classifier, and Section 4.3 examines our defense against the adaptive BPDA+EOT attack.
4.1 COMPARISON OF CONVERGENT AND NON-CONVERGENT DEFENSES
Convergent EBM Defense Accuracy Langevin Steps (Cifar-10, norm, e= 8/255)
Non-Convergent EBM Defense Accuracy Langevin Steps (Cifar-10, norm, e 8/255)
1.0
natural accuracy la robustness
accuracy
accurap
0.6
0.4
0.4
0.2-
0.2
accuracy
robustness
0.0
0.0
1000
2000
3000
4000
5000
0
200
400
600
800
1000
Langevin steps
Langevin
K-
K 250
20
K 5000
K 100%
K 200
Figure 4: Accuracy on natural images and adversarial images from BPDA+EOT attack (T)) for EBM defense with different number of Langevin steps. and images sampled from the EBM. Left: Defense with a convergent EBM. Using approximately 1500 Langevin steps yields a good balance of natural and robust accuracy. Right: Defense with non-convergent EBM. Oversaturated long-run images prevent non-convergent EBM defense from achieving high natural or robust accuracy.
We examine the effect the number of Langevin steps has on defense accuracy for the CIFAR-10 dataset (see Figure 44). Each point displays either the baseline accuracy of our stochastic classifier (blue) or the results of BPDA+EOT attack (orange) 1000 test images. The attacks used to make this diagram use reduced load of Han accuracies are slightly higher than the full attack results presented in Figure 5 Short-run Langevin with K < 100 steps yields almost no adversarial robustness. Increasing the number steps gradually increases robustness until the defense saturates at around K 2000. We chose K = 1500 steps in our experiments as a good tradeoff between robustness, natural accuracy, and computational cost.
replicates for EOT attacks so these defense
the same experiment using nonvconver EBM. The network structure and training are identical to our convergent model, except that we use Adam instead of SGD throughout training. The non-convergent EBM defense cannot achieve high natural accuracy with long-run sampling because of the oversaturated features that emerge. Without a high natural accuracy, is impossible obtain good defense results. Thus convergent EBMs that can produce realistic long-run samples are a key ingredient for the success of our method. In the context of EBM defense, short-run sampling refers trajectories of up about 100 steps where no defensive properties emerge, and long-run sampling refers to trajectories of about 1000 or more steps where defense becomes possible.
For comparison,
4.2 PGD ATTACK FROM BASE CLASSIFIER FOR CIFAR- 10 DATASET
We first evaluate our defense using adversarial images created from a PGD attack on the classifier fæ)). Since this attack does not incorporate the Langevin sampling from T(x), the adversarial images in this section are relatively easy to secure with Langevin transformations. This attack serves as a benchmark for comparing our defense to the IGEBM (Du & Mordatch] 2019) and JEM (Grathwohl

---

Published as a conference paper at ICLR 2021
al. 2020) models that also evaluate adversarial defense with ConvNet EBM T.. For all methods, we evaluate the base classifier and the EBM defense for K = 10 Langevin steps (as in prior defenses) and K 1500 steps (as in our defense). The results are displayed in Table [
Table I:
CIFAR-I0 accuracy for our EBM defense and prior EBM defenses against PGD attack from the base elassifier / (ax) with &c perturbation 8/255. (*evaluated on 1000 images)
Base
EBM Defense,
EBM Defense,
Classifier f(x)
K = 10
K 1500
Nat.
Adv.
Nat.
Adv.
Nat.
Adv.
Ours
0.9530
0.0000
0.9586
0.0001
0.8412
0.7891
(uu & Mordatch 2019
0.4714
0.3219
0.4885
0.3674
0.487*
0.375
Grathwohl al. 2020)
0.9282
0.0929
0.9093
0.1255
0.755*
0.238
Our natural classifier f(x) has a high base accuracy but no robustness. The JEM base classifier has high natural accuracy and minor robustness, while the IGEBM base classifier has significant robustness but very low natural accuracy. Short-run sampling with K 10 Langevin steps does not significantly increase robustness for any model. Long-run sampling with K 1500 steps provides dramatic increase in defense for our method but only minor increases for the prior methods. Further discussion of the IGEBM and JEM defenses be found in Appendix G
4.3 BPDA+EOT ATTACK
In this section, we evaluate our EBM defense using the adaptive BPDA+EOT attack (10) designed specifically for defense approach. This attack is recently used by Tramer al. 2020) to evaluate the preprocessing defense from Yang et al. (2019) that is similar to our method.
Table 2: Defense vs. whitebox attacks with lo perturbation E = 8/255 for CIFAR-10.
Defense
(x) Natural
Train Ims.
T(x)
Method
Attack
Nat.
Adv.
Ours
Langevin
BPDA+EOT
0,8412
0.5490
Madry al.
2018
Adversarial
PGD
0.873
0.458
Zhang et al.
2019
Adversarial
PGD
0.849
0.5643
Arrmon
2019
Adversarial
PGD
0.897
0.625
(Song et al. 2018)
Natural
Gibbs Update
BPDA
0.95
0.09
Srinivasan et
2019
Natural
Langevin
PGD
0.0048
(Yng al.
2019))
Transformed
Mask + Recon. BPDA+EOT
0.94
0.15
CIFAR-10. We random restarts of the BPDA+EOT attack in Algorithm T with the the listed parameters on the entire CIFAR-10 test set. In particular, the attacks use adversarial perturbation 8 8/255 and attack step size 2/255 in the loo norm. One evaluation of the entire approximately 2.5 days using 4x RTX 2070 Super GPUs. We compare our results to representative We include the training method for the classifier,
took
selection of AT and AP defenses in Table 2 preprocessing transformation (if any), and the strongest attack for each defense.
Our EBM defense stronger than standard AT Madry et al.. 2018) and comparable to modified AT from Zhang al. (2019). Although our results are not par with state-of-thh-art AT (Carmon et al. 2019). our defense is the first method that can effectively secure naturally-traingg classifiers.
We now examine the effect of the perturbation e, number of attacks N. and number of EOT attack replicates Hadv on the strength of the BPDA+EOT attack. To reduce the computational cost of the diagnostics,
fixed set of 1000 randomly selected test images for diagnostic attacks.
Figure 5 displays the robustness of our model compared against two AT models, standard AT (Madry al. 2018) and Semi-supervised AT (Carmon et al. 2019) for 42 and l2 attacks across perturbation size E. Our model is attacked with BPDA+EOT while AT models are attacked with PGD. Our defense more robust than standard AT for range of medium-size perturbations in the la norm and much
7

---

Published as a conference paper at ICLR 2021
Cifar-10 Robustness
Cifar-10 l Robustness across
(base
Madry
tSp atlaces)
0.8
Cormon et
accuracy
accuracy
6.6
0.4-
Our
0.2
durs aitadaa)
Camoon at
0.04
0.06
0.10
0.12
0.25
0.50
0.75
1.50
1.25
1550
1.55
2.00
perturbation size
perturbation
Figure 5: Accuracy across perturbation e for lxc and l2 attacks against our defense, standard AT (Madry et al. 2018) and Semi-supervvsed AT (Carmon et al. 2019).
Defense 1000 Whitebox Attacks (Cifar-10, la norm, é 8/255)
Effect EOT replicates (Cifar-10, I. norm, = 8/255)
0.65
Hatr
0.8
accuracy
0.500
0.7
0.55
0.6
0.50
0.5
0.45
10f
10
10a
EOT replicates
0.4
100
800
attack steps N
Figure Effect of number of attack steps N and number of EOT replicates Hadv and Haef-
more robust than standard AT for medium and large E in the l2 norm. Our model is less robust than Semi-supervised AT for the &oe norm but more robust for the l2 norm. AT-based models are trained to defend against a specific attack while our method does not use any attacks during training. This is likely a reason why our defense outperforms (oo -trained AT against l2 attacks.
Figure 6 visualizes the effect of increasing the computational power of the attacker and defender in Algorithm|| | The left figure compares our defense with AT over 1000 attacks using an increased number H. 20 of attack replicates. The majority of breaks happen within the first 50 attacks as used in the CIFAR-10 experiment in Table 2 while a small number of breaks occur within a few hundred attack steps. It is likely that some breaks from long-run attacks the result of lingering stochastic behavior from F (ar) rather than the attack itself. The right figure shows the effect of the number of EOT replicates. The strength of the EOT attack saturates when using 20 to 30 replicates. small gap attack strength remains between the 15 replicates used in our attacks and the strongest possible attack. Some of this effect is likely mitigated by our use of 5 random restarts. Defense with Har 150 is close to the maximum achievable defense for our method, although more replicates would slightly strengthen our results.
The evaluation in Table pushes the limits of what is computationally feasible with widely available resources. The diagnostics indicate that our defense report is an accurate approximation of the defense of our ideal classifier Fax) against the BPDA+EOT attack 10), although more computation would yield slightly more accurate estimate.
SVHN and CIFAR-100. The attack and defense parameters for our method are identical to those used in the CIFAR-I0 experiments. We compare results with standard AT. Overall, our defense performs well for datasets that are both simpler and more complex than CIFAR-10. 10. In future work, further stabilization of image appearance after Langevin sampling could yield significant benefits for settings where precise details need to be preserved for accurate classification. The AT results for CIFAR-100 100 are from Balaji et al. (019) and the results for SVHN from implementation.
8

---

Published as a conference paper at ICLR 2021
Table 3: Defense vs, whitehox attacks with Ice perturbation & 8/255 for SVHN and CIFAR-IOM
SVHN
CIFAR- 100
Nat.
Adv.
Nat.
Adv.
Ours
0.9223
0.6755
0,5166
0.2610
Madry et al. 2018
0.8957
0.5039
0.5958
0.2547
5 RELATED WORK
Adversarial training learns robust classifier using adversarial images created during each weight update. The method is introduced by Madry et al. (2018). Many variations of AT have been explored, some of which are related to our defense. He et al. (2019) apply noise injection to each network layer to increase robustness via stochastic effects. Similarly, Langevin updates with our EBM can be interpreted as ResNet (He et al. (016) with noise injected layers as discussed by Nijkamp al. 2019). Semi-supervised AT methods (Alayrac et al.] 2019 Carmon et al. 2019) use unlabeled images to improve robustness. Our EBM also leverages unlabeled data for defense.
Adversarial preprocessing is a strategy where auxiliary transformations are applied to adversarial inputs before they are given to the classifier. Prior forms of pre-processing defenses include rescaling (Xie et al.] 2018a), thermometer encoding (Buckman et al. 2018), feature squeezing (Xu et al.
2018), activation pruming (Dhillon al., 2018). reconstruction (Samangouei al. (018). ensemble transformations (Raff et all|2019,, addition of Gaussian noise (Cohen et al. 2019), and reconstruction of masked images (Yang al. 2019). Prior preprocessing methods also include energy-based models such as Pixel-Defend (Song et al. (2188) and MALADE (Srinivasan et al. 2019) that draw samples from a density that differs from ( It was shown by Athalyy et al. (2018); Tramer al. (2020) that many preprocessing defenses can be totally broken or dramatically weakened by simple adjustments the standard PGD attack. namely the EOT and BPDA techniques. Furthermore, many preprocessing defenses such as Cohen et al. (2019): Raff et al. (2019); Yang et al.) (2019) also modify classifier training so that the resulting defenses are analogous to AT, as discussed in Appendix FF A concurrent work called Denoised Smoothing (Salman et al. 2020) prepends Denoising Autoencoder to natural image classifier. The joint classifier fixes the natural classifier weights and the denoiser is trained by minimizing the cross-entropy loss of the joint model using noisy training inputs. Our defensive transformation provides security without information from any classifier. No prior preprocessing defense is competitive with AT when applied independent naturally-traine classifiers.
Energy-based models are a probabilistic method for unsupervised modeling. Early energy- based image models include the FRAME (Zhu et al.) 1998) and RBM (Hinton (002). The EBM is descriptive model (Guo et al.) 2003 Zhu 2003 Our EBM (T) is introduced as the DeepFRAME model by Xie al. (2016) and important observations about the learning process are presented by Nijkamp et al. (2020) 2019). Other studies of EBM learning include Gao et al. (2018): :): Xie et al. (2018b); Gao al. (2020). Preliminary investigations for using the EBM (I) for ad adversarial defense are presented by Du & Mordatch (2019); Grathwohl et al. (2200) but the results are not competitive with AT (see Section (4.).. Our experiments show that the instability of sampling paths for these non-convergent models prevents Langevin trajectories from manifesting defensive properties. We build on the convergent learning methodology from Nijkamp et al. (2020) to apply long-run Langevin sampling as defense technique.
6 CONCLUSION
This work demonstrates that Langevin sampling with a convergent EBM is an effective defense for naturally-trained classifiers. Our defense is founded on an improvement to EBM training that enables efficient learning of stable long-run sampling for complex datasets. We evaluate our defense using non-adaptive and adaptive whitebox attacks for the CIFAR- 10, CIFAR-100, 100, and SVHN datasets. Our defense is competitive with adversarial training. Securing naturally-trainee classifiers with post-training defense is a long-standing open problem that this work resolves.

---

Published as a conference paper at ICLR 2021
REFERENCES
Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In Advances Neural Information Processing Systems, volume 32, 2019.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of
security: Cireumventing defenses to adversarial examples. In Proceedings of the 35th International Conference on Machine Learning, pp. 274-283, 2018.
Yogesh Balaji, Tom Goldstein, and Judy Hoffman. Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets. arXiv preprint arXiv: 2019.
Giancarlo Benettin, Luigi Galgani, and Jean-Marie Streclyn. Kolmogorov entropy and numerical experiments. Physical Review A, 14(6):2338-2345, 1976.
Anton Bovier and Frank den Hollander. Metastability: A potential theoretic approach. International Congress of Mathematicians, 3:499-518, 2006.
Jacob Buckman, Aurko Roy, Colin Raffel, and Ian Goodfellow. Thermometer encoding: One hot way to resist adversarial examples. In International Conference on Leurning Reprexentations, 2018.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John Duchi, and Percy Liang. Unlabeled data improves adversarial robustness. In Advances in Neural Information Processing Systems, volume 32, 2019.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In Proceedings of the 36th International Conference on Machine Learning, pp. 1310-1320, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In Proceedings of the 37th International Conference on Machine Learning, 2020.
Guneet S. Dhillon, Kamyar Azizzadenesheli, Jeremy D. Bernstein, Jean Kossaifi, Aran Khanna, Zachary C. Lipton, and Animashree Anandkumar. Stochastic activation pruning for robust adver- sarial defense. International Conference on Learning Representations, 2018.
Yilun Du and Igor Mordatch. Implicit generation and modeling with energy based models. In Advances in Neural Information Processing Systems, volume 32, 2019.
Ruiqi Gao, Yang Lu, Junpei Zhou, Song- Chun Zhu, and Ying Nian Wu. Learning generative convnets via multi-grid modeling and sampling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1155-9164 2018.
Ruiqi Gao, Erik Nijkamp, Diederik P Kingma, Zhen Xu, Andrew M Dai, and Ying Nian Wu. Flow contrastive estimation of energy-based models. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7518-7528, 2020.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
Will Grathwohl, Kuan-Chieh Wang. Been-k Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. International Conference on Learning Representations, 2020.
Cheng-En Guo, Song-Chun Zhu, and Ying Nian Wu. Modeling visual patterns by integrating descriptive and generative methods. International Journal of Computer Vision, 53:5-29, 2003.
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens der Maaten. Countering adversarial images using input transformations. International Conference on Learning Representations, 2018.
10

---

Published as a conference paper at ICLR 2021
Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Patter Recognition, pp. 770-778, 2016.
Zhezhi He, Adnan Siraj Rakin, and Deliang Fan. Parametric noise injection: Trainable randomness to improve deep neural network robustness against adversarial attack. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 588 388-597, 2019.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems, volume 33, pp. 6629-6640, 2017.
Geoffrey E. Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8):1771-1800, 2002.
Nitish Shirish Keskar and Richard Socher. Improving generalization by switching from adam sgd. arXiv preprint /rXiv: 172.00662, 2017.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.
Ying-Cheng Lai, Zonghua Liu, Lora Billiings, and Ira B. Schartz. Noise-induced unstable dimension variability and transition to chaos in random dynamical systems. Physical Review E, 67, 2003.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
Erik Nijkamp, Mitch Hill, Song-Chur Zhu, and Ying Nian Wu. Learning non-convergent persistent short-run MCMC toward energy-based model. In Advances in Neural Information Processing Systems, volume 32, 2019.
Erik Nijkamp, Mitch Hill, Tian Han, Song-Chun Zhu, and Ying Nian Wu. On the anatomy of MCMC-based maximum likelihood learning of energy-based models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, 2020.
Edward Raff, Jared Sylvester, Steven Forsyth, and Mark McLean. Barrage of random transforms for adversarially robust defense. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 6528-6533 2019.
Hadi Salman, Mingjie Sun, Greg Yang, Ashish Kapoor, and J. Zico Kolter. Denoised smoothing: A provable defense for pretrained classifiers. Advances in Neural Information Processing Systems, volume 33, pp. 111945221955 2020.
Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Lefense-GANt Protecting classifiers against adversarial attacks using generative models. International Conference on Learning Representations, 2018
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Lever- aging generative models to understand and defend against adversarial examples. In International Conference Learning Representations, 2018
Vignesh Srinivasan, Arturo Marban, Klaus- Muller, Wojeiech Samek, and Shinichi Nakajima. Defense against adversarial attacks by langevin dynamics. arxiv preprint arXiv:1805.12000 2019.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. In Advances Neural Information Processing Systems, volume 33, pp. 1633-1645, 2020.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. In International Conference on Learning Representations, 20.
Jianwen Xie. Yang Lu, Song-Chin Chun Zhu, and Yingnian Wu. A theory of generative convnet. In Proceedings of the 33rd International Conference on Machine Learning, pp. 2635-2644 2016.
11

---

Published as a conference paper at ICLR 2021
Jianwen Xie, Yang Lu, Ruiqi Gao, and Ying Nian Wu. Cooperative learning of energy-based model and latent variable model via meme teaching. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018b.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In 25th Annual Network and Distributed System Security Symposium, NDSS, 2018.
Yuzhe Yang, Guo Zhang, Dina Katabi, and Zhi Xu. Me-net: Towards effective adversarial robustness with matrix estimation. In Proceedings of the 36th International Conference Machine Learning, pp. 7025-7034, 2019.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.07144 146, 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In Proceedings of the 36th International Conference on Machine Learning, pp. 7472-7482, 2019.
Song-Chun Zhu, Statistical modeling and conceptualization of visual patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25:691-712, 2003.
Song-Chur Chun Zhu, Ying Nian Wu, and David Mumford. Filters, random fields and maximum entropy (FRAME): towards a unified theory for texture modeling. International Journal of Computer Vision, 27(2):107-126, 1998.
12

---

Published as a conference paper at ICLR 2021
A ATTACK ALGORITHM
Algorithm BPDA+EOT adaptive attack to evaluate EOT defense (4)
Require: Natural images {= 1M m=11 EBM U(x), classifier [(x), Langevin noise 0.01,
Langevin updates K 1500. number of attacks N 50, attack step size
2
maximum
255
perturbation size
& EOT attack samples Havv 15, EOT defense samples Haet 150 255
Ensure: Defense record {dmm JM for m=1:M do
m=1 for each image.
Calculate larpe predicted label of the natural image ČH (aith) with 66)
if HH (x + y/m then
Natural image misclassified. dm + False. End loop iteration m.
else
dm t True.
end if
Randomly initialize Xo in the Ip e-ball centered at o. and project to [0, IjP.
for n=1:(N+1) do
Calculate small-sample predicted label HH (Xn-1) with (6).
Calculated attack gradient ABPDA-BOT n 3lm) with (O)).
if CH (Xn-1) # Vm then
Calculate large-sample predicted label ĆHA (Xn-1) with ( if @rran (Xm-1) * ym then
The attack has succeeded. dm f- False. End loop iteration m.
end if end if
Use ABPDA-EOT(X--1 Vm) with the 4p e-bounded PGD update 8 to obtain X,..
end for end for
Algorithm|] iis pseudo-code for the attack and defense framework described in Section One notable aspect of the algorithm is the inclusion of EOT defense phase to verify potentially successful attacks. Images which are identified as broken for the smaller sample used to generate EOT attack gradients checked again using a much larger EOT defense sample ensure that the break is due to the adversarial state and not random finite-sample effects. This division is done for purely computational reasons. It is extremely expensive use 150 EOT attack replicates but much less expensive to use 15 EOT attack replicates as screening method and to carefully check candidates for breaks when they are identified from time to time using 150 EOT defense replicates. experiments we find that the EOT attack is close to its maximum strength after about 15 to 20 replicates are used, while about 150 EOT defense replicates are needed for consistent evaluation of F from (4) over several hundred attacks. Ideally, the same number of chains should be used for both EOT attack and defense, in which case the separate verification phase would not be necessary.
our
B IMPROVED LEARNING OF CONVERGENT EBMS
Algorithm 2 provides pseudo-code for our improvement of the convergent learning method from Nijkamp 2020). This implementation allows efficient learning of convergent EBMs for complex datasets.
C ERASING ADVERSARIAL SIGNALS WITH MCMC SAMPLING
This section discusses two theoretical perspectives that justify the use of EBM for purifying adversarial signals: memoryless and chaotic behaviors from sampling dynamics. We emphasize that the discussion applies primarily to long-run behavior of Langevin image trajectory. Memoryless and chaotic properties do not appear to emerge from short-run sampling. Throughout our experiments, observe significant defense benefits from short-run Langevin sampling.
13

---

Published as a conference paper at ICLR 2021
Algorithm 2 ML with Adam to SGD Switch for Convergent Learning of EBM
Require: ConvNet potential (z; 0), number training steps = 150000, step to switch from
SGD to Adam JSGD = 50000, initial weight 01. training images {2 + 1 Mas
li=1* data perturbation
Tdata 0.02, step size
0.01, Langevin steps K 100, Adam learning rate TAdam 0.0001,
SGD learning rate "SGD = 0.00005.
Ensure: Weights f for energy ();
Set optimizer 9 f Adam(1Adam) Initialize persistent image bank Naata uniform noise images. for j==:(J+1) do j j=j= JSGD then
Set optimizer g SGD(1sGD).
end if
1. Draw batch images {
|m from training set, where (i) indicates a randomly selected index for sample i, and get samples X;+ = =(() + Tama Zi where Z; ~ N(O, In) i.i.d.
2. Draw initial negative samples {y; mm from persistent image bank. Update
(Y,") Timi with the Langevin equation
72 0 (Y + TZz,, y:(A) y((k-1)
2 dy
where Zi,N N(O, ID) i.idd., for K steps to obtain samples { mm
( am
j==1
Update persistent image bank with images (X (K) (x,(K) J
Update the weights by 8J+1 8, ((AB)), where g the optimizer and
00
0
E n
u(xt:
- m
-U(T::0))
00
the MI gradient approximation.
end for
Energy of Natural, Adversarial, and Noise Images
natural images
0.0010
adversarial images uniform noise
0.0008
density
0.0006
0.0004
0.0002
0.0000
-4000
72000
78000
energy U(x: 0)
Figure 7: Energy U(x; of natural, adversarial, and noise images.
C. MEMORYLESS DYNAMICS
The first justification of EBM defense is that iterative probabilistic updates will move an image from cow-brobayllity adversarial region to high-probability natural region, discussed in prior works (Song et al., 2018 Srinivasan et al.) 2019 Grathwohl et al.) (2020). Comparing the energy of adversarial and natural images shows that adversarial images tend to have marginally higher energy,
14

---

Published as a conference paper at ICLR 2021
which is evidence that adversarial images improbable deviations from the steady-state manifold of the EBM that models the distribution of natural images (see Figure (7)
The thcoretical foundation for removing adversarial signals with MCMC sampling comes from the well-known steady-state convergence property of Markov chains. The Langevin update (3) is designed to converge to the distribution p(a; 0) learned from unlabeled data after infinite number of Langevin steps. The steady-state property guarantees that adversarial signals will be erased from the sampled state long as enough steps are used because the sampled state will have no dependence on the initial state. This property is actually too extreme because full MCMC mixing would completely undermine classification by causing samples to jump between class modes.
The quasi--quilibrium and metastable behaviors of MCMC sampling (Bovier & den Hollander (006) can be as useful as its equilibrium properties. Metastable behavior refers intramodal mixing that can occur for MCMC trajectories over intermediate time-scales, in contrast to the limiting behavior of full intermodal mixing that occurs for trajectories over arbitrarily large time-scales. Although slow-mixing and high autocorrelation of MCMC chains are often viewed major shortcomings, these properties enable EBM defense by preserving class-specific features while sampling erases adversarial signals.
Our EBM samples always exhibit some dependence on the initial state for computationally feasible Langevin trajectories. Mixing within metastable region can greatly reduce the influence of an initial adversarial signal even when full mixing is not occurring. Successful classification of long-run MCMC samples occurs when the metastable regions of the EBM p(æ; 0) aligned with the class boundaries learned by the classifier network f fax). Our experiments show that this alignment naturally occurs for convergent EBMs and naturally-trained classifiers. No training modification for / (xx) is needed to correctly classify long-run EBM samples. Our defense relies on a balance between the memoryless properties of MCMC sampling that erase noise and the metastable properties of MCMC sampling that preserve the initial state.
C.2 CHAOTIC DYNAMICS
Chaos theory provides another perspective for justifying the erasure of adversarial signals with long- run iterative transformations. Intuitively, a deterministic system is chaotic if an initial infinitesimal perturbation grows exponentially in time so that paths of nearby points become distant as the system evolves (i.e. the butterfly effect). The same concept can be extended to stochastic systems. The SDE
d.X
dt -V(X)= + nE(())
(11)
where E({) is Brownian motion and 1) 0, that encompasses the Langevin equation is known to exhibit chaotic behavior in many contexts for sufficiently large 1) (Lai et al. 2003).
One can determine whether dynamical system is chaotic or ordered by measuring the maximal Lyapunov exponent À given by
1
(x) (X)
A lim log
(12)
where (t) is infinitesimal perturbation between system state at time after evolution according to (11 from an initial perturbation 8X,, (0). For ergodic dynamics, does not depend on the initial perturbation (0). Ordered systems have maximal Lyapunov exponent that is either negative or O, while chaotic systems have positive Lyapunov exponents. The SDE (1]) will have maximal exponent of least 0 since dynamics in the direction of gradient flow neither expanding nor
contracting, One can therefore detect whether a Langevin equation yields ordered or chaotic dynamics by examining whether its corresponding maximal Lyapunov exponent is 0 or positive.
We use the classical method of Benettin et al. (1976) to calculate the maximal Lyapunov exponent of the altered form Langevin transtormation given by
T, = x
V Ox + Z
(13)
for a varicty of noise strengths 77 Our results exhibit the predicted transition from noise to chaos. The value 77 = 1 which corresponds to training and defense algorithms is just beyond the transition
15

---

Published as a conference paper at ICLR 2021
0.5
0.95
1.05
Lyapunov exponent Langevin noise magnitude
maximal Lyapunov exponent
10
10
10a
10a
noise magnitude
Figure 8: Left: Maximal Lyapunov exponent for different values of The value = which corresponds our training and defense sampling dynamics is just above the transition from the ordered region where the maximal exponent is to the chaotic region that where the maximal exponent positive. Right: Appearance of steady-state samples for different values 1j. Oversaturated images appear for low values of 7). while noisy images appear for high r) Realistic synthesis is achieved in small window around rj = where gradient and noise forces are evenly balanced.
from the ordered region the chaotic region. Our defense dynamics occur in critical interval where ordered gradient forces that promote pattern formation and chaotic noise forces that disrupt pattern formation are balanced. Oversaturation occurs when the gradient dominates and noisy images occur when the noise dominates. The results are shown in Figure
The unpredictability of paths under T is effective defense against BPDA because informative attack gradients cannot be generated through chaotic transformation. Changes in adversarial perturba- tion from one BPDA attack the next attack are magnified by the Langevin transformation and becomes difficult to climb the loss landscape L(F(x), y) to create adversarial samples. Other chaotic transformations, either stochastic or deterministic. might be an interesting line of research as class of defense methods.
D EFFECT OF NUMBER OF LANGEVIN STEPS ON FID SCORE
We examine the difference between convergent and non-convergent EBM sampling paths by suring the FID score (Heusel et all 2017) of samples initialized from the data (see Figure The convergent EBM maintains reasonably low FID score across many steps so that long-run samples can be accurately classified. The non non-convergent EBM experiences quick increase in FID as oversaturated samples appear. Labels for the oversaturated samples cannot be accurately predicted by a naturally-traine classifier, preventing successful defense.
E DISCUSSION OF DEFENSE RUNTIME
Robustness does not depend on the computational resources of the attacker or defender (Athalye et al. 2018). Nonetheless, we took efforts to reduce the computational cost of our defense to make practical possible. Our defense requires less computation for classifier training but more preprocessing
computation for evaluation compared to AT due to the use of 1500 Langevin steps procedure. Running 1500 Langevin steps on a batch of 100 images with our lightweight EBM takes about 13 seconds on RTX 2070 Super GPU. While this is somewhat costly, it is still possible to evaluate our model on large test sets in reasonable amount of time. The cost of our defense also poses computational obstacle attackers, since the BPDA+EOT attack involves iterative application of the defense dynamics. Thoroughly evaluating our defense on CIFAR- 10 took about 2.5 days for the entire testing set with RTX 2070 Super GPUs
Our EBM is significantly smaller and faster than previous EBMs used for adversarial defense. Our EBM has less than 700K parameters and sampling is about 20x to 30x faster than IGEBM and JEM, as shown in Appendix G Generating adversarial images using PGD against a large base classifier is also expensive (about x slower for PGD step compared to Langevin step because base
16

---

Published as a conference paper at ICLR 2021
FID Score vs. Langevin Steps for
Data-Initialized Samples (CIFAR-10 10 dataset)
FID score (lower is better)
10a
10a
Convergent EBM
10a
Non-Convergent EBM
2000
4000
6000
8000
10000
Langevin steps
Figure 9: FID scores of samples from a convergent and non-convergent EBM that are initialized from training data. The FID of the convergent model remains reasonably low because the steady of the convergent model is aligned with the data distribution. The FID of the non-convergent model quickly increases because the oversaturated steady-state of the non-convergent model differs significantly from the data distribution. Maintaining sampling distribution close to the data distribution is essential for achieving high prediction accuracy of a natural classifier on transformed states.
classifier uses the same Wide ResNet backbone as the JEM model). Therefore PGD) steps against the base classifier takes about the same time 1500 Langevin steps with EBM, can defend images at approximately the same rate as an attacker who generates adversarial samples against the base classifier. Generating adversarial examples using BPDA+EOT is much slower because defense dynamics must incorporated. Further reducing the cost of our defense is an important direction for future work.
model
Our approach is amenable toward massive parallelization. One strategy to distribute GPU batches temporally and algorithm in high FPS with time delay. Another strategy distribute the workload of computing F'H across parallel GPU batches.
F A NOTE ON MODIFIED CLASSIFIER TRAINING FOR PREPROCESSING DEFENSES
Many works that report significant robustness via defensive transformations (Cohen et (011)) ;
Raff et al. (2019); Yang et al. (2019) and others) also modify classifier learning by training with transformed images rather than natural images. Prior defensive transformations that are strong enough to remove adversarial signals have the side effect of greatly reducing the accuracy of naturally- trained classifier. Therefore, signals introduced by these defensive transformations (high Gaussian noise, ensemble transformations, ME reconstruction, etc.) can be also considered "adversarial" signals (albeit perceptible ones) because they heavily degrade natural classifier accuracy much like the attack signals. From this perspective, modifying training to classify transformed images is direct analog of AT where the classifier is trained to be robust to "adversarial" signals from the defensive transformation itself rather than PGD attack. Across many adversarial defense works, identifying a defensive transformation that removes attack signals while preserving high accuracy of a natural classifier has universally proven elusive.
Our experiments train the classifier with natural images alone and not with images generated from our defensive transformation (Langevin sampling). Our approach represents the first successful defense
17

---

Published as a conference paper at ICLR 2021
based purely transformation and validates entirely different approach compared to defenses which modify training of the base classifier (Madry et al. (2018); [Cohen et al. (2019); Raff et al. (2019); Yang al. (2019) etc.). To our knowledge, showing that natural classifiers
be secured with post-training defensive transformation is a contribution that is unique in the literature. Our task independent approach has the potential of securing images for many applications using single defense model, while AT and relatives must learn robust model for each application.
G DISCUSSION OF IGEBM AND JEM DEFENSES
We hypothesize that the non-convergent behavior of the IGEBM (Du & Mordatch 2019) and JEM (Grathwohl et al. 2020) models limits their use as EBM defense method. Long-run samples from both models have oversaturated and unrealistic appearance (see Figure 10.. Non Non-convergent learning behavior is consequence of training implementation rather than model formulation. Convergent learning may be path to robustness using the IGEBM and JEM defense methods.
IGEBM
Running Time Number of Langevin Steps
Relative Runtime vs. Our EBM
run time (seconds)
10a
30
relative runtime
20
102
JEM
10a
Durs ICEBM
IGEBM
JEM
JEM
2500
5000
7500
10000
2500 5000
7500 10000
number of Langevin steps K
number of Langevin steps K
Figure 10: Left: Approximate steady-state samples of the IGEM and JEM models. Both exhibit oversaturation from non-convergent learning that can interfere with defense capabilities. Right: Comparison of running time for Langevin sampling with batch of 100 images. The small scale and fast sampling of our EBM are important for the computational feasibility of our defense.
Both prior works very large networks maximize result, sampling from these models can be up to 30 times slower than sampling from our lightweight EBM structure from Nijkamp et al. (2019) (see Figure I0). The computational feasibility of our method currently relies on the the small scale of our EBM. Given the effectiveness of the weaker and less expensive PGD attack in Section 4.2 and the extreme computational cost of sampling with large EBM models, we do not to apply BPDA+EOT to the IGEBM or JEM defense.
on generative modeling metrics. As
The original evaluations of the IGEBM and JEM model use end-to-end backpropagation through the Langevin dynamics when generating adversarial examples. On the other hand, the relatively weak attack Section 4.2 is as strong or much stronger than the theoretically ideal end-to-end attack. Gradient obfuscation from complex second-order differentiation might hinder the strength of end-to-end PGD when attacking Langevin defenses.
The IGEBM defense overcomes oversaturation by restricting sampling to ball around the input image, but this likely prevents sampling from being able to manifest its defensive properties. An adversarial signal will be partially preserved by the boundaries of the ball regardless of how many sampling steps are used. Unrestricted sampling, as performed in our work and the JEM defense, is essential for removing adversarial signals.
18


Reviews:
[{"review_id": "ylx_fWvNXN7", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 9, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes an interesting application of Langevin sampling with the energy-based model (EBM) to defend against adversarial attacks. Compared to adversarial training (AT), the proposed adversarial preprocessing (AP) based method can be used to secure existing models without the need for retraining.\n\nAs an EOT based defense, naturally, a larger number of replicates would lead to better adversarial robustness. The idea behind the proposed defense is to leverage the long-run sampling using EBM for the EOT based defense. However, training long-run EBM is challenging. To overcome this, the authors apply an interesting two-phase training method with Adam for the first phase to quickly train the short-run one and SGD for the second phase to gradually align the long-run one with the short-run one.\n\nThe authors compared their approach to a large number of defenses as well as the state-of-the-art AT based defense. The reported results are very impressive, with an 84.12% benign accuracy and 78.9% adversarial robustness on CIFAR-10. In addition, the proposed method can still achieve 54.9% adversarial robustness even under an adaptive attack, which is close to the state-of-the-art AT based defense. However, compared to AT, the benefit of the proposed method is that it does not require retraining the model. \n\n\nStrength:\n- Achieve robustness performance similar to adversarial training without tampering with the classifier.\n- An interesting two-phase training method to address the non-convergent problem for long-run sampling.\n- Consider an EOT attack as the adaptive attacking method.\n\nWeakness:\n- Overall, it is hard to find flaws in this paper. The only concern I have is that the timing overhead is still large (~10 sec per image when K=1500, Fig. 10) and seems can hardly be parallelized since the long-run sampling is serial. This might greatly limit the applicability of the proposed defense when timing constraint is critical.", "extra_scores": {}, "extra_text": {}}, {"review_id": "ekn8cMC46Q", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a Langevin dynamics based approach to defend naturally trained classifiers. To perform Langevin dynamics, an energy based model (EBM) is utilized, which is trained with realistic long run MCMC samples for effective purification and hence defense. \n\nThe proposal in this paper are two fold:\n1. Training of the EBM with long run dynamics along with using Adam optimizer in the initial stage and SGD for further training.   \n2. An Expectation over Transformation (EOT) defense based on Langevin dynamics.\n\n+ves: \n- The convergent learning of EBM was observed by Nijkamp et al. (2020) showing that the under this scenario short run and long-run MCMC samples have a similar appearance. \nSimilarly, Keskar and Soccer (2017) show that switching to SGD from Adam can provide improved convergence. \nThis paper utilizes the above mentioned ideas for training convergent EBM by using long run dynamics and by switching from Adam to SGD. \n\nAlthough, the proposal in this paper is interesting, I have some concerns whether the proposals are as innovative as claimed in the paper. \n\nConcerns: \n- The main innovative part of the paper is that it used long run Langevin dynamics using EBM for defending naturally trained classifiers. \nHowever Srinivasan et al. (2019) also utilized Langevin dynamics using EBM for the same task of defending against adversarial attacks, although the dynamics was performed with a modified distribution. A BPDA+EOT attack was also formulated to attack their own method in the above mentioned paper. \nThe authors do not make a clear distinction of how their method is different from the above mentioned paper. \n\n- Although Adam was found not useful and switching from Adam to SGD was found to be helpful, the images from SGD alone is not shown. \n\n- There is no clear definition of what is considered as a short-run and what is considered as a long-run dynamics. \n \n- The Figure 3 lacks a detailed explanation. Are original images used in both the left and right figures or are they adversarial samples. Why is H=5,100 considered and not 150 as used in the experiments?\n\n- Experimental settings in Table 1: JEM and IGEBM are also checked with long run dynamics. However, are the reported performance obtained by training them with long run dynamics? I suppose JEM and IGEBM can also suffer from unrealistic samples at long run due to non convergent learning as they are also EBM. In this case, the fair comparison should have been made by training both models with long run dynamics and switching from Adam to SGD. \n\n- The most interesting part of the paper, in my humble opinion, is the effect of long run dynamics vs short run dynamics for defense which is not well analyzed in the paper. \nFigure 8 in Appendix D shows a plot for increased accuracy as the number of steps increases. \nCan short run dynamics also be effective in defending by varying the hyper parameters (step size) of the Langevin dynamics after training EBM?\nAn in-depth understanding of this effect is missing. \n\nI will be open to changing my score based on the response from the authors.  ", "extra_scores": {}, "extra_text": {}}, {"review_id": "aylLeYDXEvk", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper discusses using a classifier + EBM  to construct an expectation of transformation (EOT) defense. The simple overview is that given an input image (potentially corrupted by an adversary), they use long-run Langevin dynamics seeded at the image to generate samples from the data distributions for the corresponding input image and then use samples to estimate EOT. \n1) A similar approach was used by Grathwohl et a. (2020) to improve the robustness of their model, so I was wondering if the authors can discuss their novelty here?\n\n2) The paper is not very clear in terms of the methods, descriptions, and explanation of the tables. For example, Section 4.1 refers to \"EBM defense\" but has not been defined separately, so I assumed it is the defined EOT defense in 3.1 that uses EBMs as T(x).  \n\n3) The classification accuracy of Grathwohl et al. (2020) against PGD in Table-1 is not consistent with what is reported in the paper. Grathwohl et al. (2020) reported that JEM-10 is a robust model against PGD attack (Figure-5 in their paper). Would you please discuss these inconsistencies?\n \n4) The authors claim that EOT defense is their contribution, is there any discussion on why you only tried EBMs as T? Why not VAEs? VAEs are also capable of generating samples given the corrupted images, and the sampling process is much faster than long-run Langevin dynamics!  Their output may not be as sharp as EBMs, but does it matter here?\n\n5) It would be nice to use your EBM (without the classifier) as a standalone model in Table-1 to see the comparison with IGEBM.\n\n6) What if you use WideResNet with IGEBM? This actually shows the importance of your long-run sampling.\n\n7) Could you present the effect of H for defense against PGD? I am interested to know how diverse \\hat{x}_h are when you run convergent Langevin dynamics seeded at x.  \n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "JzxmuVWuSdt", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a defense that uses MCMC sampling with an Energy-Based Model as a preprocessing before classification.\nThe stochastic transformations used in preprocessing are sampled from the Langevin updates, which has been shown to be useful for adversarial defense in prior work.\nSince there is no theoretical guarantees, I consider this method a heuristic approach with some empirical success.\nHowever, there are already many empirical pre-processing defenses, and most of them have been broken by newer attacks.\nIt is not convincing enough that the proposed defense will be different. For example, one can design better approximations for the transformation operations in BPDA, instead of using the identity function. Actually, when the transformation is larger, the identify function can be a very poor approximation.\nAlso, the proposed method is not as robust as a variant of adversarial training (Zhang et al. 2019) as shown in Table 2. The authors claim that the main benefit of their method is that it can protect a pre-existing natural model. However, from the run time analysis, it seems that it is even slower than adversarial training, so I am skeptical about the practical value of this method.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Although the technical novelty is not very high, the finding that long-run Langevin dynamics with convergently learned model provides comparable defense performance to adversarial training will give some impact to the community. 

Decision: Accept (Poster)