Online Adversarial Purification based on Self-supervised Learning
Authors: Changhao Shi
Chester Holtz
Gal Mishne
Venue: ICLR 2021 Poster (2021)
Keywords: Adversarial Robustness
Self-Supervised Learning

Abstract: Deep neural networks are known to be vulnerable to adversarial examples, where a perturbation in the input space leads to an amplified shift in the latent network representation. In this paper, we combine canonical supervised learning with self-supervised representation learning, and present Self-supervised Online Adversarial Purification (SOAP), a novel defense strategy that uses a self-supervised loss to purify adversarial examples at test-time. Our approach leverages the label-independent nature of self-supervised signals and counters the adversarial perturbation with respect to the self-supervised tasks. SOAP yields competitive robust accuracy against state-of-the-art adversarial training and purification methods, with considerably less training complexity. In addition, our approach is robust even when adversaries are given the knowledge of the purification defense strategy. To the best of our knowledge, our paper is the first that generalizes the idea of using self-supervised signals to perform online test-time purification.

Published as a conference paper at ICLR 2021
ONLINE ADVERSARIAL PURIFICATION BASED ON SELF-SUPERVISED LEARNING
Changhao Shi', Chester Holtz & Gal Mishne 12,,3 'Department of Electrical and Computer Engineering, DDpartment of Computer Science and Engineering, The Halıooğğu Data Science Institute
University of California, San Diego chholtz, gmishnee@ucsd.cco edu
ABSTRACT
Deep neural networks are known to be vulnerable to adversarial examples, where perturbation in the input space leads to amplified shift in the latent network representation. In this paper, we combine canonical supervised learning with self- supervised representation learning, and present Self-supervised Online Adversar- ial Purification (SOAP), a novel defense strategy that uses a self-supervised loss to purify adversarial examples at test-time. Our approach leverages the label- independent nature of self-supervised signals, and counters the adversarial per- turbation with respect to the self-supervised tasks. SOAP yields competitive ro- bust accuracy against state-of-thes thhe-ar adversarial training and purification meth- ods, with considerably less training complexity. addition, our approach is robust even when adversaries To the best of our knowledge, our paper is the first that generalizes the idea using self- superviedd signals to perform online test-time purification.
given knowledge of the purification defense strategy.
INTRODUCTION
Deep neural networks have achieved remarkable results in many machine learning applications. However, these networks are known to be vulnerable to adversarial attacks, i.e. strategies which aim to find adversarial examples that are close or even perceptually indistinguishable from their natural counterparts but easily mis-classified by the networks. This vulnerability raises theory-wist issues about the interpretability deep learning as well as application-wi wise issues when deploying neural networks in security -ennsiiiise applications.
Many strategies have been proposed to empower neural networks to defend against these adversaries. The current most widely used genre of defense strategies is adversarial training. Adversarial training is an on-the-fly data augmentation method that improves robustness by training the network not only with clean examples but adversarial ones as well. For example, |Madry al. (2017) propose projected gradient descent as a universal first-order attack and strengthen the network by presenting with such adversarial examples during training (c.g., adversarial training). However, this method is computationally expensive as finding these adversarial examples involves sample-wise gradient computation every epoch.
Self-supervised representation learning aims to learn meaningful representations of unlabeled data where the supervision comes from the data itself. While this seems orthogonal to the study of adversarial vulnerability, recent works use representation learning as a lens to understand as well as improve adversarial robustness (Hendrycks et al. 2019 Mao et al.) 2019 Chen al.) 2020a Naseer al. 2020). This recent line of research suggests that self-supervised learning, which often leads to a more informative and meaningful data representation, can benefit the robustness of deep networks.
In this paper, we study how self-supervised representation learning can improve adversarial robust- ness. We present Self-supervised Online Adversarial Purification (SOAP), a novel defense strategy that uses an auxiliary self- self-supervised loss purify adversarial examples at test-time, illustrated in Figure [T] During training, beside the classification task, we jointly train the network on a care- fully selected self-supervised task. The multi-task learning improves the robustness of the network

---

Published as a conference paper at ICLR 2021
2
8
Lett
2
A
(a) Joint training of classification and auxiliary.
(b) Test-time online purification
Figure I: An illustration of self-supervised online adversarial purification (SOAP). Left: joint train- ing of the classification and the auxiliary task; Right: input adversarial example is purified iteratively to counter the representational shift, then classified. Note that the encoder is shared by both classi- fication and purification.
and more importantly, enables us to counter the adversarial perturbation at test-time by leveraging the Snbel-independent nature of self-supervised signals. Experiments demonstrate that SOAP per- forms competitively various architectures across different datasets with only small computation overhead compared with vanilla training. Furthermore, we design a new attack strategy that targets both the classification and the auxiliary tasks, and show that method is robust to this adaptive adversary as well. Code is available https//github.cco//Mishee.
RELATED WORK
Adversarial training Adversarial training aims to improve robustness through data augmenta- tion, where the network is trained on adversarially perturbed examples instead of the clean original training samples (Goodfellow et al., 2014 Kurakin et al., 2016; Tramèr et al. 2017 Madry et al.] 2017 Kannan et |l. /Zhang et al. 2019). By solving a min-max problem, the network learns a smoother data manifold and decision boundary which improve robustness. However, the compu- tational cost of adversarial training high because strong adversarial examples are typically found in an iterative manner with heavy gradient calculation. Compared with adversarial training, our method avoids solving the complex inner-max problem and thus is significantly more efficient in training. Our method does increase test-time computation but it is practically negligible per sample.
Adversarial purification Another genre of robust learning focuses on shifting the adversarial examples back to the clean data representation, namely purification. Gu & Rigazio (20z exploited using a general DAE (Vincent et 2008) to remove adversarial noises; noises;|Megg Meng Chen (2017) train reformer network, which is collection of autoencoders, to move adversarial examples towards clean manifold; [iao al. 2018) train UNet that can denoise adversarial examples their clean counterparts; Samangouei et al. 2018) train GAN on clean examples and project the adversarial examples to the manifold
the generator; (onng et al. (2018) assume adversarial examples have lower probability and learn the image distribution with PixelCNN so that they can maximize the probability of given test example; Naseer et al. (020) train a conditional GAN by letting it play a min-max game with a critic network in order to differentiate between clean and adversarial examples. In contrast to above approaches, SOAP achieves better robust accuracy and does not require GAN which is hard and inefficient train. More importantly, approach exploits wider range of self-supervised signals for purification and conceptually can be applied to any format of data and not just images, given an appropriate self-supervised task.
Self-supervised learning Self-supervised learning aims to learn intermediate representations of unlabeled data that are useful for unknown downstream tasks. This done by solving self- supervised task, pretext task, where the supervision of the task comes from the data itself. Re- cently, a variety of self-supervised tasks have been proposed on images, including data reconstruc- tion (Vincent al. 2008 Rifai al. 2011). relative positioning of patches (Doersch et al. 2015
Noroozi & Favaro 2016), colorization (Zhang et al., 2016), transformation prediction (Dosovitskiy
al.
2014 Gidaris et al.) 2018 combination of tasks (Doersch & Zisserman, 2017).
More recently, studies have shown how self-supervised learning can improve adversarial robustness.
Mao al. (2099) find that adversarial attacks fool the networks by shifting latent representation to
2

---

Published as a conference paper at ICLR 2021
false class. Hendrrycks et al. (2019) observe that PGD adversarial training along with auxiliary rotation prediction task improves robustness, while Naseer et al. (2020) use feature distortion as a self-supervised signal to find transferable attacks that generalize across different architectures and tasks. Chen et al. (2020a) combine adversarial training and self-supervised pre-training to boost fine-tuned robustness. These methods typically combine self-supervised learning with adversarial training, thus the computational cost is still high. In contrast, our approach achieves robust accuracy by test-time purification which uses a variety of self-supervised signals as auxiliary objectives.
SELF-SUPERVISED PURIFICATION
3.1 PROBLEM FORMULATION
As aforementioned, Mao et al. 2019) observe that adversaries shift clean representations towards false classes to diminish robust accuracy. The small error input space, carefully chosen by ad- versaries, gets amplified through the network, and finally leads to wrong classification. A natural way to solve this is to perturb adversarial examples so as to shift their representation back to the true classes, i.e. purification. In this paper we only consider classification as our main task, but our approach should be easily generalized other tasks as well.
Consider an encoder = {a; ). classifier g(2; fass) on top of the representation 2, and the network g f a composition of the encoder and the classifier. We formulate the purification problem as follows: for adversarial example (Iadv, 3) and its clean counterpart (ar, y) (unknown to the network), purification strategy π aims to find Tpty = πTxadv) that is as close to the clean example possible: pply + However, this problem is underdetermined as different clean examples can share the same adversarial counterpart, i.e. there might be multiple or even infinite solutions for Tpyy. Thus, we consider the relaxation
min Lets ((g (1
s.t. || - Tadv S Cadv
Tply = -- -T )
i.e we accept Tpty as long as Las sufficiently small and the perturbation bounded. Here Las is the cross entropy loss for classification and Cade is the budget of adversarial perturbation. However, this problem is still vnsolvable since neither the true label y nor the budget Eady is available at test- time. We need an alternative approach that can lead to similar optimum.
3.2 SELF-SUPERVISE ONLINE PURIFICATION
Let h(2; flnx) be an auxiliary device that shares the same representation 2 with g(;; Octs). and Laux be the auxiliary self supervied objective. The intuition behind SOAP is that the shift in representation z that hinders classification will hinder the auxiliary self-supervised task as well. In other words, large Laux often implies large Les. Therefore, we propose use Laux an alternative Les in Eq. (T. Then we can purify the adversarial examples using the auxiliary self-supervised signals, since the purified examples which perform better on the auxiliary task (small Caux) should perform better on classification as well (small L.ss).
During training, we jointly minimize the classification loss and self-supervised auxiliary loss min Las ((g (x; O.,) v) + aCax ((h o fx; 0 ax,x))
(2)
where a is a trade-off parameter between the two tasks. At test-time, given fixed network parameters 0, we the indl-lindeeedentt auxiliary objective to perform gradient descent in the inpul space. The purification objective is
min Caux ((h o f)(pt))) s.t. ||<Epty - Fadv|l S Epty Tpty
T ((adv),
where (ply is the budget of purification. This is legitimate at test-time because unlike Eq. 1 the supervision or the purification signal comes from data itself. Also, compared with vanilla training the only training increment of SOAP is additional self-supervised regularization term. Thus, the computational complexity is largely reduced compared with adversarial training methods. In Sec. 4 will show that adversarial examples do perform worse auxiliary tasks and the gradient of the auxiliary loss provides useful information on improving robustness. Note that eav is replaced with Eply in Eq. and we will discuss how to find appropriate éply in the next section.
3

---

Published as a conference paper at ICLR 2021
Algorithm PGD attack
Algorithm 2 Multi-step purification
Input: æ: a test example;
Input: x: a test example;
T: the number of attack steps
T: the number of purification steps
Output: Tady: the adversarial example
Output: 2pty: the purified example
1: 8
0
1: 8
0
2: for
= 1, 1,2,.... T,, do
2: for t = 1.2....,
T do
3:
La f) 8; denc: ()
{ Laux ((h o f) + ó; oene m
4:
+ 0+ sign v.
4:
ô + 0 yiign v,
5:
ô d- min(max(( Exdv) , Endv)
Epyy ) Eply) 5 + min(max( 6, 0), 1)
5t min(max
6:
8 f- min(max( (z + 8, 0), 1)
6:
7:
end for
7: end for
8:
Tadv x 8
8: Tipfy f- +
3.3 ONLINE PURIFICATION
Inspired by the PGD (Madry et al. 2017 attack (see Alg. D. we propose multi-step purifier (see Alg. (2) which can be seen as its inverse. In contrast PGD attack, which performs projected gradient ascent on the input in order to maximize the cross entropy loss Leks. the purifier performs projected gradient descent on the input in order minimize the auxiliary loss L The purifier achieves this goal by perturbing the adversarial examples, i.e. (IIadv Iadv + ō, while keeping the perturbation under a budget, i.e. ||ô]| I Epfy. Note that is also plausible to use optimization- based algorithms in analogue some lg adversaries such CW (Carlini & Wagner 2017). however this would require more steps of gradient descent at test-time.
Taking the bound into account, the final objective of the purifier is minimize the following
min Laxx ((h (I + 8)) s.t. ||o|| Epfy, Tadv 8 E [0.
(4)
For multi-step purifier with step size y, at each step we calculate
5, 8 sign V Laux ((h ( bc-))))
(5)
For step size y Epty and number of steps T 1, the multi-step purifier becomes single-step purifier. This is analogous to PGD degrading to FGSM (Goodfellow et al., 2014) when the step size of the adversary y Cadv and the number of projected gradient ascent steps T in Alg. 1
remaining question is how to set Cpty when Cadv is unknown. Epty is too small compared to the attack, it will not be sufficient neutralize the adversarial perturbations. In the absence of knowledge of the attack eadv. we can use the auxiliary loss as a proxy to set the appropriate épty. In Figure
3 we plot the average auxiliary loss (green plot) of the purified examples for range of éply values. The "elbows" of the auxiliary loss curves almost identify the unknown Eadv in every case with slight over-estimation. This suggests that the value for which the auxiliary loss approximately stops decreasing is a good estimate of <adv- Empirically, we find that using slightly over-estimated Cply benefits the accuracy after purification, similar the claim by Song et al. (2018). This because our network is trained with noisy examples and thus can handle the new noise introduced by purification. At test-time, we use the auxiliary loss to set Epfy in an online manner, by trying a range of values for Epty and selecting the smallest one which minimizes the auxiliary loss for each individual example. In the experiment section we refer to the output of this selection procedure as émin-aux- We also empirically find for each sample the Epty that results in the best adversarial accuracy, denoted coracle in the experiment section. This is an upper-bound on the performance SOAP can achieve.
3.4 SELF-SUPERVISED SIGNALS
Theoretically, any existing sulf-supervised objective can used for purification. However, due to the nature of purification and also for the sake of efficiency, not every self-supervised task is suitable. A suitable auxiliary task should be sensitive to the representation shift caused by adversarial perturbation, differentiable with respect to the entire input, e.g. every pixel for image, and also efficient in both train and test-time. In addition, note that certain tasks are naturally incompatible with certain datasets. For example, a rotation-based self-supervised task cannot work rotation- invariant dataset. In this paper, we exploit three types of self-supervised signals: data reconstruction, rotation prediction and label consistency.
4

---

Published as a conference paper at ICLR 2021
Clean
0 2 4 5
6
7
8 9
0
2
3 4 5
6 7
8 9
Puriied Adv
0 3
4 5
6
8
7
0
0
3 9 5
6
3
7 9
0 3 4 S
6
8 7
0
(
2
3 4 S 6
7
3
Input Digits
Reconstructed Digits
Figure 2: Input digits of the encoder (left) and output digits of the decoder (right). From top to
bottom are the clean digits, adversarially perturbed digits and purified digits, respectively. Red rectangles: the adversary fools the model to incorrectly classify the perturbed digit 8 as a 3 and the purification corrects the perception back to an 8.
Data reconstruction Data reconstruction (DR), including both deterministic data compression and probabilistic generative modeling, is probably one of the most natural forms of self-supervision. The latent representation, usually lying much lower dimensional space than the input space, is required to be comprehensive enough for the decoder to reconstruct the input data.
To perform data reconstruction, to reconstruct the input from the latent representation 2. In order to better learn the underlying data manifold, as well to increase robustness, the input corrupted with additive Gaussian noise n) (and clipped) before fed into the encoder f. The auxiliary loss is the Eg distance between examples and their noisy reconstruction via the autoencoder h f:
decoder network as the auxiliary device h and require it
Laux = || (h o f) + 7)1l2
In Figure 2 we present the outputs of an autoencoder trained using Eq. 4 for clean, adversarial and purified inputs. The purification shifts the representation of the adversarial examples closer to their original class (for example, 2 4,& 8 and 9). Note that SOAP does not use the output of the autoencoder as a defense, but rather uses the autoencoder loss to purify the input. We plot the autoencdoer output here as consider it providing insight to how the trained model 'sees' these samples.
Rotation prediction Rotation prediction (RP), as an image self-supervised task, was proposed by Gidaris et al. (2018). The authors rotate the original images dataset by certain degree, then use a simple classifier to predict the degree of rotation using high-level representation by convolutional neural network. The rationale that the learned representation has be semantically meaningful for the classifier to predict the rotation successfully.
Following Gidaris et al. 2018). make four copies of the image and rotate each of them by one of four degrees: Ω = 10 90" 180 270%). The auxiliary task is a 4-way classification using representation fzx, for which we use a simple linear classifier as the auxiliary device h. The auxiliary loss is the summation of 4-way classification cross entropy of each rotated copy
Caux Σ log(h(((xw)).
where Tu is a rotated input, and b(-) is the predictive probability of it being rotated by w. While the standard rotation prediction task works well for training, we found that it tends to under-estimate Cply at test-time. Therefore, for purification we replace the cross entropy classification loss by the mean square error between predictive distributions and one-hot targets. This increases the difficulty of the rotation prediction task and leads to better robust accuracy.
Label consistency The rationale of label consistency (LC) is that different data augmentations of the same sample should get consistent prediction from the network. This exact or similar concept is widely used in semi-supervised learning (Sajjadi et al.) 2016 Laine & Aila 2016), and also successfully applied in self-supervised contrastive learning He et al. 2020 Chen et al. 20205).
We adopt label consistency perform purification. The auxiliary task here is to minimize the lg distance between two augmentations a (α) and (2 of a given image x, in the logit space given by g()). The auxiliary device of LC is the exact classifier, i.e. h = I. and the auxiliary loss
Laux
[1[g o f)(a (g (
(8)
EXPERIMENTS
We evaluate SOAP on the MNIST, CIFAR10 and CIFAR100 datasets following Madry al. 2017
5

---

Published as a conference paper at ICLR 2021
MNST
CIFAR10
CFAR10
0.45
0.30
0.45
(100 Square eree-eme
0.25
pafeed
#7venard pas
2.04
0.03
0.23
00
0.6
01
10
23
20
(a) SOAP-DR
(b) SOAP-RP
(c) SOAP-LC
Figure 3: Auxiliary loss vs. Epty. SOAP (green plot) reduces the high adversarial auxiliary loss (orange plot) to the low clean level (blue plot). The vertical dashed line is the value of Eadv- The trained models are FCN and ResNet-18 for MNIST and CIFAR1O, 10, respectively, with a PGD attack.
MNIST (LeCun et al., 1998) For MNIST, we evaluate our method on a fully-connected network (FCN) and convolutional neural network (CNN). For the auxiliary task, we evaluate the efficacy of data reconstruction. For the FCN g(.) is linear classifier and h(-) is fully-connected decoder; for the CNN g(-) is 2-layer MLP and h()) convolutional decoder. The output of the decoder is squashed into the range of [0, 1] by sigmoid function. During training, the input digits are corrupted by an additive Gaussian noise (μ 0, a 0.5). test-time, Laux of the reconstruction is computed without input corruption. SOAP runs T = 5 iterations with step size 0.1.
CIFAR1O & CIFAR100 Kriizheskky & Hinton 2009) For CIFARIO and CIFAR100, we eval- uate our method on ResNet-18 (He et al., 2 and a 10-widen Wide-ResNet-28 (Zagoruyko & Komodakis 2016). For the auxiliary task, we evaluate rotation prediction and label consis- tency. To train on rotation prediction, each rotated copy corrupted by an additive Gaussian noise (u = 0, 0.1), encoded by f(-), and classified by linear classifier g() for object recognition and by an auxiliary linear classifier h(-) for degree prediction. This results in a batch size times larger than the original. At test-time, similar to DA, compute Laux on clean input images.
To train on label consistency, we augment the input images twice using a composition of random flipping. random cropping and additive Gaussian corruption (μ = 0, 0.1). Both of these two augmentations are used to train the classifier, therefore the batch size is twice as large as the original. At test-time, we use the input image as one copy and flip-crop the image to get another copy. Using the input image ensures that every pixel in the image is purified, and using definite flipping and cropping ensures there is enough difference between the input image and its augmentation. For both rotation prediction and label consistency, SOAP runs T = 5 iterations with step size 4/255.
Note that we did not apply all auxiliary tasks all datasets due the compatibility issue mentioned in Sec. 3.4 DR is not suitable for CIFAR as reconstruction via an autoencdoer is typically challeng- ing on more realistic image datasets. RP is naturally incompatible with MNIST because the digits O, 1, and 8 are self-symmetric; and digits 6 and 9 are interchangeable with 180 degree rotation. Sim- ilarly, LC is also not appropriate for MNIST because common data augmentations such as flipping and cropping are less meaningful on digits.
4.1 WHITE-BOR BOX ATTACKS
In Tables I 3 we compare SOAP against widely-used adversarial training (Goodfellow et al. 2014
Madry et al. (017)) and purification methods (Samangouei al. 2018 Song et 2018 on a variety
of &2 and for bounded attacks: FGSM, PGD, CW, and DeepFool (Moosavi-Deziooll et al. 2016). For MNIST. both FGSM and PGD are for bounded with Eadv 0.3, and the PGD runs 40 iterations with step size of 0.01; CW and DeepFool are l, bounded with eadv 4. For CIFAR 10, FGSM and PGD are foc. bounded with Eadv = 8/255, and PGD runs iterations with a step size of 2/255; CW and DeepFool are lg bounded with éadv 2. For CW and DeepFool which are optimization-based. resulted attacks that exceed the bound are projected to the e-ball. We mark the best performance for each attack by an underlined and bold value and the second best by bold value. We do not mark out the oracle accuracy but it does serves as an empirical upper bound of purification.
For MNIST, SOAP-DR has great advantages over FGSM and PGD adversarial training all attacks when the model has small capacity (FCN). This is because adversarial training typically requires a large parameter set learn complex decision boundary while method does not have this

---

Published as a conference paper at ICLR 2021
constraint. When using larger CNN. SOAP outperforms FGSM adversarial training and comes close to Defense-Ger GAN and PGD adversarial training on & attacks. SOAP also achieves better clean accuracy compared with all other methods.
Note that FGSM AT achieves better accuracy under FGSM attacks than when there is no attack for the large capacity networks. This is due to the label leaking effect (Kurakin al. (016): the model learns to classify examples from their perturbations rather than the examples themselves.
Table 1: MNIST Results
FCN
CNN
Method
No Atk
FGSM
PGD
CW
DF
No Atk
FGSM
PGD
CW
DF
No Def
98.10
16.87
0.49
0.01
1.40
99.15
1.49
0.00
0.00
0.69
FGSM AT
79.76
80.57
2.95
6.22
17.24
98.78
99.50
33.70
0.02
6,16
PGD
76.82
60.70
57.07
31.68
13.82
98.97
96.38
93.22
90.31
75.55
Defense-GAN
95.84
79.30
84.10
95.07
95.29
95.92
90.30
91.93
95,82
95.68
SOAP-DR
Cjdy 0
97.57
29.15
0.58
0.25
2.32
99.04
65.35
27.54
0.35
0.69
Emin-aux
97.56
66.85
61.88
86.81
87.02
98, 98.94
87.78
84.92
74.61
81.27
Eply coracle
98.93
69.21
64.76
97.88
97.97
99.42
89.40
86.62
98.44
98.47
Table 2: CIFAR- 10 results
1esNet-11
Wide-ResNet-28
Method
No Atk
FGSM
PGD
CW
DF
No Atk
FGSM
PGD
CW
DF
No Def
90.54
15.42
0.00
0.00
6.26
95.13
14.82
0.00
0.00
3.28
FGSM AT
72.73
44.16
37.40
2.69
24.58
72.20
91.63
0.01
0.00
14.41
PGD AT
74.23
47.43
42.11
3.14
25.84
85.92
51.58
41.50
2.06
24.08
Pixel-Defend
79.00
39.85
29.89
76.47
76.89
83.68
41.37
39.00
79.30
79.61
SOAP-RP RP
Gdy
73.64
5.77
0.47
0.00
13.65
88.68
30.21
8.52
0.08
10.67
Fply émin-aux
71.97
35.80
38.53
68.22
68.44
90.94
51.11
51.90
83.03
82.50
Eniy curacle* *
87.57
37.60
39.40
79.80
84.34
95.55
$2.69
52.61
86.99
90.49
SOAP-LC
Epdy 0
86.36
22.81
0.15
0.00
8.52
93.40
59.23
3.55
0.01
46.98
Epdy sux
84.07
51.02
51.42
73.95
74.79
91.89
64.83
53,58
80.33
60.56
Coracle
94.06
59.45
62.29
86.94
88.88
96.93
71.85
63.10
88.96
73.66
Table CIFAR-100 results
Net-
Wide-ResNet-228
Method
Atk
FGSM
PGD
DF
Atk
FGSM
PGD
No Def
65.56
3.81
0.01
0.00
12.30
78.16
13.76
0.06
0.01
9.05
FGSM AT
44.35
20.30
17.41
4.23
18.15
46.45
88.24
0.15
0.00
13.40
PGD AT
42.15
21.92
20.04
3.57
17.90
62.71
28.15
21.34
0.65
16.57
SOAP-RP
Gpyy
40.47
2.53
0.45
0.03
11.89
60.33
13.30
4.65
0.09
12.19
Epdy
35.21
11.65
11.73
32.97
33.51
60.80
22.25
22.00
54.11
54.70
Gyty Goracle
45.57
12.44
12.04
41.13
46.51
72.03
24.42
24.19
63.04
67.86
SOAP-LC
Fply 0
57.86
6.11
0.01
12.72
74.04
16.46
0.49
9.65
Fply Emin-aux
52.91
22.93
27.55
50.26
50.57
61.01
31.40
37.53
56.09
53.79
6ply Coracle* *
69.99
27.52
31.82
62.87
68.65
82.74
37.56
47.07
71.19
73.39
For CIFARIO, on ResNet-18 SOAP-RP beats Pixel-Defend on all attacks except for FGSM and beats PGD adversarial training attacks; Wide-ResNet-28 it performs superiorly equivalently

---

Published as a conference paper at ICLR 2021
0: airplane
bird
dog
airplane
bird
deer
automobile
ship
dog
freg
ship
truck
(a) SOAP-RP
D: airplane
1: automobile
2: bird
3: cat
deer
5: dog
6: frog
7: horse
truck
adv
truck
deer
dog
airpiane
cat
deer
cat
automobile
airpiane
airplane
automobile
bird
dog
airplane
truck
(b) SOAP-LC
Figure 4: Adversarial and purified CIFAR 10 examples by SOAP with Wide-ResNet-28 under PGD attacks. True classes are shown on the top of each column and the model predictions are shown under each image.
against other methods on all attacks. SOAP-LC achieves superior accuracy compared with other methods, where the capacity is either small or large. Note that we choose Pixel-Defend as our purification baseline since Refense-GAN does not work CIFAR10. Specifically, achieves over 50% accuracy under strong PGD attack, which is 10% higher than PGD adversarial training. SOAP also exhibits great advantages over adversarial training on the & attacks. Also, compared with vanilla training ('No Def') the multi-task training of SOAP improves robustness without purification (Epfy 0), which similar on MNIST. Examples are shown Figure 4
method
For CIFARI00, SOAP shows similar advantages over other methods. SOAP-RP beats PGD ad- versarial training on PGD attacks when using large ReseeRes model and on & attacks in all cases; SOAP-LC again achieves superior accuracy compared with all other methods, where the capacity is either small or large.
Our results demonstrate that SOAP is effective under both fox and fz bounded attacks, opposed to adversarial training which only defends effectively against Eg attacks for MNIST with CNN. This implies that while the formulation of the purification in Eq. (4) mirrors an Ex bounded attack, our defense is not restricted to this specific type of attack, and the bound in Eq. serves merely as constraint the purification perturbation rather than a-prior knowledge of the attack.
Auxiliary-aware attacks Previously, we focus on standard adversaries which only rely on the classification objectives. natural question is: can an adversary easily find stronger attack given the knowledge of our purification defense? In this section, we introduce a more 'complete' white- box adversary which aware of the purification method, and show that is not straightforward to attack SOAP even with the knowledge of the auxiliary task used for purification.
In contrast to canonical adversaries, here we consider adversaries that jointly optimize the cross entropy loss and the auxiliary loss with respect to the input. As SOAP aims to minimize the auxiliary loss, the auxiliary-aware adversaries maximize auxiliary loss at the same time. The intuition behind this is that the auxiliary-aware adversaries to find the auxiliary task "on- manifold" (Stutz et al. 2019) examples that can fool the classifier. The auxiliary- aware adversaries perform gradient ascent the following combined objective
cross entropy loss while also minimizing
max (Lar(f(x) ys Oene, Oass) 3Laxx (g(x; (;; Oenc; (anx)))
where 3 is a trade-off parameter between the cross entropy loss and the auxiliary loss. An auxiliary- aware adversary degrades canonical when / in the combined objective.
8

---

Published as a conference paper at ICLR 2021
MNISS FCN
CIFARIO ResNer18
CIFARIO Resfett18
13
before porfication purffcater
before pertication
before cutilcatoon
aher purticator
purfication
0.5
0.4
0.4
0.4
0.-
0.2
0.4
550
223
220
1000
150 1000
-1.
0.3
(a) SOAP-DR
(b) SOAP-RP
(c) SOAP-LC
Figure 5: Purification against auxiliary- awwae PGD attacks. Plots are classification accuracy before (blue) and after (orange) purification.
As shown in Figure 5 adversary cannot benefit from the knowledge of the defense in a straight- forward way. When the trade-off parameter is negative (i.e. the adversary is attacking the auxiliary device as well). the attacks weakened (blue plot) and purification based on all three auxiliaries achieves better robust accuracy (orange plot) as the amplitude of 3 increases. When 3 is positive, the accuracy of SOAP using data reconstruction and label consistency increases with B. The reason for this that the auxiliary component the adapted attacks obfuscates the cross entropy gradient, and thus weakens canonical attacks. The accuracy of rotation prediction stays stable varies, i.e. it is more sensitive to this kind of attack compared to the other tasks.
4.2 BLACK-BOL ATTACKS
Table compares SOAP-DR with adversarial training against FGSM black-box attacks (Papernot ct al. 2017). Following their approach, we let white-box adversaries, e.g. FGSM, attack substitute model, with potentially different architecture, to generate the black-box adversarial examples for the target model. The substitute model is trained on limited set of 1 150 test images unseen by the target model. These images are further labeled by the target model and augmented using a Jacobian- method. SOAP significantly out-performs adversarial training on FCN; for CNN it out-performs FGSM adversarial training and comes close to PGD adversarial training.
Table 4: MNIST Black-box Results
Target
FCN
CNN
Substitute
No Atk
FCN
CNN
No Atk
FCN
CNN
No Def
98.10
25.45
39.10
99.15
49.49
49.25
FGSM AT
79.76
40.88
58.74
98.78
93.62
96.52
PGD AT
76.82
62.87
69.07
98.97
97.79
98.09
-D
Gply
97.57
78.52
92.72
99.04
95.25
97.43
Epty
Eminoux
97.56
90.35
94.51
98.94
96.02
97.80
Cpty 98.93
= Fomcle*
94.34
97.33
99.42
98.12
98.81
5 CONCLUSION
In this paper, we introduced SOAP: online defense against adversarial attacks. During training, the model Iearns a clean data manifold through joint optimization of the eross entropy loss for classification and label-independent auxil- iary loss for purification. At test-time, a purifier counters adversarial perturbation through projected gradient descent of the auxiliary loss with respect to the input. SOAP is consistently competitive across different network capacities as well as different datasets. We also show that even with knowl- edge of the self supervised task, adversaries do not gain an advantage over SOAP. While in this paper only explore how SOAP performs images, our purification approach can be extended to any data format with suitable self-supervised signals. We hope this paper can inspire future exploration on broader range of self-supervised signals for adversarial purification.
using self-supervision perform test-time purification as an
9

---

Published as a conference paper at ICLR 2021
REFERENCES
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP). pp. 39-57. IEEE, 2017.
Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From self-supervised pre-training to fine-tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 699-708, 2020a.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.055700 2020b.
Carl Doersch and Andrew Zisserman. Multi-task supervseee visual learning. In Proceedings of
the IEEE International Conference on Computer Vision, pp. 2051-2060, 2017.
Carl Doersch, Abhinav Gupta, and Alexei Efros. Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE international Conference on Computer Vision, pp- 1422-1430, 2015.
Alexey Dosovitskiy, Jost Tobias Springenberg, Martin Riedmiller, and Thomas Brox. Discrimina- tive unsupervised feature learning with convolutional neural networks. In Advances in neural information processing systems, pp. 766-774, 2014.
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv: 1803.0778, 2018.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Shixiang Gu and Luca Rigazio. Towards deep neural network architectures robust to adversarial examples. arXiv preprint arXiv:1412.5068, 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp- 770-778, 2016.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xic, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 9729-9738, 2020.
Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using self-supervised learning can improve model robustness and uncertainty. NeurIPS, 2019.
Uiwon Hwang, Jaewoo Park, Hyemi Jang, Sungroh Yoon, and Nam Cho. Puvae: variational autoencoder purify adversarial examples. IEEE Access, 7.126582-1265999 2019.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. arXiv preprint 1rX1v:1803.06333, 2018
A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Master's thesis, Department of Computer Science, University of Toronto, 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint arXiv: 1.01236, 2016.
Samuli Laine and Timo Aila. Temporal ensembling for superviee learning. arXiv preprint 1rX.v:1610.02222. 2016.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-bacs based learning applied to document recognition. Proceedings of the IEFF, 86(11):227--2322 1998.
Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using level representation guided denoiser. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 1778-1787, 2018.
10

---

Published as a conference paper at ICLR 2021
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 2017.
Chengzhi Mao, Ziyuan Zhong, Junfeng Yang, Carl Vondrick, and Baishakhi Ray. Metric learning for adversarial robustness. In Advances in Neural Information Processing Systems, pp. 480-491, 2019.
Dongyu Meng and Hao Chen. Magnet: a two-pronged defense against adversarial examples. In Proceedings of the 2017 ACM SIGSAC Conference Computer and Communications Security, pp. 135-147, 2017.
Seyed-Mohsen Moosavi-Dezfool Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference Com- puler Vision and Pattern Recognition, pp. 2574-2582, 2016.
Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli, A self- supervised approach for adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 262-2711 201, 2020.
M. Noroozi and P. Favaro. Unsupervised learning of visual representations by solving jigsaw puz- zles. In ECCV, 2016.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pp. 506-519, 2017.
Salah Rifai, Pascal Vincent, Xavier Muller, Xavier Glorot, and Yoshua Bengio. Contractive auto- encoders: Explicit invariance during feature extraction. In ICMI,, 2011.
Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. Regularization with stochastic transfor- mations and perturbations for deep semi-supervised learning. In Advances in neural information processing systems, pp. 1163- 1171, 2016.
Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-gan: Protecting classifiers against adversarial attacks using generative models. arXiv preprint arXiv: 1805 0666006605, 2018
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. In Interna- tional Conference on Learning Representations, 2018. URL httpps://openrevien net/
David Stutz, Matthias Hein, and Bernt Schiele. Disentangling adversarial robustness and general- ization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp- 6976-6987, 2019.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Bonch, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv: 17Xvv:705.007244 2017.
Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Picrre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international Conference on Machine Learning, pp. 1096-1103, 2008.
Sergey Zagoruyko and Nikos Komodakis.
Wide residual networks.
arXiv preprint
1rXiv:1605.07146. 2016.
Hongyang Zhang. Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent Ghaoui, and Michael I Jordan. Theoretically principled trade-off between robustness and accuracy. arXiv preprint arXiv: /901900813..... 2019.
Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. In European Con- ference Computer Vision, pp. 649-666 Springer, 2016.
11

---

Published as a conference paper at ICLR 2021
A APPENDIX
A.1 ILLUSTRATION OF SELF-SUPERVISED TASKS
For those readers who are not familiar with self-supervised representation learning, we detailed the self-supervised tasks here. As data reconstruction is straight-forward to understand, we simply illustrate rotation prediction and label consistency in Figure 6 As shown on the left-hand side, to perform rotation prediction, four degrees. We then use the auxiliary classifier to predict the rotation of each copy. For the label consistency auxiliary task the right-hand side, we duplicate input image to 2 copies and apply separate data augmentation to each of them. The left copy is augmented with random cropping and the right copy is augmented with random cropping well as horizontal flipping. We require the predictive distributions of these augmented copies to be close.
duplicate a input image to copies and rotate them by of the
1=
Pb
all
nehelts
Lalkln
90"
180
270"
Mean Square Error
(a) Rotation prediction
(b) Label consistency
Figure 6: An illustration of auxiliary self-supervised tasks.
A.2 HYPER-PARE PARAMETERS OF THE PURIFIER
Beyond Epty. the two additional hyper-parann parammerreesers of SOAP are the step size of the purifier 7 and the number of iterations performed by the purifier T. The selection of these dyper-paramedd is important for the efficacy purification. step size that is too small or a number of iterations that is too large can cause the purifier to get stuck in local minimum neighboring the perturbed example. This is confirmed by our empirical finding that using relatively large step size for a small number of iterations is better than using relatively small step size for large number of iterations. Although a large step size makes it hard to get the cleanest purified examples, this drawback compensated by adding noise in training. Training the model on corrupted examples makes the model robust to the residual noise left by purification.
A.3 TRAINING DETAILS
Table 5: Basic modules & specifics
Module
Specifics
Conv(m, x k s)
2-D convolutional layer with m feature maps, & x k kernel size, and stride on both directions 2-D maxpooling layer with stride on both directions Fully-connes layer with m outputs Rectified linear unit activation I-D or 2-D batch normalization Dropout layer with probability p Residual addition that bypasses the basic block
Maxpool(s)
FC(m)
ReLU
BN Dropout(p) ShortCut
12

---

Published as a conference paper at ICLR 2021
Table 6: ResNet basic blocks
Block(m, k x k,
Wide-Block(m, & ki 8, p)
Conv(m, k k, 8) BN
Conv(m x 10, k. 1)
Dropout(p) BN
ReLU
Conv(m, k k, 1)
ReLU
BN
Conv(m x 10, k. s)
ShortCut ReLU
ShortCut BN ReLU
Table 7: Architectures
FCN
CNN
ResNet-18
Rede-ResNet-28
FC(256)
Conv(32, 3 x 3, 2)
Conv(16, 3 x 3, 1) BN
Conv(16. 3 x 3, 1) BN
ReLU
ReL.U
Dropout(0.55
Conv(64, 3 3, 2)
ReLU
ReLU
FC(128)
ReLU
Block(16, 3, 1) x
Wide-Block(16,, 3 3, 0.3)
ReLU
BN FC(128)
Block(32, 3 3,
3
Dropout(0.55
Block(32, x
Wide-Block(32, 3 x,
FC(10)
ReLU
Block(64, 3 x 3,
Wide-Block(64, 3 3, 0.3) Wide-Block(64, 3 x 3, 1, 0.3)
BN
(64ck(64, x 3,
FC(10)
FC(10)
FC(10)
The architectures of the networks and training details are described as followed. Table Table desscribes the basic modules and their specifics, and Table ( describes the low-level building blocks of residual networks. Full architectures of the networks are listed in Table
2 hidden layer FCN and CNN which has the same architecture in Madry et al. (2017). FCN is trained for 100 epochs with an initial learning rate of 0.01 and CNN for 200 epochs with initial learning rate of 0.1 using SGD. The learning rate is decreased 10 times at halfway in both cases. The batch size is 128. In both FGSM and PGD adversarial training, the adversaries are loc bounded with Eddv 0.3. For PGD adversarial training, the adversary runs 40 steps of projected gradient descent with step size of 0.01. To train SOAP, the trade-off parameter Eq. (1) 100.
For MNIST, evaluate
For CIFAR10, we evaluate our method on a regular residual network ResNet- 18 and 10-widen residual network
Both networks are trained for 200 epochs using a SGD op- timizer. The initial learning rate is 0.1, which is decreased by factor of 0.1 at the 100 and 150 epochs. We use random crop and random horizontal flipping for data augmentation on CIFAR10 10. Eadv 8/255 for both FGSM and PGD adversarial training. For PGD adversarial training, the ad- versary runs steps of projected gradient descent with step size of 2/ 255.. The trade-off parameter of SOAP for rotation prediction and label consisteney are 0.5 and 1 respectively.
While we implement adversarial training and SOAP ourselves, we use authors' implementation for both Defense-Gar GAN and Pixel-Defend. Notice that our white-box Defense GAN accuracy is lower than the accuracy reported in (Samangouei et al.) 2018). Part of the reason is the difference in architecture and training scheme, but still not able replicate their accuracy using the exact same architecture following their instructions. Nonetheless, our results are close to (Hwang et al. 2019) where the authors also reported lower accuracy.
A.4 TRAINING EFFICIENCY
The comparison of training efficiency between SOAP and other methods is shown in Figure 7 To measure the training complexity, we run each training method for 30 epochs on a single Nvidia Quadro RTX 8000 GPU, and report the average epoch time consumption. When the network pacity is small, the training complexity of SOAP close to FGSM adversarial training and much lower than PGD adversarial training. When the network capacity is large, the training complexity
13

---

Published as a conference paper at ICLR 2021
PCN CNN
Restie-11 18
00
RosNoc-11
800
Time
400
20
200
NO DXP
MGSM
KD
SOAPDR
No Def
FGGM
PGD
SOAMNR
SONMLE
Defense
Defonse
(a) Data Reconstruction
(b) Rotation prediction and Label consistency
Figure 7: Comparison of training efficiency between SOAP, vanilla training ('No Def*) and adver- sarial training (FGSM and PGD). The y-axis is the average time consumption of 30 training epochs.
of SOAP higher than FGSM adversarial training but still significantly lower than PGD adversarial training.
Note that it is hard to compare with other purification methods because they are typically trained in 2 stages, the training of the classifier and the training of another purifier such GAN. While the training of those purifiers typically difficult and intense, SOAP does not suffer from this limitation as the encoder shared between the main classification network and the purifier. Therefore it is reasonable to claim that SOAP is more efficient than other purification methods.
A.5 PURIFIED EXAMPLES
We have shown some PGD examples adversarial images purified images by SOAP in Figure In Figures 10 we present some examples of every attack. The adversary is shown on the top of each column. The network prediction shown at the bottom of each example.
df
clean
8
8
8
8
8
8
8
8
8
a0v
8
8
3
8
8
8
8
8
8
3
(a) FCN
(b) CNN
Figure 8: MNIST examples with data reconstruction.
clean
automobile
autamobile
automobile
ship
automobilc
Sutamabile
automobils
ship
ship
(a) ResNet-18
(b) Wide-ResNet-28
Figure 9; CIFARI0 examples with rotation prediction.
14

---

Published as a conference paper at ICLR 2021
dlen
df
clean
fosm
automobile
automobile
automebile
automobile
ship
automobile
automobile
automobile
ship
snip
(a) ResNet-18
(b) Wide-Rees ResNet-28
Figure 10: CIFARIO examples with label consistency.
A.6 SUCCESS RATE OF & ATTACKS
To provide more details about the robustness under lg attacks, Tables Tab We report the success rate of generating le attacks. For the MNIST dataset, generating an &g attack is considered success the l2 norm of the perturbation smaller than 4; for the CIFARIO dataset, the bound is set as 2. PGD adversarial training typically results in low success rate compared with other methods; SOAP, on the other hand, typically "easy" to attack before purification because there is no adversarial training involved. Notice that the Wide-ResNet-288 model trained label consistency is robust to the DeepFool attack even before purification. This explains why SOAP-LC robust accuracy in Table is relatively low because its DeepFool perturbations are larger than other cases.
Table 8: MNIST lg attacks success rate
Attack
Architecture
No Def
FGSM AT
PGD AT
SOAP-DR
FCN
100.00
99.98
99.98
99.78
CNN
100.00
100.00
100.00
99.82
FCN
99.88
94.90
99.10
99.53
DF
CNN
99.92
100.00
18.97
95.33
Table 9: CIFARIO & attacks success rate
Attack
Architecture
No Def
FGSM AT
PGD AT
SOAP-RP
SOAP-LC
ResNet-18
100.00
97.31
96.88
100.00
100,00
CW
Tide-ResNet-28
100.00
100.00
97.95
99.92
99.99
ResNet-18
100.00
88.20
87.01
99.99
99.85
DF
Wide-Rest
Rede-Res
100.00
100.00
83.68
96.31
28.22
15


Reviews:
[{"review_id": "vp78c1ImyzJ", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "[Summary]\nOnline defenses of adversarial examples is an old topic: Given an input x (potentially adversarially perturbed) at test time, we want to sanitize x to get x', on which the trained classifier $g \\circ f$ gives the correct answer. This paper proposes a new architecture for online defenses via self supervision. There are two new things in the proposal:\n\n1. There is an explicit representation function f, namely the classifier is decomposed into $g \\circ f$. And the auxiliary self-supervised component h works on the same representation. This thus creates a Y-shape architecture that is \"syntactically\" similar to the training structure in unsupervised domain adaptation (e.g., domain adversarial neural networks). This architecture for online defense seems new (as far as I know).\n\n2. The paper leverages an interesting hypothesis that for a common f, a large classification loss happens if and only if a large self-supervision loss happens. And this paper provides solid evidence to justify this -- namely in Section 4.1 (auxiliary-aware attacks), it evaluates the defense against an adversary that is aware of h, in order to create adversarial examples that explicitly breaks the hypothesis (i.e. large classification loss but small self-supervision loss).\n\n3. For the experiments -- the paper trained f, g, and h under Gaussian corruptions, and indeed found that this online purification strategy provides robustness under adversarial perturbations, even for auxiliary-aware attacks, which is interesting.\n\n[Assessment]\n1. My first worry is that the performance of the defense is still much worse than the performance from direct adversarial training (for example, check the MNIST numbers). For example, under PGD, on CNN architecture we can achieve 80%ish accuracy. Note that for MNST, a simple discretization can already achieve almost-perfect accuracy. This is especially the case if we consider auxiliary-aware attacks.\n\n2. Following (1), what worries me more is that online-purification still needs to be aware of the attack type. Namely if one looks into equation (4), the objective has encoded norm-based attacks within it. This makes the results less interesting.\n\n3. All in all, my major doubt is what is really the benefit of reduced training complexity if we cannot achieve better robustness, and also the defense still needs to be fully aware of the attack type? For these reasons, I vote for a weak reject.\n\n[Questions]\n1. Why do we need to know the results for FCN (fully connected networks)?\n\n2. I am not sure the numbers reported for adversarial training match the state of the art reported in the MNIST challenge leaderboard: https://github.com/MadryLab/mnist_challenge. There the SOTA MNIST model always has >88% accuracy (so I am a bit skeptical about DF can bring down the accuracy to 78% for PGD AT). Also, how about applying those attacks for the self-supervision defense? (that's an additional request). Similarly, for CIFAR10, as shown by https://github.com/MadryLab/cifar10_challenge, PGD AT is never under 43%, but in Table 2, the robust accuracy is only 2% under CW attack. This is suspicious.\n\n[Post rebuttal]\n\nAfter more discussion and reading through the revision, I think this is a good paper and will be useful to the community for an instance of test-time defenses.", "extra_scores": {}, "extra_text": {}}, {"review_id": "po_6iM4JmD", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "###################\nSummary:\n\nThis paper studies adversarial defense by combing purification and self-supervised loss. During inference, the authors propose an online-purification method based on (clipped) iterative gradient ascent. The loss used by purification is from some pre-defined self-supervised tasks. During training, joint loss of softmax and self-supervised loss are used to match the purification process in inference. Experiments on MNIST10 and CIFAR10 demonstrate the effectiveness of the proposed method over several SOTA baselines. The evaluation considers both the white-box and black-box attack setup.\n\n###################\nPros\n\n1. The proposed method is well-motivated and reasonable.\n\n2. The paper is clear and easy-to-follow.\n\n\n###################\nCons\n\n1. What is the T  for the online purification? Large T will significantly slow down the test time efficiency.\n\n2. In Table 2, \"FGSM AT\" + \"PGD\", why it is \"37.4%\"? My understanding is this should be very small value, since multi-step PGD attack is pretty strong.\n\n3. I am curious to see the gain by purely online-purification, maybe using the encoder by \"PGD AT\".\n\n4. Seems like self-supervised tasks are pretty ad-hoc. Is there a principled way of selecting a good self-supervised task?\n\n5. The two datasets used in the paper represents limited visual patterns. I think larger-dataset needs to be used, like cifar100, tiny imagenet.\n\n######################### post-rebuttal\n\nI appreciate the additional explanations and experiments by the authors. I also read the public discussion threads. I raise my score to 6. Two things for future:\n- Make it work on bigger and more realistic images, imagenet, pascal, coco, etc. Now the adversarial community and deep learning community in general, highly relies on experiments, because theoretical guarantee is still mysterious. So we should push the field forward, by proving ideas on harder datasets. \n- Explore stronger attacks, particularly gradient-free attack to avoid the obfuscated gradients.\n\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "dnmwi7rSNqz", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: The paper introduces a defence for adversarial attack based on minimising a self-supervised loss on the test examples. Authors work under the assumption that minimising the self-supervised loss would be equivalent to minimising the supervised loss (to which they don't have access at test time). Authors evaluate their method on MNIST and CIFAR.\n\nStrengths:\n- The paper address the important topic of adversarial defence. Given the numerous adversarial attacks that have appeared in the last years and the deployment of novel classification methods into real world tools, I believe the field of adversarial defence is relevant.\n\n- Authors use in a smart way the self-supervised loss which is traditionally used for learning good representation as pretrainining networks. I think self-supervised learning should extend its usage over the traditional framework and this paper is one example of its potential.\n\n- Authors are also able to compute its own budget using the self-supervised loss, which I believe it's additional evidence that the usage of self-supervised learning for adversarial defence is interesting and useful. \n\n- Quantitative results show how the proposed method is competitive with methods.\n\n- Authors also evaluate the effectiveness of the method when faced with an attacker knowing which defence method is using. Although I consider the proposed attack a baseline attack (maybe other alternatives can be used), I believe it's a relevant result. \n\n\nWeaknesses:\n- I am missing evaluation of the method in larger scale datasets, or more natural images dataset. I think for this methods to be applicable and useful, authors should demonstrate its usefulness into real data. \n\n- I am missing some images of the CIFAR dataset similar to Figure 2. I know the supplementary material shows some, but it would be good to include some into the main paper.\n\n- I think authors should at least have one of the self-supervised methods (LC, RP or DR) show performance in both dataset. Given a new dataset, which methods should I select?\n\nConclusion: I believe the paper presents an interesting method with strong experimental results. The paper deserves acceptance as I believe it contains enough evidence to proof the effectiveness of the method. \n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a defense scheme for adversarial attacks, called self-supervised online adversarial purification (SOAP), by purifying the adversarial examples at test time. The novelty of this work is in its incorporation of self-supervised representation learning into adversarial defense through purification via optimizing an auxiliary self-supervised loss. This is done by jointly training the model on a self-supervised task while it is learning to perform the target classification task in a multi-task learning setting. Compared with existing adversarial defense schemes such as adversarial training and purification techniques, SOAP has a lower computation overhead during the training stage.

**Strengths:**
  * It is novel to incorporate self-supervised learning for adversarial purification at test time.
  * SOAP’s training stage based on multi-task learning incurs low computation overhead compared with the original classification task.

**Weaknesses:**
  * Although the proposed adversarial defense scheme is computationally cheaper than the other existing methods during the training stage, it does incur some overhead during test time. This may be undesirable for some applications in which efficiency during test time is an important factor to consider.
  * The choice of a suitable self-supervised auxiliary task is somewhat ad hoc. The performance varies a lot for different auxiliary tasks.
  * The experimental evaluation is only based on relatively small and unrealistic datasets even after new experiments on CIFAR-100 have been added by the authors.

It is said in the paper that SOAP can exploit a wider range of self-supervised signals for purification and hence conceptually can be applied to any format of data and not just images, given an appropriate self-supervised task. However, this claim has not been substantiated in the paper using non-image data.

Despite some limitations and that some claims still need to be better substantiated, the paper presents some novel ideas which are expected to arouse interest for follow-up work in the adversarial attack and defense research community.

Decision: Accept (Poster)