Improving Adversarial Defense with Self-supervised Test-time Fine-tuning
Authors: Zhichao Huang
Chen Liu
Mathieu Salzmann
Sabine Süsstrunk
Tong Zhang
Venue: ICLR 2022 Submitted (2022)

Abstract: Although adversarial training and its variants currently constitute the most effective way to achieve robustness against adversarial attacks, their poor generalization limits their performance on the test samples. In this work, we propose to improve the generalization and robust accuracy of adversarially-trained networks via self-supervised test-time fine-tuning. To this end, we introduce a meta adversarial training method to find a good starting point for test-time fine-tuning. It incorporates the test-time fine-tuning procedure into the training phase and strengthens the correlation between the self-supervised and classification tasks. The extensive experiments on CIFAR10, STL10 and Tiny ImageNet using different self-supervised tasks show that our method consistently improves the robust accuracy under different attack strategies for both the white-box and black-box attacks.

Under review as a conference paper at ICLR 2022
IMPROVING ADVERSARIAL DEFENSE WITH SELF- SUPERVISED BEST-TIME FINI-GUNING
Anonymous authors Paper under double-blind review
ABSTRACT
Although adversarial training and its variants currently constitute the most effec- tive way to achieve robustness against adversarial attacks, their poor generaliza- tion limits their performance the test samples. In this work, we propose im- prove the generalization and robust accuracy of adversarially-train -rraiine networks via self-supervised test-time fine-tuning. To this end, we introduce a meta adversarial training method to find good starting point for test-time fine-tuning. It incorpo- rates the test-time fine-tuning procedure into the training phase and strengthens the correlation between the self self-supervised and classification tasks. The extensive ex- periments on CIFAR 10, STL and Tiny ImageNet using different sulf-supervised tasks show that our method consistently improves the robust accuracy under dif- ferent attack strategies for both the white-box and black- box attacks.
INTRODUCTION
Adversarial training and its variants (Madry al., 2018 Wang et al. 2019 Zhang al. 2019)
currently recognized as the most effective defense mechanism against adversarial attacks. However, adversarial training generalizes poorly; the robust accuracy gap between the training and set in adversarial training is much larger than the training-test gap in standard training of deep net- works (Neyshabur et al. 2017] 7 Zhang et al. (017)) Unfortunately, classical techniques to overcome overfitting in standard training, including regularization and data augmentation, only have little ef- fect in adversarial training (Rice et al.) 2020).
The poor generalization of adversarial training is consequence of adversarial test examples lying far away from the training set (Huan et al. 2019). To adapt to these examples, we propose to fine- tune the network for each test mini-batch. Since the labels of the test images are not available, we exploit self-supervision, which is widely used the standard training of networks (Chen et al., 2020b, Gidaris et al. 2018 He et al., 2020). Fine-tuning self-supervised tasks is substitute of fine-tuning the classification loss at the inference time. Thus, minimizing this self-supervised loss function yields better generalization the test set.
To make our test-time fine-tuning strategy effective, we need to search for good starting point during training. As will be shown in experiments, adversarial training does not provide the optimal starting point. We therefore formulate the search for the network parameters that can be ef- fectively fine-tuned
bilevel optimization problem. Specifically, we introduce meta adversarial training strategy dedicated to our self-supervised fine-tuning inspired by the model-agnostic meta- learning (MAML) framework (Finn al., 2017). To this end, we treat the classification of each batch of adversarial images one task and minimize the corresponding classification error of the superveeee fine-tuned network. This meta adversarial training strategy strengthens the correla- tion between the self- superviedd and classification tasks so that self-supervised test-time fine-tuning further improves the robust accuracy.
In order to reliably evaluate our method, we follow the suggestions of (Tramer et al., 2020) and design an adaptive attack strategy that is fully aware of the test-time fine-tuning, We empirically demonstrate the effectiveness of our method on the commonly used CIFARIO (Krizhevsky et al. 2009). STL10 (Coates et al.) 2011) and Tiny ImageNet (Le & Yang. 2015) datasets under both standard (Andrisshchenko al. 2020 Croce & Hein 2020a) Madry et al. 2018) and adaptive attacks in the scenarios of both white- box and black-bor attacks. The experiments evidence that our method consistently improves the robust accuracy under all attacks.

---

Under review as a conference paper at ICLR 2022
Our contributions can be summarized as follows:
1. We introduce the framework of self-supervised test-time fine-tuning for adversarially-trainee networks and show that improves the robust accuracy of the test data.
2. We propose a meta adversarial training strategy based on the MAML framework to find a good starting point and strengthen the correlation between the self-supervised and classification tasks.
3. Following the suggestions of (Tramer et al., 2020), we design an adaptive attack strategy that is fully aware of our test-time fine-tuning. The experiments show that approach is valid on diverse attack strategies, including standard and adaptive ones in both the white-box and black- box attacks.
Our code is available through this link
2 RELATED WORK
Adversarial Training. In recent years, many approaches have been proposed defend networks
against adversarial attacks (Guo et al., 2018 Liao et al. 2018 Song et al. 2018). Among them, adversarial training (Madry et al.) 2018 stands out as one of the most robust and popular methods, even under various strong attacks (Athalye et al.) 2018 Croce & Hein 2020a). Adversarial training optimizes the loss of adversarial examples to find parameters that are robust to adversarial attacks. Several variants of adversarial training have been proposed (Wang et al. 2019: /Zhan et al., 2019). and their performance is similar to the original version of adversarial training (Rice et al. 2020))
One important problem that limits the robust accuracy of adversarial training is overfitting. Com- pared with training on clean images, the gap of robust accuracy between the training and test set is much larger in adversarial training (Rice et al., 2020). Moreover, traditional techniques to pre- vent overfitting, such as data augmentation, have little effect. Recently, some methods have at- tempted to flatten the weight loss landscape improve the generalization of adversarial training. In particular, Adversarial Weight Perturbation (AWP) (Wu et al.] (020) achieves this by designing pottuee-ert mechanism that adversarially perturbs both inputs and weights. In addition, learning-based smoothing can flatten the landscape and improve the performance (Chen et al. 2021
Sele-super Learning. In the context of non-adversarial training, many self-supervised strate- gies have been proposed, such as rotation prediction (Gidaris et al. 2018). region/ pegpoo//ommpnent fill- ing (Criminisi et al., 2004),, patch-base base spatial composition prediction (Trinh et al.. 2019) and con- trastive learning (Chen et al.] 20206 He et al. (2220). While self- supervisioo has also been employed in adversarial training (Chen et al. 2020a| Kim al., 2020 Yang & Vondrick 2020 Hendrycks et al. 2019). their methods only use self-supervised learning at training time to regularize the pa- rameters and improve the robust accuracy. By contrast, we propose to perform self-supervised fine-tuning at test time, which we demonstrate to significantly improve the robust accuracy test images. As will be shown in the experiments, the self-supervised test-time fine-tuning has larger and complementary improvements over the training time self-supervision.
Test-time Fine-tuning. Test-time fine-tuning has been used in various fields, such as image super- resolution (Shocher et al.) 2018) and domain adaption (Sun al. 2020 Wang et al. 2021). While our work is thus closely related to Test-Time Training (TTT) in (Sun et al. 2020). we target a significantly different scenario. TTT assumes that all test samples have been subject the same dis- tribution shift compared to the training data. As a consequence, it incrementally updates the model parameters when receiving new test images. By contrast, in our scenario, there is no systematic distribution shift, and it is therefore more effective to fine-tune the parameters of the original model for every new test mini-batch. This motivates our meta adversarial training strategy, which searches for the initial model parameters that can be effectively fine-tuned in a self-supervised manner.
3 METHODOLOGY
We follow the traditional multi-task learning formulation (Caruana 1997) and consider neural net- work with a backbone z = Exx 0g) and K" + heads. One head [2;;6 outputs the classification result while the other K" heads gi (2; (z:;0g))) -... gK (2; 0yk) correspond to K auxiliary self-supervised
2

---

Under review as a conference paper at ICLR 2022
tasks.
fgkk) encompasses all trainable parameters, and we further define F fofo GR gk E, 1....... ...K..
Furthermore, let = ((11, 31), (02, 122, (En, [En,yn)) denote the training set containing n images with corresponding labels, and D = { M1 (22,. 122), (zm. Fm)} } be the test set containing m images. For further illustration, the labels of the test set are shown. However, they are unknown to the networks at test time. We denote the adversarial examples of and T as z:dv and gade respectively. They satisfy the constraints ||=-do =I| S E and |ل - I S E, and is the size of the adversarial budget. For any set S, we represent the average loss over the elements in S as
L(S)
S Σ (ss))
1
where |S] is the number of elements in S. We use F(::0) and [(::0) to denote a network F and a loss function L parametrized by 8, but we often omit the symbol 0 to simplify the notation. The general classification loss, such the cross-entropy, is denoted by L.ss. We use the superseript "AT" to denote the adversarial training loss. For example, we define
C (S) =
Σ ILIBLES
max Iaande -
2. Ldl(FF( (x.d)..)) ).8.)
(3)
Because the poor generalization of adversarial training, the dnieersarillly-triinee model may not give the correct label for large portion of the adversarial test examples. That is, for test sam- ple Zado with ground truth label y. we may have arg max, F( 0); * y where F( 0); denotes the output probability of the j-th class. Since we do not know the label ÿ at test time, we cannot directly optimize the classification loss (ate(F(Fadty 77. We therefore use self-supervision as a substitute to fine-tune the backbone E. We provide more details on our self-supervised test-time fine-tuning scheme in Section 3.10 and introduce meta adversarial training method in Section 3.2 In Section 3.3] we discuss the specific self-supervised tasks used in the experiments, and further an- alyze our algorithm in Section 3.4
3.1 TEST-TIME FINE-TUNING
Our goal is to perform self-supervised learning on the test examples to overcome the overfit- ting problem of adversarial training. To this end, let us suppose that an adversarialllytrained network with parameters 0 receives a mini-batch of adversarial test examples Bads = {(āqdv 71), (Zqdv, 152),... (Zadv }b6) As the labels ( are not available, we propose to fine-tune the backbone parameters 0g by optimizing the loss function
Bas(
Σ CSs 0; 0gk)
which encompasses K self supe-supervised tasks. Here, Lss.k represents the loss function of the k-th task and { are the weights balancing the contribution of each task.
The number of images b may vary from m. b corresponds the online setting, where only one adversarial image is available at time, and the backbone parameters 0g are adapted to every new image. The online setting the most practical one, as it does not make any assumptions on the number of adversarial test images the network receives. By contrast, b m corresponds to the offline setting where all adversarial test examples are available at once. is similar to the transductive learning (Gammerman et al. 1998 Vapnik 2013). Note that our online setting differs from the online test-time training described TTT (Sun et al. (020);; we not incrementally update the network parameters as new samples come, but instead initialize fine-tuning from the same starting point 8 for each new test image.
Eqn (4) encourages OF to update in favor of the self-supervised tasks. However, as the classification head j was only optimized for the old backbone ((:6%) ). it will typically be ill-adapted to the new parameters 8%. resulting in degraded robust accuracy. Furthermore, for a small b, the model tends to overfit to the test data, reducing Las to but extracting features that are only useful for the self-supervised tasks.
3

---

Under review as a conference paper at ICLR 2022
To overcome these problems, add an additional loss function acting the training data that both regularizes the backbone and optimizes the classification head so that f remains adapted to the fine-tuned backbone g. Specifically, let B C D denote subset of the training set. We then add the regularizer
CR(B) = CAT (B)
1 B
Σ
max
(Fa(F((7 y.)))
(5)
T.BBEBB -2,1s=
the fine-tuning process. short, Eqn (5) evaluates the adversarial training loss on the training set to fine-tune the parameters By of the classification head. It also forces the backbone E to extract features that can be used to make correct prediction, i.e., prevent fg from being misled by Lss when b is small.
Combining Eqn and Eqn 5 our final test-time fine-tuning loss
Ltest (aadv B) (
X (B)(B)
Algorithm I describes our test-time seff-supervied learning algorithm in detail. As SGD is more efficient for more large amount of data, we use SGD to optimize 8 when is large (e.g. offline setting). This version of algorithm is deferred the Appendix
where sets the influence of LR.
Algorithm 1 Self-supervised Test-time Fine-tuning
Input:
Initial parameters 0"; Adversarial test images Badv {2:ee 16 ing rate vE Steps T: Weights 0 and X
Ji=1 Training data D; Learn-
Output: Prediction of ā: 3/
for t 1 to 7' do
2: Sample batch of training images B C D
3: Find adversarial dndde of training image 24 B by PGD attack.
4: Calculate Lrest in Eqn 66)
5: 6" = 0
v/ ge Lrest [audo B: 11
6: end for
7: return Prediction arg max, F F(Tadv )²))
3.2 META ADVERSARIAL TRAINING
To make the best out of optimizing Frest at test time, we should find suitable starting point 6 i.e., starting point such that test-time self-supervised learning yields better robust accuracy. We translate this into meta learning scheme, which entails a bilevel optimization problem.
Specifically, we divide the training data into small exclusive subsets D Lن Bj and let B to be adversaries of Bj. We then formulate meta adversarial learning as the bilevel minimization of
mmeta (D:0) = Σ CAT (3))) 07 = arg min Lss (Bodv;
(7)
B,CD
where Lss is the self- self-supervised loss function defined in Eqn and LAT is the loss function of adversarial training defined Eqn (3). As bilevel optimization is cone-consuming, following MAMI (Finn et al.) (017), approximate 0). That is, we compute
single gradient step of the current model parameters 0 to
8
(8)
In essence, this meta adversarial training scheme searches for a starting point such that fine-tuning with Lss will lead to a good robust accuracy. If this holds for all training subsets, then we can expect the robust accuracy after fine-tuning at test time also to increase. Note that, because the meta learning objective of Equ (7) already accounts for classification accuracy, the regularization by Cr not needed during meta adversarial learning.
Accelerating Training. To compute the gradient VoL (D; 8), we need to calculate the time- consuming second order derivatives V3 ( : V) LAT (By:77) Considering that ad- (Bddv,
versarial training is already much slower than standard training (Shafahi et al. 2019), cannot
4

---

Under review as a conference paper at ICLR 2022
afford another significant training overhead. Fortunately, as shown in (Finn al. 2017) second order derivatives have little influence on the performance of MAML. We therefore ignore them and take the gradient to be
V (iCmetaDD:00 (
CAT Vo VoyLds (By: BjCD
Σ
However, by ignoring the second order gradient, only the parameters on the forward path of the classifier F, i.e., Ag and f,. will be updated. In other words, optimizing Eqn 7 in this fashion will not update {8gk 1 k=1- To nonetheless encourage each self- supervuper head GR output the correct prediction, we incorporate an additional loss function encoding the self-supervised tasks,
-
Σ
max
Css. (x.d))))
(10)
Lande -2 Se
JEE
Note that the adversarial version of Lss to provide robustness to the self-supervised tasks, which, as shown in (Chen et al. [020a] Hendrycks et al. 2019) Yang & Vondrick) 2020). is beneficial for the classifier. The final meta adversarial learning objective therefore
Ltrain (D) Cmeta (D) + X CSS CAT (D)
(11)
where balances the two losses. Algorithm 2 shows complete meta adversarial training algorithm.
Algorithm 2 Meta Adversarial Training
Input: Training set D; I Learning rate ox, 9; Iterations T; Weights A and X Output: Starting parameters 0g for the test-time fine-tuning
1: for t = 1 to T' do
2: Sample exclusive batches of training images B1, B2, ,BB CD
3: Using PGD to find the adversaries Bodu: j2a = arg max] - sslllse Lote Lale (F(19d)), 3.y5))
addo
4: for batches B1, B2. ... By do
5:
0j 0 - V Wo (Bqdv;
6: Leeta.j LAT (Bz:07
7: end for
8: 0
8 Σ B,
V Voylmetass + XV (BVegg(BB:)))
9: end for
10: return Trained parameters 0"
3.3 SELF-SUPERVISED TASKS
In principle, any self-supervised tasks can be used for test-time fine-tuning, as long as they are positively correlated with the robust accuracy. However, for the test-time fine-tuning to remain ef- ficient, we should not use too many self-supervised tasks. Furthermore, as we aim to support the fully online setting, where only loss (Chen et al. 20206 He et al.) 2020 Kim et al. (0200) to Lss. In our experiments, we there- fore use two self seff-super tasks that have been shown to be useful improve the classification accuracy: Rotation Prediction and Vertical Flip Prediction.
image is available time, we cannot incorporate a contrastive
Rotation Prediction is widely used self-supervision task proposed in (Gidaris et al. 2018) and has been employed in adversarial training as an auxiliary task to improve the robust accuracy (Chen jet al. (Heddrycs et al.] (2019). Following (Gidaris et al. (2088), we create copies of the input image by rotating with Ω {0', 90% 180 270"), J. The task then consists of 4-way classification problem, where the head frotne aims to predict the correct rotation angle. The loss for an image is the average cross-entre ensropy over the 4 copies, given by
Lrotate (x)
Σ log Mog(Grotate ((g)CCr
(12)
wEn
where Tu is the rotated image with angle Ω, Grotate Irotate E denotes the classifier for rotation prediction, and G (+), is the predicted probability for the w angle. The head Groate is a fully-connected layer followed by softmax layer.
5

---

Under review as a conference paper at ICLR 2022
Table I: The statistics p(zaade) and two self-supervised tasks. The dataset CI- FARIO and the network is WideResNet- 34-10. 10. Adversarial budget E 0.031
Rotation
Vertical Flip
Tasks E(p(zadd)) P(p(Tudd) > 0)
25
33
6.0
1.
p(
pl
Rotation
0.15
68.51%
Figure 1: Empirical edf of p(z:dd)))
CIFAR10 and WideResNet-33-10. Adversarial budget & = 0.031
VFlip 0.22
72.16%
Vertical Flip (VFlip) Prediction is a self-supervised task similar to rotation prediction and has also been used for self-supervised learning (Saito et al.) 2020). In essence, we make two copies of the input image and flip one copy vertically. The head gvitip then contains 2-way fully-connected layer followed by softmax layer and predicts whether the image is vertically flipped not. The corresponding loss for an image is
C-lli (2) = - I (og(Grti((Eeoo)
(13)
VEV
where == [flipped, not flipped] | is the operation set and Gvtip = gvtlip E. denotes and transformed input and G- l is the probability of operation v. Note that we do flip the image horizontally as it is common data augmentation technique and classifiers typically seek to be invariant horizontal flip.
3.4 METHOD ANALYSIS
We now analyze why our test-time self-supervised fine-tuning improves the robust accuracy. To this end, we observe the correlation between the gradient of self-supervised loss Lss and the classifi- cation loss Las- For an input Todv with label Yis the gradient correlation with respect to the shared parameters 0F can be expressed as
Vo, (Tdo Ui) V 0 Las (T.d0)
p(aado)
(14)
Cos J.)llel V Las (ZQdu) )12
If p(7gdo) is significantly larger than O, then gradient descent w.r.t. Lss should act a good substitute for optimizing Lets. Let us consider small step in the direction of Css Then, the classification loss Lets can be approximated by Taylor expansion as
Di, OF V/V Css ( Less (T ,b: 0r) Les (7
(15)
( Vo 0s fets (Z: D))l2 oog (Ts ll2
As OF contains millions of parameters, the gradient norm is typically large, even small will decrease the classification loss when ((āūde) significantly larger than
Below, we further confirm this empirically. We choose cross-entropy the classification loss. For all adversarial test inputs Zadv Dadv where Dods is the adversaries of the test data, regard p(āadv) as a random variable and calculate its empirical statistics on the test dataset. Table shows the empirical statisties of an adversarially- setersaariilly-tainnee -trained model CIFAR 10, and Figure shows the cumulative distribution of p(zade)) The mean p(2,adu) is indeed significantly larger than O and P(o(zade) 0) larger than the robust accuracy of the adversarially-trainee network (50%-60%) which implies that self-supervised test-time fine-tuning helps to correctly classify the adversarial test images.
ADAPTIVE ATTACKS
In the white-bov box attacks, the attacker knows every detail of the defense method. Therefore, we need to assume that the attacker is aware of our test-time fine-tuning method and will adjust its strategy for generating adversarial examples accordingly. Here, we discuss one such strong adaptation strategy targeted to our method.

---

Under review as a conference paper at ICLR 2022
Suppose that the attacker is fully aware of the hyperparameters for test-time fine-tuning. Then, finding adversaries Bade of the clean subset B can be achieved by maximizing the adaptive loss
padd arg max Lattack (F (F(zgd") y; 0 (Badu)).
(16)
|
where Lattack refers to the general attack loss, such the cross- entsoyy the difference of logit tio (DLR) (Croce& & Hein 2020a). Let 8" be the line-tuned test-timo parameters using Algorithm
At the k-th step of the attack, it depends the input ((k) {
(k) ود)ي via the update
0++ = 0 - Vo Sreet (B(h), B),
(17)
where Liest and B are the loss function and subset of training images mentioned in Eqn 6 As
8 is a function of the input (()), we can calculate the end-to-end gradient of (²(k) E B(k) as
V a/ Vage Lattack (F (F()!")); ); 8T (B(k))). However, 0" goes through T gradient descent steps, and thus calculating the gradient V ange 10 (B(k)) requires 7'tth order derivatives of the backbone E, which is virtually impossible if T or the dimension of 0g is large. We therefore approximate the gradient as (r)
V 22 Costack (F(z!")); )²"),
(18)
which treats 8 as a fixed variable so that high-order derivatives from B (T(B(() (z{ can be avoided. Although this approximation makes Grad inaccurate, common white-box attacks use projected gradients, which are robust to such inaccuracies. For example, PGD only uses the sign of the gradient under an &oo adversarial budget. Note that solving the maximization in Eqn (16 does not necessarily require calculating the gradient Grad(z;()) ) For instance, we will also use Square Attack (Andriushchenko et al. 2020), strong score-based black-box attack, to maximize Eqn (16) and generate adversaries for B.
As another approximation to save time, one can also fixing 8T for several iterations. This leverages the intuition that attack strategies often make small changes the input X. and thus, for the interme- diate images the k-th and (k++1)-th steps, 6T (B(*) and 6T (B(+-)) should be close. Therefore, general version of our adaptive attacks only updates d" every u iterations, with a hyperparameter. We provide the algorithms for PGD, AutoPGD, FAB and Square Attack in the Appendix
EXPERIMENTS
Experimental Settings. Following previous works (Cui et al. 2020 Huang et al. 2020), we con- sider &x -orm attacks with an adversarial budget 8 0.031 8/255). We evaluate our method on three datasets: CIFARIO (Krizhessky et al. 2009). STL10 (Coates et al. 201T) and Tiny Im- ageNet (Le & Yangl 2015). We also use two different network architectures: WideResNet-34, 10 (Zagoruyko & Komodakis 2016) for CIFARIO, and ResNet18 (He et al., 2016) for STL10 and Tiny ImageNet. Following the common settings in adversarial training, we train the network for 100 epochs using SGD with momentum factor of 0.9 and a weight decay factor of 5 x 10-1, The learning rate 8 starts at 0.1 and is divided by factor of 10 at the 50-th and the 75-th epochs. The step size a in Eqn (8) the same as 3: the factor X in Eqn 11 is 1.0. We set the weight of each self-supervised task X K where K is the number of tasks. We use 10-iteration PGD with step size 0.007(= 0.031/4) to find the adversarial Badv during training. To prevent the computational burden of meta adversarial training, we use divide the classification of 32 images as one task and sample 8 mini-batches BI.... Bg in each iteration. [B,] 32 and q 8 in Algorithm 2. More detailed experimental settings are provided in the Appendix B
Attack Methods. We consider four common white-bon and black-box attack methods: PGD- 20 (Madry et al.] |l.||2118), (2018). AutoPGD (Croce & Hein| (020a)), FAB (Croce & Hein (020b) and Square Attack (Andriushchenko al., (020).. We apply both the standard and adaptive versions of these methods. Furthermore, for AutoPGI, we use both the cross-entropy and DLR (Crace & Hein 2020ah loss. More details are provided in the Appendix C
Baselines. We compare our method with the following methods: 1) Regular adversarial training
(Regular AT), which uses CAT in Eqn (3). 2) Regular adversarial training with an additional self- supervised loss, i.e., using LAT LAT for adversarial training, where LAT is given in Eqn {10)
7

---

Under review as a conference paper at ICLR 2022
Table 2: Robust test accuracy on CIFARIO of the test-time fine-tuning on both the online setting and the offline setting. We use the WideResNet-34-10 10 with an foe budget 0.031. AT means adversarial training and FT stands for fine-tuning. We underline the accuracy of the strongest attack and highlight the highest accuracy among them.
Square Attack
PGD-20
AutoPGD
FAB
Tasks
Methods
Standard Adaptive Standard Adaptive Standard Adaptive Standard Adaptive
None
Regular AT
62.51%
55.74%
52.14%
51.34%
Regular AT w/o
63.54%
56.64%
52.57%
51.87%
Meta AT w/o FT
63.96%
57.35%
53.09%
53.09%
Rotation
Online FT
65.52%
65.85%
59.52%
59.50%
57.93%
56.96%
75.58%
77.69%
Offiin FT
67.05%
65.75%
61.17%
59.71% %
58.77%
57.63 %
78. 78.12%
68.60%
Regular AT w/o FT 62.09%
55.50%%
52.79%
51.24%
Meta AT w/o FT
66.15%
59.73%
53.41%
53.02%
VFlip
Online FT
66.91% %
66. 16%
61. 11.47%
59.40%
58.74%
56.79%
75.68% %
80.57%
Offline FT
67.23%
65.60%
61. 61.82%
59.69%
59.26%
58.06 %
75.60%
72.24%
Regular AT w/o FT" 65.64%
59.19%
53.16%
53.05%
Rotation
Meta AT w/o FT
65.75%
59.51
53.99%
53.85%
+
VFlip
Online FT
67.34%
66.80%
61.79%
60.46%
59.23%
57.70%
76.39%
79.80% 71.58%
Offline FT
68.50%
66.05%
62.87%
60.54%
60.25%
58.26% %
76.89%
Table 3: STLI0 results of test-time fine-tuning. We use a ResNet18 with an foo budget E 0.031.
We underline the accuracy of the strongest attack and highlight the highest accuracy among them.
Square Attack
PGD-20
AutoPGD
FAB
Tasks
Methods
Standard Adaptive Standard Adaptive Standard Adaptive Standard Adaptive
None
Regular AT
44.83%
37.89%
35.78%
33.64%
Regular AT w/o FT 44.00%
36.92%
33.72%
33.73%
Rotation
Meta AT w/o FT
44.75%
28.66%
35.60%
35.38%
+
VFlip
Online FT
45.07%
46. 19%
40.31%
40.24%
39.53%
40.85%
51.25%
51.08%
Offline FT
47.86%
48.03%
45.21%
43.33%
43.78%
43.20%
58.49%
54.13%
This corresponds to the formulation of (Hendrycks et al. 2019). 3) Meta adversarial learning (Al- gorithm 2) without test-time fine-tuning.
We include additional figures, experiments with diffcrent adversarial budget and experiments about label leaking in Appendix A
5.1 ONLINE TEST-TIME FINE-TUNING
Let us first evaluate our method in the online setting, where only one adversarial image is available when fine-tuning the network. We fine-tune the network for 10 steps with a momentum factor of 0.9 and a learning rate 17 = 5 x 10-4, We set Å K and X 15.0.
CIFARIO. Table 2 shows the robust accuracy for different attacks and using two different tasks for fine-tuning. The adaptive attack is not applicable to models without fine-tuning. As we inject different self-supervised lasks into the adversarial training stage. and as different self-supervised tasks may impact the robust accuracy differently (Chen et al. 2020a). the robust accuracy without fine-tuning still varies. The vertical flipping task yields better robust accuracy before fine-tuning but its improvements after fine-tuning is small. By contrast, rotation prediction achieves low robust accuracy before fine-tuning, but its improvement after fine-tuning the largest. Using both tasks together combines their effect and yields the highest overall accuracy after test-time fine-tuning.
Note that our self-supervised test-time fine-tuning, together with meta adversarial learning, consis- tently improves the robust accuracy under different attack methods. Under the strongest adaptive AutoPGD, test-time fine-tuning using both tasks achieves a robust accuracy of 57.70%, significantly outperforming regular adversarial training.
STL10 and Tiny ImageNet. As using both the rotation and vertical flip prediction led to the highest overall accuracy CIFARIO, we focus on this strategy for STL10 and Tiny ImageNet. Table 3ndd 4 shows the robust accuracy on STL10 and Tiny ImageNet using a ResNet18. Our approach also significantly outperforms regular adversarial training on these datasets.
8

---

Under review as a conference paper at ICLR 2022
Table 4: Tiny ImageNet results of test-time fine-tuning. We use a ResNet18 with an & budget 0.031. We underline the accuracy of the strongest attack and highlight the highest accuracy among them.
Square Attack
PGD-20
AutoPGD
FAB
Tasks
Methods
Standard Adaptive Standard Adaptive Standard Adaptive Standard Adaptive
None
Regular AT
28.5%
20.6%
17.5%
17.2%
Regular AT w/o FT'
29.5%
22.2%
17.1%
16.7%
Rotation
Meta AT w/o FT
29.3%
23.1%
16.9%
16.8%
+
VFlip
Online FT
30.2%
30.2%
24.0%
23.2%
18.9%
18.1%
33.7%
31.6%
Offlin FT
32.4%
31.0%
25.66
24.1%
23.7%
20.6%
36.5%
27.7%
Table 5: Ablation study the online test-time fine-tuning. The dataset is CIFAR1O and the task is the "Rotation VFlip". All attacks are stan- dard attacks. Removing the Lss or LR results in lower robust accuracy than the full method, SA stands for Square Attack.
Table 6: Ablation study on the online test-time fine-tuning. The dataset is CIFARIO and the task is the "Rotation + VFlip". All attacks are stan-
dard attacks. SA stands for Square Attack.
Methods
SA
PGD-20
AutoPGD
FAB
Regular 65.64%
59.19%
53.16%
53.05%
Methods
SA
PGD-20
AutoPGD
FAB
Online FT
66.26%
60.18%
56.86%
75.26%
Before FT
65.75%
59.51%
53.99%
53.85%
Improvement 0.62%
0.99%
3.70%
22.21%
Online FT
67.34%
61.79%
59.23%
76.39%
Meta AT
65.75%
59.51%
53.99%
53.85%
Removing CR
66.83%
60.45%
57.32%
75.63%
Online
67.34%
61.79%
59.23%
76.39%
Removing Lss
65.44%
60.24%
55.64%
75.08%
Improvement 1.59%
2.28%
5.24%
22.54%
5.2 OFFLINE TEST-TIME FINE-TUNING
Let us now study the offline setting, where all test adversarial examples are available at test time. We use SGD to fine-tune the model for 10 epochs. We set À 10.0 and keep the other settings the same as for online fine-tuning. As shown in Table 2 the offline fine-tuning further improves the robust accuracy over the online version.
5.3 ABLATION STUDY
Removing Lss or CR. In our previous experiments, test-time fine-tuning was achieved using combination of two loss functions: Lss and LR. To study the effect of cach of these terms sep- arately, we remove either one of them from Ltest. In Table [ we report the robust accuracy after online fine-tuning using only La and only Lss. While, as expected, removing Lss tends to reduce more accuracy than removing Lx. It shows the benefits of our self-supervised test-time fine-tuning strategy. Nevertheless, the best results are obtained by exploiting both loss terms.
Removing Meta Adversarial Training. Our meta training strategy in Algorithm 2 aims to strengthen the correlation between the self-supervised tasks and classification. To show its effective- ness, we perform an ablation study where we fine-tune the model with regular adversarial training (i.e., setting 0 in the line 5 of Algorithm 2). We then perform the same test-time fine-tuning on the model without meta adversarial training, using the same hyperparameters as in the meta adver- sarial training case. As shown in Table 6 the robust accuracy and the improvements of fine-tuning are consistently worse without meta adversarial training.
6 CONCLUSION
We propose self cuff-supervieed test-time fine-tuning on adversarially-triue -rainned models to improve their generalization ability. Furthermore, we introduce meta adversarial training strategy to find good starting point for our self-supervised fine-tuning process. Our extensive experiments on CIFAR 10, STLIO and Tiny ImageNet demonstrate that our method consistently improves the robust accuracy under different attack strategies, including strong adaptive attacks where the attacker is aware of our test-time fine-tuning technique. In these experiments, we utilize three different sources self- supervision: rotation prediction, vertical flip prediction and the ensemble of them. While all settings lead to improved robust accuracy, some selff-supervieed tasks seem to have more impact than others. Finding better self supf-supe tasks and their combinations will be the focus of our future work.

---

Under review as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
We provide pesudocode of our algorithms and the detailed hyperparameters in the main text and Appendix. The datasets we use (CIFARIO, STL10 and Tiny ImageNet) all publicly available. In addition, we provide the code to reproduce the experiment results here
ETHICS STATEMENT
Adversarial examples bring security threats to the current machine learning models. Adversarial examples may be maliciously used to attack the safety-critical systems like autonomous driving and face verification. While adversarial training is effective in defending against adversarial attacks, suffers from poor generalization. We propose to use the self-supervised test-time fine-tuning to increase the robust accuracy of these models. It helps build safer machine learning systems.
REFERENCES
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: query- efficient black-box adversarial attack via random search. In European Conference Computer Vision, pp. 484-501. Springer, 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false of security: Circumventing defenses adversarial examples. In International Conference on Machine Learning, pp- 274-283, PMLR, 2018.
Rich Caruana. Multitask learning. Machine learning, 28(1):41-75, 1997.
Jinghui Chen and Quanquan Gu. Rays: ray searching method for hard-label adversarial attack. In Pro- ceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 1739-1747, 2020.
Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From sulf-super pre-training fine-tuning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 699-708, 2020a.
Tianlong Chen, Zhenyu Zhang. Sijia Liu, Shiyu Chang, Zhangyang Wang. Robust overfitting may be mitigated by properly learned smoothening, In International Conference on Learning Representations, 2021. URL htt.ttttt://:teeeee
net forum?
Ting Chen, Simon Korblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, pp. 1597-1607, PMLR, 2020b.
Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp. 215-223. JMLR Workshop and Conference Proceedings, 2011.
Antonio Criminisi, Patrick Pérez, and Kentaro Toyama. Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on image processing, 1300):1200-1221 1212, 2004.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter- free attacks. In International Conference Machine Learning, pp. 2206 2216. PMLR. 2020a.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. International Conference on Machine Learning, pp. 2196 2205. PMLR, 2020b.
Jiequan Cui, Shu Liu, Liwei Wang, Jiaya Learnable boundary guided adversarial training. arXiv preprint arXiv:2011.11164, 2020.
Chelsca Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, pp. 1126 1135. PMLR, 2017.
A. Gammerman, V. Vovk, and V. Vapnik. Learning by transduction. In Proceedings of the Fourteenth Confer- ence on Uncertainty in Artificial Intelligence, UAI'98, pp. 148-155, San Francisco, CA, USA, 1998. Morgan Kaufmann Publishers Inc. ISBN 155860555X.
10

---

Under review as a conference paper at ICLR 2022
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predict-
ing image rotations. In International Conference Learning Representations, 2018. URL https:
//openrvview net/ pttp/oororm.--SSAAW2Z2
Sven Gowal, Chongli Qin, Jonathan Ucsato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03593, 2020.
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens der Maaten. Countering adversarial images
using imput transformattons. In International Conference on Learning Representations, 2018. URL https:
//openreview /tt/ffounme
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729-9738, 2020.
Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song- Using self-supervised learning can improve model robustness and uncertainty.
Wallach, H. Larochelle, A. Beygelzimer, d'Alché- Buc, E. Fox, and R. Gareett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL httpsttpsss htttt:::://rceeeee /ptpc/eeeoccessssc/e/eppppc neuri httppppssccpppppp///1/////// (at.i////edaaa11110222222250 Papppee Faper.
Zhang Huan. Chen Hongge. Song Zhao, S. Boning Duane. S. Dhillon Inderjit, and Hsich Cho-Jui. The limita- tions of adversarial training and the blind spot attack. In International Conference on Learning Representa- tions, 2019. URL wtt..ttts:/// opwnreviiivveeeeeee ew. /
T
Lang Huang, Chao Zhang, and Hongyang Zhang. Self-adaptive training: beyond empirical risk minimization. In Advances Neural Information Processing Systems, volume 33, 2020.
Minseon Kim, Jihoon Tack, and Sung Ju Hwang. Adversarial self- self-supervised contrastive learning. In Advances in Neural Information Processing Systems, 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7)3, 2015.
Fangzhou Liao, Ming Liang. Yinpeng Dong. Tianyu Pang. Xiaolin Hu, and Jun Zhu. Defense against adver- sarial attacks using high-level representation guided denoiser. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1778-1787, 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations, 2018.
Behnam Neyshabur, Srinadh Bhojanapalli, David Mcallester, and Nati Srebro. Exploring generaliza- tion in deep learning. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vish- wanathan, and R. Garnett (eds.). Advances in Neural Information Processing Systems, volume 30, Cur- Associates, Inc., 2017. URL
wewrdiiisssin..n.....e.......eee wwwrips cctpp/a////////////eee///////
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In International Conference on Machine Learning, pp. 8093-8104. PMLR, 2020.
Kuniaki Saito, Donghyun Kim. Stan Sclaroff. and Kate Saenko. Universal domain adaptation through self supervision. In H. Larochelle, M. Ranzato, R. Hadsell. M. F. Balcan, and H. Lin (cds.), Advances in Neural Information Processing Systems, volume 33, pp. 16282-16292. Cur- ran Associates, Inc., 2020. URL pD4 7e16666788881aae (e65566811169fefee..e wwtpeeeeeeeeeeeaaaeeae----------
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In H. Wallach, H. Larochelle, A. Beygelzimer, F. B'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL httpttppps//ppppeeaagg..e.eeeeee påf
www.sps.eeppeccee cc/
paper/2019/file/ htt.a/////////oeeeeeeeee000000e.
Changhao Shi, Chester Holtz, and Gal Mishne. Online adversarial purification based on self-supervision. arXiv preprint 2Xix:2001.09388 2021.
11

---

Under review as a conference paper at ICLR 2022
Assaf Shocher, Nadav Cohen, and Michal Irani. "zero-shot" super-resolution using deep internal learning. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3118-3126, 2018.
Yang Song. Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand defend against adversarial examples. In International Conference on Learning Representations, 2018. URL httttss://oppreeeeeeeeee ew net ftre:/ffooumiiid-uuuxxxoo
Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self- supervision for generalization under distribution shifts. In International Conference on Machine Learning, pp. 9229 9248. PMLR, 2020.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Alcksander Madry. On adaptive attacks to adversarial example defenses. In Advances in Neural Information Processing Systems, volume 33, 2020.
Trieu Trinh, Minh-Thang Luong, and Quoe Le. Selfie: Self-supervised pretraining for image embedding. arXiv preprint arXiv: 11Ki::990.002200 02940, 2019.
Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 2013.
Dequan Wang, Evan Shelhamer. Shaoteng Liu. Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations, 2021. URL
httpss:/cpenteever net Forum? 2ddummmm---xXX
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Representa- tions, 2019.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. Advances in Neural Information Processing Systems, 33, 2020.
Junfeng Yang and Carl Vondrick. Multitask learning strengthens adversarial robustness. In European Confer-
on Computer Vision. Springer, 2020.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Edwin R. Hancock Richard C. Wilson and William A. P. Smith (eds.). Proceedings of the British Machine Vision Conference (BMVC), pp. 87.1- 87.12 BMVA Press, September 2016. ISBN 1-901725-59-6 doi: 10.5244/C.30.87. URL https://cx. doi wwwg//r///o 5244/C. 80.87
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In International Conference on Learning Representations, 2017.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing. Laurent Ghaoui, Michael Jordan. Theoretically principled tradc-off between robustness and accuracy. In International Conference on Machine Learning. pp. 7472-7482, PMLR, 2019.
12

---

Under review as a conference paper at ICLR 2022
A ADDITIONAL EXPERIMENTS
Robust Accuracy v.s Fine-tuning Steps. Figure 2 shows the robust accuracy each step of the test-time fine-tuning for different self-supervised tasks and attack methods. When using the standard version of attacks, the robust accuracy gradually increases fine-tuning proceeds. When using our adaptive attacks, the adversarial examples are generated to attack the network with H (T 10) instead of 0 Thus, when the parameters gradually change from 8 to 0", the accuracy drops.
Standard Square Attack
Standard PGD 20
Standard AutoPGD
Standard FAB
cee-
Accuracy
Accuracy
Accuracy
0.50
0.00
Rotation Fiip Ersemble
6.2
136
Steps
Steps
Steps
Steps
Adaptive Attack
Adaptiiee PGD-20
Adaptive AutoPGD
Adaptive FAB
640
0.00
Accuracy
Accuracy
Accuracy
Accuracy cy
0.00
6.78-
Vertica Fiip
Ensemble
Steps
Steps
Steps
Figure 2: Robust accuracy at different steps of the online test-time tine-tuning CIFARIO.
Accuracy Improvement on Clean Images. Our model is able improve not only the robust accuracy but also the natural accuracy of clean images on adversarially-traiee models. To evidence this, we maintain all the components of our model and simply replace the adversarial input images with clean images (i.e. replacing Bade with clean inputs in Algorithm and perform the same sulf-supe test-time fine-tuning.
As shown Table 7 our approach increases the clean image accuracy. This phenomenon further strengthens conjecture that the improvement of robust accuracy is due to the improvement of generalization instead of perturbing the model parameters, because the randomly perturbing the parameters usually lower the natural accuracy of the model. This also suggests that method is applicable to the broader problem of improving the accuracy of deep learning models with poor generalization ability.
Table 7: Accuracy on clean images. training.
Networks are trained with corresponding meta adversarial
Methods Without FT
Rotation
VFlip
Rotation + VFlip
84.77%
86.55%
86.36%
Online FT
86.10%
87.00%
87.10%
Accuracy Improvement on Inputs with Different Adversarial Budget. Our method is also able improve the robust accuracy of inputs with different adversarial budgets. As shown in Table g we set Eor budget of the adversarial inputs to be 0.015 to perform the online test-time fine-tuning. The robust accuracy is ed improved.
Table 8: Robust test accuracy on CIFARIO of the online test-time fine-tuning. We use the same WideResNeet34-10 in Table 2 which is trained with lx budget 0.031. The inputs are in the loc. ball of = = 0.015. The self-supervised task is the ensemble of rotation and vertical flip.
Square Attack
PGD-20
AutoPGD
FAB
Methods
Standard Adaptive Standard Adaptive Standard Adaptive Standard Adaptive
Meta AT w/o FT 78.01%
75.34%
72.72%
72.58%
Online FT
80.50%
79.87%
77.14%
76.75%
77.25%
74.93%
82.04%
83.76%
13

---

Under review as a conference paper at ICLR 2022
Label Leaking. One potential concern about method is whether the effect of label leaking leads to the improvement of the robust accuracy. The attacker uses the ground truth label to generate the adversarial examples. Someone may worry it leaks the information of the ground truth label to the adversarial images and the test-time fine-tuning can utilize the information to guess the correct label.
Previous experiments on clean images already show that test-time fine-tuning is effective even if there is no information of the ground truth label. We further resolve the concern of the label leaking by the following experiments. The attacker randomly lowers the score of the false label to perform the adversarial attack. And we conduct the self-supervised test-time fine-tuning on these "adversar- ial" images. If our method uses the information of leaked label to improve the robust accuracy. it will predict the false label and reduce the accuracy. However, as shown in Table the robust accuracy is also improved after the test-time fine-tuning. It demonstrates that the improvement of robust aceu- racy is not result of label leaking. It is worth noting that such attack is not targeted at the ground truth label. Therefore, the accuracy of the input images, along with the accuracy improvement of the test-time fine-tuning, are similar to the clean images in Table 7
Table 9: Experiments to rule out the possibility of label leaking. We use the WideRessNet-34110 trained with fo budget ş 0.031 and show the robust test accuracy on CIFARIO of the online test-time fine-tuning, The self-supervised task is the ensemble of rotation and vertical flip.
Square Attack
PGD-20
AutoPGD
FAB
Methods
Standard Adaptive Standard Adaptive Standard Adaptive Standard Adaptive
Meta AT w/o FT 85.43%
85.60%
85.00%
86.22%
Online FT
86.56%
87.63%
86.29%
86.68%
86. 10%
85.90%
87.61%
86.97%
Comparison with SOAP (Shi et al., 2021). Our method is different from SOAP as we are fine- tuning the model to adapt new examples instead of purifying the input. We apply SOAP-RP to the adversarially-trainee model and find that its improvement is marginal. Under AutoPGD, the accuracy is improved from 53.09% 53.57%. This improvement is much smaller than our method, whose improvement is from 53.09% to 57.93%. SOAP only has little effect when combining with the commonly-used adversarial training.
we perform a transfer attack from the static adversarial defense. We use the robust networks with the same architecture the substitute model, and the test-time fine-tuning also improves the robust accuracy.
Transfer Attack. In Table 10
Table 10: Accuracy on transfer attack on CIFARIO.
Methods
Rotation
VFlip 86.55% 87.00%
Rotation + VFlip
Without FT 84.77% 86.10%
86.36% 87.10%
Online FT
Expectation Attack. In Table 11 we show the results of the expectation attack. We modify the adaptive attack and average the gradient from 10 fine-tuned models, whose training batches are different. We evaluate the model using the ensemble of rotation and vertical flip as the self- supervised task on CIFARIO. We evaluate the model with Adaptive-AuoPGD-EOTT and Adaptive- SquareAttack-EOT.. One the strongest attack in our method and the other a black-box attack that is less likely to be affected by gradient masking. The experiment shows that the expectation attack has little influence on the improvement of our test-time fine-tuning.
Table 11: Accuracy on expectation attack CIFAR10 with the ensemble of rotation and vertical flip task.
Attacks
w/o Fine-tuning
w/ Fine-tuning
Adattive-AutoPGi
53.99%
57.70%
Aduptive-AuloGD--EOT
53.99%
57.65%
Adaptive-SquareAttacck
65.75%
66.80%
Adaptive-SquarrAAttackEOOT
65.75%
66.73%
14

---

Under review as a conference paper at ICLR 2022
Boundary Attack. We of the SOTA decision- based attacks: RayS (Chen & Gul 2020). We test it on CIFARIO with the ensemble of rotation and vertical flip. Table 12 shows that our method also improves the robust accuracy for the decision based attack.
Table 12: Accuracy on RayS on CIFARIO with the ensemble of rotation and vertical flip task.
Attacks
w/o Fine-tuning
w/ Fine-tuning
RayS Adaptive-RayS
65.61%
77.38%
75.03%
Combination with (Gowal et al. al.] 2020), We combine our test-time fine-tuning with the adversarial training using additional data (Gowal et al. 2020). We apply our Meta AT to it with the ensem- ble of rotation and vertical flip. Using a WideResNet-28- 10, it achieves robust accuracy 62.07% under AutoPGD. With our test-time fine-tuning, the robust accuracy is improved to 64.34%. The improvement of robust accuracy is 2.27%.
Inference Time. Table 13 shows the inference time for different methods. While the inference time for our method is larger than SOAP and normal method when the batch size is 1, the inference time gets closer when using larger batch size. And the batch size of 20 more is common scenario of the inference.
Table 13: Average inference time for each instance using different methods.
Batch Size
5
10
20
40
Normal SOAP (Shi et al. 2021)
17. 1s.1ms
14.5ms
13.2ms
12.8ms
11.7ms
163ms
91.2ms
75.3ms
73.1ms
72.5ms
Ours
545ms
168ms
118ms
83.9ms
82.9ms
B DETAILS OF OUR EXPERIMENTAL SETTING
Meta Adversarial Training. We consider an for norm with an adversarial budget & 0.031. We also use two different network architectures: WideResNett34-11 10 for CIFAR10 and ResNet18 for STLIO and Tiny ImageNet. Following the common settings for adversarial training, we train the network for 100 epochs using SGD with momentum factor of 0.9 and a weight decay factor of x 10 The learning rate 3 starts at 0.1 and divided by factor of 10 after the 50-th and again after the 75-th epochs. The step size in Eqn (8) is the same as B. The factor X' in Eqn (II is set to 1.0. We use 10-iteration PGD (PGD-10) with step size of 0.007 to find the adversarial image Bede training time. The weight of each self-supervised task is set d K- We set |B = 32 and sample 8 batches BI,... Bg in each iteration. Furthermore, we save the model after the 51-st epoch for further evaluation, as the model obtained right after the first learning rate decay usually yields the best performance (Rice et al. 2020).
We use PGD with the standard cross-entropy loss generate adversarial examples training time in the line 3, line 6 and line 8 of Algorithm The hyperparameters of the attacks are as follows:
Line 3: PGD-10 with step size 0.007.
Line 6: As 0j is similar to 0, the adversarial examples at this step are similar to those at Line 4. To save training time, therefore choose the starting point of the attack as the adversarial examples in Line 4 and use PGD-2 with a step size of 0.005.
Line 8: PGD-3 with step size 0.02.
Online Test-time Fine-tuning. We fine-tune the network for T = 10 steps with a momentum of 0.9 and learning rate of 17 5 x 10-4 We set A and À 15.0. In line 2 of Algorithm | sample a batch B C D containing 20 training images. In line 3, we use PGD-10 with a step size of 0.007.
Offline Test-time Fine-tuning. The algorithm for offline fine-tuning is shown Algorithm B As stochastic gradient descent more efficient for large amount of data, stochastic gradient
15

---

Under review as a conference paper at ICLR 2022
descent in the offline fine-tuning. This is the main different between [ fine- tine-tuning) and Algorithm 3 (offline tine-tuning). We also fine-tune the network for 10 epochs. The batch size of each B is 128. The other hyperparameters are the same as in the online version.
Algorithm 3 Self-supervised Test-time Fine-tuning with SGD
Input:
Initial parameters 0%; Adversarial test images Bodv (2gdo Training data D; Learning rate 77, Steps T; Weights Aς and
Output: Prediction of i:q(o::
1: for 1 to T do
2: Divide Bado into subsets Badv 1.. Badv
3: for Bade in Bade ..., Bade do
4: Sample batch of training images B C D
5: Find adversarial x. ado of training image 24 E B by PGD attack.
6: fr = f 17 Ltest Bgdd, B; 0(-1)
7: end for
8: end for
9: return Prediction ÿ = arg max, ((Gadv;0")
Attacks. The detailed settings of each attack are provided below:
PGD-20. We use 20 iterations of PGD) with step size = 0.003. The attack loss is the cross-entropy.
AutoPGD. We use both the cross-entropy and the difference of logits ratio (DLR) as the attack loss. The hyperparameters are the same as in (Croce & Hein 2020a).
FAB. We use the code from (Croce & Hein 2020a and keep the hyperparameters the same.
Square Attack. We set = 2000 and the initial fraction the elements p = 0.3. The other hyperparameters are the same in (Andriushchenko et al. 2020).
For the adaptive versions, we set the interval [T/5]]
C ALGORITHM FOR ADAPTIVE ATTACKS
In Algorithm 5 and we show the algorithms for fo norm-based adaptive PGD, AutoPGD, Square Attack FAB, respectively. The main difference between the original and adaptive version is the target loss function for maximization. The reader may refer to (Andriushchenko et al.) 20200] Croce Hein 2020a b) for more detailed description of the steps these algorithms (e.g. the condition for decreasing the learning rate in AutoPGD).
16

---

Under review as a conference paper at ICLR 2022
Algorithm 4 t0 Norm Adaptive PGD Attack
Input: Test images B { ūi) }: Attack loss Lattack; Step size 7; Iterations T; Intervals u;; Adversarial budget e; Trained parameters of the network 8".
Output: Adversarial images Budo = {
1: Add random noise to 2; in B and get B'
2: for t = 1 to T do
3: i mod = then
4: Get final parameters 0t by taking B' as input image for Algorithm 1 0²
5: end if
6:
for T. in B' do
Vz (Fat 0) 8: 1 = - ( Sign (T))
7: Grad (1)
9: end for
10:
end for
11: return Adversarial image aqde
Algorithm to Norm Adaptive AutoPGD
Input: Test images B {T;, Wi)); Attack loss Lattack: Step size y; Iterations T; Intervals u; Adversarial budget E; Parameter the adversarially-trrinee network 80; Decay iterations W (wo,..., } Momentum
Output: Adversarial image Rade [āqde]
1: Get final parameter 0² by taking B as input image for Algorithm
2: 0= 8"
3: for T; in B
do
4: 7:
=
5: Grad(T;) Vū, Lattack F( 0::0)
6: x! Clip(z,-e.t. +5] + +Siign (rad().)))
7:
1 Lattack ((7) ) Ts;
8: 1 Lattack (T Vs:
9: 1 max 10 [!]
F r if 1 = else .o = m!
11: end for
12: for = 1 to T do
13: i mod = then
14:
Get final parameter 8TH by taking B" = {o+} as input image for Algorithm 0 0T
16:
end if
17:
for = 1.... B do
18:
Grad(?)) = Vzz Cattack (F (F ():))
19:
2 Clip / + Sign (z))(
20:
4+ Clippa-c.5.+e ( E(2
1+1 2f) (1 - -1- -
21:
1+I = (F 0z:01
22:
E = i4+1 and 1,- = 16+1 - >
23:
if l E W and satisfy the condition of dropping learning rate then
7/2 and -++ =
24:
end if
26: end for
27:
end for
28: return Adversarial image zadv = FT
17

---

Under review as a conference paper at ICLR 2022
Algorithm lo Norm Adaptive Square Attack
Input: Test images B { }: Attack loss Lattacki Step size y Iterations T:: Intervals u; Image size w;; Color channels e; Adversarial budget e; Parameter of the adversariiall-trained network go
Output: Adversarial image Badv = {
1: Add noise to in B and get B'
2: for = 1 to T do
3: if' mod = then
4:
Get final parameter 0 by taking B' as input image for Algorithm |
5:
0 rT
6:
end if
7:
for h² in B' do
8:
h' +- side length of the square to modify (according to some schedule)
9:
8 + array zeros of size w x w X e E {0, ..., h+3 CN
10:
Sample uniformly
11:
for j I,.... e do Uniform( -22, 22)
12:
13:
1o-1++ .t = p 1aexh'
14:
end for
15:
Trew nnw
Clip
- +s| (T) + 8)
16:
Lattock (F ), Te5 0)
if nnew
l" then
18:
neew
nnow
20:
end if
end for
22:
end for
23:
return Adversarial image aqdv Z/
18

---

Under review as a conference paper at ICLR 2022
Algorithm Lo Norm Adaptive FAB
Input: Test images B (3 }: Step size 7; Iterations T; Intervals u; Adversarial budget E; Trained parameters of the network 80; O'max- ?), B.
Output: Adversarial images Badv { }
Add random noise to 2; in B and get B'
2: +00
3: for to T do
4: i mod = 0 then
5: Get final parameters 9T by taking B' as input image for Algorithm 8T end if
6:
7:
for Jy in B' do
8:
Grad (T))
V F(T 0),
(F( ,(.
9:
8 = arg mill
[G Cradd
8 proj (T Tay
11:
orig proj (Ti, T., C)
||6
min
||5*|| g Omax
[0
N proje: (1 (3) 178* + aaT; 178nii))
14:
if T is not classified as y; then
15:
13 Tolloo < v then
16:
| F.
17:
18:
end
19:
Ty (1 f) BT%
20:
end if
21:
end for
22:
end for
return Adversarial image zadv
19


Reviews:
[{"review_id": "ssiWpeRUE3R", "reviewer": "Reviewer_VL4k", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In summary, there isn't much to critique about this paper. The exposition was clear, barring a number of typos/grammatical errors which should be resolved in the revision. While the white-box version of the adaptive attack was unconvincing due to the gradient estimate used, the authors supplemented these results with a black-box attack that does not rely on local gradient information. Overall, a well executed paper. My only remaining concern is w.r.t. related work, while a number of papers are listed below, a comparison to existing work leveraging test-time training for adversarial robustness would be warranted. \n\nTest-time training applied to adversarial defense: https://arxiv.org/abs/2105.08714.  \nExplored addition of unlabeled data for adversarial training: https://arxiv.org/abs/2010.03593.  \nApplied self-supervised learning technique for robust learning: https://openreview.net/forum?id=bgQek2O63w\n\n- Some typos, e.g. \"As SGD is more efficient more large amount of data...\"\n- \"...we use SGD to optimize $\\theta$ when b is large (e.g. offline setting).\" Please explicitly state what's done when b is small. \n- \"... strengthens the correlation between the self-supervised and classification tasks\" what evidence was provided that meta-adversarial training strengthens the correlation between the self-supervised and classification tasks? The analysis showed that if these tasks are correlated, self-supervised fine-tuning will help, but I do not see where this particular statement in the abstract is supported. \n\n", "summary_of_the_paper": "Propose to improve the generalization and robust accuracy of adversarially-trained networks via self-supervised test-time fine-tuning. Introduce a meta adversarial training method to find a good starting point for test-time fine-tuning, which  incorporates the test-time fine-tuning procedure into the training phase. Observe consistent improvement under different attack strategies for both white-box and black-box attacks.", "summary_of_the_review": "Paper is well-written and lacks any obvious concerns regarding the content, other than the lack of comparison against existing methods for leveraging test-time training for adversarial defence.  ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "XhRRw_d9XIp", "reviewer": "Reviewer_wsz9", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The test-time fine tuning method for better adversarial robustness performance is very interesting. Recently, the test-time training techniques have gained popularity mainly due to their improved performance and adaptability to different domains. Using this strategy for enhanced adversarial robustness is definitely a nice idea. While the authors presented the method with proper motivation followed by results that shows performance improvement, there are some points I would like to be clarified on as given below.\n\n1)Basic methodology of the proposed work is mainly based on the papers [Hendrycks et al., 2019] (https://arxiv.org/pdf/1906.12340.pdf) and [Sun et al., 2020] (https://arxiv.org/pdf/1909.13231.pdf), the first one was not even mentioned anywhere in the introduction or the related work sections. While the improvements made over these mentioned papers are significant, it's very important to properly explain the previous efforts in this space and position the current work for a better understanding of the actual contribution.\n\n2)The method analysis subsection (3.4) is not well explained in my view. It's not clear to me how a correlation mean, larger than robust accuracy of the adversarially trained network, implies the effectiveness of the proposed method.\n\n3)While the results, presented for 3 different datasets, show good adversarial robustness performances against different attack methods, they were only compared with baselines that are other variations of the proposed method. It's very important to add results of other SOTA adversarial robustness methods that would improve the acceptance of the proposed method.\n\n4)The only drawback of test-time training methods could be arguably the increased inference time. As adversarial training itself is a highly time-consuming method mainly for the inner maximization i.e. generating the adversarial images, it's important to show the time required for the proposed test-time fine tuning for adversarially robust methods.", "summary_of_the_paper": "This paper presents a nice framework of test-time fine tuning through self-supervision for adversarially trained networks with the purpose of improving the robust accuracy on test data. A meta adversarial training strategy is also proposed that strengthens the correlation between the self-supervised and classification tasks to provide a good starting point for test-time fine tuning. The proposed method improved robustness performance against different attack methods including adaptive attacks where the attacker has knowledge of the fine tuning technique.", "summary_of_the_review": "An interesting work that tried to improve adversarial robustness through test-time fine tuning using self-supervision techniques. Answering/commenting on the points, that I posted under the main review, can significantly improve the quality of this work.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "JMPnsTPRxMf", "reviewer": "Reviewer_itcW", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n========\nRobust overfitting of adversarial training is a well known problem and thus require separate attention , in that context the paper is well motivated. The results are good in the sense that with Meta AT the accuracy improvement is consistent.\n\nWeaknesses:\n===========\n1. I am not sure whether it is at all feasible to have the test samples even without labels before head to do self-supervised fine tuning. The assumption of fine tuning on test mini batch thus definitely limits the scope of applications of the proposed method.\n\n2. Test time training (TTT) as a fine tuning for a full test set with the full model might be extremely in efficient and compute heavy that many inference platforms can't support. \n\n3. Though understand what the variable is,  please be specific about the definition of $\\epsilon$, the size of budget is technically not correct. \n\n4. The difference that the authors mentioned from [1], looks like disadvantage to me, as the original paper did incremental fine tuning, whereas the current paper does fine tuning starting from same weights, for each image, The reasoning to take this different strategy is unclear. Also, if the batch size is >1, how does the author manage to start from same $\\theta^0$ for each image of a batch? Please also mention what the $\\mathcal{L}_{SS}$ loss is? For example, it is definitely not any cross entropy loss. \n\n5. Also, please explicitly mention  the $\\theta_{E}$ and $\\theta_{f}$  definitions that are used for the multi task formulation.\n\n6. How is the author sure about that the $\\theta^0$ yielded by meta learner gives best starting point. I think the author meant near optimal, hence please rephrase your sentences.\n\n7. Eq. 4 uses test samples $\\hat{B}_j$ for SS-loss where as Eq. 7 uses training sample batch for  SS-loss, why? \n\n8. Please do refer and compare with the paper [2] as they seem to solve similar issues of adversarial training.\n\n9. Please provide a detailed analysis of the training and inference time and compute complexity over head due to the meta adversarial training and TTT.\n\n10. How the authors use the same starting $\\theta^0$ for batch size of $B$ > 1 is not clear to me. Please clarify.\n\n11. The comparison with Regular AT and Regular AT + w/o FT in Table 4 is unfair to me, as in these cases the user has no chance to do any fine tuning with the test samples.\n\n\n[1] Test-time training with self supervision for generalization under distribution shifts, ICML 2020.\n[2] Robust Overfitting may be mitigated by properly learned smoothening, ICLR 2021.", "summary_of_the_paper": "The paper introduces a meta adversarial training method to find a good starting point for test-time fine-tuning. It incorporates the test-time fine-tuning procedure into the training phase and strengthens the correlation between the self-supervised and classification tasks. Results on CIFAR-10, STL10 and Tiny-ImageNet show consistent robustness improvement.", "summary_of_the_review": "The authors proposes a costly training and slow fine tune based testing strategy to get rid of robust overfitting issues associated with AT. In this context the authors should justify the increased cost of training/inference with gain in performance trade-off. The authors should also compare with at least the one relevant paper I mentioned.  I can comment on the overall incremental novelty once I get to know the purpose of doing few things differently. For example the difference between TTT an proposed solution is less, and I am not convinced the $\\theta^0$ initiation is necessary always. Please justify.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "02BV-ZLj-We", "reviewer": "Reviewer_tnn3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n- **Optimization for Defense is a Worthy Topic**. Attacks optimize against defenses so it is entirely sensible that research should explore defenses that optimize back. The proposed method in particular addresses not just the test-time fine-tuning to do so, but a training-time preparation to do so by meta-learning.\n- **Standard Experimental Design**. The choice of attacks comprise the standard AutoAttack benchmark. There are standard choices of dataset (CIFAR-10) and architectures (wide resnet, standard resnet) that are common for work on adversarial defense. The baselines are sensible, and separate the effects of adversarial training, multi-task training with self-supervision, and the meta-learning variant of adversarial training.\n- **Method Variants**. The experiments cover different scopes, like online or offline optimization, and ablations of the loss components (self-supervision and regularization) and meta adversarial training. The different parts of the proposed method all help. Furthermore, offline optimization on the full test set improves over online optimization on a portion of it, which could be a positive if the fine-tuned model transfers to more attacked data without further optimization.\n- **Popular Choice of Self-Supervision**: rotation prediction is a common and effective choice for use and benchmarking (the Gidaris et al. 2018 paper on it is cited 1000+ times).\n\nWeaknesses\n- **There are Prior Test-Time Defenses**. There is existing work on test-time optimization of the model parameters (runtime masking and cleansing ICML'20), of model latents (Schott et al. ICLR'19), and of the input (SOAP/Online Adversarial Purification based on Self-Supervision ICLR'21). The last method is especially related as it optimizes self-supervised tasks at test-time to improve robustness. Apart from this recent wave, there is prior work on randomization at test-time (Raff et al., 2019; Cohen et al., 2019; Pang et al., 2020, Dhillon et al., 2018) and projecting inputs onto generative model distributions (Song et al., 2018; Samangouei et al., 2018; Hill et al., 2021). Although it could be argued this work differs from most in optimizing the model parameters, and not the input or latent variables, the point is somewhat moot because the optimization is episodic so parameter updates do not persist and affect inference on other data. None of these prior methods are included as baselines, but at least SOAP should be, as it also optimizes self-supervised tasks during testing.\n- **Computational Cost**. Optimization during inference is more intensive than a mere forward pass of a static method. Even more expensive still is joint optimization on batches of training data and test data, as done by this method. Other test-time defenses only require test data (SOAP, https://arxiv.org/abs/2103.14347, https://arxiv.org/abs/2105.08714), or small amounts of auxiliary data (https://arxiv.org/abs/2103.14222). This expense could be justified by the amount of robustness gained, but the ratio here is not reported.\n- **Main Attacks are for Static Defenses**. AutoAttack is a strong suite of attacks, but it is designed with _static_ defenses in mind, and the corresponding RobustBench benchmark only includes defenses that are deterministic and do not optimize during inference. With this in mind, improving robustness on this benchmark is necessary but not sufficient to show that a test-time defense is better. While one adaptive attack is provided, more should be done to heed well-established warnings about obfuscated gradients (Athalye et al. ICML 2018). Possibilities include a transfer attack from the static adversarial defense, an expectation attack on one test input with several samples of training and testing batches, and evaluation with a decision-based attack like boundary.\n- **Narrow Look at Self-Supervision**. Rotation prediction is a popular task, but it is just one. Vertical flip prediction is so closely related that it is not so informative as a second task. A generic task, like reconstruction/auto-encoding, or a state-of-the-art task, like contrastive learning, would be more informative for the utility of self-supervision for defense.\n\nFor Rebuttal\n- Please discuss the contributions of this work relative to the cited test-time defenses, and SOAP in particular https://openreview.net/forum?id=_i3ASPp12WS. Ideally this discussion could compare the results for optimizing the input, like SOAP, vs. optimizing the model parameters, like this work.\n- Please comment on the computational requirements of this defense. How much more expensive is it in time and FLOPs than a single forward pass?\n- Please discuss the obfuscated gradients checklist from Athalye et al. 2018 and cover how this work cannot be explained away as gradient obfuscation.\n- Please experiment with a decision based attack, such as the boundary attack.\n\nMiscellaneous Feedback\n- [related work] Alongside the already published references mentioned above, there are a number of contemporary works published or posted to arxiv this year, which this work could discuss and compare against to strengthen its scope and evaluation: https://arxiv.org/abs/2103.14222, https://arxiv.org/abs/2103.14347, https://arxiv.org/abs/2105.08714, https://arxiv.org/abs/2106.04938, https://openreview.net/forum?id=RFGkzxMFqby, https://openreview.net/forum?id=36rU1ecTFvR, https://openreview.net/forum?id=3Uk9_JRVwiF", "summary_of_the_paper": "Fine-tuning the parameters of an adversarially-trained model at test-time with self-supervised tasks improves robustness to standard adversarial attacks. By incorporating meta-learning into the adversarial training stage, the test-time fine-tuning achieves marginally higher accuracy (though not reduced optimization time, or at least this is not reported). The test-time optimization approach requires the training data be kept available for joint updates on training batches and test data. The self-supervised tasks considered are rotation prediction, a common choice, and vertical flip recognition, which is a close relative of it. Results are reported on the standard adversarial defense benchmark of CIFAR-10, and additionally models are also evaluated on STL and Tiny ImageNet. Design choices are ablated and an adaptive attack is designed and experimented with, but there other checks against obfuscated gradients (for instance those suggested by Athalye et al. 2018) are not pursued, such as attacks in expectation, or decision-based attacks. No comment is made about the computational cost of test-time fine-tuning.", "summary_of_the_review": "The contributions are summarized as (1) introducing the framework of test-time fine-tuning for defense, (2) meta adversarial training, and (3) the adaptive attack against the proposed defense. However, the first and most novel contribution (1) needs qualification, as there is substantial prior work on test-time optimization for defense, and none are included as baselines. This lack of discussion and experiment makes the work incomplete. (2) is indeed a contribution. On the other hand (3) is more a requirement for proposing a defense than an independent contribution. As outlined in the main review, more could be done here to evaluate that the defense is more robust and not superficially interfering with attacks. As there is a wave of such defenses at present, it is important to verify improvement on what has already been done, and to work toward common evaluations techniques across them. As such, I encourage the authors to further test the proposed defense, but must recommend rejection of this edition of the work.\n\n**Final Review** The rebuttal responded to the weaknesses of computational cost and the need for more attack types to assess gradient obfuscation but only partially addressed the prior and concurrent work on test-time optimization for defense. On the plus side, the potential issue of gradient masking is resolved by evaluation of a decision attack and reporting that robustness fails as the adversarial budget increases. However, on the minus side the computational cost of the method is significant and the transfer attack reduces the improvement in robustness from fine-tuning. More importantly, the response does not acknowledge the prior method of runtime masking and cleansing, which likewise updates model parameters. Furthermore, because the parameter updates in this work are _episodic_, it is perhaps not as different from purification methods as claimed. More work is needed to situate the contributions with respect to prior defenses, such as analysing when the model parameters (for this method) or input perturbation (for purification methods) is shared/unshared across the batch.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a method to improve the robust accuracy of classifiers using test-time training.  The reviewers all agree that the method is interesting, and many reviewers had a positive view of the method.  However, two main criticisms remain: (i) the method increases the runtime of inference, and (ii) comparisons to other related methods were lacking.  The authors responded to (i) by reporting runtimes for their method in the rebuttal.  Some reviewers were concerned that the runtime increase of the method is not acceptable, however I am not very concerned with this issue since I think the paper contains an interesting methodology even if it’s not ready for deployment at the industrial scale.  However, issue (ii) does not seem to have been adequately addressed.  The comparison to SOAP is a welcome addition the reviewers acknowledge, but a number of other methods, for example masking and cleansing, are closely related (but different) and so comparisons should be provided.
Decision: Reject