I MPROVING A DVERSARIAL D EFENSE WITH S ELFSUPERVISED T EST - TIME F INE - TUNING
Anonymous authors
Paper under double-blind review
A BSTRACT
Although adversarial training and its variants currently constitute the most effective way to achieve robustness against adversarial attacks their poor generalization limits their performance on the test samples
In this work we propose to im prove the generalization and robust accuracy of adversarially trained networks via self-supervised test- time ﬁne- tuning
To this end
, we introduce a meta adversarial training method to ﬁnd a good starting point for test- time ﬁne- tuning
It incorpo
- rates the test- time ﬁne-tuning procedure into the training phase and strengthens the correlation between the self-supervised and classiﬁcation tasks.
The extensive experiments on CIFAR10, STL10 and Tiny ImageNet using different self-supervised tasks show that our method consistently improves the robust accuracy under different attack strategies for both the white-box and black-box attacks
I NTRODUCTION
Adversarial training and its variants (Madry et al., 2018; Wang et al., 2019; Zhang et al., 2019) are currently recognized as the most effective defense mechanism against adversarial attacks.
However, adversarial training generalizes poorly; the robust accuracy gap between the training and test set in adversarial training is much larger than the training-test gap in standard training of deep networks (Neyshabur et al., 2017; Zhang et al., 2017).
Unfortunately, classical techniques to overcome overﬁtting in standard training, including regularization and data augmentation, only have little effect in adversarial training (Rice et al., 2020).
The poor generalization of adversarial training is a consequence of adversarial test examples lying far away from the training set (Huan et al., 2019).
To adapt to these examples, we propose to ﬁnetune the network for each test mini-batch.
Since the labels of the test images are not available, we exploit self-supervision, which is widely used in the standard training of networks (Chen et al., 2020b;
Gidaris et al., 2018;
He et al., 2020).
Fine-tuning self-supervised tasks is a substitute of ﬁne-tuning the classiﬁcation loss at the inference time.
Thus, minimizing this self-supervised loss function yields better generalization on the test set.
To make our test-time ﬁne-tuning strategy effective, we need to search for a good starting point during training.
As will be shown in our experiments, adversarial training does not provide the optimal starting point.
We therefore formulate the search for the network parameters that can be effectively ﬁne-tuned as a bilevel optimization problem.
Speciﬁcally, we introduce a meta adversarial training strategy dedicated to our self-supervised ﬁne-tuning inspired by the model-agnostic metalearning (MAML) framework (Finn et al., 2017).
To this end, we treat the classiﬁcation of each batch of adversarial images as one task and minimize the corresponding classiﬁcation error of the self-supervised ﬁne-tuned network.
This meta adversarial training strategy strengthens the correlation between the self-supervised and classiﬁcation tasks so that self-supervised test-time ﬁne-tuning further improves the robust accuracy.
In order to reliably evaluate our method, we follow the suggestions of (Tramer et al., 2020) and design an adaptive attack strategy that is fully aware of the test-time ﬁne-tuning.
We empirically demonstrate the effectiveness of our method on the commonly used CIFAR10 (Krizhevsky et al., 2009),
STL10 (Coates et al., 2011) and Tiny ImageNet (Le & Yang, 2015) datasets under both standard (Andriushchenko et al., 2020; Croce & Hein, 2020a;
Madry et al., 2018) and adaptive attacks in the scenarios of both white-box and black-box attacks.
The experiments evidence that our method consistently improves the robust accuracy under all attacks.
Our contributions can be summarized as follows:
We introduce the framework of self-supervised test-time ﬁne-tuning for adversarially-trained networks and show that it improves the robust accuracy of the test data.
We propose a meta adversarial training strategy based on the MAML framework to ﬁnd a good starting point and strengthen the correlation between the self-supervised and classiﬁcation tasks.
Following the suggestions of (Tramer et al., 2020), we design an adaptive attack strategy that is fully aware of our test-time ﬁne-tuning.
The experiments show that our approach is valid on diverse attack strategies, including standard and adaptive ones in both the white-box and black-box attacks.
Our code is available through this link.
R ELATED W ORK
Adversarial Training.
In recent years, many approaches have been proposed to defend networks against adversarial attacks (Guo et al., 2018;
Liao et al., 2018; Song et al., 2018).
Among them, adversarial training (Madry et al., 2018) stands out as one of the most robust and popular methods, even under various strong attacks (Athalye et al., 2018; Croce & Hein, 2020a).
Adversarial training optimizes the loss of adversarial examples to ﬁnd parameters that are robust to adversarial attacks.
Several variants of adversarial training have been proposed (Wang et al., 2019; Zhang et al., 2019), and their performance is similar to the original version of adversarial training (Rice et al., 2020).
One important problem that limits the robust accuracy of adversarial training is overﬁtting.
Compared with training on clean images, the gap of robust accuracy between the training and test set is much larger in adversarial training (Rice et al., 2020).
Moreover, traditional techniques to prevent overﬁtting, such as data augmentation, have little effect.
Recently, some methods have attempted to ﬂatten the weight loss landscape to improve the generalization of adversarial training.
In particular, Adversarial Weight Perturbation (AWP) (Wu et al., 2020) achieves this by designing a double-perturbation mechanism that adversarially perturbs both inputs and weights.
In addition, learning-based smoothing can ﬂatten the landscape and improve the performance (Chen et al., 2021).
Self-supervised Learning.
In the context of non-adversarial training, many self-supervised strategies have been proposed, such as rotation prediction (Gidaris et al., 2018), region/component ﬁlling (Criminisi et al., 2004), patch-base spatial composition prediction (Trinh et al., 2019) and contrastive learning (Chen et al., 2020b; He et al., 2020).
While self-supervision has also been employed in adversarial training (Chen et al., 2020a; Kim et al., 2020; Yang & Vondrick, 2020; Hendrycks et al., 2019), their methods only use self-supervised learning at training time to regularize the parameters and improve the robust accuracy.
By contrast, we propose to perform self-supervised ﬁne-tuning at test time, which we demonstrate to signiﬁcantly improve the robust accuracy on test images.
As will be shown in the experiments, the self-supervised test-time ﬁne-tuning has larger and complementary improvements over the training time self-supervision.
Test-time Fine-tuning.
Test-time ﬁne-tuning has been used in various ﬁelds, such as image superresolution (Shocher et al., 2018) and domain adaption (Sun et al., 2020; Wang et al., 2021).
While our work is thus closely related to Test-Time Training (TTT) in (Sun et al., 2020), we target a signiﬁcantly different scenario.
TTT assumes that all test samples have been subject to the same distribution shift compared to the training data.
As a consequence, it incrementally updates the model parameters when receiving new test images.
By contrast, in our scenario, there is no systematic distribution shift, and it is therefore more effective to ﬁne-tune the parameters of the original model for every new test mini-batch.
This motivates our meta adversarial training strategy, which searches for the initial model parameters that can be effectively ﬁne-tuned in a self-supervised manner.
M ETHODOLOGY
We follow the traditional multi-task learning formulation (Caruana, 1997) and consider a neural network with a backbone z = E ( x ; θ E ) and K + 1 heads.
One head f ( z ; θ f ) outputs the classiﬁcation result while the other K heads g 1 ( z ; θ g 1 ) , ..., g K ( z ; θ gK ) correspond to K auxiliary self-supervised tasks.
θ =
( θ E , θ f , θ g 1 , · · · , θ gk ) encompasses all trainable parameters, and we further deﬁne
[Equation]
Furthermore, let D = { ( x 1 , y 1 ) , ( x 2 , y 2 ) , · · · , ( x n , y n ) } denote the training set containing n images with corresponding labels, and (cid:101)
D
= { ( (cid:101) x 1 , (cid:101) y 1 ) , ( (cid:101) x 2 , (cid:101) y 2 ) , · · · , ( (cid:101) x m , (cid:101) y m ) } be the test set containing m images.
For further illustration, the labels of the test set are shown.
However, they are unknown to the networks at test time.
We denote the adversarial examples of x and (cid:101) x as x adv and (cid:101) x adv , respectively.
They satisfy the constraints (cid:107) x adv − x (cid:107) ≤ ε and (cid:107) (cid:101) x
adv − (cid:101) x (cid:107) ≤ ε , and ε is the size of the adversarial budget.
For any set S , we represent the average loss over the elements in S as
[Equation]
where | S | is the number of elements in S .
We use F ( · ; θ ) and L ( · ; θ ) to denote a network F and a loss function L parametrized by θ , but we often omit the symbol θ to simplify the notation.
The general classiﬁcation loss, such as the cross-entropy, is denoted by L cls .
We use the superscript “AT” to denote the adversarial training loss.
For example, we deﬁne
[Equation]
Because of the poor generalization of adversarial training, the adversarially-trained model may not give the correct label for a large portion of the adversarial test examples.
That is, for a test sample (cid:101) x
adv with ground-truth label (cid:101)
y , we may have arg max j F ( (cid:101) x
adv ; θ ) j (cid:54) =
(cid:101) y , where F ( (cid:101) x adv ; θ ) j denotes the output probability of the j -th class.
Since we do not know the label (cid:101) y at test time, we cannot directly optimize the classiﬁcation loss L cls ( F ( (cid:101) x adv ) , (cid:101) y ) .
We therefore use self-supervision as a substitute to ﬁne-tune the backbone E .
We provide more details on our self-supervised test-time ﬁne-tuning scheme in Section 3.1 and introduce our meta adversarial training method in Section 3.2.
In Section 3.3, we discuss the speciﬁc self-supervised tasks used in the experiments, and further analyze our algorithm in Section 3.4.
3. T EST - TIME F INE - TUNING
Our goal is to perform self-supervised learning on the test examples to overcome the overﬁtting problem of adversarial training.
To this end, let us suppose that an adversarially-trained network with parameters θ 0 receives a mini-batch of b adversarial test examples (cid:101) B adv =
{ ( (cid:101) x adv 1 , (cid:101) y 1 ) , ( (cid:101) x adv 2 , (cid:101) y 2 ) , · · · , ( (cid:101) x advb , (cid:101) y b ) } , As the labels { (cid:101)
y i } bi =1 are not available, we propose to ﬁne-tune the backbone parameters θ E by optimizing the loss function
[Equation]
which encompasses K self-supervised tasks.
Here, L SS,k represents the loss function of the k -th task and { λ k } Kk =1 are the weights balancing the contribution of each task.
The number of images b may vary from 1 to m .
b = 1 corresponds to the online setting, where only one adversarial image is available at a time, and the backbone parameters θ E are adapted to every new image.
The online setting is the most practical one, as it does not make any assumptions on the number of adversarial test images the network receives.
By contrast, b = m corresponds to the ofﬂine setting where all adversarial test examples are available at once.
It is similar to the transductive learning (Gammerman et al., 1998; Vapnik, 2013).
Note that our online setting differs from the online test-time training described in TTT (Sun et al., 2020); we do not incrementally update the network parameters as new samples come, but instead initialize ﬁne-tuning from the same starting point θ 0 for each new test image.
Eqn (4) encourages θ E to update in favor of the self-supervised tasks.
However, as the classiﬁcation head f was only optimized for the old backbone E ( · ; θ 0 E ) , it will typically be ill-adapted to the new parameters θ ∗ E , resulting in a degraded robust accuracy.
Furthermore, for a small b , the model tends to overﬁt to the test data, reducing L SS to 0 but extracting features that are only useful for the self-supervised tasks.
To overcome these problems, we add an additional loss function acting on the training data that both regularizes the backbone E and optimizes the classiﬁcation head f
so that f remains adapted to the ﬁne-tuned backbone E ( · ; θ ∗ E ) .
Speciﬁcally, let B ⊂ D denote a subset of the training set.
We then add the regularizer
[Equation]
to the ﬁne-tuning process.
In short, Eqn (5) evaluates the adversarial training loss on the training set to ﬁne-tune the parameters θ f of the classiﬁcation head.
It also forces the backbone E to extract features that can be used to make correct prediction, i.e., to prevent θ E from being misled by L SS when b is small.
Combining Eqn (4) and Eqn (5), our ﬁnal test-time ﬁne-tuning loss is
[Equation]
where λ sets the inﬂuence of L R .
Algorithm 1 describes our test-time self-supervised learning algorithm in detail.
As SGD is more efﬁcient for more large amount of data, we use SGD to optimize θ when b is large ( e.g. ofﬂine setting).
This version of algorithm is deferred to the Appendix B. a batch of training images B ⊂ D 3:
Find adversarial x advi of training image x i ∈ B by PGD attack.
Calculate L test in Eqn (6) 5: θ t
= θ t − 1 − η ∇ θ t − 1 L test ( (cid:101) B
adv , B ; θ t − 1 ) 6: end for 7: return Prediction (cid:98) y
i = arg max j F ( (cid:101) x advi ; θ T )
j
3. M ETA A DVERSARIAL T RAINING
To make the best out of optimizing L test at test time, we should ﬁnd suitable starting point θ 0 , i.e., a starting point such that test-time self-supervised learning yields better robust accuracy.
We translate this into a meta learning scheme, which entails a bilevel optimization problem.
Speciﬁcally, we divide the training data into s small exclusive subsets D = ∪ sj =1 B j and let B advj to be adversaries of B j .
We then formulate meta adversarial learning as the bilevel minimization of
[Equation]
where L SS is the self-supervised loss function deﬁned in Eqn (4) and L ATcls is the loss function of adversarial training deﬁned in Eqn (3).
As bilevel optimization is time-consuming, following
MAML (Finn et al., 2017), we use a single gradient step of the current model parameters θ to approximate θ ∗ j .
That is, we compute
[Equation]
In essence, this meta adversarial training scheme searches for a starting point such that ﬁne-tuning with L SS will lead to a good robust accuracy.
If this holds for all training subsets, then we can expect the robust accuracy after ﬁne-tuning at test time also to increase.
Note that, because the meta learning objective of Eqn (7) already accounts for classiﬁcation accuracy, the regularization by L R is not needed during meta adversarial learning.
Accelerating Training.
To compute the gradient ∇ θ L meta ( D ; θ ) , we need to calculate the timeconsuming second order derivatives − α ∇ 2 θ L SS ( B advj ; θ ) ∇ θ ∗ j L ATcls ( B j ; θ ∗ j ) .
Considering that adversarial training is already much slower than standard training (Shafahi et al., 2019), we cannot afford another signiﬁcant training overhead.
Fortunately, as shown in (Finn et al., 2017), second order derivatives have little inﬂuence on the performance of MAML.
We therefore ignore them and take the gradient to be
[Equation]
However, by ignoring the second order gradient, only the parameters on the forward path of the classiﬁer F , i.e. , θ E and θ f , will be updated.
In other words, optimizing Eqn (7) in this fashion will not update { θ gk } Kk =1 .
To nonetheless encourage each self-supervised head G k to output the correct prediction, we incorporate an additional loss function encoding the self-supervised tasks,
[Equation]
Note that we use the adversarial version of L SS to provide robustness to the self-supervised tasks, which, as shown in (Chen et al., 2020a; Hendrycks et al., 2019; Yang & Vondrick, 2020), is beneﬁcial for the classiﬁer.
The ﬁnal meta adversarial learning objective therefore is
[Equation]
) where λ (cid:48) balances the two losses.
Algorithm 2 shows complete meta adversarial training algorithm.
images B 1 , B 2 , · · · , B q ⊂ D 3:
Using PGD to ﬁnd the adversaries B advj : x advj,i = arg max (cid:107) x advj,i − x j,i (cid:107)≤ ε L cls ( F ( x advj,i ) , y j,i ) 4: for batches B 1 , B 2 , · · · , B q do 5: θ ∗ j = θ − α ∇ θ L SS ( B advj ; θ ) 6: l meta,j = L ATcls ( B j ; θ ∗ j ) 7: end for 8: θ =
[Equation]
(cid:80)
[Equation]
j (cid:104) ∇
[Equation]
(cid:48) ∇ θ
[Equation] [Equation]
; θ ) (cid:105) 9: end for 10: return Trained parameters θ 0 = θ
3. S ELF - SUPERVISED T ASKS
In principle, any self-supervised tasks can be used for test-time ﬁne-tuning, as long as they are positively correlated with the robust accuracy.
However, for the test-time ﬁne-tuning to remain efﬁcient, we should not use too many self-supervised tasks.
Furthermore, as we aim to support the fully online setting, where only one image is available at a time, we cannot incorporate a contrastive loss (Chen et al., 2020b;
He et al., 2020; Kim et al., 2020) to L SS .
In our experiments, we therefore use two self-supervised tasks that have been shown to be useful to improve the classiﬁcation accuracy: Rotation Prediction and Vertical Flip Prediction .
Rotation Prediction is a widely used self-supervision task proposed in (Gidaris et al., 2018) and has been employed in adversarial training as an auxiliary task to improve the robust accuracy (Chen et al., 2020a;
Hendrycks et al., 2019).
Following (Gidaris et al., 2018), we create 4 copies of the input image by rotating it with Ω = { 0 ◦ , 90 ◦ , 180 ◦ , 270 ◦ } .
The task then consists of a 4-way classiﬁcation problem, where the head g rotate aims to predict the correct rotation angle.
The loss for an image x is the average cross-entropy over the 4 copies, given by
[Equation]
where x ω is the rotated image with angle ω ∈ Ω , G rotate = g rotate ◦ E denotes the classiﬁer for rotation prediction, and G rotate ( · ) ω is the predicted probability for the ω angle.
The head g rotate is a fully-connected layer followed by a softmax layer.
[Table]
Vertical
[Table]
Vertical Flip (VFlip) Prediction is a self-supervised task similar to rotation prediction and has also been used for self-supervised learning (Saito et al., 2020).
In essence, we make two copies of the input image and ﬂip one copy vertically.
The head g vﬂip then contains a 2-way fully-connected layer followed by a softmax layer and predicts whether the image is vertically ﬂipped or not.
The corresponding loss for an image x is
[Equation]
(13) where V = { ﬂipped , not ﬂipped } is the operation set and G vﬂip =
g vﬂip ◦ E .
x v denotes and transformed input and G vﬂip ( · ) v is the probability of operation v .
Note that we do not ﬂip the image horizontally as it is a common data augmentation technique and classiﬁers typically seek to be invariant to horizontal ﬂip.
3. M ETHOD A NALYSIS
We now analyze why our test-time self-supervised ﬁne-tuning improves the robust accuracy.
To this end, we observe the correlation between the gradient of self-supervised loss L SS and the classiﬁcation loss L cls .
For an input (cid:101) x advi with label (cid:101)
y i , the gradient correlation with respect to the shared parameters θ E can be expressed as
[Equation]
If ρ ( (cid:101) x advi ) is signiﬁcantly larger than 0, then gradient descent w.r.t.
L SS should act as a good substitute for optimizing L cls .
Let us consider a small step η in the direction of ∇ θ E L SS ( (cid:101) x advi ) .
Then, the classiﬁcation loss L cls can be approximated by Taylor expansion as
[Equation]
As θ E contains millions of parameters, the gradient norm is typically large, even a small η will decrease the classiﬁcation loss when ρ ( (cid:101) x advi ) is signiﬁcantly larger than 0.
Below, we further conﬁrm this empirically.
We choose cross-entropy as the classiﬁcation loss.
For all adversarial test inputs (cid:101) x adv ∼ (cid:101) D adv , where (cid:101)
D adv is the adversaries of the test data, we regard ρ ( (cid:101) x adv ) as a random variable and calculate its empirical statistics on the test dataset.
Tableshows the empirical statistics of an adversarially-trained model on CIFAR10, and Figure 1 shows the cumulative distribution of ρ ( (cid:101) x adv ) .
The mean of ρ ( (cid:101) x adv ) is indeed signiﬁcantly larger than 0 and
P ( ρ ( (cid:101) x adv ) > 0) is larger than the robust accuracy of the adversarially-trained network (50%-60%), which implies that self-supervised test-time ﬁne-tuning helps to correctly classify the adversarial test images.
A DAPTIVE A TTACKS
In the white-box attacks, the attacker knows every detail of the defense method.
Therefore, we need to assume that the attacker is aware of our test-time ﬁne-tuning method and will adjust its strategy for generating adversarial examples accordingly.
Here, we discuss one such strong adaptation strategy targeted to our method.
Suppose that the attacker is fully aware of the hyperparameters for test-time ﬁne-tuning.
Then, ﬁnding adversaries (cid:101) B adv of the clean subset (cid:101) B can be achieved by maximizing the adaptive loss
[Equation]
) where L attack refers to the general attack loss, such as the cross-entropy or the difference of logit ratio (DLR) (Croce & Hein, 2020a).
Let θ T be the ﬁne-tuned test-time parameters using Algorithm 1.
At the k -th step of the attack, it depends on the input (cid:101) B ( k
[Equation]
= { ( (cid:101) x
[Equation]
j , (cid:101) y
[Equation]
bj
[Equation]
1 via the update
[Equation] [Equation]
where L test and B are the loss function and subset of training images mentioned in Eqn (6).
As θ T is a function of the input (cid:101) B ( k ) , we can calculate the end-to-end gradient of (cid:101) x
( k ) i ∈ (cid:101) B ( k ) as ∇ (cid:101) x
( k ) i
L attack ( F ( (cid:101) x
( k ) i )
; θ T ( (cid:101) B ( k ) )) .
However, θ T goes through T gradient descent steps, and thus calculating the gradient ∇ (cid:101) x ( k )
i θ T ( (cid:101) B ( k ) ) requires T -th order derivatives of the backbone E , which is virtually impossible if T or the dimension of θ E is large.
We therefore approximate the gradient as
[Equation]
which treats θ T as a ﬁxed variable so that high-order derivatives from θ T ( (cid:101) B ( k ) ( (cid:101) x ( k ) i )) can be avoided.
Although this approximation makes Grad( (cid:101) x
( k ) i ) inaccurate, common white-box attacks use projected gradients, which are robust to such inaccuracies.
For example, PGD only uses the sign of the gradient under an (cid:96) ∞ adversarial budget.
Note that solving the maximization in Eqn (16) does not necessarily require calculating the gradient Grad( (cid:101) x ( k ) i ) .
For instance, we will also use Square
Attack (Andriushchenko et al., 2020), a strong score-based black-box attack, to maximize Eqn (16) and generate adversaries for (cid:101) B .
As another approximation to save time, one can also ﬁxing θ T for several iterations.
This leverages the intuition that attack strategies often make small changes to the input (cid:101) x , and thus, for the intermediate images in the k -th and ( k + 1 )-th steps, θ T ( (cid:101) B ( k ) ) and θ T ( (cid:101) B ( k +1) ) should be close.
Therefore, a general version of our adaptive attacks only updates θ T every u iterations, with u a hyperparameter.
We provide the algorithms for PGD, AutoPGD, FAB and Square Attack in the Appendix C.
E XPERIMENTS
Experimental Settings.
Following previous works (Cui et al., 2020; Huang et al., 2020), we consider (cid:96) ∞ -norm attacks with an adversarial budget ε = 0 .
031 ( ≈ 8 / 255 ).
We evaluate our method on three datasets: CIFAR10 (Krizhevsky et al., 2009), STL10 (Coates et al., 2011) and Tiny ImageNet (Le & Yang, 2015).
We also use two different network architectures: WideResNet-3410 (Zagoruyko & Komodakis, 2016) for CIFAR10, and ResNet18 (He et al., 2016) for STL10 and
Tiny ImageNet.
Following the common settings in adversarial training, we train the network for 100 epochs using SGD with a momentum factor of 0.9 and a weight decay factor of 5 × 10 − 4 .
The learning rate β starts at 0.1 and is divided by a factor of 10 at the 50-th and the 75-th epochs.
The step size α in Eqn (8) the same as β ; the factor λ (cid:48) in Eqn (11) is 1 . 0 .
We set the weight of each self-supervised task λ k = 1 K where K is the number of tasks.
We use 10-iteration PGD with step size 0.007( ≈ 0 .
031 / 4 ) to ﬁnd the adversarial B advj during training.
To prevent the computational burden of meta adversarial training, we use divide the classiﬁcation of 32 images as one task and sample 8 mini-batches B 1 , ..., B 8 in each iteration.
( i.e. | B j | = 32 and q = 8 in Algorithm 2).
More detailed experimental settings are provided in the Appendix B.
Attack Methods.
We consider four common white-box and black-box attack methods: PGD20 (Madry et al., 2018), AutoPGD (Croce & Hein, 2020a), FAB (Croce & Hein, 2020b) and Square
Attack (Andriushchenko et al., 2020).
We apply both the standard and adaptive versions of these methods.
Furthermore, for AutoPGD, we use both the cross-entropy and DLR (Croce & Hein, 2020a) loss.
More details are provided in the Appendix C.
Baselines.
We compare our method with the following methods: 1) Regular adversarial training (Regular AT), which uses L ATcls in Eqn (3).
Regular adversarial training with an additional selfsupervised loss, i.e., using L AT cls + λ (cid:48)
[Table] [Table]
This corresponds to the formulation of (Hendrycks et al.,
[Table] [Table]
[Table]
adversarial learning (Algorithm 2) without test-time ﬁne-tuning.
We include additional ﬁgures, experiments with different adversarial budget and experiments about label leaking in Appendix A.
5. O NLINE T EST - TIME F INE - TUNING
Let us ﬁrst evaluate our method in the online setting, where only one adversarial image is available when ﬁne-tuning the network.
We ﬁne-tune the network for 10 steps with a momentum factor of 0.
and a learning rate η = 5 × 10 − 4 .
We set λ k = 1 K and λ = 15 . 0 .
CIFAR10.
Table 2 shows the robust accuracy for different attacks and using two different tasks for ﬁne-tuning.
The adaptive attack is not applicable to models without ﬁne-tuning.
As we inject different self-supervised tasks into the adversarial training stage, and as different self-supervised tasks may impact the robust accuracy differently (Chen et al., 2020a), the robust accuracy without ﬁne-tuning still varies.
The vertical ﬂipping task yields better robust accuracy before ﬁne-tuning but its improvements after ﬁne-tuning is small.
By contrast, rotation prediction achieves low robust accuracy before ﬁne-tuning, but its improvement after ﬁne-tuning is the largest.
Using both tasks together combines their effect and yields the highest overall accuracy after test-time ﬁne-tuning.
Note that our self-supervised test-time ﬁne-tuning, together with meta adversarial learning, consistently improves the robust accuracy under different attack methods.
Under the strongest adaptive
AutoPGD, test-time ﬁne-tuning using both tasks achieves a robust accuracy of 57.70%, signiﬁcantly outperforming regular adversarial training.
STL10 and Tiny ImageNet.
As using both the rotation and vertical ﬂip prediction led to the highest overall accuracy on CIFAR10, we focus on this strategy for STL10 and Tiny ImageNet.
Table 3 and 4 shows the robust accuracy on STL10 and Tiny ImageNet using a ResNet18.
Our approach also signiﬁcantly outperforms regular adversarial training on these datasets.
[Table] [Table]
[Table]
5. O FFLINE T EST - TIME F INE - TUNING
Let us now study the ofﬂine setting, where all test adversarial examples are available at test time.
We use SGD to ﬁne-tune the model for 10 epochs.
We set λ
= 10 .
and keep the other settings the same as for online ﬁne-tuning.
As shown in Table 2, 3, 4, the ofﬂine ﬁne-tuning further improves the robust accuracy over the online version.
5. A BLATION S TUDY
Removing L SS or L R .
In our previous experiments, test-time ﬁne-tuning was achieved using a combination of two loss functions: L SS and L R .
To study the effect of each of these terms separately, we remove either one of them from L test .
In Table 5, we report the robust accuracy after online ﬁne-tuning using only L R and only L SS .
While, as expected, removing L SS tends to reduce more accuracy than removing L R .
It shows the beneﬁts of our self-supervised test-time ﬁne-tuning strategy.
Nevertheless, the best results are obtained by exploiting both loss terms.
Removing Meta Adversarial Training.
Our meta training strategy in Algorithm 2 aims to strengthen the correlation between the self-supervised tasks and classiﬁcation.
To show its effectiveness, we perform an ablation study where we ﬁne-tune the model with regular adversarial training ( i.e. , setting α = 0 in the line 5 of Algorithm 2).
We then perform the same test-time ﬁne-tuning on the model without meta adversarial training, using the same hyperparameters as in the meta adversarial training case.
As shown in Table 6, the robust accuracy and the improvements of ﬁne-tuning are consistently worse without meta adversarial training.
C ONCLUSION
We propose self-supervised test-time ﬁne-tuning on adversarially-trained models to improve their generalization ability.
Furthermore, we introduce a meta adversarial training strategy to ﬁnd a good starting point for our self-supervised ﬁne-tuning process.
Our extensive experiments on CIFAR10,
STL10 and Tiny ImageNet demonstrate that our method consistently improves the robust accuracy under different attack strategies, including strong adaptive attacks where the attacker is aware of our test-time ﬁne-tuning technique.
In these experiments, we utilize three different sources of selfsupervision: rotation prediction, vertical ﬂip prediction and the ensemble of them.
While all settings lead to improved robust accuracy, some self-supervised tasks seem to have more impact than others.
Finding better self-supervised tasks and their combinations will be the focus of our future work.
R EPRODUCIBILITY S TATEMENT
We provide pesudocode of our algorithms and the detailed hyperparameters in the main text and
Appendix.
The datasets we use (CIFAR10, STL10 and Tiny ImageNet) are all publicly available.
In addition, we provide the code to reproduce the experiment results here
E THICS S TATEMENT
Adversarial examples bring security threats to the current machine learning models.
Adversarial examples may be maliciously used to attack the safety-critical systems like autonomous driving and face veriﬁcation.
While adversarial training is effective in defending against adversarial attacks it suffers from poor generalization.
We propose to use the self-supervised test-time ﬁne-tuning to increase the robust accuracy of these models.
It helps build safer machine learning systems.
A A DDITIONAL E XPERIMENTS
Robust Accuracy v.s Fine-tuning Steps.
Figure 2 shows the robust accuracy at each step of the test-time ﬁne-tuning for different self-supervised tasks and attack methods.
When using the standard version of attacks, the robust accuracy gradually increases as ﬁne-tuning proceeds.
When using our adaptive attacks
[Figure]
the adversarial examples are generated to attack the network with θ T ( T = 10 ) instead of θ 0 .
Thus, when the parameters gradually change from θ 0 to θ T , the accuracy drops.
[Figure]
-tuning on CIFAR10.
Accuracy Improvement on Clean Images.
Our model is able to improve not only the robust accuracy but also the natural accuracy of clean images on adversarially-trained models.
To evidence this, we maintain all the components of our model and simply replace the adversarial input images with clean images ( i.e. replacing (cid:101) B adv with clean inputs (cid:101) B in Algorithm 1) and perform the same self-supervised test-time ﬁne-tuning.
As shown in Table 7, our approach increases the clean image accuracy.
This phenomenon further strengthens our conjecture that the improvement of robust accuracy is due to the improvement of generalization instead of perturbing the model parameters, because the randomly perturbing the parameters usually lower the natural accuracy of the model.
This also suggests that our method is applicable to the broader problem of improving the accuracy of deep learning models with poor generalization ability.
[Table]
Accuracy Improvement on Inputs with Different Adversarial Budget.
Our method is also able to improve the robust accuracy of inputs with different adversarial budgets.
As shown in Table 8, we set (cid:96) ∞ budget of the adversarial inputs to be 0 .
015 to perform the online test-time ﬁne-tuning.
The robust accuracy is ed improved.
[Table]
Label Leaking.
One potential concern about our method is whether the effect of label leaking leads to the improvement of the robust accuracy.
The attacker uses the ground truth label to generate the adversarial examples.
Someone may worry it leaks the information of the ground truth label to the adversarial images and the test-time ﬁne-tuning can utilize the information to guess the correct label.
Previous experiments on clean images already show that test-time ﬁne-tuning is effective even if there is no information of the ground truth label.
We further resolve the concern of the label leaking by the following experiments.
The attacker randomly lowers the score of the false label to perform the adversarial attack.
And we conduct the self-supervised test-time ﬁne-tuning on these “adversarial” images.
If our method uses the information of leaked label to improve the robust accuracy, it will predict the false label and reduce the accuracy.
However, as shown in Table 9, the robust accuracy is also improved after the test-time ﬁne-tuning.
It demonstrates that the improvement of robust accuracy is not a result of label leaking.
It is worth noting that such attack is not targeted at the ground truth label.
Therefore, the accuracy of the input images, along with the accuracy improvement of the test-time ﬁne-tuning, are similar to the clean images in Table 7.
[Table]
Comparison with SOAP (Shi et al., 2021).
Our method is different from SOAP as we are ﬁnetuning the model to adapt to new examples instead of purifying the input. We apply SOAP-RP to the adversarially-trained model and ﬁnd that its improvement is marginal. Under AutoPGD, the accuracy is improved from 53.09% to 53.57%. This improvement is much smaller than our method, whose improvement is from 53.09% to 57.93%.
SOAP only has little effect when combining with the commonly-used adversarial training.
Transfer Attack.
In Table 10, we perform a transfer attack from the static adversarial defense.
We use the robust networks with the same architecture as the substitute model, and the test-time ﬁne-tuning also improves the robust accuracy.
[Table]
Expectation Attack.
In Table 11, we show the results of the expectation attack.
We modify the adaptive attack and average the gradient from 10 ﬁne-tuned models, whose training batches are different.
We evaluate the model using the ensemble of rotation and vertical ﬂip as the selfsupervised task on CIFAR10.
We evaluate the model with Adaptive-AutoPGD-EOT and AdaptiveSquareAttack-EOT.
One is the strongest attack in our method and the other is a black-box attack that is less likely to be affected by gradient masking.
The experiment shows that the expectation attack has little inﬂuence on the improvement of our test-time ﬁne-tuning.
[Table]
Boundary Attack.
We use one of the SOTA decision-based attacks: RayS (Chen & Gu, 2020).
We test it on CIFAR10 with the ensemble of rotation and vertical ﬂip.
Table 12 shows that our method also improves the robust accuracy for the decision-based attack.
[Table]
Combination with (Gowal et al., 2020).
We combine our test-time ﬁne-tuning with the adversarial training using additional data (Gowal et al., 2020).
We apply our Meta AT to it with the ensemble of rotation and vertical ﬂip.
Using a WideResNet-28-10, it achieves robust accuracy 62.07% under AutoPGD.
With our test-time ﬁne-tuning, the robust accuracy is improved to 64.34%.
The improvement of robust accuracy is 2.27%.
Inference Time.
Table 13 shows the inference time for different methods.
While the inference time for our method is larger than SOAP and normal method when the batch size is 1, the inference time gets closer when using larger batch size.
And the batch size of 20 or more is a common scenario of the inference.
[Table]
B D ETAILS OF OUR E XPERIMENTAL S ETTING
Meta Adversarial Training.
We consider an (cid:96) ∞ norm with an adversarial budget ε = 0 .
We also use two different network architectures: WideResNet-34-10 for CIFAR10 and ResNet18 for
STL10 and Tiny ImageNet.
Following the common settings for adversarial training, we train the network for 100 epochs using SGD with a momentum factor of 0.9 and a weight decay factor of 5 × 10 − 4 .
The learning rate β starts at 0.1 and is divided by a factor of 10 after the 50-th and again after the 75-th epochs.
The step size α in Eqn (8) is the same as β .
The factor λ (cid:48) in Eqn (11) is set to 1 . 0 .
We use 10-iteration PGD (PGD-10) with a step size of 0.007 to ﬁnd the adversarial image
B advj at training time.
The weight of each self-supervised task is set to λ k = 1 K .
We set | B j | =and sample 8 batches B 1 , ..., B 8 in each iteration.
Furthermore, we save the model after the 51 -st epoch for further evaluation, as the model obtained right after the ﬁrst learning rate decay usually yields the best performance (Rice et al., 2020).
We use PGD with the standard cross-entropy loss to generate adversarial examples at training time in the line 3, line 6 and line 8 of Algorithm 2.
The hyperparameters of the attacks are as follows:
• Line 3: PGD-10 with step size 0 .
• Line 6: As θ ∗ j is similar to θ , the adversarial examples at this step are similar to those at Line 4. To save training time
, we
therefore
choose the starting point of the attack as the adversarial examples in Line and use PGD-2 with a step size of 0 . 005 .
• Line 8: PGD-3 with step size 0 .
Online Test-time Fine-tuning.
We ﬁne-tune the network for T = 10 steps with a momentum of 0.
and a learning rate of η = 5 × 10 − 4 .
We set λ k = 1 K and λ = 15 . 0 .
In line 2 of Algorithm 1, we sample a batch B ⊂ D containing 20 training images.
In line 3, we use PGD-10 with a step size of 0 .
Ofﬂine Test-time Fine-tuning.
The algorithm for ofﬂine ﬁne-tuning is shown in Algorithm 3.
As we stochastic gradient descent is more efﬁcient for large amount of data, we use stochastic gradient descent in the ofﬂine ﬁne-tuning.
This is the main different between Algorithm 1 (online ﬁne-tuning) and Algorithm 3 (ofﬂine ﬁne-tuning).
We also ﬁne-tune the network for 10 epochs.
The batch size of each (cid:101) B advj is 128.
The other hyperparameters are the same as in the online version.
Attacks.
The detailed settings of each attack are provided below:
• PGD-20.
We use 20 iterations of PGD with step size γ = 0 .
The attack loss is the cross-entropy.
• AutoPGD.
We use both the cross-entropy and the difference of logits ratio (DLR) as the attack loss.
The hyperparameters are the same as in (Croce & Hein, 2020a).
• FAB.
We use the code from (Croce & Hein, 2020a) and keep the hyperparameters the same.
• Square Attack.
We set T = 2000 and the initial fraction of the elements p = 0 .
The other hyperparameters are the same as in (Andriushchenko et al., 2020).
For the adaptive versions, we set the interval u
= (cid:100) T/ 5 (cid:101) .
C A LGORITHM FOR A DAPTIVE A TTACKS
In Algorithm 4, 6, 5 and 7, we show the algorithms for (cid:96) ∞ norm-based adaptive PGD, AutoPGD,
Square Attack and FAB, respectively.
The main difference between the original and adaptive version is the target loss function for maximization.
The reader may refer to (Andriushchenko et al., 2020;
Croce & Hein, 2020a;b) for more detailed description of the steps in these algorithms ( e.g. , the condition for decreasing the learning rate in AutoPGD).
Output
−
9: l
= max { i [-+ ε ] ( (cid:101) x 
-],
1 i } 10: (cid:101) i
if i =
0 i else (cid:101) i
(cid:101) i 11: end for 12: for t = 1 to T − 1 do 13: if t then 14:
Get ﬁnal parameter T by taking (cid:101) = { (cid:101) x i as input image for Algorithm 1.
16: end if 17: for i = 1 , (cid:101) B do 18:
Grad
=
[ (cid:101) x
i − ε, (cid:101) x
i + ε ] ( (cid:101) x ti + γ Sign(Grad
( (cid:101) x ti )))
20: (cid:101) x
t +1 i
=
Clip [ (cid:101) x
i − ε, (cid:101) x
i + ε ] ( (cid:101) x ti + ξ ( z t +
i − z ti )
+ (1 − ξ )( (cid:101) x ti − (cid:101) x t − 1 i )) 21: l t +1 i
=
L attack ( F ( (cid:101) x
t +1 i ) , (cid:101)
y
i ; θ ) 22: (cid:101) x ∗ i = (cid:101) x
t +1 i and l ∗ i =
l t +1 i if l t +1 i > l ∗
i 23: if k ∈ W and satisfy the condition of dropping learning rate then 24: γ = γ/ 2 and (cid:101) x t +1 i
= (cid:101) x ∗
i 25: end if 26: end for 27: end for 28: return Adversarial image (cid:101) x
advi = (cid:101) x ∗ i do 8:
←−
9: δ ←− c 10:
Sample uniformly r, s ∈ { w − h t } ⊂ N 11: for j = 1 , ...
c do 12: ρ ←− Uniform ( − 2 ε, 2 ε ) 13:
δ r +1: r + h t ,s +1: s + h t
= ρ · 1 h t × h t 14: end
=
Clip x i − ε, (cid:101) x
i + ε ] ( (cid:101) x
i + δ
16: l
=
F y
; θ
17: if l
< l then 18: (cid:101) x
=
do i
[Table]
l = ∇
[Table] [Table]
F (
[Table]
i θ l 9: s =
arg min l (cid:54) =
y i | F ( (cid:101) x (cid:48)
i ; θ ) l − F ( (cid:101) x (cid:48)
i ; θ ) yi | (cid:107) Grad( (cid:101) x (cid:48) i )
[Table]
−
[Table] [Table]
x (cid:48) i ) yi
(cid:107)10: δ t = proj ∞ ( (cid:101) x (cid:48)
i , π s , C ) 11: δ t orig = proj ∞ ( (cid:101) x i , π s , C ) 12: α = min (cid:26) (cid:107)
[Table]
t (cid:107) ∞ (cid:107) δ
[Table]
(cid:107) ∞ + (cid:107)
[Table]
t orig (cid:107) ∞ , α max (cid:27) ∈ [0 , 1] 13: (cid:101) x (cid:48)
i =
proj C (cid:16) (1 − α )( (cid:101) x (cid:48)
i + ηδ t )
+ α ( (cid:101) x
i + ηδ t orig ) (cid:17)
14: if (cid:101) x (cid:48)
i is not classiﬁed as y i then 15: if (cid:107) (cid:101) x (cid:48)
i − (cid:101) x
i (cid:107) ∞ < v then 16: (cid:101) x advi = (cid:101) x (cid:48)
i 17: v = (cid:107) (cid:101) x (cid:48)
i − (cid:101) x
i (cid:107) ∞ 18: end if 19: (cid:101) x (cid:48) i = (1 − β ) (cid:101) x i + β (cid:101) x (cid:48)
i 20: end if 21: end for 22: end for 23: return Adversarial image (cid:101) x advi