Self-Supervised Adversarial Training via Diverse Augmented Queries and Self-Supervised Double Perturbation
Authors: Ruize Zhang
Sheng Tang
Juan Cao
Venue: NeurIPS 2024 poster (2024)
Keywords: Self-supervised Learning
Adversarial Training
Primary area: machine_vision
TL;DR: This paper provides a method to enhance self-supervised adversarial training

Abstract: Recently, there have been some works studying self-supervised adversarial training, a learning paradigm that learns robust features without labels. While those works have narrowed the performance gap between self-supervised adversarial training (SAT) and supervised adversarial training (supervised AT), a well-established formulation of SAT and its connections with supervised AT are under-explored. Based on a simple SAT benchmark, we find that SAT still faces the problem of large robust generalization gap and degradation on natural samples. We hypothesize this is due to the lack of data complexity and model regularization and propose a method named as DAQ-SDP (Diverse Augmented Queries Self-supervised Double Perturbation). We first challenge the previous conclusion that complex data augmentations degrade robustness in SAT by using diversely augmented samples as queries to guide adversarial training. Inspired by previous works in supervised AT, we then incorporate a self-supervised double perturbation scheme to self-supervised learning (SSL), which promotes robustness transferable to downstream classification. Our work can be seamlessly combined with models pretrained by different SSL frameworks without revising the learning objectives and helps to bridge the gap between SAT and AT. Our method also improves both robust and natural accuracies across different SSL frameworks. Our code is available at https://github.com/rzzhang222/DAQ-SDP.

Self-Supervised Adversarial Training via Diverse Augmented Queries and Self-Supervised Double Perturbation
Ruize Zhang
Sheng Tang"
Institute of Computing Technology.
Institute of Computing Technology,
Chinese Academy of Sciences
Chinese Academy of Sciences University of Chinese Academy of Sciences Beijing, China ts@ict.ac.cm
University of Chinese Academy of Sciences Beijing, China
zhangruize2lbBicttaccooo
Juan Cao
Institute of Computing Technology,
Chinese Academy Sciences
University of Chinese Academy of Sciences Beijing, China caojua@iict.ac.crm
Abstract
Recently, there have been some works studying self-supervised adversarial training, learning paradigm that learns robust features without labels. While those works have narrowed the performance gap between supervvpdd adversarial training (SAT) and supervised adversarial training (supervised AT), well-established for- mulation of SAT and connections with supervised AT are under-explored. Based simple SAT benchmark, robust generalization gap and degradation on natural samples. We hypothesize this due the lack of data complexity and model regularization and propose a method named as DAQ-SDP (Diverse Augmented Queries Self-supervised Dou- ble Perturbation). We first challenge the previous conclusion that complex data augmentations degrade robustness in SAT by using diversely augmented samples as queries to guide adversarial training. Inspired by previous works in supervised AT. we then incorporate self-supervised double perturbation scheme to self- supervised learning (SSL), which promotes robustness transferable downstream classification. Our work can be seamlessly combined with models pretrained by different SSL frameworks without revising the learning objectives and helps to bridge the gap between SAT and AT. Our method also improves both robust and natural accuracies across different SSL frameworks. Our code is available at https:/ggghubb.oommrrhaag22222
find that SAT still faces the problem of large
Introduction
Deep neural network has shown its power various machine learning tasks. In spite of its beneficial properties in optimization and generalization, deep neural network is vulnerable to adversarial attack as samples with carefully designed tiny perturbations may cause significant deviations of model
"Corresponing author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

predictions [16 25 12 36.. One of the most successful defenses tackling this problem is adversarial
training. which generates perturbations that makes the largest output deviation and trains the model
with perturbed samples 40 25 Many following methods 21 10
39
32 40 have further developed
more advanced adversarial training techniques based on this adversarial training framework.
The adversarial defenses mentioned above require full supervision. However, in real situations full labels may be available. For the semi-supervised setting, some works [26 3 2 suggested that unlabeled data can be useful in improving model robustness by designing auxiliary pseudo label losses. However, the performance is largely affected by the amount of available labels. Later, some works studied adversarial robustness in self-supervised learning (SAT). In this scenario, adversarial training (AT) is integrated with superviie learning (SSL) to get robust features that can be efficiently finetuned [22] 20 14 38
18
24 37 23]. Some works [22] 20 14 24 37 combined
contrastive learning (CL.) with AT to tackle this task. this paper we term those works as ACL methods.
100.00
100.00
50.00
83.32
10.00
00.00
75.59
70.00
51.44
60.00
56.55
50.00
30.08
40.00
20.00
10.00
0.00
Traledaan
Text dion
Train Robust Test Robust
Tralecden class
dean
Train Roburt Robunt
Acc.
Acc
Acc.
Acc.
Acc.
Auc.
Acc.
Figure 1: Motivation: Large robust generalization gap and reduced clean accuracy for SAT (SimCLR) on CIFAR- 1-0 and CIFAR- 10. The left part of the figure is the results CIFAR-100 and the right part is for CIFAR-10. The robust generalization gap is over 20% on both datasets.
The ACL methods above were restricted to the CL framework and could not generalize to other SSL methods. Previously, Zhang al. (38 proposed to disentangle the task of SSL and AT into two stages of learning, which first trains SSL models with natural data and then enables AT under the pseudo-supervision of the naturally trained SSL features. This learning framework brings consistent performance boost for both contrastive and non-contrastive methods. Moreover, the feature pseudo-supervision in this framework takes form similar to the supervision in supervised and semi-supervised AT, thus providing unified perspective to analyze AT paradigms under different supervision. this paper, we regard this learning framework as a simple benchmark for SAT.
While previous works on ACL and SAT have achieved remarkable performance that is comparable to supervised AT, there is neither a well-established formulation nor an analysis of the general learning process as in the supervised counterpart. To better understand the limitations of this learning paradigm, we start from the SAT benchmark proposed by Zhang al. (38), which advocates process that is much more efficient than previous ACL works (100 vs. 1000 epoches of ad adversarial training). As shown in Figure I we record finetuned train-set and test-set accuracies after SAT pretraining on 10 with ResNet-34 as backbone and find that there is robust generalization gap of around 19% and clean accuracy gap of around 6%. The results are similar for CIFAR-100, 100, which gives robust generalization gap of around 22% and clean accuracy gap of around 9%. The results show that there exists a large robust generalization gap and natural performance degradation, similar to the AT counterpart.
SAT
Given that both SSL and AT are hard tasks [38] we hypothesize these problems to be caused by insufficient data complexity and model regularization. In order to solve the problems, we propose method termed as DAQ-SDP (Diverse Augmented Queries Self-supervised Double Perturbation) and handle the problems from two aspects. First, while previous papers (24 38 concluded that complex augmentation techniques are crucial for SSL but harmful for SAT robustness, we argue that strong and diverse augmentations could help SAT if used properly. Specifically, we find that every training distribution is worth one set of BatchNorm layers SAT and propose an Augmentation-Adve -Avversrr (Aug-Adv) Cairvise-BatchNer
adversarial training method. It turns out that naturally trained SSL. models with single set of batch norm layers can provide effective guidance for multi-branch adversarial training. Second, while previous works on SAT focused on designing sample adversarial

---

perturbation for SSL, we find that self-supervised model perturbation also contributes to downstream robustness with a proper training scheme. Despite the task mismatch from pretraining to finetuning. there exists a cross-task transferability of robust generalization. Our work finds a general method that can be directly applied to different SSL. frameworks without revising learning objectives, providing insights that contribute to the understanding of SAT.
As far as we know, we are the first to analyze SAT as a general learning paradigm in an SSL- -ranssc way and solves its problems by revealing its traits related to clean performance degradation and robust generalization. The main contributions are summarized as follows:
In contrast to the conclusion of previous works [3, 24], we suggest that strong and diverse augmentations can boost self-supervised robustness and propose an Aug-Adv Pairwise- BatchNorm technique for better robust generalization and less natural degradation.
Different from previous works 22
37 23 that focused on introducing sample adversarial perturbation from supervised AT to SSL, we use model perturbation in SSL pre-traning to boost downstream robustness. We then propose self- seff-supervised double perturbation scheme in the later stage of SAT to improve robust generalization without affecting the learning of natural features.
20
4
38
18
24
We conduct experiments on CIFAR-10 and CIFAR- 100, the commonly used datasets in previous works. On CIFAR- 100, our proposed method improves over 2% on AutoAttack 77 and clean data results with ResNet-34. On CIFAR- 10, our method improves over 1 1 on AutoAttack 7 and over 2% clean data results with ResNet-34. The experimental results demonstrate the effectiveness of our method across SSL frameworks, models and datasets.
2
Related Works
The task of SAT integrates AT into SSL and aims at learning robust feature representations that enable efficient finetuning. Here we will give a brief summary of previous works related to this field.
Supervised Adversarial Training (supervised AT) Given deep neural network's 's vulnerability to adversarial perturbations, many defense methods have been proposed. Among them adversarial training, originally proposed by Goodfellow et al. [16. has become a prevalent method. Adversarial training simulates a min-max process that finds the perturbation with largest distortion and then minimize the training loss over the adversarial data. Madry et al. (255 proposed representative adversarial training method and uses random initialized ntet projected gradient descent to gen- erate adversarial samples. Many following works [21 [0 39
40 further revised the adversarial learning framework and applied more advanced techniques such as logits pairing, boundary guidance and consistency regularization for improving robustness while keeping the accuracy on clean data.
32
Self-Supervised Learning (SSL) Self-supervised learning is the task of learning feature representa- tions with no label available. In this case, a discriminative self-supervised method generally relies on a pretext task for pretraining to learn useful information. Previously, many pretext methods have been proposed [15 II] (28). One of the most successful pretext task is instance discrimination (6 4. Contrastive learning defines a instance discrimination task and learns the features using similarities between positive and negative pairs. The recent development from contrastive learning methods MoCo 6 and SimCLR to non-contrastive methods BYOL [7 and SimSiam [ has revised the contrastive loss to a positive-pair loss and further simplified the framework. Moreover, some works combined the contrastive framework with some more advanced techniques including positive and negative pair mining, prototypes and customized contrastive view crafting to enrich the information learned by the contrastive framework for effective and efficient learning.
29
13 I9
Self-Supervised Adversarial Training (SAT) The development of self-supervised learning provides a new direction for acquiring robust features. In self-supervised learning, the emergence of instance discrimination as the new state of art self-supervised pretext task provides a natural setting for adversarial robustness to fit in. While constructing reliable decision boundary using ground truth labels is not feasible. previous works [22] 20, [4] 24 (37 proposed to exploit contrastive loss adversarial training for promoting robustness. The assumption is that if the feature space near the data sample is smooth enough, the feature prediction of adversarially perturbed data will be consistent with its clean counterpart. Gupta et al. (18) suggested that contrastive learning has intrinsic sensitivity to adversarial perturbations and proposed simple method to remove false negative pairs. This
3

---

method enhances robustness in SSL without revising AT and is orthogonal to ours, Xu et al. [77 proposed use causal reasoning in ACL and used adversarial invariance regularization enhance ACI. While achieving an impressive performance, the methods above were still restricted to the contrastive learning framework.
Later, Some works have started to explore robustness in the broader SSL picture. Zhang et al. [38 formulated SAT two-stage framework that first trains an SSL model and then uses the learned features as guidance for AT. This work has set a strong baseline for this task and can be directly generalized to different SSL frameworks. Thus we take this two-stage SAT framework as the baseline to explore robustness in the broader picture of SSL. Kim et al. [23 proposed an interesting idea that carefully crafted targeted adversarial perturbations can help enhancing robustness for non-contrastive SSL methods. However, the improvements on contrastive frameworks are not as consistent as in the non-contrastive case. Moreover, some works 38 24 suggested that complex augmentations are crucial for SSL but destructive for SAT. In this work, we approach the task of SAT by studying its learning process and drawing an analogy to supervised AT, with the goal of acquiring better understanding of the difference and similarity between these two learning paradigms. We then propose a method to tackle the potential problems in SAT. In the following sections, we will introduce the problem statement and then describe our motivation and method.
Preliminary
Self-Supervised Learning As the most representative contrastive learning framework, SimCLR uses data in the same batch as negative pairs and optimizes the following objective:
exp(sim(z1,23)) 2;)/1)
(2), = - -log exp(sim 2,)/t) + Σxzi exp(sim (2, 2))t)
In the equation above,r, (Ir), (2)) are augmented views of the same image. = g f(T((x)) is the projected feature of the corresponding view. 2; and 2j are a positive pair. is the number of negative samples.
Contrastive SSL relies on large batch size or extra maintained queue for negative pairs and can be computationally expensive. In contrast, positive-only frameworks only include positive pairs. The learning objective of SimSiam [5), representative positive-only method, can be formulated as:
P1- stopgrad(=2) P1 |1112|12211
P2 stopgrad(21) |P2
La (TT(
(2)
2 22
In the equation above, 7I (æ), T2(1)) are two augmented views of the same image. 2; = g. (()
and p: = h- 2; the projected and predicted feature of the corresponding view, in which g is projector helps to preserve instance dicriminative features and is a predictor helps to prevent model callapse.
Adversarial Contrastive Learning Based on the framework contrastive learning (CL), multiple previous works have proposed adversarial contrastive learning (ACL) methods, which aims at improving the robustness of the learned features. Pevious works have adopted such a learning In general, those methods can be
framework but each had some revision of the loss term 20 14 formulated as:
cady
CL = (T))
where
adv x + arg max fax
In the equations above, padv is adversarial contrastive loss with an extra variable for the adversarial view. It is often calculated as the average of the pairwise contrastive loss [4] y.adv is the adversarial sample generated with this loss.

---

Self-Supervised Adversarial Training To find general method that can improve robustness for different SSL frameworks, we start from a basic SAT framework [3]]
stagel
LSSL:
(5)
and
Esugez = Sim(f2(2), fi(2)) + Sim(f2(1 f2(x)),
(6)
where
adv x arg (iinnx(SSim(((x-+t
This SAT framework separates the adversarial training process into two stages. In the first stage, an SSL model fi is trained with clean data. Then the features predicted by the clean model are used as superv--uperisiion for adversarial training of f2 in the second stage. The clean samples providing guiding features can be regarded as queries that help to distill useful information from the clean model. Compared to ACL methods mentioned above, this type of method is more general and also more computationally efficient. In our work, we adopt this framework as the baseline.
4
Method
In this paper, we aim at finding method that can solve the problem of robust generalization and clean accuracy degradation in SAT. Note that our work proposes a general method that can be directly combined with different pretrained SSL models for improvements instead of requiring adversarially re-training the models from scratch. Figure demonstrates the overall framework of our method. In the following sections, we will introduce each part of our method.
4.1 Diverse Augmented Query
Previously in supervised AT, some works have discussed about the effects of complex augmentation
strategies robustness [3 sample distributions to reduce overfitting, However, in SAT, there is no ground truth label to provide supervision on training or generalized data distributions. We hypothesize that SSL models trained with natural data, especially instance discrimination based ones, already contain certain level of generalization capability the feature space is learned with large amount of data under strong and complex augmentations. However, this capability of generalization can be lost during AT. Thus we propose to use the diversely augmented clean features as supervision for AT.
30 M The idea is fit the model to the labels the generalized
In the field of ACL and SAT, previous works 38 24] have concluded that strong and diverse augmentations essential for SSL but harmful for robustness. Thus Luo al. (24 proposed to gradually reduce the strength of augmentation during the training and Zhang et al. only use random resized crop and horizontal flip in adversarial training. In this paper, we argue that diversely augmented samples help SAT given that the model has sufficient capacity, as it's actually essential for the robust model distill rich information to keep the capability of generalization.
38 proposed to
Note that data augmentation strategies including AutoAugment 9 and RandAugment X require labels to caleulate validation accuracy in the process of searching for optimal augmentation policies, thus can not be directly applied to SAT. In this paper, we propose to use TrivialAugment (27) which dataset-independent and search-free method that randomly samples the augmentation policy and strength. Since we need to improve the generalization while also specialize on the testing distribution, need fit the adversarial model with clean model features both on the diversely augmented and basic augmented distributions. Inspired by previous work in supervised AT [. we adopt a multi-stream structure that takes samples from different distributions as inputs. In the previous ACL and SAT works, there were no clear conclusion on the usage of BatchNorm layers. While some works (2 [4] suggested to use different BatchNorm layers for adversarial sample, Zhang et al. (38 suggested that separate BatchNorm layers are not necessary. In this work, we find that each training distribution is worth one set of BatchNorm parameters in SAT and even the clean model only contains one set of BatchNorm layers, it can provide effective guidance for the multi-Baad BatchNorm adversarial model. Specifically, we define four different input streams based on the combination of their adversarial type (adversarial vs. natural) and augmentation type (strong vs. weak), where the basic augmentation contains random resized crop and horizontal flip while diverse augmentation contains Trivial Augmentation 27 and basic augmentation. In the first stage
5

---

Xweak adv
Frozen Encoder
Xstrong-adv
Encoder
Projecter
Predictor
(single-BathNorm)
x
Luagee
weakk-nat
Adv. Self-Perturbed Encoder
(Pairwise-BatchNomm)
Xatrong nat
Stage1: Self-Supervised Learning
Stage2: Adversarial Training
Figure 2: A demonstration of our proposed DAQ-SDP. The single BatchNorm encoder of pretrained model extracted as teacher for our Haarwise-BatchNom robust encoder for adversarial training.
of our method, pretrained SSL model with no consideration for robustness. Then in the second stage, we propose an Aug-Adv pairwise- Batchhorm strategy for adversarial training, where each stream of features is pscudo-superviae by the features predicted by the clean model. After training, we only keep the basic-adv BatchNorm layers.
either single of BatchNorm to train clean model directly use
The formulation of our method is:
Cdiverec-oup Lelean + A ladvs
(8)
where
Leeean
Sim(s2-auge clean (Faug, ), (Faug,) aug.
Σ
(9)
and
Ladv Sim (fn 2 augaaaa-aaa ado (1 fe-auge- aag.au---eaas clean (Zaug, }),
(10)
and
Pang arg max Sim(ff(rrit (Faug
(11)
In the equations above, Sim(.,.) .) is the cosine similarity between two features and aug, corresponds to the augmentation type mentioned above. fe-aug.-add aug and fa-oug. are the student model with the corresponding pairwise- -attssh-Bthhommm layers. Our method forces the geeudo-superviee adversarial model inherit the generalization capability from clean model with diversely augmented queries. The AugAdv pairwise-BA, BiN helps features without interfering with each other. The experimental results demonstrate that diverse and complex augmentations can improve SAT robustness. This finding helps to narrow the gap between improving SAT and supervised AT.
of each training distribution to fit the clean feature counterparts
4.2 Adversarial Self- haf-Peeturbed Weight
Previous works in ACL and SAT have borrowed the idea of adversarial sample perturbation from supervised AT to SAT and made revisions either on the specific training loss term 220 22 or the effective way of generating sample perturbations [14 23). However, whether more advanced ideas in supervised AT can bring improvements in SAT is under-explored. In this section, we suggest that adversarial weight perturbation can introduced into SSL pretext task for downstream robustness. Note that in the method proposed by Wu et al. different supervised frameworks including vanilla PGD (55), TRADES (39], RST and MART 34). However, all those methods require full or partial labels and are based classification loss.
35 adversarial weight perturbation is applied to

---

The adversarial weight perturbation can be expressed as:
LawP = max Len(fa(x; v)) + BTad (S (Ja(nadv ,y)),
(12)
0Eμ(()
where j is the perturbation size of the model weight.
In contrast, in our work the weight perturbation is introduced into the SSL pretext task. Thus the perturbation doesn't 't regularize the weight classification-loss landscape, but works on the feature similarity loss in label-free paradigm instead. This transition of learning paradigm makes it interesting to see whether such perturbations respect to the SSL objective can benefit downstream robust generalization. Specifically, the self-supervised weight adversarial perturbation perturbs the model weight in the direction of enlarging the self-supervised cosine similarity loss and increases the smoothness of this similarity loss landscape. The weight perturbation be formulated
02 arg min (faufa( ) f1 (eaug-weuk) + X Sim (a weak/ ), f2 (af weak) (13) (2Ep(0)
The weight perturbation finds the "worst" adversarial scenario which is beneficial for model robust- ness. However, this extra adversarial component also further increases the difficulty of the task. In supervised AT, the existence of ground truth labels helps the model converge despite the enlarged difficulty. However, both AT and SSL are difficult tasks. In the early stage of SAT, regulating the model to learn this rather difficult objective with respect to insufficiently learned adversarial features could impede the learning of natural features and we propose to apply this weight adversarial perturbation only the later stage of learning when the pseudo-supen supervised learning clean and adversarial features is stabilized. Without our weight self-perturbation scheme, there is clean accuracy drop of 0.7% and PGD robust accuracy drop of 0.8% on CIFAR-10 10 with ResNet-34.
The adversarial weight perturbation calculated on weakly augmented data is combined with sample adversarial perturbations over the four sample distributions the previous section. The overall learning objective is:
Lawap- divep---vvese-aau
Leleon + X Ludes Σ
(14)
angi
where
Edeen Sim( fog aug, clan (Daug, ) fe, (Faugi))
(15)
and
lado Sim(So,-au -ung. ado (1 S f2-auu clean (Fangs)) ))-
(16)
100.00
100.00
88.02
10.00
10.00
10:00
53,62
50.63
$8.50
50.00
$0.00
33.08
30.00
10:00
20.00
10:00
10.00
10:00
0.00
deann
Robuat
Traindoas
Text clean Asc.
Thai Rebust Tosk Robut Ass.
Act.
Figure 3: The generalization gap of SAT (SimCLR) with proposed DAQ-SDP. The left part is the result on CIFAR-100. The right part is the result on CIFAR-10.
As shown Figures and 3 the robust generalization gap is reduced by around 3% and the test clean accuracy improves by more than 1.5% average with proposed method, which means properly regulating the smoothness of weight SSL-loss landscape in pre-training can improve the robust generalization of downstream classification despite the lack of labels.
4.3 Towards Unified Understanding of SAT and supervised AT
Despite the task difference between the learning paradigms, our method steps forward an unified understanding of SAT and supervised AT by revealing their similar characteristics with respect to
7

---

071
435
+.0
1115
4175
-15
Figure 4: 1D visualization of the downstream weight loss landscape. The plot the left is for the baseline method and the plot on the right shows that for our method with self-perturbed weight. The X-axis represents the magnitude move the model weight.
Table 1: Results on ResNet-34 trained on CIFAR- 10 with SimCLR framework.
Evaluation
Method
Clean
PGD
AutoAttack
DynACL AIR 37
79.79
51.07
47.61
Simple Linear
TARO 23
84.23
53.36
45.68
Finetuning
DecoupledACI
38
82.46
56.86
47.99
DAQ-SDP (ours)
84.57
58.57
49.22
generalization. In this work, We also provide methods that focus on SAT with single SSL framework. We look forward to seeing future works with general approaches that improve adversarial training across different supervision settings.
a "higher" level of perspective than previous narrower
Experiment
In this section, we demonstrate the effectiveness of our method. First, we evaluate the effectiveness when DAQ-SDP is plugged into contrastive and positive-pair only SSL. frameworks. Then we conduct an ablation study for each part of our method. We also visualize the representation learned by our method through t-SNE 33 in Appendix.
Table 2: Results on ResNet-34 trained on CIFAR-100 with SimCLR framework.
Evaluation
Method
Clean
PGD
AutoAttack
DynACL|24| + 37
47.02
23.91
20.66
Simple Linear
TARO
51.28
29.46
21.14
Finetuning
DecoupledACL DAQ-SDP (ours)
33
51.44
30.68
21.31
53.54
33.09
23,42
Experimental Sctup:
We apply our method on top of SimCLR [4), SimSiam (5) and BYOL 17 to evaluate the performance improvements across SSL frameworks. We also conduct extensive experiments on ResNet-18, ResNet-34 and ResNet-50. We find that the improvements are less significant on the smaller model ResNet-18. The rationale may be that simply increasing data view could decrease the training
complexity for ResNet-18 by using extra set clean accuracy from 54.45% to 52.78% on CIFAR- 100, suggesting insufficient model capacity to distill richer information while fitting well on clean data. This is understandable model capacity takes a significant role in adversarial robustness and many techniques in supervised AT [30] 1 35 require models with sufficient capacity to be effective. Given the complexity of AT and SSL, expect a model larger than ResNet-18 18 is needed to learn rich information from the training data.
All SSL models in our method are first trained with clean data for 1000 epochs, then adversarially trained with 5-step PGD attack with the epsilon size of 8/255. Methods previous works are adversarially trained for 1000 epoches as mentioned in their papers. The robustness is evaluated with
8

---

Table 3: Results on other SSL frameworks with ResNet-34 backbone. Note that most previous works are based on SimCLR and can not be used in Positive-Pair only SSL, frameworks. The dataset is CIFAR-10.
SSL Framework
Method
Clean
PGD
AutoAttack
SimSiam
TARO[23]
81.71
52.61
44,46
SimSiam
DecoupledACL
38
78.40
57.17
47.20
SimSiam
DAQ-SDP (ours)
80.42
58.53
47.69
BYOL
TARO 233
86.84
52.01
44.76
BYOL
DecoupledACIL 38
83.15
55.22
47.67
BYOL
DAQ-SDP (ours)
85.91
56.53
49.07
Table Results on ResNet-18 trained on CIFAR-10 with SimCLR framework.
Method
Clean
PGD
AA
DynACL + 37
78.08
49.12
45.17
TARO 23
82.86
52.44
43,99
DecoupledACI
38
80.17
53.95
45.31
DAQ-SDP(ours)
81.76
55.15
45.12
AutoAttack 77 and PGD attack with 20 iterations and epsilon size of 8/255, is set to We use double adversarial perturbation after 60 epochs of training and weight perturbation size constraint of 0.002. The SL.F and AFF finetuning details are the same as previous works [14 38 with 25 steps of training and initial learning rate of 0.1. The experimental results in our method is the average of runs, with a maximal variation range of =0.5 for clean accuracy and +0.35 for robust accuracy. All experiments are conducted on RTX 3090 GPUs.
5.1 Effectiveness of DAQ-SDP across SSL Frameworks, Models and Datasets
a) We first conduct experiments with different SSL frameworks ResNet-34. As shown in Table T and Table 2 our method contributes to significant improvements on both clean and robust accuracy on both CIFAR- 100 and CIFAR-10 10 with SimCLR [4) Note that TARO [23 is adversarial sample generation method that needs to be combined with specific SAT baselines. In this work we combine with the same baseline framework
for better performance and fair comparison. Although TARO (3 gives slightly better clean accuracy on CIFAR-10, our method outperforms TARO (3 on robust accuracy by large margin. On CIFAR-100. both our clean and robust accuracy outperforms TARO 23 Also note that methods except TARO 23). DecoupledACL. (38) and ours are contrastive based methods and don't 't generalize to positive-pair only SSL frameworks. In Table we compare our method with previous works on SimSiam [5] and BYOL [17]. As shown in the results, our work provides consistent improvements for different SSL frameworks. This is because
treat differently trained models as the teacher that provides supervision for the clean data feature space. Once we obtain the supervision, we take an SSL-framework agnostic process for robustness improvements.
b) We then conduct experiments on ResNet- 18 and ResNet-50. From Table 4 and Table 5 our method shows improvement across model sizes.
c) We also provide cross-dataset transfer learning results. Table 6 shows transfer learning from CIFAR-100 to CIFAR-10. in which our work outperforms other SAT methods. In Table 7 we provide the transfer learning SLF results from CIFAR-10 to STL-10, which shows that our method can transfer well across datasets from more different domains.
Table Results on ResNet-50 trained on CIFAR- 10 with SimCLR framework.
Method
Clean
PGD
AA
DynACL AIR
80.67
47.56
TARO 84.57
53.60
46,86
DecoupledACL
38
83.32
55.70
48.24
DAQ-SDP(ours)
85.22
58,05
49,49

---

Table 6: Cross-dataset transfer learning from CIFAR-100 to CIFAR-10. Note that the methods compared here are not restricted to specific SSL. framework. We use both simple linear finetuning (SLF) and adversarial full finetuning (AFF) in this experiment. We use ResNet-34 as the backbone model.
SLF
AFF
Method
Clean
PGD
Clean
PGD
DecoupledACI, 38
55.03
25.78
86.16
52.97
[33
57.13
23.99
86.00
52.71
DAQ-SDP (ours)
57.66
26.83
86.83
53.08
Table 7: Cross-dataset transfer learning from CIFAR-10 to STL-10. We use ResNet-34 as the backbone model.
Method Baseline
Clean
PGD
63.84
40.66
DAQ-SDP(ours)
66.79
40.75
Table 8: Ablation study for our method with SimCLR on CIFAR-100. We use ResNet-34 as the backbone model.
Method
Clean
PGD
Baseline
51.44
30.68
DAQ(single-BN)
52.27
31.56
Diverse Augmented Query
52.67
32.11
Weight Self- Self-Perturbed Scheme
51.56
32.37
DAQ-SDP(ours)
53.54
33.09
5.2 Ablation Study
In this section we evaluate each part of our method. Table shows the results on CIFAR-100. With Diverse Augmented Query, the clean accuracy increases by 1.23% and the PGD robust accuracy increases by 1.43%. With our Diverse Augmented Query and Weight Self-Perturbed Scheme, the clean accuracy increases by 2.10% and the robust accuracy increases by 2.41% % As shown in Figures [ and 3 both robust generalization and clean accuracy are improved compared to the baseline. These improvements brought by our proposed method demonstrate the effectiveness of the enhanced sample complexity and model regularization with the SSL loss term.
The smoothness of the weight loss landscape has been shown to be important for robust generalization
in supervised AT 35) In Figure 4 we also analyze the effect of our method to downstream weight loss landscape through 1D visualization as previous work in supervised AT (35) did. From Figure we can see that the regularizing effect transferred to downstream loss landscape is actually much attenuated compared with directly regularizing classification loss in supervised AT (35), showing difficulty of such a transferred improvement from SSL pretext tasks.
Conclusion
In this paper, we observe that SAT has the similar problem of large robust generalization gap and clean accuracy degradation as in supervised AT. We then propose a general method solve this problem, which can be directly combined with different pretrained SSL models without further changing learning objectives. We first challenge the previous conclusion that diverse and strong augmentations harms SAT and propose a diversely augmented query based method with Aug-Ady BatchNorm distill generalizable and diverse information from clean model. Second, different from previous works that focused on introducing sample perturbation to the SSL pretext task, we suggest that regulating the smoothness of the SSL loss landscape by adversarial weight self- self-perturbation boosts robust generalization transferable to downstream classification. Our method not only improves the performance across different SSI. frameworks, but also provides insights for narrowing the gap between the study of these two adversarial learning paradigms.
10

---

Acknowledgements
This research work is supported by the Project of Chinese Academy of Sciences (E141020).
References
[1] Sravanti Addepalli, Samyak Jain, and Venkatesh Babu R. Efficient and effective augmentation strategy for adversarial training. In NeurIPS, pages 1488-1501, 2022.
[2] Caan-Bapisste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In NeurIPS, pages 1-24, 2019.
[3] Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. Unlabeled data improves adversarial robustness. In NeurIPS, pages 1-12, 2019.
[4] Ting Chen, Simon Korbllihh, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. pages 1597-1607, 2020.
[5] Xinlei Chen and Kaiming He. Exploring simple siamese representation learning, 2020. M.kiv:2011.105660
[6] Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning, 2020. arXiv:2003.04297.
[7] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICMI., pages 2206-221 2216, 2020.
[8] Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le. Randaugment: Practical automated data augmentation with reduced search space. Mmxi:1199.113711
[9] Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoe V. Le. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[10] Jiequan Cui, Shu Liu, Liwei Wang. and Jiaya Jia. Learnable boundary guided adversarial training. In ICCV, pages 15721-15730, 2021.
[11] Carl Doersch, Abhinav Gupta, and A Alexci Efros. Unsupervised visual representation learning by context prediction. In ICCV, pages 1422-1430, 2015.
[12] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In CVPR, pages 1-9, 2018.
[13] Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre Sermanet, and Andrew Zisserman. With little help from my friends: nearest-neighhor contrastive learning of visual representations. In ICCV, pages 9588-9597, 2021.
[14] Lijie Fan, Sijia Liu, Pin- Yu Chen, Gaoyuan Zhang, and Chuang Gan. When does contrastive learning preserve adversarial robustness from pretraining to finetuning? In NeurlPS, pages 21 21480-21492, 2021.
[15] Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations, 2018. #rXiv:1803.0772S
[16] Ian Goodfellow, Jonathon Shlens, and Christian Szegedy, Explaining and harnessing adversarial examples. In ICLR, pages 1-11, 2015.
[17] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, koray kavukeuoglu, Remi Munos, and Michal Valko. Bootstrap your own latent new approach to self-supervised learning. In NeurIPS, pages 2127 122844,, 2020.
11

---

[18] Rohit Gupta, Naveed Akhtar, Ajmal Mian, and Mubarak Shah. Contrastive self-superrvsed learning leads higher adversarial susceptibility, 2023. urXiv:2207.10862.
[19] Qianjiang Hu, Xiao Wang, Wei Hu, and Guo-Jun Qi. Adco: adversarial contrast for efficient learning of unsupervised representations from self-trained negative adversaries. CVPR, pages 1074-1083, 2021.
[20] Ziyu Jiang. Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning. In NeurIPS, pages 11199-1210,, 2020.
[21] Harini Kannan, Alexey Kurakin, and J. lan Goodfellow. Adversarial logit pairing, 2018. CaRR:abb/1803.0633
[22] Minseon Kim, Jihoon Tack, and Sung Ju Hwang. Adversarial self-supervised contrastive learning. In NeurlPS, pages 2983-2994, 2020.
[23] Minseon Kim. Hyeonjeong Ha. Sooel Son, and Sung Ju Hwang. Effective targeted attacks for adversarial self-supervised learning. In NeurIPS, pages 56885-56902, 2023.
[24] Rundong Luo, Yifei Wang, and Yisen Wang. Rethinking the effect of data augmentation in adversarial contrastive learning, 2023, arXiv:2303.01289.
[25] Aleksander Madry, Aleksander Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, pages 1-28, 2018.
[26] Takeru Miyato, Shin-Ichi Macda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: regularization method for supervised and semi-supervised learning. IEEE TPAMI, 41(8): 1979-1993, 2019.
[27] Samuel G. Müller and Frank Hutter. Trivialaugment: Tuning-free yet state-of-the-art data augmentation. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pages 774-782, October 2021.
[28] Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In ECCV, pages 69-84, 2016.
[29] Xiangyu Peng, Kai Wang, Zheng Zhu, Mang Wang. and Yang You. Crafting better contrastive views for siamese representation learning. In CVPR, pages 16031-16040, 2022.
[30] Sylvestre-Alvise Rebufñi Sven Gowal, Dan Andrei Calian, Florian Stimberg, Olivia Wiles, and Timothy A Mann. Data augmentation can improve robustness. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and Wortman Vaughan, editors, Advances in NeurIPS, volume 34, pages 29935-29948, 2021.
[31] Jihoon Tack, Sihyum Yu. Jongheon Jeong, Minseon Kim, Sung Ju Hwang, and Jinwoo Shin. Consistency regularization for adversarial robustness, 2022. arXiv:2103.04623.
[32] Florian Tramér, Alexey Kurakin, Nicolas Papernot, J. Ian Goodfellow, Dan Boneh, and
D. Patrick McDaniel. Ensemble adversarial training: Attacks and defenses. In ICLR, 2018.
[33] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, ((86))2579-26055 2008.
[34] Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan. Gu. Improving adversarial robustness requires revisiting misclassifed examples. In ICLR, 2020.
[35] Dongxian Wu, Shu-tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization, 2020. arXiv:2004.058. 058884
[36] Cihang Xie, Zhishuai Zhang. Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L. Yuille. Improving transferability of adversarial examples with input diversity. In CVPR, pages 2730-2739, 2019.
12

---

[37] Xilie Xu, Jingfeng ZHANG, Feng Liu, Masashi Sugiyama, and Mohan S Kankanhalli. Enhanc- ing adversarial contrastive learning via adversarial invariant regularization. In NeurlPS, pages 16783-16803, 2023.
[38] Chaoning Zhang, Kang Zhang, Chenshuang Zhang, Axi Niu, Jiu Feng, Chang D. Yoo, and In So Kweon. Decoupled adversarial contrastive learning for self-suuerrvied adversarial robustness. In ECCV, 2022.
[39] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, El Laurent Ghaoui, and I Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In /CML, 2019.
[40] Haizhong Zheng, Ziqi Zhang, Juncheng Gu, Honglak Lee, and Atul Prakash. Efficient adversar- ial training with transferable adversarial examples. In CVPR, pages 1-10, 2020.
13

---

A Appendix supplemental material
A.1 Feature Visualization
In this section, we visualize the the learned features of the test set of CIFAR-10 10 with 1-SNE Each data point colored with its label. As shown i the features of different classes learned by our DAQ-SDP has a clearer boundary than the baseline method.
62-
-00-
-42
-40
-40
-50
-35
-50
-75
-25
25
Figure 5: 'T-NNE results of the test set features. The left part of the figure is the features predicted by our adversarial training baseline and the right part of the figure is the features predicted DAQ-SDP.
A.2 Societal Impacts
Our work is useful for pretraining robust models with no labels. However, generated one more adversarial data and perturbed weights, which takes more computation. So it can cause more consumption of energy and pollution. Despite this limitation, we believe method is still beneficial for the society and promotes model robustness in real life.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We claim that we find a general method to solve the robust generalization and clean accuracy reduction problem for different SSL frameworks. This claim is accurately reflected and supported in the analysis and experiments.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We mentioned in the introduction that distill diverse and generalizable information from clean model, we need to have backbone model that has sufficient capacity.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
15

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: In this work we analyzed the problem of current SAT methods and provided experimental results for our method.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
We provided all the hyperparameters and formulas for our method.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
16

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [No]
Justification:
The URL of the code will be released got accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cc
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ //nips. Siiaps.cccpublie. wwtppbbiccgguueeeeeooooooccpcped
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
All the training and testing details are provided.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: For the computation complexity of adversarial training and self-supervised learning, we didn't t include confidence interval or sigma error bars. However, we run each experiments for times and took the average results and also included the range of variation of our results in the experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We included the compute resources in the experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: htt////euripppp cwtuurpp..ccpullleee hccpppllc/ttiiiccuueeeeeeeeee
Answer: [Yes]
Justification:
We conformed to the code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our work helps to build robust models under no label conditions, which can be useful in real life.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
18

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The contents used in this paper are cited
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The model and code will be released if got accepted.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper doess not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
20

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
21


Reviews:
[{"review_id": "pK89EZo99L", "reviewer": "Reviewer_mcfF", "summary": "This paper proposes a method to solve the robust generalization problem for self-supervised adversarial training in general. Starting by showing the generalization gap in existing self-supervised adversarial training framework, it proposes to solve the problem from the aspects of data complexity and model regularization and provides self-supervised remedies. This paper reduces the gap between the understanding for traditional adversarial training and self-supervised adversarial training by demonstrating the connection of their characteristics and helps to build a “unified perspective for adversarial training” under different scenarios. Robust accuracy and clean accuracy are improved across multiple self-supervised learning frameworks.", "questions": "1.Correcting the typos in the formulas and explain the important notations are needed.\n2.Additional ablation results for the “pairwise-BatchNorm” are needed.", "limitations": "The authors addressed the requirement for sufficient model capacity based on the complexity of this adversarial training task. This is acceptable. There is no negative societal impact.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This is a paper with novelty and good quality overall: \n1.It provides novel explorations and insights for the self-supervised adversarial learning paradigm, which has a shift of learning objective and different “task complexity” from the traditional supervised adversarial training. This paper can be regarded as the first work that analyzes the training problem of self-supervised adversarial learning across self-supervised frameworks in general as in the supervised scenario, while such a study is absent in previous researches. In the end, the connection with the study of traditional supervised adversarial training is built. I agree that this paper provides insight to help understand adversarial training under different supervision conditions in a broad view and has the potential to inspire future works in this field. \n2.In the methodology level, the method proposed is adaptable and general. It’s mentioned in this paper that the method is easily plugged into the self-supervised learning frameworks or pretrained models for improvements. As the authors suggest, one advantage of the method is that it doesn’t require reformulating the learning objectives, finetuning complex hyperparameters or adversarially retraining the model from scratch. \nThe experiment result on multiple self-supervised frameworks shows consistent improvement.", "weaknesses": "1.To illustrate the usage of the “pairwise-BatchNorm” technique, the authors may need to provide ablation study that uses normal BatchNorm instead for comparison.\n2.Some notations lack explanations in the paper. As an example, the notation related to theta are inconsistent in equation (10) and (11). Which theta in equation (10) is used in equation (11)?\n3.There are typos or missing signs in the formulas, e.g. the adversarial data generation in equation (11).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer mcfF,\n\nWe appreciate your positive comments. Here is our response for your questions:\n\n1. We have added new experiments on ResNet18 and ResNet50 to prove the effectiveness of our method.\n\nResNet18:\n| Method | Clean | PGD | AA |\n|---|---|---|---|\n| DynACL+AIR| 78.08 | 49.12 | 45.17 |\n| TARO | 82.86 | 52.44 | 43.99 |\n|DecoupledACL| 80.17 | 53.95 | 45.31 |\n|DAQ-SDP(ours)| 81.76 | 55.15 | 45.12 |\n\nResNet50:\n| Method | Clean | PGD | AA |\n|---|---|---|---|\n| DynACL+AIR| 80.67 | / | 47.56 |\n| TARO | 84.57 | 53.60 | 46.86 |\n|DecoupledACL| 83.32 | 55.70 | 48.24 |\n|DAQ-SDP(ours)| 85.22 | 58.05 | 49.49 |\n\n2. We have added more ablation study including the BN technique.\n| Method | Clean | PGD |\n|---|---|---|\n|Baseline | 51.44 | 30.68 |\n|DAQ(single-BN) | 52.27 | 31.56 |\n|Diverse Augmented Query | 52.67 | 32.11 |\n|Weight Self-Perturbed Scheme | 51,56 | 32.37 |\n|DAQ-SDP(ours) | 53.54 | 33.09 |\n\n3. We added some explanations for the notations in our paper:\nThe $aug_i$ in the equations represents either strongly or weakly augmented data. $f_{2-aug_{i}-clean}$ and $f_{2-aug_{i}-adv}$ in equation 10 represent the parameters of the student model with the pairwise-BN decided by augmentation strength and adversarial category. Since the adversarial data generation in equation 11 works on the adversarial branch, the parameter used in equation 11 is $f_{2-aug_{i}-adv}$. $f_{2-aug_{i}-adv}$ is the same as $f_{\\theta_2-aug_{i}-adv}$ in equation 15.\n4. We have added additional Societal Impact section:\n\nOur work is useful for pretraining robust models with no label. However, it generates one more adversarial data and self-perturbed weight, which take more time and computation. So it can cause more consumption of energy and pollution. Despite this limitation, we believe our method is still beneficial for the society and promotes model robustness in real life.\n\nThanks again for your positive comments!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "945YStUFmT", "reviewer": "Reviewer_dErS", "summary": "The paper proposes a method to improve self-supervised adversarial training. This method consists of two stages. First, a standard self-supervised model is trained on clean images to learn a feature extractor network F-1. In the second stage, a robust feature extractor F_2 is trained based on the features generated by F_1. The total loss is a linear combination of standard self-supervised loss, where F-2 learns to mimic the features generated by F_1, and an adversarial loss. The adversarial loss follows adversarial training framework where the targets are provided by F_1. The method applies weak and strong augmentations in both stages. However, in order to process different streams of data, the method uses four sets of batch-norm layers for weak and strong augmentations and adversarial and clean images. Moreover, the method applies adversarial weight perturbation for downstream robustness.", "questions": "There are no experiments to show the outcome of weight perturbation on the loss landscape of the proposed method. Overall the paper is missing qualitative experiments.", "limitations": "The authors discuss aspects of limitations of their method in introduction, however potential negative societal impact is not included.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The paper contextualizes well within the literature. The method brings a combination of current ideas from supervised adversarial training to self-supervised adversarial training.", "weaknesses": "The proposed approach does not include substantial novelty, where the main idea is following the two stage adversarial training in [38]. The addition of weight perturbation from [35] does not prompt a creative combination of ideas. \n\nThe presentation clarity can be substantially improved. There are several terms that are never defined in the equations such as aug_i and  f2_augi_clean in eqs 9,10,11, and g and h in lines 144, 150. In eqs 3 and 4 l_CL is a three-variable function, whereas it is defined as a two-variable function in eq 1. There are several typos such as line 171 \"we aims\", 185 \"In the filed of\", 242 \"loss landscape respect\". \n\nThe claim of unifying self-supervised and supervised adversarial training in section 4.3 is not supported empirically or theoretically.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer dErS, \n\nWe appreciate your time and efforts in reviewing our paper. We are sorry the phrases of “unified perspective” and “unified understanding” cause confusion. First please let us explain the following points:\n\na)\tThere are actually two types of generalization in the self-supervised adversarial training context. The first is the generalization between tasks from the SSL perspective. For instance, SSL robustness transferring to other datasets and downstream classification tasks. At least to the best of our knowledge, all previous works explore the extent of this generalization. In this work we are curious about the robust generalization of SSL methods from the adversarial training (training-testing) perspective, which is related to the robust overfitting concept in supervised AT. Since SSL operates on features and needs an extra linear finetuning for classification loss, this overfitting is often overlooked in previous works. Although we don’t observe decreasing validation robustness during training (robust overfitting) as in supervised adversarial training, we do observe a large robust generalization gap (shown in Figure 1). Our method helps to solve this generalization problem (in Figure 3) and improves the results from this AT perspective for both contrastive and non-contrastive SSL methods (shown in the same-dataset or cross-dataset finetuned results). The robust overfitting phenomenon is studied a lot in supervised AT but not studied in the SSL context. We believe the identification and analysis of such traits of self-supervised AT create a space for further improvements. That is what we mean by “bridging the gap and providing a unified perspective for adversarial training under different supervisions”. We look forward to your suggestions about making this phrase clearer.\n\nb)\tIn supervised AT, the phenomenon of robust overfitting is prevalent and AWP helps to improve robustness. However, whether AWP works depends on the trade-off between generalization gap and decreased training accuracy. Since there is a difference in the difficulty of supervised AT and self-supervised AT (which is “a challenging problem due to its two mixed challenging goals” and “clearly even more challenging than the semi-supervised AT setting” in [38]), it is unclear whether the generalization gap for self-supervised AT is large enough to compensate for the trade-off to achieve robustness improvement and transferred to classification.\n\nSecond, we add some additional experiments.\na)\tTables below show results on ResNet18 and ResNet50 with SimCLR on CIFAR10：\n\nResNet18:\n| Method | Clean | PGD | AA |\n|---|---|---|---|\n| DynACL+AIR| 78.08 | 49.12 | 45.17 |\n| TARO | 82.86 | 52.44 | 43.99 |\n|DecoupledACL| 80.17 | 53.95 | 45.31 |\n|DAQ-SDP(ours)| 81.76 | 55.15 | 45.12 |\n\nResNet50:\n| Method | Clean | PGD | AA |\n|---|---|---|---|\n| DynACL+AIR| 80.67 | / | 47.56 |\n| TARO | 84.57 | 53.60 | 46.86 |\n|DecoupledACL| 83.32 | 55.70 | 48.24 |\n|DAQ-SDP(ours)| 85.22 | 58.05 | 49.49 |\n\nb)\tThis table shows ablation study with self-perturbed weight：\n\n| Method | Clean | PGD |\n|---|---|---|\n|Baseline | 51.44 | 30.68 |\n|DAQ(single-BN) | 52.27 | 31.56 |\n|Diverse Augmented Query | 52.67 | 32.11 |\n|Weight Self-Perturbed Scheme | 51,56 | 32.37 |\n|DAQ-SDP(ours) | 53.54 | 33.09 |\n\nc)\tThis table shows transfer learning from CIFAR10 to STL10：\n| Method | Clean | PGD |\n|---|---|---|\n|Baseline | 63.84 | 40.66 |\n|DAQ-SDP(ours) | 66.79 | 40.75 |\n\nd)\tFigure 4 in the newly uploaded pdf shows the effects of baseline and our method with self-perturbed weight on downstream weight loss landscape with 1D visualization. The regularizing effect transferred to downstream loss landscape is actually much attenuated compared with regularizing classification loss as in [35], showing difficulty of such a transferred improvement.\n\nThird, thanks for pointing out the clarity problem. $CL$ loss in eq1 is defined as a two-variable function and $ACL$ loss in eqs 3 and 4 is defined as a three-variable because of the added adversarial view of data. Different works have slightly different way to deal with this extra variable. For instance, [14] averages the two-variable version. Projector $g$ is a component that preserves instance discriminative feature. Predictor $h$ is a component to prevent model callapse. $aug_i$ represents differently augmented data. $f_{2-aug_{i}-clean}$ means the parameters of the student model with the pairwise-BN decided by augmentation strength and adversarial type. \n\nWe respectfully argue that lots of previous works have a simple but effective idea. For instance, [14] combines pseudo-supervised classification, an idea used in semi-supervised AT [26], and high frequency component, a technique shown to be effective for robustness in [41], with ACL. [18] has a simple idea that similar features should be removed from negative pairs. [24] and [38] has the simple idea that AT and SSL should have separate hyperparameters. Even the first ACL works [20][22] are based on an idea that sounds straightforward: if the feature cannot be perturbed to be far from its original place, then the feature is robust. This is actually a simple combination of the notion of adversarial attack and instance discrimination. Identifying generalization problem in Self-Supervised AT and adjusting traits of downstream loss landscape via regularizing pretext tasks is actually not straightforward in the way it sounds.\n\nWe have included the Societal Impact section:\nOur work is useful for pretraining robust models with no label. However, it generates one more adversarial data and perturbed weight, which take more computation. So it can cause more consumption of energy and pollution. Despite this limitation, we believe our method is still beneficial for the society and promotes model robustness in real life.\n\n[41] Haohan Wang, Xindi Wu, Zeyi Huang and Eric P. Xing. High-frequency Component Helps Explain the Generalization of Convolutional Neural Networks. In CVPR, 2020.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eO7e1Yc1ip", "reviewer": "Reviewer_6QGn", "summary": "The paper introduces the DAQ-SDP (Diverse Augmented Queries Self-supervised Double Perturbation) method to solve the problem of large robust generalization gap and clean accuracy degradation in self-supervised adversarial training. The experimental results demonstrate the effectiveness of the DAQ-SDP method.", "questions": "1. Could you share the reason why you chose these hyperparameters (In lines 284-288) and whether you have done experiments with other hyperparameters? \n2. How does DAQ-SDP compare to DecoupledACL and TARO in terms of runtime?\n3. Can other models be used to prove the effectiveness of the approach?", "limitations": "The author mentioned that this approach is only applicable to a sufficiently large model (ResNet34).", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. The experimental results show that DAQ-SDP is better than other baselines such as DecoupledACL and TARO.\n2. Unlike most works that only perturb samples, this paper introduces the weight perturbation into the SSL pretext task.", "weaknesses": "1. Assumptions lack reasonable basis. (In line 56)\n2. There are some typos:\n\t- Incorrect citation and reference:\n\t\t- Using \\citet is better. (In lines 39, 122, 132, etc.）\n\t\t- In lines 314 and 315, \"Figure 1\" is linked to Table 1.\n\t- Inconsistent names:\n\t\t- \"Pairwise-BatchNorm\" (in line 63) and \"pairwise BatchNorm\" (in line 209)\n\t\t- \"resNet34\" (in caption of Table 1) and \"ResNet34\" (in caption of Table 2)\n\t\t- \"Taro\" (in lines 297, 300, 301, 302) and \"TARO\" (in Tables 1, 2, 5)\n\t- Some typos:\n\t\t- \"clean\" instead of \"lean\" in Eq. 8.\n\t\t- \"note\" instead of \"Note\" in Line 302.\n\t\t- \"where\" instead of \"Where\" in Line 234.\n\t\t- \"a result\" ? in Line 308.\n3. Figures seem to be a bit fuzzy.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer 6QGn,\n\nWe appreciate your efforts and time in reviewing our paper. The following is our response.\nFirst, the assumption in line 56 is also supported by [38]. This work includes the following description for self-supervised adversarial learning: \n\n1.\t“A challenging problem due to its two mixed challenging goals”.\n2.\t“Clearly … can be even more challenging than the semi-supervised AT setting.”\n\nSSL models typically require 1000 epochs to train. In the adversarial training of SSL models, no information about decision boundary is available and the features are required to be robust against attacks towards any directions away from its original place. That explains why  we describe them as hard tasks in our paper.\n\nSecond, we choose the hyperparameters in line 284 to 288 for the following reasons:\n\n1.\tThe training epochs, attack steps, attack range in training and attack range in testing are the standards reported in a series of previous works including [22][20][14][38][37].\n\n2.\tWe conducted experiments and finds that using the 60-epoch self-perturbed AWP scheme provides a 0.70% and 0.80% improvements over self-perturbed AWP for the whole training process on CIFAR10. Without the scheme there is a 0.88% improvement on PGD accuracy but 0.81% reduction on natural accuracy compared with the result without self-perturbed AWP.\n\n3.\tWe choose this weight self-perturbation range because it’s in the middle of the perturbation range used in supervised adversarial learning work [35] to give a good trade-off on robust generalization gap and training accuracy. We also show the effectiveness of our method with this hyperparameter with the experimental improvements. Please check the following ablation study results:\n\n\nThird, we have added the experiments on ResNet18 and ResNet50. The results of ResNet18 are not included in our initial submission for the result is not as obvious as in the larger model case. Please note that the DYNACL[24]+AIR[37] paper doesn’t include the PGD accuracy on ResNet50 or checkpoint and it take us too long to train for this result. So only clean and AA attack accuracy are provided. The experiments are conducted on CIFAR10.\n\nResNet18:\n| Method | Clean | PGD | AA |\n|---|---|---|---|\n| DynACL+AIR| 78.08 | 49.12 | 45.17 |\n| TARO | 82.86 | 52.44 | 43.99 |\n|DecoupledACL| 80.17 | 53.95 | 45.31 |\n|DAQ-SDP(ours)| 81.76 | 55.15 | 45.12 |\n\nResNet50:\n| Method | Clean | PGD | AA |\n|---|---|---|---|\n| DynACL+AIR| 80.67 | / | 47.56 |\n| TARO | 84.57 | 53.60 | 46.86 |\n|DecoupledACL| 83.32 | 55.70 | 48.24 |\n|DAQ-SDP(ours)| 85.22 | 58.05 | 49.49 |\n\nThe epoch time for the baseline, diverse augmented query and weight self-perturbation in the later epochs is 2min52s, 5min24s and 6min29s respectively. We have included a new Social Impact section for this limitation.\n\nWe are sorry that the figures are fuzzy and reuploaded the Figure 1 and Figure 3 in the one-page pdf. Also we have corrected the typos including reference, consistent method name and “the result” in line 308. We will change \\citep to \\citet in our new version. \n\nThanks for your helpful advice. You can also check our explanation about the motivation in other responses.\nWe appreciate your efforts and time. Thank you!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "L06jQ3MfjX", "reviewer": "Reviewer_iqHN", "summary": "The authors aim to further close the gap between the clean generalization gap and the robust generalization gap for SSL models. Several engineering improvements have been made to the SAT pipeline, such as including strong data augmentation, adversarially perturbing weight, and using separate BN. The resulting algorithm exhibits non-trivial improvements on benchmark datasets.", "questions": "(1) How does the algorithm perform on ResNet-18?", "limitations": "See weakness.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) The paper is overall well written. I find it easy to follow the motivations and implementations of the components.\n\n(2) The argument about the usage of strong data augmentation is interesting and supported by experimental results.\n\n(3) The performance gain seems non-trivial. An average improvement of 1% in robust accuracy can be expected.", "weaknesses": "Major:\n\n(1) Baseline + Weight Self-Perturbed Scheme should be included in Table 4.\n\n(2) The authors claim that \" diversely augmented samples help SAT given that the model has sufficient capacity\". I am wondering whether the difference between ResNet-18 (used in previous papers) and ResNet-34 (used in this paper) is large enough for the claim to hold. How about using even larger models, like ResNet-101 or Wide-ResNet-28-10? \n\n(3) Also, I believe the ResNet-18 results should not be entirely excluded. Previous works only tuned their hyper-parameters on ResNet-18, while the hyper-parameters for the proposed algorithm are tuned on ResNet-34. The comparison may not be fair. \n\n(4) The claims made in Section 4.3 are not well supported by the results, especially the \"unified understanding of SAT and AT\". I cannot see why applying AWP to SAT can lead to such \"unified understanding\".  Besides, \"robustness and robust generalization can be transferable across SSL and classification\" is also evidenced by previous works.\n\n\nMinor:\n\n(1) Figure 1 and Figure 3 seem blur. Are you using the JPEG images directly?\n\n(2) Typo errors: the caption of Table 1. Equation 8.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer iqHN,\n\nWe appreciate your positive comments! Here is our response:\n\n1. We added the results with ResNet18 and ResNet50 on CIFAR10 (larger model is hard for us to train in limited time). The results of ResNet18 are not included in our initial submission for it is not as obvious as in the larger model. The AIR[37] paper doesn’t include PGD accuracy on ResNet50 or checkpoint and it can take us a long time to train. So only clean and AA attack accuracy are provided.\n\nResNet18:\n| Method | Clean | PGD | AA |\n|---|---|---|---|\n| DynACL+AIR| 78.08 | 49.12 | 45.17 |\n| TARO | 82.86 | 52.44 | 43.99 |\n|DecoupledACL| 80.17 | 53.95 | 45.31 |\n|DAQ-SDP(ours)| 81.76 | 55.15 | 45.12 |\n\nResNet50:\n| Method | Clean | PGD | AA |\n|---|---|---|---|\n| DynACL+AIR| 80.67 | / | 47.56 |\n| TARO | 84.57 | 53.60 | 46.86 |\n|DecoupledACL| 83.32 | 55.70 | 48.24 |\n|DAQ-SDP(ours)| 85.22 | 58.05 | 49.49 |\n\n2.\tWe added weight perturbation in the ablation study experiments:\n| Method | Clean | PGD |\n|---|---|---|\n|Baseline | 51.44 | 30.68 |\n|DAQ(single-BN) | 52.27 | 31.56 |\n|Diverse Augmented Query | 52.67 | 32.11 |\n|Weight Self-Perturbed Scheme | 51,56 | 32.37 |\n|DAQ-SDP(ours) | 53.54 | 33.09 |\n\n3.\tWe added transfer learning results from CIFAR10 to STL 10:\n| Method | Clean | PGD |\n|---|---|---|\n|Baseline | 63.84 | 40.66 |\n|DAQ-SDP(ours) | 66.79 | 40.75 |\n\n4.\tWe hope to argue that comparison between our method and previous ones on ResNet34 are fair. First, DynACL[24]+AIR[37] results are the same as reported in the paper. Second, DecoupledACL[38] hyperparameters on larger models are finetuned for clean-robust trade-off. The hyperparameter to tune is weight decay in adversarial training. On CIFAR100, weight decay of 2e-4 gives 51.44% (clean) and 30.68% (PGD) but a larger regularization with weight decay of 5e-4 gives 49.94% and 31.99%. On CIFAR10, weight decay of 5e-4 gives 82.46 (clean) and 56.86% (PGD) but a larger regularization of 1e-3 gives 80.57% and 56.48%. TARO[23] is an attack method that can be combined with different frameworks. The initial one-stage training results reported in the paper is much lower and we combined it with our model for improved results. The hyperparameter that matters is the batch size for choosing target to attack. We use the same batch size of 512. On ResNet50, the hyperparameter is finetuned in the same manner. An increase of weight decay from 5e-4 to 6e-4 on CIFAR10 typically causes a robustness reduction of around 1.00% in these methods.\n\n5.\tWe are glad to discuss your questions about the claim of “unified perspective” and “robust generalization”. Indeed, the whole adversarial self-supervised learning literature is about robustness and generalization transferable to downstream tasks. And we hope to clarify the following points:\n\na)\tThere are actually two types of generalization in the self-supervised AT context. The first is the generalization between tasks from the SSL perspective. For instance, SSL robustness transferring to other datasets and downstream classification tasks. At least to the best of our knowledge, all previous works explore the extent of this generalization. In this work we are actually curious about the robust generalization of SSL methods from the adversarial training (training-testing) perspective, which is related to the robust overfitting concept in supervised AT. Since SSL operates on features and needs an extra linear finetuning for classification loss, this generalization is often overlooked in previous works. Although we don’t observe decreased validation robustness during training (robust overfitting) as in supervised AT, we do observe a large robust generalization gap (in Figure 1 of the paper). Our method helps to solve this generalization problem (in Figure 3) and improves the results from this AT perspective for both contrastive and non-contrastive SSL (shown in the same dataset and cross-dataset downstream classification results). The related robust overfitting phenomenon is studied a lot in supervised AT but not studied in the SSL context. We believe the identification and analysis of such traits of self-supervised AT create a space for further improvements. That is what we mean by “providing a unified perspective across AT under different supervisions”. We look forward to your suggestions about making these phrases clearer.\n\nb)\tIn supervised AT, phenomenon of robust overfitting is prevalent and AWP helps to improve robustness. However, whether AWP works depends on the trade-off between reduced generalization gap and decreased training accuracy. Since there is a difference in the difficulty of supervised AT and self-supervised AT (which is “a challenging problem due to its two mixed challenging goals” and “clearly can be even more challenging than the semi-supervised AT setting” in [38]), it’s unclear whether the generalization gap for self-supervised AT is large enough to compensate for the trade-off for improvement. It’s also interesting that using the self-perturbed AWP scheme provides a 0.70% and 0.80% improvements over self-perturbed AWP for the whole training process on CIFAR10. Without the scheme there is a 0.88% improvement on PGD acc. but 0.81% reduction on clean acc. compared with the result without self-perturbed AWP. This can also be related to the traits of self-supervised AT. \n\nc)\tTransferring preferrable AT properties except robustness itself, e.g. loss landscape smoothness, from SSL to classification is not considered in previous works and not as easy as one might think. In Figure 4 of the uploaded one-page pdf, we show the 1d visualization of resulting loss landscape of the baseline and our method. Although regularizing SSL objective smooths downstream loss landscape, the transferred regularization effect is much attenuated than directly regularizing classification loss as in [35], showing the difficulty of such an improvement.\n\n6.\tThe figures are uploaded in the one-page pdf. Thanks again for your time and patience.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We have uploaded a one-page pdf file containing unclear figures in the original submission and additional visualizations.

Meta-review: This work analyzes and improves self-supervised adversarial training (SAT) as it relates to adversarial training (AT). The results improve across a variety of self-supervised schemes and the reviewer consensus is positive. Importantly this is an informative study at a higher "level" than prior narrower studies of SAT, in its proposal of a general approach across specific self-supervision methods, which can promote more research on adversarial training without supervision. The AC sides with acceptance.

Four reviewers with expertise in domain adaptation, self-supervision, distribution alignment, and adversarial vote for acceptance (mcfF: 7) and borderline acceptnace (dErS: 5, 6QGn: 5, iqHN: 5). The authors provide a rebuttal, all reviewers engage with the rebuttal, and the authors and reviewers discuss across multiple threads and posts.

To summarize, the main argument for acceptance is the improved understanding and improved empirical results for SAT relative to AT which are delivered by a general approach consisting of the novel coordination of data and model perturbations alongside the factorization of model statistics. While elements exist in prior work, they are appropriately credited, and the proposed method is sufficiently novel. Lastly the empirical findings counter past evaluations and therefore provide valuable signal. On the other hand, the weaknesses raised during review are (1) issues in notation and lacking ablation (mcfF), (2) questions of clarity (mcfF, dErS, 6QGn), (3) concern for novelty vs. two-stage AT (dErS), and (4) lacking coverage across model sizes like a smaller ResNet-18 in particular and more generally a question of limitations (iqHN). Each of the weaknesses was satisfactorily resolved by the rebuttal and discussion, and during the AC-reviewer discussion phase no additional issues were raised.

The productive discussion and clarifications led to higher scores for mcfF and 6QGn and iqHN. The thread with iqHN is significant, because it was the most negative, but it ultimately resulted in support of borderline acceptance. The area chair monitored this thread closely, and commends the authors for preserving a factual and respectful tone, and acknowledges the reviewer for the respectful and thoughtful conclusion of the thread.

Miscellaneous feedback:

- There is a typo in the abstract for "generation gap" vs. "generalization gap".
- This is a matter of style, but the number of significant digits in plots such as Figures 1 & 3 are perhaps distractingly high for the range of results, so consider lesser precision for presentation.
Decision: Accept (poster)