Title: Effective Targeted Attacks for Adversarial Self-Supervised Learning
Venue: NeurIPS 2023
Accepted: yes
Keywords: Adversarial self supervised learning

targeted attack

self supervised learning

contrastive learning

positive mining
Abstract: Recently, unsupervised adversarial training (AT) has been highlighted as a means of achieving robustness in models without any label information. Previous studies in unsupervised AT have mostly focused on implementing self-supervised learning (SSL) frameworks, which maximize the instance-wise classification loss to generate adversarial examples. However, we observe that simply maximizing the self-supervised training loss with an untargeted adversarial attack often results in generating ineffective adversaries that may not help improve the robustness of the trained model, especially for non-contrastive SSL frameworks without negative examples. To tackle this problem, we propose a novel positive mining for targeted adversarial attack to generate effective adversaries for adversarial SSL frameworks. Specifically, we introduce an algorithm that selects the most confusing yet similar target example for a given instance based on entropy and similarity, and subsequently perturbs the given instance towards the selected target. Our method demonstrates significant enhancements in robustness when applied to non-contrastive SSL frameworks, and less but consistent robustness improvements with contrastive SSL frameworks, on the benchmark datasets.

Effective Targeted Attacks for Adversarial Self-Supervised Learning

Abstract

Recently, unsupervised adversarial training (AT) has been highlighted as a means of achieving robustness in models without any label information. Previous studies in unsupervised AT have mostly focused on implementing self-supervised learning (SSL) frameworks, which maximize the instance-wise classification loss to generate adversarial examples. However, we observe that simply maximizing the self-supervised training loss with an untargeted adversarial attack often results in generating ineffective adversaries that may not help improve the robustness of the trained model, especially for non-contrastive SSL frameworks without negative examples. To tackle this problem, we propose a novel positive mining for targeted adversarial attack to generate effective adversaries for adversarial SSL frameworks. Specifically, we introduce an algorithm that selects the most confusing yet similar target example for a given instance based on entropy and similarity, and subsequently perturbs the given instance towards the selected target. Our method demonstrates significant enhancements in robustness when applied to noncontrastive SSL frameworks, and less but consistent robustness improvements with contrastive SSL frameworks, on the benchmark datasets.

1 Introduction

Enhancing the robustness of deep neural networks (DNN) remains a crucial challenge for their real-world safety-critical applications, such as autonomous driving. DNNs have been shown to be vulnerable to various forms of attacks, such as imperceptible perturbations [14], various types of image corruptions [20], and distribution shifts [25], which can lead DNNs to make incorrect predictions. Many prior studies have proposed using supervised adversarial training (AT) [29, 40, 38, 37] to mitigate susceptibility to imperceptible adversarial perturbation, exploiting class label information to generate adversarial examples. However, achieving robustness in the absence of labeled information has been relatively understudied, despite the recent successes of self-supervised learning across various domains and tasks.

Recently, self-supervised learning (SSL) frameworks have been proposed to obtain transferable visual representations by learning the similarity and differences between instances of augmented training data. Such prior approaches include those utilizing contrastive learning between positive and negative pairs (e.g., Chen et al. [6] (SimCLR), He et al. [19] (MoCo), Zbontar et al. [39] (Barlowtwins)), as well as those utilizing similarity loss solely between positive pairs (e.g., Grill et al. [17] (BYOL), Chen and He [7] (SimSiam)). To achieve robustness in these frameworks, Kim et al. [23] and Jiang et al. [22] have proposed adversarial SSL methods using contrastive learning [6], which generate adversarial examples that maximize the instance-wise classification loss.

Unfortunately, deploying this contrastive framework often becomes computationally expensive as it requires a large batch size for training in order to attain a high level of performance [6]. Specifically, when a given memory and computational budget is limited, such as with edge devices, performing

adversarial

after perturbation

clean

clean

clean

adversarial

after perturbation

clean

clean

clean

adversarial

after perturbation

clean

clean

clean

maximize

negative pairs

Non-effective

adversarial

Effective

adversarial

maximize

target image

positive pairs

!

minimize

maximize

negative pairs

Non-effective

adversarial

Effective

adversarial

maximize

target image

positive pairs

!

minimize

maximize

negative pairs

Non-effective

adversarial

Effective

adversarial

maximize

target image

positive pairs

!

minimize

(b) Contrastive based attack

!

target image

adversarial

Effective

positive pairs

after perturbation

maximize

minimize

!

target image

Non-effective

adversarial

Effective

adversarial

positive pairs

after perturbation

maximize

minimize

!

target image

Non-effective

adversarial

Effective

adversarial

positive pairs

after perturbation

maximize

minimize

adversarial

Non-effective

contrastive SSL becomes no longer viable or practical as an option, as it may not obtain sufficiently high performance using a small batch size.

Alternatively, non-contrastive, positive-only SSL frameworks have been proposed resort to maximizing consistency across two differently augmented samples of the same instance, i.e., positive pairs, [17, 7, 39], without the need of negative instances. These approaches improve the practicality of SSL for those limited computational budget scenarios. However, leveraging prior adversarial attacks that maximize the self-supervised learning loss in these frameworks results in extremely poor performance compared to those of adversarial contrastive SSL methods (Table 1). The suboptimality of the deployed attacks causes to learn limited robustness and leads to the generation of ineffective adversarial examples, which fail to improve robustness in the SSL frameworks trained using them. As shown in Figure 1c, the attack in the inner loop of the adversarial training loss, designed to maximize the distance between two differently augmented samples, perturbs a given example to a random position in the latent space. Thus, the generated adversarial samples have little impact on the final robustness. The suboptimality of the attacks also can be occurred in contrastive adversarial SSL that also contains positive pairs, when simply maximizing the contrastive loss. As shown in Figure 1b, contrastive learning treats all positive and negative pairs equally regardless of their varying importance in generating effective adversarial examples.

To address this issue, we propose T argeted A ttack for RO bust self-supervised learning (TARO). TARO is designed to guide the generation of effective adversarial examples by conducting targeted attacks that perturb a given instance toward a target instance to enhance the robustness of an SSL framework (Figure 1). The direction of attacks is assigned using our target selection algorithm that chooses the most confusing yet similar sample for a given instance based on the entropy and similarity. By targeting the attacks toward specific latent spaces that are more likely to improve robustness on positive-pairs, TARO improves the robustness of SSL, regardless of the underlying SSL frameworks. Notably, as the positive-pair only SSL has gained attention in recent times, our proposed method becomes crucial for the ongoing safe utilization of these frameworks in real-world applications.

The main contributions can be summarized as follows:

We observe that simply maximizing the training loss of self-supervised learning (SSL) may lead to suboptimality of attacks as the main cause of the limited robustness in SSL frameworks, especially those that rely on maximizing the similarity between the single pair of augmented instances.

To address this issue, we propose a novel approach, T argeted A ttack for RO bust selfsupervised learning (TARO), which aims to improve the robustness of SSL by conducting targeted attacks on the positive-pair that perturb the given instance toward the most confusing yet similar latent space, based on entropy and similarity of the latent vectors.

We experimentally show that TARO is able to obtain consistently improved robustness of SSL, regardless of underlying SSL frameworks, including contrastive- and positive-pair only SSL frameworks.

2 Related Work

Adversarial training Szegedy et al. [34] showed that imperceptible perturbation to a given input image may lead a DNN model to misclassify the input into a false label, demonstrating the vulnerability of DNN models to adversarial attacks. Goodfellow et al. [14] proposed the fast gradient sign method (FGSM), which perturbs a given input to add imperceptible noise in the gradient direction of decreasing the loss of a target model. They also demonstrated that training a DNN model over perturbed as well as clean samples improves the robustness of the model against FGSM attacks. Follow-up works [27, 2] proposed diverse gradient-based strong attacks, and Madry et al. [29] proposed a projected gradient descent (PGD) attack and a robust training algorithm leveraging a minimax formulation; they find an adversarial example that achieves a high loss while minimizing the adversarial loss across given data points. TRADES [40] proposed minimizing the Kullback-Leibler divergence (KLD) over clean examples and their adversarial counterparts, thus enforcing consistency between their predictions. Recently, leveraging additional unlabeled data [3] and conducting additional attacks [38] have been proposed. Carmon et al. [3] proposed using Tiny ImageNet [28] images as pseudo labels, and Gowal et al. [16] proposed using generated images from generative models to learn richer representations with additional data.

Self-supervised learning Due to the high annotation cost of labeling data, SSL has gained a wide attention [11, 41, 35, 36]. Previously, SSL focused on solving a pre-task problem of collaterally obtaining visual representation, such as solving a jigsaw puzzle [30], predicting the relative position of two regions [11], or impainting a masked area [31]. However, more recently, SSL has shifted to utilizing inductive bias to learn the invariant visual representation of paired transformed images. This is accomplished through contrastive learning, which utilizes both positive pairs and negative pairs, that is differently transformed images and other images from the same batch, respectively [6, 19]. Additionally, some studies have proposed using only positive pairs in SSL and have employed techniques such as momentum networks [17] or stop-gradient [7]. In this paper, we annotate these approaches as contrastive SSL, and positive-pair only SSL, respectively.

Adversarial self-supervised learning The early stage of adversarial SSL methods [23, 22] employed contrastive learning to achieve a high level of robustness without any class labels. Adversarial self-supervised contrastive learning [23, 22] generated an instance-wise adversarial example that maximizes the contrastive loss against its positive and negative samples by conducting untargeted attacks. Both methods achieved robustness, but at the cost of requiring high computation power due to the large batch size needed for contrastive learning. On the other hand, Gowal et al. [15] utilized only positive samples to obtain adversarial examples by maximizing the similarity loss between the latent vectors from the online and target networks, allowing this method greater freedom regarding the batch size. However, it exhibited relatively worse robustness than the adversarial self-supervised contrastive learning frameworks. Despite the advances in the SSL framework (i.e., positive-pair only SSL), a simple combination of untargeted adversarial learning and advanced SSL does not guarantee robustness. To overcome such a vulnerability in positive-pair only SSL, we propose a targeted attack leveraging a novel score function designed to improve robustness.

3 Positive-Pair Targeted Attack in Adversarial Self-Supervised Learning

Adversarial SSL and supervised adversarial learning utilize adversarial examples in a similar manner. Specifically, adversarial SSL generates instance-wise adversarial examples in the direction of maximizing the training loss for better robustness. However, this approach exhibits an insufficient level of robustness especially in the positive-pair only self-supervised learning framework due to generating highly suboptimal adversarial examples.

We argue that simply maximizing the training loss, dubbed as an untargeted attack, in positive-pair only SSL limits the diversity of adversarial examples which eventually leads to limited robustness. We theoretically show that range of perturbation is smaller when the positive-pair only SSL objective is employed in an untargeted attack than the contrastive objective in simple two-class tasks. Furthermore, we empirically demonstrate poorer robustness when we naively merge untargeted attack and positivepair only SSL approaches [17, 7], compared to contrastive-based adversarial SSL [23, 22].

To remedy such a shortcoming, we propose a simple yet effective targeted adversarial attack to increase the diversity of the generated attack. Moreover, we empirically suggest novel positive mining the target for the targeted adversarial attack that contributes to generating more effective and stronger adversarial examples, thus improving the robustness beyond that of previous adversarial SSL approaches. In this section, we first recap supervised adversarial training, self-supervised learning, and previous adversarial SSL methods. We then demonstrate theoretical intuition on our motivation and describe our proposed targeted adversarial SSL framework, TARO, in detail.

3.1 Preliminary

Supervised adversarial training We first recap supervised adversarial training with our notations. We denote the dataset D = { ( x i , y i ) } , where x i ∈ R D is a input, and y i ∈ R N is its corresponding label from the N classes. In this supervised learning task, the model is f θ : X → Y , where θ is a set of model parameters to train.

Given D and f θ , an adversarial attack perturbs a given source image that maximizes the loss within a certain radius from it (e.g., ℓ ∞ norm balls). For example, ℓ ∞ attack is defined as follows:

where B (0 , ϵ ) is the ℓ ∞ norm-ball of radius ϵ , Π is the projection function to the norm-ball, α is the step size of the attacks, and sign ( · ) is the sign of the vector. Also, δ represents the perturbations accumulated by α sign ( · ) over multiple iterations t , and L CE is the cross-entropy loss. In the case of PGD [29], the attack starts from a random point within the epsilon ball and performs t gradient steps, to obtain a perturbed sample. Adversarial training (AT) is a straightforward way to improve the robustness of a DNN model; it minimizes the training loss that embeds the adversarial perturbation ( δ ) in the inner loop (Eq. 1).

Self-supervised learning Recent studies on self-supervised learning (SSL) have proposed methods to allow their models to learn invariant features from transformed images, thus learning semantic visual representations that are beneficial for diverse tasks [6, 19, 17, 7, 39]. In this paper, we aim at improving the robustness of the two most popular types of SSL frameworks: positive-pair only SSL (e.g., BYOL, SimSiam) and contrastive SSL (e.g., SimCLR) frameworks.

We start by briefly describing a representative contrastive SSL, SimCLR [6]. SimCLR is designed to maximize the agreement between different augmentations of the same instance in the learned latent space while minimizing the agreement between different instances. Differently augmented examples from the same instance are defined as positive pairs, and all other instances in the same batch are considered negative examples. Then, the training loss of SimCLR is defined as follows:

where z is the latent vector of input x , pos , neg stands for positive pair and negative pairs of x , respectively, and sim denotes the cosine similarity function.

A representative positive-pair only SSL framework is SimSiam [7]. SimSiam consists of the encoder f , followed by the projector g , and then the predictor h ; both g and h are multi-layer perceptrons (MLPs). Given the dataset D = { X } and the transformation function t ∼ T that augments the images x ∈ X , it is designed to maximize the similarity between the differently transformed images and avoid representational collapse by applying the stop-gradient operation to one of the transformed images as follows:

where z = g ◦ f (t 1 ( x )) , z pos = g ◦ f (t 2 ( x )) , and p = h ◦ z , p = h ◦ z pos are output vectors of the projector g and predictor h , respectively. Before calculating the loss, SimSiam detaches the gradient on the z , which is called the stop-gradient operation. This stop-gradient operation helps the model prevent representational collapse without any momentum networks, by making an encoder to act as a momentum network.

Adversarial SSL To achieve robustness in SSL frameworks, prior studies have proposed adversarial SSL methods [22, 23, 15]. They generate adversarial examples by maximizing the training loss, dubbed as an untargeted attack, of their base SSL frameworks. For example, the inner loop of an adversarial attack for Kim et al. [23] is structured as follows:

where the perturbation maximizes the L . For adversarial contrastive SSL approaches [22, 23], L = L nt-xent is the contrastive loss in Eq. 2, so that adversarial examples are generated to minimize the similarity between positive pairs and maximize the similarity between negative pairs. For the positive-pair only SSL, adversarial examples are generated to maximize the similarity loss, L = L ss (Eq. 3), between positive-pairs only. However, as shown in Table 1, positive-pair only SSL results in significantly poor robustness compared to the adversarial contrastive SSL approaches. This is because using the naive training loss function of positive-pair only SSL in the attack hinders the generation of effective attack images for robust representation, as we theoretically show the range of perturbations is smaller (Section 3.2). To address this issue, we propose a targeted adversarial attack that can select more effective examples to make more diverse perturbations.

3.2 Theoretical Motivation: Adversarial Perturbations in Positive-only SSL

A model is considered to have a better generalization of adversarial robustness when the model can maintain its performance across a wide range of adversarial perturbations. Hence, the ability of the attack loss to generate a diverse range of perturbations during training is a crucial factor that influences the model's final robust generalization.

However, we found the theoretical motivation that positive-pair only SSL loss ( L ) could not provide

*na¨ ıve adversarial training applied in SimSiam ss a wide range of adversarial perturbations as contrastive loss ( L nt-xent ) does. We simplify the problem into simple binary classification with the linear layer model to demonstrate our theoretical motivation. Let us denote adversarial perturbations that are generated with both losses as follows,

where we approximate the loss of L ss in the ℓ 1 distance function between the positive pair and the loss of L nt-xent into combination of two ℓ 1 distance functions of one positive- and one negative- pair. In both cases, a δ maximizes the respective loss, subject to the constraint that the norm of δ is less than or equal to ϵ . The objective in positive-only SSL is to make the perturbed and original samples dissimilar as follows,

while the objective of nt-xent is to make the perturbed sample dissimilar to the positive pair and similar to the negative pair as follows,

Theorem 3.1 (Perturbation range of self-supervised learning loss) . Given a model trained under the positive-only distance loss, the adversarial perturbations δ ss are likely to be smaller than those perturbations δ nt-xent from a model trained under the positive-pair and negative-pair distance loss. Formally, ∥ δ ss ∥ ∞ < ∥ δ nt-xent ∥ ∞ .

These theoretical insights are also supported by the empirical experiments in Table 1 that a model trained with adversarial examples generated using positive- and negative- paired contrastive loss ( L nt-xent ) have better adversarial robustness generalization because it is exposed to a wider range of perturbations during training than models that are trained with the positive-only similarity loss ( L ss ). The detailed proof and the empirical analysis are in the Supplementary.

3.3 Targeted Adversarial SSL

Wepropose a simple yet effective targeted adversarial attack to generate effective adversarial examples in a positive-only SSL scenario. In this section, we first show the theoretical intuition of our approach and describe our overall framework to further improve the robustness of the adversarial SSL method by performing targeted attacks wherein targets are selected according to the proposed score function.

Targeted adversarial attack to different sample We argue that leveraging untargeted adversarial attacks in positive pairs only SSL still leaves a large room for better robustness. To enlarge the diversity of the attacks, we propose simple targeted adversarial attacks for positive-pair only SSL. The loss for such adversarial attacks is as follows:

where L targeted-attack is L ours-ss = -L ss , and x ′ is a selected target within the batch.

Therefore, in the previous simplified scenario described in Section 3.2, conducting the randomly selected targeted attack could increase the range of the perturbation that is generated with positive-pair only similarity loss as follow,

Through triangle inequality, the targeted attack may increase the range of the perturbation and eventually leverage the overall robustness.

Theorem 3.2 (Perturbation range of targeted attack) . Given a model trained under the L targeted-attack loss, the adversarial perturbations δ targeted-attack are larger than the adversarial perturbations δ ss from a model trained under the L ss . Formally, ∥ δ targeted-attack ∥ ∞ > ∥ δ ss ∥ ∞ .

However, these are theoretical expectations in a simplistic scenario. To further substantiate this, we empirically observed that even a simple targeted attack, with a random target in the batch, significantly improves robustness in a positive-pair only SSL scenario, as shown in Table 2. Therefore, based on these theoretical and empirical insights, we propose to search more effective target for positive-pair targeted attack to boost the robustness of the self-supervised learning frameworks

*adversarial training applied in SimSiam through experimental observations. The detailed proof of Theorem 3.2 is in the Supplementary.

Similarity and entropy-based target selection for targeted attack In our theoretical analysis and empirical observations, we established that targeted attacks can significantly enhance overall robustness in SSL, except for the target itself. To this end, we propose a score function, denoted as S ( x, · ) , which aims to identify the most suitable target that is distinct from the input while effectively contributing to improved robustness. Following the studies of Kim et al. [24], Ding et al. [10], Hitaj et al. [21], we prioritize high-entropy examples or those located near decision boundaries as crucial for generating effective adversarial examples in supervised adversarial training. Accordingly, we recommend selecting a target distinct from itself, yet induces confusion, creating adversarial examples that are located close to decision boundaries (Eq.11). The score function yields the most potent target ( x ′ ) for a given base image ( x ). Subsequently, the targeted attack generates a perturbation, maximizing the similarity to the target x ′ for the base image x .

To this end, we design the score function based on the similarity and entropy values, without using any class information, as follows:

where p = h ◦ g ◦ f ( x ) and e = f ( x ) are output vectors of predictor h and encoder f , respectively. Overall, the score function S incorporates both cosine similarity and entropy. The cosine similarity

is calculated between features of base images and candidate images in the differently augmented batch. The entropy is calculated with the assumption that the vector p represents the logit of an instance as Caron et al. [4], Kim et al. [24]. Our score function is designed to select an instance ( x ′ ) that is different but confused with the given image ( x ), thus facilitating the generation of effective adversarial examples for targeted attack (Figure 1). The experimental results in Figure 2b verify that the score function successfully selects such instances, as intended.

Robust self-supervised learning with targeted attacks The TARO framework starts by selecting a target image based on the score function ( S ). It then generates adversarial examples using the selected target and performs adversarial training with them.

For a positive pair, represented as differently transformed augmentations t 1 ( x ) , t 2 ( x ) , the target images t 2 ( x ′ ) and t 1 ( x ′ ) are selected respectively, as ones with the maximum score within the batch from the score function ( S ) in Eq. 11. Then, we generate adversarial examples, i.e., t 1 ( x ) adv , t 2 ( x ) adv , for each transformed input with our proposed targeted attack (Eq. 8), where the targeted loss L targeted-attack = -L ss maximizes the similarity to the selected target t 2 ( x ′ ) and t 1 ( x ′ ) , respectively. Finally, we maximize the agreement between the representations of adversarial images ( t 1 ( x ) adv and t 2 ( x ) adv ) and the clean image t 1 ( x ) as follows:

where L is Eq. 3 for the SimSiam framework. Since all three instances have the same identity, we maximize the similarity between the clean and adversarial examples.

TARO could be also applied to positive pairs in contrastive adversarial SSL methods (e.g., RoCL [23], ACL [22]). Since contrastive SSL does not have a predictor, we use the output of the projector as p in Eq. 10 to select the target for positive-pair. Then, when we apply our targeted attack to their instance-wise attacks, as follows:

where the adversarial loss is a sum of the modified nt-xent loss [6] and similarity loss. Since TARO alters the untargeted attack of the positive pair with a targeted attack between the base image ( t 1 ( x ) ) and target image ( t 1 ( x ′ ) ), we eliminate the positive pair term in nt-xent loss and add similarity loss instead. The similarity loss maximizes the cosine similarity between the t 1 ( x ) images and the t 1 ( x ′ ) images which are searched by the score function. Overall, we generate adversarial examples that maximize the L ours-rocl loss as shown in Algorithm 1.

4 Experiment

In this section, we extensively evaluate the efficacy of TARO with both contrastive and positive-pair only adversarial SSL frameworks. First, we compare the performance of our model to previous adversarial SSL methods that do not utilize any targeted attacks in Section 4.1. Moreover, we evaluate the robustness of the learned representations across different downstream domains in Section 4.2. Finally, we analyze the reason behind the effectiveness of targeted attacks in achieving better robust representations compared to models using untargeted attacks in Section 4.3.

Experimental setup We compare TARO against previous contrastive and positive-pair only adversarial SSL approaches. Specifically, we adapt TARO on top of two contrastive adversarial SSL frameworks, RoCL [23], ACL [22] and a positive-pair only SSL framework, SimSiam [7], to demonstrate its efficacy in enhancing their robustness. All models use the ResNet18 backbones that are trained on CIFAR-10 and CIFAR-100 with ℓ ∞ PGD attacks with the attack step of 10 and epsilon 8 / 255 . We evaluate the robustness of our method against two types of attack, AutoAttack * [8] and ℓ ∞ PGD attacks, with the epsilon size of 8 / 255 , using the attack step of 20 iterations. Clean denotes the classification accuracy of the ResNet18 backbone on the original images. We further describe the experimental details in Appendix B. Code is available in https://github.com/Kim-Minseon/TARO.git

4.1 Efficacy of Targeted Attacks in Adversarial SSL

Wefirst validate whether the proposed targeted attacks in TARO contribute to improving the robustness of positive-pair adversarial SSL frameworks. To evaluate the quality of the learned representations with the SSL frameworks, we utilize linear and robust linear evaluation, as shown in Table 3. Then, we validate the generality of TARO to contrastive-based adversarial SSL frameworks (Table 5).

Robustness improvements in positive-pair only SSL We evaluate the efficacy of TARO by comparing those to untargeted attacks on positive-pair only SSL frameworks, i.e., SimSiam and BYOL. As shown in Table 3, when replacing untargeted attacks with TARO in the positive-only SSL, TARO contributes to attaining significant gains in both robustness accuracy against PGD attacks and clean accuracy. This is due to the inherent limitations of untargeted attacks in positive-pair only SSL frameworks. In such frameworks, perturbations in any direction away from the other pair of samples will

*na¨ ıve adversarial training applied in SimSiam inevitably increase the SSL loss, making it challenging to generate effective adversarial examples. However, with the guidance provided by TARO, the model is able to generate stronger attack images, leading to meaningfully improved performance both on clean and adversarially perturbed images. Furthermore, we show that the untargeted attacks are not only ineffective for learning robust features, but also hinder the learning of good visual representation for clean images.

Switching from an untargeted to a targeted attack approach leads to a substantial increase in performance across both contrastive-based and positive-pair only approaches, as shown in Table 4. This advancement is particularly evident when addressing the challenge of selecting appropriate targets within positive pairs. As we have discussed in the Limitations section, our empirical score function may not be the absolute optimal algorithm for target selection. Nevertheless, it is clear that concentrating on targeted attacks in the context of positive pairs is crucial for enhancing robust representation, applicable to both clean and adversarial examples.

Robustness improvements in contrastive adversarial SSL The robustness gains through TARO in contrastive adversarial SSL, specifically RoCL and ACL, are demonstrated in Table 5. Given that our TARO algorithm mines the positive-pair in contrastive loss, its effects on contrastive-based SSL might be more limited compared to positive-pair SSL. Despite this, TARO enhances RoCL's robustness against PGD attacks from 42.89% to 45.37% without compromising the clean accuracy. In the case of ACL, TARO fortifies the robustness against PGD attacks while maintaining performance comparable to AutoAttack.

4.2 Evaluation on CIFAR-100

Robustness on larger benchmarks datasets We further validate our method on a larger dataset, CIFAR-100. In Table 6, TARO demonstrates consistent robust accuracy when compared with those of the adversarial SSL frameworks using untargeted attacks, with notably significant robustness improvements on the positive-pair only SSL. Although the clean and original robust accuracy of the positive-only SSL method is noticeably lower than that of the contrastive learning method on this particular dataset, it achieves significantly higher robust accuracy than the contrastive counterpart

when using our targeted attack. The results further suggest that the proposed targeted attack plays a crucial role in creating effective adversarial examples.

Transferable robustness The main objective of SSL is to learn transferable representations for diverse downstream tasks. Therefore, we further evaluate the transferable robustness of the pretrained representations trained using our targeted attack on novel tasks from a different dataset. We adopt the experimental setting from the previous works on supervised adversarial transfer learning [32] which freeze the encoder and train only the fully connected layer. We pretrained the model on CIFAR-100 and evaluate the robust transferability to

CIFAR-10. In Table 7, our model also shows impressive transferable robustness both with contrastive and positive-pair only SSL, compared to those obtained by the representations learned with untargeted adversarial SSL.

4.3 Effectiveness of TARO

In this section, we further analyze the effect of the targeted attacks in adversarial SSL to see how and why it works. 1) Analysis of the selected images by S , 2) Visual representation of adversarial examples that are generated with untargeted attack/targeted attack, and 3) ablation experiment on each component of the score function.

Analysis of the selected target To analyze which target images are selected by our score function ( S ), we use a supervised adversarial training (AT) model. We select the target images of a single class (airplane) with the score function, and forward them to the supervised AT model to obtain their class distribution. To further examine which are the most confusing classes for the original images, we forward the base airplane images to the supervised AT model as well. As shown in Figure 2a, airplane images are easily confused with the ship class and the bird class. Surprisingly, 1/3 of the target images are selected using our target selection function for airplane images belonging to either ship or the bird class, which are the most confusing classes for the images belonging to the airplane class (See Figure 2b). These results strongly support that our score function effectively selects targets that are similar yet confused, as intended, without using any label information.

0

0.2

0.4

plane

ship

car

truck

bird

cat

deer

dog

frog

horse

(a) Mean predict proba-

bility of base images

All samples

Misclassified samples

Number of Samples

0

200

400

plane

ship

car

truck

bird

cat

deer

dog

frog

horse

(b) Distribution of class of targeted images

(a) Untargeted attack

(b) Targeted attack

Visualization of embedding space To examine the differences between images that are generated with targeted and untargeted attacks, we visualize their embedding space. In Figure 3, black markers represent adversarial examples, and light blue markers represent clean examples, both belonging to the same class. As shown in Figure 3a, untargeted adversarial examples are located near clean examples,

and far from the class boundaries. On the other hand, targeted adversarial examples are located near the class boundaries (Figure 3b), although it is generated in an unsupervised manner without any access to class labels. This visualization shows that our targeted attack generates relatively more effective adversarial examples than untargeted attacks, which is likely to push the decision boundary to learn more discriminative representation space for instances belonging to different classes.

Ablation study of the score function To demonstrate the effect of each component in our score function, we conduct an ablation study of the score function S . The score function consists of two terms: the entropy term and the cosine similarity term (Eq. 10), which together contribute to finding an effective target that is different but confusing. We empirically validate each term by conducting an ablation

experiment using only a single term in the score function during adversarial SSL training in Eq. 11. The experimental results in Table 8, suggest that the entropy term leads to good clean accuracy while the similarity term focuses on achieving better robust performance. Thus the combined score function enables our model to achieve good robustness while maintaining its accuracy on clean examples.

5 Conclusion

In this paper, we demonstrate that a simple combination of supervised adversarial training with self-supervised learning is highly suboptimal due to the ineffectiveness of adversarial examples generated by untargeted attacks in positive-pair only SSL, which perturb to random latent space without considering decision boundaries. To address this limitation, we proposed an instance-wise targeted attack scheme for adversarial self-supervised learning. This scheme selects the target instance based on similarity and entropy, such that the given instance is perturbed to be similar to the selected target. Our targeted adversarial self-supervised learning yields representations that achieve better robustness when applied to any type of adversarial self-supervised learning, including positive-pair only SSL and contrastive SSL. We believe that our work paves the way for future research in exploring more effective attacks for adversarial self-supervised learning.

Limitations

Our method's main constraint is that our score function's design relies on empirical design based on the previous works. Establishing the most optimal score function theoretically for a high-dimensional, non-linear deep learning model is a complex task. Despite this, we've provided a theoretical basis for how a targeted attack can improve robustness in a simple scenario for positive pairs. Our experimental results also confirm our score function's effectiveness, suggesting we've made various efforts to counterbalance our limitations. Additionally, our method demands more computational time than a simple untargeted adversarial training, given the need to select a target instance. Yet, this extra computational time is less than 5% compared to original training time. Considering the significant boost in robustness, we believe it's a reasonable trade-off to implement our method. Despite these limitations, we've identified a significant vulnerability in the untargeted attack method-an essential discovery for adversarial self-supervised learning. Moreover, we suggest a simple yet effective way to address this vulnerability in adversarial self-supervised learning.

References

Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: a query-efficient black-box adversarial attack via random search. In European Conference on Computer Vision , pages 484-501. Springer, 2020.

Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE symposium on security and privacy (sp) , pages 39-57. IEEE, 2017.

Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John C Duchi. Unlabeled data improves adversarial robustness. Advances in Neural Information Processing Systems , 2019.

Mathilde Caron, Hugo Touvron, Ishan Misra, Herv´ e J´ egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In IEEE Conference on Computer Vision and Pattern Recognition , pages 9650-9660, 2021.

David M Chan, Roshan Rao, Forrest Huang, and John F Canny. Gpu accelerated t-distributed stochastic neighbor embedding. Journal of Parallel and Distributed Computing , 131:1-13, 2019.

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning , 2020.

Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15750-15758, 2021.

Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference on Machine Learning , pages 2206-2216. PMLR, 2020.

Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack. In International Conference on Machine Learning , pages 2196-2205. PMLR, 2020.

Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Mma training: Direct input space margin maximization through adversarial training. In International Conference on Learning Representations , 2020.

Alexey Dosovitskiy, Philipp Fischer, Jost Tobias Springenberg, Martin Riedmiller, and Thomas Brox. Discriminative unsupervised feature learning with exemplar convolutional neural networks. IEEE transactions on pattern analysis and machine intelligence , 38(9):1734-1747, 2015.

Lijie Fan, Sijia Liu, Pin-Yu Chen, Gaoyuan Zhang, and Chuang Gan. When does contrastive learning preserve adversarial robustness from pretraining to finetuning? Advances in Neural Information Processing Systems , 34, 2021.

Lianli Gao, Qilong Zhang, Jingkuan Song, Xianglong Liu, and Heng Tao Shen. Patch-wise attack for fooling deep neural network. In European Conference on Computer Vision , pages 307-322. Springer, 2020.

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations , 2015.

Sven Gowal, Po-Sen Huang, Aaron van den Oord, Timothy Mann, and Pushmeet Kohli. Selfsupervised adversarial robustness for the low-label, high-data regime. In International Conference on Learning Representations , 2021.

Sven Gowal, Sylvestre-Alvise Rebuffi, Olivia Wiles, Florian Stimberg, Dan Andrei Calian, and Timothy A Mann. Improving robustness using generated data. Advances in Neural Information Processing Systems , 34:4218-4233, 2021.

Jean-Bastien Grill, Florian Strub, Florent Altch´ e, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent: A new approach to self-supervised learning. Advances in Neural Information Processing Systems , 2020.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition , pages 770-778, 2016.

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In IEEE Conference on Computer Vision and Pattern Recognition , 2020.

Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations , 2019.

Dorjan Hitaj, Giulio Pagnotta, Iacopo Masi, and Luigi V Mancini. Evaluating the robustness of geometry-aware instance-reweighted adversarial training. International Conference on Learning Representations , 2021.

Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning. In Advances in Neural Information Processing Systems , 2020.

Minseon Kim, Jihoon Tack, and Sung Ju Hwang. Adversarial self-supervised contrastive learning. Advances in Neural Information Processing Systems , 2020.

Minseon Kim, Jihoon Tack, Jinwoo Shin, and Sung Ju Hwang. Rethinking the entropy of instance in adversarial training. In First IEEE Conference on Secure and Trustworthy Machine Learning , 2023.

Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton Earnshaw, Imran Haque, Sara M Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. Wilds: A benchmark of in-the-wild distribution shifts. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research , pages 5637-5664. PMLR, 18-24 Jul 2021.

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems , pages 1097-1105, 2012.

Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533 , 2016.

Ya Le and X. Yang. Tiny imagenet visual recognition challenge. In TinyImageNet , 2015.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations , 2018.

Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In European Conference on Computer Vision , pages 69-84. Springer, 2016.

Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 2536-2544, 2016.

Ali Shafahi, Parsa Saadatpanah, Chen Zhu, Amin Ghiasi, Christoph Studer, David Jacobs, and Tom Goldstein. Adversarially robust transfer learning. International Conference on Learning Representations , 2020.

Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation , 23(5):828-841, 2019.

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 , 2013.

Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. In European Conference on Computer Vision , 2020.

Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning. In Advances in Neural Information Processing Systems , 2020.

Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Representations , 2019.

Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. Advances in Neural Information Processing Systems , 33, 2020.

Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and St´ ephane Deny. Barlow twins: Selfsupervised learning via redundancy reduction. In International Conference on Machine Learning , pages 12310-12320. PMLR, 2021.

Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael I Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning , 2019.

Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. In European Conference on Computer Vision , pages 649-666. Springer, 2016.

Effective Targeted Attack for Adversarial Self-Supervised Learning

Supplementary Material

A Baselines.

RoCL [23]. RoCL is SimCLR [6] based adversarial self-supervised learning methods. We experiment with the official code † . To make a fair comparison, we set the attack step to 10 as other baselines. We train the model with 1,000 epochs under the LARS optimizer with weight decay 2 e -6 and momentum with 0 . 9 . For the learning rate schedule, we also followed linear warmup with cosine decay scheduling. We set a batch size of 512 for all datasets (CIFAR-10, CIFAR-100, STL10). For data augmentation, we use a random crop with 0.08 to 1.0 size, horizontal flip with a probability of 0.5, color jitter with a probability of 0.8, and grayscale with a probability of 0.2 for RoCL training.

ACL [22]. ACL is SimCLR [6] based adversarial self-supervised learning methods. We conduct the experiment with the official code ‡ . To make a fair comparison, we set the attack step to 10 as other baselines. We train the model with 1,000 epochs. We set a batch size of 512 for STL10 dataset. For CIFAR-10, and CIFAR-100, we use the official pretrained checkpoints. For data augmentation, we use a random crop with 0.08 to 1.0 size, horizontal flip with a probability of 0.5, color jitter with a probability of 0.8, and grayscale with a probability of 0.2 for ACL training. We set PGD dual mode which calculates both clean and adversarial during the training.

BYORL [15] BYORL is BYOL [17] based adversarial self-supervised learning methods for low label regime. Since there is no official code for BYORL we implement the BYORL by ourselves. We implement based on BYOL from a self-supervised learning library § . We use the same CIFAR-10 setting in the library except for normalization. We exclude normalization in the data augmentation. To make a fair comparison, we implement on the ResNet18 with attack step 10 of PGD. As shown in supplementary materials in [15], when the model is trained with 10 steps in ResNet34 it shows 37.88% of robustness. We conjecture that we have a different performance from the original paper because the original paper employs 40 steps of PGD in WideResNet34 to obtain the reported robustness which requires extraordinary computation power.

AdvCL [12]. AdvCL is SimCLR [6] based adversarial self-supervised learning which employ pseudo labels from the model that is pretrained on ImageNet [26] data. Even though the outstanding performance of AdvCL, we exclude this model as our baseline because the proposed methods require the model that is trained with the labels of ImageNet which we assume to have no label information for training.

B Detailed description of experimental setups.

B.1 Resource description.

All experiments are conducted with a two NVIDIA RTX 2080 Ti, except for the experiments with CIFAR-100 experiments. For CIFAR-100 experiments, two NVIDIA RTX 3080 are used. All experiments are processed in Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz.

B.2 Training detail.

For all methods, we train on ResNet18 [18] with ℓ ∞ attacks with attack strength of ϵ = 8 / 255 and step size of α = 2 / 255 , with the number of inner maximization iterations set to K = 10 . For the optimization, we train every model for 800 epochs using the SGD optimizer with the learning rate of

Algorithm 1 Targeted Attack Robust Self-Supervised Learning (TARO) for contrastive-based SSL

0.05, weight decay of 5e -4 , and the momentum of 0.9. For data augmentation, we use a random crop with 0.08 to 1.0 size, horizontal flip with a probability of 0.5, color jitter with a probability of 0.8, and grayscale with a probability of 0.2. We exclude normalization for adversarial training. We set the weight of adversarial similarity loss w as 2.0. We use batch size 512 with two GPUs.

In the score function, we calculate the similarity score term and the entropy term as shown in Equation 11. First, to exclude the positive pairs' similarity score we set the similarity score between positive pairs to -1 . Then, to calculate the overall score, after obtaining the similarity score and entropy of each sample, we normalize each component with Euclidean normalization to balance each component to the score function. Further, the detailed algorithm of TARO for contrastive SSL is described in Algorithm 1 and Eq. 14.

B.3 Evaluation details.

PGD ℓ ∞ attack. For all PGD ℓ ∞ attacks used in the test time, we use the projected gradient descent (PGD) attack with the strength of ϵ = 8 / 255 , with the step size of α = 8 / 2550 , and with the number of inner maximization iteration set to K = 20 with the random start.

AutoAttack. We further test against a strong gradient-based attack, i.e., AutoAttack (AA) [8]. AutoAttack is an ensemble attack of four different attacks (APGD-CE, APGD-T, FAB-T [9], and Square [1]). AGPD-CE is an untargeted attack, APGD-T and FAB-T are targeted attacks. The Square is a black-box attack. We use an official code to test models ¶ .

Self-supervised learning. For self-supervised learning, we denote linear evaluation when we use only clean images to train the fully connected (fc) layer after the pretraining phase. When we denote robust linear evaluation, we train the fc layer with adversarial examples. While ACL uses partial fine-tuning to obtain their reported accuracy and robustness, to make a fair comparison, we freeze the encoder and train only the fc layer. Robust fine-tuning is training all parameters including parameters of the encoder with adversarial examples. For linear evaluation, we followed the baseline hyperparameters for each model. We train the baseline models with 150 epochs, 25 epochs, and 50 epochs for RoCL, and ACL, respectively. We also followed their learning rate of 0.1, 0.1, and 2 × 10 -3 for RoCL, and ACL, respectively. On the other hand, we train our model with 100 epochs with a learning rate of 0.5 for linear evaluation. We use AT loss for robust linear evaluation except for ACL. For ACL, we use TRADES loss as the official code.

C Experimental Details of Analysis.

Analysis the distribution of target class. To analyze the target from the score function ( S ), we employ an adversarially supervised trained model. We calculate the score function that is trained with our TARO on SimSiam. We use a train set. For each class, we calculate the mean predict probability, which is the average of all softmax outputs of target images from the supervised trained

model. Further, we also count the number of samples that are predicted for each class. In Figure 2, the results are target images of the airplane as a base image. There is a similar tendency even though we change the base class to other classes as shown in the following Figure 4.

All samples

Misclassified samples

Number of Samples

0

200

400

airplane

bird

cat

deer

dog

frog

horse

ship

truck

automobile automobile

0

0.2

0.4

airplane

bird

cat

deer

dog

frog

horse

ship

truck

(b) Mean predict probability of automobile

automobile

All samples

Number of Samples

0

200

400

airplane

bird

cat

deer

dog

frog

horse

ship

truck

Misclassified samples

(c) Distribution of class of target of deer

automobile

0

0.2

0.4

airplane

bird

cat

deer

dog

frog

horse

ship

truck

Visualization of embedding space. To visualize the embedding of our targeted attack and untargeted attack, we use t-Distributed Stochastic Neighbor Embedding (t-SNE) [5] with the cosine similarity metric. Our TARO model is trained on CIFAR-10 as a feature extractor. We sample a few examples and conduct two types of attack, the untargeted attack and the targeted attack. To visualize more effectively we ignore the other seven classes in CIFAR-10. We visualize clean examples from three classes and then visualize adversaries that are generated with our targeted attack and untargeted attack, respectively, with dark blue.

D Additional Experiment

Contrastive based adversarial self-supervised learning with TARO. Our TARO could be also applied to positive pairs in contrastive-based adversarial self-supervised learning (e.g., RoCL [23], ACL [22]). We applied our TARO in instance-wise attack of the contrastive-based approaches as follow,

where attack loss is consists of original attack loss nt-xent loss [6] and similarity loss. The similarity loss additionally constrains the positive pairs as the TARO that maximize the similarity between the x with the j th index images which is searched by our TARO score function. Overall, we generate adversarial examples that maximizes the L attack loss. Surprisingly, when we apply TARO on the contrastive learning based approach, previous work could achieve marginally better clean accuracy and robustness. This shows that our empirical assumption also holds on contrastive-based SSL but since there is ( 1 /batch size) effects on the total loss the gain could be marginal.

Robustness against black box attack We conduct black box attack to verify our model is robust to gradient free attacks. We generate black box adversaries with AT [29] model, RoCL [23] model and our models. Then, we test adversaries to each other. As show in the table, our model is able to defend the black box attack from AT model than the RoCL model. Moreover, our model generates stronger black box adversaries than RoCL since AT model shows more weak robustness.

Robustness against diverse attacks We tested our approach against diverse types of adversarial attacks, including the Carlini-Wagner (CW) attack [2], black-box attack, i.e., Pixle [33], and Patchattack, i.e., PIFGSM [13], as shown in Table 10. Since our approach already showed improved performance against Autoattack, which includes black-

box Square attacks, our approach is able to consistently demonstrates enhanced robustness against both the CW attack and black-box attacks.

E Proof of the Theorem

Let us consider the problem as a simple binary task using a linear layer model to demonstrate our theoretical motivation. The dataset D = X, · consists of training examples, where x ∈ X represents a training example without any class label. We assume there is a single positive pair and a single negative pair. The linear model is denoted as f ( · ) . The adversarial perturbations generated using both losses are as follows:

where we approximate the cosine similarity distance loss into ℓ 1 distance function. In both cases, a δ maximizes the respective loss, subject to the constraint that the norm of δ is less than or equal to ϵ . The objective in positive-only SSL is to make the perturbed and original samples dissimilar as follows,

The range of adversarial attack of each loss is then calculated as follow,

If there are more negative pairs, the difference in perturbation range between positive-pair-only attacks and contrastive attacks could become more pronounced.

Theorem E.1 (Perturbation range of self-supervised learning loss) . Given a model trained under the positive-only distance loss, the adversarial perturbations δ ss are likely to be smaller than those perturbations δ nt-xent from a model trained under the positive-pair and negative-pair distance loss. Formally, ∥ δ ss ∥ ∞ < ∥ δ nt-xent ∥ ∞ .

When applying a random targeted attack within the positive-pair-only self-supervised learning framework, we can effectively increase the range of perturbations. Let us assume that the target instance x target is different from the original instance x , and the distance between them is greater than the threshold δ . The perturbations generated through the targeted attack are as follows:

Let us denote target instance x target as x ′ for simple equations,

The random targeted attack, which targets instances that are at a greater distance than δ from the original input, can potentially increase the perturbation range and ultimately enhance overall robustness.

Theorem E.2 (Perturbation range of targeted attack) . Given a model trained under the L targeted-attack loss, the adversarial perturbations δ targeted-attack are likely to be larger than those from a model trained under the L ss . Formally, ∥ δ targeted-attack ∥ ∞ > ∥ δ ss ∥ ∞ .

F Broader Impacts

The pursuit of adversarial robustness against malicious attacks within deep neural networks remains an unsolved, yet fundamental area of deep learning research. To date, several self-supervised adversarial training approaches have been proposed, primarily based on the contrastive learning framework. However, the attainment of robustness via a 'positive-pair only' self-supervised learning approach is still under-explored. Consequently, self-supervised frameworks have evolved from large batch contrastive learning to a focus on single 'positive-pair only' learning paradigms. The area of self-supervised learning that we are targeting aims to delve into the robustness of these new learning frameworks through our tailored attacks. Furthermore, we believe that achieving superior robustness in self-supervised learning is a crucial research path towards achieving authentic robustness in representation. We hope that our work will inspire more research aimed at achieving generalizable robustness in unseen domains and datasets by leveraging the potential of various self-supervised frameworks.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper highlights that untargeted attacks for adversarial self-supervised learning result in poor downstream robustness. Instead, they suggest utilizing targeted attacks and propose a scoring method for selecting the target samples. They show that this approach results in significant robustness improvements over the untargeted attacks.\n\nSTRENGTHS:\n1. The paper highlights a major issue with adversarial self-supervised learning for non-contrastive models, that the untargeted attacks typically used fail to deliver good downstream robustness.\n2. The proposed method shows significant robustness improvement when compared to positive-only untargeted attack.\n3. The paper is well-written and the definitions and argumentation are easy to follow.\n\nWEAKNESSES:\n1. The proposed method, TARO, has two components but their individual contributions are not studied. TARO improves over the positive-only unsupervised attack by i. changing it to a targeted attack and ii. by selecting the target using a score function. However, there are indications in the prior work and in the results in this paper that the majority of the improvement comes from i. and not ii. For example, (Petrov and Kiwatkowska, 2022) also compare targeted and untargeted adversarial self supervision and show that targeted attacks across the whole batch result in more certifiably robust models than the untargeted ones.  The fact that the improvements in Table 4 in this paper are significantly lower than the improvements in Table 3 is further evidence that it might be i. that drives the improvement rather than ii. Hence, the authors should consider adding an additional baseline to all their results where instead of using their score function, they select a random sample from the batch. This would also be computationally easier, as the paper reports a 5% overhead for the target selection.\n2. The paper does not address the fact that the training loss and the loss used for computing the adversarial samples need not be the same. In fact, a number of prior works have this precise setting, e.g., (Alayrac et al., 2019), (Nguyen et al., 2022). Hence, it is not particularly novel that the loss used for training a positive-only model can be different from the one used for the attacks.\n3. Only RoCL and ACL are compared against the novel method, but a number of other unsupervised adversarial training methods have been proposed as listed above. In order to claim that the target selection via a score function is better than them, the authors should compare with a larger range of methods. In my view, it is especially important to compare against methods that generate adversarial examples using the whole batch, rather than a single target as these methods might be stronger than the proposed TARO. Some examples of such prior work are (Ho and Vasconcelos, 2020), (Fan et al., 2021), (Petrov and Kwiatkowska, 2022) and the Adversarial-to-Adversarial and Dual Stream methods by (Jiang et al., 2020).\n4. In the “Visualization of embedding space” paragraph and Fig. 3, it is actually not that clear whether the targeted attacks generate more samples on the boundary than the untargeted attacks. Visually the two plots seem quite similar. \n\nReferences:\n\nJean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness?, 2019\n\nChih-Hui Ho and Nuno Vasconcelos. Contrastive learning with adversarial examples, 2020\n\nZiyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning, 2020\n\nLijie Fan, Sijia Liu, Pin-Yu Chen, Gaoyuan Zhang, and Chuang Gan. When does contrastive learning preserve adversarial robustness from pretraining to finetuning?, 2021\n\nA. Tuan Nguyen, Ser Nam Lim, and Philip Torr. Task-agnostic robust representation learning, 2022\n\nAleksandar Petrov and Marta Kwiatkowska. Robustness of Unsupervised Representation Learning without Labels, 2022\n\nQUESTIONS:\n1. I did not understand the “Analysis of the selected target”. What is the AT model used? How was it trained? And why was an AT model preferred to a standardly trained one? It is also not particularly clear what Fig. 2a shows. Perhaps more clarity in the writing could help.\n2. The notation in the “Similarity and entropy-based target selection for targeted attack” paragraph is confusing. $p’$ and $p$ are used in the equations but $p_i$ is defined below. Is $S_\\text{entropy}$ a vector or a scalar? If $p$ is a vector of logits then isn’t $S_\\text{entropy}$ a vector? \n\nTypo:\n- The sentence starting on line 361, an unnecessary “are” and “belonging” should be “belong”.\n\nLIMITATIONS:\nAs mentioned in Weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new adversarial attack on self-supervised learning methods which is called TARO. The method performs an attack \"on the positive-pair that perturb the given instance toward the most confusing yet similar latent space, based on entropy and similarity of the latent vectors.\" Authors show that this attack can be used to improve adversarial robustness of the underlying semi-supervised learning framework.\n\nSTRENGTHS:\n- The experimental evaluation shows that TARO can be used to improve adversarial robustness of positive-pair-only self-supervised learning approaches (SimSiam and BYOL).\n- The paper is mostly well-written and comprehensible. It also gives a useful overview of the necessary backgrounds.\n\nWEAKNESSES:\n- Mathematical notation can be improved at some points:\n\t1. In Section 3.1 the authors state that the dataset $\\mathcal{D} = \\\\{X, Y\\\\}$ is a set of the training data and the corresponding labels. This does not make sense: $\\mathcal{D}$ has to be a tuple and X and Y also need to be ordered sets.\n\t2. For me it is not clear what is meant by ${z_{pos,neg}}$ in Equation 2. The used notation suggests that it is a set containing a single element. However, that does not make sense as the sum in the denominator should go over all positive and negative samples. \n\t3. In Equation 3 it is not specified at all over which entries the sum is calculated. \n- The number of evaluated attacks is too small. The authors only compare against PGD and Autoattack. It might be interesting to compare against other attacks, e.g. Carlini-Wagner or black-box attacks.\n- Theorem 3.2 states that \"...adversarial perturbations are *likely* to be larger than...\". However, the next sentence states that this is always the case. I suggest that the authors reformulate that sentence.\n- Figure 1 is difficult to understand. I had to read some more sections first to understand that the positive-pair only attack is proposed in the paper. Maybe, the term should be mentioned earlier (perhaps even in the abstract.)\n\nMinor comment:\n- Using verbatim in formulars is quite uncommon. I suggest the authors use \\text{}, but this is a matter of taste.\n- \"positive-pair only\" vs. \"positive-pair-only\"\n\nOverall, I see merit in this submission and willing to recommend acceptance after the weaknesses mentioned in my review were addressed.\n\nQUESTIONS:\n- How improves adversarial training using TARO robustness against other attacks, e.g. Carlini-Wagner or black-box attacks?\n- Are there any other SSL methods besides BYOL and SimSiam which could benefit from TARO?\n\nLIMITATIONS:\nLimitations are not discussed in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
The paper proposed the targeted attack for adversarial self-supervised learning to solve the suboptimal learning issue in positive pair-only self-supervised learning. To improve the robustness of adversarial self-supervised learning (SSL), the author leverages the targeted selection mechanism based on the score function and generate the samples by selected targets for adversarial training. The paper empirically validates that target attacks can improve the adversarial robustness of several previous adversarial SSL methods including BYORL and SimSiam.

STRENGTHS:
The paper is well-organized and easy to follow. The proposed idea is clear and has good motivation.

WEAKNESSES:
For the transferable robustness part, the author mentioned that TARO can transfer robustness from one task to another. In my opinion, CIFAR100 and CIFAR10 share a similar domain and is not trivial to transfer the robustness. Although the authors evaluate on AutoAttack, which is a well-known strongest attack so far, one thing that I might be curious about is the transferability from the PGD attack (known attack) to other kinds of unknown attacks such as the Patch attack, CW_l2.

QUESTIONS:
Can the proposed method be applied to the transformer-based model, such as ViT?

LIMITATIONS:
yes, the authors adequately addressed the limitation
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

comments: SUMMARY:
This paper investigates the problem of unsupervised adversarial training. It claims that previous non-contrastive, positive-only SSL frameworks suffer from ineffective learning with untargeted adversarial samples. Based on this, this paper proposes a new TARO paradigm, which conducts targeted attacks to select the most confusing but similar samples for guiding the gradients toward a desired direction. Experiments show some great results.

STRENGTHS:
1. The motivation of this paper is clear, which addresses the limitation so existing positive-only SSL frameworks.
2. The paper is well-organized and easy to read.
3. Supplementary file is provided.

WEAKNESSES:
1. The novelty is incremental. Actually, the technical design of targeted adversarial SSL is too straightforward. Although many theorems are provided in the paper to support it, I don't think they are critical to the final model design.

2. Missing many related works. The references are out-of-data, they are all published before 2022. There are many recent papers of this topic, the authors should carefully revisit them and discuss with them.

3. The experiments are unconvincing. The baselines are old, the authors should conduct experiments with latest baselines. Moreover, the ablation study is also insufficient.

QUESTIONS:
See above

LIMITATIONS:
n/a
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-12