ASTrA: Adversarial Self-supervised Training with Adaptive-Attacks
Authors: Prakash Chandra Chhipa
Gautam Vashishtha
Settur Jithamanyu
Rajkumar Saini
Mubarak Shah
Marcus Liwicki
Venue: ICLR 2025 Poster (2025)
Keywords: Self-supervised Adversarial Training
Robustness
Self-supervised Learning
Adversarial Attacks
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: Self-supervised adversarial training powered by self-supervised learnable attack strategy.

Abstract: Existing self-supervised adversarial training (self-AT) methods rely on hand-crafted adversarial attack strategies for PGD attacks, which fail to adapt to the evolving learning dynamics of the model and do not account for instance-specific characteristics of images. This results in sub-optimal adversarial robustness and limits the alignment between clean and adversarial data distributions. To address this, we propose $\textit{ASTrA}$ ($\textbf{A}$dversarial $\textbf{S}$elf-supervised $\textbf{Tr}$aining with $\textbf{A}$daptive-Attacks), a novel framework introducing a learnable, self-supervised attack strategy network that autonomously discovers optimal attack parameters through exploration-exploitation in a single training episode. ASTrA leverages a reward mechanism based on contrastive loss, optimized with REINFORCE, enabling adaptive attack strategies without labeled data or additional hyperparameters. We further introduce a mixed contrastive objective to align the distribution of clean and adversarial examples in representation space. ASTrA achieves state-of-the-art results on CIFAR10, CIFAR100, and STL10 while integrating seamlessly as a plug-and-play module for other self-AT methods. ASTrA shows scalability to larger datasets, demonstrates strong semi-supervised performance, and is resilient to robust overfitting, backed by explainability analysis on optimal attack strategies. Project page for source code and other details at https://prakashchhipa.github.io/projects/ASTrA.

Published as a conference paper at ICLR 2025
ASTRA: ADVERSARIAL SELF-SUPERVISED TRAINING WITH ADAPTIVE-ATTACK
Prakash Chandra Chhipa' Gautam V Settur Anantha Sai Jithamanyu Rajkumar Saini', Mubarak Shah', and Marcus Liwicki
"Luleå Tekniska Universitet, Sweden, Indian Institute of Technology, Gandhinagar, India Indian Institute of Technology, Madras, India, UUniversity of Central Florida, USA
(fprakash. chandra chhipa, zajkumar saini, marcus liwwiwwikkk wttp:sswwwwwikii..uuuo... eltuuse 2 "gautam.pv@alammm ieemiiiiiggeen witgn.aae ac. in, Sed20b0588 smail. iitm. tt..t...... ac.
shahe crev.
ABSTRACT
Existing self-supervised adversarial training (self-AT) methods rely hand- crafted adversarial attack strategies for PGD attacks, which fail to adapt to the evolving learning dynamics of the model and do not account for instance- specific characteristics images. This results in sub-optimal adversarial robust- ness and limits the alignment between clean and adversarial data distributions. To address this, we propose AS7rA (Adversarial Self-supervised Training with AdaptiveAAttacks) novel framework introducing learnable, self-supervised attack strategy network that autonomously discovers optimal attack parameters in single training episode. ASTrA leverages a reward mechanism based on contrastive loss, optimized with REINFORCE, enabling adaptive attack strategies without labeled data additional hyperpa- rameters. We further introduce a mixed contrastive objective to align the dis- tribution of clean and adversarial examples in representation space. ASTrA achieves tfae-offthhe-rtt results CIFARIO, CIFAR100, and STL10 while in- tegrating seamlessly as a plug-and-play module for other self-AT methods. AS- TrA shows scalability to larger datasets, demonstrates strong semi-supervised per- formance, and is resilient to robust overfitting, backed by explainability analysis optimal attack strategies. Project page for source code and other details
through exploration-exploitaii
INTRODUCTION
In era where Convolutional Neural Networks (CNNs) are increasingly deployed in wide range of critical applications across domains like medical image analysis (Wang et al. (2019); Ma et al.
2021 Kaviani et al. (2022)). object detection (Wang et al. (2021a); Hoory et al. (2020): Wei et al. 2018 facial recognition (Vakhshiteh et al. (2020) Akhtar et al. 2021): Biswas et al. (2021), ) autonomous driving (Cao a (2019): Sun al. (2020) Tu et al. (2020) among others, their susceptibility to adversarial attacks has become a pressing concern (Szegedy 20:3); Hendrycks & Dietterich (2019)). These attacks, often imperceptible human observers referred to as Adversarial Examples (AEs), can cause deep models to fail catastrophically. undermining trust in Al systems.
Adversarial Training (AT) emerges as the most prominent defense against adversarial attacks in learning. This approach injects Adversarial Examples (AEs) as part of the training regime of Deep Neural Networks (DNNs). By exposing DNNs to both standard and perturbed samples during training, AT improves generalization to adversarial inputs within a specified c-ball in the input space, resulting in increased invariance to perturbations Madry (017))
Bearring
Although existing works in supervised Adversarial Training (sup-AT) have shown improvements in adversarial robustness (Madry (2017); (Zhan et al. (2019); Wang et al. (2021b)), their dependence on true class labels for crafting adversarial examples (AEs) limits their broader applicability. Most sup- AT methods utilize hand hand-crafted attack strategies, such as the Projected Gradient Descent (PGD)
"qqual contribution

---

Published as a conference paper at ICLR 2025
attack (Mkadry et al. (2017), with predefined parameters like maximal perturbation of 8, 10 iterations, and a step size of 2.
The past few years have seen significant progress in self-supervised learning (SSL) (Misra & Maaten Chen et al. (220)); Noroozi & Favaro (0166; Gidaris et al. (2018)), more specifically Con- trastive Learning (CL) for learning representations without the need for ground truth (GT) labels.
(2020); Chen
Prominent SSL methods, such as SimCLR (Chen et al.] (2020), learn robust representations by
enforcing instance discrimination, treating each instance and its views as a separate class (Purush-
walkam & Gupta 2020)).
Inspired by this progress, multiple recent works (Kim al. 2020); Jiang al. (2020): Fan et al. (021]: Luo et al. (2023)) have positively attempted to leverage unlabelled data for achieving ad- versarial robustness (self-AT). Built as a min-max optimization strategy in Adversarial Contrastive Learning, an 'attacker' crafts input perturbations that attempt to minimize representation similarity for worst-case robustness, and the target network, i.e 'defender', targets maximizing the representa- tion similarity for improved robustness against such perturbed adversarial attacks.
ACL [Jiang et al. (0200) integrates continual learning with adversarial training to establish self-AT framework. Similarly, RoCL. (Kim et al. (2020)) enhances adversarial CL by aligning the distribu- tions of clean and perturbed images. AdvCI. (Fan et al. (202T)) employs knowledge distillation using pseudo labels from pretrained self-supervised models. DeACL (Zhang et al. (2022) intro- duces a two-stage approach that distills a standard pretrained encoder through adversarial training. Recently, DynACL (Luo et al. (023) investigated the impact of augmentation strength adver- sarial pretraining.
While these existing self-AT methods made significant progress in improving robustness, they en- hance representation learning without considering the impact of attack strategies on the learning dynamics. They rely on heuristic, domain-specific techniques craft adversarial examples, using attack strategies derived from supervised adversarial training (sup-AT) heuristics. This limits their ability dynamically adapt attack strategies, potentially compromising the robustness and effective- ness of the learned representations. In contrastive learning, representations are uniformly distributed in the feature space, adhering to the principle of maximum entropy (Jaynes (1957) and typically residing on an n n-dimensional hyper-sphere (Ermolov et al. (202T): Gupta et al. (202)). Since most instances are positioned near class boundaries with dispersed class clusters, effective attack strate- gies should leverage knowledge of the learned representations to perturb samples minimally toward boundaries, thereby avoiding class confusion. This approach is not followed by any of the existing methods.
Although some studies have explored the effects of varying and adaptive attack strategies in sup-AT Tramer et al. ) Yao et al. (2021): Jia et al. ). it is crucial to investigate these impacts in the context of self-AT. This is because, unlike sup-AI, self-AT does not utilize ground truth labels, meaning that insights and methods from sup-AT cannot be directly applied. In sup-AT, attack strate- gies often rely on label information to craft targeted adversarial examples, optimize perturbations based class-specific gradients, and evaluate attack success using label-based metrics. Without access to such labels, Sclf-AT must generate and assess adversarial examples based solely on the data and learned representations. This leads to the following research questions:
1) What is the impact of employing different attack strategies at various training stages in self-AT?
2) How does sample-level variation in attack strategy selection affect robust representation learning in AT?
To answer these questions, our preliminary investigation results, shown in Figure I demonstrate that varying attack strategies both across training stages (left) and at the sample level (right) can enhance adversarial robustness. Specifically, implementing different attack schedules during vari- ous training phases leads to modest improvements over the baseline ACL. Additionally, introducing sample-level variation through random perturbations slightly outperforms the baseline ACL, high- lighting the benefits of adaptive attack strategies. This suggests that samples have varying degrees of vulnerability to adversarial perturbations, and a one-size-fits-all attack strategy may not effectively challenge the model across all instances. However, these experiments still rely heavily on hand- crafted heuristics and domain-specific knowledge, which limits their generalizability. As training progresses, the most effective adversarial examples for enhancing robustness may change, neces-
2

---

Published as a conference paper at ICLR 2025
Effect Varying Attack Strategies during Different Training Stages
Ablation on PGD perturbation
42
llseiine
40
1n7
39
38.21
37.55
--40
Auto
37
ACL basellnne
Epoch
8./255
10.255
11./255
12.255 Random strategy
PGD perturbation
Figure 1: Ablation on PGD perturbations in ACL method. (left): Varying attack strategy by chang- ing perturbations at different steps following different schedules. (right): Experiments with different perturbation values in handcrafted strategy followed by Random strategy where sample-level pertur- bations are chosen randomly from range 3 to 14.
sitating adaptive strategies that respond to the model's evolving performance. Moreover, existing instance-leve variations do account for sample characteristics, resulting in attack strategies that are not truly sample-dependent
ougmertted
original
Self-AT Loss
perees
LeEEN Luv
Lyp
onggeettdd view
Ausiitic attnck moragy
(a)
target
Self-AT Loss for Target
portma
Lyves
Leuean Lyurro Laov
emEek storgy
target
Ley
Strebagy
strategy
Lyw
mosimizatios
REINTONCE grodients
Figure 2: (a) Conventional self-AT with heuristics PGD attack. Here, samples are perturbed us- ing handcrafted attack strategies causing limited scope of adversarial robustness. (b) ASTrA with learnable adaptive attack strategy and mixed contrastive objective. In this case, attack strategies are selected by the adaptive Strategy Network depending upon sample-characteristis and training dy- namics of the target model.
To address these challenges, we propose Adversarial Self-supervised Training with Adaptable At- tacks (ASTrA). ASTrA incorporates novel learnable attack strategy module (refer Fig. (bb that employs an -etrrationnxplliiaation approach (refer Fig. (bb) within single episodic framework to identify optimal attack strategies for each training instance. This is achieved via a novel optimization formulation that leverages policy gradient methods that enable the strategy network to adjust attack strategies by directly influencing the target network's ' learning trajectory to maximize the robust- ness of the network. In the exploration phase, the strategy network explores wide range of attack parameters to gather information about how different strategies affect the model's 's learning. Over time, it converges towards the most effective attack strategies based on feedback from the model's performance, refining the attack parameters to continually challenge the model appropriately, which can be seen in Fig. 3b Our approach uniquely applies adaptive attacks during the self-supervised
3

---

Published as a conference paper at ICLR 2025
Adaptive Strategy Selection of ASTrA
Proportion of Samplis (cans to 100%)
Exploration
Exproitution
T.
Rignardted
oetarke
samplic
(a)
Figure 3: (a) ASTrA's mixed contrastive loss objective facilitating distribution alignment between clean and perturbed samples. Comparison with RoCL in sec. A.3 in appendix. (b) Exploration and Exploitation trends of ASTrA. The strategy model initially explores across perturbation values (exploration) and then assigns optimal values (exploitation) as training progresses for better generalization.
7, 11, 13) maximal proportion of images
pretraining phase, contrast to adaptive sup-AT methods (Ja et al. in fully supervised context.
2022 that utilize adaptability
Additionally, we propose Mixed Contrastive objective (refer Fig. 3a integrated in our framework mitigate the misalignment between adversarial and standard data distributions. Refer sec. A. for details on strategy and target network interactions and sec. A.2 for ASTrA's algorithm.
Our framework outperforms existing self-AT methods in adversarial robustness across multiple datasets, including CIFARIO, CIFAR100, and STL100, and scales effectively to larger datasets such ImageNet-1000 Furthermore, ASTrA operates as plug-and-play module, demonstrating enhanced performance over several state-of-the the--rte art self-AT approaches (see 5).
Our main contributions are:
(1) Self-supervised Learnable Attack Strategy: We introduce novel, self-supervised. adaptable adversarial attack strategy that eliminates the need for human supervision and heuristics. This learn- able strategy optimizes attack strategies at the instance level to maximize adversarial robustness, whilst maintaining standard accuracy, surpassing the capabilities of conventional self-AT methods.
(2) Mixed Contrastive Objective: We propose mixed contrastive objective to address the distri- bution alignment challenge between clean and perturbed sample representations, thereby enhancing generalization and robustness.
2
METHODOLOGY
2.1
PIPELINE OF THE PROPOSED FRAMEWORK
The proposed ASTrA framework integrates two main components: Self-supervised Target Network and Self-supervised Strategy Network. This framework addresses the challenges outlined in the introduction by adaptively optimizing attack strategies and improving representation robustness against adversarial perturbations without relying on handcrafted heuristics or ground truth labels (refer Fig. 26
Self-supervised Target Network. Denoted á (w). where represents target network parameters, this network learns robust feature representations by processing dual input streams: clean augmented views and adversarially perturbed counterparts following ACLDDang et al. (2020). Standard augmentation techniques (e.g., cropping, rotation, jitter) are applied to clean images, while the adversarial perturbations are generated based on strategies learned by the Strategy Network.
Self-supervised Strategy Network. We introduce a novel, learnable, self-supervised strategy net- work designed to overcome the limitations fixed attack strategies employed by existing self-AT methods. Unlike handcrafted strategies, the strategy network curates sample-specific attacks con- sidering both the sample characteristics and the training dynamics the target model to generate optimal attacks that best improve robust generalization (refer Fig. 2b). Self-supervision drives at- tack strategy selection through contrastive rewards. Initially, the strategy network explores wide range of attack strategies (exploration), and training progresses, shifts towards exploiting the
4

---

Published as a conference paper at ICLR 2025
learned dynamics and using the most suitable attack strategies (exploitation). This entire process curs within a single training episode, as shown in Figure (row 1). Formally, the strategy network generates an attack strategy (e.g., PGD step size a, number of iterations 7, perturbation strength e). These parameters are drawn from conditional distribution p(a| 0), parameterized by 0, and updated in real-time based on the feedback loop created by the target network's 's performance. ASTTA addresses the gradient-based updating of the Strategy Network's parameters by utilizing the REINFORCE algorithm WWilliams (1992). This allows us to bypass the non-differeniiabiliy of the adversarial example generation pro- cess and optimize the strategy network's attack policies through policy gradients. Details of this optimization are discussed in Section 2.4
{a,, 02 .... am3 } A, where each an is a discrete attack parameter
Adversarial Example Generator. The adversarial example generation process central to the interaction between the two networks. For a given input 2, an adversarial example Fade is crafted as:
(1) where 8 represents the adversarial perturbation and g(-) encapsulates the perturbation method (typ- ically PGD). Conventional self-supervised adversarial training methods (ACL Jiang et al. (2020).
Fadv
6 where 8 g(z a(0), w),
utilize fixed strategies, fizedd: for generating adversarial examples. The adversarial examples challenge the Target Network to improve its robustness. The attack strategies are sampled from a probability distribution p(a) (x; 0). where 0 is updated during the training.
2.2 THE REWARD FOR THE STRATEGY NETWORK
The reward function for the Strategy Network is designed to balance adversarial robustness with feature consistency across clean examples, leveraging contrastive loss terms that are inherently self- supervised and label-free. This reward mechanism allows the Strategy Network to craft adaptive attack strategies that optimize the perturbation without access to ground truth labels while ensuring the model generalizes well on the clean data distribution.
The adversarial contrastive loss is structured to measure the dissim- ilarity between features of adversarially perturbed augmented views of the same image. By maxi- mizing this dissimilarity, the Strategy Network encouraged to explore adversarial strategies that create stronger perturbations, thereby enhaneing the model's robustness against adversarial attacks: (x fu (u((g(")))/
Advessaiial Contrastive Loss:
Lady (w. 0) = log
(2)
Less (sp fm (24d)))/7²
where undd and oi depend upon 0 as per D and uodd represents all adversarial samples in the hatch excluding 2ndo The temperature parameter controls the sensitivity of the softmax distribution.
Clean Contrastive Loss: Conversely, the clean contrastive loss minimizes the distance between fea- tures of unperturbed augmented views. This ensures that essential information from non-adversarial data is preserved, allowing the model to maintain high performance on the clean data distribution:
exp(sim(f ( (zf'ean), f, (aglea))))
Lekenn (w) - log Exels exp(sim(ffu (zşlean), ), fo (aglean))
where s.lean and sgleaan are unperturbed augmented views of x, and ok slllann includes all clean sam- ples in the batch excluding xcleon_ By minimizing this loss, the Strategy Network ensures that perturbations do not overly distort the learned representations, preserving the model's ability to generalize to standard clean inputs.
Reward Objective:
The composite reward objective for the Strategy Network, aimed at striking balance between robust and standard performance can be written as:
Retrategy (0, w fizee) En [am=p(al=:0) [a (0, fizedd Letean (w Fined)]] }]]
(4)
where a and y are hyperparameters that balance the trade-off ensuring feature consistency (through clean loss minimization) and enhancing adversarial robustness (through adversarial loss maximiza- tion).
The reward objective adapt to sample- variations by continuously learning from the evoly- ing state of the Target Network during training. As the Target Network processes each sample, the Strategy Network adjusts its attack strategy based on the reward feedback. This dynamic interaction
5

---

Published as a conference paper at ICLR 2025
enables the Strategy Network to generate instance-specific perturbations tailored to both the input data and the model'ss training progress, leading to more effective adversarial challenges without dependence on any predefined, heuristic-based attack schedules.
2.3 THE Loss TERMS FOR THE TARGET NETWORK
The Target Network trained using composite loss function that integrates three distinct con- trastive losses: clean, adversarial, and mixed. Each loss plays critical role in ensuring generaliza- tion on clean data while maintaining robustness against adversarial attacks.
The clean and adversarial contrastive losses are aligned with those in the Strategy Network (refer to eq. and 2}. The clean loss preserves the network's s performance
Clean and Adversarial Contrastive Losses:
benignly augmented views, while the adversarial loss minimizes the dis- tance between adversarially perturbed views, enhancing robustness. Minimizing these losses ensures the network performs effectively both clean and perturbed inputs.
This novel proposed loss focuses on aligning the distribution of clean and adversarial samples in the representation space. This alignment prevents robust overfitting by ensuring generalization across a wide range of adversarial attacks:
Mixed Contrastive Loss:
exp(sim(f au (a.tean), (a (aq(d))))/ Iiixed (. 8) log
(5)
Exe; exp(sim( an (nglean), ), Su ()/T
Composite Objective: The overall objective can be written as: Ltarges (ogee(00fzeee (flized w) Ez n
Sace-D(ooaaaae La= (lized) + +B L Olized) LLetean (w))
where a, 3, and y balance the contributions of each term. The mixed loss facilitates continuous alignment between clean and adversarial distributions, supporting the generation of adaptive adver- sarial strategies while maintaining robust generalization.
2.4 NOVEL OPTIMIZATION FORMULATION OF ASTRA USING REINFORCE
The standard formulation of the adversarial training involves a min max formulation of the objective function with adversarial samples generated using hand-crafted adversarial attacks.
min Ex-D [(x) + X max C(x + 8)
BES
In our framework, we use two separate objective functions, one for the optimization of the strategy network and one for the target network. The interaction between the two networks occurs through adversarial sample generation from the attack strategies generated by the strategy network. Given equation 4 anddd 6 we formulate the min-max optimization of ASTrA with for adversarial training as
follows:
min max Ttarger (ffized w) + Retrategy (0, u Fixedy
key challenge in optimizing the Strategy Network comes from the non-differentiable nature of ad- versarial example generation, making traditional gradient-base based methods ineffective. The selection of attack parameters like intensity and perturbation type involves non-differentiable operations that backpropagation cannot handle.
To alleviate this issue, we employ the REINFORCE algorithm Williams (1992), a policy gradient method that allows optimization without requiring differentiable operations. This method enables the Strategy Network to update its parameters based on rewards derived from the Target Network's response, ensuring dynamic learning of attack strategies without labels.
The objective function J(f) for the Strategy Network is maximize the expected reward, which evaluates the success of adversarial attacks:
J(Ø) = E
Σ
R(x, a; ()p(ala:;))
an>p(a/a;;))
where R(x, a: f) measures the effectiveness of generated adversarial examples. The REINFORCE algorithm estimates the gradient of J(O) as:

---

Published as a conference paper at ICLR 2025
VoJ(O) = EnnD
R(x, oz O)p(p x; 0; logp(a z; x;
a>pp(a) (x:0)
RigDD [Baxp(alu:e)
[R(x, a; 0VVo log x; 0]]
This gradient approximation, based on sampled strategies, allows the Strategy Network to refine its attacks by updating parameters according to the observed rewards from the Target Network's performance.
Gradient Ascent Update: The Strategy Network updates its parameters via gradient ascent:
0c+1 0. + e
(12)
where is the learning rate. These updates iteratively refine the attack strategies by adjusting 0 to increase adversarial success.
REINFORCE facilitates managing the non-differentiable updates between the Target and Strategy Networks. While the Strategy Network optimizes attack strategies, the Target Network concurrently updates its parameters to minimize contrastive losses. Reward from the Target Network informs the Strategy Network's s gradient updates, creating a synchronized, co-evolving system that adapts continuously in self-supervised manner. Refer to section A.1 for more details.
Synchronization:
3
EXPERIMENTS
We evaluate ASTrA on the benchmarks CIFARIO, CIFARIO0 Krizhevsky et al. (2009), and STL10 Coates et al. (20TT), comparing against existing self-AT methods: RoCL Kim et al. (2020), ACL Jiang et al. (0200). AdvCL Fan et al. (02T). DeACL Zhang et al. (2022). DYNACL Luo et al.
(2023), , and DYNACL-AIR Xu et al. 2024). Additionally, assess the scalability of ASTrA
the ImageNet-100 Tian et al. (2020).
the target network, incorporating mixed contrastive loss term with a weighting parameter 7 = 0.5, following the protocol from Jiang et al. (222)). ResNet-18 is also used as the adaptive strategy network, with a learning rate of 0.1, LARS optimizer, step sizes ranging from I to 6, attack iterations between 3 and 14, and perturbation range 3 15. Reward weights a and are both set to 1.0 for adversarial and clean losses, respectively. In ResNet, we use bottleneck projector head of size 2048x512, performance comparison with ACL (Jing et (2020) using ACL projector head is in sec. AA.4.4 in appendix. We set 3 to 0.5 for mixed contrastive loss term. ASTrA++ is longer pretraining variant of ASTrA with 2000 epochs.
Pretraining. We use ResNet-18 He et al. (016)
Evaluation. The learned representa- tions are evaluated using three protocols: standard linear finetuning (SLF), adver- sarial linear finetuning (ALF), and ad- versarial full finetuning (AFF) for three accuracy metries Auto Attack Accu- racy (AA), accuracy under PGD-20 as Robust Accuracy (RA), and Standard Accuracy (SA). SI.F and ALF freeze the encoder and the classifier using nat- ural (SLF) or adversarial (ALF) samples with cross-entropy loss. For AFF, the pretrained encoder is used as initializa- tion, and entire model is trained, fol- lowing the approach in ACL Jiang et al. (2200. ACL, RoCL, AdvCL, DeACL results reported from DeACL (Zhang et al. (2022)).
Table 1: SLF results CIFAR1O, CIFAR100, and STL.0. All the methods are evaluated with ResNer18 un- condition following Jiang al. (020). For
der the all metrics (AA, RA, SA), Top two performances high- lighted in bold. ASTrA++ denotes longer pre-training for 2000 epochs.
CIFARIO
STL10
AA
RA
SA
AA
RA
SA
AA
RA
SA
Supervised
46.23
47.50
84.35
23.27
25.86
58.98
29.21
31.34
49,38
RoCL
26.12
28.40
77.90
8.72
11.52
42.93
26.51
28.21
78.19
ACL
37.62
79.32
15.68
17.10
45.34
33.24
AdvCL
37.46
40.54
73.23
15.45
17.05
37.58
45.26
46.18
72.11
DeACL
DYNACI.
45.04
46.72
77.41
19.25
21.40
45.73
46.59
47.38
69.67
DYNACL-AIR
45.17
78.08
20.45
47.66
72.30
ASTA
46.40
54,02
80.54
21.34
24.28
53.20
47.62
48.82
78.00
ASTA+
21.95
4 RESULTS AND ANALYSIS
We evaluate ASTrA across multiple benchmarks against existing self-AT and sup-AT methods. AS- TrA outperforms prior approaches robustness, scalability, and adaptability across various evalua- tion protocols and datasets.

---

Published as a conference paper at ICLR 2025
4.1 ROBUSTNESS ON MULTIPLE BENCHMARKS
In Table we compare the robustness of supervised AT and various self-AT methods on CIFARIO,
CIFAR 100, and STLI0. ASTrA outperforms prior self-AT methods, improving AA accuracy over ACL Jiang et al. (2020) by 8.78% on CIFARIO (to 46.40%), 5.66% on CIFAR 100 (to 21 21.34%) and 2.36% on STLIO (to 47.62%). +STrA++ further enhances robustness, adding 0.52% on CI- FARIO, 0.61% on CIFARI00, and 0.59% on STL10. Both ASTrA and ASTrA++ surpass recent self-AT methods like DYNACL Luo al. (2023) and DYNACL-AIR Xu et al. (2024), demonstrat- ing generalization and scalability. Notably, ASTrA++ exceeds supervised vanilla AT on CIFARIO and STL marking significant advancement in self-AT. Sec. [A.4.3] in the appendix shows the detailed sensitivity analysis of the strategy network's 's learning rate in ASTrA.
4.2 ROBUSTNESS ON DIFFERENT EVALUATION PROTOCOLS
In Table and 3 we evaluate ASTrA and ASTrA++ under different protocols: SLF, AFF, and ALF. ASTrA consistently demonstrates state-of-the- art robustness compared to other self- AT methods across all protocols. Under ALF, ASTrA++ surpasses existing self- AT methods, including DYNACL and DYNACL-AIR, with notable AA im- provement of 1.18% on CIFARIO. In the challenging AFF settings, ASTrA improves upon the sup-AT baseline by 1.88% (from 48.96% to 50.84%) and
Table 2: AFF results CIFARIO, CIFARIO0, and STL.0. ++ denotes longer pre-training.
SSLAT
CIFARIO
CIFARI00
STL.00
AA
RA
SA
AA
RA
SA
AA
RA
SA
Supervised
48.96
49.90
80,23
22.16
26.38
53.34
RoCL
47.88
22.28
27.49
ACL
49.27
52.82
82.19
23.63
29.38
56.61
34.85
35.42
75.11
AdvCL
76.20
DEACI.
50.39
54.18
83.95
25.48
29.65
59.86
47.35
48.24
77.30
DYNACL
50.54
54.26
81.94
25.05
29.10
59.30
48.12
49.85
73.75
DYNACL-AIR
50,60
82.14
25.34
57.44
48.
73.10
ASTIA
50.84
54,90
82.68
26.10
30.22
59.92
49.65
52,40
80.20
ASTIA+
51,20
55.01
83.72
26.45
31,00
60.25
50.15
53,28
79.70
shows superior performance on CIFAR 10 and STLIO. Overall, ASTTA and + achieve state-
of-the-art results across diverse protocols, confirming their robustness and flexibility.
4.3 ASTRA'S SCALABILITY AND ABLATIONS ON CONTRIBUTED COMPONENTS
In Table we evaluate ASTrA's scalability CIFARIO, CIFARION 100, STL10, and the larger ImageNet- 100 dataset us-
Table ALF results on CIFARIO.
ing the AFF protocol. ASTrA effectively scales to larger datasets, achieving competitive performance on ImageNet- 100 demonstrating adaptability to more complex tasks. We
Pretmining Method
AA RA
SA
Sup-AT
47.00
48.12
83,22
RoCI.
29.69
28.72
75.62
also conduct ablations on ASTrA's core components across all datasets. Including the mixed contrastive loss (MC) en- hances performance, improving AA accuracy-for example, by 0.77% on CIFARIO. The adaptive attack strategy (A- Attack) yields further improvements, such as a 1.79% AA gain on CIFAR100.
ACL
40.91
42.00
76.57
AdvCL
37.28
40.58
73.15
DYNACL
45.72
46.90
72,87
DYNACL-AIR
46.01
77.42
ASTrA
46.54
47.62
78.23
ASTIA+
78.77
Combining both compo- nents achieves the high- est gains, with AA im- provements of 1.57% on CIFARIO and 2.59% LITARIOO. These re- sults underscore the im- portance of both Mixed Contrastive and adaptive adversarial policies in en- hancing ASTrA's robust- ness across datasets. The effect of batch frequency updates is detailed in sec. A.4.5
Table 4: Ablations the effect of each component of ASTIA. AFF re- sults and scalability analysis. MC:Mixed Contrastive, A-Atlack: Adap- live Attack Strategy, A-Attack MC: Complete ASTrA.
SSL-AT
CIFARIO
CIFARI00
STL.0
ImageNet-100
AA
SA
AA
SA
AA
SA
AA
SA
ACI.
49.27
5282
82.19
23.63
29.38
56.61
34.85
35.42
75.11
09.24
11.51
19.22
MC
50.04
53.58
82.34
24.6
29.82
58.22
47.05
48.40
77.62
10.48
14.18
21.74
A-Altack
50.58
$4.34
82.56
25.42
30.10
59.40
48.40
51.10
79.00
11.02
15.12
22,50
A-AIlack
50.84
82.68
26.10
59.92
80.20
23.01
Computational
overhead analysis on strategy network is in sec. A.4.I
Comparison of +STrA++ with post process-
ing variants of other methods is in sec. A.4.2
4.4 ASTRA AS PLUG-N- BUIG-N-PLAN --LLAY FRAMEWORK
8

---

Published as a conference paper at ICLR 2025
Table 5: ASTrA as plug-N-play framework with RoCL and DYNACL. AFF on CIFARIO.
As shown in Table integrating ASTrA's adap- tive attack policy into RoCL and DYNACL enhances their robustness across all metrics. Specifically, ASTrA improves AA by 1.22% and RA by 1.89% when combined with RoCL., and boosts AA by 1.38% and RA by 0.78%
Metries
RoCL
RoCL+ASTIA
DYNACL DYNACL+ASTAA
AA
47.88
49.10
50.54
51.92
RA
$1.35
53.24
$4,26
55.04
81.01
82.01
82.55
with DYNACL. These results demonstrate ASTrA's effectiveness plug-and-play framework.
4.5 ASTRA UNDER SEMI-SUPERVISER SETTINGS
As shown in Figure a ASTrA outperforms ACL in semi-supervised settings on CIFARIO, demon- strating clear improvements in Auto-Attack (AA), Robust Accuracy (RA), and Standard Accuracy (SA) across all label ratios. At 50% label ratio, ASTrA achieves significant 2.58% improve- ment in AA and a 1.78% gain in RA over ACL. Even with fewer labels, ASTrA maintains superior performance, highlighting its robustness with limited labeled data.
Auto Accuracy
Accuracy (RA)
Standard Accuracy
50
46:53 47.17
48.50
52.5
55.0
50 50
$1.27
50.86
52.02
80.0
78.54
78.95
79.23
75.20
60.00
E..
44.94 45.25
50.0
27.5-
72.15
33
(S4
2 47.5
E 75.0-
40-
8 45.0
% 72.5
42.5
00.068.8
35.22
ASTIA
40.0
67.56.50
10
20
40
50
10
30
40
20
30
Label Ratio (%6
Label Rabio (%6
Label Ratio 1961
Figure 4: ASTrA consistently outperforms ACL in semi supersupe settings. AFF on on CIFARIO.
4.6
ASTRA ON ROBUST OVERFITTING
Bfrect longur protreining (CHAA-118)
Repular mratraiing
Regusar Probroksin
Rugurr Protroping
20
Epoch
Epoch
Epech
Figure 5: Longer AT ACL ASTrA CIFAR10. SA drops off for ACL.
ASTTA effectively mitigates robust overfitting during longer pretraining on CIFAR1O. as shown in Figure 5] Over extended epochs, ASTrA maintains both Robust Accuracy (RA) and Standard Accuracy (SA), reaching RA of 55.01% % and SA of 83.72% after 2000 epochs. contrast, ACL experiences significant decline in SA after 1000 epochs. indicating overfitting. This demonstrates ASTrA's superior ability to balance robustness and generalization during long-ten training.
4.7 ASTRA ON FINDING OPTIMAL ATTACK PARAMETERS
ASTrA autonomously discovers optimal attack parameters, including step size, perturbation, and attack iterations, by employing a dynamic ixploration-exploiataii approach. Initially, the strategy network explores a wide range of values, searching for effective attack configurations. ASTrA shifts from exploration to exploitation as training progresses, converging on optimal values that balance adversarial robustness and generalization. This adaptive process Figure (row 1), is powered by the novel reward mechanism, which aligns model dynamics to fine-tune attack strategies. Our anal- ysis across datasets, including CIFARIO, 10, STLI0, and ImageNet- 100 (Fig. (row 2)), further shows how ASTrA tailors its attack policies based dataset-specific characteristics. Starting with broad exploration, ASTrA gradually narrows the range of perturbation values, finding optimal points for each dataset. This process requires no manual hyperparameter tuning, ASTrA efficiently balances exploration and exploitation throughout training. ensuring robust defenses without sacrificing clean data performance. Ablations discretization of the attack parameters is in sec. A.4.6

---

Published as a conference paper at ICLR 2025
Perturbation())
Attack Rerations
Tredertion of (aura to 1100
Fromette
600
606
5000
200
1000
6000
300
3900
OPAR-19
$T1-30
3010
Expleration
Exploitartion
Exploration
Ceplottation
Coploration
Eiploitation
Propecaana waregee
Baasch
Spaic
Figure 6: Exploration-explootation phenomena of ASTrA. (row 1) ASTrA on CIFARIO 10 finding opti- mal attack policy (PGD iterations, perturbation(e), step size) by exploration exploration-exploitatioo exploitation phenomena by learnable self-supervised strategy module where entire training a single episode (row 2) ASTrA finding optimal value(s) of perturbation by exploration exploitation across three datasets 10, STL10, and ImageNet100.
exploration-explootttio
4.8 COMPARING OF ASTRA'S ADAPTIVE WITH RANDOM AND HANDCRAFTED ATTACKS
In the introduction, we questioned whether static or random strategies could adapt to the dynamic na-
ture of adversarial training. While the random and
scheduled strategies show slight improvements (Fig- ure 7) they still fall short in fully optimizing at-
tack parameters due to their lack of adaptability.
The random strategy lacks convergence, and the sched- uled approach remains inflexible the evolving model dynamics. However, these initial results indicated that an adaptive strategy could improve robustness by adjusting these changing conditions.
Figure 7: Perturbation strategies.
Portatbation Mothod
Building on this insight, ASTrA's learnable adaptive strategy dynamically balances exploration and exploita- tion throughout training. As shown in Table 6 ASTrA surpasses both random and scheduled approaches, au- tonomously finding optimal attack parameters. The flat- ter loss landscape (refer sec. (A.5) further demonstrates ASTrA's effectiveness in maintaining both adversarial ro- bustness and generalization, validating the need for an adaptive strategy raised in the introduction.
Table 6: Comparison of attack strategies
SLF evaluation CIFAR 10.
Strategy
AA(%)
ACL
37.65
Random
38.21
Scheduled
38. 48.40
ASTrA (Adaptive)
46.40
CONCLUSION
This work addressed the limitations of static adversarial strategies in self-AT by introducing AS- TrA, self-supervised, adaptive attack framework. ASTrA autonomously optimizes attack param- eters through contrastive reward mechanism, using REINFORCE to enable dynamic, fabel-free adaptation. The framework significantly enhances adversarial robustness and generalization across benchmarks like CIFARIO, CIFAR100, and STL10, with scalability to larger datasets. Our analy- sis shows that adaptive, instance-specifii strategies not only mitigate robustness overfitting but also provide deeper insights into finding optimal adversarial attack parameters, further strengthening the framework's s ability to counter diverse adversarial threats.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Naveed Akhtar, Ajmal Mian, Navid Kardan, and Mubarak Shah. Advances in adversarial attacks and defenses in computer vision: A survey. IEEE Access, 9:155161-155196 2021.
Rubel Bíswas, Víctor González-Castro, Eduardo Fidalgo, and Enrique Alegre. new perceptual hashing method for verification and identity classification of occluded faces. Image and Viston Computing, 113:104245, 2021.
Qi-Zhi Cai, Min Du, Chang Liu, and Dawn Song. Curriculum adversarial training. arXiv preprint 10X.::1805004807, 2018.
Yulong Cao, Chaowei Xiao, Dawei Yang. Jing Fang, Ruigang Yang, Mingyan Liu, and Bo Li. Adver- sarial objects against lidar-based autonomous driving systems. arXiv preprint arXiv: arXiv:1907.000 1907.05418. 2019.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning. pp. 1597-1607. PMLR, 2020.
Adam Coates, Andrew Ng. and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelli- gence and statistics, pp. 215-223. JMLR Workshop and Conference Proceedings, 2011.
Aleksandr Ermolov, Aliaksandr Siarohin, Enver Sangineto, and Nicu Sebe. Whitening for self- supervised representation learning. In International conference on machine learning, pp. 3015- 3024. PMLR, 2021.
Lijie Fan, Sijia Liu, Pin- Yu Chen, Gaoyuan Zhang, and Chuang Gan. When does contrastive learning preserve adversarial robustness from pretraining to finetuning? Advances in neural information processing systems, 34:21480-2144 2021.
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv: 1803.07728, 2018.
Rohit Gupta, Naveed Akhtar, Ajmal Mian, and Mubarak Shah. Contrastive self-supervised learning leads to higher adversarial susceptibility. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 14838-148 14846, 2023.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp- 770-778, 2016.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. arXiv preprint arXiv: 1903. /2261, 2019.
Shahar Hoory, Tzvika Shapira, Asaf Shabtai, and Yuval Elovici. Dynamic adversarial patch for evading object detection models. arXiv preprint mrXiv:2010.13070 2020.
Edwin T Jaynes. Information theory and statistical mechanics. ii Physical review, 108(2):171, 1957.
Xiaojun Jia, Yong Zhang, Baoyuan Wu, Ke Ma, Jue Wang, and Xiaochun Cao. Las-at: adversarial training with learnable attack strategy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2022.
Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning. Advances in neural information processing systems, 33:16199-16210, 2020.
Sara Kaviani, Ki Jin Han, and Insoo Sohn. Adversarial attacks and defenses on ai in medical imaging informatics: survey. Expert Systems with Applications, 198:1 168 15, 2022.
Minseon Kim, Jihoon Tack, and Sung Ju Hwang. Adversarial sulf-supe contrastive learning.
Advances in neural information processing systems, 33:2983-2994, 2020.

---

Published as a conference paper at ICLR 2025
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images, 2009.
Ananya Kumar, Aditi Raghunathan, Robbie Matthew Jones, Tengyu Ma, and Percy Liang. Fine- tuning can distort pretrained features and underperform out-of-distribttin. In International Con- ference on Learning Representations.
Rundong Luo, Yifei Wang, and Yisen Wang. Rethinking the effect of data augmentation in adver- sarial contrastive learning. arXiv preprint arXiv:2303.01289 2023.
Xingjun Ma, Yuhao Niu, Lin Gu, Yisen Wang, Yitian Zhao, James Bailey, and Feng Lu. Under- standing adversarial attacks on deep learning based medical image analysis systems. Pattern Recognition, 10:10733,, 2021.
Aleksander Madry. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 100600033, 2017.
Ishan Mísra and Laurens van der Maaten. Self-supervised learning of pretext-invariant representa- tions. In Proceedings of the IEEE/CVF conference on computer vision and patter recognition, pp. 6707-6717, 2020.
Aleksander Mkadry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. stat, 1050(9), 2017.
Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In European conference on computer vision, pp. 69-84. Springer, 2016.
Senthil Purushwalkam and Abhinav Gupta. Demystifying contrastive self-supervised learning: In- variances, augmentations and dataset biases. Advances in Neural Information Processing Systems, 33:3407-3418, 2020.
Chawin Sitawarin, Supriyo Chakraborty, and David Wagner. Sat: Improving adversarial training via curriculum-based loss smoothing. In Proceedings of the 14th ACM Workshop on Artificial Intelligence and Security, pp. 25-36, 2021.
Jiachen Sun, Yulong Cao, Qi Alfred Chen, and Z Morley Mao. Towards robust LLiDAR-basedd per- ception in autonomous driving: General black-box adversarial sensor attack and countermeasures. In 29th USENIX Security Symposium (USENIX Security 20), pp. 877-894, 2020.
C Szegedy. Intriguing properties of neural networks. arXiv preprint arXiv: 1312.6199 2013.
Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part 16. pp. 776-794. Springer, 2020.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. adaptive attacks to adversarial example defenses. Advances in neural information processing systems, 33:1633-
1645, 2020.
James Tu, Mengye Ren, Sivabalan Manivasagam, Ming Liang, Bin Yang, Richard Du, Frank Cheng, and Raquel Urtasun. Physically realizable adversarial examples for lidar object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13716- 13725, 2020.
Fatemeh Vakhshiteh, Raghavendra Ramachandra, and Ahmad Nickabadi. Threat of adversarial attacks on face recognition: A comprehensive survey, arXiv preprint arXiv:2007.11709 2020.
Yajie Wang, Haoran Lv, Xiaohui Kuang, Gang Zhao, Yu-an Tan, Quanxin Zhang, and Jingjing Hu. Towards physical-world adversarial patch for blinding object detection models. Information Sciences, 556:459 4711, 2021a.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International conference on learning representations, 2019.
12

---

Published as a conference paper at ICLR 2025
Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. arXiv preprint arxiv:2112.08304, 2021b.
Xingxing Wei, Siyuan Liang, Ning Chen, and Xiaochun Cao. Transferable adversarial attacks for image and video object detection. arXiv preprint 1rXiv:1811.12641, 2018.
Ronald J Williams. Simple statistical gradient- following algorithms for connectionist reinforcement learning. Machine learning, 8:229-250 222,, 1992.
Xilie Xu, Jingfeng Zhang, Feng Liu, Masashi Sugiyama, and Mohan S Kankanhalli. Enhancing adversarial contrastive learning via adversarial invariant regularization. Advances Neural In- formation Processing Systems, 36, 2024.
Chengyuan Yao, Pavol Bielik, Petar Tsankov, and Martin Vechev. Automated discovery of adaptive attacks on adversarial defenses. Advances in Neural Information Processing Systems, 34:26858- 26870, 2021.
Chaoning Zhang, Kang Zhang, Chenshuang Zhang, Axi Niu, Jiu Feng, Chang D Yoo, and So Kweon. Decoupled adversarial contrastive learning for self-supervised adversarial robustness. In European Conference on Computer Vision, pp. 725-742. Springer, 2022.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pp. 7472-7482. PMLR, 2019.
13

---

Published as a conference paper at ICLR 2025
A
APPENDIX
A.1
INTERACTION BETWEEN STRATEGY AND TARGET NETWORKS IN ASTRA
Iteration ()) sampled from distribution-1
strategy
opsiion (o) sampled from distribution-2
X
fx(
step-size (9) sampted distribution-3
Perturbation perameters (1, s)
backprop
target
CLEAN
LIXXD + LAV
X
f w (x)
3
backprop
REINFORCE
Reward
strategy
LADV LCLEAN
Figure 8: Strategy and Target Network interaction during ASTrA pretraining phase.
Figure x iillustrates the ASTrA framework involving an adaptive attack strategy network interacting with target network in an adversarial training setup. The framework focuses on dynamically craft- ing adversarial examples and aligning the clean and adversarial representations for robust learning, explained in the following steps.
1.
The strategy network generates probability distributions for each perturbation parameter from which iteration (t), epsilon (e), and step size (s) are sampled. The perturbation (at- tack) parameters are sampled according to conditional probability distribution, given by p(a[z, 0).
2. These perturbed parameters (i, e, s) are used craft adversarial examples from the input images for target network. The target network feed-forward both clean and adversarially perturbed images and computes three loss terms (a) contrastive loss on clean image views (Lossst CleAn), (b) contrastive loss on perturbed views (LossNT ADV), and (c) contrastive loss on clean to perturbed views (Lossnt.MIXO)
and pack-propagated target network.
3. The clean loss (a) and adversarial loss (b) terms are used compute reward for strategy net- work. This computed reward is fed to REINFORCE (Williams (1992) algorithm to com- pute gradients required update the strategy network. The sampling process p(a (x; B) is not differentiable with respect to 6, making traditional gradient- based optimization meth- ods inapplicable. The REINFORCE algorithm is specifically designed handle such situ- ations by using the log-derivative trick, enabling gradient-based optimization even when sampling is involved. The objective of the strategy network is to maximize the expected reward (0), which is defined as:
(()) = EEwD [Earppialat (a:0) [Reusssg (Ir, a; 0)]]
(
To compute the gradient of J(0) with respect to A, the property of probability distributions:
Vop(ale; 0) p(a|2; 0) log
14

---

Published as a conference paper at ICLR 2025
The gradient of ( is expanded
(0) Eew VoBa=p(a(z [Ratratgy (z, o; 0)]
(10)
Using the log-derivative trick, the inner gradient is expressed as:
VoBa-p(olz;0) [Rurategy] = (amp(a)z;0) (Ratattgy Vo Polgp(aa
(11)
Substituting back, the gradient of J(0) becomes:
VaJ(0) Bz~D [Ba-p(a|z;0) [aarategyyz V logp(a/2;; [)]]]
(12)
The reward Ratateg scales the gradient update, encouraging the strategy network to favor attack parameters that yield higher rewards. This ensures that the strategy network learns attack strategies that balance adversarial robustness and generalization.
A.1.1 EFFECT OF STRATEGY NETWORK ON TRAINING STABILITY
The incorporation
strategy network makes the training process adaptive and stable by dynami- cally adjusting attack parameters at the instance level, leveraging the learning dynamics of the target network at each step through observations of standard and adversarial loss terms. This stability evident based the smoothness of the loss landscape visualization in Figure 11. Additionally, at- tack parameter bin configurations identified empirically for CIFAR-10 10 are successfully reused for STL-10 and ImageNer100, demonstrating that the model is not sensitive to initial bin configurations and consistently adapts toward convergence.
A.1.2 ROLE OF LELEAN AND LADV
The reward function Rstrategy incorporates both Lady and Letean-
Ratrateyy (z, 0) atL (I,, a; 0) o Letean (z, Wixed).
(13)
Here:
Ladv depends on 0, as adversarial examples Tadv are crafted using attack parameters a, which are influenced by 0.
Leken does not depend 0, as it is computed using clean data and the fixed target network Whixed
Substituting Ratratgy into the gradient:
(0) = Ez-D [Eawp(alz;0) [a (z, a; 0 log 2 0)]
(14)
E E (ampp(la:)) Letean (I, Wixed) Vo al
1.3.3 EFFECT OF LICERAN
While Lekea does not directly depend on 0, it indirectly affects the updates to 0 by scaling the reward Rstrttgy
*
LLarge Lelean reduces Rstratgyy discouraging attack strategies that degrade clean performance.
Small Lelean increases Ratratgyy reinforcing attack strategies that preserve generalization.
This ensures that the strategy network learns attack parameters a that balance robustness (via a T and generalization (via Letean)
1.1.4 FINAL GRADIENT EXPRESSION
The final gradient is:
VoJ(O) EZLD (a=p(aa [a (z, a; 8V 0)]
(15)
E Ea- Lo Wixed) ) V log p(a x;; 0)]
This ensures the updates to 9 produce controlled attack strategies that improve adversarial robustness while maintaining clean accuracy.
15

---

Published as a conference paper at ICLR 2025
A.2 ASTRA ALGORITHM
Below the details of the proposed ASTrA algorithm:
Algorithm ASTrA Algorithm
Require: Training dataset D, target model fw. strategy model se
1:
for each epoch do
2:
for each batch in D do
3:
augmented views of images in B if update interval reached then
4:
5:
Set Suo to eval mode, sp to train mode a f- sp(r)
6:
b Sample attack parameters Generate adversarial examples
7:
Fade < (..
8:
r - ComputeReward fwr T, Todo) REINFORCEUPDATEEG, a, r, x)) end if
9:
10:
11:
Set fw to train mode, s9 to eval mode
12:
4 sx(x)
Select attack parameters
13:
Tadv +- PGD(fw. I, a)
> Generate adversarial examples
14:
t fw (z. 'normal') )
Clean features
15:
Endo + fro (Iod: 'pgd')
> Adversarial features
16:
Ldeaan N'T-Xent(3 =)
Contrastive loss on clean samples & Contrastive loss on adversarial samples
17:
Lado - NT-Xent( Zadv)
Csim + NTT-Xent[[z, Zadv])
D Similarity loss
19:
C + (Celean + Ladv)/ ALsim
Compute gradients of with respect to fw 21: Update fo parameters using computed gradients
end for
23: end for
Algorithm 2 CowputeReward Function
1: function COMPUTEREWARD((u. 2, Fodv)
2: + f2, 'normal')
Clean features Adversarial features
3: Zadu <- fu (Tados 'pgd')
4:
Letean NT-Xent(z)
5: Ladv t NT-Xent(eae NNT-Xee((Zee))) Zadv)
6:
Lsim + NNT-Xent Zodul)
7:
F Wi Lado
Wy Celean
8:
return r
9: end function
Algorithm REINFORCE Update
1: function REINFORCEUPDATTESS. a, r, x)
2: Compute log probability: log π log (a(x))
3:
Compute gradient: (oJ( +- rV log
4:
Update parameters: < 0 + (
5: end function
16

---

Published as a conference paper at ICLR 2025
Algorithm 4 PGD Attack
1: Initialize 8 ~ Uniform(-ce
2: for k to K do
3: Compute gradient: k Vohad (sr + +))
4: Update perturbation: 5 <- ā + a sign(g)
5: Project perturbation: d clip( -, E)
6: Ensure valid pixel range: zado + clip( + 6,0,1)
7: end for
A.3 CLEAN AND ADVERSARIAL DISTRIBUTION SAMPLE ALIGNMENT ROCL VS ASTRA
As shown in Fig. RoCL and ASTrA differ in how they align clean and perturbed distributions using contrastive loss. RoCL creates an augmented view from the original image, then perturbs generate adversarial view. The contrastive loss applied between the original image, clean augmented view, and adversarial view, resulting in a less structured alignment with only one ad- versarial view involved. In contrast, ASTrA generates two augmented views, each paired with its
RoCL contrastive term between clean and adversarial views
ASTrA contrastive term between clean and adversarial views
Loss ane
Lo5s over
original
augmented view
aiversarialview
original
augmerted
perturbation adoptive atteck
adversarftl
mored contrastive
perturbation handcrofted
Loss
Loss
apy
Figure 9: clean and adversarial distribution alignment approach is compared between RoCL Kim et al. (2020) and ASTrA. For clarity, only clean to adversarial contrastive loss is shown.
corresponding adversarial view. The contrastive loss is computed between each clean view and its adversarial counterpart, enforcing more direct and balanced alignment. This "mixed contrastive" objective in ASTrA enhances the alignment between clean and perturbed samples, improving ro- bustness without compromising generalization due to this direct pairing, allowing the model to adapt better to diverse adversarial scenarios and generalize more effectively than RoCL's single perturbation method.
clean data. ASTrA's approach more effective
A.4 EXTENDED ANALYSIS AND ABLATIONS
A.4.1 COMPUTATION ANALYSIS OF ASTRA
The Table highlights the computation analysis of onboarding different strategy networks within the ASTrA Tramework. The results show that adding strategy network introduces slight increase in computation time. For instance, the compute time increases from 20.5 hours for the ACL baseline (which lacks strategy network) to 23.4 hours for ResNet18, which is the largest architecture in this analysis. This represents an additional compute overhead of less than 3 hours. For smaller networks such MobileNetV1
CustomCNN, the increase in compute time is even smaller, around to 1.5 hours. These results indicate that the computational overhead introduced by the strategy network remains minimal and manageable in all cases.
17

---

Published as a conference paper at ICLR 2025
Table 7: Ablation (SLF evaluation) on Strategy network choices. Compute-robustmes trade-off. Training conducted on single H100 GPU. ASTrA tends network agnostic for strategy network and even with smaller architectures outperform ACL (Cingi et al. (020) and choices of networks like ResNet10, EfficientNet-BO, and DenseNet-122 121 achieves SoTA.
Method Strategy Network/Parameter
AA
RA
SA
Compute Time (hrs.)
ACI
37.62
40.02
79.32
20.50
ASTEA
CustomCNN (5-layers))2.5M
45.22
53.12
80.02
21.10
ASTrA
MobileNetV1/422M
45.38
53.35
80.21
21.60
ASTrA
ResNet10/5.1M
45,80
53.63
80.32
21.75
ASTrA
EfffcientNettB0//S.M
45.94
53,86
80.40
21.75
ASTrA
BenseNet-121/7.98M
46.05
53.88
80.48
22.60
ASTrA
ResNet
46.40
54.02
80.54
23.40
Referring Table 7 ASTrA proves to be network-agnostic, achieving consistent performance im- provements across both parametric and con-parametric strategy network choices. Regardless of the architecture, ASTrA outperforms the ACI. baseline (Jiang et al.] (2020)) showcasing its versatil- ity and robustness. Smaller, lightweight architectures such as MobileNetVi and CustomCNN still achieve competitive performance, while larger architectures such as ResNet10, EfficientNet-BD and DenseNet-121 improve state-of-the-an results.
The results also reveal a positive trend where increasing the complexity of the strategy network leads incremental gains in adversarial and robust accuracy metrics. This demonstrates ASTrA's ability to leverage the capacity of various strategy networks effectively, reinforcing its robustness and generalization capabilities while keeping computational overhead minimal.
A.4.2 COMPARISON OF ASTRA++ WITH METHODS HAVING POST-PROCESSING VERSIONS
The DYNACL++ (Lu al. (023) and DYNACL-AIR++ (Xu et al. (2044) methods extend two- stage self-superviied adversarial training by introducing a third post-processing stage to enhance
representation robustness. This additional stage involves generating pseudo-labels using clustering on pretraining embeddings. followed by Linear Probing and Adversarial Full Finetuning (LP-AFF Kumar et al.)). ASTIA++ which is longer pretraining (2000 epochs) version ASTrA focuses solely extended pretraining improve performance without relying pseudo-labels addi- tional training phases. We compare ASTrA+ with DYNACL+ and DYNACL-AIR++ in Table Performance of DYNACL-AIR method is compared with ASTrA and other methods and incorpo- rated in respective tables in updated manuscript.
Table compares ASTrA++ with DYNACL+ and DYNACL-AIR+ CIFAR-10, CIFAR-100, and STL-10 10 under SLF and AFF evaluation protocols. Despite being limited to two-stage training framework, ASTrA++ demonstrates improved performance AFF metrics across all datasets, high- lighting the efficacy of extended pretraining in achieving robust and generalized representations. For SLF evaluation, ASTrA++ achieves performance comparable to the ofate-of-the-art (SoTA), demon- strating its ability to match exceed the robustness of methods that incorporate additional post- processing stages.
A.TTA++ does exhibit some limitations, particularly in SLF results, where the improvements are less effective compared to its gains in AFF metrics. This suggests that while extended pretraining is effective for adversarial robustness, it may not fully address the requirements for improving standard linear evaluation scenarios. Future investigations could explore integrating post-processing stages, such pseudo-label-based adversarial finetuning, further enhance ASTrA++'s performance in SLF settings while retaining its strengths in adversarial robustness.
ASTIA be extended with post-pros post-processing used by DYNACL+ (uo et al. (2023) and DYNACL-AIR++ (Xu et al. (2024)) however may considered improved version of ASTrA which violets the submission policy.
18

---

Published as a conference paper at ICLR 2025
Table 8: SLF and AFF evaluation on CIFARIO, CIFARI00, and STLIO. in DYNACL++ and DYNACL-AIR+ indicates additional post-processing phase which uses clustering following by Pseudo Adversarial Training. ASTrA+ denotes longer pre-training for 2000 epochs.
Dataset
Pre-training
SLF
AFF
AA (%)
SA (%)
AA (%)
SA ( (%)
DynACL++
46.46
79.81
50.31
81.94
CIFAR-10
DynACI-AIR++
46.99
81.80
50.65
82.36
ASTrA++
46.92
80.46
50.84
83.72
DynACL++
20.07
52.26
25.21
57.30
CIFAR-100
DynACL-AIR++
20.61
53.93
25.48
57.57
ASTrA+
21.95
53.58
26.45
60.25
DynACL++
47.21
70.93
41.84
72.36
STL-10
DynACL-AIR++
47.90
71.44
44.09
72.42
ASTIA+
48.21
78.72
50.15
79.70
A.4.3 LEARNING RATE OF STRATEGY NETWORK
Table shows the critical role of the learning rate in optimizing ASTrA's strategy network during the pretraining stage, directly influencing the effectiveness of the adaptive attack strategy. The perfor- mance across CIFARIO and STL10 reveals that low learning rates (0.001, 0.01) hinder the strategy network's ' ability to explore and exploit optimal adversarial attacks, leading to under-performance in both AA and RA.
Table 9: Effect of learning rates for the Strategy network of ASTrA. Adversarial full finetuning is performed.
Learning Rate
CIFAR10
STL10
AA
RA
SA
AA
RA
SA
0.001
47.65
49.86
79.50
46.30
49.80
77.62
0.01
49.36
52.54
81.20
48.15
51.28
79.30
0.1
50.84
54.90
82.68
49.65
52.40
80.20
0.5
50.20
54.10
82.10
49.10
51.95
79.85
The sharp performance gains learning rate of 0.1 indicate that this value strikes the ideal bal- ance between exploration and exploitation within the adversarial space, allowing ASTrA's strategy network to generate more adaptive and potent perturbations. At 0.5, the marginal decline in per- formance suggests that too high a learning rate disrupts the stability of adaptive attacks, potentially leading overly aggressive perturbations that compromise alignment with the clean distribution. This analysis highlights the role of learning rate in achieving optimal adaptive attack strategies for ASTrA's towards a balanced optimization of adversarial robustness and generalization.
A.4.4 PROJECTOR HEAD OF TARGET NETWORK
The projector head ablation results (refer Fig. 10 provide valuable insights into the impact of architectural choices for the target network during ASTrA's pretraining. The comparison between a smaller projection head (512, 512) and Bottleneck configuration (2048, 512) across CIFAR 10, CIFAR 100, and ST demonstrates that the Bottleneck consistently outperforms the smaller head across AA, RA, and SA metries.
The performance improvement of the Bottleneck can be attributed to its larger latent dimensionality. which enables richer representation learning during adversarial supervueeeeeeed pretraining, lead- ing to enhanced adversarial robustness and generalization clean data. These results emphasize the importance of projection head capacity in optimizing adversarial and standard accuracy during pretraining.
19

---

Published as a conference paper at ICLR 2025
CIFAR-
CIFAR-10 RA
CIFAR-10
Bottlenack
(00)
(56)
2013
50
SA.30
07:05
24:02
10:21
Wi-3
2000
113
CIFAR-100
CIFAR-100
00:03
30.22
59.92
50.00
20.0
23.00
2000
AA (%)
(%)
00:25
Figure 10: Bottleneck projector head (2014,512) shows improvement over ACL's project head (512, 512).
A.4.5 BATCH UPDATES FREQUENCY OF STRATEGY NETWORK
Table illustraae the impact of the frequency of strategy network updates relative to target network updates on performance metrics. The frequency parameter controls how often the strategy network is updated relative to the target network. Specifically. the strategy network is updated every time the target network has been updated a certain number of times, as dictated by the frequency value. From the table, it is clear that frequency of 10 leads the best performance across all metrics
Table 10: Effect of batch update frequency for the Strategy network of ASTrA. Adversarial full finetuning performed.
Frequency
CIFARIO
STL10
AA
RA
SA
AA
RA
SA
47.70
50.10
79.90
46.80
48.25
78.44
50.34
54.58
82.12
49.25
52.02
79.86
10
50.84
54.90
82.68
49.65
52.40
80.20
20
50.50
54.65
82.40
49.50
52.05
80.00
50
50.45
54.62
82.35
49.45
52.22
80.05
(AA, RA, SA), indicating that this update interval strikes the right balance between exploration and exploitation in the strategy network. This frequency allows the strategy model maintain balance between exploring new adversarial strategies and stabilizing around effective attacks without reacting too quickly small changes in the target network. Overly frequent updates may cause the strategy network to overfit to short-term changes in the target network, leading to less effective adversarial attacks overall. The adversarial examples generated by the network are not challenging enough for the target network to generalise better. The frequent changing of the attack strategies, due to frequent strategy network updates, provides less chance for the target network to adapt to the attacks, leading drop in performance. Conversely, updating the strategy network too frequently (e.g., at frequency of 20 or 50) leads to slight drop in performance, particularly in AA and RA. Although these settings still perform close to the optimal, the slight decrease suggests that less frequent updates may hinder the strategy network's 's ability to adapt rapidly enough to evolving target network behavior.
20

---

Published as a conference paper at ICLR 2025
A.4.6 DISCRETIZATION OF THE ATTACK PARAMETERS
To find the suitable granularity of attack parameter bins on downstream performance, we earlier conducted experiments by varying the discretization levels of perturbation c, PGD iterations, and step size. The ablations are in Tables
12 and with following analysis.
Table 1I: Ablation (SLF evaluation) on discretization of the attack parameter perturbation.
Approach
Bins (Perturbation)
AA
RA
SA
small-bins
[3, 11, 15 3, 5. 7.9. 11, 11, 3, 3,4,5,6,74 7. 13, 13,
38.10
41.00
79.30
small-bins
40.18
42.12
79.62
original
13
46.40
54.02
80.54
large-bins
3,3.5, 4,4...., 14, 14,1,,
44.88
52.05
79.38
large-bins + 2k epochs
4,4,,,,,,,,,,,
46.34
54.00
80.36
Table 12: Ablation (SLF evaluation) discretization of the attack parameter step size.
Approach
Bins (Step-size) 1,3.5]
AA
RA
SA
small-bins
37.90
40.22
79.55
original
1,2,3,4,5,6
46.40
54.02
80.54
large-bins
1.5, 2, 2.5, 5.5,, 6
5.5,
45.02
53.06
79.12
large-bins + 2k epochs
[1, 1.5, 2,, 46.37
53.92
80.20
Effeet of Coarser Diseretization (Smaller Bins) Using coarser bins for attack parameters simpli- fies the action space for the strategy network but limits its ability to find the most effective attack strengths. As shown in the tables, when we use smaller bins (e.g.. [3. 7, 11, 15] for perturbation c, re- fer Table 11p. there is noticeable decrease in adversarial accuracy (AA) and robust accuracy (RA). Specifically. ly,, AA drops from 46.40% (original bins) to 38. 10% with coarser bins for c, though better than baseline ACL Jiang al. (2020). This confirms that limited set of attack parameter choices hampers the strategy network's capacity to adaptively challenge the model, leading to sub-optimal robustness.
Table 13: Ablation (SLF evaluation) on discretization of the attack parameter PGD iterations.
Approach small-bins
Bins (PGD iterations)
AA
RA
SA
3, 7, 11, 14 3. 5. 7, 11, 13
38.20
40.80
79.22
small-bins
39.80
41.10
79.80
original
8,4,5,6,7,88 8...., 13, 14]
46.40
54.02
80.54
Effect of Finer Discretization (Larger Bins) Introducing finer bins increases the granularity of attack parameter choices, potentially allowing the strategy network to find more optimal strategies. However, as observed, the performance gains with larger bins are marginal compared to the original settings. For instance, with finer bins for step size (Table AA improves slightly to 45.02%, but does not surpass the original setting. Moreover, the computational complexity increases due the expanded action space, which may require longer training to converge. Notably, when we extend the pretraining to 2000 epochs with larger bins, the model attains results comparable the original settings (e.g.. AA of 46.34% vs. 46.40% for perturbation in Table II. indicating that longer training can compensate for the increased complexity.
Empirical Findings and Transferability Through these experiments, we found that the original bin settings offer a good balance between performance and efficiency. The optimal bin settings for all three attack parameters were determined empirically CIFARI0 and successfully applied to other datasets without significant performance degradation. This suggests that the optimal parameters are transferable and not highly sensitive to dataset-specific characteristics, enhancing the practicality of our method across different domains.
Limitations on Bin Approach While the empirically found parameters demonstrate transferability, ASTrA currently lacks a proven foundation for selecting optimal bin ranges. Dynamically adapting the bins for attack parameters during training based on learning dynamics of target models is one
21

---

Published as a conference paper at ICLR 2025
of the possibility. Incorporating adaptive binning strategies for parameter selection could further improve performance and efficiency. As a plug-and-play framework, ASTrA can be extended in future work to include these capabilities, potentially enhancing its adaptability and robustness.
A.4.7 EXPLANATION ON EXPLORATION-EXPLOITTTII VISUALIZATION
The transition from exploration to exploitation in ASTrA caused by the Strategy Network's adap- tive learning process updates its policy to maximize expected rewards using the REINFORCE algorithm. There is no explicit endpoint for exploration; instead, as the network learns which at- tack strategies are most effective, it gradually increases the probability of selecting those strategies, naturally shifting toward exploitation.
As the Strategy Network continuously adapts to gradients from the Target Network. there is no definitive boundary where exploration ends, and exploitation begins. The delineation is purely il- lustrative, serving to emphasize the conceptual transition. Post this point, the model's selection behavior becomes increasingly exploitative, predominantly favoring specific parameter values opti- mized for Target Model training. This concept is further visualized in Figure[T]] where the parameter values are visualized discretely across training epochs. The figure clearly demonstrates significant increase in the proportion of samples selecting optimal parameter values as the model progresses, highlighting the transition to the exploitation phase.
Epsilon Distribution Across Epochs (100-Epoch Binned, Up to 1000 Epochs)
0.8
Perontngg
0.6
0.2
6.0g
Figure 11: Exploration-EExpoiiation alternate visualization for Fig 6.
A.5 LOSS LANDSCAPE ANALYSIS
To further support the effectiveness of ASTrA's adaptive strategy, the loss landscape comparison, presented in Appendix (Figure 12]. demonstrates that ASTrA achieves a flatter loss landscape com- pared to ACL. This flatter landscape indicates better generalization and robustness, showing how ASTrA not only improves adversarial accuracy but also maintains high performance on clean data. This adaptability and dynamic optimization make ASTrA a more effective solution than static or ran- domly exploring strategies, demonstrating its ability to find optimal attack policies without relying on brute-force methods.
A.6 ASTRA AND ADVERSARIAL CURRICULUM LEARNING
ASTrA shares a fundamental similarity with adversarial curriculum learning (A-CL) approaches such SAT (Sitawarin al. (202T) and CAT (Cai et al. (2018)) in that it seeks to improve adver- sarial training by dynamically adjusting the strength of adversarial examples during training. Both ASTrA and A-CL approaches aim to enhance model robustness while maintaining generalization by optimizing the process of adversarial example generation over time. The motivation behind AS- TrA partially overlaps with A-CL. While A-CL methods focus on incrementally increasing attack difficulty (weaker attacks to stronger attacks) to prevent catastrophic forgetting and achieve smooth transitions, ASTrA is motivated by creating an autonomous and adaptive attack framework to estab- lish instance level attack parameters based learning dynamics of network itself.
22

---

Published as a conference paper at ICLR 2025
Landscape:
100
+000
+359
LOWNTION
Figure 12: ASTrA vs ACL Loss landscape comparison.
How ASTrA differs from adversarial curriculum approaches? ASTrA's reliance on adaptive, autonomous parameter optimization and its self-supervised foundation mark significant departure from curriculum-based methods. Following are some important observations.
Attack mechanism: A- A-CL approaches rely on a predefined or gradually increasing curricu- lum for adversarial attack strength, which is often heuristic-based whereas ASTrÀ employs a strategy network guided by reinforcement learning to autonomously adjust attack param- eters (e.g., iteration, epsilon, step size). This eliminates the need for predefined rules heuristics, making ASTrA more adaptable to diverse datasets and training dynamics.
Optimization: ASTrA introduces reward baed optimization that evaluates the balance between adversarial loss and clean loss, enabling the strategy network to align clean and adversarial distributions effectively. A-CL methods do not typically incorporate such ex- plicit reward-based optimization for attack strategies.
Learning approach: ASTrA designed for a self-supervised setting, more specifically self- supervised adversarial training (self-AT), making it suitable for learning robust represen- tations against adversarial attacks utilizing unlabeled data through its contrastive learning framework. Other side, A-CL methods are designed for supervised settings, where label information often guides the curriculum.
A.7 CODE REPRODUCIRILIIT
The source code of ASTrA is made available at pro project//ASTT and results can be reproduced with PyTorch 2.0 on CUDA 12.x version.
github. io/
23


Reviews:
[{"review_id": "ToXwdurEPM", "reviewer": "Reviewer_iFgE", "summary": "The paper proposes a new method ASTrA for self-supervised training of adversarially robust features. Specifically, the paper introduces several novel and important components to boost the performance of adversarial self-supervised training: mixed contrastive loss and adaptive strategy network. With these two new component, the paper improves the robust accuracy by 1-2% on commonly used datasets like CIFAR10. Ablation study verifies the necessity on these components. Further experiments demonstrates the scalability and potential for wide applications when combining with other methods.", "questions": "1. The last term of reward (eq (4)) does not contain the optimized parameter $\\theta$, why is it necessary to add it to the reward function?\n\n2. As the number of parameters of ResNet 18 is already higher than the number of training example times the number of action space, is it possible to use the non-parametric modeling to replace the strategy network?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper studies an important topic, self-supervised adversarial training, and proposes novel components to improve the performance of the self-supervised AT.\n\n2. The empirical experiments are extensive, showing improvement of ASTrA over existing method. The ablation study and analysis are thorough, demonstrating the necessity on each component.", "weaknesses": "1. Lacking of empirical experiments on larger scale to really reflect the effectiveness of self-supervised AT. As CIFAR10, CIFAR100 and STL-10 are all supervised datasets with clear label, supervised adversarial training may be enough to achieve good robust accuracy. Using unlabel data may be a good way to further improve the performance. However, the paper lacks of the experiments in this aspect.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OkHfgYWtXx", "reviewer": "Reviewer_veWg", "summary": "This work attempts to improve the generation of adversarial examples for use in the adversarial self-supervised training (self-AT). Self-AT is made difficult by the fact that ground truth labels are not known. The first contribution of this paper consists of a Strategy Network, which is a module that can be incorporated during constrastive training to adaptively select parameters for adversarial example generation, conditioning on both the input sample and the contrastive model's current parameters. The Strategy Network is designed to balance adversarial contrastive loss, which encourages a high dissimilarity between perturbed representations, and clean contrastive loss, which ensures that the Target Network maintains high performance on the clean data distribution. For training the Target Network, a new Mixed Contrastive Loss term is introduced, which prevents robust overfitting by penalizing excess distance between clean and adversarial perturbations. While these modifications to contrastive training allow for an improved tradeoff between strong adversarial perturbations during training and high clean accuracy, they do introduce difficulties in training due to the non-differentiable nature of adversarial example generation. To remedy this, the REINFORCE algorithm is used to estimate the gradients of the Strategy Network's objective function. The complete method is evaluated by pretraining the ResNet18 architecture using three different datasets (CIFAR10, CIFAR100, and STL10), and then finetuning using one of three approaches (standard linear finetuning, adversarial linear finetuning, and adversarial full finetuning). Both clean and robust accuracy are computed, and compared against several other robust contrastive learning methods. ASTrA is shown to improve both robust and clean accuracy over the other training methods tested. An ablation study shows that the individual components each contribute to this improvement, and it is shown how ASTrA can be used as a plug and play component to improve the performance of the other methods.", "questions": "- Would the methods in this paper (particularly the strategy network) be suitable for standard adversarial training in a supervised, non-contrastive setting? It seems that adaptively choosing attack parameters might be beneficial in adversarial training more generally.\n- I'm not sure I understand what causes the exploration phase to end and the exploitation phase to begin. For example, in Figure 6, why are the dotted lines drawn where they are? The changes don't always seem consistent between the plots. I also found this plot somewhat difficult to understand in general, it wasn't immediately clear to me that the different colors represented different choices of attack parameters.\n- Why is Figure 8 a line graph? It seems like it could be more effectively presented as a table.\n- How do you think your work relates to the body of research studying adversarial curriculum learning (i.e. [1,2])? At a high level, these works seek to improve adversarial training by adjusting the strength of adversarial examples used during the training process. Do you see ASTrA as having similar motivation and design inspiration to these techniques? How does ASTrA differ from these kinds of curriculum approaches?\n- What limitations does your approach have?\n\n[1] Sitawarin, Chawin, Supriyo Chakraborty, and David Wagner. \"Sat: Improving adversarial training via curriculum-based loss smoothing.\" In Proceedings of the 14th ACM Workshop on Artificial Intelligence and Security, pp. 25-36. 2021.\n\n[2] Cai, Qi-Zhi, Min Du, Chang Liu, and Dawn Song. \"Curriculum adversarial training.\" arXiv preprint arXiv:1805.04807 (2018).", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Using reinforcement learning to optimize adversarial attack generation is well motivated in this paper and seems like a powerful technique to improve robust training. I could imagine this technique being beneficial in applications beyond self-supervised contrastive training.\n- The experimental section appears well designed. Several state of the art methods are used as comparison, and four datasets are utilized (CIFAR-10, CIFAR-100, STL-10, and Imagenet-100).\n- The plug and play framework presented here could potentially offer a significant leap in robustness for self-supervised contrastive models.\n- Code is provided for reproduction purposes.", "weaknesses": "- The evaluation section could include more information on the impact of ASTrA on other metrics (like training time or training stability). I would assume that the additional complexity introduced by the strategy network might make the training process more difficult, although I would be interested in hearing if this wasn't the case. I do believe the effects on the training process warrant more explanation in the paper.\n- I think one underexplored aspect of this approach is how the discretization of the attack parameters affects the downstream performance of the model. I imagine there would be some sort of tradeoff between training efficiency and performance as you vary the number of parameter choices that the strategy network can make. Were there any experiments where the discretization of the attack strategy was changed?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "offP2agogf", "reviewer": "Reviewer_w4mR", "summary": "This paper proposes a novel adaptive attack strategy for adversarial contrastive learning that autonomously discovers optimal attack parameters for generating adversarial examples. By leveraging these parameters in adversarial training, the proposed method enhances adversarial robustness while maintaining standard accuracy.", "questions": "1. Could you explain how attack parameters are sampled from the conditional distribution $p(a|x;\\theta)$? Also, could you provide a step-by-step explanation of the interaction between the target and strategy networks? Particularly, it is unclear how $L_{\\text{clean}}$ in the reward objective updates $\\theta$, as it seems that $\\theta$ is not involved in that loss.\n2. Could you compare the computational complexity of the proposed method with existing methods? Additionally, could you analyze the computational costs of crafting instance-specific optimal attack strategies and training the strategy network?\n3. Could you expand the experimental comparisons to include state-of-the-art methods like DynACL++ and DynACL-AIR++ for a more comprehensive evaluation of the proposed method's strengths and limitations? \n    - DynACL-AIR++ is proposed by Xu et al. in Enhancing Adversarial Contrastive Learning via Adversarial Invariant Regularization, NeurIPS 2023.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The proposed method generates effective adversarial examples by dynamically adjusting attack parameters based on the characteristics of the input data and the model's training dynamics. This adaptive approach outperforms existing methods relying on fixed attack strategies.\n2. The proposed method serves as a play-and-play module, allowing seamless integration with existing adversarial contrastive learning methods.", "weaknesses": "1. The paper has several readability issues. For instance, equation (3) uses $\\theta$ as an input for $L_{\\text{clean}}$ while equation (4) switches to using $w^{\\text{fixed}}$. This inconsistency causes confusion in understanding the proposed method. Additionally, Table 1 references an undefined [19], making it difficult to verify results. Overall, the writing lacks clarity.\n2. The paper lacks clear explanations of the strategy network, a key component of the proposed method.\n    - There is insufficient detail on how attack parameters are sampled from the conditional distribution $p(a|x;\\theta)$. Since the effectiveness of the adaptive attack strategy depends on this process, a more thorough explanation of how the strategy network determines these parameters is crucial.\n    - The interaction between the strategy and target networks is unclear, particularly how $\\theta$ is optimized during training. More details are needed on how feedback from the target network and the REINFORCE algorithm update $\\theta$ for learning optimal attack strategies.\n3. The adaptive attack strategy introduces additional computational overhead. However, there is no analysis of its impact on overall training time compared to existing methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)