Stay Focused is All You Need for Adversarial Robustness

Authors: bingzhi  chen
ruihan  liu
yishu  liu
jiahui  pan
guangming  lu
zheng  zhang
xiaozhao  fang
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-09 11:11:29

Abstract: Due to the inherent vulnerability of neural networks, adversarial attacks present formidable challenges to the robustness and reliability of deep learning models. In contrast to traditional adversarial training (AT) methods that prioritize semantic distillation and purification, our work pioneers a novel discovery attributing the insufficient adversarial robustness of models to the challenges of spatial attention shift and channel activation disarray. To mitigate these issues, we propose a robust spatial-aligned and channel-adapted learning paradigm, which we term the StayFocused, that integrates spatial alignment and channel adaptation to enhance the focus region against adversarial attacks by adaptively recalibrating the spatial attention and channel responses. Specifically, the proposed StayFocused mainly benefits from two flexible mechanisms, i.e.,   Spatial-aligned Hypersphere Constraint (SHC) and Channel-adapted Prompting Calibration (CPC). Specifically, SHC aims to enhance intra-class compactness and inter-class separation between adversarial and natural samples by measuring the angular margins and distribution distance within the hypersphere space. Inspired by the top-$K$ candidate prompts from the clean sample, CPC is designed to dynamically recalibrate channel-wise feature responses by explicitly modeling interdependencies between channels. To comprehensively learn feature representations, the StayFocused framework can be easily extended with additional branches in a multi-head training manner, further enhancing the model's robustness and adaptability. Extensive experiments on multiple benchmark datasets consistently demonstrate the effectiveness and superiority of our StayFocused over state-of-the-art baselines.

Main paper: 
 introduction: 
 In the past decade, Deep Neural Networks (DNNs) have emerged as indispensable tools for addressing complex real-world challenges across diverse domains within the multimedia field. These applications encompass a broad spectrum of scenarios, ranging from biomedical imaging [3, 4] and face recognition [34] to image retrieval [25] . However, due to the inherent susceptibility of neural networks, adversarial attacks and perturbations [31, 47] can expose significant security vulnerabilities in DNN-based models. Consequently, adversarial robustness has become a crucial metric for evaluating the reliability and trustworthiness of these models.To enhance the robustness of deep learning models, early studies have concentrated on refining variations against adversarial attacks, such as integrating additional regularization terms [17] , introducing core set-based training strategies [8] , and adjusting the perturbation size of training data [42] . By employing a large number of adversarial samples as augmented data, adversarial training (AT) approaches [26, 27] have garnered significant attention from the research community. Inevitably, a significant gap exists between the training robustness and test robustness of adversarially trained models [43] . With the advancement of knowledge distillation techniques, adversarial distillation (AD) [16, 32, 46] is dedicated to enhancing the robustness of lightweight networks by distilling valuable insights from adversarially pre-trained models. Despite the progress achieved by pioneering efforts, most mainstream models still suffer from inadequate adversarial robustness.While semantic distillation and purification have been integral in refining the semantic robustness of adversarially trained models, our research is driven by a fresh cognitive viewpoint beyond semantic interpretation. We demonstrate that the challenges of spatial attention shift and channel activation disarray posed by adversarial attacks and perturbations can be regarded as the main culprits responsible for reducing the robustness of models. As illustrated in Figure 1 (b), spatial attention shift refers to the phenomenon where adversarial perturbations cause a displacement in the model's attention towards irrelevant features, significantly impacting its performance and reliability. Intuitively, as shown in Figure 2 (a), channel activation disarray arises when adversarial attacks disrupt the activation patterns of individual channels, which can hinder the model's ability to extract meaningful representations from the data. By identifying these challenges, our work strives to provide a new solution to the challenges posed by adversarial attacks, improving the robustness of deep learning systems.In this paper, we propose a robust spatial-aligned and channeladapted learning paradigm called "StayFocused", which aims to stay focused on discriminative features against adversarial attacks by adaptively recalibrating the spatial attention and channel responses. Specifically, the proposed StayFocused incorporates two flexible mechanisms, i.e., Spatial-aligned Hypersphere Constraint (SHC) and Channel-adapted Prompting Calibration (CPC). On the one hand, the main purpose of SHC is to implicitly perform spatial alignment by facilitating intra-class compactness and inter-class separation between adversarial and natural samples. Based on the angular margin measurement within the hypersphere space, it aims to minimize the hyperspherical distribution distance within the same category while maximizing the distinct margin from different categories. On the other hand, CPC empowers the network to recalibrate channel-wise feature responses by explicitly capturing the interdependencies among different channels. Benefiting from the top-𝐾 candidate class prompts, it dynamically adjusts the channel magnitudes to prioritize important features while suppressing noise and irrelevant features, leading to more accurate representations against adversarial attacks. By incorporating diverse adversarial objectives related to masking ratio, a flexible multi-head training strategy is also proposed to learn more comprehensive feature representations. Our main contributions are summarized as follows:• Our work pioneers a novel perspective by identifying the challenges of spatial attention shift and channel activation disarray as critical factors contributing to the insufficient adversarial robustness of deep learning models.  
related works 2.1 adversarial attack: 
 In the research community, adversarial attacks can be broadly classified into white-box and black-box attacks. It is noted that our work is dedicated to defending against a variety of white-box attacks [2, 11, 26, 31] while maintaining discriminative capability on clean samples. Sezgedy et al. [31] introduced the concept of adversarial samples, which involves adding imperceptible noise to natural samples to cause misclassification by DNNs [33] . After that, numerous influential techniques for white-box attacks have emerged to combat adversarial examples. For example, Goodfellow et al. [11] introduced the Fast Gradient Sign Method (FGSM) that utilizes gradient information to identify the most aggressive perturbation within a specified range. Inspired by FGSM, Madry et al. [26] proposed a multi-step perturbation strategy called projected gradient descent (PGD) to generate stronger adversarial samples. Different from the previous gradient search perturbation, Carlini Wagner et al. [2] proposed an optimization-based attack method as C&W, which is widely used to evaluate the robustness of deep learning models. Furthermore, Croce et al. [7] explored an extension of PGD attacks and integrated them with existing attacks, namely AutoAttack, to evaluate the robustness of adversarially trained model.Adversarial Training. Among the various existing defense strategies, adversarial training (AT) [19, 26, 35, 45] is widely acknowledged and employed to enhance model robustness against adversarial attacks and perturbations. Given the classification task based on the batch training set X = {(𝑥 1 , 𝑦 1 ), ..., (𝑥 𝑛 , 𝑦 𝑛 )}, where 𝑛 denotes the batch size, each sample 𝑥 with the ground-truth label 𝑦 is drawn from the data distribution D. Theoretically, the function of AT can be defined as a min-max optimization problem [9, 10, 36, 49] ,EQUATIONwhere F represents a DNN-based model with weight parameters 𝜃 , 𝒙 ′ is the adversarial example within the 𝐿 𝑝 -norm ball B 𝜖 (𝒙) = {𝒙 ′ : ∥𝒙 ′ -𝒙 ∥ 𝑝 ≤ 𝜖} centered at 𝑥 and 𝑦 is its corresponding label. In the context of a classification task, L refers to the loss function, e.g., the cross-entropy loss. Here, the inner maximization problem depends on adversarial examples 𝒙 ′ generated within the 𝜖-ball. In contrast, the outer minimization problem optimizes model parameters under worst-case perturbations based on the inner maximization process.Adversarial Distillation. The goal of knowledge distillation is to transfer the knowledge learned by the large robust model (teacher) to the lightweight target model (student), enabling the student model to achieve similar performance with reduced computational resources [12] . Given a well-trained fixed teacher network T with higher capacity, previous AD works [6, 10, 18, 20, 50] have attempted to incorporate knowledge distillation with AT to enhance the adversarial robustness of the trainable student networks S, which can be formulated as the following optimization:EQUATION)where 𝛼 is the trade-off factor and 𝜏 is a temperature constant, L CE represents the cross-entropy (CE) loss that encourages the student S to maximize the natural accuracy, and L KL denotes the Kullback-Leibler (KL) divergence that aims to minimize the distribution difference across teacher-student domains.Advanced Adversarial Robustness. Recently, variants of advanced defense strategies have been proposed to enhance adversarial robustness. For instance, the channel-wise activation suppressing (CAS) strategy [1] suppressed redundant activations caused by adversarial perturbations. To achieve a better trade-off, Zhang et al. [45] decomposed the adversarial prediction error into the natural error and boundary error, proposing TRADES to simultaneously control both terms. Similarly, the Channel-wise Importance-based Feature Selection (CIFS) [41] generated non-negative multipliers for channels to manipulate channel activations for specific layers. In contrast, MART [35] additionally considered misclassified examples during adversarial training. Kim et al. [21] proposed a recalibration strategy called Feature Separation and Recalibration (FSR) to recapture its potential discriminative clues. Furthermore, Yin et al. [43] proposed an effective method named AGAIN to obtain the attribution span of the model under real and random labels, aiming to enlarge the learned attribution span. 
methodology: 
 Technically, the proposed StayFocused is driven by a robustness paradigm shift, specifically focusing on the challenges of spatial attention shift and channel activation disarray posed by adversarial attacks. Figure 3 provides a detailed pipeline of our proposed StayFocused framework, comprising three essential modules: 1) Multi-branches Feature Embedding, 2) Spatial-aligned Hypersphere Constraint, and 3) Channel-adapted Prompting Calibration.To proficiently capture distributed feature representations from input images, a well-established multi-branch architecture is designed as the backbone, comprising adversarial and clean branches. Each encoder is initialized with a pretrained ResNet-18 model. In the clean branches, StayFocused incorporates the concept of maskdenoising [13, 39, 40] to explore the semantic-relevant local image. By robustly capturing localized patterns from randomly masked patches of images, we aim to capture the semantic relevance between different local images through two branches that share parameters, which can be formulated as follows:EQUATIONwhere 𝜙 represents the network parameters of the clean encoder.Inspired by contrastive learning [5, 14, 29] , our SHC mechanism aims to implicitly perform spatial alignment by enhancing intraclass compactness and inter-class separation between adversarial and natural samples. By measuring the angular margins and distribution distance, it is engineered to anchor robust features within the distribution of natural features, while dispersing features from different classes across the hypersphere.Typical Contrastive Learning. Contrastive learning serves as a self-supervised learning paradigm, where the primary objective is to drive the model to draw similar data points nearer within the embedding space, while simultaneously pushing apart dissimilar data points. As mentioned in [5] , each instance 𝑥 contains a set of positive views and a set of negative views [5] . In particular, the where sim(𝑥 𝑖 , 𝑥 𝑗 ) denotes a similarity metric between samples 𝑥 𝑖 and 𝑥 𝑗 , N (𝑖) represents the set of negative embeddings.Definition of Angular Margin. For the binary classification task, let Δ(, ) represent the angle between each pair of feature embeddings. Suppose the learned adversarial feature 𝑥 ′ is given, where Δ(𝑥 ′ , 𝑥 𝑎 ) and Δ(𝑥 ′ , 𝑥 𝑏 ) denote the angles between the adversarial sample and natural samples from different ground-truth categories,EQUATIONTo classify 𝑥 ′ in the spherical space, it is necessary to ensure that Δ(𝑥 ′ , 𝑥 𝑎 ) is greater than the angles of the other class,EQUATIONAs shown in Figure 4 , the decision boundary can be formulated as:EQUATIONwhere 𝑚 ≥ 1 is an integer coefficient. According to Eq. ( 5) and Eq. ( 7), the hyper-spherical angular margin M [23] between classes 𝑎 and 𝑏 can be calculated as:EQUATIONHypersphere Contrastive Learning. Based on the aforementioned analysis, the essence of the angular margin lies in constraining the arc length on the unit circle, which in turn amplifies the discriminative power of the features learned on the hypersphere. Geometrically, we leverage the concept that the dot product of two vectors can yield an angle, thereby transforming the optimization of the feature vector into angle optimization on the unit sphere. Formally, the multiplication of the adversarial and nature vectors can be expressed as:EQUATIONIn this way, we can transform the comparison of traditional feature similarity matrices into an optimization problem on a hypersphere with an angular margin M. Compared with standard contrastive learning, the objective function of hypersphere contrastive learning is formulated as follows:EQUATION)To combat challenges arising from channel activation disarray, the CPC mechanism is meticulously designed to suppress redundant channels while maintaining the activation of relevant channels.Benefiting from the top-𝐾 candidate class prompts, this strategic modulation enables the model to recalibrate its visual attention towards crucial channels, enhancing its resilience and robustness against adversarial attacks and perturbations. Top-K Class Prompts. In parallel with advancing the adversarial branch, we introduce a standard training branch to pinpoint the centroids of distinct class clusters by incorporating category prototypes during the training phase. It can provide a more detailed representation and understanding of each class's inherent characteristics and distributions in the feature space. After that, we compute the cosine similarity between its feature representation and the prototypes of all classes. As a result, the categories with top-𝐾 confidence scores are selected as the class prompts. This selection process ensures that the model focuses on the most relevant and informative classes for each sample, enhancing its discriminative capability and robustness against adversarial attacks.Channel-wise Recalibration. In this part, we initially acquire the preliminary channel feature response 𝑧 ′ for each adversarial embedding via a forward inference process. Following the concept of class activation mapping [48] , we leverage the top-𝐾 class prompts to recalibrate the channel-wise feature responses,EQUATIONwhere 𝑘 ∈ {1, ..., 𝐾 }, 𝛼 is a trade-off factor to balance the weights between different classes, and 𝑤 𝑦 𝑙 and 𝑤 𝑘 𝑙 denote the channel weight of the 𝑙-th feature map corresponding to ground-truth class 𝑦 and top-𝐾 candidate classes, respectively. Given that label information and nature samples are unavailable during the testing phase, the class obtaining the highest score in similarity calculation with the prototype is considered as the initial predicted class,EQUATIONwhere 𝑤denotes the channel weights of the 𝑙-th feature map provided by its corresponding class prototype vector. Notably, the internal maximization the optimization objective depends on estimating the gradients of input pixels based on the loss function. During the adversarial sample generation phase, our CPC strategy is maintained in a frozen state to ensure effective gradient computation for various attack algorithms, without discarding any informative channels.Theoretical and Robust Analysis. Based on Eq. ( 11), the objective function for classification during the adversarial training phase can be formulated as follows:EQUATIONwhere 𝜎 represents the softmax activation function, 𝑤 ′ denotes the class weights corresponding to candidate classes. By incorporating the cross-entropy loss, L CPC can be transformed to:EQUATION)where 𝑁 is the number of classification categories. Suppose ℎ = 𝑧 ′ 𝑤 𝑦 -𝑧 ′ 𝑤 ′ [43], Eq. ( 14) can be reformulated as:EQUATIONTheoretically, under ideal classification conditions,EQUATION)Consequently, we can infer the following:EQUATION)According to Eq. ( 17), a reasonable trade-off mechanism with 𝛼, e.g., 𝛼 = 0.5, allows the model to focus on the activated features associated with the predicted class while also considering the activated features of related classes. Therefore, it is evident that a comprehensive fusion strategy used in Eq. ( 11) enhances the model's reliability and robustness against adversarial perturbations.Intuitively, our flexible multi-head training approach serves as a pivotal enhancement to the model's capabilities. By integrating insights from various heads, StayFocused gains a richer understanding of the data distribution, leading to improved generalization and resilience against adversarial attacks. Meanwhile, each head focuses on capturing different aspects of the data, allowing the model to understand and adapt to various complexities and nuances presented by adversarial inputs.Total Objective Function. Based on the above analyses, the training objective of the proposed StayFocused approach is a combination of multiple loss functions from different modules, i.e., Through the joint optimization of these losses, our approach can further improve the robustness and reliability of deep learning models. Our experimental results demonstrate that a straightforward summation of the objective loss, without incorporating weight constraints, yields satisfactory outcomes across various benchmark datasets. The training algorithm of StayFocused is shown in our supplementary materials.EQUATION) 
experiments 4.1 datasets, baselines, and metrics: 
 In our experiments, we evaluate the effectiveness of our proposed StayFocused framework on two benchmark datasets, i.e., CIFAR-10 [22] and SVHN [28] . In particular, a wide range of state-of-theart baselines are introduced, including two AT methods, namely AT [26] and TRADES [45] , and five advanced adversarial robustness methods, including CAS [1] , CIFS [41] , FSR [21] , AGAIN [43] and AGAIN with AWP [38] . To make a fair comparison, we utilize natural/clean accuracy on natural test samples and robust accuracy on adversarial test samples as the primary evaluation criteria.Following the existing studies, we adopt the ResNet-18 [15] and WideResNet- 34-10 [44] as the backbone architecture for the proposed StayFocused method.Training Phase. Following the conventional settings in existing works [21] , we utilize the stochastic gradient descent (SGD) optimizer with momentum 0.9, weight decay 5×10 -4 , an initial learning rate of 0.1 for CIFAR-10 and 0.01 for SVHN, which is divided by 10 at the 75th and 90th epochs. We apply our method to adversarial training PGD-10 (10-step PGD) with a step size of 2/255 and the perturbation 𝜖 in the adversarial attack under 𝐿 ∞ -norm is set to 8/255 for all methods.Evaluation Phase. The robustness of the model is evaluated by measuring the correct accuracy of the model under different adversarial attacks. We choose several adversarial attack methods to attack the trained model, including single-step attack algorithm FGSM [11] , multi-step attack algorithm PGD [26] (PGD-10, PGD-20, PGD-50 and PGD-100), C&W [2] , and AutoAttack (AA) [7] . The maximum perturbation strength of all attack methods under 𝐿 ∞norm is set to 8/255 and the step size is 0.1 × 𝜖. This comprehensive evaluation ensures a robust assessment of the model's resilience against a wide range of adversarial perturbations.To evaluate the effectiveness of our StayFocused in enhancing adversarial robustness, we conduct comprehensive experiments against a variety of state-of-the-art baselines on CIFAR-10 and SVHN datasets.The comparative results are summarized in Table 1 and Table 2 .Evaluation on CIFAR-10 Dataset. We can observe that our proposed StayFocused method clearly outperforms all the comparative baselines on benchmark datasets. Compared to the current best-performing method, i.e., AGAIN-AWP, StayFocused trained by ResNet-18 model on the CIFAR-10 achieves an average improvement of 3.28% (89.80% vs. 86.52%) in clean accuracy and an average improvement of 16.09% (74.94% vs. 58.85%) under the standard 100-step PGD attack. Particularly, when facing the challenging scenario of comprehensive attack method AA, the classification accuracy of StayFocused shows a 15.4% improvement (67.29% vs. 51.89%) and a 9.01% improvement (62.60% vs. 53.59%) in ResNet-18 and WideResNet-34 on CIFAR-10, respectively. Furthermore, the evaluation results on the ResNet-18 model reveal a substantial improvement of 11.05% in test accuracy (72.24% vs. 61.19%) for Stay-Focused compared to the AGAIN-AWP baseline under C&W attack. Therefore, the above experimental results show that our StayFocused method effectively uses prior knowledge from the standard training branch to recalibrate visual attention and perform spatial alignment, and has better robust performance compared with SOTA in the same field.Evaluation on SVHN Dataset. To evaluate the generalizability of StayFocused on large datasets, we conduct extensive experiments on the real-world dataset SVHN. The proposed StayFocused framework surpasses the current state-of-the-art approaches by a large margin. In particular, StayFocused achieves substantial improvements in terms of Top-1 accuracy on clean samples, surpassing the state-of-the-art approaches by 1.97% (93.54% vs. 91.57%) and 1.88% (95.56% vs. 93.68%). Consistently, StayFocused increases PGD-100 attack accuracy by 10.85% on ResNet-18 and 7.42% on WideResNet-34. Furthermore, the AA accuracy increased significantly by 5.74% (58.99% vs. 53.25%) and 3.32% (56.94% vs. 53.62%) on the ResNet-18 and WideResNet-34 models respectively, further confirms the We study an exhaustive parameter analysis of our proposed Stay-Focused method under different parameter configurations. Specifically, we focus on analyzing the effects of two key parameters: the trade-off factor of 𝛼 and the hyperparameter of 𝐾 in Eq. ( 11).Hyperparameter 𝛼. The hyperparameter 𝛼 controls the ratio between true labels and the top-𝐾 candidate prompts. As 𝛼 increases, the model allocates greater attention to feature maps associated with the true label. When 𝛼 = 1, only feature maps relevant to the true label are amplified. Conversely, as 𝛼 decreases, the model prioritizes feature maps of the top-𝐾 candidate prompts, with 𝛼 = 0 exclusively enhancing these maps. We experimented with different 𝛼 values using ResNet-18 on CIFAR-10 to find the optimal 𝛼. Through experimentation and analysis in Figure 5 . When 𝛼 equals 0 or 1, the model exclusively attends to the feature map under top-𝐾 candidate prompts and those under true labels, respectively. Based on experimental results, 𝛼 = 0.5 yields the optimal outcome. Thus, we adopt 𝛼 = 0.5 during experiments to prioritize the feature map associated with true labels while enabling the model to learn additional feature knowledge.Hyperparameter 𝐾. The hyperparameter top-𝐾 dictates the number of candidate prompts considered. When 𝐾 = 1, only real labels are utilized. As 𝐾 increases, more pertinent candidate feature maps are incorporated. We vary 𝐾 and conduct experiments on the CIFAR-10 dataset using ResNet-18 to ascertain the optimal top-𝐾. The experimental results are depicted in Figure 6 . Observing the experimental results, the best performance is achieved when 𝐾=2. However, with further increments in the number of top-𝐾, the model absorbs an excessive number of redundant feature maps, leading to a reduction in discriminative capability.Effect on Each Component. We systematically evaluate the impact of each component on the model's performance. The comparative results are presented in Table 3 . "StayFocused w/o CPC" shows reduced adversarial robustness, emphasizing CPC's critical role in suppressing redundant channels and recalibrating visual attention. Effect on Hypersphere Constraint. We also comprehensively compare the hyper-sphere contrastive learning used in our SHC mechanism. The comparative results are presented in Table 4 . Compared with two popular measurement methods, cosine similarity [37] and InfoNCE [14, 24] , experimental results show that SHC substantially improves clean accuracy and adversarial robustness under various adversarial attacks. The results from the ablation studies above underscore the significance of our SHC mechanism in enhancing both intra-class compactness and inter-class separation.Spatial Alignment. As illustrated in Figure 1 , we adopt the class activation mapping approach to identify the relevant attentional visual regions. In contrast to traditional methods susceptible to adversarial attacks, as demonstrated in Figure 1 our approach consistently directs its attention towards the most relevant regions associated with the ground-truth labels.Channel Recalibration. Meanwhile, Figure 2 (c) visualizes the averaged channel magnitudes derived from StayFocused. This visualization vividly showcases the efficacy of our approach in counteracting channel activation disarray. By recalibrating the channelwise feature responses, our method ensures a more coherent and stable representation of visual information.T-SNE Visualization. To further showcase the effectiveness of StayFocused, we employ the T-SNE technique [30] to represent the distributions of feature representations for both clean and adversarial samples. As depicted in Figure 7 and Figure 8 , the T-SNE [30] plots resulting from our StayFocused method exhibit a substantial increase in intra-class compactness, while revealing a notable improvement in the separation between different classes. 
conclusion: 
 In this paper, we discovered the phenomenon of adversarial perturbation causing spatial attention shift and channel activation disarray and proposed a StayFocused paradigm to recalibrate the spatial attention and channel responses from a new perspective to solve the above problems. The proposed StayFocused method implicitly conducts spatial alignment and leverages top-K candidate class prompts to maintain focus on discriminative features, combined with a multi-head training strategy to enhance resilience against adversarial attacks. Comprehensive experiments prove that our method is sufficiently effective and universal, and can be integrated into existing advanced frameworks. 
 
 Appendix:  


Decision: Overall it is a good paper present an interesting study into adversarial robustness. Some initial concerns of the reviewers were adequately addressed by the rebuttal.

Reviewer PKbZ has a questionable final rating: he/she initially praised the novelty and gave a "weak accept", but then complained about the novelty and changed to "weak reject" without giving a convincing argument. Hence I am discounting this reviewer's rating.

Reviews: review: paper_summary: paper_summary: This paper proposes a novel paradigm called StayFocused for better adversarial robustness, which consists of a Spatial-aligned Hypersphere Constraint (SHC) and Channel-adapted Prompting Calibration (CPC). Extensive experiments show the efficacy of StayFocused to help models gain better inter-class compactness and intra-class separation. StayFocused achieves significant improvement in robustness against many adversarial attacks.
strength_weakness: strength_weakness: 1. The paradigm introduced in the paper is novel and intuitive. The theoretical analysis provided in the paper aligns with the main claim, as well as the experimental results.
2. Experiments are conducted on two datasets and various attacks, showing certain effectiveness of StayFocused. Visualization is also properly employed as an auxiliary for understanding and showcasing the effectiveness of StayFocused.
3. The overall writing of the paper is clear and coherent. The methods and motivations are delineated in a reading-friendly way. Analysis and discussion are also sufficiently thorough and well-covered.
limitations: limitations: 1. Experiments are only conducted on relatively small-scale datasets, larger datasets such as ImageNet (or at least Tiny-ImageNet), are absent for validation.

2. Some attacks including adaptive attacks[1] are not included in evaluations. Black-box and transfer-based attacks should also be considered in the evaluation of adversarial robustness.

3. Would the inclusion of MLP in CPC be regarded as a modification to models and thus be unfair for evaluations of other methods?

4. Line 412  should be "smaller than the angles of other classes". Besides, according to Eq.5-7, should Eq.8 be |\Delta(x',x^b) - \Delta(x',x^a)|? 

5. In Eq. 15, why didn't the two exp(zw') terms get cancelled out? 

6. This paper is that this paper seems to focus explicitly on adversarial robustness in CV, which does not involve any other modality. It would be more suitable if this work provides attempts or justification for expanding to models that involve multi-/cross-modality. could the authors provide the feasibility of expanding their methods to cross-modality?

[1] Tramer, F., Carlini, N., Brendel, W., & Madry, A. (2020). On adaptive attacks to adversarial example defenses. Advances in neural information processing systems, 33, 1633-1645.
paper_summary: paper_summary: This paper proposed a spatial-aligned and channel-adapted learning strategy to improve model robustness with two modules: spatial-aligned hypersphere constraint and channel-adapted prompting calibration. Experiments on two datasets are conducted to demonstrate the performance.
strength_weakness: strength_weakness: 1. The paper is well-motivated and well-written.
2. The experimental results outperform comparison methods by a large margin.
limitations: limitations: 1. More explanations are needed for Section 3.2. How is $\mathcal{M}$ used in the loss of SHC module? How is $\cos(m\cdot\Delta(x’,x))$ computed in Eq. 10 as only $\cos(\Delta(x’,x))$ is available from the dot product.
2. The effect of CPC and SHC is evaluated in the ablation study. What about the ablation study for the multi-head component?
3. The experiments are conducted on two datasets. Are there any results from datasets with larger images?
paper_summary: paper_summary: The paper proposes a novel framework called StayFocused for enhancing the adversarial robustness of deep learning models.    The framework incorporates spatial alignment and channel adaptation to improve the focus region against adversarial attacks.    It introduces two mechanisms, Spatial-aligned Hypersphere Constraint (SHC) and Channel-adapted Prompting Calibration (CPC), to recalibrate spatial attention and channel responses.
strength_weakness: strength_weakness: (1) The paper introduces a novel perspective on the challenges of spatial attention shift and channel activation disarray in adversarial attacks, and proposes a unique solution to address these challenges.
(2) The proposed StayFocused framework incorporates innovative mechanisms, SHC and CPC, to enhance adversarial robustness by recalibrating spatial attention and channel responses.
limitations: limitations: The two mechanisms, Spatial-aligned Hypersphere Constraint (SHC) and Channel-adapted Prompting Calibration (CPC) proposed in this paper are a little lack of novelty,  such as [1-2], they have already done the research of adversarial robustness on the hypersphere space based angular margin. If you could clarify the novelty of the mechanisms, i would change my score.


[1] Pang T, Yang X, Dong Y, et al. Boosting adversarial training with hypersphere embedding[J]. Advances in Neural Information Processing Systems, 2020, 33: 7779-7792.
[2] Fakorede O, Nirala A, Atsague M, et al. Improving adversarial robustness with hypersphere embedding and angular-based regularizations[C]//ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023: 1-5.
paper_summary: paper_summary: This paper raises an interesting finding based on the activation maps to show that the adversarial samples will destroy the current attention regions of the classifier, thus leading to misclassification. Then, the author indicates two root causes: 1) spatial attention shift and 2) channel activation disarray. To solve these problems, the author proposes "StayFocused", an adversarial training framework, to enhance the attention regions of the classifier. The overall experimental results show that it performs better.

However, current experiments cannot show its efficiency due to the weaknesses (detailed in the Sec. Limitation). Meanwhile, there is some confusion about the methodology (also see in Sec. Limitation)). Therefore, we rate it as "borderline reject".
strength_weakness: strength_weakness: 1. This paper proposes an interesting finding that the adversarial samples will destroy the correct attention regions of the classifier, thus leading to misclassification. Meanwhile, this paper gives a detailed analysis to show the root causes for this finding, which is innovative.

2. The overall method performs better than the baselines mentioned in this paper.
limitations: limitations: 1. The experiments are insufficient. 1) The experiments lack the necessary comparison with SOTA methods. According to the records in the RobustBench for adversarial training (AT), the SOTA performance for AT in CIFAR-10 with the backbone of WideResNet 28-10 against autoattack is 92.44% and 67.31% for standard and robust accuracy, respectively [2]. It seems that this paper's performance does not achieve SOTA. 2) The dataset lacks the CIFAR-100 and ImageNet. To prove the efficiency of AT, it is necessary to report the related experiments on CIFAR-100 and the sub-set of ImageNet.

[1] RobustBench: a standardized adversarial robustness benchmark. Croce, Francesco and Andriushchenko, Maksym and Sehwag, Vikash and Debenedetti, Edoardo and Flammarion, Nicolas. arXiv preprint arXiv:2010.09670, 2022.

[2] Better Diffusion Models Further Improve Adversarial Training. Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, Shuicheng Yan. ICML 2023, 2023.

2. There is some confusion in the methodology parts: 

-- How do we make sure Eq. 5 is valid on multi-label classification conditions? As mentioned by the author, they assume that for " binary classification", the angle between two categories could be represented by assuming that the adversarial samples are located in the tangent surface between two categories. This is reasonable, but under multi-label conditions, this assumption is strong. It may be better to give more details to illustrate its validity.  

-- How do we calculate the scale factor $m$ during training? If I have missed anything, please correct me. I have checked all the details, including the appendix, and cannot find the details about $m$. 

-- For $\alpha$ in Eq. 11, the overall theory to prove $\alpha = 0.5$ is little unreasonable. To begin with, according to Eq. 17, let $\alpha =0.5$ cancel it, which seems that it is not necessary to introduce this factor to balance the weights between $w^{y}$ and the top-k candidates in Eq. 11. In this view, the following question is why $w^{y}$ is equally important compared to the top-k candidate? since top-k candidates are not always positive based on the ablation study. Then, what $z_{l}$ means? Based on Figure 3, I guess $z_{l}$ are the feature embeddings from different layers. Eq. 16 will be questioned in this view since this assumption is too strong. As shown in Eq. 11 and Eq. 12, inputs differ during the training and inference phases. The assumption may work in the training phase, but how do we ensure it works in the inference phase? Besides, how can each layer's class weight be calculated during the inference phase?

3. The top-k strategy is a little invalid. The reason to raise this confusion is that the ablation study shows a negative influence when $k > 2$ without further analysis. As the author said, " We leverage the top-k class prompts to recalibrate the channel-wise feature responses." The top-k strategy could be regarded as offering more negative samples to enhance the positive ones. In this condition, more negative samples should increase the overall performance. It may be better to offer more details about this.