RAND: Disrupting Adversarial Synchronization in Transformers via Redundancy-Aware Noise Defense
Authors: Lian Duan, Hanzhang Wang, Yuchun Fang
Accepted at: acl25


Reviewer paper summaries: Transformer models are vulnerable to word substitution attacks due to redundant attention heads synchronizing adversarial perturbations. RAND disrupts this synchronization by injecting noise into redundant heads, enhancing robustness without requiring adversarial training or architectural changes. Experiments on BERT, RoBERTa, and LLaMA show RAND effectively mitigates attacks like TextFooler and BERT-Attack with minimal accuracy loss.

The paper proposes a defense method against the word substitution adversarial attack. It observes the “adversarial synchronization”, where small token changes are amplified by redundant attention heads. The proposed RAND addresses this by clustering heads by layers, then injecting controlled noise into non-representative heads, thereby disrupting synchronized shifts. The experiment results on BERT, RoBERTa, and LLaMA show the improved robustness against two word substitution attack. The attack is model-agnostic and does not require adversarial training.  


The paper defines the redundant attention heads: heads exhibiting similar attention aptterns across layers, shift synchronously in response to adversarial perturbations. This phenomenon is called “Adversarial Synchronization”.
Summary of strengths: - This writing is clear, well-structured, and easy to follow.
- This paper identifies an intriguing phenomenon in transformer-based models, namely adversarial synchronization, which enables word substitution attacks to exploit attention redundancy, thereby amplifying their impact.
- This paper proposes a clustering-based defense mechanism that enhances the adversarial robustness of large language models by regulating attention head redundancy.

1. To mitigate the adversarial, the paper defines the redundant attention heads, and injects small noise perturbations intot the redundant attention heads to disrupt their synchronized shifts. The idea is straight forward but efficient.

2.The proposed method is lightweight and training free. It injects minimal overhead and requires no adversarial retraining.
Summary of weaknesses: - The proposed defense mechanism is relatively simple and easy to understand. However, the number of clusters significantly impacts clustering errors. The authors do not provide any analysis regarding the influence of the number of clusters on attack performance. Additionally, it remains uncertain whether the proposed elbow-curve analysis method accurately identifies the most suitable number of clusters.
- There appears to be a lack of experiments involving adaptive attacks. In adaptive attacks, it should be assumed that the attacker is aware of the proposed defense mechanism and incorporates adaptive strategies to counteract it. For example, when generating substitute words, attackers could use large language models deployed with the proposed method as the victim model. The authors should evaluate the defense performance under such conditions.
- As shown in Table I, the proposed method does not achieve the best performance in certain cases. The authors should provide a detailed explanation of these unexpected results.

1.The redundant attention heads, utilizes the difference of attention head ( scores ) before and after adding the trigger words. Meanwhile, it only works on the substitution word attack. If it is other types of attack, the proposed method may encounter an issue. 


2.The paper claims the defense agains BERT, RoBERTa, and LLaMA. However, the main experiments only contain BERT and RoBERTa. There are only ablation study on LLaMA. It is a bit overclaim. The paper should either only claim BERT and RoBERTa, or did more experiments on LLaMA (e.g., more attack baselines and defense baselines).


3.In order to identify the redundant attention heads, the author needs to inject the malicious words. How to define the malicious words? what if the inserted words is not the adversarial words, or can activate the adversarial by accident (original model is good)?


4.The paper only contains two attack baselines. It would be better to involve more attack baselines. 

5.Would the method be robust to the number of clusters? I saw the ablation study in Figure 5. What is the number used for the experiments. When inserting the noise to the attention head, the same noise is added to all redunant attention heads, or each cluster of redundant attention heads use different noise?

6.When computing the cluster, why use Jensen-Shannon distance, what about other distances. Are there any evidence (ablation study) proves that the JS distance is better?
Comments, suggestions and typos: - Since the number of clusters significantly impacts clustering errors, the authors should conduct a thorough analysis of its effect on attack performance. This could involve varying the number of clusters and evaluating its impact on both defense effectiveness and model robustness. Additionally, further validation is needed to determine whether the proposed elbow-curve analysis method accurately identifies the optimal number of clusters. Providing empirical results and theoretical justification for this selection would strengthen the study.
- The authors should assume that attackers are aware of the defense strategy and implement adaptive mechanisms accordingly.
- The authors should provide a detailed analysis of these cases, identifying potential reasons for the variations and unexpected results.

Please answer the above questions
Overall assessment scores: 2

3
Metareview score: 3.5
Metareview: This paper presents an interesting and novel adversarial defence approach. Considering the reviews and associated discussions, the paper on the whole is a meaningful contribution to the community. Please see the suggested revisions that would strengthen the paper's conviction.
Summary of reasons to publish: 1. Effective adversarial defence approach - they beat or match baselines
2. The biggest strength is that the approach proposed is independent of an attack and does not require training. Hence, it is very practical to apply in real-world situations
3. An interesting proposition behind the defence approach: is it the redundancy in attention heads that is exploited by adversarial perturbation.
4. Adaptive adversarial attacks are not a concern for bypassing RAND