Title: Boosting the Transferability of Adversarial Attack on Vision Transformer with Adaptive Token Tuning
Venue: NeurIPS 2024
Accepted: yes
Keywords: Adversarial Attack

Adversarial Transferability

Black-box Attack
Abstract: Vision transformers (ViTs) perform exceptionally well in various computer vision tasks but remain vulnerable to adversarial attacks. Recent studies have shown that the transferability of adversarial examples exists for CNNs, and the same holds true for ViTs. However, existing ViT attacks aggressively regularize the largest token gradients to exact zero within each layer of the surrogate model, overlooking the interactions between layers, which limits their transferability in attacking black-box models. Therefore, in this paper, we focus on boosting the transferability of adversarial attacks on ViTs through adaptive token tuning (ATT). Specifically, we propose three optimization strategies: an adaptive gradient re-scaling strategy to reduce the overall variance of token gradients, a self-paced patch out strategy to enhance the diversity of input tokens, and a hybrid token gradient truncation strategy to weaken the effectiveness of attention mechanism. We demonstrate that scaling correction of gradient changes using gradient variance across different layers can produce highly transferable adversarial examples. In addition, introducing attentional truncation can mitigate the overfitting over complex interactions between tokens in deep ViT layers to further improve the transferability. On the other hand, using feature importance as a guidance to discard a subset of perturbation patches in each iteration, along with combining self-paced learning and progressively more sampled attacks, significantly enhances the transferability over attacks that use all perturbation patches. Extensive experiments conducted on ViTs, undefended CNNs, and defended CNNs validate the superiority of our proposed ATT attack method. On average, our approach improves the attack performance by 10.1% compared to state-of-the-art transfer-based attacks. Notably, we achieve the best attack performance with an average of 58.3% on three defended CNNs. Code is available at https://github.com/MisterRpeng/ATT.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Adaptive Token Tuning (ATT) to improve the transferability of adversarial examples generated from ViTs. ATT is an improvement over traditional gradient-based algorithms, consisting of three independent methods. The first method reduces gradient variance by rescaling the token gradients computed throughout different modules and layers in the ViT. The second method is a scheduled, semantic-guided patch-out approach to increase the diversity of input during the iterative gradient ascent procedure. The third method is a truncation approach for attenuating the model-specific attention occurring at the deeper layers of the ViT. Experimental results on ImageNet show an improvement over existing transfer-based attacks.\n\nSTRENGTHS:\n**Originality:**\n\nThe proposed ATT algorithm includes three strategies that work in combination to improve the transferability of perturbations generated from vision transformers. Each method is developed based on insights from previous work and improves upon them. The proposed method is original.\n\n\n**Quality/Clarity:** \n\nThe presentation is clear, and the methods are well-explained. A thorough evaluation of transferability on a variety of target models, including transformers and both undefended and defended CNNs.\n\n\n**Significance:** \n\nThe proposed work is a valuable addition to methods for generating more transferable perturbations from vision transformers.\n\nWEAKNESSES:\n1. Ln76: Blackbox attack also includes those when target model has limited access, i.e., query-based attacks.\n\n2. The statement in ln 294-295 requires further consideration. The analysis itself does not show that the ATT perturbation captured more features than those generated by other algorithms. While the improvement suggests this, there is no analysis proving the statement.\n\n3. Several statements in the paper requires references, for instance, Ln218-219.\n\nQUESTIONS:\n1. Do we have results showing the benefit of the adaptive strategy in variance reduction in terms of transferability improvement?\n\n2. In (2), is it possible for the scaling factor to be negative? Wouldn't that change the direction of the gradient, resulting in a less optimal argmax of (1), and potentially making the perturbation less transferable?\n\n3. For the Hybrid Token Gradient Truncation Method, many design decisions are made without clear explanation. For example, why do we have two different types of truncations? I appreciate the authors for including the analysis in A.5 on the number of truncated layers. For the experiment in 4.1, how is $\\ell'$ selected for each model? Do we need to run a sweep for each model?\n\n4. For the results in Tables 1 and 2, do all methods, including the baselines and ours, have Patchout?\n\n5. In Ln 271, shouldn't the number of discards be equal between Patchout and SPPO for a fair comparison?\n\nLIMITATIONS:\nNo discussion on limitation of the work. The checklist says the limitation is discussed in Sec 2 4 and Appendix A1, but I could not find any discussion.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

comments: SUMMARY:
The paper proposes an Adaptive Token Tuning (ATT) method to enhance the transferability of adversarial attacks on Vision Transformers (ViTs). The method introduces three optimization strategies: adaptive gradient re-scaling to reduce token gradient variance, a self-paced patch out strategy to enhance input diversity, and a hybrid token gradient truncation strategy to reduce the effectiveness of the attention mechanism. Extensive experiments demonstrate the superiority of ATT over existing methods in terms of attack success rate and transferability across various models.

STRENGTHS:
1. The combination of adaptive gradient re-scaling, self-paced patch out, and hybrid token gradient truncation is novel and well-motivated, providing a fresh perspective on improving adversarial attack transferability for ViTs.
2. The experimental results show that ATT significantly outperforms state-of-the-art methods, achieving higher attack success rates and better transferability.

WEAKNESSES:
1. In the paper's formulation, there are multiple hyperparameters. Do these hyperparameters interact with each other? Why is γ set to 0.5?
2. In Figure 2, is the model used for prediction the same as the target model for the attack? From the figure, it appears that the proposed method indeed makes the model focus on the classification target more quickly. However, how do you explain that as the number of iterations increases, the attention no longer focuses on the classification target? The targets shown in the figure are all dogs, just different types, so the attention should still focus on the dog.
3. Have the effects been tested on defended ViTs?

QUESTIONS:
Refer to Weakness section.

LIMITATIONS:
No.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-11

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nn this paper, the authors investigate three strategies to boot the transferability of adversarial attacks on Vision Transformers, including an adaptive gradient re-scaling strategy, a self-paced patch out strategy, a hybrid token gradient truncation.\n\nSTRENGTHS:\n1 The experiments are solid.\n\n2 The soundness of the method is good.\n\n3 This paper is well written.\n\nWEAKNESSES:\n1 The title of the  paper might be imappropriate to conclude the techniques proposed by authors. The overall methods are composed of three partitions:\n\n- An adaptive gradient re-scaling strategy to reduce the overall variance of token gradients. It is related to the back propagation for crafting adversarial samples. \n\n- A self-paced patch out strategy to enhance the diversity of input tokens. It is related to the adaptive forward propagation for calculating the adversarial loss. \n- A hybrid token gradient truncation strategy to weaken the effectiveness of attention mechanism. It is also related to the back propagation for crafting adversarial perturbations. \n\nWe can see that none of these points are related to the token tuning. (Maybe I am wrong, I hope authors can point out my mistakes.) As I understand it, token tuning usually refers to optimizing the token after tokenizing the input images.  Since the designed attacks craft pertubations directly on input images like most of previous attacks. I think it might be imappropriate to conclude the proposed methods with the phrase \"Adaptive Token Tuning\". \n\n2 In line 72, the authors claim that \"robustly trained defense models still fail to meet security requirements\".  The observation contradicts with the findings observed in [1].  I conjecture this is because the authors choose to attack secure models which is published in 6 years ago. Considering the rapid development in the adversarial community, I recommend to attack recent sucure models trained by $l_{\\infty}$ AT provided by robustbench [2].\n\n3 The novelty of this paper is fair. The design principles of the proposed methods originates from previous papers and in this paper, they simply make them adaptive. Higher flexibility often brings better transferability. However, it also brings additional costs: it will increases the difficulty of hyperparameter tuning. In addition, more new insights are needed to guide community better leverage ViT to attack deep models.\n\n4  The conparison with the baseline methods in Table 1 and Table 2 might be unfair. It seems that the proposed methods acquire better transferability than other attacks. However, considering the proposed attack proposed in this paper combines the power of three attacks: TGR, PNA, PatchOut. I think the appropriate baseline is TGR+PNA+PatchOut. It helps us more precisely pinpoint the gains brought by the proposed method.\n\nQUESTIONS:\nIn Table 2, the results reveal that the proposed method can better attack the robust CNNs. How about robust ViTs, such as DiT-S in [3], Swin-B in [4], Xcit-S in [5]?\n\nLIMITATIONS:\nThere is no single section to discuss the limitation and the broader Impact of the paper. Can you explain them with more details?\n\n[1] https://github.com/Trustworthy-AI-Group/TransferAttacks\n\n[2] https://robustbench.github.io/\n\n[3] Are Transformers More Robust Than CNNs? in NeurIPS 2021.\n\n[4] When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture, in NeurIPS 2022.\n\n[5] A Light Recipe to Train Robust Vision Transformers, in SaTML 2023.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the enhancement of adversarial attack transferability on Vision Transformers (ViTs) through innovative adaptive token tuning techniques. It addresses the vulnerability of ViTs to adversarial attacks by introducing three main optimization strategies: an adaptive gradient re-scaling method to uniformly reduce gradient variance across ViT layers, a self-paced patch out strategy to increase input diversity and mitigate overfitting by dynamically discarding less important perturbation patches, and a hybrid token gradient truncation method designed to attenuate the attention mechanism's effectiveness, adjusting truncation factors across different modules for optimal balance. Extensive experiments demonstrate that these methods not only improve the transferability of adversarial examples across ViTs and CNNs but also significantly enhance attack success rates by an average of 10.1% compared to state-of-the-art transfer-based attacks. Notably, this approach achieves a remarkable average attack performance of 58.3% on defended CNNs, highlighting the ongoing challenges in securing robustly trained defense models against sophisticated adversarial strategies.\n\nSTRENGTHS:\n1. This paper significantly enhances the transferability of adversarial attacks on Vision Transformers (ViTs) by introducing three main optimization strategies: an adaptive gradient re-scaling method, a self-paced patch out strategy, and a hybrid token gradient truncation method. It also provides a comprehensive theoretical analysis of the proposed methods, detailing the mechanisms through which each contributes to improving adversarial robustness and effectiveness.\n\n2. Extensive experimental results substantiate the effectiveness of the proposed methods, demonstrating their efficacy across various settings and models.\n\nWEAKNESSES:\n1. This paper enhances the transferability of adversarial attacks on Vision Transformers (ViTs) through adaptive token tuning, introducing three main optimization strategies: an adaptive gradient re-scaling method, a self-paced patch out strategy, and a hybrid token gradient truncation method. However, it lacks ablation experiments for these three strategies.\n\n2. The authors propose an improvement on the PatchOut method, termed Self-Paced Patch Out, which is an input transformation approach that intuitively could enhance adversarial transferability. However, comparing it in Table 1 alongside methods that do not incorporate input transformation, such as TGR, seems unfair.\n\nQUESTIONS:\nThe authors mention that the mild re-scaling strategy can back-propagate some important feature information in the largest token gradient, which is correlated with important feature information, unlike TGR. However, it is unclear how retaining important feature information contributes to enhanced transferability of adversarial attacks. Could the authors elaborate on the connection between these retained features and the observed improvements in transferability?\n\nLIMITATIONS:\nThe authors suggest that attention modules in certain ViT layers are redundant for image classification and adversarial perturbation generation, potentially leading to overfitting. This assertion raises questions about the underlying mechanisms. Could the authors provide more detailed analysis or empirical evidence on how these redundant attention modules affect the transferability of adversarial attacks?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}