Paper: AutoDAN: Automatic and Interpretable Adversarial Attacks on Large Language Models (ICLR-2024)

Reviewer comment (Experiments):
How is the computational resource required for AutoDAN? And, how is the efficiency of the proposed attack? The backpropagation through the LLMs to select the candidate subset could be computationally heavy.

Author response:
Please see our **General Response - 3. Computational Complexity Analysis** for the complexity of our method in comparison with prior work (GCG), and the actual time cost. We do backpropagation once in each step to get the gradient at the new token's location. Since we only need the gradient for one location (as compared to all weights in model training), the time and space complexity is slightly less than a forward pass due to the shorter path in the computational graph.

Reviewer comment (Reproducibility):
The proposed attack seems to require using the probability of the next token. However, in a black-box setting (e.g., attack GPT-3.5 API), it is difficult (or impossible) to obtain the probabilities. Therefore, the proposed method could be difficult to adapt to the latest LLMs.

Author response:
This is indeed a limitation of our method (and all other gradient-based methods alike, including GCG), as it is impossible to obtain the gradient from black-box LLMs. Nevertheless, AutoDAN-generated interpretable prompts demonstrate transferability, suggesting that different LLMs share a somewhat similar vulnerability landscape. These interpretable prompts thus provide a way for red-teaming researchers to identify and exploit these common vulnerabilities and potentially strengthen the existing black-box attacks using the strategies found by AutoDAN.

Reviewer comment (Evaluation):
The major objective of this paper is to "generate attack prompts that achieve high attack success rates while having perplexity scores lower than most human-written prompts" (Page 2 the summary of contributions). Namely, the proposed method should have high ASR and meanwhile low perplexity. However, the paper cannot convince me that the proposed method can solve (or avoid) the trade-off between ASR and readability. Specifically, the proposed method introduces a loss function to drive the generated jailbreak suffix readable and add a weight parameter to balance the attack strength and readability. However, the experiment shows that the proposed method compromises ASR when achieving better readability (Page 7 Table 1), and a similar trade-off exists in GCG and GCG-reg. So how can the proposed method achieve "high attack success rates" and "low perplexity scores"? From my perspective, answering this question may need more comprehensive evaluations. In a nutshell, I think the current weakness in soundness is that the paper has not shown the ability to address (or ease) the trade-off between attack strength and readability, in other words, the proposed method seems more like "achieving lower perplexity scores by compromising ASR" rather than achieving both.

Author response:
> **Weakness 1.1:** However, the paper cannot convince me that the proposed method can solve (or avoid) the trade-off between ASR and readability.

Thank you for raising this important point. Let's break down the concept of the trade-off between ASR and readability, as it's not a universally acknowledged notion in the existing literature and may vary depending on the target model and definition of ASR.
1. **Existence of the trade-off:** The ASR-readability trade-off may not necessarily exist for some models. For instance, LLMs like GPT3.5, GPT4, and Vicuna are vulnerable to manually designed prompts that are both perfectly readable and universally effective (jailbreakchat.com provides many such examples). Similarly, some prompts generated by our method also exhibit this dual quality and show no sign of such a trade-off.

2. **Defining ASR:** The notion of the trade-off depends on how we define ASR. If we consider training ASR, then prioritizing readability might constrain our choice of prompts, suggesting a potential trade-off. However, when considering test ASR (same model, unforeseen behaviors) or transfer ASR (different models, unforeseen behaviors), readable prompts often show better generalization and transferability than their unreadable counterparts, as evidenced in our Table 1 (generalization) and Table 2 (transferability). This suggests that the trade-off may not exist in the latter case. The test ASR and transfer ASR matter because attackers can copy and paste any pretrained prompts to jailbreak LLMs without warming up their own GPUs, thus posing a threat to the safety of LLM applications.

3. **Defining target model:** The notion of the trade-off also depends on how we define the target model. If we consider the safety alignment during RLHF as part of the target model, then a perplexity filter attached to the model may also be considered as part of it. If this is the case, then the trade-off may not exist since unreadable prompts cannot bypass the filter to jailbreak the model.

In summary, while the trade-off between ASR and readability is an interesting concept, its existence and significance may vary based on how we define ASR and the target LLM, and our method is not necessarily subject to such trade-off.

> **Weakness 1.2:** However, the experiment shows that the proposed method compromises ASR when achieving better readability (Page 7 Table 1), and a similar trade-off exists in GCG and GCG-reg. So how can the proposed method achieve "high attack success rates" and "low perplexity scores"?

By "high attack success rates," we mean AutoDAN can achieve an average of 88% ASR on both training and test behaviors and actually 100% ASR if we allow choosing the best prompts out of several independent runs (Figure 1). By "low perplexity scores," we mean it achieves a median perplexity of 12, which is lower than the median perplexity of human-written prompts in ShareGPT. GCG and GCG-reg cannot achieve both high ASR and readability in that even the GCG-reg can only achieve a median perplexity of 1143, and we can see from Table 10 that their generated prompt samples are not readable compared to AutoDAN's.

> **Weakness 1.3:** In a nutshell, I think the current weakness in soundness is that the paper has not shown the ability to address (or ease) the trade-off between attack strength and readability, in other words, the proposed method seems more like "achieving lower perplexity scores by compromising ASR" rather than achieve the both.  

In summary, we would like to argue that the "trade-off" between readability and ASR is not well-defined in the literature, as it may depend on whether to consider the generalization and transferability in defining the ASR, and also the target model setting. Furthermore, even if we consider only the training ASR and exclude defending mechanisms of the target LLMs, our method still achieves "high" ASR and "low" perplexity, and better Pareto optimal than GCG and GCG-reg (Figure 1, note the log-scale x-axis).

Reviewer comment (Experiments):
Lack of evaluations on more robust LLMs. In this paper, the proposed method is evaluated on Vicuna-7B and 13B, Guanaco-7B, and Pythia-12B. Existing works such as GCG have shown that these models have more severe vulnerability of jailbreaks (e.g., 90+ ASR) compared with LLMs that have better alignment, for example, the widely known open-sourced model Llama2. While effectiveness on models like Vicuna-7B is not meaningless, given the trade-off between attack strength and readability, it becomes necessary to conduct evaluations on more robust (aligned) LLMs to show the ability to address this trade-off. Otherwise, as shown on (Page 7 Table 1 first column), all methods achieve 100 ASR in different LLMs, making it hard to draw accurate conclusions about whether the proposed method is not compromising much attack strength.

Author response:
### Weakness 2b. Evaluations: lack of evaluations on more robust LLMs
> I'm not saying that the effectiveness demonstrated in models such as Vicuna-7B is totally not persuading. However, back to my point of soundness, as there exists a trade-off between attack strength and readability, it becomes necessary to conduct evaluations on more robust (aligned) LLMs to show the ability to address this trade-off of the proposed method. Otherwise, as we can see on (Page 7, Table 1 first column), the methods all achieve 100 ASR in different LLMs, so it becomes hard to gain accurate conclusions about whether the proposed method is not compromising much attack strength.

As suggested by the reviewer, we additionally jailbreak Llama2-chat in the individual behavior setting. Note that this is the setting that most jailbreak work (except GCG) considers solely. We show that our method achieves slightly lower training ASR but better test ASR than GCG in this setting.

Reviewer comment (Experiments):
Lack of ablation studies. As the proposed method is aimed at solving the aforementioned trade-off and proposes a dual-target loss function, it is surprising that no ablation studies on these components are provided. For example, how do the weight parameters w_1 and w_2 affect the training process and the final scores? How does the proposed readability loss influence ASR and perplexity? This kind of ablation study is quite important for papers introducing new loss constraints, especially if the proposed loss might conflict with the original loss (target attack loss).

Author response:
### Weakness 2c. Evaluations: Lack of ablation studies
> As the proposed method is aimed at solving the aforementioned trade-off and proposed a dual-target loss function, it surprise me that the paper has not provided ablations studies on these two parts, for example, how are the weigh parameters w_1 and w_2 affecting the training process and the final scores. This leaves many important questions not answered, for example, how the proposed readability loss is influencing ASR and perplexity. From my perspective, this kind of ablation study is quite important for papers like introducing new loss constraints, especially if the proposed loss is somewhat in conflict with the original loss (target attack loss).

We thank the reviewer for pointing this out! We show how the two parameters affect the training process (convergence speed) in our **General Response - 3. Computational Complexity Analysis**, and their effect on the final ASR and perplexity in the **General Response - 4. Hyperparameter Ablation**.

Reviewer comment (Evaluation):
Lack of evaluations on computational cost. There are no evaluations of the computational cost of the proposed method. Such evaluation can make readers more familiar with aspects like convergence speed. Additionally, it might be preferable to maintain a similar (or smaller) computational cost compared with the existing GCG method.

Author response:
### Weakness 2d. Evaluations: Lack of evaluations on computational cost
> There are no evaluations of the computational cost of the proposed method. Such evaluation can make readers more familiar with the aspects like convergence speed. And it may be better to keep a similar (or smaller) computational cost compared with the existing GCG method.

Thanks for this suggestion! We report the computational cost in **General Response - 3. Computational Complexity Analysis**. Considering the individual behavior, our method uses a similar amount of time to generate a similarly powerful prompt (~50 token long, 200 steps) as GCG (200 steps), and about x1.5 the time cost when we generate ~100 token long prompt (400 steps) as GCG (400 steps).