Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLM
Authors: Xikang Yang, Xuehai Tang, Jizhong Han, Songlin Hu


Reviewer paper summaries: This research paper introduces "Chain of Attack" (CoA), a method designed to identify vulnerabilities in large language models (LLMs) during multi-turn conversations. CoA analyzes model responses and conversation context to generate increasingly effective attack prompts, potentially leading LLMs to generate unsafe or harmful content. A core assumption in CoA is that the semantic similarity between the model's responses and the target query will increase as the conversation progresses and the attacker successfully steers the model toward the harmful goal. Experiments demonstrate CoA's efficacy in exposing vulnerabilities across a range of LLMs, including open-source and closed-source models.

While the paper seemingly proposes a novel and robust attack method, several important aspects are not described clearly and unambiguously.
Summary of strengths: 1. The proposed attack appears novel and interesting, and the experiments demonstrate robust results.
2. The authors provide code that allows for reproducibility.
3. The authors included a Statement on the Use of AI Assistants to clarify how AI was used in their research and writing.
4. The authors evaluated the LLM as a judge (GPT-3.5-turbo) by comparing its outputs to the annotations of human experts.
Summary of weaknesses: As mentioned in the summary, many important details in the paper are not clearly described, making it difficult for readers to fully understand the proposed approach and for reviewers to assess the novelty, soundness, and contribution of this work.

1. The seed attack chain generation (section 4.1) is not described clearly, leaving much room for ambiguity. The paper mentions that "multiple candidate multi-turn attack prompts are generated based on the chosen target task." Does this mean that the seed model generates multiple sequences of prompts, with each sequence containing a prompt for each turn of the conversation?
2. In line 236, when talking about the "Seed Attack Chain Generator" the authors mention that "Each update to the prompt varies within a certain range." What is meant by "update"? Does this refer to the progression of turns?
3. In line 243, the authors observed a rising trend in the toxicity score and semantic similarity of the generated "initial" attack chain. What is meant by "initial"? Is this an observed phenomenon or the intended design of the sequence?
4. In line 257, it is unclear if the target model is attacked using the seed chain in a dialogue fashion, where in each turn, the previous turns are provided in the context.
5. It is also not clear how the generator is able to create semantically closer prompts as a function of the turn. Does this process create independent chains, or alternatively, are independent prompts created for each turn? Or are tree-like chains created? A more formal description of this stage would substantially enhance understanding.
6. Figure 4 is unclear. It should give an overview of how the seed attacks are created and how the attack chain is updated, but it fails to do so. The uninformative caption adds to this lack of clarity.
7. In Figure 5, it is not clear how the toxicity was calculated. Was it using the OpenAI moderation tool?
8. The authors use terms inconsistently and unclearly. Although I am not a native English speaker, the wording of many sentences appears odd to me and seriously hinders understanding, especially given the complexity of the proposed approach. Additionally, the authors use odd terms and alternate between synonyms, making it hard to follow. For instance, in line 160, the authors use the term "command" instead of "prompt." They also use "evaluator" and "judger" interchangeably. Furthermore, it is unclear if the word "response" refers to the output of the red teaming model (indicated as \( u \) in line 4 of Algorithm 1) or the response of the target model, as used in many other places in the text.
Comments, suggestions and typos: Minor comments:
1. Use unified notation: either E(O,r_i) or E(O|r_i)
2. Line 154: Italic T?  
3. Figure 3: The figure text is too small and truncated.
4. Line 200: fix the mathematical notation 
5. Line 354: Why aren't the equations at the beginning of page 6 numbered?
6. Line 355: Missing closing bracket in the definition of S'(t)
7. Algorithm 1: use either M_{tgt} or M_{target} for consistency
8. Line 433: what do you mean by the discriminator? The LLM as a judge?
9. In Section 5.2 it is mentioned that only vicuna and Llama are used as the target model,  but in the caption of Table 1, several more models are compared. Why?
10. Line 419: Use either PAIR or Pair for consistency.
Overall assessment scores: 3