Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialogue
Authors: Zhenhong Zhou, Jiuyang Xiang, Haopeng Chen, Quan Liu, Zherui Li, Sen Su


Reviewer paper summaries: This paper explores a jailbreaking technique in multi-turn dialogues by breaking down harmful queries into seemingly harmless sub-queries across several turns. Leveraging the propensity of LLMs to be helpful, this attack gathers pieces of information throughout the conversation to ultimately answer the harmful prompt, without surpassing the misalignment threshold at any single turn. 
Since query decomposition is challenging, the authors utilize GPT-4 and few-shot demonstrations (i.e., in-context learning) to achieve this successfully.
Through experiments with popular LLMs like ChatGPT, Claude, and Gemini, the authors show how this "Speak Out of Turn" attack enables the models to gradually produce harmful information, even though each individual turn seems benign.
Summary of strengths: - The paper is generally well-written, with figures that are well-referenced and clearly explained.
- The proposed attack is intriguing, novel, and appears to be robust.
- The proposed attacks were tested and achieved some success against commercial state-of-the-art models.
-The author provides a compelling and engaging discussion on why LLMs are susceptible to multi-turn attacks, relating their approach to the "Competing Objectives" and "Mismatched Generalization" concepts of LLM safety.
Summary of weaknesses: - The authors mention that their evaluation focuses on the entire conversation, including both the intermediate responses to decomposed questions and the final answer. It is unclear why the entire conversation is evaluated. Why not evaluate the harmfulness of the final answer given the initial malicious question?

- Please clarify why three evaluation systems are needed. If the goal is to use multiple judges for higher precision, why not combine their scores and present the final results as a mixture of experts to simplify the comparison?

- While the authors mention in the introduction that they propose several mitigation strategies, I didn't find any specific practical approaches. They argue in their discussion that it is "essential to enhance the model’s understanding of context further," but they do not suggest how to achieve this.

- I didn’t fully understand the Multi-turn Ablation experiment (Sec 4.3.3). It would be helpful to include a figure comparing this approach to the regular approach. Specifically, it is not clear what you mean by "We combine the sub-queries into one turn and conduct baseline generation experiments." Do you combine all the decomposed questions into a single prompt, or are the sub-queries prompted in separate sessions versus a single session dialogue? If the latter, are sub-queries that require answers from previous sub-queries also included? Are these answers provided in the prompt? This approach seems more evasive because providing the entire conversation, including previous turns, gives the model a chance to detect the malicious intent. Can you clarify this?

- Additionally, the authors mention in line 437 that "GPT-4’s judgments and average scores for harmfulness significantly decreased in a single-turn dialogue." They conclude that "turn dialogues are real." However, this is not a direct comparison. An alternative explanation could be that the judge is biased by length, or when the judge sees the entire conversation, the malicious intent is revealed from the context.
Comments, suggestions and typos: Minor comments:
1. Line 397: missing references.
2. 50: LLM s -> LLMs
3. Line 132: what is "TB-level" tokens?
4. Line 143: DPO is "Direct Preference Optimization" and not "Decision Process Optimization".
5. Lone 158: missing space before reference.
6. Line 204: The "201" appears to be a typo.
7. Table 1 is cluttered with numbers, making it difficult to read. Consider adding a line to separate the LLaMA Guard and GPT-4 results, and bolding the best result in each subcategory.
8. In table 1: which model decomposed the questions?
9. Is Table 2 an extension of Table 1? If so, why not combine them? They are located far apart, making comparison difficult.
Since models are used for: 1. decomposing the question; 2. serving as the target model (the model under attack); and 3. acting as a judge, to avoid confusion, make it clear in the text and tables that the models are the target model.
Overall assessment scores: 3