Shake-It-Off: Jailbreaking Black-Box Large Language Models by Shaking Off Objectionable Semantics

Authors: Yanbo Chen, Yang Cao, Weiwei Liu
Primary area: foundation or frontier models, including LLMs
Keywords: Jailbreaking Attacks, Large Language Models

Abstract:
Large language models (LLMs) are vulnerable to jailbreaking attacks (Zou et al., 2023; Liu et al., 2024), in which attackers use adversarially designed prompts to bypass the model’s safeguard and force the model to generate objectionable content. The present paper studies jailbreaking attacks from a red team’s viewpoint and proposes a novel black-box attack method, called Shake-It-Off (SHAKE), that only requires the response generated by the victim model. Given objective query $T_{obj}$, our method iteratively shakes off the objectionable semantics of $T_{obj}$, making it gradually approximates a pre-defied decontaminated query $T_{dec}$. We conduct extensive experiments on multiple baseline methods and victim LLMs. The experimental results show that SHAKE outperforms the baseline methods in attack success rates while requiring much less running time and access to the victim model.

Decision: reject
Reviewer scores: 3, 6, 5, 5

Reviews and discussion:
<thread>
creation date:
1730431373958

thread_type:
official_review

<thread_object>
creator: reviewer_nrrx

<content>
summary:
This paper presents SHAKE (Shake-It-Off), a new efficient method for jailbreaking large language models. SHAKE iteratively modifies model outputs to bypass safeguards and generate objectionable content, using only the model's responses. Experiments show it outperforms baseline methods in attack success while requiring less time and model access. The study aims to understand these attacks from a red team perspective, likely to improve future LLM defenses.

strengths:
1. The jailbreaking attack is an interesting topic.
2. The reported attack success rate appears to be notably effective, which underscores the importance of developing robust defense mechanisms for large language models.

weaknesses:
1. The novelty of the proposed approach appears to be somewhat limited in the context of existing research.
2. Given that the authors present their attack as a black-box method, it's surprising that they didn't extend their evaluation to include prominent commercial models such as Claude and ChatGPT. This omission raises questions about the method's broader applicability.
3. The paper would benefit from a more comprehensive discussion of potential defense strategies. It would be valuable to see an evaluation of the proposed method against representative defense mechanisms, providing a more balanced perspective on the attack's effectiveness in real-world scenarios.

questions:
Please see comments.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730602205523

thread_type:
official_review

<thread_object>
creator: reviewer_8edi

<content>
summary:
The paper presents a new jailbreaking attack against LLMs in the black box setting. Borrowing techniques from PAIR and HSJ attacks, the attack proceeds in multiple steps: remove sensitive words/phrases from a query, construct an initial adversarial prompt, and iterate while the prompt is rejected by the model. Each iteration includes sampling of prompts and keeping the ones closest to the decontaminated one in the embedding space. The paper evaluates the attack against llama 2 and vicuna models and compares the performance against three baselines: autoDAN, GCG, and ReNeLLM. The evaluation shows higher success with higher efficiency.

strengths:
The paper describes the proposed attack clearly and also provides the algorithm and walks the reader through the algorithm steps. Furthermore, the appendix shows the prompts used for sampling and shows the sensitive words/phrases used for decontamination. The paper also includes a threat model, which contextualizes the proposed attack.

The attack approach is computationally efficient because the sampling approach is guided by similarity to a decontaminated prompt, potentially resulting in lower query complexity (to the model under attack).

The evaluation shows that the proposed attack performs better than white box or black box baselines.

weaknesses:
The paper needs to present a clearer argument regarding the contributions over the PAIR attack. It appears as if the paper borrows several techniques from the PAIR attacks, but adds a sampling step so that the adversarial sample is closer to a benign sample. The evaluation section does not show if the proposed attack improves over PAIR. Most probably, the authors forgot to add those results to the evaluation section because the introduction to that section mentions PAIR as a baseline. As such, the evaluation section needs to compare against the PAIR attack and provide an ablation study showing that the added sampling step improves adversarial success and improves efficiency by reporting queries to the model under attack. 

Moreover, it would be useful to the reader to see if the attack works against commercial models, such ChatGPT or other large models such as Mistral or Llama 3. The evaluation is performed against models that are considered by today's standards to be smaller and older. The PAIS paper, for example, shows evaluation over GPT, Gemini, and Claude. It should be feasible to run the attack against such models. 

It would have also been useful to show examples of the generated adversarial prompts.

The paper does not include an ethics discussion, nor does it discuss potential defenses or mitigation. It is important for the paper to have a discussion section about these issues especially as it generates harmful prompts for aligned language models.

questions:
How does the performance of the proposed attack compare against PAIR on commercial models?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730631592532

thread_type:
official_review

<thread_object>
creator: reviewer_yten

<content>
summary:
The authors propose a novel black-box jailbreaking strategy, SHAKE, which iteratively adjusts the target query in the direction of  a decontaminated version, gradually removing objectionable semantics and ultimately bypassing the model's safeguards.

strengths:
1. Simple but effective strategy for jailbreaking.
2. Paper is well written.
3. Comprehensive comparison with prior work.

weaknesses:
1.The evaluation section has very limited experiments. 

2. Lack insights on why the attack works. See questions below.

questions:
1. Your comparison is currently limited to prior work on smaller LLMs. It would be valuable to see how your attack performs on larger models, such as GPT-4, Claude, or Gemini, given that your attack is black-box in nature. Additionally, comparing your method with other black-box attacks, like ReNeLLM and PAIR, under these conditions would improve the evaluation.

2. I didn't see a clear reasoning for why the decontaminated query is constructed by replacing objectionable semantics with [PAD] tokens. Why not use other tokens, or even allow the target LLM to decontaminate the query itself to make it less objectionable? An ablation study on this approach could provide better insights into why the attack is effective.

3. Can you provide more insight on why other black-box attacks like PAIR and ReNeLLM don't work as well given there is significant overlap between those strategies and yours except the fact that you move in the direction of  the less objectionable query per step. I think results for Question 2 may provide answers to this as well.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730757402728

thread_type:
official_review

<thread_object>
creator: reviewer_tpgy

<content>
summary:
The paper introduces SHAKE, a black-box jailbreaking attack method for LLMs. SHAKE iteratively removes objectionable semantics from adversarial prompts, leading to successful bypassing of model safeguards. The authors validate SHAKE's effectiveness and efficiency on various datasets and LLMs, showing improvements in attack success rates and reduced running time compared to existing methods.

strengths:
The method is clearly explained, and the writing is well-executed.

weaknesses:
All of the evaluated LLMs, such as Vicuna and LLaMA2, are open-weight models that have already been shown to be quite fragile, casting doubt on how effective the attack would be against more robust, modern models. Furthermore, the method demonstrates only a marginal improvement over the current state-of-the-art.

I don't believe this type of "yet another jailbreak" paper provides significant value to the research community.

questions:
Q1: All of the evaluated LLMs, such as Vicuna and LLaMA2, are already shown to be quite fragile against jailbreaks. Pushing the ASR on these models from ~90% to 100% is not a novel contribution. It's unclear how effective the attack would be on more robust, modern models. 

Q2: How does the attack perform on proprietary models? The attack is described as a black-box method, but none of the evaluated models seem to be true black-box models.
</content>
</thread_object>

</thread>


Metareview: