ObscurePrompt: Jailbreaking Large Language Models via Obscure Input
Authors: Yue Huang, Jingyu Tang, Dongping Chen, Bingda Tang, Yao Wan, Lichao Sun, Xiangliang Zhang


Reviewer paper summaries: This paper proposes ObscurePrompt, which leverages GPT-4 to continuously modify the original harmful prompts, rendering their expressions obscure and difficult to understand in order to achieve jailbreak. This approach exploits the inherent limitations of Large Language Models (LLMs) when handling Out-of-Domain (OOD) data. Experiments demonstrate that both open-source and closed-source models exhibit a certain degree of vulnerability when confronted with these obscure harmful prompts.
Summary of strengths: 1. This paper aims to investigate the jailbreak of LLMs, which is a direction worthy of attention and discussion.

2. The proposed method is simple and does not require access to the model weights.
Summary of weaknesses: 1. After undergoing Obscure-Guided Transformation, the semantics of the sentence may shift, leading to inaccurate descriptions of the initial harmful prompt. The authors should evaluate the extent of this semantic shift and its impact on the success of the jailbreak.

2. The evaluation metrics are not comprehensive. The ASR metric based on keyword matching is not accurate and can result in many false positives. Even if the responses generated by LLMs do not contain words like "sorry" or "I can't," they may still not be harmful. The authors should introduce evaluations using GPT-4 [1][2][3] and sample some data for manual annotation to ensure the accuracy of ASR.

[1] AUTODAN: GENERATING STEALTHY JAILBREAK PROMPTS ON ALIGNED LARGE LANGUAGE MODELS

[2] Jailbreaking Black Box Large Language Models in Twenty Queries

[3] A Wolf in Sheep’s Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily

3. Regarding Attack Integration, the results in Table 2 show each baseline using this evaluation method, i.e., generating multiple jailbreaks and considering it a success if any one of them works. If not, then the comparisons in the table are unfair. In addition, Attack Integration is more like an ensemble method, so the average jailbreak ASR should also be reported. Specifically, the authors should run ObscurePrompt five times across the entire dataset and calculate the average ASR for these five runs.
Comments, suggestions and typos: 1. In section 5.1, the Advbench dataset should comprise 520 samples instead of 521.

2. In section 5.3, it is stated that "This also indicates that OBSCUREPROMPT is robust to PPL-filtering defender." However, in the Limitations section, it is mentioned that "Furthermore, our attack tends to underperform in scenarios involving perplexity filtering, due to the inherently high perplexity associated with obscure text." This is contradictory. I believe that PPL filtering can indeed significantly defense OBSCUREPROMPT.
Overall assessment scores: 2.5