Continuous Embedding Attacks via Clipped Inputs in Jailbreaking Large Language Models
Authors: Zihao Xu, Yi Liu, Gelei Deng, Kailong Wang, Yuekang Li, Ling Shi, Stjepan Picek


Reviewer paper summaries: This paper proposes a novel approach for input attacking on LLMs for jailbreaking. The study conducted experiments by optimizing over discrete and continuous suffixes, which are predominant in white-box setting jailbreaking. The main contributions of the paper include:

- Mitigating and relaxing the overfitting problem for long gradient updates through a novel approach of clipping embeddings in hidden space with mean and standard deviation values. This improvement can be clearly seen from the results presented in the tables.

- Introducing a low-resource computing method for jailbreaking that might benefit research aimed at closing this backdoor vulnerability in LLMs.
Summary of strengths: Well written with a great deal of attention to how their work is situated in the landscape of jailbreaking, and a strong understanding of the limitations of their model, with room for follow-up to this work. The authors demonstrate the ability of the proposed model to jailbreak open-source LLMs. This method:

- Fundamental idea isn't completely new, however, it presents a unique application within its specific methodological context.
- Allows other researchers to explore internal bias within LLMs' embedding space.
- Provides a simple and effective approach that can be reproduced with a single GPU setting.
Summary of weaknesses: There are the following weaknesses which might contribute to not accepting this paper:

- The Attack Success Rate (ASR) measurement proposed in the paper is not evaluated against existing methods for jailbreaking LLMs. It would be beneficial to see the performance compared to other white-box continuous or discrete suffix optimization methods, especially those of (Zou et al., 2023b) and (Schwinn et al., 2023).

- The authors mentioned that random patterns they were trying to remove persist after applying engineering methods. Although the justification states that (1) overfitting with contrastive vector, (2) overfitting vector, and (3) jailbroken vector are separable after t-SNE projection, it is still not convincing that the patterns are indeed separable.

- It is unclear why shorter suffixes tend to be better regularizers than longer ones
Comments, suggestions and typos: This approach can be done only on open-source LLMs such as LLaMA, which has been targeted in the paper. The paper also acknowledges the need for investigations on the explainability of regularizers, and it is not clear how input embedding changes over long iterations, which could be valuable for understanding its behavior.

Additionally, I would recommend that the authors specify which versions of LLaMA and Vicuna they studied. Although it is obvious that in one GPU scenario with RTX series, smaller LLMs are used, it is still worth mentioning explicitly.

Finally, it would also be worth adding a discussion on the potential negative impacts of this work from the authors' point of view.
Overall assessment scores: 3