Imperceptible Jailbreaking against Large Language Models
Authors: name: Kuofeng Gao, name: Yiming Li, name: Chao Du, name: Xin Wang, name: Xingjun Ma, name: Shu-Tao Xia, name: Tianyu Pang
Organization: Sea AI Lab

Jailbreaking attacks on the vision modality typically rely on imperceptible
adversarial perturbations, whereas attacks on the textual modality are
generally assumed to require visible modifications (e.g., non-semantic
suffixes). In this paper, we introduce imperceptible jailbreaks that exploit a
class of Unicode characters called variation selectors. By appending invisible
variation selectors to malicious questions, the jailbreak prompts appear
visually identical to original malicious questions on screen, while their
tokenization is "secretly" altered. We propose a chain-of-search pipeline to
generate such adversarial suffixes to induce harmful responses. Our experiments
show that our imperceptible jailbreaks achieve high attack success rates
against four aligned LLMs and generalize to prompt injection attacks, all
without producing any visible modifications in the written prompt. Our code is
available at https://github.com/sail-sg/imperceptible-jailbreaks.