Do Anything Now: Characterizing In-the-Wild Jailbreak Prompts
Date: 2023-08
Affected: GPT-3.5, GPT-4, ChatGPT
Attack vector: jailbreak
Category: research
Severity: High

Shen et al. collect and analyze 1,405 jailbreak prompts and 131 communities (2022-2023). They find five jailbreaks reaching 0.95 ASR on GPT-3.5/4 and persisting online for 240+ days. Published at ACM CCS 2024.


Mitigations:

Architectural layers (MAESTRO):