## 1 Introduction  
Large Language Models (LLMs) like ChatGPT [1] are highly anticipated for applications across a wide range of fields including education, research, social media, marketing, software engineering, and healthcare[2, 3, 4, 5, 6]. However, the use of extremely diverse texts for training LLMs [7] often leads to the generation of ethically harmful content [8]. This poses a significant barrier to the real-world application of LLMs. LLM vendors are acutely aware of this issue and have implemented safeguards such as reinforcement learning with human feedback to align LLMs with human values and intentions [9], and external systems to detect and block ethically harmful inputs (prompts) and outputs (responses), thus preventing the generation of problematic texts [10].  
Nevertheless, these safeguards can be bypassed, enabling the generation of ethically harmful content by LLMs [11, 12]. Often referred to as \"jailbreaks\", this represents a key these prompts often contain unnatural (unreadable) texts, making them detectable based on this criterion [19; 20], thus enabling their blockage.  
However, jailbreak attack methods have been evolving, particularly with recent efforts focused on creating prompts in natural language and formulating high-performance jailbreak prompts for black-box LLMs [21; 22]. Yet, existing methods often rely on white-box LLMs or require complex prompt designs, resulting in high computational costs and complexity. Furthermore, when considering the development of practical state-of-the-art methods [18] that are versatile (not limited to adversarial suffixes) and easily implemented, it can still be argued that jailbreak attacks remain somewhat limited.  
In this study, contrary to expectations, we demonstrate that jailbreak prompts written in natural language, which are highly effective against black-box LLMs, can be created with remarkable ease. Specifically, we propose a simple black-box method for jailbreak attacks and illustrate its effectiveness. Our method involves targeting a black-box LLM and repeatedly rewriting ethically harmful questions (prompts) into expressions deemed harmless. By using these rewritten prompts as inputs, the method successfully jailbreaks the LLM. This approach is based on the hypothesis that it is possible to sample expressions with the same meaning as the original prompt directly from the target LLM, thereby bypassing safeguards. In simpler terms, this method involves inducing the target LLM to confess its own jailbreak prompts.  
The contributions of this study are as follows:  
* Proposal of an extremely simple black-box method for jailbreak attacks. Compared to existing research, the proposed method is extremely easy to implement. It does not require the design of sophisticated prompts and is composed only of simple prompts. There is no need for white-box LLMs and high-spec computing environments to operate them; the method can be implemented in a general user's computing environment using only the application programming interface (API) for black-box LLMs.
* High attack success rate and high efficiency. The proposed method demonstrates high attack performance against a wide range of ethically harmful questions in various scenarios, compared to manual jailbreak prompts and other existing methods. Empirically, the proposed method can create jailbreak prompts with fewer iterations than expected. In experiments using ChatGPT (GPT-3.5 and GPT-4) and Gemini-Pro, an attack success rate of over 80% was achieved in an average of five iterations.
* Simple jailbreak prompts written in natural language. Since the prompts are rewritten by LLMs, they are naturally in natural language. Furthermore, compared to existing methods, the jailbreak prompts are significantly shorter.
* High evasiveness against defense. The jailbreak prompts generated by the proposed method, due to their nature, evade the versatile and practically implementable state-of-the-art defense method, maintaining a high attack success rate.