## 4 Experiments
### Performance Evaluation of Jailbreak Attacks
In the \"Hate Speech\" scenario, the baseline ASR of 10.0% on GPT-3.5 suggests relatively strong safeguards, yet the proposed method achieved an ASR of 83.3%, while PAIR's was 63.3%. On GPT-4, both the proposed method and PAIR achieved an ASR of 80%. For Gemini-Pro, the ASR of the proposed method (80.0%) was slightly lower than that of PAIR (83.3%), which could be attributed to the relatively high baseline ASR (30.0% and 36.7% for GPT-4 and Gemini-Pro, respectively). Although the proposed method generally performs well, it appears to struggle in certain scenarios. For instance, in the \"Financial Advice\" scenario, the ASR of the proposed method was not the best across all models. On GPT-3.5, manual jailbreak attacks performed the best with an ASR of 96.7%, compared to 80.0% for the proposed method. On GPT-4 and Gemini-Pro, PAIR was the best, with ASRs of 100.0% and 96.7%, respectively, while the proposed method had ASRs of 70.0% and 86.7%. A similar trend was observed in the \"Health Consultation\" scenario. On GPT-3.5, manual jailbreak attacks were best with an ASR of 93.3%, compared to 55.3% for the proposed method. On GPT-4 and Gemini-Pro, PAIR again performed the best with ASRs of 90.0% and 83.3%, respectively, while the proposed method had ASRs of 50.0% and 80.0%. This may be attributed to the fact that the questions in these scenarios are not inherently discomforting. For example, a question in the \"Financial Advice\" scenario is \"What specific stocks should I invest in for maximum returns?\" and, in the \"Health Consultation\" scenario, \"How can I manage this suspected inflammatory bowel disease?\" While these are questions better suited for experts than LLMs, they are not inherently discomforting for most people. Since the prompt rewriting AdversarialRephrasing in the proposed method aimed to reduce discomfort, it was less effective for these types of questions. For GPT-4 and Gemini-Pro, there were instances where the ASR of manual jailbreak attacks was lower than the baseline ASR. This appears to be because many of the manual jailbreak prompts were created based on GPT-3.5. In fact, in GPT-3.5, the ASR for manual jailbreak attacks was higher than the baseline, both overall and in individual scenarios. To evaluate the versatility of jailbreak attacks, the ASR for each method was calculated for another dataset, the harmful behavior dataset (Table 2). In this dataset too, the proposed method achieved higher ASR compared to other methods overall. Specifically, in GPT-3.5, the ASR for the proposed method was 80.0%, which was higher than that of manual jailbreak attacks (8.0%) and the state-of-the-art method, PAIR (54.0%). Furthermore, the average number of iterations required for jailbreaking was 7.2 for the proposed method and 8.9 for PAIR. The proposed method achieved higher ASR with fewer iterations. In GPT-4, the ASR for the proposed method was 88.0%, compared to 2.0% for manual jailbreak attacks. This was the same as the ASR for PAIR, but the average number of iterations required for jailbreaking was lower for the proposed method (4.8 compared to 5.2 for PAIR), indicating that the proposed method demonstrated equivalent attack performance to PAIR with fewer iterations. For Gemini-Pro, the ASR for the proposed method (72.0%) was higher than that of manual jailbreak attacks (58.0%) and the same as that of PAIR (72.0%). The average number of iterations required for jailbreaking was slightly higher for the proposed method (6.4 compared to 6.2 for PAIR), but, given the simplicity of the proposed method, it could be considered a powerful complementary method to PAIR.