Kati8/ICLR_2023_2025 huggingface dataset:
authors: Seanie Lee
authors: Haebin Seong
authors: Dong Bok Lee
authors: Minki Kang
authors: Xiaoyin Chen
authors: Dominik Wagner
authors: Yoshua Bengio
authors: Juho Lee
authors: Sung Ju Hwang
reviews: 
Comments: This paper focuses on distilling the safeguarding ability of LLMs into a smaller one, particularly through the data augmentation perspective.
Strengths: 1. Overall, this is a well-written paper. The motivation and setting are clearly stated.
1. The problem of distilling large safeguard models into small ones is novel and has good applications.
1. The proposed HarmAug method of using ICL examples is simple and easy to understand.
1. The experiment is very comprehensive.
1. Code and other resources are provided.
Weaknesses: 1. While not explicitly instructed in the Call for papers, the Reproducibility statement and Ethics statement should be in \section*{} format (following previous years).
1. The overall prompt design shares similar notions with 2 existing works, which may should be cited in Section 3.2: 
    - using prefixes to bypass the safety guardrails of the LLMs is similar to Prefix Injection [1]
    - using ICL examples to bypass the safety guardrails of the LLMs is similar to in-context attack [2]
1. Are there other applications of HarmAug beyond model distillation? This point was not well-discussed in the paper.

[1] Jailbroken: How Does LLM Safety Training Fail? https://arxiv.org/pdf/2307.02483

[2] Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations https://arxiv.org/pdf/2310.06387
Recommendation: 10
Reviewer confidence: 4
Soundness/correctness: 4
Clarity: 4