Title: Red-Teaming Text-to-Image Systems by Rule-based Preference Modeling
Venue: NeurIPS 2025
Accepted: yes
Keywords: Text-to-Image Systems

Rule-based Preference Modeling

Red-Teaming
Abstract: Text-to-image (T2I) models raise ethical and safety concerns due to their potential to generate inappropriate or harmful images. Evaluating these models' security through red-teaming is vital, yet white-box approaches are limited by their need for internal access, complicating their use with closed-source models. Moreover, existing black-box methods often assume knowledge about the model's specific defense mechanisms, limiting their utility in real-world commercial API scenarios. A significant challenge is how to evade unknown and diverse defense mechanisms. To overcome this difficulty, we propose a novel Rule-based Preference modeling Guided Red-Teaming (RPG-RT), which iteratively employs LLM to modify prompts to query and leverages feedback from T2I systems for fine-tuning the LLM. 
RPG-RT treats the feedback from each iteration as a prior, enabling the LLM to dynamically adapt to unknown defense mechanisms. Given that the feedback is often labeled and coarse-grained, making it difficult to utilize directly,  we further propose rule-based preference modeling, which employs a set of rules to evaluate desired or undesired feedback, facilitating finer-grained control over the LLM’s dynamic adaptation process. Extensive experiments on nineteen T2I systems with varied safety mechanisms, three online commercial API services, and T2V models verify the superiority and practicality of our approach. Our codes are available at:  https://github.com/caosip/RPG-RT.


Peer reviews:
comments: SUMMARY:
This paper introduces a novel framework for red-teaming text-to-image (T2I) models, called Rule-based Preference modeling Guided Red-Teaming (RPG-RT). The framework aims to evaluate and enhance the security of T2I systems by iteratively modifying input prompts and utilizing feedback to bypass safety mechanisms and generate harmful content, while maintaining semantic similarity to the original image.

STRENGTHS AND WEAKNESSES:
Strengths:

Innovative Approach: RPG-RT uses iterative feedback and rule-based preference modeling, allowing it to adapt to evolving defense mechanisms.

High Attack Success Rate: It significantly outperforms existing methods in bypassing defenses and generating harmful content.
Generalization Across Models: It works effectively across various T2I systems and defense types, showing robustness and flexibility.

Semantic Similarity Preservation: RPG-RT maintains semantic similarity with the original image while generating harmful content.

Weaknesses:
Dependence on Feedback Quality: The effectiveness relies on the quality of the feedback from T2I systems, which may be coarse-grained.

Computational Cost: Initial training and fine-tuning of the LLM require significant computational resources.

Ethical Concerns: Its ability to generate harmful content raises concerns about potential misuse in malicious contexts.

QUESTIONS:
See weakness

LIMITATIONS:
NA
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents RPG-RT (Rule-based Preference modeling Guided Red-Teaming), a novel framework for red-teaming text-to-image (T2I) systems in commercial black-box settings. Existing red-teaming methods for T2I models struggle in realistic commercial scenarios because they assume knowledge of specific defense mechanisms. Real-world commercial APIs deploy unknown combinations of detection filters, content removal, and safety alignment. The authors propose RPG-RT and uses an iterative approach with 1) Prompt Modification: Uses an LLM agent to generate multiple modifications of harmful prompts 2) Rule-based Preference Modeling: Develops a scoring model that separates harmful content intensity from benign semantic similarity using a novel CLIP representation with a reference image generated from a T2I model M_0 without defense mechanisms. 3) Direct Preference Optimization (DPO): Fine-tunes the LLM based on feedback 2). The results on 19 T2I systems with various defenses, plus commercial APIs (DALL-E 3, Leonardo.ai, SDXL). RPG-RT achieves significantly higher attack success rates (ASR) while maintaining semantic similarity.\n\nSTRENGTHS AND WEAKNESSES:\n1. Realistic Problem Setting: Addresses the most challenging and practical scenario - commercial black-box systems with unknown defenses.\n\n2. Comprehensive Evaluation: Extensive testing across 19 different defense mechanisms and real commercial APIs demonstrates robustness.\n\n3. Based on the results in this paper, current commercial black-box systems should be aware of their limitation on rejecting unsafe query. This will further improve the safety of current generative models.\n\nQUESTIONS:\n1. Ethical Concerns: Provides a powerful tool for generating harmful content from commercial systems and this tool could enable users to bypass safety measures more effectively. However, in the paper, the authors should provide discussion on potential ways to avoid being attacked by such methods.\n\n2. In Line 198-200, the authors mentioned that \"with the images M0(P) generated by the original prompt P on the reference T2I model M0 without defense mechanisms.\" Does the selection of this model M0 without defense mechanisms influences the result of attack and how does it influence the results?\n\n3. Efficiency. The proposed pipeline operates in an iterative way. The reviewer is wondering how efficient it is. More quantitative results on efficiency should be provided.\n\nLIMITATIONS:\n1. Ethical Concerns: Provides a powerful tool for generating harmful content from commercial systems and this tool could enable users to bypass safety measures more effectively. However, in the paper, the authors should provide discussion on potential ways to avoid being attacked by such methods.\n\n2. In Line 198-200, the authors mentioned that \"with the images M0(P) generated by the original prompt P on the reference T2I model M0 without defense mechanisms.\" Does the selection of this model M0 without defense mechanisms influences the result of attack and how does it influence the results?\n\n3. Efficiency. The proposed pipeline operates in an iterative way. The reviewer is wondering how efficient it is. More quantitative results on efficiency should be provided.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 1, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes RPG-RT, a rule-based preference modeling framework for red-teaming commercial text-to-image (T2I) systems. The method uses an LLM to iteratively modify prompts and employs a scoring model based on decoupled CLIP embeddings to fine-tune preferences via DPO. The authors claim state-of-the-art performance across multiple defenses and real-world APIs such as DALL-E 3, Leonardo.ai, and SDXL.\n\nSTRENGTHS AND WEAKNESSES:\n1. The paper tackles an important and timely problem - “how to effectively red-team commercial text-to-image systems without knowing their internal safety mechanisms”. This black-box setup is realistic and relevant.\n2. The core idea - using LLMs to iteratively adapt prompts based on feedback, and guiding them with rule-based preferences is creative and shows some technical novelty.\n3. Demonstrates consistent ASR improvement over prior methods. Adapts to diverse NSFW categories and multiple commercial models.\n\nQUESTIONS:\n1. The paper relies entirely on automated detectors (NudeNet, CASCo, Q16, etc.) and custom scoring models to judge NSFW severity and semantic similarity. No human validation is conducted to confirm whether the images are genuinely harmful or inappropriate, which is critical given the subjective nature of NSFW content.\n\n2. (a) While the paper reports ASR, FID, and CLIP similarity, there are no confidence intervals, no statistical tests, and no variance analysis across seeds or repeated runs. This makes it hard to assess robustness. (2. b) There exist several new SOTA evaluation metrics to compare. So, evaluation based on only these three (ASR, FID, and CLIP similarity) metrics is not reliable. (2. c) Metrics like FID and CLIPSim do not reflect real-world risk or ethical severity. A lower FID doesn’t always mean a worse attack. There is no ethical alignment or societal risk score, which would be more meaningful in red-teaming evaluations.\n\n3. They repeatedly query the same prompt if it generates a Type-3 image (NSFW bypassed), which might bias the model and inflate ASR. This is acknowledged but not fairly ablated.\n\n4. While the authors test on DALL-E 3, Leonardo.ai, and SDXL, they don’t report which prompts succeed/fail or the actual content (only aggregate ASR). A qualitative inspection with a case-by-case breakdown is essential.\n\n5. RPG-RT is evaluated directly on unseen prompts, whereas baseline methods are “re-optimized” on new prompts. This unequal setup could be misleading. The comparison isn't fair unless both are tested in the same inference-only setting.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to perform red-teaming upon the black-box T2I models where the defense mechanism nor the model weights of the T2I models are unknown. The proposed method (named as RPG-RT) iteratively employs LLMs to modify prompts and leverages feedbacks from T2I models for finetuning the LLMs, in which the LLMs ideally would finally produce the prompts which are able to successfully evade the defense mechanism of T2I models. Basically, given an initial query prompt, the LLM will produce multiple modifications (named as modified prompts) for obtaining the feedbacks/generated images (composed of three main categories, NSFW, SFW, and Reject) from the T2I models, in which these feedbacks are further collected to construct the training set for learning the fine-grained scoring model (where the CLIP embeddings of the generated images are decoupled into the representations of the harmful content and the other innocuous semantics). With leveraging the types of feedbacks and the scoring model to form the preference rules, the binary partial order of modified prompts can be evaluated and further used to finetune the LLM via Direct Preference Optimization (DPO, in which such optimization mechanism is similar to the DRPO in the deepseek). According to the extensive experimental results (conducted upon different defense mechanisms, including detection-based, removal-based, safety-alignment, and their combinations), the proposed RPG-RT is able to strike a better balance between maximizing the harmfulness (i.e. better attack success rate) and maintaining the semantic similarity (with respect to the images generated by the T2I models without any defense mechanisms), compared to various black-box/white-box baselines and state-of-the-arts.\n\nSTRENGTHS AND WEAKNESSES:\n+ Strengths:\n1) The main contribution and the novelty comes from the proposed preference modeling, where the coarse feedbacks (composed of three main categories, NSFW, SFW, and Reject) from the T2I generated images are further used to train a scoring model for providing additional fine-grained preference rules. The design of the four objectives for the scoring model to learn the decomposition of CLIP embeddings into harmful content and the other innocuous semantics is also inspiring.\n2) The significant superior performance across various defense mechanisms and NSFW types compared to both black-box and white-box red-teaming approaches not only demonstrates the efficacy of the proposed method but also its generalizability. Moreover, the effectiveness of the proposed method upon online commercial T2I APIs is also well verified.\n\n- Weaknesses:\n1) As the training of fine-grained scoring model highly relies on having the diverse feedbacks, having the initial modifications of the original prompt which are able to produce diverse feedbacks (composed of both NSFW and SFW images) becomes the key of the entire proposed method. Nevertheless, currently in both main manuscript and the appendix there exists no clear description upon the efficient modifications.\n2) Furthermore, it seems that the proposed method can be seamlessly combined with the idea of FLIRT approach (which adopts in-context learning to modify the prompt) for enhancing the probability of generating diverse feedbacks.\n\nQUESTIONS:\nAlthough the proposed method has the significant novelty upon preference modeling and learning the fine-grained scoring model to drive the LLM finetuning (in order to produce more effective modified prompts to evade the defense mechanism of T2I models which preserving the semantic similarity), and its efficacy and generalizability are well experimentally demonstrated, the authors should provide the corresponding details upon the prompt modifications (which plays the key role to collect diverse feedbacks for learning the scoring model) in the rebuttal.\n\nLIMITATIONS:\nno additional concern upon the potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-06"}