Title: Towards Safe Concept Transfer of Multi-Modal Diffusion via Causal Representation Editing
Venue: NeurIPS 2024
Accepted: yes
Keywords: Diffusion based models

Safe generation

Concept Transfer

Representation editing
Abstract: Recent advancements in vision-language-to-image (VL2I) diffusion generation have made significant progress. While generating images from broad vision-language inputs holds promise, it also raises concerns about potential misuse, such as copying artistic styles without permission, which could have legal and social consequences. Therefore, it's crucial to establish governance frameworks to ensure ethical and copyright integrity, especially with widely used diffusion models. To address these issues, researchers have explored various approaches, such as dataset filtering, adversarial perturbations, machine unlearning, and inference-time refusals. However, these methods often lack either scalability or effectiveness. In response, we propose a new framework called causal representation editing (CRE), which extends representation editing from large language models (LLMs) to diffusion-based models. CRE enhances the efficiency and flexibility of safe content generation by intervening at diffusion timesteps causally linked to unsafe concepts. This allows for precise removal of harmful content while preserving acceptable content quality, demonstrating superior effectiveness, precision and scalability compared to existing methods. CRE can handle complex scenarios, including incomplete or blurred representations of unsafe concepts, offering a promising solution to challenges in managing harmful content generation in diffusion-based models.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the potential misuse of VL2I diffusion models, such as copying artistic styles without permission, which could lead to legal and social issues. The paper introduces an early exploration of safe concept transfer in MLLM-enabled diffusion models using a novel framework called Causal Representation Editing (CRE). CRE allows for effective inference-time removal of unsafe concepts from noisy images while retaining other generated content. This is achieved through fine-grained editing based on identifying the causal period during which unsafe concepts appear. The approach reduces the editing overhead by nearly half compared to existing methods. The paper presents extensive evaluations demonstrating that CRE outperforms existing methods in terms of effectiveness, precision, and scalability, even in complex scenarios with incomplete or blurred features of unsafe concepts.\n\nSTRENGTHS:\n1. The paper introduces an innovative approach to address the emerging concern of safe concept transfer in multimodal diffusion models. The originality is evident in its novel application of CRE to selectively remove unsafe concepts from generated images.\n2. The research is methodologically sound, with comprehensive evaluations and rigorous testing of the proposed CRE framework.\n3. The paper is well-written and clearly structured, making it accessible to both experts and those new to the field.\n4. The significance of this work lies in its potential to influence the future development of safe AI-generated content. As AI models become increasingly integrated into creative industries, the ability to safely and efficiently remove unsafe or unwanted concepts is critical.\n\nWEAKNESSES:\n1. The paper lacks a user study or feedback from practitioners who might use the CRE framework in their workflows.\n2. Although the paper provides extensive evaluations, it could benefit from a more diverse set of evaluation metrics and benchmarks. The effectiveness, precision, and scalability are well-documented, but additional metrics such as computational efficiency, user satisfaction, or real-world applicability could provide a more comprehensive assessment of the method's performance. Furthermore, comparisons with a wider range of existing techniques would strengthen the argument for the superiority of the proposed approach.\n3. While the paper highlights the importance of safe content generation, it could delve deeper into the ethical and legal implications of using CRE. For example, the criteria for defining \"unsafe concepts\" are not fully explored, which could lead to subjective or inconsistent applications of the method. Moreover, the potential misuse of the technology for censorship or manipulation of content raises important ethical concerns that the paper does not adequately address. A more detailed discussion on these aspects would enhance the comprehensiveness of the research.\n\nQUESTIONS:\nPlease see weaknesses.\n\nLIMITATIONS:\nThe authors have adequately described the limitations and potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-07"}

comments: SUMMARY:
The authors studied an important problem about misuse of Text-to-image (T2I) diffusion model, leading to legal and social issues. They propose a causal representation editing (CRE) method, extends representation editing from large language models to diffusion-based models. CRE improves safe content generation by intervening at diffusion timesteps linked to unsafe concepts, allowing precise removal of harmful content while preserving quality. The extensive experiment results have shown the effectiveness of their model.

STRENGTHS:
1. The authors focus on an important problem about misuse of Text-to-image (T2I) diffusion model, leading to legal and social issues. 
2. The writing is clear and easy to follow.
3. The authors conduct extensive experiments to verify the effectiveness of their proposed method, CRE.

WEAKNESSES:
1. The authors may need the experiment to compare the inference time with other Inference-time Safe Concept Transfer models to further verify the efficiency of their model.
2. It’s better to provide more related works about representation editing in the related works.
3. The contribution of the design of selecting editing timesteps to performance of the model is unclear compared with random selection.

QUESTIONS:
In ProtoRe, they also have similar editing function in their Eq (7). The contribution in this paper is about adding the term related to discriminator. Can authors explain more about the contribution compared with ProtoRe?

LIMITATIONS:
Can be found in weaknesses part.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Causal Representation Editing (CRE) for vision-language-to-image (VL2I) models to prevent undesirable concept generation. CRE prevents unsafe image generation through the following process:\n1. Using a discriminator, it detects whether an unsafe concept is present in the user prompt.\n2. If the discriminator identifies an unsafe concept, it projects out the representation $\\tilde{A}$ corresponding to the unsafe concept from the existing attention representation $A$.\n3. The projection out is not performed at every diffusion timestep but within the specific interval $[t_s, t_e]$ identified as causally influential for unsafe concept generation through assess-with-exclusion.\nThe authors demonstrated the efficacy of CRE through object, style, and multiple-style censoring.\n\nSTRENGTHS:\n(Here, I will refer to \"preventing unsafe image generation\" as censoring.)\n\n1. Ability to Handle Unsafe Image Prompts\nMost existing safety methods are designed for T2I models and focus on preventing the generation of unsafe text concepts. The method proposed in this paper has the advantage of being applicable regardless of the prompt domain, whether text or image.\n\n2. Censoring Performance\nTable 1 shows that CRE performs better in censoring compared to existing methods.\n\nWEAKNESSES:\n1. Using an external discriminator is highly inefficient.\n\nFor example, let's say we need to remove K unsafe concepts. Using an external discriminator adds two types of overhead: (1) training a discriminator for each concept and (2) performing discriminator inference for each concept. While (1) can be somewhat mitigated with models like CLIP that have good zero-shot classification performance, (2) remains problematic. In my opinion, the advantage of test-time guidance methods like ActAdd and SLD is that they don't require training. The use of a discriminator diminishes the benefits they offer over inference-time refusal or machine unlearning methods.\n\n2. Inadequate experiments\n\nThe current paper lacks significant experiments needed in the safety unlearning field. \n\nThere are no experiments showing the impact of CRE on image fidelity. While a perfect discriminator would be ideal, real-world discriminators are not perfect. Can you report the COCO 30k dataset FID? Considering limited computational resources, knowing the results when censoring \"cassette player\" and \"Disney\" would be sufficient.\n\nThis paper uses a T2I diffusion model as an image decoder, so it could compare machine unlearning techniques for T2I diffusion models as baselines. Are there any comparisons with safety unlearning techniques for T2I diffusion models? State-of-the-art methods include SPM [1] and MACE [2], but considering the submission time, comparing with CA [3] and UCE [4] would suffice.\n\nThere are no experiments on NSFW concept removal. I'm curious if CRE can robustly prevent adversarial prompt attacks like Ring-A-Bell [5] or concept inversion [6] through red teaming.\n\n3. Methodological novelty (minor)\nDirectly using the representation editing method ActAdd from LLMs is relatively less novel.\n\n---\n\n[1] : One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications, https://arxiv.org/abs/2312.16145\n\n[2] : MACE: Mass Concept Erasure in Diffusion Models, https://arxiv.org/abs/2403.06135\n\n[3] : Ablating Concepts in Text-to-Image Diffusion Models, https://arxiv.org/abs/2303.13516\n\n[4] : Unified Concept Editing in Diffusion Models, https://arxiv.org/abs/2308.14761\n\n[5] : Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?, https://arxiv.org/abs/2310.10012\n\n[6] : Circumventing Concept Erasure Methods For Text-to-Image Generative Models, https://arxiv.org/abs/2308.01508\n\nQUESTIONS:\nQ1: Can you provide the specific values of $[t_s, t_e]$ obtained through assess-with-exclusion? The current paper includes the method but lacks actual causal tracing results or precise values for each concept.\n\nLIMITATIONS:\nThe author describes the limitations caused by discriminator.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a framework called Causal Representation Editing (CRE) to address the ethical and copyright concerns in vision-language-to-image (VL2I) diffusion models. CRE enhances safe content generation by intervening at diffusion timesteps linked to unsafe concepts, effectively removing harmful content while preserving quality. The approach is more effective, precise, and scalable than existing methods. CRE also offers a solution for complex scenarios, providing insights into managing harmful content in diffusion-based models. Comprehensive evaluations demonstrate CRE's superiority in various benchmarks.\n\nSTRENGTHS:\n1. Innovative Causal Representation Editing Framework: The paper introduces a novel framework called Causal Representation Editing (CRE) that effectively extends representation editing techniques from language models to diffusion-based generative models. This framework enhances the efficiency and flexibility of safe content generation, providing a new approach to addressing ethical and copyright concerns in vision-language-to-image (VL2I) models.\n\n2. Comprehensive Handling of Unsafe Concepts: CRE demonstrates superior effectiveness, precision, and scalability compared to existing methods. It can handle complex scenarios, including incomplete or blurred representations of unsafe concepts, ensuring that harmful content is precisely removed while maintaining acceptable content quality.\n\n3. Detailed Experimental Validation: The paper provides extensive evaluations and experiments to validate the effectiveness of the proposed method. The results show that CRE surpasses existing methods in various benchmarks, highlighting its potential for managing harmful content generation in diffusion-based models.\n\nWEAKNESSES:\n1. Dependency on Discriminator Accuracy: The effectiveness of CRE is heavily reliant on the accuracy of the unsafe concept discriminator. If the discriminator fails to accurately identify unsafe concepts, CRE might incorrectly edit safe content, leading to unnecessary modifications and potentially impacting the quality of the generated images.\n\n2. Additional Inference Overhead: Compared to safe generation methods that use fine-tuned diffusion models, CRE introduces additional inference overhead. This can lead to increased computation time and resource usage, which might be a significant drawback in practical applications where efficiency is crucial.\n\n3. Limited Applicability to Complex Unsafe Concepts: While CRE is effective for well-defined unsafe concepts, its performance might degrade when dealing with more complex or nuanced unsafe concepts that are difficult to categorize or describe. This limitation restricts its applicability in real-world scenarios where unsafe content is not easily defined.\n\nQUESTIONS:\n1. Could you explain how your method handles cases where the discriminator accuracy is low, and CRE might perform representation editing even for safe prompts? Are there any mechanisms to mitigate this issue?\n\n2. The paper mentions that the additional overhead introduced by representation editing is within a tolerable range. Could you provide more details on how this overhead might scale with larger datasets or more complex models?\n\nLIMITATIONS:\nsee weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}