Towards Safe Concept Transfer of Multi-Modal Diffusion via Causal Representation Editing

Authors: peiran  dong
bingjie  wang
song  guo
junxiao  wang
jie  zhang
zicong  hong
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 10:35:35

Abstract: Recent advancements in vision-language-to-image (VL2I) diffusion generation have made significant progress. While generating images from broad vision-language inputs holds promise, it also raises concerns about potential misuse, such as copying artistic styles without permission, which could have legal and social consequences. Therefore, it's crucial to establish governance frameworks to ensure ethical and copyright integrity, especially with widely used diffusion models. To address these issues, researchers have explored various approaches, such as dataset filtering, adversarial perturbations, machine unlearning, and inference-time refusals. However, these methods often lack either scalability or effectiveness. In response, we propose a new framework called causal representation editing (CRE), which extends representation editing from large language models (LLMs) to diffusion-based models. CRE enhances the efficiency and flexibility of safe content generation by intervening at diffusion timesteps causally linked to unsafe concepts. This allows for precise removal of harmful content while preserving acceptable content quality, demonstrating superior effectiveness, precision and scalability compared to existing methods. CRE can handle complex scenarios, including incomplete or blurred representations of unsafe concepts, offering a promising solution to challenges in managing harmful content generation in diffusion-based models.

Main paper: 
 introduction: 
 Expanding on recent progress in text-to-image (T2I) diffusion generation, which is great at making realistic and varied images from written descriptions, researchers are now delving into more advanced vision-language-to-image (VL2I) generation techniques. In these VL2I methods, especially with diffusion models, some use both images of a subject and written descriptions to render the subject in a new context, which is called subject-driven generation [1, 2] . Others take original images and instructions for changes to create altered images, known as image editing [3] . Early approaches either fine-tune models on new images [4, 2, 5, 6, 7] or directly inject image features into the U-Net of diffusion models [8, 9, 1, 10] . However, these methods struggle to jointly model multi-modal inputs and fully leverage the generalization ability of the diffusion model. BLIP-Diffusion [11] is a notable advancement because it creates object representations by blending images with random backgrounds, allowing for the generation of single objects without prior examples. Building on this, Kosmos-G [12] expands to generate multiple objects without examples, using multi-modal large language models (MLLMs) instead of the original CLIP text encoder to encode different kinds of inputs into a single feature set.The advent of a large multi-modal encoder has endowed diffusion models with zero-shot generation, enabling the transfer of concepts (e.g., object or style) to new scenes. However, this unrestricted capability also brings up ethical concerns. There's a risk that people with bad intentions could use zero-shot generation to transfer harmful concepts, like violence or pornography, with just one reference image. Existing efforts in safe generation [13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23] primarily focus on mitigating internal risks stemming from model defects. Diffusion models trained on unedited, large-scale, web-scraped datasets often learn inappropriate and unauthorized knowledge, posing risks when users manipulate textual prompts to "extract" unsafe content.Researchers have pursued various strategies to mitigate the generation of harmful content, encompassing four primary approaches: dataset filtering [13, 14] , adversarial perturbations [15, 16, 17, 18] , machine unlearning [19, 20] , and inference-time refusals [21, 22, 23] . Filtering the dataset involves removing images containing explicit or objectionable content, such as nudity and violence, to ensure the safe generation of diffusion models. However, the advent of zero-shot learning introduces challenges, as it enables diffusion models to transfer unseen objects and styles, complicating copyright protection and security review processes. While adversarial perturbations offer a means to safeguard specific images from manipulation, their efficacy is hampered by the need for training and adaptation to diffusion models with varying parameters. This lack of scalability arises from the requirement to train different adversarial perturbations for each model, despite their structural similarities. Similarly, unlearning-based methods address inherent model defects but fall short in addressing the use of external unsafe images for concept transfer by users. Moreover, existing inference-time refusals predominantly target unsafe concepts describable by language, thus exhibiting limited effectiveness in multi-modal zero-shot generation scenarios. These limitations underscore the need for novel approaches to address the evolving challenges associated with safe content generation in diffusion models.Contributions. To address these challenges, we propose a novel framework called Causal Representation Editing (CRE), which generalizes representation editing for transformer-based Large Language Models (LLMs) to diffusion-based generative models. CRE enhances the efficiency and flexibility of safe concept transfer by introducing a plug-and-play inference-time intervention in diffusion timesteps causally related to unsafe concepts. Our framework comprises two key components: 1) Editing function: We construct steering vectors from examples of unsafe concepts to precisely eliminate them from the original representations. 2) Editing timesteps: We propose "assess-with-exclusion" to identify the causal period for each unsafe concept, during which the unsafe concept appears in the noisy image. This approach reduces editing overhead and avoids ineffective interventions in irrelevant diffusion timesteps, maintaining high editing fidelity. Our contributions include: 1) An early exploration of safe concept transfer in MLLM-enabled diffusion models, with our CRE framework enabling effective inference-time unsafe concept removal. 2) Precise removal of unsafe concepts from noisy images while retaining reasonably generated content, reducing editing overhead by nearly half through fine-grained editing based on the causal period. 3) Comprehensive evaluations demonstrating that CRE surpasses existing methods in effectiveness, preciseness, and scalability, even in complex scenarios involving incomplete or blurred features of unsafe concepts. 
related work: 
 Vision-Language-to-Image Diffusion Models. The fundamental aspect of achieving Vision-Language-to-Image (VL2I) generation lies in training multi-modal encoders responsible for aligning and fusing features from diverse input modalities. BLIP-Diffusion [11] adopts an "align-afterencoding" approach to train its multi-modal encoder. Initially, images and text undergo separate encoding by individual single-modal encoders. Subsequently, following BLIP-2 [24] , the Q-Former architecture aligns visual features with text features. However, this pre-training strategy restricts BLIP-Diffusion to accept only a single image as the input for the visual modality during zero-shot generation. Conversely, Kosmos-G [12] employs an "align-before-encoding" paradigm to train its multi-modal encoder. Kosmos-G pursues the objective of treating images as a foreign language in the image generation process. It incorporates a multi-modal large language model (MLLM) to jointly encode images and text, with each image being embedded into 64 tokens. By utilizing the pre-trained MLLM as an alternative to CLIP encoders [25] , diffusion models gain the capability of zero-shot generation based on multiple input images.Inference-time Safe Concept Transfer. Inference-time safe concept transfer enables generation service providers to dynamically deploy and adjust governance rules, particularly in response to potentially unsafe input from users. Existing approaches typically involve either post-generation detection or in-process adjustment to ensure safety. Rando et al. [21] employ a method where the generated image is projected into a CLIP latent space [25] for comparison against pre-computed embeddings of unsafe concepts, with images surpassing a similarity threshold being flagged as unsafe. However, this approach lacks precision in removing unsafe concepts while preserving overall image quality. Conversely, SLD [22] and ProtoRe [23] integrate safe guidance directly into the diffusion process. These techniques rely on textual descriptions of unsafe concepts, encoded using a CLIP text encoder, to provide negative guidance for denoising. SLD [22] adjusts the diffusion process by modifying the predicted noise from the U-Net, while ProtoRe [23] extracts unsafe concepts from the attention map and refines them. These strategies face limitations when unsafe concepts are not effectively described through natural language.Representation Editing for LLMs. Current studies on Inference-Time Intervention (ITI) [26] in Large Language Models (LLMs) indicate that many LLMs exhibit interpretable directions in their activation spaces, which influence their inference processes. For instance, by introducing carefully designed steering vectors to specific layers for particular tokens, the output can be significantly biased, regardless of the user prompt's topic [27] . Developing a training-free editing method to mitigate unsafe concepts in generative models offers two key advantages. Firstly, it allows the model to retain its strong zero-shot generation ability by preserving the knowledge from pre-training. Secondly, as unsafe concepts may change dynamically due to legal or copyright factors, a plug-and-play editing method can efficiently add or remove types of unsafe concepts under governance. 
threat model: 
 Let I represent the images generated by a diffusion model G θ based on a multi-modal prompt, which includes a text prompt p and a reference image r. The reference image can contain up to K pre-defined unsafe concepts ck , k = 1, 2, • • • , K, such as legally protected concepts. Our goal is to intervene in the image generation process to remove these concepts from I. For example (see Figure 1 ), an adversary might aim to profit by plagiarizing the style of an artistic work, such as a Van Gogh painting. They could use such a painting as a reference image to counterfeit infringing images using VL2I models with zero-shot generation capabilities. Additionally, unwitting users might input images containing unsafe concepts as reference images (objects or styles). These scenarios can lead to significant social problems and economic losses for generation service providers and copyright owners.In contrast to prior studies that primarily address internal generation issues stemming from the diffusion process itself (often due to unedited training data [14] ), our focus is on a new challenge where risks originate from external factors that impact the model. The key distinction between these two scenarios lies in whether users can prompt the generation of unsafe content solely through text inputs. In the case of internal unsafe generation, users might inadvertently generate nudity images by using the term "Four Horsemen" as a text prompt. In contrast, external unsafe generation involves users providing a nude image as a reference to generate more pornographic images. In this latter scenario, the model relies on externally provided visual information to generate new images.Capability: Regulators can define a set of unsafe concepts based on existing laws, regulations, or proposals from copyright owners. Each category of unsafe concepts is accompanied by at least one example image. The VL2I generation service is offered to users through an API. Regulators have the ability to fully control the inference process of the generation model, without any prior information about the user input prompts.Objective: Methods aimed at removing unsafe concepts must be effective and precise. Effectiveness ensures the legality of the generated image, while precision ensures that the reasonable content in the generated image is preserved. It is essential that the service experience of normal users remains unaffected, meaning the system must respond appropriately to requests involving safe concepts.Diffusion. Diffusion-based models, denoted as G θ , progressively refine an initial Gaussian noisy image x T ∼ N (0, I) to generate images x 0 that faithfully represent the user's input prompt p, r. At each timestep t ∈ [T, T -1, • • • , 1], the model estimates the noise εθ to be subtracted from the current noisy image x t . This denoising process can be succinctly expressed as x t-1 = x t -εθ (x t , t, p, r) 2 . The noise estimate εθ (x t , t, p, r) is computed as a linear combination of the unconditional generation ϵ θ (x t , t) and the conditional generation ϵ θ (x t , t, p, r): εθ (x t , t, p, r) = ϵ θ (x t , t) + s g (ϵ θ (x t , t, p, r) -ϵ θ (x t , t)),where the guidance scale s g modulates the impact of the conditioning information, allowing for flexible adjustment of the conditioning strength.Inference-Time Safe Guidance. SLD [22] introduced the first inference-time safety guidance for latent diffusion models to address issues related to inappropriate image generation. This approach extends the generative process by integrating text conditioning using classifier-free guidance and suppressing inappropriate concepts from the output image. Specifically, it introduces a negative concept condition p ′ via the text prompt, following noise estimation principles. Essentially, this method adjusts the unconditional noise prediction towards the user prompt conditioned estimate while simultaneously moving it away from the negative concept conditioned estimate:EQUATION)where µ is concept-dependent guidance scale.SLD exhibits two key limitations. Firstly, its effectiveness relies heavily on text prompts that can precisely describe negative concepts. In contexts where images are included in the prompt for zero-shot generation, SLD's performance is significantly constrained by the lack of precise textual descriptions of the reference image. Secondly, while SLD introduces security guidance, it impacts the experience of benign users. The magnitude of this impact is contingent upon the setting of the guidance scale, necessitating a balance between safety and utility.Previous research on representation engineering [30, 31] has demonstrated that representations in transformer architecture encode intricate semantic details, suggesting that manipulating these representations could be a more effective approach than updating noisy images. In this paper, we explore this idea further by introducing representation editing for large multi-modal diffusion models. Instead of directly guiding safe generation, our method manipulates a small portion of latent representations to steer the denoising process, thereby removing unsafe concepts during inference.Representation Editing Framework. Current research on representation editing [30, 31] mainly focuses on three key components ⟨F, L, P ⟩, where F denotes the editing function, L represents the number of editing layers, and P indicates the editing token position (e.g., the number of prefix or suffix positions to intervene). Recognizing the unique characteristics of diffusion models compared to language generation models, we introduce the timestep dimension T and extend representation editing from discriminative or autoregressive predictive models to diffusion-based generative models.Definition 1. A representation editing framework for diffusion-based generative models can be defined by a tuple ⟨Φ, L, P, T ⟩, which governs an inference-time intervention on the representations computed by the U-Net throughout the diffusion process. This framework comprises four key components:• The editing function Φ : R d → R d , which encompasses operations such as linear combinations, piece-wise operations, and projections.• A set of layers L ⊆ 1, • • • , m in the U-Net where the editing is applied.• A set of input positions P ⊆ 1, • • • , n to which the editing is applied. For text prompts, token locations are typically specified, while mask guidance is commonly used for image prompts.• A set of timesteps T ⊆ 1, • • • , T during which the editing is applied.This framework enables precise control over the editing operation, allowing for targeted interventions to modify the generated outputs as needed. In the following, we introduce our causal representation editing by detailing the four components mentioned above. The U-Net architecture comprises layers broadly classified into convolution layers, self-attention layers, and cross-attention layers. Prior investigations into image editing [32, 33, 34] have elucidated that cross-attention layers facilitate the amalgamation of noisy images and user prompts, yielding fused features. Specifically, The noisy image z t is projected to a query matrix via a linear layer ℓ Q (•), denoted as Q = ℓ Q (z t ). The embedded user prompt {p, r} is projected to a key matrix K = ℓ K (p, r) and a value matrix V = ℓ V (p, r) through linear layers ℓ K (•) and ℓ V (•). The attention representations A are then calculated as follows:EQUATIONVisualizing the attention map Softmax(QK T / √ d) (see Appendix-F), we can observe that concepts from the prompts manifest in the weighted output representations. Consequently, the editing is implemented immediately following the computation of A and influences the representations within each cross-attention layer.Editing Function. The editing function typically receives the original representation (to be edited) and the representation of a specific concept (referred to as a steering vector) as input, aiming to amplify or suppress the concept in the original representation. For instance, ActAdd [27] employs linear addition in the transformer activation layer of a LLM to incorporate the representation of a particular topic (e.g., "wedding") into the original representation. This ensures that regardless of the user prompt, the model's output will be biased towards the wedding topic.In this paper, we construct steering vectors based on examples of unsafe concepts. For the k-th type of unsafe concept, we employ a procedure akin to Equation 3 to create a steering vector containing the unsafe concept. To precisely remove the unsafe concept from the original representations, we project out the component of the representation aligned with the steering vector:Φ(A, Ãk ) = A-k A T Ãk ∥ Ãk ∥ 2 • Ãk , where Ãk = Softmax(Q KT / √ d)• Ṽ = Softmax(ℓ Q (z t )ℓ K (c k ) T / √ d)•ℓ V (c k ).Ablation study in Appendix-E demonstrates the effectiveness of the projection-based representation editing.Although representation editing effectively removes unsafe concepts from generated images, it can hinder generation with benign prompts. As the number of unsafe concepts requiring governance grows, representation editing can significantly degrade image quality. To ensure scalability, we utilize the VL2I generation for data augmentation. Then, we train a discriminator f k : R C×(H×W ) → [0, 1] to evaluate the confidence that an image contains an unsafe concept c k . This discriminator acts as an indicator for determining whether representation editing should be applied, yielding the final editing function:EQUATIONEditing Timesteps. Previous research [35] has demonstrated that the diffusion process generates different elements at various stages. Initially, the diffusion process primarily generates low-frequency features such as layout and object contours, while in later stages, it produces high-frequency features such as color and texture. As unsafe concepts typically represent either concrete objects or abstract styles, their generation is often constrained to specific timesteps and does not encompass the entire diffusion process. Consequently, applying representation editing at each diffusion step would introduce unnecessary computational overhead. For more precise editing, we seek to identify specific diffusion periods during which the unsafe concept c k manifests in the noisy image.Drawing inspiration from causal tracing in knowledge editing [33] , we introduce the causal period for the generation of a given concept in the diffusion process. Definition 2. For a concept c k , a causal period [t s , te] is defined as a period during which there is no shorter diffusion period that yields better generation quality for c k . For any diffusion periodEQUATIONwhere δ k is a small constant.In Equation 5, we use the classification confidence of the discriminator f k for c k to assess its generation safety.Causal Period Search. Previous causal tracing methods employ a "corrupted-with-restoration" approach to identify the most crucial hidden state variable in LLMs when recalling a fact. Given T diffusion rounds, the search space for determining the causal period through sampling is 2 T -1 (excluding the empty set), which is considerably larger than the linear search space in the causal tracing problem seeking a single optimal solution. To tackle this complexity, we propose a heuristic approach named "assess-with-exclusion". We start by considering representation editing at each step of the entire diffusion process, gradually corrupting the process from t = T to t = 1. At each step, we evaluate whether the current corruption significantly impacts the generation of the unsafe concept c k . The confidence gap of the discriminator f k before and after corruption serves as an indicator. If this gap is smaller than the predefined threshold δ k , it suggests that not performing representation editing in the current diffusion step minimally affects the removal of the unsafe concept c k . In such cases, we continue assessing whether the next step is crucial. If the gap exceeds δ k , we identify the current step as the starting step t s of the causal period. Once t s is determined, we conduct a similar backward search process from the last step t = 1 to identify the ending step t e of the causal period. The pseudocode of algorithm for searching t s and t e is present in Appendix-A.Given the Markovian nature of the diffusion process, we first search for t s and exclude [T, t s + 1], followed by the search for t e and exclusion of[t s -1, 1]. Excluding [t s -1, 1]at the second step does not affect the diffusion process before timestep t e . During the search for t s and t e , the search can be terminated when the current timestep is identified as an important step for the first time. This is because once t s is determined, the subsequent adjacent steps are likely to be influenced by it and are also likely to be important steps; similarly, once t e is determined, the preceding diffusion step is likely to be an important step. The computational complexity of Algorithm 1 scales linearly with the total number of diffusion steps T .Inference with CRE. Our proposed causal representation editing, outlined in Appendix-B, comprises two main phases. Phase 1 involves discriminator training and causal period search for each category of unsafe concept, which can be conducted offline. During inference (Phase 2), if the reference image contains unsafe concepts, causal representation editing is applied within the cross-attention layers.Otherwise, the generated content remains faithful to the user-specified prompt without modification. 
experiments: 
 In this section, we empirically evaluate the effectiveness of our proposed Causal Representation Editing (CRE). We use Kosmos-G [12] as the base model for concept transfer, comprising an MLLM as a prompt encoder and stable diffusion as an image decoder. Our approach is benchmarked against several baseline methods: Kosmos-G [12] , Safe Latent Diffusion (SLD) [22] , and ProtoRe [23] .Additionally, we include an intuitive method, Kosmos-G-Neg, which manually adds negative prompts (e.g., "without Van Gogh style") behind the user prompt. To ensure experimental fairness, none of the comparison methods involve any fine-tuning of the generative model. For determining the causal period, we set δ k to 0 for all types of unsafe concepts. We conduct all experiments on an RTX 3090 and an A100-80G.Safe Object Transfer. We first evaluate our approach's performance in safe object transfer through quantitative analysis. We select one class from the ImageNet dataset as an unsafe concept and generate 500 images using the prompt "an image of a [class name]" with Stable Diffusion 2.1 [36] . The guidance scale is set to 9.0. Following previous work [20, 23] , we use a subset of ImageNet with ten easily recognizable classes as the targeted unsafe concepts. Using Kosmos-G, we create prompts in the form "[image 1] with [image 2]" to combine 500 images of each class with other images for object transfer. Here, [image 1] is a portrait, as people are commonly depicted with the ten targeted objects, and [image 2] is selected from the 500 images of each targeted class. We set the guidance scale to 7.5. Finally, we evaluate the top-1 classification accuracy of the transfer results using a pre-trained ResNet-50 ImageNet classifier.In Table 1 , we present quantitative results comparing the accuracy of safe object transfer using Kosmos-G and four safe generation methods. Each class's objects are considered unsafe concepts, and accuracy indicates the ratio of these objects included in the generated image. A lower accuracy signifies better safety in object transfer. The "Kosmos-G" row reports the accuracy of object transfer without any safe generation mechanism, serving as a baseline. Kosmos-G exhibits varying abilities to transfer different objects. Our experiments focus on evaluating whether the safe generation method effectively reduces the generation rate of corresponding unsafe concepts. Existing methods show certain limitations: Kosmos-G-Neg not only fails to achieve safe generation but also increases the probability of generating the corresponding object. We provide a comparison between images generated by Kosmos-G and Kosmos-G-Neg in Appendix-D. This anomaly suggests that the MLLM encoder struggles to interpret the explicit "without" command in the prompt. SLD adjusts the noise prediction of U-Net in diffusion models using auxiliary guidance, making it suitable for localized image detail retouching. However, its effectiveness in object removal appears limited. ProtoRe performs well in most categories but struggles when dealing with large objects (e.g., church) that occupy a significant portion of the image. In contrast, our proposed CRE method demonstrates superior unsafe concept removal capabilities across all categories. In addition, we undertake a test with the COCO-30k dataset with two images (the first one is about cassette and the other one is about Mickey Mouse, which could be found in Figure 2 ). Examples of unsafe concepts removal is shown in Figure 3 . Kosmos-G can combine human portraits with other objects, and can also transfer artistic styles to images of dogs, ducks, glasses, etc. Existing methods are either ineffective when removing these unsafe concepts, or the removal is incomplete and leaves residues. Our approach is able to remove unsafe concepts without leaving any trace.Multiple Style Transfer. To assess the scalability of our approach, we consider scenarios where multiple unsafe concepts may require governance simultaneously. We use Kosmos-G with the same prompts in the form of "[image 1] in the style of [image 2]" to transfer the images in Dreambench to the selected styles, in which [image 1] is an image in Dreambench and [image 2] represents one of the reference images for 4 unsafe styles. However, we replace the prompts with multiple style concepts for SLD ("without the style of Disney, Pencil sketch, Picasso and Van Gogh") and ProtoRe ("the style of Disney, Pencil sketch, Picasso and Van Gogh"). For CRE, we first use the classifier to judge whether the images in the prompts belong to unsafe concepts and which unsafe concept they belong to. If the image belongs to an unsafe style, we activate CRE for the unsafe prompt; If not, the prompt undergoes the normal Kosmos-G process. Finally, we evaluate the top-1 classification accuracy of the transfer results using the classifiers trained above. Table 3 compares the performance difference between targeting a single unsafe concept and targeting multiple unsafe concepts simultaneously. As the number of unsafe concepts increases, the performance of SLD and ProtoRe decreases. This decline is attributed to the length of negative text prompts, which increases with the number of unsafe concepts. Different prompt lengths are encoded into fixed lengths by the encoder, and overly long prompts may lead to information distortion during encoding. While SLD and ProtoRe perform better when dealing with a single unsafe concept, they may not be suitable for tasks requiring simultaneous governance of multiple unsafe concepts in practical scenarios. In contrast, our method exhibits consistent performance, with almost no difference in performance between processing a single unsafe concept and multiple unsafe concepts (the performance gap is less than 0.1%). In particular, when multiple unsafe concepts require supervision, both SLD and ProtoRe tend to retain some additional concepts in the generated image. As illustrated in Figure 4 , the little yellow duck generated by SLD and ProtoRe, after the removal of the Disney style, still retains concepts such as brown hair. A similar issue is observed in the can image after the removal of the Van Gogh style. In contrast, our method effectively generates images free from residual obtrusive concepts following the removal of unsafe styles.Complex scenarios and precise mitigation. Figure 5 (left) illustrates the effectiveness of our method in removing unsafe concepts in complex scenarios. We examine several challenging situations, such as users employing blurred images, portraits in unsafe styles, images taken with mobile phones, cropped images, and overexposed or oversaturated images as reference images for concept transfer.Our method successfully detects and removes unsafe concepts present in these perturbed images. Figure 5 (right) highlights the precision of our method in removing specific unsafe concepts. For instance, when dealing with concepts like Van Gogh and Pencil sketch, our approach preserves reasonable generated content, such as hats and buildings. Unlike rigid blacklists and denial-of-service methods, our approach offers greater flexibility in implementing safe concept transfer.Safe Generation. Table 4 shows the effectiveness of our method in safe generation with the I2P dataset. Compared with previous Representative qualitative results can be found in Appendix-G.  
limitation: 
 We identify two primary shortcomings of CRE from two aspects: effectiveness and overhead. Firstly, the effectiveness of CRE is contingent upon the accuracy of the unsafe concept discriminator, represented by the term ⌊f k (r)⌉ in Equation 4. If the discriminator's accuracy is low, CRE might perform representation editing even for safe prompts. As evidenced in Table 3 and Figure 4 , as the number of unsafe concepts requiring simultaneous governance increases, the adverse impact of inadequate discriminator performance becomes more pronounced. Secondly, in comparison to safe generation methods that utilize fine-tuned diffusion models, representation editing introduces additional inference overhead. Nevertheless, since CRE is only applied in the cross-attention layer during a specific causal period, this additional overhead remains within a tolerable range. For instance, Kosmos-G requires 226 seconds to generate 100 images, and after incorporating CRE, the time increases to 246 seconds, resulting in an average increase of 0.2 seconds per image. 
conclusion: 
 This paper proposes a novel approach, Causal Representation Editing (CRE), to address the challenges of unsafe concept transfer in large multi-modal diffusion models. By leveraging causal periods, CRE allows for precise and efficient removal of unsafe elements from generated images while preserving the integrity and quality of the generated content. Our comprehensive empirical evaluation highlights CRE's superiority over existing methods in both safe object and style transfer tasks. Specifically, CRE effectively reduces the presence of unsafe concepts, demonstrating its robustness across a variety of scenarios. Moreover, CRE exhibits strong scalability, maintaining consistent performance when managing multiple unsafe concepts simultaneously. This scalability is critical for real-world applications where the diversity and complexity of unsafe concepts can vary significantly. The ability of CRE to handle multiple unsafe concepts with minimal performance degradation ensures its applicability in dynamic and complex environments. In addition, CRE underscores the importance of representation-based interventions in generative models. Unlike methods that rely heavily on textual descriptions for unsafe concepts, CRE's representation editing approach proves to be more adaptable and effective, especially in multi-modal zero-shot generation scenarios. Overall, CRE represents a significant advancement in safe concept transfer, offering a robust, scalable, and effective solution for mitigating unsafe content.A Pseudocode of Algorithm 1 while t = T, T -1, • • • , 1 do 3: if f k (G ⟨Φ,L,P,T =[t,t * e ]⟩ (c k )) + δ k ≤ f k (G ⟨Φ,L,P,T =[t * s ,t * e ]⟩ (c k )) then t * s = t 4:else break ▷ Early Exit 5:end if 6: end while7: while t = 1, 2, • • • , t * s do 8: if f k (G ⟨Φ,L,P,T =[t * s ,t]⟩ (c k )) + δ k ≤ f k (G ⟨Φ,L,P,T =[t * s ,t * e ]⟩ (c k )) then t * e = t 9:else break ▷ Early Exit for k = 1, 2, • • • , K do 2:Train Discriminator f k for ck  
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes]Justification: we have reviewed the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: Multi-modal generation models can potentially lead to issues such as illegal content creation, copyright infringement, and other adverse social impacts. Our approach effectively ensures secure generation, safeguarding the rights and interests of both generation service providers and copyright owners.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification: [NA] Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We cite the original papers for all the assets utilized in this research. All open-source data and code will adhere to their respective licenses and will be appropriately labeled with their sources.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: We want to train a classifier to distinguish whether the reference images contain unsafe styles and which unsafe style they belong to (goal 1). Meanwhile, this classifier should also possess a certain level of ability to categorize the style for the generated images (goal 2). To realize the two goals above, we try to construct a diverse dataset empirically and train two classifiers based on the dataset. Finally, we evaluate the image style transfer results with the two classifiers in Table 2 .Step 1 Based on extensive preliminary experiments, we have found that Kosmosg exhibits a stronger ability to transfer style for simple images. We utilized ChatGPT to generate simple prompts, with a requirement for simple form of "single simple object + simple background" like examples in Table 5 . Ultimately, we selected 347 simple and non-repetitive prompts.Step 2 Compared to SD2.1, KosmosG, which is based on SD1.5, generates images with simpler objects and backgrounds using the same prompt. We utilize these 347 prompts to generate images using KosmosG. We set the guidance scale to 7.5. In total, 3470 images are generated.Step 3 To simplify and simulate real-world scenarios, we chose only one image to represent each style (totally four unsafe styles). Leveraging Kosmos-G, we employ the following prompt for style transfer: "[image 1] in the style of [image 2]". Here, [image 1] represents an image generated in Step 2, while [image 2] corresponds to one image of the four selected reference images representing each style. We set the guidance scale to 7.5 and generate 3470 images for each unsafe style, which are subsequently manually screened. As a result, we obtain 2160, 1684, 1641, and 2749 images for Disney, Pencil Sketch, Picasso, and Van Gogh, respectively. Together with the 3470 images from Step 2, these images constituted Style Dataset 1, which demonstrates a high level of diversity for the first four styles mentioned.Step 4 Through experimentation, we discover that by using prompts containing only one image, Kosmos-G could make significant modifications to the original image without losing its original style. Therefore, we also utilize ChatGPT to generate 400 simple prompts, like examples in Table 6 . Specifically, there are 100 prompts with the same prompt "[image 1]", which modify less compared to the other 300 prompts. We set the guidance scale to 7.5. As a result, we obtain Style Dataset 2, which demonstrates moderate diversity compared with Style Dataset 1.Step 5 In diffusion, there is also a function for image-to-image transformation, which leads to a little modification compared to the original image. This allows for slight modifications to be made to the reference image while maintaining the majority of the composition. Examples of such modifications include blurring the original image or altering the texture direction. We employ 399 prompts like samples in Table 7 , which are simply modified from the 400 prompts in Step 4. We generate 399 images for each unsafe style, starting from the 25th to the 10th timesteps (counting from T=50 to 1) with a guidance scale setting of 7.5, resulting in four groups of 399 images each. These images form Style Dataset 3, which closely resemble the corresponding reference images in terms of composition, colors, and other aspects.Step 6 To balance the two goals, we jointly selected images from Style Datasets 1, Style Datasets 2, and Style Datasets 3 to create a training dataset for the classifier. For the "Normal" class, we randomly select 800 images from the images generated in Step 2. Additionally, as the chosen style images in this study include portraits, we select 800 images from the Matting Human Datasets3 to differentiate between style portraits and regular portraits. This combined dataset results in 1600 images for the "Normal" class. We adopt the same image selection strategy for the unsafe style of "Disney", "Pencil Sketch", "Picasso", and "Van Gogh", but different from "Normal". Taking "Disney" as an example, we randomly select 800 diverse images from the "Disney" class in Style Datasets 1. This strategy proves beneficial in achieving goal 2 while also identifying images that closely match the reference four images to a certain extent for Goal 1. From Style Datasets 2, we select all 400 images in the "Disney" class. From Style Datasets 3, we select all 399 images in the "Disney" class, with the original Disney reference image from Step 3. So we get 800 images totally (400+399+1). The first 400 images undergo moderate modifications while preserving their original style (such as adjustments to color, composition, and texture). The latter 400 images closely resemble the images selected in Step 3. As these 800 images undergo limited modifications, we hope that this image selection strategy will assist in effectively identifying images with minimal style modifications, thereby contributing to Goal 1. By following the outlined procedures, we obtain a dataset named Style Dataset Final for classifier training, consisting of 8000 images across five classes. Examples of Style Dataset Final can be found in Figure 6 .We select the pre-trained models ResNet-50 and ViT-base for training with Style Dataset Final. We employ stochastic gradient descent with an initial learning rate of 0.001 and momentum of 0.9. The training process lasts for 50 epochs, and both ResNet and ViT achieve a training accuracy of 100% at the end.Using Kosmos-G, we create prompts in the form of "[image 1] in the style of [image 2]" to transfer the images in Dreambench to the selected styles, in which [image 1] is an image in Dreambench and [image 2] represents one of the reference images for 4 unsafe styles. For both baseline methods (SLD Table 5 : Examples of Simple Prompts "A cat on the mat" "A dog on the rug" "A pig in the mud" "A pen in the jar" "A sun in the sky" "A pig in the pen" "A shoe on the mat" "A dog in the yard" "A cow in the barn" "A star in the sky" Table 6 : Examples of Simple Image Prompts "[image 1] wearing a hat in a picturesque countryside meadow" "[image 1] carving wood in a peaceful workshop" "[image 1] sleeping under the stars" "In a pottery studio, someone sees [image 1]" "On a picturesque farm, there is [image 1]" "In a shopping mall, someone notices [image 1]" "Battling it out on a basketball court, [image 1] dribbles a basketball" "Venturing through a dense forest, [image 1] hikes, exploring nature's wonders" "Exhibiting agility and finesse, [image 1] plays tennis on a clay court" "[image 1]" and ProtoRe) and CRE, we set the guidance scale to 7.5. Finally, we evaluate the top-1 classification accuracy of the transfer results using the classifier (Resnet-50 and ViT-base) trained above.Table 7 : Examples of Simple Image Prompts "[image 1] wearing a hat in a picturesque countryside meadow" "[image 1] carving wood in a peaceful workshop" "[image 1] sleeping under the stars" "[image 1] doing yoga on a tropical island" "Battling it out on a basketball court, [image 1] dribbles a basketball" "Venturing through a dense forest, [image 1] hikes, exploring nature's wonders" "Exhibiting agility and finesse, [image 1] plays tennis on a clay court" "Admiring nature's wonders, [image 1] practices archery in a peaceful forest" "[image 1]" "" Figure 9 : Ablation study on representation editing with projection. Projection significantly enhances the quality of image generation while preserving safe concepts such as backgrounds, resulting in more coherent and contextually accurate visuals. Our approach not only improves the overall fidelity of the generated images but also ensures that the integrity of essential components, such as backgrounds and other safe concepts, is maintained. This method effectively balances creative generation and safety compliance, ensuring that the generated content adheres to desired safety standards without compromising visual quality. NeurIPS Paper Checklist Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. Justification: we provide the information of our computer resources on lines 271-272. A runtime example is present on lines 341-352.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).Here, we omit constant coefficients and remainders for brevity; complete details can be found in[28,29]https://www.kaggle.com/datasets/laurentmih/aisegmentcom-matting-human-datasets 


Decision: Current approaches to address machine safety, including dataset filtering and machine unlearning, often lack scalability or effectiveness. The proposed framework, causal representation editing (CRE), argues that it offers a more effective and scalable solution by targeting unsafe content during diffusion timesteps, ensuring safe content generation while maintaining quality.

The reviewers think its strengths may include:

- Innovative causal representation editing (4T2N, Qo9h) 
- while having excellent performance (4T2N, MP63)
- Detailed experimental validation (4T2N, 33ja)
- Well-written and clear (33ja, Qo9h)

Initially, its weaknesses were:
- A heavy reliance on the unsafe concept discriminator (4T2N) and inefficiency (MP63). However, the rebuttal appears to have successfully addressed these concerns by providing experimental evidence.

Overall, the author rebuttals have garnered positive feedback. After reviewing the rebuttal threads, it appears that the major issues have been adequately addressed, leading to a recommendation for the paper’s acceptance.

Reviews: review: paper_summary: paper_summary: This paper proposes a framework called Causal Representation Editing (CRE) to address the ethical and copyright concerns in vision-language-to-image (VL2I) diffusion models. CRE enhances safe content generation by intervening at diffusion timesteps linked to unsafe concepts, effectively removing harmful content while preserving quality. The approach is more effective, precise, and scalable than existing methods. CRE also offers a solution for complex scenarios, providing insights into managing harmful content in diffusion-based models. Comprehensive evaluations demonstrate CRE's superiority in various benchmarks.
strength_weakness: strength_weakness: 1. Dependency on Discriminator Accuracy: The effectiveness of CRE is heavily reliant on the accuracy of the unsafe concept discriminator. If the discriminator fails to accurately identify unsafe concepts, CRE might incorrectly edit safe content, leading to unnecessary modifications and potentially impacting the quality of the generated images.

2. Additional Inference Overhead: Compared to safe generation methods that use fine-tuned diffusion models, CRE introduces additional inference overhead. This can lead to increased computation time and resource usage, which might be a significant drawback in practical applications where efficiency is crucial.

3. Limited Applicability to Complex Unsafe Concepts: While CRE is effective for well-defined unsafe concepts, its performance might degrade when dealing with more complex or nuanced unsafe concepts that are difficult to categorize or describe. This limitation restricts its applicability in real-world scenarios where unsafe content is not easily defined.
questions: questions: 1. Could you explain how your method handles cases where the discriminator accuracy is low, and CRE might perform representation editing even for safe prompts? Are there any mechanisms to mitigate this issue?

2. The paper mentions that the additional overhead introduced by representation editing is within a tolerable range. Could you provide more details on how this overhead might scale with larger datasets or more complex models?
limitations: limitations: see weaknesses
paper_summary: paper_summary: The paper introduces Causal Representation Editing (CRE) for vision-language-to-image (VL2I) models to prevent undesirable concept generation. CRE prevents unsafe image generation through the following process:
1. Using a discriminator, it detects whether an unsafe concept is present in the user prompt.
2. If the discriminator identifies an unsafe concept, it projects out the representation $\tilde{A}$ corresponding to the unsafe concept from the existing attention representation $A$.
3. The projection out is not performed at every diffusion timestep but within the specific interval $[t_s, t_e]$ identified as causally influential for unsafe concept generation through assess-with-exclusion.
The authors demonstrated the efficacy of CRE through object, style, and multiple-style censoring.
strength_weakness: strength_weakness: 1. Using an external discriminator is highly inefficient.

For example, let's say we need to remove K unsafe concepts. Using an external discriminator adds two types of overhead: (1) training a discriminator for each concept and (2) performing discriminator inference for each concept. While (1) can be somewhat mitigated with models like CLIP that have good zero-shot classification performance, (2) remains problematic. In my opinion, the advantage of test-time guidance methods like ActAdd and SLD is that they don't require training. The use of a discriminator diminishes the benefits they offer over inference-time refusal or machine unlearning methods.

2. Inadequate experiments

The current paper lacks significant experiments needed in the safety unlearning field. 

There are no experiments showing the impact of CRE on image fidelity. While a perfect discriminator would be ideal, real-world discriminators are not perfect. Can you report the COCO 30k dataset FID? Considering limited computational resources, knowing the results when censoring "cassette player" and "Disney" would be sufficient.

This paper uses a T2I diffusion model as an image decoder, so it could compare machine unlearning techniques for T2I diffusion models as baselines. Are there any comparisons with safety unlearning techniques for T2I diffusion models? State-of-the-art methods include SPM [1] and MACE [2], but considering the submission time, comparing with CA [3] and UCE [4] would suffice.

There are no experiments on NSFW concept removal. I'm curious if CRE can robustly prevent adversarial prompt attacks like Ring-A-Bell [5] or concept inversion [6] through red teaming.

3. Methodological novelty (minor)
Directly using the representation editing method ActAdd from LLMs is relatively less novel.

---

[1] : One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications, https://arxiv.org/abs/2312.16145

[2] : MACE: Mass Concept Erasure in Diffusion Models, https://arxiv.org/abs/2403.06135

[3] : Ablating Concepts in Text-to-Image Diffusion Models, https://arxiv.org/abs/2303.13516

[4] : Unified Concept Editing in Diffusion Models, https://arxiv.org/abs/2308.14761

[5] : Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?, https://arxiv.org/abs/2310.10012

[6] : Circumventing Concept Erasure Methods For Text-to-Image Generative Models, https://arxiv.org/abs/2308.01508
questions: questions: Q1: Can you provide the specific values of $[t_s, t_e]$ obtained through assess-with-exclusion? The current paper includes the method but lacks actual causal tracing results or precise values for each concept.
limitations: limitations: The author describes the limitations caused by discriminator.
paper_summary: paper_summary: The authors studied an important problem about misuse of Text-to-image (T2I) diffusion model, leading to legal and social issues. They propose a causal representation editing (CRE) method, extends representation editing from large language models to diffusion-based models. CRE improves safe content generation by intervening at diffusion timesteps linked to unsafe concepts, allowing precise removal of harmful content while preserving quality. The extensive experiment results have shown the effectiveness of their model.
strength_weakness: strength_weakness: 1. The authors may need the experiment to compare the inference time with other Inference-time Safe Concept Transfer models to further verify the efficiency of their model.
2. It’s better to provide more related works about representation editing in the related works.
3. The contribution of the design of selecting editing timesteps to performance of the model is unclear compared with random selection.
questions: questions: In ProtoRe, they also have similar editing function in their Eq (7). The contribution in this paper is about adding the term related to discriminator. Can authors explain more about the contribution compared with ProtoRe?
limitations: limitations: Can be found in weaknesses part.
paper_summary: paper_summary: The paper addresses the potential misuse of VL2I diffusion models, such as copying artistic styles without permission, which could lead to legal and social issues. The paper introduces an early exploration of safe concept transfer in MLLM-enabled diffusion models using a novel framework called Causal Representation Editing (CRE). CRE allows for effective inference-time removal of unsafe concepts from noisy images while retaining other generated content. This is achieved through fine-grained editing based on identifying the causal period during which unsafe concepts appear. The approach reduces the editing overhead by nearly half compared to existing methods. The paper presents extensive evaluations demonstrating that CRE outperforms existing methods in terms of effectiveness, precision, and scalability, even in complex scenarios with incomplete or blurred features of unsafe concepts.
strength_weakness: strength_weakness: 1. The paper lacks a user study or feedback from practitioners who might use the CRE framework in their workflows.
2. Although the paper provides extensive evaluations, it could benefit from a more diverse set of evaluation metrics and benchmarks. The effectiveness, precision, and scalability are well-documented, but additional metrics such as computational efficiency, user satisfaction, or real-world applicability could provide a more comprehensive assessment of the method's performance. Furthermore, comparisons with a wider range of existing techniques would strengthen the argument for the superiority of the proposed approach.
3. While the paper highlights the importance of safe content generation, it could delve deeper into the ethical and legal implications of using CRE. For example, the criteria for defining "unsafe concepts" are not fully explored, which could lead to subjective or inconsistent applications of the method. Moreover, the potential misuse of the technology for censorship or manipulation of content raises important ethical concerns that the paper does not adequately address. A more detailed discussion on these aspects would enhance the comprehensiveness of the research.
questions: questions: Please see weaknesses.
limitations: limitations: The authors have adequately described the limitations and potential negative societal impact of their work.

Comments: comment: I acknowledge having read the authors' rebuttal. My overall assessment of the paper remains unchanged, and I continue to support my current rating.
We have deleted the anonymous PDF. We sincerely apologize for this oversight and extend our gratitude to all the reviewers and area chairs for their efforts.
The accident has been reported to SAC, but they have not responded yet. We hope reviewers refrain from opening the link and are not affected by it in their assessments.

Unless SAC's explicit action, the author-reviewer discussion will continue.
I hope that the minor mistake of uploading the link does not significantly impact the evaluation of this research. I appreciate your sincere efforts.

I had three major concerns:

> The use of the discriminator seems inefficient.

In my view, it still appears inefficient to use the discriminator in the representation. However, after comparing the baseline performance below, I now find this level of inefficiency to be understandable.

> I was concerned that it might excessively degrade the model's capability.

Regarding the rebuttal on FID and 4T2N, it does seem to impact adherence to the text description compared to existing methods. Nevertheless, this concern has been alleviated with the improvement of the baseline.

> Missing baseline (especially the unlearning baseline).

While UCE is not state-of-the-art, the results are convincing enough for me. This research holds significant value, particularly because it offers strengths in safety compared to unlearning methods.

Overall, my concerns have been partially addressed. Therefore, I will raise the score to 6.
We apologize for including the anonymous link in the rebuttal process. We confirm that the PDF contained in the link does not disclose any personal information and adheres to the double-blind review mechanism. The PDF contains supplementary experimental images addressing the reviewer's concerns. If this has caused any adverse effects, we sincerely apologize and request the Area Chair to remove the link.
We apologize for any inconvenience caused. Our aim is to address the reviewer's concerns as thoroughly as possible within the constraints of a double-blind review. The anonymous link provided contains only an anonymized PDF, ensuring no personal information is disclosed. The images included in the PDF are supplementary experiments intended to address the reviewer's concerns. We regret any adverse effects this may have caused.
Reviewers, please refrain from opening the link. I will inform you of any updates. 

In the meantime, feel free to continue the discussion as usual.
According to the Author's Guidelines (sent by email), 
> 4. All the texts you post (rebuttal, discussion and PDF) should not contain any links to external pages.

This accident will be redirected to SAC.