Paper: Circumventing Concept Erasure Methods For Text-To-Image Generative Models (ICLR-2024)

Reviewer comment (Novelty):
The CI attack method is straightforward. Basically, this paper uses the existing methods, such as Textual Inversion (Gal et al., 2023), for the Concept Inversion Method. The CI methods lack insightful and novel design.

Author response:
While we mostly utilize Textual Inversion for Concept Inversion, we do think the fact that an existing and straightforward method can circumvent current concept erasure methods emphasizes the vulnerability of post hoc erasure methods. 
We also want to note that vanilla Textual Inversion does not work for SLD and NP, and we had to invent two novel variants that can search for the appropriate soft embeddings in these methods, where Textual Inversion fails. In particular, section 4.2.3 provides the description of Concept Inversion for NP, and section C.4 in the Appendix contains the pseudocode for Concept Inversion for SLD.

Reviewer comment (Experiments):
The authors propose their attack methods under relatively lenient conditions, as evidenced by assumptions (1) and (2) in the PRELIMINARIES section. These assumptions are closer to a white-box setting. However, for most MLaaS (Machine Learning as a Service) platforms, attackers would not have access to the internal details of the model, let alone the ability to add new tokens to its dictionary for executing Concept Inversion attacks.

Author response:
We agree that our attack scenario is closer to the white-box setting. We would like to point out that we want to use Concept Inversion attacks to better understand current erasure methods through their failure cases. Our results on Concept Inversion and the transferability of the learned embeddings demonstrate that current concept erasure methods are not fully excising the targeted concept but rather performing some input filtering. We have updated the main draft to clarify this better.

Reviewer comment (Evaluation):
The TRANSFERABILITY method outlined by the authors in Appendix 4.4 also seems to be non-transferable to a black-box environment due to the inability of the attacker to modify the model's internal dictionary. Therefore, the current CI attacks appear to still have limited effectiveness in a black-box setting.

Author response:
We agree that our attack scenario is closer to the white-box setting. We would like to point out that we want to use Concept Inversion attacks to better understand current erasure methods through their failure cases. Our results on Concept Inversion and the transferability of the learned embeddings demonstrate that current concept erasure methods are not fully excising the targeted concept but rather performing some input filtering. We have updated the main draft to clarify this better.

Reviewer comment (Experiments):
There is curiosity about the behavior of the learned concept placeholder embedding in comparison to the embedding of the original concept name. This comparison could provide valuable insights into the effectiveness of the proposed approach.

Author response:
**There is curiosity about the behavior of the learned concept placeholder embedding in comparison to the embedding of the original concept name. This comparison could provide valuable insights into the effectiveness of the proposed approach.**

We agree this is an important question. While optimizing for concept placeholder embedding which is very close to the initial concept does result in deteriorated performance, the placeholder embeddings further away from the original word behave as well within a sentence as the original word embeddings (Please see Figure 7). We leave further investigation of the properties with these embeddings for future work.

Reviewer comment (Presentation):
Typo: inconsistency in the caption of Figure 7, which does not align with the order of the figure.

Author response:
**Typo: inconsistency in the caption of Figure 7, which does not align with the order of the figure.**

We have updated the caption to better align with the figure.

Reviewer comment (Writing):
Confusion regarding Section 4.4 — how to understand ‘remapping the concept in token space’? Since the original text encoder and the embedding of the original text tokens are not updated, right?

Author response:
**How to understand ‘remapping the concept in token space’? Since the original text encoder and the embedding of the original text tokens are not updated, right?**

You are absolutely correct that the original text encoder and the embedding of the original text tokens are not updated. What we are trying to convey is that current concept erasure methods are suppressing the generation of the targeted concept when conditioned on the input embeddings corresponding to the concept text, i.e. Van Gogh. However, there exist word embeddings that trigger the generation of the erased concept in SD 1.4 that also work on the erased model. This means that some degree of input-filtering is occurring. Moreover, since the behavior of the post-erasure model is changed due to fine-tuning the UNet or interference of the generation process, we believe that the erased concept is also mapped to some input embeddings that do not necessarily transfer to the pre-erasure model. We will add further clarifications in the revised manuscript.

Reviewer comment (Experiments):
Confusion regarding Section 4.4 — how the authors conducted experiments to demonstrate the editability, as the left figure of Figure 7 does not appear to clearly showcase this aspect?

Author response:
**how the authors conducted experiments to demonstrate the editability, as the left figure of Figure 7 does not appear to clearly showcase this aspect?**

Following Gal et al. 2022, we conducted the editability experiment by first performing Concept Inversion on the erased model, and then generating images using prompts containing the special learned token (e.g. “A painting of the beach in the style of <special-token>"). We then measure the CLIP [1] similarity between the generated images and the text prompt without the special token (e.g.  “A painting of the beach”). This allows us to evaluate whether the learned embeddings can be used in various contexts.

*__Reference:__*

[1] Radford, Alec, et al. "Learning transferable visual models from natural language supervision." International Conference on Machine Learning, 2021.

Reviewer comment (Evaluation):
The paper evaluates black-box methods using a white-box approach. For example, Kumari et al. [1] already acknowledged their limitation in the white-box setting.

Author response:
We fully acknowledge the transparency by Kumari et al. [1] in the white-box setting. Nevertheless, this limitation has not been clearly stated in the remaining concept erasure methods. Moreover, while Kumari et al. mention that their method does not prevent the re-introduction of the erased concept in the white-box setting, we are the first to point out that it is their method (and 6 other concept erasure methods covered in our work) mostly provide no protection against more sophisticated inputs.

Reviewer comment (Experiments):
Clarification is needed regarding the attack scenarios presented. Specifically, it is unclear why an adversary with white-box access would seek to bypass a black-box model.

Author response:
Through an adversary with white-box access, our work aims to understand better the failure cases of concept erasure methods. Our experiments reveal that the seemingly erased concepts still persist within the model post-erasure. Hence, a malicious user can exploit this false sense of security and trigger the erased model to generate sensitive content.

Reviewer comment (Theory):
Assumption 5 in section 3 appears inconsistent with a white-box setting, particularly when considering Assumption 4, which grants the adversary computational resources. It raises the question of why an adversary could not alter the weights of the "erased" model under these conditions.

Author response:
We would like to clarify that we are not merely trying to generate erased concepts from sanitized models. From a scientific perspective, our goal in this paper is to investigate two questions:

 (1) does the erased concept still persist in the model post-erasure? 

 (2) are current concept erasure methods performing input filtering?

While fine-tuning the entire model on samples containing the erased concept can re-introduce that concept back into the model, it does not help us answer the previously mentioned questions. Hence, we want to restrict the adversary from altering the weights of the erased model.

Reviewer comment (Evaluation):
Regarding the practicality of the proposed attack method, it is important to note that widely-used public services such as Stability AI and DALL-E 3 do not currently accept text-embedding inputs to my knowledge. This may render the evaluation less critical, especially considering that existing methods effectively erase explicitly mentioned words.

Author response:
We agree with the reviewer regarding the practicality of our proposed attack method towards closed models which are available only through API access. However, the main goal of our work is to take a step toward understanding current concept erasure methods and their vulnerabilities. We hope that our work can shed light on the limitations of these methods to the research community, which can encourage future works to focus more on practical attacks and appropriate defenses. Moreover, our experiments with white-box access demonstrate that (1) current concept erasure methods are susceptible to soft-embedding attacks, and (2) there exists soft-embeddings that can trigger both the original (unerased) model and the fine-tuned (erased) model to generate the target concepts. One possible attack under the black-box assumption is exhaustively performing discrete optimization (black-box) attacks on the unerased model until we find the hard prompts that “jailbreak” the erased model. We leave further explorations of black-box attacks for future work.