Unmasking the Veil: An Investigation into Concept Ablation for Privacy and Copyright Protection in Images
Authors: Shivank Garg
Manyana Tiwari
Venue: Accepted by TMLR ()

Abstract: In this paper, we extend the study of concept ablation within pre-trained models as introduced in 'Ablating Concepts in Text-to-Image Diffusion Models' by $\citep{Kumari2022}$.  Our work focuses on reproducing the results achieved by the different variants of concept ablation proposed through predefined metrics. We also introduce a novel variant of concept ablation—trademark ablation. This variant combines the principles of memorization and instance ablation to tackle the nuanced influence of proprietary or branded elements in model outputs. Further, our research contributions include an observational analysis of the model's limitations. Moreover, we investigate the model's behavior in response to ablation leakage-inducing prompts, which aim to indirectly ablate concepts, revealing insights into the model's resilience and adaptability. We also observe the model's performance degradation on images generated by concepts far from its target ablation concept, which is documented in the appendix.

Published in Transactions on Machine Learning Research (06/2024)
Unmasking the Veil: An Investigation into Concept Ablation for Privacy and Copyright Protection in Images
Shivank Gargl
shivank @0mff..ütr.ac..
Deportment of Data Science and Artificial Intelligence
II Roorkee
Manyana Tiwarf
m Mivarimmaa wwt..tttt.....mmm.mammmmmmomm. witr.ac..i.........
Deportment of Mathematics
IIT Roorkee
Reviewed on OpenReview: https: ope.rvveeeer...e.e.emeeeee net/ pet//frum? id= TYYApLajal
Abstract
In this paper, extend the study of concept ablation within pre-trained models as intro- duced in 'Ablating Concepts in Text-to- Text-to-Image Diffusion Models' by (Kumari et al.] 2022). Our work focuses on reproducing the results achieved by the different variants of concept ablation proposed and validated through predefined metrics. We also introduce a novel vari- ant concept ablation, namely 'trademark ablation'. This variant combines the principles of memorization and instance ablation to tackle the muanced influence of proprictary or branded elements in model outputs. Further, our research contributions include an obser- vational analysis of the model's limitations. Morcover, we investigate the model's behavior in response to ablation leakage-inducing prompts, which aim to indirectly ablate concepts, revealing insights into the model's 's resilience and adaptability. We also observe the model's performance degradation on images generated by concepts far from its target ablation con- cept, documented in the appondix.
1 Introduction
With the advent of sophisticated large-scale text-to-image diffusion models, there have been significant improvements in the quality and compositional ability of the generated images. This is attributed to the uti- lization of extensive training data, which often contains copyrighted material and licensed images. Therefore, these models have the ability to replicate the styles of various artists or memorize exact training samples. Hence, there is an imminent need to develop techniques to unlearn(ablate) the effect of a specific 'target concept" from the output generated by the model.
Concept ablation is defined as the task of preventing the generation of the desired image corresponding to a given target concept that needs to be ablated. This is done by modifying the generated images to match the image distribution corresponding to more generalized anchor text prompt instead of the specific target prompt. A simple approach maximizes the diffusion training loss for the target prompt-image pairs. However, this produces poor relevance to the 'anchor concept" and results on the generated images.
The recent paper 'Ablating Concepts in Text-to-Image Diffusion Models' Kumari et al. (2023 provided an efficient method for ablating concepts from pre-trained model without retraining the model from scratch. Their work aims to improve the degree of retention of the anchor and surrounding concepts while ablating largel concept.
Through this paper, we have attempted to reproduce the original findings and produce new insights and metrics to analyze the accuracy of the proposed method. We have introduced a new variant of ablation,
"Equal Contribution

---

Published in Transactions on Machine Learning Research (06/2024)
namely trademark ablation. We discuss the results and motivation behind this variant in later sections. We also study the robustness of the method in circumventing leakage-inducing prompis. Our observations also showcase that the model performs sub-optimally on ablating concepts far from the anchor concepts.
Paper Outline Through Section 2 we build a background of the working the proposed ablation models.
We provide details of training and baseline metrics in Section 3 Further, in Section we expand on the analysis of our variation and limitations of the proposed algorithm.
2 Model breakdown and Intuition
The major contribution the original paper is significantly reducing the latency ablate specific target prompt under 5 minutes. For this, the authors have utilized loss function based on minimizing the KL Divergence between the distribution of the target concept image (to be ablated) and the distribution of the anchor concept images Kumari et al. 2022
2.1 Methods
The authors have introduced two different methods for ablation, namely basse and model-based abla- tion. After evaluation both methods, it was observed that the noise-based method achieves comparable performance to the model-based ablation in optimal cases, and sub-par in some cases. It also has slower convergence rate overall, thus being an inferior method. Hence, all results in this study are computed solely on the model-based ablation method.
Breaking down the notations used in the following equations:
DEL the KL Divergence, which we aim to optimize.
((((o...)))))) is the distribution of the target concept conditioned on anchor concept
(o. lc") is the model's distribution the target concept conditioned on a set text prompts o*
ag is the strength of gaussian noise
Vat + vI oge is the intermediate latent image with iterative noise addition. x
(xx,, c*2 t) is the denoising network that generates rt-1i whose parameters we aim to estimate.
2.1.1 Naive Approach
It essential to mention the standard diffusion training objective that was introduced as a naive approach performing concept ablation. This objective maximized the diffusion loss on the text-image pairs corre- sponding to the target concept, while regularizing the weights. (Kong & Chaudhuri 2023
arg DKL (p(X ll P (o((o...)))c"))
(1)
2.1.2 moise-based Ablation
The anchor distribution can be redefined as the pairing of the target prompt and the anchor prompt. An image, denoted by x, is created using the anchor prompt. Introducing randomly sampled noise to represented by 6, produces a noisy image I; at time-step f. The model is then fine-tuned using these redefined target-anchor pairs, employing the standard diffusion training loss.
Cnokse (X,, c, c²) Exx.ex. /wu,/e $(x,, c.,)
(2)
2

---

Published in Transactions on Machine Learning Research (06/2024)
2.1.3 Model-based Ablation
The method proposed is similar to the standard diffusion model training objective. The authors show that minimizing the KT divergence objective between the joint distribution noisy latent variables conditioned on anchor and target concept, i.e. loss in the main paper, can be reduced the difference between the predicted noise vectors. This reduced form is then used for training. (Kumari et al. 2023
DE (p(X || ps (X (x......))))
(3)
Reduces to
arg min E. e.c. | c,t) - (x,,,xx 4
(4)
A similar modified loss function is also used, which minimizes the reverse KL Divergence instead. Since the reduced form for both the losses is the same (i.e., the difference in noise vectors), they produce sufficiently similar results.
However, this requires extensive memory and computation. To bypass this, we sample 2, using the forward diffusion process and assume that the model remains similar for the anchor concept during fine-tuning. Therefore, we use the network & with stopgrad to get the anchor concept prediction. Thus, our final training objective is:
m c, 2) = & [| c, sg( 4xx,, t.ll...
(5)
CriPainting by Van Gagh
Diffusion Model (U-Net)
X
12 loss
C:Palnnting
v1 11- dl
Dxffusion Model (U-Vet)
c-N(O,1)
Figure 1: The method for fine-tuning the model involves adjusting the model weights to alter the distribution of generated images related to the target concept. The distribution for the anchor images is created from the diffusion model itself, with conditioning on the anchor concept. For instance, when removing Van Gogh's style, the goal is to align its distribution with the distribution generic paintings in the diffusion model.
Through the following graphs, we provide a comparative study between the performances of model-based and noise-basr based ablation methods.
3

---

Published in Transactions on Machine Learning Research (06/2024)
Traiiig
0.5
sole aeget connon
Kcap Steet Sancurdin Croupt
Rodd Ornest Conuge
Figure 2: comparison the proposed methods, specific to Object Ablation. The resulting CLIP scores are based on ablating the concept "grumpy cat".
Twinng
sole naggl Coang
Nra Sired Sancurding Group:
Hodd Coropt.
Figure 3: A comparison of the proposed methods, specific to Style Ablation. The resulting CLIP Scores are based on ablating the concept 'van gogh'.
From the preceding observations, we are able to prove empirically that Noise-based ablation is worse off than Model-based ablation method, quantified by trends in CLIP scores for both style and instance ablation cases. We can view a sharp decline in CLIP scores of the target concept for Model-based ablation, thus validating the claims made by the authors. Hence, have used the Model-based variant itself substantiate our study further.
3 Reproducibility Results
In this section, we report the results obtained on reproducing the methods of the original paper. We have documented the ablation results for the three different variants proposed and added visuals and analysis for all of them, some of which are not included in the original paper.
The variants are:
Style: The molivation behind this ablation variant is to remove an artist's specific influence from an image. This focuses on ablating certain specific textures and patterns.
Instance: This variant aims to remove specific object instances or events within images.
Memorized Images: This variant memorizes and ablates exact recurring training data concepts.
4

---

Published in Transactions on Machine Learning Research (06/2024)
3.1 Training Details
All the slandard training parameters are documented here.
Prompt Generation We used ChatGPT (ChttGPT) to generate 200 random prompts that contain the anchor prompt and 10 separate evaluation prompts. The prompts we used followed the format:
Training prompts: "Give 200 prompts for a text-to-iie text-to-image model containing <anchor-concept>>> and ensure that the prompts mention <anchor-concept> explicitly. Also, very few of these should contain <largel-concepl>'.
We ensured the prompts were descriptive and generic enough avoid extensive emphasis on the larget concept, which might mislead results by overfitting the model.
Evaluation prompts: "Give 10 prompts for a text-to-image model containing <concept- type>", where the <concept-lype> took up values <largel-concepl>> <anchor-concepl>, <surroumding-coneep
Baseline Image Generation We use the pre-trained checkpoint of the Stable Diffusion model (ckpt v1-4) (Rombach et al. 2022) to generate 200 images for baseline comparison on the same set of prompts.
Finc-tune Parameters We have reproduced results by fino-tuning the following architectural com- ponents of the diffusion model.
Pross-Attention The key and value projection matrices in the diffusion model's U-Net. (Kumari
ct al. 2022
Embedding: The text embedding in the text transformer. Gaa et al. 2022
Full Weights: All parameters of the U-Net. (Ruiz et al. 2022
3.2 Evaluation Metrics
We have utilized the CLIP Score as quantitative metric to assess the model performance. The CLIP Score provides a value to establish relevance between the prompt and the image generated from it. This is computed by minimizing the distance between matching image-lext pairs and maximizing il for non-matching ones using a contrastive loss function. (Hessel al. 2022). We have used the CLIP-vit-base-pate base-patch66 version of the CLIP Model. (Source code)
In the following section, we provide the results of our training corresponding to each concept ablation variant.
3.3 Evaluation Results
We computed CLIP scores for 10 different surrounding concepts. The surrounding concepts were chosen manually based on the image and prompts. For each of the Style and Instance variants, the final CLIP score is an average of 25 image scores, and for the Memorization variant it is an average of 10 image scores.
Our purpose for reproducing was to showcase the results with an emphasis on trends rather than the numerical accuracy of the scores. The graphs for Memorization ablation are based on standard CLIP score calculation, done without normalization (which was used by the authors). Hence, the CLIP scores are slightly deflated. [Zhengwentai] 2023) However, in order to reproduce results for style and instance ablation, have used the normalized version of the CLIP Score, used by the authors.
Note:
3.3.1 Style Variant
5

---

Published in Transactions on Machine Learning Research (06/2024)
Concapts
0.55
Seroad casoell Deeckire
summad by canopt -o starthn
Bige maagee inaedding
Coou deatie
Tralning
Figure 4: CLIP Scores for Style Ablation. We ablated images with the larget concept "Van Gogh' and the anchor concept 'Painting'. We evaluated on surrounding concepts other artists to show the robustness of the model.
Figure 5: Example of ablation of Van Gogh Style. Left: Images produced by the baseline diffusion model. Right: Images produced by ablated model
3.3.2 Instance Variant
vs. Training Steps Nifferee Concopts
0.5
0.55
Samoan to canopt weblldigg
Somuad epcoree Cos-iderter
Brst craatt Bravedns
umaat hull Waats
Archer cebtar untecdrg
archer canaar weight
Figure 6: CLIP Scores for Instance Ablation. We ablated images with the target concept "Cat+Grumpy Cat", and the anchor concept being "cat".

---

Published in Transactions on Machine Learning Research (06/2024)
Figure 7:
Example ablation Grumpy Cat Instance. Left: Images produced by bascline diffusion model. Right: Images produced by ablated model
3.3.3
Memorization Variant
CL Score Train Steps Different Concepts and Methods
0.73OM
022
teperrree siviggs nlwaye hillagee
100
230
300
Taaning sieps
Figure 8: CLIP Scores for Memorization Ablation.
NAVVELL
MARERAL
Figure 9: Example of memorized ablation. Left: Images produced by baseline diffusion model. Right: Images produced by ablated model
4 Additional Analysis and Limitations
4.1 Trademark Ablation
We propose a new variant of ablation based on the limitation we observed after performing experiments on the three categories.

---

Published in Transactions on Machine Learning Research (06/2024)
4.1.1 Motivation
While experimenting with instance ablation, observed a limitation, that the method could not completely ablate certain symbols of well-known since it is still visually possible to determine the proprietary information about the object. To tackle this, we proposed combining the hyperparameters of Memorization ablation with the basic configuration of Instance ablation.
objects. This undermines the vision for ablation
4.1.2 Proposed Method
We have modeled our method on the instance ablation variant, introducing the following changes to its configuration:
1. Freezing parameters: We changed the set of parameters to be fine-tuned, fixing them to 'all parameters' of the U-Net. (-Nie. et al. 2022
Augmentations: We have restricted augmenting the images, since we require the model to mem- orize the specific orientation and features of the target trademark concept.
3. Evaluation: We will evaluate this variant generic anchor concept, 'logo'. This restrains any overfitting and generalizes the anchoring instead of emphasizing alternate trademarks.
4.1.3 Experiment Results
The results of the trademark variant support our claims of better ablation the specific object. We achieved lower CLIP score for the target concept, showcasing better target ablation.
not able to observe drastic decrease in the CLIP sual absence of the logo. This is because CLIP compares the overall image with the entire prompt. (Radford ct al. 2021) Therefore, the image retains large semantic similarity with the prompt, even without the presence of the trademark logo.
However,
despite the complete vi-
Training Staps for Differant Methods
0.34
0.3
0.3
C.PSOORE
Cnpetaeeent: indenas alanko
Ancer cncept beannce Aileton
Accher mncret Todkerak Alchion
0
30
100
150
200
250
300
350
Training steps
Figure 10: CLIP Scores after finetuning on our trademark variant configuration. We can observe lowered CLIP score for the target concept, indicating better ablation. However, we can also observe a lowered CLIP score for the anchor concept. We hypothesize this is due to the model learning to ablate the entire logo/Uracd trademarkk class in the process of ablating a proprietary trademark symbol.
8

---

Published in Transactions on Machine Learning Research (06/2024)
Figure 11: Examples of complete ablation of the Starbucks logo. We kept the number of training steps and the global seed the same for both variants ensure the fairness of the experiment. Left: Generated by the instance ablated model. Right: Generated by our trademark ablation model.
4.2 Ablation leakage
An important metric to judge the success of ablation is if it is able to circumvent indirect laakage-inducingg target prompts, i.e, when the prompt does not explicitly include the target concept but describes it in similar terms, which induces the model into generating it. We showcase the inability of the proposed methods to prevent ablation in such cases. We evaluated the performance of the variants by sampling generated images and contrasting them with that of the pre- reetraiied model.
4.2.1 Instance and Style variant
We observed that the instance and style variants are relatively leakage- For example, while observing the performance after ablating the larget concept of "R2D2", we were not able to generate images that contained the r2d2 concept despite using prompts such as "Visualize a smart and brave astromech droid with a white and blue design, flat base with two legs, and a head that resembles radar dish.'
Leakage prompt evaluation on r2d2. Left: Generated by baseline stable diffusion model. Right: Generated by instance ablated variant.
Figure 12:
Similarly, while observing the performance after ablating gogh style, used prompts such "Cafe terrace at night" and "Vase with 15 sunflowers". However, the model ceased to generate gogh style.
9

---

Published in Transactions on Machine Learning Research (06/2024)
Figure 13: Leakage prompt evaluation on van gogh paintings. Left: Generated by baseline stable diffusion model. Right: Generated by style ablated variant.
4.2.2 Memorization variant
In comparison, we could induce the memorization ablated model to produce larget concept of "bilcoin', even with vague prompts such as "Cryptocurrency investments can be highly volatile, with prices fluctuating rapidly". (Zhang et al. 2023b
B
B
B
B
Figure 14: Left: Images generated by the baseline Right: Images generated by ablated model.
5
Discussion
In this section, we further hesiteesize reasons for the limitations of the proposed methods.
The degradation of surrounding concepts whilst ablating a target concept can be due to similar probability distributions. This similarity can be verified by examining the method of encoding tokens. While training, the shift the target distribution to the anchor distribution might also result a similar shift for surrounding concepts. (Gandikota et al. 2023)
We also provide some strategies that could be useful in mitigating the listed limitations, specifically the degraded performance of far-away concepts. This includes additional optimixations. We propose utilizing the following paths:
The decision boundaries of classification tasks be used to generate rehearsal samples that are more susceptible to forgetting. Retraining the model on those examples will ensure better perfor- mance on far-away concepts. (Cywiñski et al. 2024
Guided diffusion processes, such the ControlNets can be used to supervise the output produced by the diffusion model, and guide the output using conditional losses. (Zhang al. 2023a
10

---

Published in Transactions on Machine Learning Research (06/2024)
The CLIP Score is skewed metric for verifying the absolute performance of concept ablation model. This is due to the reasoning that the CLIP compares the entire textual prompt. with the image. Hence, the reasoning for dip in the CLIP score cannot be attributed solely to the success of ablation. This is why we cannot determine a certain threshold below which we can deem a model successful. (Stein et al. 2023
Note:
6 Conclusion
Through the course of this work, we have reproduced the claims made by the authors, along with providing additional visual metrics to analyze the results. We experimented with a new variant of concept ablation, the trademark variant. We showcased its performance superseding that of instance ablation on the specific objective of completely ablating proprietary identification symbols on objects. We also showcased various limitations of the proposed methods.
Broader Impact Statement
The work of this paper encourages thought exploring the ablation of proprictory objects and products. This provides institutions with the choice of retaining the privacy of their products.
Acknowledgements
We would like to thank Aniket Agarwal for providing invaluable guidance throughout, as well as the members of VLG ITT for providing valuable leedback and participating in the user study. We would also like thank the authors of the original work for making their code publicly available, allowing conduct our experiments effectively.
References
ChatGPT. URL hathh:tt///hhhtt httphttps///chtt.pppaaa... wwt.:tt.cooaacccceeeeoo
Bartosz Cywiński, Kamil Deja, Tomasz Trzciński, Bartlomiej Twardowski, and Lukasz Kuciński. Guide: Guidance-based incremental learning with diffusion models, 2024.
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit, Bermano, Gal Chechik, and Daniel Cohen-Or: An image is worth one word: Personalizing text-to-image generation using textual inversion. 2022.
Rohit Gandikota, Joanna Materzyúska, Jaden Fiotto-Kaufnan, and David Bau. Erasing concepts from diffusion models. In Proceedings of the 2023 TEEE International Conference on Compuler Vision, 2023.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. A reference-free evaluation metric for image captioning. 2022.
Zhifeng Kong and Kamalika Chaudhuri. Data redaction from pre-trained gans. 2023.
Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yar Zhu. Multi-concept cus- tomization of text-to-image diffusion. 2022.
Nupur Kumari, Bingliang Zhang, Sheng-Yu Wang, Eli Shechtman, Richard Zhang, and Jun-Yan Zhu. Ab- lating concepts in text-to-image diffusion models. 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang (eds.), Proceedings of the 98th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 8748-8763. PMLR, 18 24 Jul 2021. URL https::/prooeedingg.. mlr. press/v11//rafford2aa..k html
11

---

Published in Transactions on Machine Learning Research (06/2024)
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the TEEE/CT Conference on Compuler Vision and Pattern Recognition (CVPR), pp. 10684 10695, 2022.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfr Aberman. Dream- booth: Fine tuning text-to-image diffusion models for subject-driven generation. 2022.
George Stein, Jesse C. Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L. Caterini, J. Eric T. Taylor, and Gabriel Loaiza-Ganeni Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models, 2023.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023a.
Yang Zhang, Teoh Tze Tzun, Lim Wei Hern, and Kenji Kawaguchi Haonan Wang. Investigating copyright issues of diffusion models under practical scenarios. 2023b.
SUN Zhengwentai, clip-score: CLIP Score for PyTorch. httpss//gthuub.cc//aaiae/////11 March 2023. Version 0.1.1.
A Appendix
Visual observations of our experimentations.
A.1
Image generation for far-off concept
We provide some observations on the degrading quality of images generated by model that ablates far-off concept.
Figure 15: The model here trained to ablate the target instance 'Snoopy Dog". We observe the poor compositional ability of the model to generate images of cats.
12

---

Published in Transactions on Machine Learning Research (06/2024)
Figure 16: The model here was trained to ablate 'Grumpy Cat'. While generating images of an unrelated concept of "dogs, observe poor compositional ability.
Figure 17: Some more images of the above example where we observe poor compositional ability.
A.2 Trademark Ablation Examples
loca
LabT
m
U
Figure 18: Our variant of the trademark ablated model ablated the target concept of the McDonald's Logo. We get promising results, as shown.
A.3 Misc Generated Examples
13

---

Published in Transactions on Machine Learning Research (06/2024)
Figure 19: The model was trained to ablate Van Gogh style, on sampling various images for the prompt 'New York in Van Gogh style", get the above results
A.4 User Study
In order to support the legibility of our claims, we performed a user study. The participants were askod Trademark ablation model and the other to the
to choose between pairs of images, one belonging proposed Instance ablation model. The orientation of our images was shuffled constantly between left and right, ensuring no prior bias. We performed this study on 33 participants.
A.4.1 Results
Taking the average of feedback from all participants:
For the ablated concept, the metric of choice is the success of ablation, given the target prompt. We evaluated these on a set pairs images:
Trademark ablation better: 38/45 /5 84.5%
Similar performance: 6/45 ~ 13.3%
Instance ablation better: 1/45 2.2%
For the anchor concept, the metric of choice is the compositional ability, given the anchor concept.
We further added 20 pairs for tosting:
Trademark ablation beiter: 46/65 xs 70.8%
Similar performance: 12/65 = 18.5%
Instance ablation better: 7/65 10.7%
14


Reviews:
[{"review_id": "lifKXNmoEm", "reviewer": "Reviewer_YwiN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This is essentially a reproduction study, which I think is a good thing for science. They also introduce a new ablation concept. They consider three reproduction results: when the style of the artist is to be removed, when a specific instance or event has to be removed, and when memorization based image generation. \nThey have a clear evaluation plan and metric. They present their work as it is without trying to conflate their finding. I find this as a good welcome even pointing out the limitations of their method.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper under submission is largely a work to reproduce previous results using various proposed ablation concepts. \nThe second contribution is to introduce a new ablation technique called trademark ablation. They also provide the limitations of the models and also study the model's behavior in response to ablation leakage-inducing prompts.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Nothing to add here.", "broader_impact_concerns": "There are none.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "LgEB9CIDZw", "reviewer": "Reviewer_8vNy", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. This paper showcases the limitations of existing concept ablation studies against leakage inducing prompts.  \n\nWeaknesses:\n1. The experiments are only demonstrated on a few experimental data points and hence, as an empirical paper, the main messages may not be well supported with the limited empirical evidence. \n2. There is a lack of understanding on how we may improve existing concept ablation approaches. It is also unclear what is the contribution of reproducing the claims made in prior work. I think more interesting results are ones that point out the limitations of previous works and then also proposing some tentative solutions. \n3. There is a lack of understanding on how to qualitatively view the CLIP scores presented in the paper. Is below certain CLIP score threshold considered well-enough for practical applications?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper extends the study of concept ablation within pre-trained diffusion models, which is done in Kumari et al. (2022). This paper first produces the claims made in the prior working then proposes a new variant of concept ablation called trade-mark ablation, which achieves better visual performance compared to instance ablation. The authors also showed the limitations of existing approaches against leakage-inducing prompts.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Add description of the notation in the paper. It is hard to parse the technical part without having a proper knowledge about the notations.\n2. Record more data points in terms of different training steps so that we can have understandings about the trend with finder details. \n3. Discuss about the possible reasons behind the limitations of existing approaches and ideally, can also propose some (tentative) mitigation solutions. I would suggest removing the reproducibility results to allow space for more interesting results.", "broader_impact_concerns": "The broader impact is addressed in the paper and is sufficient.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "uLAdi2W9Gy", "reviewer": "Reviewer_izoN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n- The concept of ablation is one of the tools that can be used for privacy and copyright protection. It is an important topic for ensuring the generated images do not incur copyright infringement. The concepts of ablation and trademark ablation studied have shown very promising results. It could have a large impact on the existing practices of releasing image-generation models to the public. The model owner can use the ablation for privacy and copyright protection.\n- The experimental results are very promising, and observational analyses are very straightforward. \n\nWeaknesses\n\n- It might be better to include more numerical analysis. For example, a comparison between Eq (1), Eq (2), and Eq (4) for these observational analyses.\n- Minor typos.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the concept of ablation, which is an important topic for privacy and copyright protection. Specifically, this paper investigates image generation models. This paper reproduced results from existing work by Kumari et al., 2022. Additionally, the authors have introduced a new variant of concept ablation, the trademark ablation. It aims to remove well-known brands/trademarks from the generated images. Throughout several observational analyses, this paper demonstrated that trademark ablation can successfully remove the logo while not affecting other similarly generated images.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Typo of the single quotation marks. In both of the abstract and the Section 1.\n- It would be great to use Section XXX rather than just XXX for the in-text references. E.g., in the paper outline. This could improve the readability.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper is a valuable reproducibiltiy study with some additional insights and as such clearly meets the bar for TMLR.
Decision: Accept (poster)