DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing

## Abstract

Abstract Recent advances in diffusion models have introduced a new era of text-guided image manipulation, enabling users to create realistic edited images with simple textual prompts.
However, there is significant concern about the potential misuse of these methods, especially in creating misleading or harmful content.
Although recent defense strategies, which introduce imperceptible adversarial noise to induce model failure, have shown promise, they remain ineffective against more sophisticated manipulations, such as editing with a mask.
In this work, we propose DiffusionGuard, a robust and effective defense method against
unauthorized edits by diffusion-based image editing models, even in challenging setups.
Through a detailed analysis of these models, we introduce a novel objective that generates adversarial noise targeting the early stage of the diffusion process.
This approach significantly improves the efficiency and effectiveness of adversarial noises.
We also introduce a mask-augmentation technique to enhance robustness against various masks during test time.
Finally, we introduce a comprehensive benchmark designed to evaluate the effectiveness and robustness of methods in protecting against privacy threats in realistic scenarios.
Through extensive experiments, we show that our method achieves stronger protection and improved mask robustness with lower computational costs compared to the strongest baseline. Additionally, our method exhibits superior transferability and better resilience to noise removal techniques compared to all baseline methods. Our source code is publicly available at https://github.com/choi403/DiffusionGuard .

## 1 Introduction

Text-to-image diffusion models trained on large-scale datasets have demonstrated impressive results in generating high-quality images from text prompts (Betker et al., ; Sauer et al., ; Saharia et al., ).
These models have expanded beyond simple image generation to support text-guided image editing (Wang et al., ; Brooks et al., ; Yenphraphai et al., ), enabling users to modify existing images with both ease and precision.
For instance, Imagen Editor (Wang et al., ) allows users to manipulate images using masks and textual descriptions. This facilitates detailed and intuitive adjustments to specific areas of an image.
Similarly, Image Sculpting (Yenphraphai et al., ) identifies 3D objects in photos. With this tool, users can then manipulate these objects directly, unlocking new possibilities for altering images.
These approaches improve the user-friendliness of image editing tools, which significantly enhance the creative process by allowing for precise modifications based on textual input.

However, alongside these advances, there exists a significant concern regarding the potential misuse of text-guided image editing models.
With their ability to create highly realistic and convincing content, image editing models can be exploited for malicious purposes such as generating fake news, spreading disinformation, and creating deceptive visual content.
For example, with open-sourced text-to-image models (Rombach et al., ), one could easily manipulate a photo to falsely depict a celebrity being arrested, as shown in the bottom row of Fig. [1](#S1.F1).
As text-to-image models become more powerful, it is paramount to address these risks and implement safeguards to prevent misuse.

Figure: Figure 1: Protecting against the misuse of text-to-image models using DiffusionGuard. (Bottom row) Images without protection are vulnerable to malicious editing, such as altering the background while preserving the face to create fake images (*e.g.*, a celebrity being arrested). (Top row) DiffusionGuard protects the image by focusing on the face, a defining feature of personal identity. It disrupts diffusion models and results in failed edits when attackers attempt malicious changes.
Refer to caption: /html/2410.05694/assets/x1.png

To mitigate the potential risks associated with the misuse of text-to-image diffusion models, protection methods based on adversarial noises have shown promise recently (Liang et al., ; Liang & Wu, ; Salman et al., ; Xue et al., ).
These techniques involve adding imperceptible adversarial noise to the original image, which is designed to cause the model to fail in generating high-quality images (see Fig. [1](#S1.F1)).
By publishing images with this adversarial noise, the cost and difficulty of malicious editing are significantly increased.
However, current methods do not provide robust protection against real-life scenarios, such as editing with freely chosen masks by adversaries, which can bypass the protection.
This issue is especially problematic as adversaries may select the smallest possible region containing sensitive identities (*e.g.*, a person’s face), thereby minimizing the effectiveness of these protection methods.

##### Contributions

In this work, we introduce DiffusionGuard, a robust and effective defense method against text-guided image editing models in challenging setups, such as editing with user-selected masks.
Specifically, we propose a novel objective to generate adversarial noises targeting the early stage of the diffusion process.
Through our analysis, we observe that editing models tend to generate key regions within the mask during these initial diffusion steps.
Therefore, by directing adversarial perturbations at the early stages, we prevent the models from maintaining the key regions, which are crucial for high-quality editing.
Additionally, we propose a mask-augmentation method to find robust adversarial perturbations that are effective against mask inputs of various shapes.

For concrete evaluation, we introduce InpaintGuardBench, a challenging evaluation benchmark designed to assess defense methods against image editing models.
InpaintGuardBench comprises images paired with handcrafted masks of diverse shapes and text prompts for editing, enabling a comprehensive evaluation of robustness against various misuse scenarios.
We conduct human surveys and measure qualitative metrics to evaluate DiffusionGuard.
Through extensive experiments, we demonstrate both qualitatively and quantitatively that DiffusionGuard is effective, and most importantly, robust against changes in mask inputs.
This makes it exceptionally useful in real-life scenarios.
Moreover, our method proves to be more compute-efficient and performs well even in low noise budget setups compared to existing baselines (Liang et al., ; Salman et al., ).

## 2 Preliminaries

This section provides an overview of text-to-image diffusion models, emphasizing
inpainting models and adversarial examples against them.

### 2.1 Diffusion models

We consider denoising diffusion models (Sohl-Dickstein et al., ; Ho et al., ; Dhariwal & Nichol, ) in discrete time.
Suppose $\mathbf{x}\sim p_{\textrm{data}}(\mathbf{x})$ represents the data distribution.
A diffusion model defines a sequence of latent variables with noise scheduling functions $\alpha_{t},\sigma_{t}$ such that the log signal-to-noise ratio $\lambda_{t}=\log(\alpha_{t}^{2}/\sigma_{t}^{2})$ decreases with $t$.
The forward process of a diffusion model is described by gradually adding noise to the data $\mathbf{x}$, where the marginal distribution is given as $q(\mathbf{x}_{t}|\mathbf{x})=\mathcal{N}(\mathbf{x}_{t};\alpha_{t}\mathbf{x},\sigma_{t}^{2}\mathbf{I})$.
For sufficiently large $\lambda_{T}$, $\mathbf{x}_{T}$ becomes indistinguishable from pure Gaussian noise, and for sufficiently small $\lambda_{0}$, $\mathbf{x}_{0}$ is nearly identical to the data distribution.
The reverse process starts from random noise $\mathbf{x}_{T}$, and sequentially denoises it to generate $\mathbf{x}_{0}$, which matches the training distribution.

##### Text-to-Image diffusion models

Text-to-image (T2I) diffusion models (Rombach et al., ; Saharia et al., ; Betker et al., ) are a class of diffusion models specifically designed to generate images conditioned on text prompts.
These models incorporate text embeddings extracted from pre-trained text encoders like T5 (Raffel et al., ) or CLIP (Radford et al., ) to guide the image generation process.
Given a pair of image $\mathbf{x}$ and text $y_{\tt{text}}$,
these models commonly employ a noise prediction model $\bm{\epsilon}_{\theta}(\mathbf{x}_{t};t)$ and are trained using a noise prediction loss as follows:

$$ $\mathcal{L}_{\tt{diff}}(\theta;\mathbf{x})=\mathbb{E}_{t\sim\mathcal{U}(1,T),\bm{\epsilon}\sim\mathcal{N}(\bm{0},\mathbf{I})}\big{[}\omega(\lambda_{t})\|\bm{\epsilon}_{\theta}(\mathbf{x}_{t};y_{\tt{text}},t)-\bm{\epsilon}\|_{2}^{2}\big{]}\text{,}$ (1) $$

where $\omega(\lambda_{t})$ is a weighting function of a timestep $t$.

##### Text-guided inpainting models

In addition to T2I generation, it is of a great interest to edit a desired region of a given image with text prompts.
To this end, T2I image inpainting models (Nichol et al., ; Saharia et al., ; ) propose to fine-tune pretrained T2I diffusion models to leverage its rich generative prior.
In specific, inpainting models are fine-tuned by adding conditions of source image $\mathbf{x}_{\tt{src}}$ and binary mask $M$ that designates the region to infill to the noise prediction loss in Eq. [1](#S2.E1).
During fine-tuning, random regions of an image are masked, and the source image and a mask are concatenated to the noisy latent $\mathbf{x}_{t}$ as an input of the diffusion model.
The training objective of diffusion-based inpainting models is given as follows:

$$ $\mathcal{L}_{\tt{Inpaint}}(\theta;\mathbf{x}_{\tt{src}},M)=\mathbb{E}_{t\sim\mathcal{U}(1,T),\bm{\epsilon}\sim\mathcal{N}(\mathbf{0},\mathbf{I})}\big{[}\omega(\lambda_{t})\|\bm{\epsilon}_{\theta}(\mathbf{x}_{t};y_{\tt{text}},t,M,\mathbf{x}_{\tt{src}})-\bm{\epsilon}\|_{2}^{2}\big{]}\text{.}$ (2) $$

### 2.2 Adversarial examples against diffusion models

Adversarial examples are deliberately fabricated data to manipulate model behaviors (Szegedy et al., ; Biggio et al., ), often with malicious intent.
Given a clean image $\mathbf{x}$ and a model, an adversarial example adds a perturbation $\delta$ to $\mathbf{x}$ so that $\mathbf{x}+\delta$ deceives the model.
These perturbations are typically crafted to be imperceptible to human eyes, *e.g.*, via constrained optimization using $\ell_{\infty}$ bound $\|\delta\|_{\infty}\leq\eta$ for some $\eta>0$.
In this paper, we consider crafting an adversarial example for text-guided image editing models,
where we aim to find a perturbation $\delta$ of source image that enforces the editing models to generate low-quality images.
A line of research (Liang et al., ; Liang & Wu, ; Xue et al., ; Salman et al., ) has investigated adversarial examples of this purpose, using them as a protective measure against unauthorized image editing.
These works either perturb each individual step of the denoising process to maximize the diffusion model training loss (*i.e.*, Eq. [1](#S2.E1)),
or force diffusion models to generate a specifically undesirable image as follows:

$$ $\displaystyle\delta=\operatorname*{arg\,min}_{||\delta||_{\infty}\leq\eta}~{}\mathbb{E}_{\bm{\epsilon}\sim\mathcal{N}(\bm{0},\mathbf{I})}\big{[}\|\widehat{\mathbf{x}}(\mathbf{x}_{\tt{src}}+\delta;\bm{\epsilon},y_{\tt{text}},M)-\mathbf{x}_{\tt{target}}\|_{2}^{2}\big{]}\text{,}$ (3) $$

where $\widehat{\mathbf{x}}$ is a generated image
given source image $\mathbf{x}_{\tt{src}}+\delta$, prompt $y_{\tt{text}}$, and mask $M$.

## 3 Main method

In this section, we outline DiffusionGuard, a method designed to protect images against inpainting methods in challenging scenarios (Sec. [3.1](#S3.SS1)).
First, based on the unique behaviors of inpainting models, we develop a novel objective to target the early stages of the reverse diffusion process (Sec. [3.2](#S3.SS2)).
Next, we propose a mask-augmentation method to find a robust adversarial perturbation that remains effective against mask inputs of various shapes (Sec. [3.3](#S3.SS3)).

### 3.1 Problem setup

Previous protection methods (Liang et al., ; Liang & Wu, ; Xue et al., ) typically consider a *global* perturbation $\delta$ applied across the entire image, *i.e.*, $\mathbf{x}+\delta$, as described in Sec. [2.2](#S2.SS2).
However, such methods become ineffective against diffusion inpainting models, where a binary mask $M$ is additionally applied to the source image, *i.e.*, it processes masked source images $(\mathbf{x}+\delta)\odot M$.
This realistic setup poses a unique challenge for adversarial defense, given that now only the part of adversarial noise that intersects with the mask $M$ can affect the model’s behavior.

##### Threat model

We assume that a malicious user attempts to successfully edit an image protected by adversarial perturbations applied by a defender.
This malicious user can freely choose the mask input $M$, and text prompt $y_{\texttt{text}}$ for editing.
Because it is challenging to develop a defense method against any arbitrary mask, we consider a feasible yet practical scenario where both defender and malicious user share a common understanding of the *sensitive region* in the source image; typically, in a portrait, this could be the face or the body of a person, while in other contexts, it might be a specific object.
We assume that the defender uses this sensitive region as a training mask $M_{\texttt{tr}}$ in generating adversarial noises.
Meanwhile, a malicious user can utilize a different mask $M_{\texttt{te}}$ but based on the same conceptual sensitive region.

Figure: (a) Denoising process of standard diffusion models
Refer to caption: /html/2410.05694/assets/x2.png

### 3.2 Perturbing the early stages of the diffusion process

In this section, we introduce a novel objective that specifically exploits a unique behavior we have observed in inpainting models.
As shown in Fig. [2(a)](#S3.F2.sf1), it is well-known that during the denoising process of diffusion models, coarse features (such as the image outline) emerge first, while fine details are generated in the later stages (Ho et al., ; Hertz et al., ).
However, we have found that this pattern does not hold for inpainting models.
As illustrated in Fig. [2(b)](#S3.F2.sf2),
these models first produce fine details (*e.g.*, facial features) even at the first denoising step.

This unique behavior likely originates from the additional inputs given during the fine-tuning process of inpainting models.
Unlike standard diffusion models that only receive random noises as input, inpainting models are fine-tuned to utilize two additional inputs by modifying the input channel of the noise prediction model $\bm{\epsilon}_{\theta}$.
Specifically, these models take a binary mask $M_{\texttt{tr}}$, and a masked source image $\mathbf{x}_{\texttt{src}}\odot M_{\texttt{tr}}$ as inputs.
Inpainting models are fine-tuned using a reconstruction loss (Eq. [2](#S2.E2)), which encourages them to copy and paste the unmasked region of the image, leading to the emergent behavior observed in Fig. [2(b)](#S3.F2.sf2).

Inspired by the unique behavior of inpainting models, we develop a novel objective that targets the initial step of the denoising process.
Suppose we have a source image $\mathbf{x}_{\texttt{src}}$ to protect, an inpainting model $\bm{\epsilon}_{\theta}$, and a binary mask $M_{\texttt{tr}}$ which designates the part of the image to keep while rest of the image is recreated.
We aim to find an adversarial perturbation $\delta$ that maximizes the $\ell_{2}$ norm of the initial predicted noise only (*e.g.*, see Fig. [3](#S3.F3)):

$$ $\displaystyle\delta=\operatorname*{arg\,max}_{||\delta||_{\infty}\leq\eta}\left\|\bm{\epsilon}_{\theta}(\mathbf{x}_{T};y_{\tt{text}},T,M_{\tt{tr}},\mathbf{x}_{\tt{src}}+\delta)\right\|_{2}^{2},$ (4) $$

where $T$ corresponds to the initial denoising step and $\mathbf{x}_{T}$ is random noise.
Our proposed objective focuses on targeting the early stage of the diffusion process, in contrast to prior methods that target the entire diffusion process Liang et al. () or the output images Salman et al. ().
This approach makes generating adversarial noise both efficient and effective because only one forward pass through the noise prediction model is necessary.
Additionally, unlike previous methods that aim to maximize reconstruction loss (Eq. [1](#S2.E1)) or minimize the distance to an arbitrary target image (Eq. [3](#S2.E3)), we propose to increase the norm of the noise.
We have found that this maximum norm objective is more effective than previous approaches (see Fig. [6(a)](#S4.F6.sf1) for supporting results).

### 3.3 Mask-robust adversarial perturbation

In practice, malicious users may utilize a mask that differs from the mask $M_{\texttt{tr}}$ that is seen during the generation of adversarial noise.
Therefore, it is crucial to find robust perturbations that are effective across various mask shapes.
To achieve this, we propose a mask augmentation $\mathcal{A}(\cdot)$ that generates a new binary mask with a similar shape to $M_{\texttt{tr}}$.
Specifically, we first obtain the points along the contours of $M_{\texttt{tr}}$ using contour detection.
We then adjust these points inward by a random offset to define a new contour.
The area inside this new contour is filled to form the augmented mask (see Fig. [3](#S3.F3)).

To make these modifications more realistic, we apply smoothing to the random offsets.
The full procedure of mask augmentation is summarized in Algorithm [1](#alg1).

Figure: Figure 3: Overview of DiffusionGuard. We propose (a) mask augmentation for improving robustness, and (b) early state perturbation loss for generating effective noises.
Refer to caption: /html/2410.05694/assets/x4.png

Using the proposed mask augmentation function $\mathcal{A}(\cdot)$, we generate a robust $\eta$-bounded adversarial perturbation $\delta$ by minimizing the following loss over the set $\mathcal{M}$ of augmented masks $\mathcal{A}(M_{\texttt{tr}})$:

$$ $\displaystyle\delta=\operatorname*{arg\,max}_{||\delta||_{\infty}\leq\eta}\mathcal{L}_{\tt{adv}}(\theta;\mathbf{x}+\delta,M_{\tt{tr}})=\mathbb{E}_{M\sim\mathcal{A}(M_{\tt{tr}})}\big{[}\big{\|}\bm{\epsilon}_{\theta}(\mathbf{x}_{T};y_{\tt{text}},T,M,\mathbf{x}+\delta)\big{\|}_{2}^{2}\big{]}\text{,}$ (5) $$

where $\mathcal{L}_{\tt{adv}}$ is from our adversarial loss in Eq. [4](#S3.E4).(^1^11Note that this applies to any mask-dependent adversarial loss (Salman et al., ), see Appendix [E.3.2](#A5.SS3.SSS2).)
In practice, we optimize $\delta$ by stochastically sampling masks from $\mathcal{M}$ during the adversarial noise generation.
Each iteration, we sample a mask $M\sim\mathcal{A}(M_{\texttt{tr}})$ and perform a projected gradient descent (PGD) step (Madry et al., ) to update $\delta$:

$$ $\displaystyle\delta\leftarrow\mathrm{Proj}_{||\delta||_{\infty}\leq\eta}\left(\delta-\gamma\cdot\mathrm{sign}(\nabla_{\delta}\mathcal{L}_{\tt{adv}})\right),$ (6) $$

where $\gamma$ is the step size and $\mathrm{Proj}_{||\delta||_{\infty}\leq\eta}(\cdot)$ projects $\delta$ onto the $\ell_{\infty}$ ball of radius $\eta$. By iteratively updating $\delta$ using different augmented masks, we effectively minimize the expected adversarial loss over the set of masks $\mathcal{M}$. This stochastic optimization approach allows us to find a perturbation $\delta$ that is robust to various mask shapes similar to $M_{\tt{tr}}$.

Figure: (a) Edit prompt: "A man in a hospital"
Refer to caption: /html/2410.05694/assets/x5.png

## 4 Experiments

### 4.1 InpaintGuardBench: Inpainting-specialized protection benchmark

##### Benchmark dataset

To thoroughly validate protection effectiveness and mask robustness, we construct a benchmark specialized for masked inpainting models.
Our benchmark, named InpaintGuardBench, consists of 42 images, each associated with five unique masks.
Out of these, one mask per image is generated using SAM (Kirillov et al., ), a state-of-the-art segmentation method, and the remaining four masks are handcrafted using the most common tools employed by end-users.
The most commonly used tool for drawing a mask is the circle brush, where users select the region to keep by painting over the image.
This method is employed by popular inpainting tools such as OpenAI DALL-E inpainting (Ramesh et al., ) and Stable Diffusion web UI,(^2^22[https://github.com/AUTOMATIC1111/stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui)) the most widely used open-source GUI for diffusion models.
We also incorporate simple handcrafted mask shapes such as rectangles and circles.
Our benchmark contains 42 images, divided into three categories: 32 celebrity portraits, 5 inanimate objects, and 5 animals.
We consider 10 edit prompts for each image, resulting in a total of 2,100 edit tasks (42 images, 5 masks, and 10 prompts).

##### Baselines

We compare our method to various baseline methods.
Our primary baseline is PhotoGuard (Salman et al., ), a protection method specialized for inpainting models.
It adds perturbation within the mask region, optimized to cause diffusion models to generate incorrect images, as detailed in Eq. [3](#S2.E3).
We also consider AdvDM (Liang et al., ), a protection method that targets standard text-to-image diffusion models.
This approach involves perturbing the entire image to disrupt the denoising process, aiming to maximize reconstruction loss (Eq. [1](#S2.E1)).
Furthermore, we consider Mist (Liang & Wu, ) and SDS(-) (Xue et al., ), both of which build on top of AdvDM to improve the method.
These three methods originally propose to add perturbation to the entire image, without considering specific mask regions.
In our main experiments, we report the results obtained from the original approaches of these methods.
However, we also consider variants of these methods that introduce perturbations only within the mask region $M_{\tt{tr}}$, and the results of this modified approach are reported in Appendix [E.3.1](#A5.SS3.SSS1).

##### Setup and evaluation metrics

As the target model, we use Stable Diffusion Inpainting (Rombach et al., ), an open-sourced inpainting diffusion model.
For generating adversarial noises, we use the SAM-generated mask as the training ("seen") mask.
We then evaluate the effectiveness of the generated adversarial perturbations on all 5 masks, including the handcrafted 4 "unseen" masks.

For evaluation, we employ quantitative metrics to measure the fidelity of the prompt and the quality of the image.
We use the following metrics:

- •
CLIP directional similarity (Gal et al., ): This measures the alignment between the deviation in the image (from the source to the edited result) and the deviation in the text (from the source caption to the edit instruction). The source caption, which describes the source image, is generated using the BLIP-Large model (Li et al., ).
- •
CLIP similarity (Radford et al., ): This measures the text fidelity of the edited image using the cosine similarity between their CLIP embeddings.
- •
ImageReward (Xu et al., ): A human-aligned vision-language model (Li et al., ) fine-tuned on a human preference dataset, assesses both the resulting image quality and text fidelity.

Additionally, we measure the PSNR between the edited results of unprotected and protected images, as done by (Salman et al., ), to quantify the differences in the edited result compared to the unprotected version.

### 4.2 Main results

**Table 1: Results on InpaintGuardBench. Our method achieves strong protection in both Seen and Unseen set, across all metrics. The lower number represents better protective effectiveness, indicating failed edits. All methods were optimized using constraint of $\|\delta\|_{\infty}=\nicefrac{{16}}{{255}}$.**
| Method | PSNR $\downarrow$ | CLIP Dir. Sim. $\downarrow$ | ImageReward $\downarrow$ | CLIP Sim. $\downarrow$ |
| --- | --- | --- | --- | --- |
| Seen (1 Mask, Train set) |  |  |  |  |
| Unprotected | N/A | 24.40 | -1.365 | 30.15 |
| PhotoGuard (Salman et al., ) | 12.87 | 21.17 ($\Delta$-3.23) | -1.537 ($\Delta$-0.172) | 27.89 ($\Delta$-2.26) |
| DiffusionGuard | 12.60 | 18.95 ($\Delta$-5.45) | -1.807 ($\Delta$-0.442) | 26.55 ($\Delta$-3.60) |
| Unseen (4 Masks, Test set) |  |  |  |  |
| Unprotected | N/A | 24.29 | -1.315 | 30.72 |
| PhotoGuard (Salman et al., ) | 14.53 | 23.30 ($\Delta$-0.99) | -1.357 ($\Delta$-0.042) | 30.30 ($\Delta$-0.42) |
| DiffusionGuard | 13.19 | 21.84 ($\Delta$-2.45) | -1.557 ($\Delta$-0.242) | 29.05 ($\Delta$-1.67) |
| AdvDM (Liang et al., ) | 13.37 | 24.27 ($\Delta$-0.02) | -1.361 ($\Delta$-0.046) | 30.97 ($\Delta$+0.25) |
| Mist (Liang & Wu, ) | 14.51 | 23.93 ($\Delta$-0.36) | -1.307 ($\Delta$+0.008) | 30.79 ($\Delta$+0.07) |
| SDS(-) (Xue et al., ) | 14.32 | 23.85 ($\Delta$-0.44) | -1.237 ($\Delta$+0.078) | 30.78 ($\Delta$+0.06) |

We compare DiffusionGuard with the baseline methods (PhotoGuard, AdvDM, Mist, SDS(-)) on InpaintGuardBench.
For all experiments, we ensure a fair comparison by running the protection methods for an equal amount of GPU time.

##### Qualitative comparison

As shown in Fig. [4](#S3.F4), DiffusionGuard demonstrates superior protective effectiveness, causing the edit result to become nearly devoid of any recognizable content in most examples.
In contrast, the baseline methods generate images that are more realistic and well-aligned with the prompts.
Notably, the protected results of DiffusionGuard effectively prevent the diffusion inpainting model from ’recognizing’ the object, as illustrated in the final example where a different dog is drawn in place of the original.
Additionally, DiffusionGuard demonstrates its robustness against mask changes, in contrast to PhotoGuard (Salman et al., ), a protection method utilizing mask information, which loses the protective effectiveness even with small deviations in mask shape.

##### Quantitative results

Table [1](#S4.T1) reports the quantitative metrics for DiffusionGuard and the baseline methods.(^3^33Baselines optimized without mask information are omitted from Seen group of the table for fair comparison.)
DiffusionGuard exhibits strongest protection among all methods for both Seen and Unseen masks.
Note that DiffusionGuard outperforms the baseline methods in both mask categories by a significant margin, in line with the results presented in Fig. [4](#S3.F4).

##### Human evaluation

We also conduct a human evaluation as follows:
for each edit instance, defined by a triplet of source image, mask shape, and edit instruction, we display two edit results using DiffusionGuard and PhotoGuard in random order.
Human raters then select which result is better or tie (*i.e.*, the two editing results are similar) for all 2100 pairs.
We ask human evaluators to make decisions based on both image quality and edit prompt fidelity simultaneously.
We calculate the win rates of a protection method by counting how often its result was not chosen (*i.e.*, deemed worse).
As shown in Fig. [5(a)](#S4.F5.sf1), DiffusionGuard results in a superior win rate against PhotoGuard in both Seen and Unseen sets of masks, with approximately a 17% higher win rate gap over the baseline.

### 4.3 Comparison under resource-restricted scenarios

Figure: (a) Human survey
Refer to caption: /html/2410.05694/assets/x10.png

In this section, we compare our method against baselines in two resource-restricted scenarios.
First, we evaluate each method with varying running times to compare computational efficiency.
Second, we test each method under limited noise budget by setting the noise threshold $\|\delta\|_{\infty}$ to ${4/255,6/255,8/255,12/255,}$ and $16/255$ in order to compare them under tighter noise constraints.

##### Comparison under limited compute budget

Fig. [5(b)](#S4.F5.sf2) shows that DiffusionGuard is more effective than PhotoGuard when both are optimized for an equal number of steps.
Specifically, our method with compute budget of 11 seconds achieves a similar PSNR of PhotoGuard at 90 seconds.
Additionally, when comparing the performance gaps between the Unseen and Seen mask categories for each method, the gap is notably smaller for DiffusionGuard (blue).
These results demonstrate that our method is faster, cheaper, and more effective than PhotoGuard.

##### Comparison under limited noise budget

Fig. [5(c)](#S4.F5.sf3) shows that DiffusionGuard consistently achieves stronger performance (*i.e.*, lower PSNR) under a tighter noise budget.
In particular, our method with a noise budget of $6/255$ is similar to PhotoGuard even when PhotoGuard uses a higher budget of $16/255$.
These results show that DiffusionGuard maintains high levels of protection even with reduced perturbations (*i.e.*, less visible).
This makes it suitable for real-life applications where generating less detectable noise and preserving the original image quality are crucial.

### 4.4 Ablation study

We conduct a comprehensive analysis on the effects of loss functions in adversarial noise generation, mask augmentation, and the efficacy of using an inpainting-specialized method.

##### Loss functions in adversarial noise generation

To verify the effectiveness of the early stage perturbation loss (Eq. [4](#S3.E4)) in generating adversarial noises, we compare it with image-space loss (used in PhotoGuard (Salman et al., )) and reconstruction loss (used in AdvDM (Liang et al., )).
To isolate the effects of mask augmentation, we use a single fixed mask $M_{\texttt{tr}}$ for generation and evaluate using the Seen set of InpaintGuardBench.
As shown in Fig. [6(a)](#S4.F6.sf1), our early stage perturbation loss consistently outperforms both the other losses across most metrics.

##### Effect of mask augmentation

Additionally, we evaluate how much mask augmentation improves the protection strength on the Unseen mask set by measuring the performance of DiffusionGuard with and without mask augmentation.
Fig. [6(b)](#S4.F6.sf2) shows that mask augmentation consistently improves all metrics for the Unseen set of InpaintGuardBench, clearly demonstrating its effectiveness in enhancing mask robustness.
We provide qualitative examples in Appendix [E.1](#A5.SS1).

##### Comparison with mask-free protection

We compare DiffusionGuard with a mask-free protection method that applies a global perturbation over the entire image.
As a baseline, we use AdvDM, a mask-free protection method based on reconstruction loss (Eq. [1](#S2.E1)).
Fig. [6(c)](#S4.F6.sf3) presents the results on the Unseen set of InpaintGuardBench, showing that DiffusionGuard, by focusing on the mask region, provides significantly stronger protection than AdvDM across all metrics.
We also remark that the noise perceptibility is much lower with DiffusionGuard.
This is because the per-pixel noise threshold $\|\delta\|_{\infty}$ is identical between the two methods, but DiffusionGuard adds $\delta$ over a smaller, focused region, whereas in AdvDM $\delta$ occupies the entire image, making it more visible.

Figure: (a) Optimization loss comparison
Refer to caption: /html/2410.05694/assets/x13.png

### 4.5 Resilience against purification methods

We also evaluate the robustness of DiffusionGuard and the baseline methods against noise purification tools.
Specifically, we applied commonly used purification tools such as JPEG (Wallace, ) compression, crop-and-resize, and AdverseCleaner (Zhang, ), an algorithmic adversarial perturbation remover, to the protected images.
As visualized in Fig. [7(a)](#S4.F7.sf1), DiffusionGuard (blue) shows superior protection strength, maintaining its effectiveness even after noise removal.
This demonstrates its reliable protection in real-world scenarios, where malicious users may attempt to circumvent defenses by eliminating noise.
We include the full results for the purification experiments in Appendix [F](#A6).

### 4.6 Black-box transfer

Finally, we verify whether our method and the baselines can be black-box transferred to a different model.
Our adversarial perturbations are optimized against Stable Diffusion Inpainting (SD Inpainting), which is based on the pre-trained model weights of SD 1.2.
In this section, we test these perturbations on SD 2 Inpainting, which is based on the weights of SD 2.0 (Rombach et al., ).
Since SD 2.0 was trained from scratch and does not build on SD 1.2, it represents a different model family, making this evaluation a test of black-box transfer.
Fig. [7(b)](#S4.F7.sf2) shows the edited results for DiffusionGuard and PhotoGuard (Salman et al., ) using 4 different editing prompts ("A man in a hospital", "A man in a gym", "A man getting on a bus", "Photo of a construction worker" from left to right).
As presented, DiffusionGuard maintains its effectiveness even when transferred to another model, resulting in failed edits, while PhotoGuard loses its protection and results in successful edits that are aligned with the editing prompt.
We include the full evaluation results and comparisons with other baselines in Appendix [G](#A7).

Figure: (a) Purification comparison
Refer to caption: /html/2410.05694/assets/x16.png

## 5 Related work

##### Safety concerns of generative diffusion models

Generative diffusion models have raised public safety concerns, particularly regarding the potential misuse for generating realistic but harmful content.
There have been various studies that seek to mitigate these concerns by developing safety measures against them.
Some works focus on watermarking the images to identify manipulated images (Cui et al., ; Fernandez et al., ; Peng et al., ; Zhao et al., ).
A line of studies explore ways to remove
potentially harmful concepts (Gandikota et al., ; ; Heng & Soh, ).
Another line of research explores and identifies biases embedded in diffusion models and ways to mitigate them (Friedrich et al., ; Choi et al., ; Shen et al., ).
These safety issues need continued focus and attention to ensure the responsible use of generative diffusion models.

##### Adversarial examples against diffusion-based image editing

Several works have explored adversarial examples against diffusion-based image editing to protect images from being manipulated by these models.
AdvDM (Liang et al., ) proposed to maximize diffusion model training loss to create adversarial examples.
Xue et al. () further improved AdvDM by applying score distillation sampling (Poole et al., ).
However, most prior works did not specialize for inpainting models specifically.
PhotoGuard (Salman et al., ) considered a defense against inpainting models but did not evaluate the protection under a challenging setup such as mask robustness.

## 6 Conclusion

This work presents DiffusionGuard, a robust and effective defense method against malicious diffusion-based image editing.
We introduce a novel adversarial objective in order to target the vulnerability of inpainting models and disrupt the early stages of the denoising process, where key regions are generated.
Furthermore, we identify key limitations in existing protection methods, such as their inability to handle various shapes of masks, and address this with mask augmentation to develop a robust protection method.
By leveraging these strategies, our method achieves stronger protection and improved mask robustness with lower computational costs, when compared to several baselines.
Additionally, DiffusionGuard demonstrates robustness against various purification methods, and proves effective in black-box transfer settings.

## Ethics statement

Text-to-image diffusion models have demonstrated superior capabilities in generating and editing images based on text prompts and additional user inputs such as masks, offering significant potential across creative industries, including art, graphics, and entertainment. However, it comes with the risk of misuse, such as creating fake or deceptive visual contents. For example, malicious users can manipulate images to spread misinformation, deliberately fabricate events, or misrepresent individuals using the generated fake content.

Our work presents a defense mechanism against the malicious usage of text-to-image models, in specific, we focus on protecting sensitive areas of images from unauthorized modifications.
While our method provides robust defenses, we recognize a remaining risk that advanced adversaries may find ways to circumvent protections. As such, we highlight that the necessity of ethical guidelines and legal frameworks to prevent any malicious use.

Therefore, we have made a concerted effort to evaluate and document the limitations of our approach through rigorous testing. Specifically, we conducted purification experiments (*e.g.*, Sec. [4.5](#S4.SS5) and Appendix [F](#A6)) to assess the resilience of our method against attempts to remove adversarial noise, as well as transfer experiments (*e.g.*, Sec. [4.6](#S4.SS6) and Appendix [G](#A7)) to evaluate its generalizability under a black-box transferring scenario. These experiments demonstrate the robustness of our method
even when attackers employ noise-removal techniques or apply it to different models.

Nevertheless, we acknowledge the possibility of future threats and emphasize the future research on strengthening these defenses. We encourage open discussion and regulatory measures to mitigate the broader ethical implications of advancements in text-to-image models.

## Reproducibility statement

To ensure the reproducibility of our results, we provide detailed descriptions of our methods, datasets, and evaluation criteria in Sec. [4](#S4), Appendix [C](#A3), and Appendix [D](#A4).
Specifically, we describe how we generate adversarial perturbations, how we evaluate the protective effectiveness and the robustness of these perturbations against different test-time mask shapes, and the setup for our experiments. Additionally, we attach the executable code for adversarial perturbation generation, mask augmentation, and the evaluation suites
in the supplementary materials. Full experimental details regarding setups, used inpainting models, and the evaluation metrics are in
Sec. [4](#S4), Appendix [C](#A3) and Appendix [D](#A4).

## Acknowledgements

We thank Dongjun Lee, Dongyoon Hahm, Haeone Lee, and Daewon Chae for helping conduct the human survey experiments. We also appreciate Juyong Lee and Seojin Kim for providing feedback and reviewing the paper.

## Appendix A Mask augmentation algorithm

The full procedure of mask augmentation is summarized in Algorithm [1](#alg1).

Figure: Algorithm 1 Mask augmentation via contour shrinking

## Appendix B InpaintGuardBench

To assess the ability of a protection method to prevent unauthorized adversaries from editing an image in a challenging yet practical scenario as outlined in Sec. [3.1](#S3.SS1), we construct a benchmark out of various images, mask shapes, and edit prompt instructions.

### B.1 Dataset

#### B.1.1 Images

To take into account realistic scenarios of privacy threat posed by inpainting models, we collect 42 images consisting of 32 images of celebrities and 10 images of non-human objects. The 32 celebrity images were collected from the web, and consist of 20 front-view images and 12 side-view images of racial and domain diversity.
Out of each, 30 images are focused on faces, and 2 images focus on the body of the person.
10 non-human images were sourced from the DreamBooth (Ruiz et al., ) dataset. Out of them, 5 images contain animals, and 5 images include inanimate objects. We visualize all images that we have used in Fig. [8](#A2.F8) and all masks that we have used in Fig. [9](#A2.F9).

Figure: Figure 8: All images used in InpaintGuardBench. Best seen zoomed in.
Refer to caption: /html/2410.05694/assets/x18.png

Figure: Figure 9: All masks used in InpaintGuardBench. Best seen zoomed in.
Refer to caption: /html/2410.05694/assets/x19.png

#### B.1.2 Masks

In order to measure the robustness of a protection method against mask variations, we prepare 5 masks per image. For the first mask, we obtain a training mask $M_{\texttt{tr}}$, which determines the sensitive region (*e.g.* face, body or object) using an automated segmentation tool (Kirillov et al., ) (see Sec. [3.1](#S3.SS1) for more details about the definition of the sensitive region). This mask is used for training in both DiffusionGuard and the baselines.
For the remaining 4 masks, we handcraft 4 additional masks that contains the same sensitive region. The handcrafted masks are drawn using either circle brush or simple shapes such as rectangles or circles. Circle brush is a simple yet the most commonly used user interface (UI) to draw a mask, and it is used by popular inpainting tools such as DALL-E 3 ChatGPT integration (Betker et al., ), DALL-E 2 playground (Ramesh et al., ), or Stable Diffusion web UI.(^4^44[https://github.com/AUTOMATIC1111/stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui))

#### B.1.3 Edit text prompts

Finally, we use 10 different editing text prompts in order to take into account the robustness of each protection method against different editing prompt choices. All prompts are available in Table [2](#A2.T2) and Table [3](#A2.T3).

**Table 2: All prompts for portrait images.**
| A [man/woman] in a hospital |
| --- |
| A [man/woman] riding a motorcycle |
| A [man/woman] walking in the street |
| A [man/woman] driving a car |
| A [man/woman] dancing in a club |
| A [man/woman] dressed up in halloween costume |
| A [man/woman] in the gym |
| A [man/woman] in a gaming convention |
| A photo of a construction worker |
| A [man/woman] getting on a bus |

**Table 3: All prompts for non-portrait images.**
| A [object] in a hospital |
| --- |
| A [object] on a motorcycle |
| A [object] in the street |
| A [object] in a car |
| A [object] in a club |
| A [object] in halloween |
| A [object] in the gym |
| A [object] in a gaming convention |
| A photo of [object] at a construction site |
| A [object] on a bus |

## Appendix C Evaluation details

### C.1 Quantitative metrics

In order to quantitatively measure the protection strength of each method, we employ multiple metrics in order to measure both edit instruction fidelity and edit image quality (*i.e.* how realistic the generated image is). Because these metrics measure the degree of alignment, and our goal is to stop adversaries from obtaining desirable edits, these metrics should be lower if the protection is better.

#### C.1.1 CLIP similarity

Contrastive Language-Image Pre-training (CLIP) (Radford et al., ) is a set of vision and text encoder trained together to align vision and text representations. To measure edit instruction fidelity, we calculate the cosine similarity between the textual description $\text{CLIP}_{\texttt{text}}({y_{\texttt{edit}}})$ and the actual edited image representation $\text{CLIP}_{\texttt{image}}(\mathbf{x}_{\texttt{edit}})$, where $\mathbf{x}_{\texttt{edit}}$ is the edit result image, and CLIP is the CLIP encoder. Higher similarity scores indicate that the edit more closely aligns with the desired instruction. This metric helps us evaluate how accurately the edits reflect the specified changes.

#### C.1.2 CLIP directional similarity

CLIP directional similarity (Gal et al., ) is a metric specifically intended to measure the performance of a text-guided image editing model. Specifically, CLIP directional similarity measures the alignment between the deviation in the text space (from the source caption to the edit instruction) and the deviation in the image space (from the source to the edited result). The source caption is a caption that describes the source image and in our case, it is obtained using BLIP-Large model (Li et al., ), which is an open-source captioning model. The formulation of CLIP directional similarity can be written as follows:

$$ $\displaystyle\text{CLIP directional similarity}=\frac{(\mathbf{e}_{\text{image, edit}}-\mathbf{e}_{\text{image, source}})\cdot(\mathbf{e}_{\text{text, edit}}-\mathbf{e}_{\text{text, source}})}{\|\mathbf{e}_{\text{image, edit}}-\mathbf{e}_{\text{image, source}}\|\|\mathbf{e}_{\text{text, edit}}-\mathbf{e}_{\text{text, source}}\|}.$ $$

#### C.1.3 ImageReward

ImageReward (Xu et al., ) is a human-aligned vision-language model and a reward model, which is fine-tuned on a human preference dataset. As stated and used by several works (Ye et al., ; Fan et al., ; Black et al., ), ImageReward is suitable for evaluating edit prompt fidelity as well as overall image quality, and shows improvement especially in terms of the ability to measure prompt-image alignment.

#### C.1.4 PSNR

Peak Signal-to-Noise Ratio (PSNR) is a widely used metric to assess the similarity between two images by calculating the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the quality of its representation. In our context, PSNR is used to measure the similarity between the edit result of an unprotected clean image $\mathbf{x}_{\texttt{edit}}$ and a protected image $\mathbf{x}_{\texttt{src}}+\delta$. This serves as an indicator of how much the protection alters the edited result compared to the edited result of a clean image. PSNR is defined as follows:

$$ $\displaystyle\text{PSNR}(\mathbf{x}_{\text{edit, protected}},\mathbf{x}_{\text{edit, unprotected}})=20\cdot\log_{10}\bigg{(}\frac{\text{MAX}(\mathbf{x}_{\text{edit, unprotected}})}{\sqrt{\text{MSE}(\mathbf{x}_{\text{edit, unprotected}},\mathbf{x}_{\text{edit, protected}})}}\bigg{)}$ $$

where $\text{MAX}({\mathbf{x}_{\text{edit, unprotected}}})$ is the maximum possible pixel value of the unprotected edited result image, and MSE is the mean squared error. Lower PSNR values indicate that the edited result of the protected image is different from the edited result of the unprotected image, indicating that the protection alters the edited result of the image.

### C.2 Human survey

In order to assess the edited result of the protected images perceived by human eyes, we perform a human survey with the 2,100 edit instances from InpaintGuardBench. We collected 10,500 labels using the Amazon Mechanical Turk (Crowston, ) platform, from 5 unique human annotators for each edit instance. An edit instance is defined by a triplet of (source image, mask, edit instruction), with fixed random seed value. We draw one edit instance from each of the two methods that are compared and present them to the rater in a shuffled order. Then, the rater is instructed to choose the method with better edited result in terms of the criteria, or whether it is tie. For detailed explanation about the human survey criteria, refer to Appendix [C.2.1](#A3.SS2.SSS1).

We created the labeling interface using HTML and CSS, which is the default method accepted by Amazon Mechanical Turk. We shuffled the 2,100 edit instances in a random order and split it into 42 batches, each with 50 edit instances. Then, we distributed the 42 batches on Amazon Mechanical Turk with 5 unique annotators assigned for each batch, resulting in 10,500 annotations in total.

When aggregating the results, we applied majority voting to the collected 5 votes per each edit instance and counted all ambiguous, tied cases as ties. Additionally, we report the 95% confidence interval of the voting results, as visualized as error bars in Fig. [5(a)](#S4.F5.sf1).

Figure: Figure 10: The human survey labeling user interface, with the instruction given to the annotators included.
Refer to caption: /html/2410.05694/assets/x20.png

#### C.2.1 Human survey criteria

The purpose of the protection is to prevent adversaries from achieving desired edit results that are aligned with their edit instructions, and are natural and realistic enough to spread malicious information. In order to assess this, we ask raters to choose the edit result that is better in terms of the following criteria and count the cases where a given method was not chosen (*i.e.* had worse results) as a winning case. The actual instruction given to the raters are visualized in Fig. [10](#A3.F10).

- •
Overall image quality: Raters are instructed to assess how natural, realistic, and high-quality the edited image is.
- •
Edit prompt fidelity: Raters are instructed to assess how aligned the edit result image and the edit prompt are.

Due to the need for a large quantity of labels, we optimized the annotation process based on feedback from human annotators. One major issue was confusion when asked to choose a worse image, leading to slower labeling and less accurate annotations. As a result, we decided to ask annotators to select the better edit instance, counting the cases where a method was not chosen as the winner. Details on the win rate computation are provided in Appendix [C.2](#A3.SS2).

#### C.2.2 Baseline for human survey

For the baseline, we choose PhotoGuard (Salman et al., ) as our baseline, as (1) PhotoGuard achieves the best result overall in terms of quantitative metrics as presented in Table [1](#S4.T1), Fig. [6](#S4.F6), and Fig. [4](#S3.F4), which is also visually notable, and (2) PhotoGuard proposed to target the diffusion model in a mask-dependent manner, which is more aligned with our setup outlined in Sec. [3.1](#S3.SS1), allowing a fairer comparison in contrast to other baselines, which are not necessarily mask-specific.

## Appendix D Experimental details

In this section, we outline the experimental details of our experimental setup for reproducibility. We conduct all our experiments on a single NVIDIA H100 80GB HBM3 GPU. For fair comparison, we match the time taken for running PGD optimization to 90 seconds in all comparisons throughout the paper. Additionally, we fix the random seed for a reliable comparison of the edited results of different methods, and we also follow the same projected gradient descent (PGD) (Madry et al., ) optimization configuration proposed by each method.

## Appendix E More experimental results

### E.1 More editing results

In this section, we include additional editing results using DiffusionGuard. We attach the additional editing results in Fig. [20](#A8.F20), Fig. [21](#A8.F21), Fig. [22](#A8.F22), Fig. [23](#A8.F23).

**Table 4: Results of each protection method using various noise strength threshold values. Our method achieves strong protection, exhibiting best protection strength in every evaluation even under tighter noise budget, in both Seen and Unseen set, across all metrics. Noticeably, the gap between our method and the baselines grows as noise budget gets tighter, showing that our method is more robust under limited noise budget scenario. Lower metric is better, indicating failed edits.**
| $|\delta|_{\infty}$ | Method | PSNR $\downarrow$ | CDS $\downarrow$ | IR $\downarrow$ | CS $\downarrow$ | PSNR $\downarrow$ | CDS $\downarrow$ | IR $\downarrow$ | CS $\downarrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | Seen (1 Mask, Train set) | Unseen (4 Masks, Test set) |  |  |  |  |  |  |
| $\nicefrac{{16}}{{255}}$ | PhotoGuard | 12.87 | 21.17 | -1.537 | 27.89 | 14.53 | 23.30 | -1.357 | 30.30 |
| AdvDM | 13.62 | 23.77 | -1.438 | 30.04 | 13.37 | 24.27 | -1.361 | 30.97 |  |
| Mist | 14.22 | 24.25 | -1.368 | 30.27 | 14.51 | 23.93 | -1.307 | 30.79 |  |
| SDS(-) | 14.44 | 23.42 | -1.337 | 29.89 | 14.32 | 23.85 | -1.237 | 30.78 |  |
| Ours | 12.60 | 18.95 | -1.807 | 26.55 | 13.19 | 21.84 | -1.557 | 29.05 |  |
| $\nicefrac{{12}}{{255}}$ | PhotoGuard | 13.19 | 21.79 | -1.499 | 28.42 | 15.07 | 23.32 | -1.365 | 30.26 |
| AdvDM | 13.99 | 24.10 | -1.404 | 30.19 | 14.02 | 24.27 | -1.302 | 30.84 |  |
| Mist | 14.87 | 23.96 | -1.374 | 30.04 | 15.05 | 24.13 | -1.281 | 30.75 |  |
| SDS(-) | 15.07 | 23.85 | -1.333 | 29.92 | 14.88 | 23.96 | -1.207 | 30.64 |  |
| Ours | 12.78 | 20.08 | -1.762 | 27.15 | 13.55 | 22.13 | -1.499 | 29.25 |  |
| $\nicefrac{{8}}{{255}}$ | PhotoGuard | 13.67 | 22.25 | -1.475 | 28.76 | 15.84 | 23.64 | -1.337 | 30.29 |
| AdvDM | 14.89 | 24.15 | -1.386 | 30.02 | 15.01 | 24.18 | -1.302 | 30.71 |  |
| Mist | 15.84 | 24.10 | -1.383 | 30.02 | 16.01 | 23.90 | -1.313 | 30.50 |  |
| SDS(-) | 15.66 | 24.09 | -1.369 | 29.87 | 15.58 | 23.92 | -1.250 | 30.58 |  |
| Ours | 13.12 | 20.53 | -1.674 | 27.43 | 14.12 | 22.65 | -1.455 | 29.48 |  |
| $\nicefrac{{6}}{{255}}$ | PhotoGuard | 14.23 | 23.18 | -1.432 | 29.35 | 16.55 | 23.57 | -1.341 | 30.26 |
| AdvDM | 15.83 | 24.49 | -1.409 | 30.22 | 15.88 | 24.08 | -1.308 | 30.64 |  |
| Mist | 16.54 | 24.42 | -1.367 | 30.01 | 16.75 | 24.01 | -1.321 | 30.52 |  |
| SDS(-) | 16.22 | 24.14 | -1.341 | 29.82 | 16.31 | 23.92 | -1.279 | 30.45 |  |
| Ours | 13.63 | 21.65 | -1.594 | 28.05 | 14.79 | 22.93 | -1.415 | 29.66 |  |
| $\nicefrac{{4}}{{255}}$ | PhotoGuard | 15.00 | 23.77 | -1.397 | 29.58 | 17.59 | 23.78 | -1.315 | 30.40 |
| AdvDM | 17.37 | 24.61 | -1.398 | 30.10 | 17.62 | 24.21 | -1.297 | 30.66 |  |
| Mist | 17.54 | 24.41 | -1.392 | 29.97 | 17.98 | 24.00 | -1.327 | 30.50 |  |
| SDS(-) | 17.36 | 24.09 | -1.361 | 29.94 | 17.35 | 23.95 | -1.298 | 30.53 |  |
| Ours | 14.55 | 22.68 | -1.533 | 28.68 | 15.87 | 23.16 | -1.387 | 29.92 |  |

Figure: Figure 11: Image editing results of each protection method using varying noise budget ($\|\delta\|_{\infty}$) values (Seen mask). Editing prompt is "A man in a hospital".
Refer to caption: /html/2410.05694/assets/x21.png

Figure: Figure 12: Image editing results of each protection method using varying noise budget ($\|\delta\|_{\infty}$) values (Seen mask). Editing prompt is "A woman in a gaming convention".
Refer to caption: /html/2410.05694/assets/x22.png

### E.2 Additional analysis under scenarios with limited resources

We compare our method to the baseline method in Sec. [4](#S4) by optimizing the adversarial perturbation using under two different scenarios with limited resources. The two scenarios were using tighter noise strength threshold (*i.e.* noise budget), and using constrained optimization time (*i.e.* compute budget).

Verifying whether a protection method is still effective under such constraints is crucial, as they are closely related to the usefulness in realistic scenarios.
For instance, a stricter noise budget plays a crucial role in generating adversarial perturbations that are both subtle and imperceptible to human eyes or automated detection systems. This ensures that the noises remain less noticeable and at the same time preserve the quality and integrity of the original image, making the protection method more applicable in practical scenarios.
Tighter compute budget, on the other hand, is vital for developing a cost-effective defense measure as the defender likely has to protect a large number of images in real-life scenarios, where computational resources are limited and deploying efficient defenses quickly becomes critical for maintaining security without compromising the protective effectiveness.

#### E.2.1 Limited noise budget scenario

In this section, we conduct a comprehensive evaluation of DiffusionGuard and all baseline methods using 5 different noise strength threshold values, and present the evaluation results in Table [4](#A5.T4).
As shown in the table, DiffusionGuard (noted as "Ours" in the table, highlighted in grey color) results in the best protection strength in every case across all noise perturbation threshold and across all evaluation metrics, providing better stealthiness compared to the baseline methods. We also visualize the qualitative results for each noise budget value in Fig. [11](#A5.F11) and Fig. [12](#A5.F12), in which DiffusionGuard sustains its protective strength even when the noise budget gets tighter (*i.e.* lower), while other methods quickly lose protection and results in successful edits as the noise budget value decreases. This makes DiffusionGuard especially more useful in real-life applications, in which small noise threshold is crucial for a less detectable image protection and for preserving the original image quality.

**Table 5: Results of each protection method under various compute budget values. The compute budget is defined by the time limit of PGD optimization, given in seconds. Our method achieves strong protection, exhibiting best or second-to-best protection strength in most evaluations even under tighter compute budget, in both Seen and Unseen set. All methods were trained using constraint of $|\delta|_{\infty}=\nicefrac{{16}}{{255}}$. Lower metric is better, indicating failed edits.**
| second-to-best |
| --- |

#### E.2.2 Limited compute budget scenario

In this section, we evaluate DiffusionGuard and all baseline methods by applying projected gradient descent (PGD) optimization with varying time constraints for each method.
Specifically, we optimize DiffusionGuard and all baseline methods with 5 different time durations, and present the evaluation results in Table [5](#A5.T5). As shown in the table, DiffusionGuard ("Ours" in the table, highlighted in grey color) results in the best or the second best protection effectiveness in most cases across all most PGD optimization durations and across all evaluation metrics, exhibiting better resource efficiency compared to the baseline methods. This makes DiffusionGuard more useful in practical scenarios where multiple images need to be protected using a limited amount of computational resources.

### E.3 Additional analysis on mask region

In this section, we conduct additional analysis to further explore the behavior and the protective effectiveness of the methods related to mask regions selected during generation of the adversarial perturbation.

#### E.3.1 Using DiffusionGuard and baseline methods with a fixed mask

We evaluate whether the loss function of DiffusionGuard provides better protective effectiveness compared to the baseline methods when ruling out the influences of the mask, *i.e.* by using the same, fixed mask during optimization. Specifically, we fix the mask used for optimization to $M_{\texttt{tr}}$ for all methods including DiffusionGuard and all baseline methods, and assess the performance on the Seen mask set (which only consists of $M_{\texttt{tr}}$).

Baseline methods like AdvDM (Liang et al., ), and methods that build on top of AdvDM such as Mist (Liang & Wu, ) or SDS(-) (Xue et al., ) originally propose to add a perturbation over the entire image, without considering specific mask regions, as discussed in Sec. [4](#S4). For this analysis, however, we adapt these methods to apply perturbations only within the mask region $M_{\texttt{tr}}$. Similarly, we modify DiffusionGuard by removing the mask augmentation component and fixing the mask to $M_{\texttt{tr}}$, as done for the baseline methods. This ensures a fair comparison by isolating the impact of the loss function after eliminating the mask factor. Note that no modifications were made to PhotoGuard (Salman et al., ), as it already applies perturbations exclusively to the mask region $M_{\texttt{tr}}$.

Figure: Figure 13: Evaluation results on the Seen mask set after generating adversarial perturbation using each method with $M_{\texttt{tr}}$ mask. Note that Seen mask set consists only of $M_{\texttt{tr}}$ mask. We report PSNR, CLIP directional similarity (CDS), ImageReward (IR), and CLIP similarity (CS). Lower metrics indicate failed edits, representing better protection effectiveness of the defense method.
Refer to caption: /html/2410.05694/assets/x23.png

The evaluation results are visualized in Fig. [13](#A5.F13).
As shown in the figure, DiffusionGuard achieves significantly better protective effectiveness compared to the baseline methods across most metrics, verifying that even when the mask factor is isolated, DiffusionGuard exhibits the most effective protection strength.

#### E.3.2 Using baseline methods in combination with mask augmentation

In this section, we evaluate the performance of each baseline method when combined with mask augmentation, a component of DiffusionGuard introduced in Sec. [3](#S3). As discussed in Appendix [E.3.1](#A5.SS3.SSS1), baseline methods can be adapted to use different masks. For AdvDM, Mist, and SDS(-), we modified them to add adversarial perturbations only within the mask region given by $M_{\texttt{tr}}$. The purpose of this experiment was to analyze the effects of the loss function exclusively, after ruling out the mask-related factors. This section extends the previous analysis by applying the same mask augmentation algorithm from DiffusionGuard during the optimization process of all methods, while preserving their original loss functions.

As illustrated in Fig. [14](#A5.F14), DiffusionGuard demonstrates superior performance compared to the baseline methods when the baselines are used in combination with mask augmentation. These experimental results, combined with those presented in Appendix [E.3.1](#A5.SS3.SSS1) (where the mask is fixed to $M_{\texttt{tr}}$ to isolate the effects of the loss function), verify that (1) the early-stage perturbation loss proposed by DiffusionGuard provides the strongest protection, and (2) both components of DiffusionGuard work synergistically to provide effective protection against diffusion-based malicious image editing. We note that no changes were made to DiffusionGuard in this experiment, as it already incorporates mask augmentation.

Figure: Figure 14: Evaluation results on the Unseen mask set after generating adversarial perturbation using each method in combination with mask augmentation. We report PSNR, CLIP directional similarity (CDS), ImageReward (IR), and CLIP similarity (CS). Lower metrics indicate failed edits, representing better protection effectiveness of the defense method.
Refer to caption: /html/2410.05694/assets/x24.png

#### E.3.3 DiffusionGuard with and without mask augmentation

In this section, we report a detailed analysis of the effect of mask augmentation on the performance of DiffusionGuard, specifically the performance of DiffusionGuard with and without mask augmentation in both Seen and Unseen sets of InpaintGuardBench. The results are illustrated in Fig. [15](#A5.F15). As visualized, while mask augmentation does not have a notable impact or only marginally improves the performance of the protection in the case of Seen masks, it significantly improves the protection in the case of Unseen masks.

Figure: Figure 15: Evaluation results of DiffusionGuard and DiffusionGuard after removing mask augmentation. We evaluate using the Seen set (left) and the Unseen set (right) of InpaintGuardBench. We report PSNR, CLIP directional similarity (CDS), ImageReward (IR), and CLIP similarity (CS). Lower metric indicates failed edits, representing better protection effectiveness of the defense method.
Refer to caption: /html/2410.05694/assets/x25.png

**Table 6: Editing results of each protection method after applying adversarial noise purification with various algorithms. Our method achieves strong protection, exhibiting best or second-to-best protection strength in most evaluations even after applying noise purification, in both Seen and Unseen set, across all metrics. All methods were optimized using the constraint of $|\delta|_{\infty}=\nicefrac{{16}}{{255}}$. Lower metric is better, indicating failed edits.**
| second-to-best |
| --- |

Figure: Figure 16: Image editing results after applying purification method to images protected using each method (Unseen mask). Editing prompt is "A man dancing in a club".
Refer to caption: /html/2410.05694/assets/x26.png

Figure: Figure 17: Image editing results after applying purification method to images protected using each method (Seen mask). Editing prompt is "A man dancing in a club".
Refer to caption: /html/2410.05694/assets/x27.png

## Appendix F Protection resilience against noise purification

In this section, we report the protection effectiveness of each method after applying purification methods. It is known that adversarial perturbations can be "purified" by applying modification (*e.g.* JPEG compression, crop-and-resize) to the image which contains an adversarial perturbation. Because adversarial perturbations added by image protection methods are optimized in pixel-space using the PGD algorithm, any operation that modifies the image may impair the adversarial perturbation and decrease the protection effectiveness. Commonly used methods include the following methods.

- •
Crop-and-resize: The image is cropped at the center and is resized up to match the original resolution.
- •
JPEG compression: A universally adopted algorithmic image compression method. It accepts a quality parameter ranging from 0 to 100, where lower values result in worse image quality (hence greater noise removal and stronger purification).
- •
AdverseCleaner (Zhang, ): An algorithmic filter which removes adversarial perturbation targeted at diffusion models.

In this section, we conduct additional experiments extending from Sec. [4](#S4) to evaluate DiffusionGuard and the baseline methods after processing the protected images using each of the purification methods and present the full results in Table [6](#A5.T6).
As presented, As shown in the table, DiffusionGuard (noted as "Ours" in the table, highlighted in grey color) results in the best or the second best protection strength in most cases across all most PGD optimization durations and across all evaluation metrics, exhibiting better resource efficiency compared to the baseline methods.
Additionally, we include the qualitative results for each protection method and each purification method in Fig. [16](#A5.F16) and Fig. [17](#A5.F17) (edited using Unseen and Seen in order).
As shown in the figures, DiffusionGuard is resilient against various noise purification methods compared to the baselines, maintaining its protective strength even after a malicious user attempts to remove the adversarial perturbation.

**Table 7: Editing results of each protection method when black-box transferred to Stable Diffusion 2.0 Inpainting. Our method achieves strong protection, exhibiting best or second-to-best protection strength in most evaluations even when transferred to a different model, in both Seen and Unseen set, across all metrics. All methods were trained using constraint of $|\delta|_{\infty}=\nicefrac{{16}}{{255}}$. Lower metric is better, indicating failed edits.**
| second-to-best |
| --- |

Figure: Figure 18: Black-box transfer to Stable Diffusion 2.0 Inpainting from Stable Diffusion Inpainting, comparison between DiffusionGuard and baselines (Unseen mask). Editing prompts for each row are "A man in a hospital", "A man in the gym", "A photo of a construction worker", and "A man getting on a bus" in order.
Refer to caption: /html/2410.05694/assets/x28.png

Figure: Figure 19: Black-box transfer to Stable Diffusion 2.0 Inpainting from Stable Diffusion Inpainting, comparison between DiffusionGuard and the baseline methods (Seen mask). Editing prompts for each row are "A man in a hospital", "A man in the gym", "A photo of a construction worker", and "A man getting on a bus" in order.
Refer to caption: /html/2410.05694/assets/x29.png

## Appendix G Transferability to black-box models

In this section, we conduct additional experiments to assess the black-box transfer effectiveness of DiffusionGuard and the baseline methods.
Specifically, we use Stable Diffusion Inpainting 1.0 (Rombach et al., ) for generating adversarial examples, and test them on Stable Diffusion 2.0 Inpainting.
We note that Stable Diffusion Inpainting 1.0 is a fine-tuned checkpoint of Stable Diffusion 1.2, and Stable Diffusion 2.0 Inpainting is a fine-tuned checkpoint of Stable Diffusion 2.0.
Because Stable Diffusion 2.0 was pre-trained from scratch independently from the weights of Stable Diffusion 1.2, the weights of the two inpainting models are significantly different, making this evaluation setup a black-box transfer setting.
The full evaluation results are presented in Table [7](#A6.T7). As shown in the table, DiffusionGuard exhibits strong protection when transferred to a different model (*i.e.* used against a different editing model), achieving either best or second-to-best value in most cases across both Seen and Unseen of InpaintGuardBench.
We also present the qualitative results in Fig. [18](#A6.F18) and Fig. [19](#A6.F19), respectively for Unseen and Seen masks. As visualized, DiffusionGuard maintains its protective effectiveness when transferred to a different model, while baseline methods fail to protect the image and results in successful edits.

## Appendix H Limitation

There are several limitations and interesting future directions in our work:

- •
Black-box setups: Although we demonstrate the effectiveness of DiffusionGuard in black-box settings in Sec. [4.6](#S4.SS6) and Appendix [G](#A7), further investigations are required against more advanced closed models, such as DALL-E 3 (Betker et al., ).
- •
Extension to personalization: Text-to-image diffusion models have shown remarkable success in generating personalized subjects based on a few reference images (Ruiz et al., ).
Because such personalized models can be misused to generate harmful content, developing defense methods against personalization methods would be an important direction for future research.

Figure: Figure 20: Edited results after using DiffusionGuard and the baseline protection methods (all masks shown). Editing prompt is "A man walking in the street".
Refer to caption: /html/2410.05694/assets/x30.png

Figure: Figure 21: Edited results after using DiffusionGuard and the baseline protection methods (all masks shown). Editing prompt is "A woman in a hospital".
Refer to caption: /html/2410.05694/assets/x31.png

Figure: Figure 22: Edited results after using DiffusionGuard and the baseline protection methods (all masks shown). Editing prompt is "A man in a gym".
Refer to caption: /html/2410.05694/assets/x32.png

Figure: Figure 23: Edited results after using DiffusionGuard and the baseline protection methods (all masks shown). Editing prompt is "A man in a gaming convention".
Refer to caption: /html/2410.05694/assets/x33.png

## Peer Reviews

confidence: 2
contribution: 2
initial_rating: 3
presentation: 3
questions: See weaknesses.
soundness: 2
strengths: - The proposal of mask augmentation is sensible.
- DiffusionGuard outperforms the baselines in all metrics. Qualitative figures show that it often causes the inpainting models to generate plain and blurry inpainted output backgrounds.
summary: The paper proposes DiffusionGuard, an image-cloaking algorithm to defend against malicious diffusion-based text-guided inpainting. Compared to previous works, it has two main proposals. First, instead of optimizing any denoising step using either image-space loss or reconstruction loss, DiffusionGuard only optimizes at the early stage (t = T) and aims to increase the norm of the noise. Second, it employs mask augmentation to improve the robustness of the proposed algorithm for different mask variations at test time. Experiments verified that DiffusionGuard outperforms the previous baselines on this task.
weaknesses: - The title is misleading. The work only focuses on diffusion-based text-guided inpainting, e.g., Stable Diffusion Inpainting. It does not consider other diffusion-based image editing methods such as Instruct-Pix2Pix, MasaCtrl... The authors should revise the title to better specify the scope of the work.
- The work only tests with Stable Diffusion Inpainting variants. Recent inpainting models, e.g., MagicBrush [1], should be mentioned and tested.
- L191-200: the mentioned "unique behavior" of inpainting models sounds misleading. In the early denoising stage, the fine details only appear on the unchanged region, which is basically copied from the input. The inpainting regions, i.e., the background, still do not have fine details and behave as in normal diffusion models.
- L200: The reason for targeting the early steps is not convincing. From the presented results, the proposed method affects only the inpainting regions outside of the face, which have similar behavior as in normal diffusion models. In the ablation studies, the author should add an extra experiment to test the case when Eq (4) is applied in all time steps instead of only in the early one.
- In mask augmentation, the mask is shrunk to be smaller. What happens if the mask used at test time is bigger?
- The PSNR metric used in Table 1 and Fig. 5 is not reliable. Given the same image and mask, we can have different editing results that match the input prompt. Hence, a small PSNR does not necessarily imply a successful defense; good editing can still produce a low PSNR score.
- The test set is small, with only 42 images. It is better to test on a much larger set of images.
- The authors ran experiments with 5 masks per testing image. From Fig.4, the masks are pretty similar; hence, the effect of changing the mask is not significant. I would trade the number of masks and prompts to have more testing images.
- The authors should provide a qualitative figure showcasing the cloaked images to see whether the added noise is obvious or not. Quantitative numbers (PSNR, SSIM) for it are also recommended.
- Fig.4: The first 3 examples are very good; the inpainted backgrounds are plain and blurry. However, the last example does not show that behavior. The authors should explain why. Fig.5a confirms that DiffusionGuard is not always that good and still loses to Photoguard 22-25% of the time.

[1]. Zhang, K., Mo, L., Chen, W., Sun, H. and Su, Y., 2024. Magicbrush: A manually annotated dataset for instruction-guided image editing. Advances in Neural Information Processing Systems, 36.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Will different editing prompts have effects on the results? Are the perturbations generated with a single prompt or several different text prompts?

For other questions, please see the Weaknesses part.
soundness: 3
strengths: 1. The observations, that the inpainting models produce fine details of masked region at early steps, are interesting and insightful. 

2. Using augmented masks is a reasonable and effective method to improve robustness.

3. The paper proposes a benchmark to evaluate different methods. Extensive results show the effectiveness and robustness of the method.
summary: The paper proposes a effective and robust method against malicious diffusion-based image editing. The method is interesting and insightful. With the proposed benchmark, the paper shows the superior results compared to baseline methods.
weaknesses: There are two main concerns.

1. Did the authors try some specifically designed purification methods for such perturbations in diffusion models? Such as the method in [1].

2. Only focusing on mask-based image editing may be a little limited. Currently many editing methods do not require such masks, such as InstructPix2Pix[2]. Can the proposed method be used in these methods? Will the proposed method still be more effective and robust?




[1] Bochuan Cao et al. IMPRESS: Evaluating the Resilience of Imperceptible Perturbations Against Unauthorized Data Usage in Diffusion-Based Generative AI, 2023.

[2] Tim Brooks et al. InstructPix2Pix: Learning to Follow Image Editing Instructions, 2023.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: Please refer to the weaknesses.

If the authors address my concerns during the rebuttal, I would be open to adjusting my score.
soundness: 4
strengths: 1. An insight is provided that inpainting models generate fine details during the very early stages of the denoising process.
2. A new objective specifically designed to prevent image inpainting is introduced.
3. A benchmark is introduced.
summary: This work reveals that inpainting models generate fine details in the very early stages of the denoising process, leading to the development of a defense method against unauthorized image inpainting. A mask augmentation technique is proposed to enhance robustness. Additionally, a benchmark is introduced to evaluate the effectiveness of protection against unauthorized image inpainting.
weaknesses: 1. The mask augmentation is achieved by shrinking the contours inward. If malicious users provide masks larger than those used during training, will this affect performance?

2. The diffusion model's sampling can begin from different timesteps, and various sampling schedulers may start at different timesteps. For example, when sampling with 50 steps of DDIM, T is typically around 981, whereas for 25 steps of DPM-Solver, T might be around 961. If the user uses a different sampler from the one used during training, or the same sampler but starts from a different timestep T, will the proposed algorithm still work in this case?

3. The problem setting may be somewhat narrow. While the title suggests it is "against image editing," the method is only effective for a specific type of editing—image inpainting. It remains unclear whether the method can prevent other forms of editing that don't involve masks, such as instruction-guided editing [1][2].

4. Several recent references [3,4,5] on harmful concept removal are missing.

---
[1] InstructPix2Pix: Learning to Follow Image Editing Instructions

[2] MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing

[3] One-dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications

[4] MACE: Mass Concept Erasure in Diffusion Models

[7] Separable Multi-Concept Erasure from Diffusion Models

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. In line 242, the author wrote "by minimizing the following loss ...". Should it be minimizing or maximizing? I was a little bit confused. I hope the author could clearify this.
2. The author proposed a new objective or loss function for the PGD-like attack. Could the author tell me why you choose to maximize the L2 norm of the predicted noise in the earlier step? Why not L1 or total variation or focal norm? I didn't see any explaination of why max(L2) works.
3. The author didn't show that the influence of different early step `t`. For example, from 1 to 10, there is no ablation about it. I'd like to see what's the influence of different `t`, and why.
4. Although the author has done the ablation for attack steps (comp. budget), the author didn't do the ablation for learning rate choice. Some previous papers mentioned the larger learning rate choice may cause better performance when attacking the generation task in some cases, which is counter-intuitive. I hope the author can do the ablation for this as well, to find the best hyperparameters settings.
5. regarding the line 175 to 183, the authors mention that they only apply perturbations in sensitive areas most commonly used by malicious users. This is reasonable in most cases, but if a malicious user only wants to edit the eyes in a face photo, and wants to change the shape of the pupil or the position where the line of sight is focused, will inpainting be successful in this case? I would be inclined to think that editing or inpainting would still succeed in this case.
    Therefore, I would like to ask the author to make a demo. For example, in a face image, the facial features are masked separately for attack, and then a perturbation composed of several sub-perturbations will be obtained. I want to see how effective a certain sub-perturbation is in this case. Whether the inpainting of the sub-entity will be successful and whether the editing of the entire entity will be successful.
    I expect the author can take series of experiments and show me the results, and it would be better if it could be analyzed quantitatively.
6. Some figures are too tiny to read such as Figure 7a, author may prefer to make the 7a wider to have a better visualization.
soundness: 3
strengths: 1. The author captured the key problem that the global adversarial perturbation will loss its adversarial semantic in inpainting due to the mask
2. The proposed method only need to run one step of U-Net in each attack step
3. The experiment is quite comprehensive
summary: The author proposed an attack method that targeting the LDM-based inpainting task. The method came with a new loss fucntion, and a new data agumentation for inpainting mask. The author also proposed a new benchmark for evaluate the anti-inpainting methods. The experiments showed the good performance. Generally it's a complete work.
weaknesses: 1. The motivation for the loss is not clear
2. There is no ablation study for the hyperparameters

More details see questions

Meta-review: This work introduces a novel adversarial noise-based defense method designed to protect images from unauthorized edits by diffusion-based image editing techniques. The authors propose an objective that targets the early stages of the diffusion process to enhance attack performance, complemented by a mask-augmentation technique that further improves robustness. The reviewers unanimously support the paper's acceptance in positive ratings (i.e., 6.0 on average), recognizing its contribution to addressing the challenges of diffusion-based image manipulation with an efficient and robust protection method. They also commend the innovative approach of crafting adversarial noise by focusing on the early denoising stages. Additionally, reviewers WYYe, QU4Q, and ScsB highlight the effectiveness of the mask augmentation method in enhancing robustness during test time.

Based on these positive evaluations, we have decided to accept the paper.

Decision: Accept (Poster)