SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Bochuan Cao
bccao@psu.edu
Ting Wang
inbox.ting@gmail.com
University of Illinois, Urbana Champaign
Changjiang Li
meet.cjli@gmail.com
Jinyuan Jia
Jinghui Chen
jzc5917@psu.edu
Diffusion-based image generation models, such as Stable Diffusion or DALL-E 2, are able to learn from given images and generate high-quality samples following the guidance from prompts. For instance, they can be used to create artistic images that mimic the style of an artist based on his/her original artworks or to maliciously edit the original images for fake content. However, such ability also brings serious ethical issues without proper authorization from the owner of the original images. In response, several attempts have been made to protect the original images from such unauthorized data usage by adding imperceptible perturbations, which are designed to mislead the diffusion model and make it unable to properly generate new samples. In this work, we introduce a perturbation purification platform, named IMPRESS, to evaluate the effectiveness of imperceptible perturbations as a protective measure. IMPRESS is based on the key observation that imperceptible perturbations could lead to a perceptible inconsistency between the original image and the diffusion-reconstructed image, which can be used to devise a new optimization strategy for purifying the image, which may weaken the protection of the original image from unauthorized data usage (e.g., style mimicking, malicious editing). The proposed IMPRESS platform offers a comprehensive evaluation of several contemporary protection methods, and can be used as an evaluation platform for future protection methods.
Diffusion-based image generation models, e.g., Stable Diffusion (Rombach et al., 2022b) and DALL-E 2 (Ramesh et al., 2022), have gained increasing attention due to their exceptional performance in synthesizing high-quality samples by leveraging given images. Despite their superior performance, existing studies (Shan et al., 2023) showed that they could be abused to generate new images without proper authorization from the original data owner. For instance, they could be used to learn from the original artworks of a specific artist and generate artistic images that mimic his/her style without authorization, which may lead to violation of intellectual property or copyrights (Dixit, 2023; Joseph Saveri Law Firm LLP, 2023; Edwards, 2022). They can also be used to maliciously edit the images
Task:
Sample1:
Sample2:
Style Mimicking, Aim Artist: Claude Monet
Protected Image
Purified Image
Figure 1: Examples of imperceptible perturbations that are less effective in preventing unauthorized data usage after being purified by IMPRESS. Left: Stable diffusion generated images mimicking the style of Claude Monet using clean images, GLAZE (Shan et al., 2023) protected images, and IMPRESS purified images for model fine-tuning. Right: images edited by Stable diffusion using the prompt "A person in an airplane" with clean images, PhotoGuard (Salman et al., 2023) protected images, and IMPRESS purified images as model input.
Image Editing, Prompt: "A person in an airplane"
of celebrities downloaded from the Internet to create disinformation (Pranav Dixit, 2023; Li et al., 2022a). Those unauthorized data usage without the consent from the original data owner creates severe ethical concerns and causes profound societal harm (Shan et al., 2023).
To prevent unauthorized data usage, some recent studies (Shan et al., 2023; Salman et al., 2023) proposed to add imperceptible perturbations to images such that the latent diffusion model (LDM) (Rombach et al., 2022b), a state-of-the-art diffusion-based image generation model, is not able to leverage perturbed images to generate high-quality samples. For instance, PhotoGuard (Salman et al., 2023) proposed to add imperceptible perturbations to the images such that LDMs cannot generate realistic images when attempting to edit upon the perturbed image. GLAZE (Shan et al., 2023) proposed to add imperceptible perturbations to the artworks of an artist such that the LDMs cannot learn from perturbed images to generate samples that mimic the style of the artist. These works share a common trait: they all aim to add imperceptible perturbations to the original images to protect them from being "correctly" processed by the LDMs without proper authorization.
Although those protection methods (Shan et al., 2023; Salman et al., 2023) show some promising demos on preventing unauthorized data usage, we still lack a systematic understanding of their robustness and practical performances. For instance, since the perturbations added by those methods are imperceptible, we could potentially purify a perturbed image to disable its perturbation.
In this work, we aim to bridge the gap by conducting a systematic examination of those protection methods. We argue that this examination is essential from two aspects. First, the effectiveness of those protection methods under strong, adaptive perturbation purification is unknown. As a result, the examination could help users obtain a better understanding on the effectiveness of those protection methods in different scenarios. Second, the examination could also shed light on the design of new protection methods. The developed system can also be used as an evaluation platform for testing the effectiveness of future protection methods.
Our key observation is that the current "successful" imperceptible perturbations usually lead to a perceptible inconsistency between the original image and the diffusion-reconstructed image (i.e., the image-to-image output of the LDM without given any prompt), which can be used to devise a new optimization strategy for purifying the image and disabling the added perturbation. Specifically, we derive two conditions that the purified image should satisfy: similarity condition and consistency condition. The similarity condition suggests that the purified image should be visually close to the perturbed image, and thus visually close to the original image (since the perturbation is imperceptible). The consistency condition means that the reconstructed image by a LDM for the purified image should be similar to itself. This condition stems from the observations that the clean image can reconstruct the original image and satisfy such consistency condition while the perturbed image cannot. We defer the detailed derivation of those two conditions to Section 4. Given those two conditions, we respectively design two losses to quantify them, enabling us to formulate the purification of a
perturbed image as an optimization problem. Finally, we solve the optimization problem and obtain the purified image. In summary, our contributions are as follows:
- We have explored and analyzed existing protection methods for unauthorized data usage in diffusion models and pointed out the potential risks. We found that existing methods heavily rely on adding an imperceptible perturbation. Yet such an imperceptible perturbation usually leads to a perceptible inconsistency between the original image and the diffusion-reconstructed image.
- Based on the empirical findings, we propose IMPRESS, i.e., IMperceptible Perturbation REmoval SyStem, a new platform for evaluating the effectiveness of imperceptible perturbations as a protective measure by purifying a perturbed image with consistency-based losses.
- We also test the effectiveness of adaptive protection designs which also feature our consistency-based loss upon those existing protection methods. We find that it is difficult to generate effective imperceptible perturbations through such a simple adaptive design, highlighting the need for more advanced methods to prevent unauthorized data usage in LDMs.
Diffusion-based Image Generation Systems. Recently, Diffusion Probabilistic Models (DPMs) (Ho et al., 2020) have achieved impressive results in the field of image generation (Croitoru et al., 2023), including applications such as unconditional image generation(Ho et al., 2020; Nichol and Dhariwal, 2021; Song et al., 2020a), text-to-image synthes (Shi et al., 2022; Rombach et al., 2022a; Sahara et al., 2022b), image-to-image translation (Wolleb et al., 2022; Li et al., 2022b; Wang et al., 2022; Zhao et al., 2022; Sahara et al., 2022a), image editing (Avrahami et al., 2022; Meng et al., 2021), and image inpainting(Lugmayr et al., 2022; Bugeau and Bertalmio, 2009). DPMs typically employ a U-Net (Ronneberger et al., 2015) architecture as the underlying neural backbone, which naturally adapts to image-like data (Dhariwal and Nichol, 2021; Ho et al., 2020; Song et al., 2020b; Ronneberger et al., 2015).
Latent Diffusion Models. In image generation tasks, training and evaluating DPMs in the original image feature space can lead to low inference speed and high training costs. Recent works have been trying to address this issue by using advanced sampling strategies (Song et al., 2020a; Kong and Ping, 2021; San-Roman et al., 2021), hierarchical approaches (Ho et al., 2022; Vahdat et al., 2021), and feature compression strategy (Rombach et al., 2022b). Among them, latent diffusion model (Rombach et al., 2022b) employs a pre-trained image encoder and decoder, enabling the DPM to work in a compressed, low-dimensional latent space, thus reducing the computational cost of training and accelerating inference speed while maintaining almost the same quality of synthesized images.
Unauthorized Data Usage in Diffusion Models. The powerful image generation and editing capabilities of AI have also raised ethical concerns, including malicious image editing (Goodfellow et al., 2014; Mirza and Osindero, 2014; Salimans et al., 2016; Isola et al., 2017; Zhu et al., 2017; Zhang et al., 2017; Karras et al., 2018; Brock et al., 2019; Karras et al., 2019; Rombach et al., 2022b; Ramesh et al., 2022) or training image generation models using unauthorized images from artists and mimicking their styles (Shan et al., 2023). Some works have attempted to address this issue, such as removing certain knowledge from models (Gandikota et al., 2023) or making images unlearnable or uneditable (Huang et al., 2021; Fu et al., 2021; Shan et al., 2023; Salman et al., 2023). However, other studies argue that such imperceptible perturbations are fragile (Radiya-Dixit et al., 2021b).
We briefly introduce the preliminaries on Latent Diffusion Models (Rombach et al., 2022b) and existing protection methods (Salman et al., 2023; Shan et al., 2023).
Latent Diffusion Models (LDMs). Latent diffusion model (Rombach et al., 2022b) aims to train the diffusion model on a lower-dimensional latent space to generate high-quality samples while saving computation costs. Moreover, it can utilize additional information, such as text, to guide the generation process, enabling it to generate or edit images with guidance from a prompt (e.g., a text). Given an image  $\mathbf{x}$ , LDM first uses an image encoder  $\mathcal{E}$  to embed it into a latent representation, i.e.,  $\mathbf{z} = \mathcal{E}(\mathbf{x})$  and then perform  $T$  forward diffusion steps by progressively adding Gaussian noise  $\epsilon \sim \mathcal{N}(\mathbf{0},\mathbf{I})$  to the latent space. For simplicity, we use  $\mathbf{z}_1,\mathbf{z}_2,\dots ,\mathbf{z}_T$  to denote latent representations at each forward diffusion step. Suppose we have a prompt  $y$  (e.g., a certain text phrase), a feature extractor  $\tau_{\theta}$
and we denote the embedding of  $y$  as  $\tau_{\theta}(y)$ . The goal of the LDM is to train a conditional denoising network  $\epsilon_{\theta}$ , e.g., a UNet (Ronneberger et al., 2015), to model the conditional distribution  $p(\mathbf{z}|y)$  by gradually recovering  $\mathbf{z}$  from  $\mathbf{z}_T$  based on  $\tau_{\theta}(y)$ . Suppose  $\epsilon_{\theta}(\mathbf{z}_t, t, \tau_{\theta}(y))$  is the Gaussian noise estimated in the  $t$ -th step. We can train  $\epsilon_{\theta}$  based on the following loss:
$$
L _ {\mathrm {L D M}} := \mathbb {E} _ {\mathcal {E} (\mathbf {x}), y, \epsilon \sim \mathcal {N} (\mathbf {0}, \mathbf {I}), t} \left[ \| \epsilon - \epsilon_ {\theta} \left(\mathbf {z} _ {t}, t, \tau_ {\theta} (y)\right) \| _ {2} ^ {2} \right]. \tag {3.1}
$$
Suppose  $\widetilde{\mathbf{z}}$  is the latent representation obtained after  $T$  denoising steps. We can use an image decoder  $\mathcal{D}$  to obtain the generated image  $\widetilde{\mathbf{x}}$  based on  $\widetilde{\mathbf{z}}$ , i.e.,  $\widetilde{\mathbf{x}} = \mathcal{D}(\widetilde{\mathbf{z}})$ . For simplicity, we denote the end-to-end framework to obtain  $\widetilde{\mathbf{x}}$  based on  $\mathbf{x}$  and  $y$  as  $\widetilde{\mathbf{x}} = f_{\mathrm{LDM}}(\mathbf{x}; y)$ . Given a prompt, LDMs can be used to generate a sample or edit an image. Note that when there is no prompt  $y$ , the LDM reconstructs its input image  $\mathbf{x}$ , i.e.,  $\mathbf{x}$  would be very similar to  $f_{\mathrm{LDM}}(\mathbf{x})$ .
PhotoGuard. PhotoGuard (Salman et al., 2023) proposes to add carefully crafted perturbations to an image before making it publicly available (e.g., uploading it to the Internet) to raise the cost of malicious image editing. When using LDMs to edit the perturbed images, the output is usually less realistic. Specifically, PhotoGuard proposes two approaches: encoder attack and diffusion attack.
The goal of the encoder attack is to add a perturbation  $\delta_{\mathrm{enc}}$  to an image  $\mathbf{x}$  such that the image encoder  $\mathcal{E}$  produces similar outputs for  $\mathbf{x} + \delta_{\mathrm{enc}}$  and a target image  $\mathbf{x}_{\mathrm{target}}$  (e.g., a pure gray image or random noise). Formally,  $\delta_{\mathrm{enc}}$  can be obtained by solving the following optimization problem:
$$
\delta_ {\mathrm {e n c}} = \underset {| | \boldsymbol {\delta} | | _ {\infty} \leq \Delta} {\operatorname {a r g m i n}} \| \mathcal {E} (\mathbf {x} + \boldsymbol {\delta}) - \mathcal {E} (\mathbf {x} _ {\text {t a r g e t}}) \| _ {2} ^ {2}, \tag {3.2}
$$
where  $\Delta$  is the  $L_{\infty}$  norm perturbation budget. The noise added to the image could be imperceptible to human eyes when  $\Delta$  is small. Thus when LDMs are used to edit  $\mathbf{x} + \delta_{\mathrm{enc}}$ , the resulting output would be similar to the target image instead. The diffusion attack, on the other hand, is more direct and aims to craft a perturbation  $\delta_{\mathrm{diff}}$  such that the output of the LDM is similar to  $\mathbf{x}_{\mathrm{target}}$ :
$$
\boldsymbol {\delta} _ {\text {d i f f}} = \underset {| | \boldsymbol {\delta} | | _ {\infty} \leq \Delta} {\operatorname {a r g m i n}} \| f _ {\mathrm {L D M}} (\mathbf {x} + \boldsymbol {\delta}) - \mathbf {x} _ {\text {t a r g e t}} \| _ {2} ^ {2}. \tag {3.3}
$$
Compared to the encoder attack which only targets the image encoder, the diffusion attack considers the whole LDM model with prompts, achieving better empirical performance but is less efficient.
GLAZE. GLAZE (Shan et al., 2023) aims to add perturbations to the artworks of an artist such that LDMs cannot learn the correct style of the artist from perturbed artworks. Given an image  $\mathbf{x}$ , GLAZE first chooses a target style  $T$  that is sufficiently different from the style of  $\mathbf{x}$ . Then, GLAZE transfers  $\mathbf{x}$  to the target style  $T$  using a pre-trained style transfer model  $\Omega$ . For simplicity, we use  $\Omega(\mathbf{x}, T)$  to denote the style-transferred image. Given  $\Omega(\mathbf{x}, T)$ , GLAZE crafts the perturbation  $\delta_{\mathrm{GLAZE}}$  by solving the following optimization problem:
$$
\delta_ {\text {G L A Z E}} = \min  _ {\delta} \| \mathcal {E} (\Omega (\mathbf {x}, T)), \mathcal {E} (\mathbf {x} + \delta) \| _ {2} ^ {2} + \lambda \cdot \max  \left(\text {L P I P S} (\mathbf {x}, \mathbf {x} + \delta) - \Delta_ {L}, 0\right), \tag {3.4}
$$
where LPIPS (Learned Perceptual Image Patch Similarity) (Zhang et al., 2018) measures user-perceived image distortion,  $\Delta_L$  is the perturbation budget, and  $\lambda$  is a hyper-parameter adjusting the strength of the LPIPS regularization term. Roughly speaking, GLAZE aims to perturb images (e.g., artworks of an artist) such that a LDM generates samples with the target style instead of the original style when learning from the perturbed images.
In this section, we formally study the potential vulnerabilities in existing protection methods and discuss our proposed method to disable perturbations added by those methods.
4.1 Analyzing Potential Vulnerabilities of Existing Methods
We notice that existing protection methods (e.g., PhotoGuard and GLAZE) all aim to add imperceptible perturbations such that LDMs cannot "correctly" learn or process the perturbed image for style mimicking or malicious editing. While such perturbations certainly raise the cost of unauthorized data usage, it is not flawless. Specifically, one special use case of LDM is to let it reconstruct the image, i.e., let an image  $\mathbf{x}$  run through the entire LDM (encoder, forward diffusion, backward diffusion,
Figure 2: Examples of inconsistencies observed when using LDMs to reconstruct the protected images. Left: the diffusion-reconstructed clean image looks similar to the original image; Right: images protected by GLAZE (Shan et al., 2023) leads to noticeable inconsistency after reconstruction.
decoder) without any prompt. In such a case, the output of the LDM (denoted by  $f_{\mathrm{LDM}}(\mathbf{x})$ ) should be close to the original input  $\mathbf{x}$ . However, since the imperceptible perturbations are able to deceive the LDMs (to believe that the image is from another style or that the image's latent embedding is close to a target image), its reconstruction result  $f_{\mathrm{LDM}}(\mathbf{x}_{\mathrm{ptb}})$  may no longer be close to the perturbed input  $\mathbf{x}_{\mathrm{ptb}}$ . Rather, it could become similar to a target image or target style instead.
Figure 2 shows an example painting by Claude Monet and its protected version by GLAZE (Shan et al., 2023). When using LDMs to reconstruct the two images respectively, we can observe that the protected image leads to noticeable inconsistency after reconstruction while the clean image stays roughly the same. In other words, the reconstructed version of the perturbed image is no longer close to itself, rather, it is more close to the targeted style (e.g., Cubism by Picasso). By contrast, a clean image and its reconstructed version do not have such inconsistency. This inspires us to leverage the inconsistency to purify the perturbed image to disable such imperceptible perturbations.
4.2 Our Proposed Method
Based on our previous empirical findings, given a perturbed image  $\mathbf{x}_{\mathrm{ptb}}$  created by existing protection methods via imperceptible perturbations, we aim to purify the perturbed image to disable its perturbation. We first derive the two conditions that the purified image  $\mathbf{x}_{\mathrm{pur}}$  should satisfy, then formulate IMPRESS, i.e., IMperceptible Perturbation REMoval SyStem, as an optimization problem based on those two conditions and provide our solution to solve it, and finally show our complete algorithm.
Two conditions for purified image. The purified image  $\mathbf{x}_{\mathrm{pur}}$  should satisfy two conditions: similarity condition and consistency condition. Next, we respectively derive them.
1. Similarity condition. Our ultimate goal is to make sure the purified image  $\mathbf{x}_{\mathrm{pur}}$  stays visually close to the original image  $\mathbf{x}$ , i.e.,  $\mathbf{x}_{\mathrm{pur}} \approx \mathbf{x}$ . Recall that existing methods (Salman et al., 2023; Shan et al., 2023) aim to add an imperceptible perturbation to a given image  $\mathbf{x}$  to craft a perturbed image  $\mathbf{x}_{\mathrm{ptb}}$  (see Section 3 for details). Therefore, the perturbed image  $\mathbf{x}_{\mathrm{ptb}}$  is already close to the original image  $\mathbf{x}$ . Based on this observation, since we only have access to the perturbed image  $\mathbf{x}_{\mathrm{ptb}}$ , requiring the purified image  $\mathbf{x}_{\mathrm{pur}}$  to stay visually close to the perturbed image  $\mathbf{x}_{\mathrm{ptb}}$  would be sufficient to guarantee  $\mathbf{x}_{\mathrm{pur}} \approx \mathbf{x}$ . This leads to similarity condition.
2. Consistency condition. As our goal is to find a purified image  $\mathbf{x}_{\mathrm{pur}}$  close to the original image  $\mathbf{x}$ ,  $\mathbf{x}_{\mathrm{pur}}$  should share some similar traits as  $\mathbf{x}$ . Our observation in Section 4.1 suggests that a clean image can stay roughly the same after diffusion-reconstruction while the perturbed image cannot. Therefore, to make  $\mathbf{x}_{\mathrm{pur}}$  close to  $\mathbf{x}$ , we also require the purified image  $\mathbf{x}_{\mathrm{pur}}$  to be able to reconstruct itself using LDMs, i.e.,  $\mathbf{x}_{\mathrm{pur}} \approx f_{\mathrm{LDM}}(\mathbf{x}_{\mathrm{pur}})$ . This leads to consistency condition.
Formulating IMPRESS as an optimization problem. Our key idea is to define two losses that respectively quantify the similarity condition and consistency condition.
1. Quantifying similarity condition. Our similarity condition means that  $\mathbf{x}_{\mathrm{pur}}$  would be visually close to  $\mathbf{x}_{\mathrm{ptb}}$ . Following previous studies (Shan et al., 2023), we use LPIPS (Learned Perceptual Image Patch Similarity) (Zhang et al., 2018) to quantify the visual difference between  $\mathbf{x}_{\mathrm{pur}}$  and  $\mathbf{x}_{\mathrm{ptb}}$ . Formally, we have the loss max(LPIPS  $(\mathbf{x}_{\mathrm{pur}}, \mathbf{x}_{\mathrm{ptb}}) - \Delta_L, 0)$ , where  $\Delta_L$  is the perceptual perturbation budget. We note that LPIPS is widely adopted in various fields (Cherepanova et al.,
Figure 3: A sketch of our method consists of two losses: the similarity loss and the consistency loss. The similarity loss requires the purified image  $\mathbf{x}_{\mathrm{pur}}$  to stay close to  $\mathbf{x}_{\mathrm{ptb}}$  while the consistency loss requires the encoder/decoder reconstructed image to stay close  $\mathbf{x}_{\mathrm{pur}}$  itself.
2021; Laidlaw et al., 2020; Rony et al., 2021) as it provides a measure of human-perceived image distortion. Compared to  $\ell_p$ -norm constraint, LPIPS offers more flexible constraint ranges. If the modified image still appears similar to the original image from a human perspective, it allows for larger changes in certain parts of the image, which aligns with our goal of preserving as much semantic information from  $\mathbf{x}_{\mathrm{ptb}}$  as possible.
2. Quantifying consistency condition. Our consistency condition implies that  $\mathbf{x}_{\mathrm{pur}} \approx f_{\mathrm{LDM}}(\mathbf{x}_{\mathrm{pur}})$ . Intuitively, we can define the consistency loss as  $||\mathbf{x}_{\mathrm{pur}} - f_{\mathrm{LDM}}(\mathbf{x}_{\mathrm{pur}})||_2^2$ . However, it is very challenging to directly optimize this loss as it involves the complicated diffusion process (i.e., gradually adding and removing Gaussian noise). To address the challenge, we simplify the loss by removing the diffusion process in it. In particular, we define the loss  $||\mathbf{x}_{\mathrm{pur}} - \mathcal{D}(\mathcal{E}(\mathbf{x}_{\mathrm{pur}}))||_2^2$  to quantify the consistency condition, where  $\mathcal{E}$  and  $\mathcal{D}$  are the image encoder and decoder in the LDM. Note that  $||\mathbf{x}_{\mathrm{pur}} - \mathcal{D}(\mathcal{E}(\mathbf{x}_{\mathrm{pur}}))||_2^2$  can approximate  $||\mathbf{x}_{\mathrm{pur}} - f_{\mathrm{LDM}}(\mathbf{x}_{\mathrm{pur}})||_2^2$  as the goal of the diffusion process (without any prompt) is to reconstruct latent representation. Moreover, such approximation enables us to solve the optimization problem efficiently.
Combining the above two losses together, our final optimization problem is as follows:
$$
\min  _ {\mathbf {x} _ {\text {p u r}}} \underbrace {\left\| \mathbf {x} _ {\text {p u r}} - \mathcal {D} \left(\mathcal {E} \left(\mathbf {x} _ {\text {p u r}}\right)\right) \right\| _ {2} ^ {2}} _ {\text {c o n s i s t e n c y l o s s}} + \alpha \cdot \underbrace {\max  (\mathrm {L P I P S} \left(\mathbf {x} _ {\text {p u r}} , \mathbf {x} _ {\text {p t b}}\right) - \Delta_ {L} , 0)} _ {\text {s i m i l a r i t y l o s s}}, \tag {4.1}
$$