Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI

## Abstract

Abstract Artists are increasingly concerned about advancements in image generation models that can closely replicate their unique artistic styles.
In response, several protection tools against style mimicry have been developed that incorporate small adversarial perturbations into artworks published online.
In this work, we evaluate the effectiveness of popular protections—with millions of downloads—and show they only provide a false sense of security.
We find that low-effort and “off-the-shelf” techniques, such as image upscaling, are sufficient to create robust mimicry methods that significantly degrade existing protections.
Through a user study, we demonstrate that all existing protections can be easily bypassed , leaving artists vulnerable to style mimicry.
We caution that tools based on adversarial perturbations cannot reliably protect artists from the misuse of generative AI, and urge the development of alternative protective solutions.

## 1 Introduction

*Style mimicry* is a popular application of text-to-image generative models. Given a few images from an artist, a model can be finetuned to generate new images in that style (e.g., a spaceship in the style of Van Gogh).
But style mimicry has the potential to cause significant harm if misused. In particular, many contemporary artists worry that others could now produce images that copy their unique art style, and potentially steal away customers (Heikkilä, ).
As a response, several protections have been developed to protect artists from style mimicry (Shan et al., ; Van Le et al., ; Liang et al., ).
These protections add adversarial perturbations to images that artists publish online, in order to inhibit the finetuning process.
These protections have received significant attention from the media—with features in the New York Times (Hill, ), CNN (Thorbecke, ) and Scientific American (Leffer, )—and have been downloaded over 1M times (Shan et al., ).

Yet, it is unclear to what extent these tools actually protect artists against style mimicry, especially if someone actively attempts to circumvent them (Radiya-Dixit et al., ).
In this work, we show that state-of-the-art style protection tools—*Glaze* (Shan et al., ), *Mist* (Liang et al., ) and *Anti-DreamBooth* (Van Le et al., )—are ineffective when faced with simple *robust mimicry methods*. The robust mimicry methods we consider range from low-effort strategies—such as using a different finetuning script, or adding Gaussian noise to the images before training—to multi-step strategies that combine off-the-shelf tools.
We validate our results with a user study, which reveals
that robust mimicry methods can produce results indistinguishable in quality from those obtained from unprotected artworks (see Figure [1](#S1.F1) for an illustrative example).
(^†^†Code and images released at: [https://github.com/ethz-spylab/robust-style-mimicry](https://github.com/ethz-spylab/robust-style-mimicry).)

We show that existing protection tools merely provide a false sense of security.
Our robust mimicry methods do not require the development of new tools or fine-tuneing methods, but only carefully combining standard image processing techniques
*which already existed at the time that these protection tools were first introduced!*. Therefore, we believe that even low-skilled forgers could have easily circumvented these tools since their inception.

Although we evaluate specific protection tools that exist today, the limitations of style mimicry protections are inherent. Artists are necessarily at a disadvantage since they have to act first (i.e., once someone downloads protected art, the protection can no longer be changed).
To be effective, protective tools face the challenging task of creating perturbations that transfer to *any* finetuning technique, even ones chosen adaptively in the future. A similar conclusion was drawn by Radiya-Dixit *et al.* (Radiya-Dixit et al., ), who argued that adversarial perturbations cannot protect users from facial recognition systems.
We thus caution that *adversarial machine learning techniques will not be able to reliably protect artists from generative style mimicry*, and urge the development of alternative measures to protect artists.

We disclosed our results to the affected protection tools prior to publication, so that they could determine the best course of action for existing users.

Figure: Figure 1: Artists are vulnerable to style mimicry from generative models finetuned on their art. Existing protection tools add small perturbations to published artwork to prevent mimicry (Shan et al., ; Liang et al., ; Van Le et al., ). However, these protections fail against *robust mimicry methods*, giving a false sense of security and leaving artists vulnerable. Artwork by *@nulevoy* (Stas Voloshin), reproduced with permission.
Refer to caption: /html/2406.12027/assets/x1.png

## 2 Background and Related Work

Text-to-image diffusion models.  A latent diffusion model consists
of an image autoencoder and a denoiser. The autoencoder is trained to encode and decode images using a lower-dimensional latent space. The denoiser predicts the noise added to latent representations of images
in a diffusion process (Ho et al., ).
Latent diffusion models can generate images from text prompts by conditioning the denoiser on image captions (Rombach et al., ). Popular text-to-image diffusion models include open models such as Stable Diffusion (Rombach et al., ) and Kandinsky (Razzhigaev et al., ), as well as closed models like Imagen (Saharia et al., ) and DALL-E (; Betker et al., ).

Style mimicry.  Style mimicry uses generative models to create images matching a target artistic style.
Existing techniques vary in complexity and quality (see [Appendix G](#A7)). An effective method is to finetune a diffusion model using a few images in the targeted style. Some artists worry that style mimicry can be misused to reproduce their work without permission and steal away customers (Heikkilä, ).

Style mimicry protections.  Several tools have been proposed to prevent unauthorized style mimicry. These tools allow artists to include small perturbations—optimized to disrupt style mimicry techniques—in their images before publishing. The most popular protections are Glaze (Shan et al., ) and Mist (Liang et al., ). Additionally, Anti-DreamBooth (Van Le et al., ) was introduced to prevent fake personalized images, but we also find it effective for style mimicry. Both Glaze and Mist target the encoder in latent diffusion models; they perturb images to obtain latent representations that decode to images in a different style (see [Section H.1](#A8.SS1)). On the other hand, Anti-DreamBooth targets the denoiser and maximizes the prediction error on the latent representations of the perturbed images (see [Section H.2](#A8.SS2)).

Circumventing style mimicry protections.  Although not initially designed for this purpose, adversarial purification (Yoon et al., ; Shi et al., ; Samangouei et al., ) could be used to remove the perturbations introduced by style mimicry protections. DiffPure (Nie et al., ) is the strongest purification method and Mist claims robustness against it. Another existing method for purification is upscaling (Mustafa et al., ). Similarly, Mist and Glaze claim robustness against upscaling. Section [4.1](#S4.SS1) highlights flaws in previous evaluations and how a careful application of both methods can effectively remove mimicry protections.

IMPRESS (Cao et al., ) was the first purification method designed specifically to circumvent style mimicry protections.
While IMPRESS claims to circumvent Glaze, the authors of Glaze critique the method’s evaluation (Shan et al., ), namely the reliance on automated metrics instead of a user study, as well as the method’s poor performance on contemporary artists. Our work addresses these limitations by considering simpler and stronger
purification methods, and evaluating them rigorously with a user study and across a variety of historical and contemporary artists.
Our results show that the main idea of IMPRESS is sound, and that very similar robust mimicry methods are effective.

Unlearnable examples .
Style mimicry protections build upon a line of work that aims to
make data “unlearnable” by machine learning models (Shan et al., ; Huang et al., ; Cherepanova et al., ; Salman et al., ).
These methods typically rely on some form of adversarial optimization, inspired by adversarial examples (Szegedy et al., ).
Ultimately, these techniques always fall short of an *adaptive* adversary that enjoys a second-mover advantage: once unlearnable examples have been collected, their protection can no longer be changed, and the adversary can thereafter select a learning method tailored towards breaking the protections (Radiya-Dixit et al., ; Fowl et al., ; Tao et al., ).

## 3 Threat Model

The goal of style mimicry is to produce images, of some chosen content, that mimic the style of a targeted artist.
Since artistic style is challenging to formalize or quantify, we refrain from doing so and define a mimicry attempt as successful if it generates new images that a human observer would qualify as
possessing the artist’s style.

We assume two parties, the *artist* who places art online (e.g., in their portfolio), and a *forger* who performs style mimicry using these images.
The challenge for the forger is that the artist first *protects* their original art collection before releasing it online, using a state-of-the-art protection tool such as Glaze, Mist or Anti-DreamBooth.
We make the conservative assumption that *all* the artist’s images available online are protected. If a mimicry method succeeds in this setting, we call it *robust*.

In this work, we consider style forgers who finetune a text-to-image model on an artist’s images—the most successful style mimicry method to date (Shan et al., ).
Specifically, the forger finetunes a pretrained model $f$ on protected images $X$ from the artist to obtain a finetuned model $\hat{f}$.
The forger has full control over the protected images and finetuning process, and can arbitrarily modify to maximize the mimicry success. Our *robust mimicry methods* combine a number of “off-the-shelf” manipulations that allow even low-skilled parties to bypass existing style mimicry protections.
In fact, our most successful methods require only black-box access to a finetuning API for the model $f$, and could thus also be applied to proprietary text-to-image models that expose such an interface.

## 4 Robust Style Mimicry

We say that a style mimicry method is *robust* if it can emulate an artist’s style using only *protected* artwork.
While methods for robust mimicry have already been proposed, we note a number of limitations in these methods and their evaluation in Section [4.1](#S4.SS1).
We then propose our own methods (Section [4.3](#S4.SS3)) and evaluation (Section [5](#S5)) which address these limitations.

### 4.1 Limitations of Prior Robust Mimicry Methods and of Their Evaluations

##### (1) Some mimicry protections do not generalize across finetuning setups.

Most forgers are inherently ill-intentioned since they ignore artists’ genuine requests *not* to use their art for generative AI (Heikkilä, ).
A successful protection must thus resist circumvention attempts from a reasonably resourced forger who may try out a variety of tools.
Yet, in preliminary experiments, we found that Glaze (Shan et al., ) performed significantly worse than claimed in the original evaluation, even before actively attempting to circumvent it.
After discussion with the authors of Glaze, we
found small differences between our off-the-shelf finetuning script, and the one used in Glaze’s original evaluation (which the authors shared with us).(^1^11The two finetuning scripts mainly differ in the choice of library, model, and hyperparameters. We use a standard HuggingFace script and Stable Diffusion 2.1 (the model evaluated in the Glaze paper).)
These minor differences in finetuning are sufficient to significantly degrade Glaze’s protections (see [Figure 2](#S4.F2) for qualitative examples).
Since our off-the-shelf finetuning script was not designed to bypass style mimicry protections, these results already hint
at the superficial and brittle protections that existing tools provide: artists have no control over the finetuning script or hyperparameters a forger would use, so protections must be robust across these choices.

Figure: (a) Original artwork
Refer to caption: /html/2406.12027/assets/plots/process/original/0009.jpg

##### (2) Existing robust mimicry attempts are sub-optimal.

Prior evaluations of protections fail to reflect the capabilities of moderately resourceful forgers, who employ state-of-the-art methods (even off-the-shelf ones). For instance, Mist (Liang et al., ) evaluates against *DiffPure* purifications using an outdated and low-resolution purification model. Using DiffPure with a more recent model, we observe significant improvements. Glaze (Shan et al., ) is not evaluated against any version of DiffPure, but claims protection against *Compressed Upscaling*, which first compresses an image with JPEG and then upscales it with a dedicated model. Yet, we will show that by simply swapping the JPEG compression with Gaussian noising, we create *Noisy Upscaling* as a variant that is highly successful at removing mimicry protections (see Figure [29](#A9.F29) for a comparison between both methods).

##### (3) Existing evaluations are non-comprehensive.

Comparing the robustness of prior protections is challenging because the original evaluations use different sets of artists, prompts, and finetuning setups. Moreover, some evaluations rely on automated metrics (e.g., CLIP similarity) which are unreliable for measuring style mimicry (Shan et al., , ). Due to the brittleness of protection methods and the subjectivity of mimicry assessments, we believe a unified evaluation is needed.

### 4.2 A Unified and Rigorous Evaluation of Robust Mimicry Methods

To address the limitations presented in Section [4.1](#S4.SS1), we introduce a unified evaluation protocol to reliably assess how existing protections fare against a variety of simple and natural robust mimicry methods. Our solutions to each of the numbered limitations above are: (1) The attacker uses a popular “off-the-shelf” finetuning script for the strongest open-source model that all protections claim to be effective for: Stable Diffusion 2.1. This finetuning script is chosen independently of any of these protections, and we treat it as a black-box. (2) We design four robust mimicry methods, described in Section [4.3](#S4.SS3). We prioritize simplicity and ease of use for low-expertise attackers by combining a variety of off-the-shelf tools. (3) We design and conduct a user study to evaluate each mimicry protection against each robust mimicry method on a common set of artists and prompts.

### 4.3 Our Robust Mimicry Methods

We now describe four robust mimicry methods that we designed to assess the robustness of protections. We primarily prioritize simple methods that only require *preprocessing* protected images. These methods present a higher risk because they are more accessible, do not require technical expertise, and can be used in black-box scenarios (e.g. if finetuning is provided as an API service). For completeness, we further propose one white-box method, inspired by IMPRESS (Cao et al., ).

We note that the methods we propose have been considered (at least in part) in prior work that found them to be *ineffective* against style mimicry protections (Shan et al., ; Liang et al., ; Shan et al., ). Yet, as we noted in Section [4.1](#S4.SS1), these evaluations suffered from a number of limitations. We thus re-evaluate these methods (or slight variants thereof) and will show that they are significantly more successful than previously claimed.

##### Black-box preprocessing methods.

✦ *Gaussian noising.* As a simple preprocessing step, we add small amounts of Gaussian noise to protected images. This approach can be used ahead of any black-box diffusion model.

✦ *DiffPure.* We use image-to-image models to remove perturbations introduced by the protections, also called DiffPure (Nie et al., ) (see [Section I.1](#A9.SS1)). This method is black-box, but requires two different models: the purifier, and the one used for style mimicry. We use Stable Diffusion XL as our purifier.

✦ *Noisy Upscaling.* We introduce a simple and effective variant of the two-stage upscaling purification considered in Glaze (Shan et al., ). Their method first performs JPEG compression (to minimize perturbations) and then uses the Stable Diffusion Upscaler (Rombach et al., ) (to mitigate degradations in quality).
Yet, we find that upscaling actually *magnifies* JPEG compression artifacts instead of removing them. To design a better purification method, we observe that the Upscaler is trained on images augmented with Gaussian noise. Therefore, we purify a protected image by first applying Gaussian noise and then applying the Upscaler. This Noisy Upscaling method introduces no perceptible artifacts and significantly reduces protections (see Figure [29](#A9.F29) for an example and [Section I.2](#A9.SS2) for details).

##### White-box methods.

✦ *IMPRESS++.* For completeness, we design a white-box method to assess whether more complex methods can further enhance the robustness of style mimicry. Our method builds on IMPRESS (Cao et al., ) but adopts a different loss function and further applies *negative prompting* (Miyake et al., ) and *denoising* to improve the robustness of the sampling procedure (see [Section I.3](#A9.SS3) and Figure [30](#A9.F30) for details).

## 5 Experimental Setup

##### Protection tools.

We evaluate three protection tools—Mist, Glaze and Anti-DreamBooth—against four robust mimicry methods—Gaussian noising, DiffPure, Noisy Upscaling and IMPRESS++—and a baseline mimicry method.
We refer to a combination of a protection tool and a mimicry method as a *scenario*. We thus analyze fifteen possible scenarios. [Appendix J](#A10) describes our experimental setup for style mimicry and protections in detail.

##### Artists.

We evaluate each style mimicry scenario on images from 10 different artists, which we selected to maximize style diversity. To address limitations in prior evaluations (Shan et al., ), we use five historical artists
as well as five contemporary artists who are unlikely to be highly represented in the generative model’s training set (two of these were also used in Glaze’s evaluation).(^2^22Contemporary Artists were selected from *Artstation*. We keep them anonymous throughout this work—and refrain from showcasing their art—except for artists who gave us explicit permission to share their identity and art. We will share all images used in our experiments upon request with researchers.) All details about artist selection are included in [Appendix J](#A10).

##### Implementation.

Our mimicry methods finetune Stable Diffusion 2.1 (Rombach et al., ), the best open-source model available at the time when the protections we study were introduced.
We use an off-the-shelf finetuning script from HuggingFace
(see [Section J.1](#A10.SS1) for details).
We first validate that our style mimicry pipeline is successful on unprotected art using a user study, detailed in Appendix [K.1](#A11.SS1). For protections, we use the original codebases to reproduce Mist and Anti-Dreambooth. Since Glaze does not have a public codebase (and the authors were unable to share one), we use the released Windows application binary (version 1.1.1) as a black-box.
We set each scheme’s hyperparameters to maximize protections.
See [Section J.2](#A10.SS2) for details on the configuration for each protection.

We perform robust mimicry by finetuning on 18 different images per artist. We then generate images for 10 different prompts. These prompts are designed to cover diverse motifs that the base model, Stable Diffusion 2.1, can successfully generate. See Appendix [K](#A11.SS0.SSS0.Px3) for details about prompt design.

##### User study.

To measure the success of each style mimicry scenario, we rely only on human evaluations since previous work found automated metrics (e.g., using CLIP (Radford et al., )) to be unreliable (Shan et al., , ). Moreover, style protections not only prevent style transfer, but also reduce the overall quality of the generated images (see Figure [3](#S5.F3) for examples). We thus design a user study to evaluate image quality and style transfer as independent attributes of the generations.(^3^33The user study was approved by our institution’s IRB.)

Figure: Figure 3: Examples of robust style mimicry for two different artists: @greg-f (contemporary) and Edvard Munch (historical). Cherry-picked examples with strong protections and successful robust mimicry. We apply Noisy Upscaling for prompts: “a shoe” and “an astronaut riding a horse”.
Refer to caption: /html/2406.12027/assets/x2.png

Our user study asks annotators on Amazon Mechanical Turk (MTurk) to compare image pairs, where one image is generated by a robust mimicry method, and the other from a baseline state-of-the-art mimicry method that uses *unprotected* art of the artist. A perfectly robust mimicry method would generate images of quality and style indistinguishable from those generated directly from unprotected art. We perform two separate studies: one assessing image quality (e.g., which image looks “better”) and another evaluating stylistic transfer (i.e., which image captures the artist’s original style better, disregarding potential quality artifacts). Our results show that these two metrics obtain very similar results across all scenarios. [Appendix K](#A11) describes our user study and interface in detail.

As noted by the authors of Glaze (Shan et al., ), the users of platforms like
MTurk might not have high artistic expertise. However, we believe that the judgment of non-artists is also relevant as they ultimately represent a large fraction of the *consumers* of digital art. Thus, if lay people consider mimicry attempts to be successful, mimicked art could hurt an artist’s business. Also, to mitigate potential issues with the quality of annotations (Kennedy et al., ), we put in place several control mechanisms to filter out low-quality annotations to the best of our abilities (details in Appendix [K](#A11)).

##### Evaluation metric.

We define the *success rate* of a robust mimicry method as the percentage of annotators (5 per comparison) who prefer outputs from the robust mimicry method over those from a baseline method finetuned on *unprotected* art (when judging either style match or overall image quality). Formally, we define the success rate for an artist in a specific scenario as:

$$ $\texttt{success rate}=\frac{1}{{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}10}\cdot{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}5}}\;\;{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\sum_{\text{prompt}}^{10}}\hskip 5.0pt{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\sum_{\text{annotator}}^{5}}\mathds{1}{[\textit{robust mimicry}\text{ \small{preferred over} }\textit{unprotected mimicry}]}$ (1) $$

A perfectly robust mimicry method would thus obtain a success rate of 50%, indicating that its outputs are indistinguishable in quality and style from those from the baseline, unprotected method. In contrast, a very successful protection would result in success rates of around 0% for robust mimicry methods, indicating that mimicry on top of protected images always yields worse outputs.

## 6 Results

In [Figure 4](#S6.F4), we report the distribution of success rates per artist (N=10) for each scenario. We averaged the quality and stylistic transfer success rates to simplify the analysis (detailed results can be found in Appendix [C](#A3)). Since the forger can try multiple mimicry methods for each prompt, and then decide which one worked best, we also evaluate a “best-of-4” method that picks the most successful mimicry method for each generation (according to human evaluators).

Figure: Figure 4: Success rate per artist (N=10) on all mimicry scenarios. Box plots represent success rates for most protected, quartiles, median and least protected artists, respectively. Success rates around 50% indicate that robust mimicry outputs are indistinguishable in style and quality from mimicry outputs based on unprotected images. *Best-of-4* selects the most successful method for each prompt.
Refer to caption: /html/2406.12027/assets/x3.png

### 6.1 Main Findings: All Protections are Easily Circumvented

We find that all existing protective tools create a false sense of security and leave artists vulnerable to style mimicry. Indeed, our best robust mimicry methods produce images that are, on average, indistinguishable from baseline mimicry attempts using unprotected art.
Since many of our simple mimicry methods only use tools that were available before the protections were released, style forgers may have already circumvented these protections since their inception.

Noisy upscaling is the most effective method for robust mimicry, with a median success rate above 40% for each protection tool (recall that 50% success indicates that the robust method is indistinguishable from a mimicry using unprotected images). This method only requires preprocessing images and black-box access to the model via a finetuning API. Other simple preprocessing methods like Gaussian noising or DiffPure also significantly reduce the effectiveness of protections. The more complex white-box method IMPRESS++ does not provide significant advantages. Sample generations for each method are in Appendix [B](#A2).

A style forger does not have to use a single robust mimicry method, but can test all of them and select the most successful.
This “best-of-4” approach always beats the baseline mimicry method over unprotected images (which attempts a single method and not four) for all protections.

Appendix [A](#A1) shows images at each step of the robust mimicry process (i.e., protections, preprocessing, and sampling). Appendix [B](#A2) shows example generations for each protection and mimicry method.
Appendix [C](#A3) has detailed success rates broken down per artist, for both image style and quality.

### 6.2 Analysis

We now discuss key insights and lessons learned from these results.

##### Glaze protections break down without any circumvention attempt.

Results for Glaze without robust mimicry (see “Naive mimicry” row in [Figure 4](#S6.F4)) show that the tool’s protections are often ineffective.
Without any robustness intervention, 30% of the images generated with our off-the-shelf finetuning are rated as better than the baseline results using only unprotected images.
This contrasts with Glaze’s original evaluation, which claimed a success rate of at most 10% for robust mimicry.(^4^44The original evaluation in Glaze directly asks annotators whether a mimicry is successful or not, rather than a binary comparison between a robust mimicry and a baseline mimicry as in our setup. Shan et al. () report that mimicry fails in 4% of cases for unprotected images, and succeeds in 6% of cases for protected images. This bounds the success rate for robust mimicry—according to our definition in [Equation 1](#S5.E1)—by at most $10\%$.) This difference is likely due to the protection’s brittleness to slight changes in the finetuning setup (as we illustrated in Section [4.1](#S4.SS1.SSS0.Px1)).
With our best robust mimicry method (noisy upscaling) the median success rate across artists rises further to 40%, and our best-of-4 strategy yields results indistinguishable from the baseline for a majority of artists.

##### Robust mimicry works for contemporary and historical artists alike.

Shan et al. () note that one of IMPRESS’ main limitations is that “purification has a limited effect when tested on artists that are not well-known historical artists already embedded in original training data”. Yet, we find that our best-performing robust mimicry method—Noisy Upscaling—has a similar success rate for historical artists (42.2%) and contemporary artists with little representation in the model’s training set (43.5%).

##### Protections are highly non-uniform across artists.

As we observe from Figure [4](#S6.F4), the effectiveness of protections varies significantly across artists: the least vulnerable artist (left-most whisker) enjoys much stronger mimicry protections than the median artist or the most vulnerable artist (right-most whisker). We find that robust mimicry is the least successful for artists where the baseline mimicry from unprotected images gives poor results to begin with (cf. results for artist $A_{1}$ in Appendix [C](#A3) and Appendix [K.1](#A11.SS1)).
Yet, since existing tools do not provide artists with a way to *check* how vulnerable they are, these tools still provide a false sense of security for all artists. This highlights an inherent asymmetry between protection tools and mimicry methods: protections should hold for *all* artists alike, while a mimicry method might successfully target only specific artists.

Figure: Figure 5: Randomly selected comparisons where all 5 annotators preferred mimicry from unprotected art over robust mimicry. Both use Noisy Upscaling for robust mimicry.
Refer to caption: /html/2406.12027/assets/x4.png

##### Robust mimicry failures still remove protection artifacts.

We manually checked the cases where all annotators ranked mimicry from unprotected art as better than robust mimicry with Noisy Upscaling. Figure [5](#S6.F5) shows two examples. We find that in many instances, the model fails to mimic the style accurately even from unprotected art. In these cases, robust mimicry is still able to generate clear images that are similar to unprotected mimicry, but neither matches the original style well.

## 7 Discussion and Broader Impact

##### Adversarial perturbations do not protect artists from style mimicry.

Our work is not intended as an exhaustive search for the best robust mimicry method, but as a demonstration of the brittleness of existing protections.
Because these protections have received significant attention, artists may believe they are effective.
But our experiments show *they are not*.
As we have learned from adversarial ML, whoever acts first (in this case, the artist) is at a fundamental disadvantage (Radiya-Dixit et al., ).
We urge the community to acknowledge these limitations and think critically when performing future evaluations.

##### Just like adversarial examples defenses, mimicry protections should be evaluated adaptively.

In adversarial settings, where one group wants to prevent another group from achieving some goal, it is necessary to consider “adaptive attacks” that are specifically designed to evade the defense (Carlini & Wagner, ).
Unfortunately, as repeatedly seen in the literature on machine learning robustness, even after adaptive attacks were introduced, many evaluations remained flawed and defenses were broken by (stronger) adaptive attacks (Tramer et al., ).
We show it is the same with mimicry protections: simple adaptive attacks significantly reduce their effectiveness.
Surprisingly, most protections we study claim robustness against input transformations (Liang et al., ; Shan et al., ), but minor modifications were sufficient to circumvent them.

We hope that the literature on style mimicry prevention will learn from the failings of the adversarial example literature:
performing reliable, future-proof evaluations is much harder than proposing a new defense.
Especially when techniques are widely publicized in the popular press, we believe it is necessary to provide users with exceptionally high degrees of confidence in their efficacy.

##### Protections are broken from day one, and cannot improve over time.

Our most successful robust style mimicry methods rely solely on techniques that existed before the protections were introduced.
Also, protections applied to online images cannot easily be changed (i.e., even if the image is perturbed again and re-uploaded, the older version may still be available in an internet archive) (Radiya-Dixit et al., ).
It is thus challenging for a broken protection method to be fixed retroactively. Of course, an artist can apply the new tool to their images going forward, but pre-existing images with weaker protections (or none at all) will significantly boost an attacker’s success (Shan et al., ).

Nevertheless, the Glaze and Mist protection tools recently received significant updates (after we had concluded our user study). Yet, we find that the newest 2.0 versions do not protect against our robust mimicry attempts either (see Appendix [E](#A5) and [F](#A6)).
A future version could explicitly target the methods we studied, but this would not change the fact that all previously protected art would remain vulnerable, and that future attacks could again attempt to adaptively evade the newest protections.
The same holds true for attempts to design similar protections for other data modalities, such as video (Passananti et al., ) or audio (Gokul & Dubnov, ).

##### Ethics and broader impact.

The goal of our research is to help artists better decide how to protect their artwork and business. We do not focus on creating the *best* mimicry method, but rather on highlighting limitations in popular perturbation tools—especially since using these tools incurs a cost, as they degrade the quality of published art.
We will disclose our results to the affected protection tools prior to publication, so that they can determine the best course of action for their users.

Further, we argue that having *no* protection tools is preferable to having *insecure* ones. Insecure protections may mislead artists to believe it is safe to release their work, enabling forgery and putting them in a worse situation than if they had been more cautious in the absence of any protection.

With respect to our paper, all the art featured in this paper comes either from historical artists, or from contemporary artists who explicitly permitted us to display their work. We hope our results will inform improved non-technical protections for artists in the era of generative AI.

##### Limitations and future work.

A larger study with more than 10 artists and more annotators may help us better understand the difference in vulnerability across artists.
The protections we study are not designed in awareness of our robust mimicry methods. However, we do not believe this limits the extent to which our general claims hold: artists will always be at a disadvantage if attackers can design adaptive methods to circumvent the protections.

## Acknowledgements

We thank all the MTurkers that engaged with our tasks, especially those that provided valuable feedback during our preliminary studies to improve the survey. We thank the contemporary artists Stas Voloshin (@nulevoy) and Gregory Fromenteau (@greg-f) for allowing us to display their artwork in this paper. JR is supported by an ETH AI Center doctoral fellowship.

## Appendix A Detailed Art Examples

This section illustrates how images look like at every stage of our work. We include (1) original artwork from a contemporary artist (@nulevoy)(^5^55The artist gave explicit permission for the use of their art) as a reference in Figure [6](#A1.F6), (2) the original artwork after applying each of the available protections in Figure [8](#A1.F8), (3) one image after applying the cross product of all protections and preprocessing methods in Figure [10](#A1.F10), (4) baseline generations from a model trained on unprotected art in Figure [11](#A1.F11), and (5) robust mimicry generations for each scenario in Figure [13](#A1.F13).

Figure: Figure 6: 4 samples from the original artwork from @nulevoy.
Refer to caption: /html/2406.12027/assets/plots/process/original/0000.jpg

Figure: (a) Glaze
Refer to caption: /html/2406.12027/assets/plots/process/glaze/0000.jpg

Figure: (a) Glaze
Refer to caption: /html/2406.12027/assets/plots/process/glaze/0009.jpg

Figure: Figure 11: Generations in the style of @nulevoy after finetuning on *unprotected* images. Each generation is sampled with a different seed.
Refer to caption: /html/2406.12027/assets/plots/process/generations/clean/1.jpeg

Figure: (a) Glaze
Refer to caption: /html/2406.12027/assets/plots/process/generations/glaze/none.jpeg

## Appendix B Robust Mimicry Generations

Figure: Figure 14: Style mimicry for all protections using *naive mimicry*—no robust method is used and we finetune directly on protected images. We randomly chose artists and prompts. Each image pair shows the protected generation and generation from unprotected art.
Refer to caption: /html/2406.12027/assets/plot-img0.jpeg

Figure: Figure 15: Style mimicry for all protections using *Gaussian Noising*. We randomly chose artists and prompts. Each image pair shows the protected robust generation and generation from unprotected art.
Refer to caption: /html/2406.12027/assets/plot-img0.jpeg

Figure: Figure 16: Style mimicry for all protections using *DiffPure*. We randomly chose artists and prompts. Each image pair shows the protected robust generation and generation from unprotected art.
Refer to caption: /html/2406.12027/assets/plot-img0.jpeg

Figure: Figure 17: Style mimicry for all protections using *IMPRESS++*. We randomly chose artists and prompts. Each image pair shows the protected robust generation and generation from unprotected art.
Refer to caption: /html/2406.12027/assets/plot-img0.jpeg

Figure: Figure 18: Style mimicry for all protections using *Noisy Upscaling*. We randomly chose artists and prompts. Each image pair shows the protected robust generation and generation from unprotected art.
Refer to caption: /html/2406.12027/assets/plot-img0.jpeg

## Appendix C Detailed Results

### C.1 Mimicry Quality Versus Style

This section includes the detailed results from our user study. As mentioned in Section [5](#S5), we ask users to assess quality and stylistic fit separately in our study. Figure [19](#A3.F19) and [20](#A3.F20) show the results for each of these evaluations separately (the results in the main body represent the average of the two). Finally, Table [1(b)](#A3.T1.st2) includes numerical results for each scenario.

Figure: Figure 19: Quality evaluation. User preference ratings of all style mimicry scenarios but only for the quality question: “Based on noise, artifacts, detail, prompt fit, and your impression, which image has higher quality?”.
Refer to caption: /html/2406.12027/assets/x5.png

Figure: Figure 20: Style evaluation. User preference ratings of all style mimicry scenarios but only for the quality question: “Overall, ignoring quality, which image better fits the style of the style samples?”.

**Table 1: Success rates averaged across artists for all style mimicry scenarios. Higher percentages indicate more successful mimicry, and 50% would indicate perfect mimicry.**
| Method | Naive mimicry | Gaussian noising | IMPRESS++ | DiffPure | Noisy Upscaling | Best-of-4 |
| --- | --- | --- | --- | --- | --- | --- |
| Protection |  |  |  |  |  |  |
| Anti-DB | 11.6% | 20.6% | 32.2% | 26.6% | 45.0% | 56.6% |
| Glaze | 22.2% | 29.6% | 35.4% | 32.0% | 39.4% | 56.6% |
| Mist | 9.0% | 21.0% | 37.4% | 35.8% | 42.8% | 62.0% |

### C.2 Results Broken Down per Artist

We present next the results obtained for each artist in each scenario. Table [2](#A3.T2) plots the success rate for each method against each protection for all artists, and Table [3(b)](#A3.T3.st2) includes the detailed success rates.

**Table 2: Success rates per artist for style and quality questions, respectively. Each line plot shows, for a given protection and artist, the success rate with Gaussian noising ( ), naive mimicry ( ), IMPRESS++ ( ), DiffPure ( ), Noisy Upscaling ( ), and Best-of-4 ( ) on a scale from 0% to 77%, where the bar $\mid$ demarcates 50%.**
| Attack | Anti-DB | Glaze | Mist |
| --- | --- | --- | --- |
| Artist |  |  |  |
| $A_{1}$ |  |  |  |
| $A_{2}$ |  |  |  |
| $A_{3}$ |  |  |  |
| $A_{4}$ |  |  |  |
| $A_{5}$ |  |  |  |
| Albrecht Durer |  |  |  |
| Alphonse Mucha |  |  |  |
| Anna O.-Lebedeva |  |  |  |
| Edvard Munch |  |  |  |
| Edward Hopper |  |  |  |

**Table 3: User preference ratings of all style mimicry scenarios $\mathcal{S}\in{\mathbb{M}}$ for each artist $A\in{\mathbb{A}}$ by name. Each cell states the percentage of votes that prefer an image generated under the corresponding scenario $\mathcal{S}$ and artist $A\in{\mathbb{A}}$ over a matching image generated under clean style mimicry. Higher percentages indicate weaker attacks or better defenses.**
|  | Method | Naive mimicry | Gaussian noising | IMPRESS++ | DiffPure | Noisy Upscaling | Best-of-4 |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Protection | Artist |  |  |  |  |  |  |
| Anti-DB | $A_{1}$ | 4% | 6% | 8% | 18% | 26% | 30% |
|  | $A_{2}$ | 14% | 48% | 54% | 32% | 50% | 62% |
|  | $A_{3}$ | 10% | 8% | 18% | 16% | 40% | 46% |
|  | $A_{4}$ | 14% | 22% | 20% | 14% | 54% | 70% |
|  | $A_{5}$ | 16% | 16% | 22% | 24% | 54% | 60% |
|  | Albrecht Durer | 2% | 22% | 32% | 26% | 42% | 70% |
|  | Alphonse Mucha | 16% | 22% | 44% | 42% | 60% | 66% |
|  | Anna O.-Lebedeva | 38% | 40% | 56% | 40% | 44% | 76% |
|  | Edvard Munch | 2% | 14% | 40% | 40% | 46% | 56% |
|  | Edward Hopper | 0% | 8% | 28% | 14% | 34% | 30% |
| Glaze | $A_{1}$ | 8% | 20% | 22% | 10% | 12% | 24% |
|  | $A_{2}$ | 12% | 42% | 40% | 28% | 44% | 60% |
|  | $A_{3}$ | 12% | 26% | 18% | 26% | 34% | 52% |
|  | $A_{4}$ | 22% | 20% | 20% | 54% | 54% | 60% |
|  | $A_{5}$ | 18% | 34% | 34% | 24% | 40% | 52% |
|  | Albrecht Durer | 2% | 16% | 40% | 28% | 26% | 54% |
|  | Alphonse Mucha | 40% | 44% | 58% | 42% | 56% | 66% |
|  | Anna O.-Lebedeva | 42% | 46% | 54% | 44% | 34% | 70% |
|  | Edvard Munch | 40% | 16% | 42% | 42% | 38% | 62% |
|  | Edward Hopper | 26% | 32% | 26% | 22% | 56% | 66% |
| Mist | $A_{1}$ | 0% | 6% | 20% | 4% | 12% | 28% |
|  | $A_{2}$ | 14% | 50% | 50% | 46% | 48% | 76% |
|  | $A_{3}$ | 0% | 10% | 22% | 24% | 60% | 60% |
|  | $A_{4}$ | 0% | 16% | 24% | 36% | 66% | 70% |
|  | $A_{5}$ | 12% | 22% | 40% | 28% | 50% | 54% |
|  | Albrecht Durer | 10% | 24% | 28% | 46% | 38% | 60% |
|  | Alphonse Mucha | 32% | 18% | 60% | 56% | 54% | 66% |
|  | Anna O.-Lebedeva | 20% | 38% | 54% | 50% | 34% | 74% |
|  | Edvard Munch | 2% | 22% | 54% | 44% | 28% | 72% |
|  | Edward Hopper | 0% | 4% | 22% | 24% | 38% | 60% |

### C.3 Inter-Annotator Agreement

Figure: Figure 21: Inter-annotator agreement for generations from robust mimicry with Noisy Upscaling and generations from models finetuned on protected art directly (naive mimicry). We plot the percentage of comparisons for which the preferred option was selected by 3, 4 or 5 annotators, respectively. The graph shows a higher consensus for naive mimicry, since the differences are clearer, and more variance for robust mimicry.

## Appendix D Differences with Glaze Finetuning

In Section [4.1](#S4.SS1) and Figure [2](#S4.F2), we discussed the brittleness of Glaze protections against small changes in the finetuning script. We also found our finetuning setup to be better at baseline style mimicry from unprotected art (see Figure [22](#A4.F22)).

Figure: (a) Original artwork
Refer to caption: /html/2406.12027/assets/plots/process/original/0009.jpg

## Appendix E Findings on Glaze 2.0

After concluding our user study, Glaze (Shan et al., ) released an updated version of their tool (v2.0). According to the official release, “This new version significantly improved Glaze robustness against the newest AI models”. Although we could not run the entire user study with the latest protections, we reproduced some of our experiments to verify if protections were more robust under robust mimicry. We believe this comparison is fair to Glaze since we are using newer models—such as Stable Diffusion XL for upscaling. These models, although released before Glaze 1.1.1, may not have been considered in the tool’s design and are now explicitly accounted for.

The official release specifically mentions “Significantly improved robustness against Stable Diffusion 1, 2, SDXL, especially for smooth surface art (e.g. anime, cartoon)”. Therefore, we decided to test this new tool with the contemporary artist *nulevoy*, who draws in a cartoon style and gave us permission to display their artwork. As with the previous version, we only have access to the publicly available Windows application that uses unknown parameters. We protect the images using the “highest” protection option. Our main findings are:

- 1.
Glaze v2.0 introduces more visible perturbations uniformly over the images. See Figure [23](#A5.F23).
- 2.
Glaze v2.0 does not improve protection under robust mimicry. Noisy Upscaling still achieves almost perfect style mimicry. See Figure [24](#A5.F24).
- 3.
Noisy Upscaling is able to to remove visible perturbations during preprocessing as before. See Figure [25](#A5.F25).

Figure: (a) Glaze v1.1.1
Refer to caption: /html/2406.12027/assets/x6.png

Figure: (a) Robust style mimicry on Glaze v1.1.1
Refer to caption: /html/2406.12027/assets/x8.png

Figure: (a) Original artwork
Refer to caption: /html/2406.12027/assets/x10.png

## Appendix F Findings on Mist v2

After responsibly disclosing our work to defense developers, authors from Mist brought to our attention the recent release of their latest Mist v2 with improved resilience (Zheng et al., ). As we did with Glaze v2.0 (see Section [E](#A5)), we reproduced some of our experiments with the latest protections to verify the success of robust mimicry. Their original implementation still uses the outdated version 1.5 of Stable Diffusion. We change to SD 2.1 to match our previous experiments(^6^66Both models share the same encoder for which protections are optimized.).

Our findings, as we saw with Glaze v2.0, highlight that improved protections are still not effective against low-effort robust mimicry. More specifically, the latest version of Mist:

- 1.
introduces visible perturbations over the images. See Figure [26](#A6.F26)
- 2.
does not improve protections against robust mimicry. See Figure [27](#A6.F27)
- 3.
creates protection that are easily removable with Noisy Upscaling. See Figure [28](#A6.F28).

Figure: (a) Mist v1
Refer to caption: /html/2406.12027/assets/x12.png

Figure: (a) Robust style mimicry on Mist v1
Refer to caption: /html/2406.12027/assets/x14.png

Figure: (a) Original artwork
Refer to caption: /html/2406.12027/assets/x16.png

## Appendix G Methods for Style Mimicry

This section summarizes the existing methods that a style forger can use to perform style mimicry. Our work only considers *finetuning* since it is reported to be the most effective (Shan et al., ).

### G.1 Prompting

Well-known artistic styles contained in the training data (e.g. Van Gogh) can be mimicked by prompting a text-to-image model with a description of the style or the name of the artist.
For example, a prompt can be augmented with “ painted in a cubistic style” “ painted by van Gogh” to mimic those styles, respectively. Prompting is easy to apply and does not require changes to the model. However, it fails to mimic styles that are not sufficiently represented in the training data of model—often from the most vulnerable artists.

### G.2 Img2Img

Img2Img creates an updated version of an image with guidance from a prompt. For this, Img2Img processes image $x$ with $t$ timesteps of a diffusion process to obtain the diffused image $x_{t}$. Then, Img2Img uses the model with guidance from prompt $P$ to reverse the diffusion process into the output image variation $x_{P}$.
Analogous to prompting, a prompt suffices to transfer a well-known style, but Img2Img also fails for unknown styles.

### G.3 Textual Inversion

Textual inversion (Gal et al., ) optimizes the embedding of some $n$ new tokens ${\bm{t}}=\left[t_{1},\ldots,t_{n}\right]$ that are appended to image prompts $P$ so that generations closely mimic the style of a given set of images. The tokens are optimized via gradient descent on the model training loss so that $P+{\bm{t}}$ generates images that mimic the target style. Textual inversion requires white-box access to the target model, but enables the mimicry of unknown styles.

### G.4 Finetuning

Finetuning updates the weights of a pretrained text-to-image model to introduce a new functionality. In this case, finetuning allows a forger to “teach” the generative model an unknown style using a set of images in the target style and their captions (e.g. *an astronaut riding a horse*). First, all captions are augmented with some special word, like the name of the artist, to create prompts $P_{x}=C_{x}+\textrm{``by }w_{*}\textrm{''}$. Then, the model weights are updated to minimize the reconstruction loss of the given images following the augmented prompts. At inference time, the forger can append $\textrm{``by }w_{*}\textrm{''}$ to any prompt to obtain art in the target style

The authors of Glaze identify this finetuning setup as the strongest style mimicry method (Shan et al., ). We validate the success of our style mimicry with a user study detailed in Appendix [K.1](#A11.SS1)

## Appendix H Existing Style Mimicry Protections

##### Naming convention.

Depending on the context, style mimicry protections may be viewed either as attacks or as the targets of attacks. In an artistic setting, artists see style mimicry as an attack and utilize methods like Glaze as a defense. Conversely, in the context of adversarial robustness, Glaze can be seen as an attack against style mimicry methods through adversarial perturbations. The research community has not reached a consensus on terminology: Glaze’s authors consider style mimicry an attack and label Glaze as a defense, while the authors of Mist and Anti-DreamBooth describe their approaches as attacks. In our work, we distance ourselves from the attack/defense terminology and instead refer to these mechanisms as protections, and to the party performing mimicry as the “style forger”.

Existing protections can either target the encoder or the decoder of text-to-image models. We classify them accordingly.

### H.1 Encoder Protections

Encoder protections include adversarial perturbations in the images $X$ so that the encoder $\mathcal{E}_{{\bm{\phi}}}$ of the model maps images to latent representations that, when reconstructed, recover images in a different style. Concretely, an encoder protection
first defines a target latent representation ${\bm{t}}_{x}\in\mathrm{Latent}$ for each image $x\in X$ that is different to its own style. For instance, the target latent representation for Edvard Munch could be Vincent Van Gogh. Then, protection $\mathcal{P}$ optimizes the objective

$$ $\begin{gathered}\min_{{\bm{\delta}}_{x}}\mathrm{d_{\mathrm{Lat}}}\!\left(\mathrm{\mathcal{E}_{{\bm{\phi}}}}\!\left(x+{\bm{\delta}}_{x}\right),{\bm{t}}_{x}\right)\\ \text{subject to}\quad\mathrm{d_{\mathrm{Img}}}\!\left(x+{\bm{\delta}}_{x},x\right)\leq p.\end{gathered}$ (2) $$

Glaze (Shan et al., ) is an instance of an encoder protection. Glaze first selects an adversarial target style $\mathcal{S}_{\textrm{adv}}$ that style mimicry should learn instead of the style $\mathcal{S}$ to be protected. Then, Glaze uses Img2Img style transfer to create a variation $x_{\mathcal{S}_{\textrm{adv}}}$ in style $\mathcal{S}_{\textrm{adv}}$ of each image $x\in X$. The latent representation of variation $x_{\mathcal{S}_{\textrm{adv}}}$ is used
as the target latent representation ${\bm{t}}_{x}$ for each image $x\in X$.

Glaze selects the target style $\mathcal{S}_{\textrm{adv}}$ from a pre-defined set of 50 styles ${\mathbb{S}}_{\textrm{adv}}$. First, Glaze computes the distance between the mean CLIP embedding of the images $X$ and the prompt $P_{S^{\prime}}$ corresponding to each style $S^{\prime}\in{\mathbb{S}}_{\textrm{adv}}$. Then, Glaze randomly samples target style $\mathcal{S}_{\textrm{adv}}$ from the 50 to the 75 percentile of target styles ${\mathbb{S}}_{\textrm{adv}}$ sorted by distance.

Glaze implements [Equation 2](#A8.E2) with the penalty method (Wright, ) as

$$ $\min_{{\bm{\delta}}_{x}}\left\|\mathrm{\mathcal{E}_{{\bm{\phi}}}}\!\left(x+{\bm{\delta}}_{x}\right),{\bm{t}}_{x}\right\|_{2}^{2}+\alpha\cdot\mathrm{\mathrm{max}}\!\left(\mathrm{\mathrm{LPIPS}}\!\left(x+{\bm{\delta}}_{x},x\right)-p,0\right)$ (3) $$

where LPIPS (Zhang et al., ) is a choice for metric $d_{\mathrm{Img}}$ that aims to measure user-perceived image distortion. Glaze then optimizes [Equation 3](#A8.E3) with the Adam (Kingma & Ba, ) optimizer.

$\textrm{Mist}_{\phi}$ (Liang et al., ) is a different encoder protection from the Mist project(^7^77Mist project also contains a denoiser attack that we fail to reproduce as a robust protection.). $\textrm{Mist}_{\phi}$ optimizes perturbations with PGD to minimize the squared $L_{2}$-induced distance between the latent representation of the artists’ images and some unrelated target image.

In their original work, Mist is only evaluated against DreamBooth, Style Transfer, and Textual Inversion, but not against finetuning. Also, the original Mist work refers to $\textrm{Mist}_{\phi}$ as Mist operating in textural mode.

### H.2 Denoiser Protections

Denoiser protections use the prediction error of the denoiser ${\epsilon}_{{\bm{\theta}}}$ as a proxy of the quality of style mimicry, making it a feasible target for adversarial optimization. Current Denoiser protections, such as Mist (Liang et al., ) and Anti-DreamBooth (Van Le et al., ) assume that poorly reconstructed images will fail to mimic style

Anti-DreamBooth (Van Le et al., ) uses the prediction error of the denoiser ${\epsilon}_{{\bm{\theta}}_{\textrm{adv}}}$ as a proxy for the mimicry quality, where denoiser ${\epsilon}_{{\bm{\theta}}_{\textrm{adv}}}$ corresponds to the denoiser from a finetuned model trained on images with the style to be protected. Since perturbations maximizing the error with the pretrained decoder can be easily circunvented with finetuning, Anti-DreamBooth uses a technique they refer to as Alternating Surrogate and Perturbation Learning (ASPL). The intuition behind ASPL is trying to simulate finetuning on the art and maximizing the error during finetuning. For this purpose, they interleave finetuning steps with perturbation optimization steps.

## Appendix I Robust Mimicry Methods

This section details the robust mimicry methods we use in our work. These methods are not aimed at maximizing performance. Instead, they demonstrate how various ”off-the-shelf” and low-effort techniques can significantly weaken style mimicry protections.

Formally, given protected images $X$ and a pretrained text-to-image model $f$, we define a general robust mimicry pipeline that finetunes a model ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{f}}$ and then produces an image $Z$ for a given *prompt* as follows (a successful method may not require modifications in all stages):

$$ $\displaystyle{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{f}}\leftarrow\texttt{Finetune}({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}f};\texttt{PreProcess}({\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}X}))$ $\displaystyle{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}Z}\leftarrow\texttt{PostProcess}(\texttt{Sample}({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{f}},{\color[rgb]{.75,.5,.25}\definecolor[named]{pgfstrokecolor}{rgb}{.75,.5,.25}\mathrm{``prompt"}})).$ $$

### I.1 DiffPure

DiffPure (Nie et al., ) uses image generation diffusion models to adversarially purify images $X_{\textrm{prot}}$. DiffPure processes
each image $x_{\textrm{adv}}\in X_{\textrm{prot}}$ with $t$ timesteps of a diffusion
process to obtain the diffused image $x_{\textrm{adv}}^{t}=\sqrt{\alpha_{t}}\cdot x_{\textrm{adv}}+\sqrt{1-\alpha_{t}}\cdot{\bm{{\epsilon}}}$, where $\alpha$ is the noise schedule of the diffusion process and noise ${\bm{{\epsilon}}}$ is sampled from $\mathrm{\mathcal{N}}\!\left(0,{\bm{I}}\right)$. Then, DiffPure constructs the purified image $\mathrm{\mathrm{DiffPure}}\!\left(x_{\textrm{adv}}\right)$ by applying reverse diffusion to image $x_{\textrm{adv}}^{t}$ for $t$ timesteps with an image generation diffusion model $\mathrm{DM}$. prove that under certain idealized conditions, DiffPure is likely to weaken adversarial
perturbations in image $x_{\textrm{adv}}$.

If the text-to-image model $\mathrm{M}$ supports unconditional image generation,
then we can use model $\mathrm{M}$ for the reverse diffusion process. For example, Stable Diffusion (Rombach et al., ) generates images unconditionally
when the prompt $P$ equals the empty string. Under these conditions, Img2Img is equivalent to DiffPure. Therefore, in the context of defenses for style mimicry,
we refer to Img2Img applied with an empty prompt $P$ as unconditional DiffPure, and to Img2Img applied with a non-empty prompt $P$ as conditional DiffPure.

### I.2 Noisy Upscaling

Upscaling increases the resolution of an image by predicting new pixels that enhance the level of detail. Upscaling images can purify adversarially perturbed images (Mustafa et al., ). However, we discover that applying upscaling directly on protected images fails to remove the perturbations.

We define Noisy Upscaling as a way to address the shortcomings of upscaling. Noisy Upscaling
first applies Gaussian noising and then upscales the noisy image.
Noisy Upscaling has a more profound effect than the sum of its parts: Gaussian noising only adds noise
to an image $x_{\textrm{adv}}$, but does not remove the adversarial perturbation ${\bm{\delta}}_{x}$. Similarly, we observe upscaling to roughly preserve perturbation ${\bm{\delta}}_{x}$. In contrast, $\mathrm{\mathrm{NoisyUpscale}}\!\left(x_{\textrm{adv}}\right)$ shows neither visually perceptible noise, nor adversarial perturbations. Figure [29](#A9.F29) illustrates the improvements. We explain these phenomena as follows.

First, we use the Stable Diffusion Upscaler ($\mathrm{Upscale}_{\text{SD}}$), which is trained on noise-augmented images and accepts the corresponding noise level $L$ as a class-conditioning label. We can therefore condition $\mathrm{Upscale}_{\text{SD}}$ on the noise level $L_{\sigma^{2}}$, corresponding to the variance $\sigma^{2}$ used by $\mathrm{GaussianNoising}$, to remove the noise that $\mathrm{GaussianNoising}$ adds.

Second, we note that upscaling has shown success against adversarial perturbations for classifiers (Mustafa et al., ), but not against adversarial perturbations for generative models (Liang et al., ; Shan et al., ).

Figure: (a) Original artwork
Refer to caption: /html/2406.12027/assets/plots/noisyups/0001_nulevoy_clean.jpg

### I.3 IMPRESS++

We enhance the IMPRESS algorithm (Cao et al., ). We change the loss of the reverse encoding optimization from patch similarity to $l_{\infty}$ and include two additional steps: negative prompting and post-processing. All in all, IMPRESS++ first preprocesses protected images with Gaussian noise and reverse encoder optimization, then samples using negative prompting and finally post-processes the generated images with DiffPure to remove noise.

##### Reverse encoder optimization.

Reverse encoder optimization is a preprocessing defense against encoder protections.
It adds additional perturbations $\Delta\!\!\!\!\Delta^{\prime}$ to images $X_{\textrm{prot}}$ so that the latent representation ${\bm{t}}_{x_{\textrm{adv}}^{\prime}}=\mathrm{\mathcal{E}_{{\bm{\phi}}}}\!\left(x_{\textrm{adv}}^{\prime}\right)$ of each protected image
$x_{\textrm{adv}}^{\prime}=x_{\textrm{adv}}+{\bm{\delta}}_{x_{\textrm{adv}}}$ satisfies

$$ $\mathrm{\mathcal{D}_{{\bm{\phi}}^{\prime}}}\!\left({\bm{t}}_{x_{\textrm{adv}}^{\prime}}\right)\approx x_{\textrm{adv}}^{\prime}$ (4) $$

and each perturbation ${\bm{\delta}}_{x_{\textrm{adv}}}\in\Delta\!\!\!\!\Delta^{\prime}$ satisfies

$$ $\mathrm{d_{\mathrm{Img}}}\!\left(x_{\textrm{adv}}+{\bm{\delta}}_{x_{\textrm{adv}}},x_{\textrm{adv}}\right)\leq p.$ (5) $$

If [Equation 4](#A9.E4) holds, then style mimicry finetuning learns the
style of images $X_{\textrm{prot}}^{\prime}$.
In addition, the combination of [Equation 5](#A9.E5) with the image similarity constraint $\mathrm{d_{\mathrm{Img}}}\!\left(x+{\bm{\delta}}_{x},x\right)\leq p$ in [Equation 2](#A8.E2) ensures that the defended images $X_{\textrm{prot}}^{\prime}$ look similar to the original images $X$. Therefore, style mimicry finetuning on images $X_{\textrm{prot}}^{\prime}$ should learn a style similar to style $\mathcal{S}$.

Reverse encoder optimization aims to achieve [Equation 4](#A9.E4) and [Equation 5](#A9.E5) by optimizing the objective

$$ $\begin{gathered}\min_{{\bm{\delta}}_{x_{\textrm{adv}}}}\mathrm{d_{\mathrm{Lat}}}\!\left(\mathrm{\mathcal{E}_{{\bm{\phi}}}}\!\left(x_{\textrm{adv}}+{\bm{\delta}}_{x_{\textrm{adv}}}\right),\mathrm{\mathcal{E}_{{\bm{\phi}}}}\!\left(x_{\textrm{adv}}\right)\right)\\ \text{subject to}\quad\mathrm{d_{\mathrm{Img}}}\!\left(x_{\textrm{adv}}+{\bm{\delta}}_{x_{\textrm{adv}}},x_{\textrm{adv}}\right)\leq p\end{gathered}$ (6) $$

with PGD.

##### Negative prompting.

Negative prompting (Miyake et al., ) is a technique to guide image generation of a diffusion-based text-to-image model $\mathrm{M}$ away from a prompt $P_{\textrm{neg}}$. To this end, negative prompting manipulates the
classifier-free guidance (Ho & Salimans, ), which computes the denoiser output of model $\mathrm{M}$ as

$$ $\mathrm{\tilde{{\epsilon}}_{{\bm{\theta}}}}\!\left({\bm{z}},t,P\right)=\left(1+w\right)\cdot\mathrm{{\epsilon}_{{\bm{\theta}}}}\!\left({\bm{z}},t,P\right)-w\cdot\mathrm{{\epsilon}_{{\bm{\theta}}}}\!\left({\bm{z}},t,\text{``''}\right)$ (7) $$

where parameter $w$ controls the guidance strength.
Negative prompting simply substitutes the empty string “” with $P_{\textrm{neg}}$ to obtain

$$ $\mathrm{\tilde{{\epsilon}}_{{\bm{\theta}}}}\!\left({\bm{z}},t,P\right)=\left(1+w\right)\cdot\mathrm{{\epsilon}_{{\bm{\theta}}}}\!\left({\bm{z}},t,P\right)-w\cdot\mathrm{{\epsilon}_{{\bm{\theta}}}}\!\left({\bm{z}},t,P_{\textrm{neg}}\right).$ (8) $$

We design a routine for $\mathcal{D}_{\mathrm{In_{F}}}$ that leverages negative prompting to guide model $\mathrm{M}$ away from adversarial generations.
To this end, we first apply Textual Inversion with adversarial images $X_{\textrm{prot}}$ to encode the style of adversarial generations $\mathcal{S}_{\textrm{adv}}$ into a special word $w_{*}$. We then set prompt $P_{\textrm{neg}}=$ “art by $w_{*}$​”.

Naive negative prompting offers no strength control. Too little strength may fail to guide model $\mathrm{M}$ away from the adversarial style $\mathcal{S}_{\textrm{adv}}$. Too much strength may guide towards the style opposite to style $\mathcal{S}_{\textrm{adv}}$ in the latent space of model $\mathrm{M}$, which is not necessarily the desired style $\mathcal{S}$. We use negative prompt weights (muerrilla, ) to control the strength of negative prompting. The negative prompt weights technique introduces the strength
control parameter $c$ to interpolate between [Equation 7](#A9.E7) and [Equation 8](#A9.E8) as

$$ $\mathrm{\tilde{{\epsilon}}_{{\bm{\theta}}}}\!\left({\bm{z}},t,P\right)=\left(1+w\right)\cdot\mathrm{{\epsilon}_{{\bm{\theta}}}}\!\left({\bm{z}},t,P\right)-w\cdot\left(\left(1+c\right)\cdot\mathrm{{\epsilon}_{{\bm{\theta}}}}\!\left({\bm{z}},t,P_{\textrm{neg}}\right)-c\cdot\mathrm{{\epsilon}_{{\bm{\theta}}}}\!\left({\bm{z}},t,\text{``''}\right)\right).$ (9) $$

Figure [30](#A9.F30) illustrates the improvements introduced by each additional step.

Figure: (a) Original IMPRESS
Refer to caption: /html/2406.12027/assets/plots/impressplus/antidb_nulevoy_revopt.jpeg

## Appendix J Experimental Setup

This section describes our general experimental setup and specifies the settings and hyperparameters of the methods we use. When possible, we use default values from the machine learning literature. For implementation details see our official repository: [https://github.com/ethz-spylab/robust-style-mimicry](https://github.com/ethz-spylab/robust-style-mimicry)

### J.1 Style Mimicry Experimental Details

As described in [Section 3](#S3), our threat model
considers style mimicry with a latent diffusion text-to-image model $\mathrm{M}$
that is finetuned on a set of images $X$ in a style $\mathcal{S}$. This section specifies our choices for model $\mathrm{M}$, images $X$, style $\mathcal{S}$, the hyperparameters for finetuning $\mathrm{M}$, and the hyperparameters for generating images with the finetuned model. Where possible, we
try to replicate the style mimicry setup used by
to evaluate Glaze, and highlight any differences.

##### Model

We use Stable Diffusion version 2.1 (Stability AI, ), the same model used to optimize the protections we evaluate (Shan et al., ; Liang et al., ; Van Le et al., ).

##### Dataset.

We collate 10 image sets $\left\{X^{A}:A\in{\mathbb{A}}\right\}$ from 10 different artists ${\mathbb{A}}$.
Each image set $X^{A}$ contains 18 images that we choose manually to follow
a consistent style $\mathcal{S}_{A}$. We select the artists ${\mathbb{A}}$ from contemporary and historical artists: We select 5 contemporary artists from ArtStation(^8^88[www.artstation.com](www.artstation.com)) and 5 historical artists from the WikiArt dataset (Tan et al., ). We found 2 of the 4 artists used by Glaze and included them in our evaluation. We manually select the remaining 8 artists to cover a broad variety of styles.
Glaze additionally verified that the images of the contemporary artists in their evaluation are not included in the training dataset of the model $\mathrm{M}$. Unfortunately, the LAION-5B dataset (Schuhmann et al., ) used to train SD 2.1 was taken offline (Cole, ), so we are unable to perform this verification. Instead, we verify for each contemporary artist $A\in{\mathbb{A}}$ that SD 2.1 is unable to mimic the style $\mathcal{S}_{A}$ by
manually inspecting SD 2.1 generations for prompts of the form
“An {object} by {artist}”. We center-crop each image $x$ to $512\times 512$ pixels and generate a caption $C_{x}$ for $x$ with the BLIP-2 model (Li et al., ).

##### Finetuning hyperparameters.

Glaze does not specify which finetuning script they use, but they claim to “follow the same training parameters as (Rombach et al., ). We use $5\cdot 10^{-6}$ learning rate and batch size of 32.” This batch size misfits their small finetuning image sets that contain no more than 34 images. Moreover, the finetuning code that kindly sent us upon request
uses DreamBooth finetuning with Stable Diffusion 1.5, instead of version 2.1 as described in their work.

In light of these discrepancies, and assuming that mimicry protections should be agnostic to the finetuning setup used, we use an “off-the-shelf” HuggingFace finetuning script for Stable Diffusion (von Platen et al., ) and manually tune hyperparameters for optimal style mimicry before protections are applied. Concretely, we use $2{,}000$ training steps, batch size $4$, learning rate $5\cdot 10^{-6}$, and set the remaining hyperparameters to their default values. We pair each image $x$ with the prompt
$P_{x}=C_{x}+$“ by $w_{*}$”, where $w_{*}=$ “nulevoy”(^9^99@nulevoy is the first ArtStation artist that we experimented with. In our experiments, we found “nulevoy” a suitable choice for the special word $w_{*}$ and use it for all artists. We check that all of nulevoy’s images are published after the release date of LAION-5B to ensure that SD 2.1 has no prior knowledge about nulevoy’s style.).

##### Generation hyperparameters

We use the DPM-Solver++(2M)
Karras (Lu et al., ; Karras et al., ) scheduler for 50 steps to generate images
of size $768\times 768$. This scheduler generates images with slightly higher quality than the PNDM (Liu et al., ) scheduler used by Glaze.

### J.2 Protections Experimental Details

We evaluate three different protections: Mist (Liang et al., ), Glaze (Shan et al., ), and Anti-DreamBooth (Van Le et al., ). For a fair comparison, we fix the perturbation budget $p$ for each adversarial perturbation ${\bm{\delta}}_{x}$ created by Mist and Anti-DreamBooth to $p=8/255$, which is the same budget that use to evaluate Mist. It is not possible to evaluate Glaze with exactly this perturbation budget, for three reasons: First, Glaze uses LPIPS for the image similarity measure $d_{\mathrm{Img}}$, which does not bound the $L_{\infty}$ norm.
Second, Glaze implements the metric $d_{\mathrm{Img}}$ as a soft bound in [Equation 3](#A8.E3), which offers no hard bound guarantees. Third, Glaze is closed-source software
whose perturbation budget control only offers the settings
Default, Medium, and High. Upon request, the Glaze authors refused to share a codebase where we could control the hyperparameters. Therefore, we evaluate Glaze through their official public tool with
the setting High to evaluate our protections under the highest protections. In our evaluation, we perceive images processed
with Glaze to be equally or less perturbed than images processed with Mist and Anti-DreamBooth.

Next, we describe specific hyperparameters we use to reproduce each of the protections.

#### J.2.1 Anti-DreamBooth

implement Anti-DreamBooth against
DreamBooth finetuning. We adapt their implementation to our vanilla finetuning for style mimicry, using the same hyperparameters where
possible: We set the number of iterations to $N=50$,
the PGD perturbation budget to $p=8/255$, the PGD step size to $\alpha=5\cdot 10^{-3}$, and the number of PGD steps per ASPL iteration to $N_{\mathrm{PGD}}=6$. We minimize the loss
$\mathcal{L}_{\mathrm{Finetune}}$ with the vanilla finetuning setup
in [Section J.1](#A10.SS1) for $300$ training steps.

#### J.2.2 Mist ϕ subscript Mist italic-ϕ \textrm{Mist}_{\phi}

We replicate the evaluation that use to evaluate
$\textrm{Mist}_{\phi}$ against Stable Diffusion. We set the PGD perturbation budget to $p=8/255$, the number of PGD iterations
to $N_{\mathrm{PGD}}=100$, the PGD step size to $\alpha=1/255$, and the target image to $T=\text{Target_Mist}$ shown in [Figure 31](#A10.F31).

Figure: Figure 31: The Mist target image Target_Mist. Target_Mist is the default target image in the reference Mist implementation and one of the successful target images evaluated by .
Refer to caption: /html/2406.12027/assets/media/Target_Mist.png

#### J.2.3 Glaze

The Glaze authors were unable to share a codebase upon request. We thus use their publicly released Windows application binary. We use the latest available version of Glaze, v1.1.1. We set Intensity to High and Render Quality to Slowest, to obtain the strongest protections. Appendix [E](#A5) includes qualitative results on an updated version released after we concluded our user study.

### J.3 Robust Mimicry Methods Experimental Details

#### J.3.1 Gaussian noising

We manually tune the Gaussian noising strength to $\sigma_{2}=0.05$.

#### J.3.2 DiffPure

We use conditional DiffPure with the best-performing publicly available image generation diffusion model, Stable Diffusion XL 1.0 (SDXL) (Podell et al., ). We implement conditional DiffPure using the HuggingFace AutoPipelineForImage2Image pipeline. We use classifier-free guidance scale $\text{{guidance_scale}}=7.5$ with prompt $P=C_{x}$ for image $x$.
We manually tune the number of diffusion timesteps $t$ via the strength pipeline argument to $\text{{strength}}=0.2$.

#### J.3.3 IMPRESS++

##### Reverse Optimization

Like $\textrm{Mist}_{\phi}$, we set the PGD perturbation budget to $p=8/255$ and the PGD step size to $\alpha=1/255$. We manually tune the number of PGD iterations to $N_{\mathrm{PGD}}=400$.

##### Noisy Upscaling

We manually tune the Gaussian noising strength to
$\sigma=0.1$. We then use the Stable Diffusion Upscaler (^10^1010[www.huggingface.co/stabilityai/stable-diffusion-x4-upscaler](www.huggingface.co/stabilityai/stable-diffusion-x4-upscaler)) with the maximum denoising strength $L$.(^11^1111We inadvertently set the denoising strength to $L=320$ instead of the actual maximum denoising strength $L=350$. We observe no qualitative difference in the generated images.).

We note that the Stable Diffusion Upscaler is trained on
diffused images of the form $x_{\alpha}=\sqrt{\alpha}\cdot x+\sqrt{1-\alpha}\cdot\mathrm{\mathcal{N}}\!\left(0,{\bm{I}}\right)$. In contrast,
noisy upscaling noises images additively, that is, without the factor $\sqrt{\alpha}$. However, we note that for $\sqrt{1-\alpha}=\sigma=0.1$, we have $\sqrt{\alpha}=0.995\approx 1$. In practice,
we observe no qualitative difference in the generated images.

##### Negative Prompting

We manually tune the negative prompting strength to $c=0.5$. We use the Stable Diffusion web UI (^12^1212[https://github.com/AUTOMATIC1111/stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui)) to apply Textual Inversion on
the adversarial images $X_{\textrm{prot}}$. We follow the Textual Inversion
setup used by to evaluate Mist and set the length of the token vector ${\bm{t}}$ to $n=8$, the embedding initialization text
to “style *”, the learning rate to $\gamma=0.005$, the batch size
to $1$, and the number of training steps to $500$.

##### DiffPure post subscript DiffPure post \text{DiffPure}_{\text{post}}

To make IMPRESS++ work under a single-model availability, we apply $\text{DiffPure}_{\text{post}}$ with the same model that we use for image generation, SD 2.1. We implement $\text{DiffPure}_{\text{post}}$ using the HuggingFace AutoPipelineForImage2Image pipeline. We use the classifier-free guidance scale $\text{{guidance_scale}}=7.5$ with prompt $P=C_{x}+\text{``, artistic''}$ for image $x$.
We manually tune the number of diffusion timesteps $t$ via the strength pipeline argument to the value $\text{{strength}}=0.2$.

## Appendix K User Study

This user study was approved by our institution’s IRB.

##### Design.

Our user study asks annotators to compare outputs from one robust mimicry method against a baseline where images are generated from a model trained on the original art without protections—for a fixed set of prompts ${\mathbb{P}}$.

We present participants with both generations and a gallery with original art in the target style.
We ask participants to decide which image is better in terms of style and quality, separately. For this, we ask them two different questions:

- 1.
Based on noise, artifacts, detail, prompt fit, and your impression, which image has higher quality?
- 2.
Overall, ignoring quality, which image better fits the style of the style samples?

For each comparison, we collect data from 5 users. We randomize several aspects of our study to minimize user bias. We randomly select the order of robust mimicry and baseline generations. Second, we randomly shuffle the order of all image comparisons to prevent all images from the same mimicry method to appear consecutively. Finally, we also randomly sample the seeds that models use to generate images to prevent repeating the same baseline image across different comparisons.

##### Differences with Glaze’s user study.

Our study does not exactly replicate the design of Glaze’s user study for two reasons. First, the Glaze study provided annotators with four AI-generated images and four original images, asking if the generated images successfully mimicked the original artwork. This evaluation fails to account for the commonly encountered scenario where current models are incapable of reliably mimicking an artist’s style even from unprotected art. Second, we believe the relative assessment recorded in our study (“Which of these two mimicry attempts is more successful?”) is easier for humans than the absolute assessment used in the Glaze study (“Is this mimicry attempt successful”).

##### Prompts.

We curate a small dataset of 10 prompts ${\mathbb{P}}$. We design the prompts to satisfy two criteria:

- 1.
The prompts should cover diverse motifs with varying complexity. This ensures that we can detect if a scenario compromised the prompt-following capabilities of a style mimicry model.
- 2.
The prompts should only include prompts for which our finetuning base model $\mathrm{M}$, SD 2.1, can successfully generate a matching image. This reduces the impact of potential human bias against common defects of SD 2.1.

To satisfy criterion 1 and increase variety, we instruct ChatGPT to generate prompt suggestions for four different categories:

- 1.
Simple prompts with template “a {subject}”.
- 2.
Two-entity prompts with template “a {subject} {ditransitive verb} a {object}”.
- 3.
Entity-attribute prompts with template “a {adjective} {subject}”.
- 4.
Entity-scene prompts with template “a {subject} in a {scene}”.

The chat we used to generate our prompts can be accessed at [https://chatgpt.com/share/ea3d1290-f137-4131-baca-2fa1c92b3859](https://chatgpt.com/share/ea3d1290-f137-4131-baca-2fa1c92b3859). To satisfy criterion 2, we generate images with SD 2.1 on prompts suggested by ChatGPT and manually filter out
prompts with defect generations (e.g. a horse with 6 legs). We populate the final set of prompts ${\mathbb{P}}$ with 4
simple prompts, 2 two-entity prompts, 2 entity-attribute prompts,
and 2 entity-scene prompts (see [Figure 32](#A11.F32)).

Figure: Figure 32: Our set of prompts. We manually wrote the prompts “a astronaut riding a horse” and “a village in a thunderstorm”. ChatGPT wrote the remaining prompts.

##### Quality control.

We first run a pilot study where we directly ask users to answer the previous questions about style and quality. This study resulted in very low-quality responses that are barely better than random choice. We enhanced the study to introduce several quality control measures to improve response quality and filter out low-quality annotations:

- 1.
We limit our study to desktop users so that images are sufficiently large to perceive artifacts introduced by protections.
- 2.
We precede the questions we use for our study with four dummy questions about
the noise, artifacts, detail, and prompt matching of the images. The dummy questions force annotators to pay attention and gather information useful to answer the target questions.
- 3.
We precede our study with a *training session* that shows for question 1, 2, and each of the four dummy questions an image pair with a clear, objective answer. The training session helps users to understand the study questions. We introduced this stage after gathering valuable feedback for annotators.
- 4.
We add *control comparisons* to detect annotators who did not understand the tasks or were answering randomly. We generated several images from the baseline model trained on the original art. For each of these images, we created two ablations. For question 1 (quality), we include Gaussian noise to degrade its quality but preserve the same information. For question 2 (style), we apply Img2Img to remove the artist style and map the image back to photorealism using the prompt “high quality photo, award winning”. We randomly include control comparisons between the original generations and these ablations, and we only accept labels from users who answered correctly at least 80% of the control questions.

##### Execution.

We execute our study on Amazon Mechanical Turk (MTurk). We design and evaluate an MTurk Human Intelligence Task (HIT) for each artist $A\in{\mathbb{A}}$, shown in [Figure 33](#A11.F33). Each HIT includes image pair comparisons for a single artist $A$ under all scenarios $\mathcal{S}\in{\mathbb{M}}$, as well 10 quality control image pairs, 10 style control image pairs, and 6 training image pairs. We generate an image pair for each of the 10 prompts and each of 15 scenarios, for a total of $10\cdot 15+10+10+6=176$ image pairs per HIT. We estimate study participants to
spend 5 minutes on the training image pairs and 30 seconds per remaining image pair, so 90 minutes in total. We compensate
study participants at a rate of $\$16/\text{hour}$, so $\$24$ per HIT.

Figure: Figure 33: The interface of our user study.
Refer to caption: /html/2406.12027/assets/media/mturk_ui_2.png

### K.1 Style Mimicry Setup Validation

We execute an additional user study to validate that our style mimicry setup in [Appendix G](#A7) successfully mimics style from
unprotected images.

For each prompt $P\in{\mathbb{P}}$ and artist $A\in{\mathbb{A}}$, our validation study uses the baseline model trained on uprotected art to generate one image. Inspired by the evaluation by Glaze (Shan et al., ), we ask participants to evaluate the style mimicry success by answering the question:

- How successfully does the style of the image mimic the style of the style samples? Ignore the content and only focus on the style.

To answer this question, we show a participant the image $x_{A}^{\mathcal{O}}{}$ and the images $X^{{\mathbb{A}}}$ that serve as style samples.
The participant can answer the question on a 5-point Likert scale with options

- 1.
Not successful at all
- 2.
Not very successful
- 3.
Somewhat successful
- 4.
Successful
- 5.
Very successful

We also execute the style mimicry validation study on MTurk. We design and evaluate a single HIT for all questions, shown in [Figure 36](#A11.F36). We estimate study participants to spend 15 seconds on each question, and to spend 1 minute to familiarize themselves with a new style, so 35 minutes in total.
We compensate study participants at a rate of $18/hour, so $10.50 per HIT.

We find that style mimicry is successful in over 70% of the comparisons. Results are detailed in Figure [34](#A11.F34).

Figure: Figure 34: User ratings of clean style mimicry success. Each bar indicates the percentage of votes for the corresponding success level for clean style mimicry generations. [Figure 35](#A11.F35) breaks the ratings down by artist.

Figure: Figure 35: User ratings of clean style mimicry success. Each bar indicates the percentage of votes for the corresponding success level over all clean style mimicry generations for the corresponding artist.

Figure: Figure 36: The interface of our style mimicry setup validation study.
Refer to caption: /html/2406.12027/assets/media/mturk_ui_val.png

## Appendix L Compute Resources

[Table 4](#A12.T4) reports the compute resources for our experiments.

**Table 4: Compute resources for our experiments. *Execution time per image / (artist)* reports the execution time of the method to compute a single image, or the combined execution time for all samples of an artist, if the method operates on all samples of an artist at once. $\dagger$ Google Cloud $\ddagger$ IMPRESS++ requires an additional 2 seconds per image generation.**
| Method | GPU | CPU | Memory | Storage | Execution time per<br>image / (artist) | Overall execution time |
| --- | --- | --- | --- | --- | --- | --- |
| Finetuning | RTX A6000 | EPYC 7742 | 5 GB | 5 GB | (40 minutes) | 100 hours |
| Image generation | RTX A6000 | EPYC 7742 | 5 GB | 5 GB | 15 seconds | 13 hours |
| Anti-DB | RTX A6000 | EPYC 7742 | 5 GB | 10 GB | 29 minutes | 88 hours |
| Glaze | T4 | 16 vCPUs on GCP^† | 5 GB | 5 GB | 4 minutes | 12 hours |
| Mist | RTX A6000 | EPYC 7742 | 5 GB | 5 GB | 18 seconds | 54 minutes |
| Gaussian noising | None | EPYC 7742 | 0 GB | 0 GB | 143 milliseconds | 26 seconds |
| IMPRESS++ | RTX A6000 | EPYC 7742 | 5 GB | 5 GB | (27 minutes)^‡ | 370 minutes^‡ |
| DiffPure | RTX A6000 | EPYC 7742 | 7 GB | 7 GB | 48 seconds | 144 minutes |
| Noisy Upscaling | RTX A6000 | EPYC 7742 | 3.5 GB | 3.5 GB | 217 seconds | 651 minutes |

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 8
presentation: 3
questions: See weaknesses.
soundness: 4
strengths: - The paper is well-written and easy to follow.
- It works on an important problem and provides critical insights: all protection methods today cannot protect artworks from diffusion-based mimicry. Though works like Glaze have been widely accepted by artists, they actually perform bad.
- The proposed robust mimicry methods are simple but effective.
- The authors did extensive experiments to support the claims.
summary: The authors critically revisit the current efforts towards protecting artists' work from diffusion model mimicry. The author proposes that most protection nowadays cannot really protect artists, since the protection can by easily bypassed using some tricks of purifications e.g. Gaussian Noising, Diff-Pure and Up-scalers. Extensive experiments are done to support the claim in this paper. Overall, the paper is well-written, easy to follow and the proposed method is simple but effective.
weaknesses: Clarification:
- The title of this paper is 'ADVERSARIAL PERTURBATIONS CANNOT RELIABLY PROTECT ARTISTS FROM GENERATIVE AI', while style-based mimicry is one part of diffusion-based mimicry, there are more basic mimicry e.g. inpainting, style-transfer by diffusion model, which are also tested in previous papers of protection e.g. Mist. Fine-tuning a diffusion model seems to have a more complicated mechanism compared with image-to-image applications of diffusion models. 
- I wonder does the proposed method also work for inpainting/image-to-image SDEdit, if it works, the proposed method becomes more general.

Methods:
- I noticed that the perturbation used in this paper is quite small, if the noise is scaled up, will the purification be worse
- While Glaze and Mist are popular, there are many other protection methods that can be studied to get a safer conclusion. e.g. MetaCloak [3] and SDS [4].

Related Papers:
[1, 2] are highly related to this paper, [1] also find that the current attacks are vulnerable to purifications, [2] proposed that latent diffusion models can be easily purified by pixel-space diffusion models. 

[1] Can Protective Perturbation Safeguard Personal Data from Being Exploited by Stable Diffusion?

[2] Pixel is a Barrier: Diffusion Models Are More Adversarially Robust Than We Think.

[3] MetaCloak: Preventing Unauthorized Subject-driven Text-to-image Diffusion-based Synthesis via Meta-learning

[4] Toward effective protection against diffusion-based mimicry through score distillation

---

confidence: 4
contribution: 3
initial_rating: 8
presentation: 4
questions: 1. It would be interesting to see the results on Glaze 2.1 (the newest version) for completeness’s sake. Does this induce greater quality degradation compared to its previous versions?

2. The “Best-of-4” method is not really a practical method since it depends on the human ratings and the same human ratings are used to evaluate the method. For fair comparison, the raters need to be split into validation and test raters, and the method selection should utilize only the validation raters and be evaluated by the test raters, with averaging across splits via cross-validation.
soundness: 3
strengths: 1.  While it was anticipated that prominent style protection techniques like Glaze might have weaknesses, the extent of their fragility is striking. The paper demonstrates that these methods fail even against rudimentary attacks, with Glaze unable to withstand a mere change in the fine-tuning script. This revelation underscores a concerning lack of “security mindset” within the style protection research community, which is particularly alarming given the recognition Glaze has received, including multiple awards from USENIX Security.

2. The MTurk study is well constructed and the authors have taken pains to ensure that their work adheres to commonly accepted ethical standards.
summary: This paper examines the effectiveness of existing style protection methods (such as Glaze, Mist, and Anti-DreamBooth) against both black-box and white-box attacks. The authors demonstrate that these protections can be easily circumvented using simple techniques like DiffPure, upscaling, and adding Gaussian noise, as well as more sophisticated methods like IMPRESS. The findings are supported by an MTurk study, where participants were asked to distinguish between images generated using style mimicry on unprotected vs protected artworks. The paper highlights the inherent asymmetry between attackers (style mimickers) and defenders (artists), cautioning against relying on adversarial perturbations for style protection due to the potential false sense of security they may provide.
weaknesses: 1. The work has limited novelty since it was predictable that style protection would be vulnerable to DiffPure-like purification methods. It would be good if the purification methods evaluated in the paper could be packaged into a standard baseline, say on the lines of AutoAttack.

2. In my opinion, the paper overstates the general case against style protection techniques based on adversarial perturbation. The presented argument makes an assumption that artists have the choice to release their artworks on the internet, and they may choose to withhold their artworks if they believe that AI models may be trained on their artworks. However, digital artists are extremely dependent on the internet to grow their customer pool and advertise their works, so this is likely not a feasible option. The appropriate counterfactual to artists using Glaze-like methods would be not using any protections at all. Also, since these methods seem to be improving against simple attacks at least, it may be enough to use them as a deterrent rather than as fool-rpoof security.


I struggled to decide whether to rate this paper as borderline accept or clear accept due to the stated weaknesses. Ultimately, I decided to rate this paper as clear accept as the argument may indeed be persuasive to a small minority of artists who may decide to not publish any of their works if there is no secure style protection mechanism.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: See weaknesses above.
soundness: 2
strengths: 1. The writing is clear and easy to follow.
2. The topic is interesting and important.
summary: The paper presents a user study evaluating pre-processing techniques against current adversarial perturbations, finding that such methods can significantly reduce their effectiveness.
weaknesses: The main weakness is the unreliable evaluation and whether the experimental results reflect real-world cases.

1. The study heavily relies on MTurk workers, who may not be suitable for this task. They might lack the expertise to classify artistic styles or assess whether artwork quality has been degraded. Artists themselves, such as those mentioned in [1], should be included in the main evaluation. Additionally, these workers may not represent potential consumers of artists' works, contrary to what the authors propose (L 338). A more thorough study should report the workers' relevance to the art domain (e.g., how often they purchase art or visit museums).
2. The effectiveness of the fine-tuning procedure is unclear. As shown in Fig. 3 (especially the bottom row) and Fig. 5, the generated images, even without protection, do not closely resemble the style of the training images. In contrast, previous work like Glaze [1] uses a fine-tuning setting with much stronger resemblance between generated images and training data (see Fig. 2 in [1]). The focus should be on whether adversarial perturbations effectively defend against mimicry once it has been successfully achieved. Even in Appendix K, where the limitations of using MTurk for evaluation are acknowledged, fewer than 50% of cases were rated as better or equal to “successful” mimicry.
    
    In an extreme case, if fine-tuning involved only one step, adversarial perturbations would likely fail to defend against mimicry. The core issue seems to be underestimating how closely the model need to fit training images, including adversarial perturbations,  to learn the style, which may result in an underestimation of their impact.
    
3. Even if the above weaknesses are overlooked, the paper provides no new insights on solving the problem of mimicry. Both pre-processing and adversarial perturbations degrade image quality, raising the question of whether removing adversarial perturbations is worth the cost, given that pre-processing might degrade the image quality in ways that hinder style recognition.
4. The paper overlooks broader ethical implications, such as the removal of adversarial perturbations used as a watermark. Similar to traditional watermarks (such as a simple icon on the corner) that can be removed but are illegal to do so in many countries, stronger watermarks complicate removal and leave evidence. A discussion on these ethical issues would contribute more meaningfully to the community.
5. The proposed methods lack novelty, largely offering improved hyper-parameter tuning of existing approaches.

[1] Shan S, Cryan J, Wenger E, et al. Glaze: Protecting artists from style mimicry by {Text-to-Image} models[C]//32nd USENIX Security Symposium (USENIX Security 23). 2023: 2187-2204.

---

confidence: 4
contribution: 3
initial_rating: 8
presentation: 3
questions: Overall, the technical details in the paper are clearly described, so I have no questions about the technical aspects. However, I do have some open questions for the authors that I would like to discuss:

+ Q1: Practical and scalable evaluations? The core claim of this paper is to demonstrate the ineffectiveness of existing art mimicry protection methods. For this purpose, using a user study as an evaluation method is sufficient. However, for future papers proposing alternative protective solutions, what could be a scalable way to evaluate their effectiveness?

+ Q2: Is it technically possible to protect artists from mimicry? The conclusions drawn from this work appear to be somewhat pessimistic. Given that generative AI has a strong capability to fit any content, it seems that the potential for mimicry might be inevitable. What could be potential solutions to mitigate this issue?
soundness: 3
strengths: + S1: The motivation is clear and compelling, effectively serving the paper's purpose: to caution researchers about the limitations of using adversarial perturbations for protection against art mimicry.

+ S2: The paper is well-organized and easy to follow. The experiments conducted are solid.

+ S3: The conclusions drawn from this work are potentially significant for the community and could reshape the landscape of this research field. Researchers are encouraged to reconsider the current paradigms of artwork / copyright data protection in light of their practical effectiveness.
summary: This paper studies the practicality of "using adversarial perturbations to protect art works from mimicry". The paper argues that, existing research works which uses adversarial noise for art copyright protection -- although published in top ML/Security conferences -- do not robustly achieve their claimed goals. Simple technical means are enough to bypass these existing protections. The work argues that researchers and practitioners should rethink the solution for art mimicry problem and develop alternative protections.
weaknesses: + Typos
    1. Line 219, fare -> fail?
    2. Figure 11, 13, 14, 15 seem to have the wrong label - now they are all labelled as "gaussian noising"

Meta-review: This paper revisits the solution of "using adversarial perturbations to protect artworks from mimicry". The paper argues that existing research works that use adversarial noise for art copyright protection do not robustly achieve their claimed goals. Simple technical means are enough to bypass these existing protections. Hence, the work indicates that researchers and practitioners should rethink the solution for the art mimicry problem and develop alternative protections. Reviewers agree that this work has the following advantages: (1) It works on an important problem and provides critical insights. It demonstrates that even the well-known protection work has serious weaknesses (2) The proposed robust mimicry methods are simple but effective. (3) The work did extensive experiments to support the claims. (4) The writing is clear and easy to follow. The reviewers also raise some questions: (1) Extension to other generation models like inpainting/image-to-image SDEdit. (2) Extension to other protection methods. (3) Limited novelty. All concerns are well responded by authors during the rebuttal process.

Decision: Accept (Spotlight)