Paper: Toward effective protection against diffusion-based mimicry through score distillation (ICLR-2024)

Reviewer comment (Theory):
The paper lacks insight into why minimizing semantic loss works better.

Author response:
It is an interesting phenomenon that minimizing the semantic loss $\mathcal{L}_{S}(x)$ counter-intuitively fools the diffusion model, here we provide some insights into the possible reasons behind it. 

Remember we have the semantic loss in SDS form can be regarded as the weighted probability density distillation loss (Eq.10 in Appendix G).

Here we can get some insights from the equation, minimizing the semantic loss means minimizing the KL divergence between $q(z_t|x)$ and $p_{\theta}(z_t)$, where $p_{\theta}(z_t)$ is the marginal distribution sharing the same score function learned with parameter $\theta$. Since $p_{\theta}(z_t)$ does not depend on $x$, it actually describes the learned distribution of the dataset smoothed with Gaussian. 

Let's assume in a simple case that the data distribution is exactly the Dirac Delta Distribution of data points in the dataset. Then we have $p_{\theta}(z_t)$ as a Gaussian-smoothed composition of Dirac Delta Distribution if $s_{\theta}$ is perfectly learned. Then minimizing the semantic loss $\mathcal{L}_{S}(x)$ turns into making $q(z_t|x)$ closer to the Gaussian-smoothed composition of Dirac Delta Distribution, and the optimization direction is to make $x$ closer to the spatially average value in the dataset, which brings blurred $x$, and it turns out to be a good protection.

Reviewer comment (Novelty):
The major concern is the limited contribution of this paper. The observation that the encoder is more vulnerable than the denoiser only contributes to this region, and it is limited in developing more secure LDM. Besides, the experiments empirically indicate this conclusion, but it lacks theoretical proofs.

Author response:
> The observation that the encoder is more vulnerable than the denoiser only contributes to this region, it is limited in developing more secure LDM. Besides, the experiments empirically indicate this conclusion. It lacks theoretical proofs.

We believe that `the encoder is more vulnerable` is  a meaningful finding and can contribute new insights to the community to show that the bottleneck of attacking a LDM is the encoder. 


We further show in the **Appendix C** in revision that attacking a pixel-based DM using semantic loss can not work, which proves that attacking a DM is very different from attacking a LDM, where the sucess of attacking a LDM really comes from the encoder.

We showed a lot of evidences in Section 4 to support our claim,  we do not try to focus on give a theoretical analysis but rather call emphasis on our insights. We write further clarification about our contribution in the **General Response**.

Reviewer comment (Novelty):
The authors resort to the existing method, Score Distillation Sampling, to fasten the backpropagation of the semantic loss.

Author response:
> The authors resort to the existing method, Score Distillation Sampling, to fasten the backpropagation of the semantic loss.


SDS is a free lunch, but it is fully omitted by previous methods [1,2,3], which turns out to be too expensive to run. We are the first to apply SDS into the protection framework and did extensive experiments to show that it is effective. Also, we provide theoratical analysis to better explain the SDS loss from the perspective of gradient of KL divergence in revision of **Appendix F** in the protection framework.

Reviewer comment (Evaluation):
The authors find that applying gradient descent over the semantic loss leads to more harmonious adversarial images with the original ones. The authors should delve into this phenomenon.

Author response:
> The authors find that applying gradient descent over the semantic loss leads to more harmonious adversarial images with the original ones. The authors should delve into this phenomenon.

It is an interesting phenomenon that minimizing the semantic loss $\mathcal{L}_{S}(x)$ counter-intuitively fools the diffusion model, here we provide some insights into the possible reasons behind it. 

Remember we have the semantic loss in SDS form can be regarded as the weighted probability density distillation loss (Eq.10 in Appendix G).

Here we can get some insights from the equation, minimizing the semantic loss means minimizing the KL divergence between $q(z_t|x)$ and $p_{\theta}(z_t)$, where $p_{\theta}(z_t)$ is the marginal distribution sharing the same score function learned with parameter $\theta$. Since $p_{\theta}(z_t)$ does not depend on $x$, it actually describes the learned distribution of the dataset smoothed with Gaussian. 

Let's assume in a simple case that the data distribution is exactly the Dirac Delta Distribution of data points in the dataset. Then we have $p_{\theta}(z_t)$ as a Gaussian-smoothed composition of Dirac Delta Distribution if $s_{\theta}$ is perfectly learned. Then minimizing the semantic loss $\mathcal{L}_{S}(x)$ turns into making $q(z_t|x)$ closer to the Gaussian-smoothed composition of Dirac Delta Distribution, and the optimization direction is to make $x$ closer to the spatially average value in the dataset, which brings blurred $x$, and it turns out to be a good protection.

Reviewer comment (Experiments):
From my understanding, this method needs the gradient from the misused models. That is, the protection perturbation is generated on the exact model used to maliciously manipulate the image. It's not clear if this generated perturbation is only effective on that model, or if it can protect the image from misuse by an unknown model.

Author response:
That is a good question. Here we focus on the **white-box settings** the same as previous works  [1, 2, 3]. For the black-box settings as the reviewer mentioned, recent paper [4] found that the adversarial samples can transfer between the most popular publicly-avaible LDMs like SD-v1.4, SD-v1.5 and SD-v2. We also did experiments to show that the adversarial samples generated on the basic SD-v1.4 can be transferred to SD-v1.5 and SD-v2. We put the results in Appendix E of the revision following your suggestion.

Reviewer comment (Theory):
It's not clear why minimizing the loss (encouraging LDM to make better predictions) can protect the images. It's very counter-intuitive and needs a better explanation. I can understand that minimizing the loss with respect to certain targets as Photoguard can work because it encourages the LDM to generate an image different from the original image to protect. But why can encouraging the LDM to generate a better image of the original image (the opposite goal of AdvDM) can help? Maybe it's kind of trapping the sample into a local optimum so that the later optimization/editing is also stuck? But this also needs to be justified.

Author response:
It is an interesting phenomenon that minimizing the semantic loss $\mathcal{L}_{S}(x)$ counter-intuitively fools the diffusion model, here we provide some insights into the possible reasons behind it. 

Remember we have the semantic loss in SDS form can be regarded as the weighted probability density distillation loss (Eq.10 in Appendix G).

Here we can get some insights from the equation, minimizing the semantic loss means minimizing the KL divergence between $q(z_t|x)$ and $p_{\theta}(z_t)$, where $p_{\theta}(z_t)$ is the marginal distribution sharing the same score function learned with parameter $\theta$. Since $p_{\theta}(z_t)$ does not depend on $x$, it actually describes the learned distribution of the dataset smoothed with Gaussian. 

Let's assume in a simple case that the data distribution is exactly the Dirac Delta Distribution of data points in the dataset. Then we have $p_{\theta}(z_t)$ as a Gaussian-smoothed composition of Dirac Delta Distribution if $s_{\theta}$ is perfectly learned. Then minimizing the semantic loss $\mathcal{L}_{S}(x)$ turns into making $q(z_t|x)$ closer to the Gaussian-smoothed composition of Dirac Delta Distribution, and the optimization direction is to make $x$ closer to the spatially average value in the dataset, which brings blurred $x$, and it turns out to be a good protection.

Reviewer comment (Evaluation):
In Section 4.3, the conclusion of denoiser being more robust needs more evidence. Figure 2 shows the $\delta_z$ can be less than 10x $\delta_x$ for many cases. So I think the 10x budget (Figure 10 has even larger budgets) should be large enough to conduct some successful attacks. It's not clear why they fail.

Author response:
That is a good question. This exactly supports our claim that the bottleneck is the encoder, when attacking the z-space alone, we do not use the gradient from the encoder, which brings a weak attack to the encoder itself. As we can see in Figure 10, although the budgets are getting even larger, the edited results do not diverge a lot from x_adv (which may be quite noisy since we perturb the z-space).

In conclusion, even given the same budget, simply attacking the z-space can not give us a good direction to attack the LDM. Encoder is really important and is the key when attacking a LDM.

To better support our conclusion, we did experiments (in **Appendix C** in revision) on a pixel-based diffusion model without encoder-decoder, and we found that gradient-based methods fail to work. That is, if we attack the DM using PGD and semantic loss, the generated adversarial sample cannot even fool the DM, the reason behind this may be the randomness brought by Gaussian noise term, making the denoiser quite robust to be attacked directly.

Reviewer comment (Experiments):
Photoguard has a diffusion attack. Why is only the encoder attack mentioned in the related work and evaluated in the experiments?

Author response:
The diffusion attack in Photoguard is an end-to-end attack against SDEdit-based inpainting, which means we need to final edit results over the inputs, which contain the chain of U-Net. It will take like 50G and even more on our settings, which is actually not a practical attack, also in previous works [1, 3] we only compare the encoder attack in PhotoGuard.

Reviewer comment (Reproducibility):
The detailed setup for the experiments is missing. For example, how many iterations are used for the baselines and proposed method? What is the learning rate? What's the perturbation budget? How large is the fixed $\delta_x$ in Figure 2?

Author response:
For all the methods (baselines and our method) we use the PGD attack to generate adversarial samples, and we use $100$ as the number of timesteps and $\delta=16/255$ (fixed $\delta$ in Figure 2.) and step-size as $1/255$. We put this in the revision in **Appendix B.2**.

Reviewer comment (Reproducibility):
The pseudocode in Appendix A doesn't allow `x` to have gradients because `x=x.detach().clone()` detaches `x` from the computational graph.

Author response:
Thanks for your careful reading. We miss `x.requires_grad=True` to enable the gradient calculation in the pseudo-code. We have fixed this in the revision.

Reviewer comment (Evaluation):
I suggest briefly introducing the metrics and human studies and having a more detailed explanation of the experimental results in the main text.

Author response:
We are glad to consider this suggestion in our final version and add more explanation of the experimental in the main text and move some quantitative results to the appendix for enough space.

Reviewer comment (Experiments):
To show the effectiveness of SDS approximation, one may visualize the loss changes and the gradient landscapes with the original loss function and the SDS loss.

Author response:
That is a good suggestion, we followed it and visualized the loss function in **Appendix F** of the new revision.
Also, we provide theoretical analysis to better explain the SDS loss from the perspective of the gradient of KL divergence in the revision of **Appendix F**. We borrowed the idea from [5] to write the proof.

Reviewer comment (Experiments):
Can this protection effect transfer to different LDMs? That is, assume the image is protected using a LDM A, when the adversary uses another LDM B to edit the image, will it fail?

Author response:
That is a good question. Here we focus on the **white-box settings** the same as previous works  [1, 2, 3]. For the black-box settings as the reviewer mentioned, recent paper [4] found that the adversarial samples can transfer between the most popular publicly-avaible LDMs like SD-v1.4, SD-v1.5 and SD-v2. We also did experiments to show that the adversarial samples generated on the basic SD-v1.4 can be transferred to SD-v1.5 and SD-v2. We put the results in Appendix E of the revision following your suggestion.

Reviewer comment (Experiments):
Can image transformations such as JPEG compression, cropping, rotation, and rescaling invalidate the protection?

Author response:
We test some famous defending methods, including JPEG compression, AdverseClean (https://github.com/lllyasviel/AdverseCleaner) and crop-and-rescale on the adversarial samples. We put additional results in Appendix D following your suggestion, we found that crop-and-resize turns out to be relatively effective, but it still cannot work well to remove the perturbations. This finding is in accordance with that also found in [3].

Reviewer comment (Theory):
Why can minimizing the loss (encouraging LDM to make better predictions) protect the images?

Author response:
It is an interesting phenomenon that minimizing the semantic loss $\mathcal{L}_{S}(x)$ counter-intuitively fools the diffusion model, here we provide some insights into the possible reasons behind it. 

Remember we have the semantic loss in SDS form can be regarded as the weighted probability density distillation loss (Eq.10 in Appendix G).

Here we can get some insights from the equation, minimizing the semantic loss means minimizing the KL divergence between $q(z_t|x)$ and $p_{\theta}(z_t)$, where $p_{\theta}(z_t)$ is the marginal distribution sharing the same score function learned with parameter $\theta$. Since $p_{\theta}(z_t)$ does not depend on $x$, it actually describes the learned distribution of the dataset smoothed with Gaussian. 

Let's assume in a simple case that the data distribution is exactly the Dirac Delta Distribution of data points in the dataset. Then we have $p_{\theta}(z_t)$ as a Gaussian-smoothed composition of Dirac Delta Distribution if $s_{\theta}$ is perfectly learned. Then minimizing the semantic loss $\mathcal{L}_{S}(x)$ turns into making $q(z_t|x)$ closer to the Gaussian-smoothed composition of Dirac Delta Distribution, and the optimization direction is to make $x$ closer to the spatially average value in the dataset, which brings blurred $x$, and it turns out to be a good protection.

Reviewer comment (Presentation):
How are the $z$ and $z_{adv}$ visualized in the figures such as Figure 2?

Author response:
The size of the z-space in a diffusion model is actually 4*64*64, we visualize the first three channels in RGB in figures such as Figure 2, which can already effectively show the large differences in the z-space.

Reviewer comment (Evaluation):
Missing details when comparing the magnitude of embedding change versus input change. The authors use absolute value to compare adversarial noise. While the embedding can be of different scale. The authors shall provide a relative scale of perturbation magnitude.

Author response:
As we mentioned in Sec 4.1: line 5, we normalized the perturbations in the two spaces for comparison. Specifically, we scale the perturbations in the z-space back to [0, 1] as in the x-space.

Reviewer comment (Theory):
Unclear math derivation. In equation 5, why can we approximately remove the gradient parts from denoiser?

Author response:
This is a good question. Approximating the jacobian of U-Net as an identity matrix is used in many works, such as distillation from 2D [1, 2] and gradient approximation in adversarial attack [3]. While approximation is the most straightforward explanation, it can also be regarded as the weighted probability density distillation loss, we provide detailed analysis in **Appendix G** in the revision.

Also, by visualizing the loss curve in **Appendix F**, we better support the claim and it turns out that using SDS loss is a free lunch that should be noticed and applied when designing a protection method.


[1] DreamFusion: Text-to-3D using 2D Diffusion

[2] ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation

[3] Diffusion-based adversarial sample generation for improved stealthiness and controllability