Paper: Understanding and Improving Adversarial Attacks on Latent Diffusion Model (ICLR-2024)

Reviewer comment (Experiments):
Only one defense method is evaluated. Are the adversarial samples robust to other transformations such as compression or adding Gaussian noises? Also, no adaptive defense is evaluated. If the infringers know about this adversarial attack, can they adaptively mitigate the adversarial effects?

Author response:
We have conducted experiments on other common defending methods (adding gaussian noises, JPEG compression and resizing). Numerical and visualization results show that our method is also robust to these defenses. We’ve also explored some adaptive defense strategies, and we highly agree on the academic significance of such defenses. However, as the goal of our method is to help the community to protect their copyrighted content, we think publishing techniques that aim to overcome the attack is not socially responsible. Therefore, we’ll not release any details of such strategies in our paper.

Reviewer comment (Novelty):
From my understanding, Liang & Wu, 2023 and Salman et al., 2023 are not just "targeting the VAE" as claimed in this paper. They attacked the UNet as well.

Author response:
Yes indeed. Thanks for your correction! We’ve corrected our description in the revision.

Reviewer comment (Writing):
Many issues in the writing.

Author response:
Thanks for your detailed review, we’ve corrected and updated the paper based on the given points.

      4.1  We’ve reconstructed our paper in a more fluent way and corrected grammar mistakes.

      4.2  We’ve updated the references and the figures.

      4.3  We’re sorry for the typographical errors. They have been fixed in the revision.

      4.4  We did have inconsistent symbol definition in different sections. In the new version, we carefully choose our symbols and define them at the very beginning. 

      4.5  We’ve corrected our citation mistakes in the revision.

Reviewer comment (Writing):
On Page 2, "serve as a means to" -> "mean".

Author response:
Thanks for your detailed review, we’ve corrected and updated the paper based on the given points.

      4.1  We’ve reconstructed our paper in a more fluent way and corrected grammar mistakes.

      4.2  We’ve updated the references and the figures.

      4.3  We’re sorry for the typographical errors. They have been fixed in the revision.

      4.4  We did have inconsistent symbol definition in different sections. In the new version, we carefully choose our symbols and define them at the very beginning. 

      4.5  We’ve corrected our citation mistakes in the revision.

Reviewer comment (Presentation):
In Section 2.2, it says "As shown in Figure 1 adversarial attacks create an adversarial example that seems almost the same as real examples". However, Figure 1 only contains the generated images by the infringers instead of the adversarial examples as indicated by the title.

Author response:
Thanks for your detailed review, we’ve corrected and updated the paper based on the given points.

      4.1  We’ve reconstructed our paper in a more fluent way and corrected grammar mistakes.

      4.2  We’ve updated the references and the figures.

      4.3  We’re sorry for the typographical errors. They have been fixed in the revision.

      4.4  We did have inconsistent symbol definition in different sections. In the new version, we carefully choose our symbols and define them at the very beginning. 

      4.5  We’ve corrected our citation mistakes in the revision.

Reviewer comment (Presentation):
The references to figures and tables are incorrect such as "Figure 5.3", "Table 5.2", "Table 5.3", etc. In the ablation study, the caption of the table is "Figure 5".

Author response:
Thanks for your detailed review, we’ve corrected and updated the paper based on the given points.

      4.1  We’ve reconstructed our paper in a more fluent way and corrected grammar mistakes.

      4.2  We’ve updated the references and the figures.

      4.3  We’re sorry for the typographical errors. They have been fixed in the revision.

      4.4  We did have inconsistent symbol definition in different sections. In the new version, we carefully choose our symbols and define them at the very beginning. 

      4.5  We’ve corrected our citation mistakes in the revision.

Reviewer comment (Writing):
Some math symbols are not defined where they first appear. For example, It would be better to mention that $\phi$ in Section 2.1 means the VAE. I suggest to use $\mathcal{E}\_{\phi}$ or $\mathcal{E}$ consistently. What does $\sqrt{\bar{\alpha\_t}}$ (the line below equation 5) mean? The text below Equation 10, for "$q\_{\phi}(v\_t | x')$ and $q\_{\phi}(v\_t | x')$", the second one should be $q\_{\phi}(v\_t | x)$. It would be better to briefly explain the N, M, and K in Algorithm 2.

Author response:
Thanks for your detailed review, we’ve corrected and updated the paper based on the given points.

      4.1  We’ve reconstructed our paper in a more fluent way and corrected grammar mistakes.

      4.2  We’ve updated the references and the figures.

      4.3  We’re sorry for the typographical errors. They have been fixed in the revision.

      4.4  We did have inconsistent symbol definition in different sections. In the new version, we carefully choose our symbols and define them at the very beginning. 

      4.5  We’ve corrected our citation mistakes in the revision.

Reviewer comment (Writing):
The citation of SR in section E.2 is wrong. "Salman et al., 2023" -> "Mustafa et al. 2019".

Author response:
Thanks for your detailed review, we’ve corrected and updated the paper based on the given points.

      4.1  We’ve reconstructed our paper in a more fluent way and corrected grammar mistakes.

      4.2  We’ve updated the references and the figures.

      4.3  We’re sorry for the typographical errors. They have been fixed in the revision.

      4.4  We did have inconsistent symbol definition in different sections. In the new version, we carefully choose our symbols and define them at the very beginning. 

      4.5  We’ve corrected our citation mistakes in the revision.

Reviewer comment (Experiments):
Can the adversarial examples be effective for different or unknown models/methods?

Author response:
Yes they can, as mentioned in the first point in the weaknesses section.

Reviewer comment (Theory):
According to section E.1, the target $\mathcal{T}$ in Equation 15 and 16 is defined as $\mathcal{E}(x^{\mathcal{T}})$. I can understand for the $\mathcal{L}\_{vae}^{\mathcal{T}}$, it's meaningful to encourage $\mathcal{E}(x')$ to be close to $\mathcal{E}(x^{\mathcal{T}})$. However, for the UNet part, what's the rationale to encourage the predicted noise at each timestep to be close to $\mathcal{E}(x^{\mathcal{T}})$? Because I think the final output $z_0$ should be close to $\mathcal{E}(x^{\mathcal{T}})$, but not the intermediate predicted Gaussian noise.

Author response:
The most intuitional way is indeed to let the output $z_0$ be close to $\mathcal{E}(x^T)$. And that is what the targeted version of Anti-DreamBooth intends to achieve(for detailed mathematical formulation of targeted Anti-DB please refer to section 3.1). However, results have shown the inferiority of this method, even compared to un-targeted ones. The intuition of our method is to “trick” the prediction of U-Net to have consistent errors under various timesteps, which can then be accumulated in the denoising process. By trying to “pull” the prediction close to a fixed target, we can control the prediction error to be consistent and have semantic similarity to the target ($\mathcal{E}(x^T)$), therefore, adding strong target-dependent semantic distortion to the output image. We have a great visualization for this intuition and give intuitive explainations for why targeted Anti-DB doesn’t work(Please refer to Appendix B.2).

Reviewer comment (Experiments):
More thorough examination accounting for a wider range of generative techniques could further validate the method's real-world utility and limitations. While the proposed attack focuses on the prevalent LDM framework, its generalization to other powerful generative paradigms like SDXL, DALL-E, and Deep Floyd remains untested.

Author response:
Unfortunately we don’t have enough time and resources to deploy SDXL and DeepFloyd IF and conduct experiments on them within the rebuttal period. But we evaluated our method on SD1.4,SD1.5 and SD2.1. Additionally, we add new experiments on cross-model transferability in Section 5.3. It shows that our attacks not only work well on Stable Diffusion 1.4 and Stable Diffusion 2.1 but also enjoy cross-model transferability that attacks whose backbone is SD1.5 work well on SD1.4 and SD2.1. Hence, we believe this can answer the question. Supported by cross-model experiment results, we’re quite confident about the performance of this method in the range of LDMs. We’ll continue on exploring our methods’ effect on more powerful generative paradigms in the future and update the results.

Reviewer comment (Experiments):
A more powerful baseline of PhotoGuard, i.e. Diffusion Attack is not compared to. This comparison could help gauge the true leadership of the new method. Without including this more powerful adversarial technique, the paper's claims about the proposed attack outperforming the current approaches remains uncertain.

Author response:
We highly agree that the comparison between Diffusion Attack and ours is needed for deciding the leader of this field. However, the high demand of Diffusion Attack on GPU has been a challenge to our limited computation resources. Diffusion Attack requires a lot of time and more importantly, gpu memory. Under bf16/fp16 and our memory efficient optimization, Diffusion Attack still requires about 17G of gpu memory to run an attack on one single image and requires a much longer time for one round of PGD attack than other methods. This makes it hard to make a fair comparison between Diffusion Attack and ours. We’ll conduct the experiment as soon as our computation resource is upgraded(will soon!). Before we have the results, we add an additional explanation in the paper about this issue.

Reviewer comment (Experiments):
The authors assert a memory-efficient design but do not provide details to support this claim. Further explanation or experimental evaluation of memory usage compared to alternative approaches would help validate the proposed method's efficiency advantages.

Author response:
Thank you for your advice! We admit the details in our original version are not sufficient, and we recognize the importance of memory efficiency, as most personal computers wouldn’t have as much GPU memory as researchers do. In further experiment, we find that the numerical results of original results is higher than the actual usage, possibly due to a more aggressive caching strategy on GPUs with larger memory. We retest our result after disentangling the caching factor and add more detailed information about our strategy. We’ve also tested other methods and made numerical evaluations, showing our advantages in memory efficiency. Please refer to section 4 for more details.

Reviewer comment (Novelty):
The proposed theoretical framework is not sufficiently innovative.

Author response:
We agree that the previous version of our paper includes theoretical results that are trivial and cannot explain some observations. Hence, we remove the theory and replace it with the intuitive process that yields the proposed method. We sincerely hope that the reviewer can read the new version of our paper and give advice, which will significantly help us refine our paper furthermore.

Reviewer comment (Novelty):
The key formulation of minimizing the conditional likelihood is trivial. The given theoretical proof is complicated. Actually, the likelihood equivalent to the KL divergence has been well-known. From this perspective, the proof is somehow trivial.

Author response:
We agree that the previous version of our paper includes theoretical results that are trivial and cannot explain some observations. Hence, we remove the theory and replace it with the intuitive process that yields the proposed method. We sincerely hope that the reviewer can read the new version of our paper and give advice, which will significantly help us refine our paper furthermore.

Reviewer comment (Theory):
There exists many wrong equations. In the first paragraph of Sect 3.2, q(v_t|v_{0:t-1} is mistakenly formulated.

Author response:
b) We agree that it is wrongly formulated. However, the theoretical speculation afterwards only exploits q(v_t|v_0), which has a closed form. Hence, this error does not influence the speculation a lot.

Reviewer comment (Theory):
There exists many wrong equations. In the last paragraph of Page 4, z_{t-1} is mistakenly formulated.

Author response:
c) We agree that it is wrongly formulated and should be $z_t\sim q(z_t|x)$.

Reviewer comment (Theory):
There exists many wrong equations. In Eq. (3), the sum in terms of z is mistakenly formulated given the expectation.

Author response:
d) Eq 3. is the definition of the adversarial attack on LDM. Hence, we are not 
sure what mistakes there are in the definition. We guess it might be the formulation of expectation that we put the condition outside of the probability. We respond to this problem in e.

Reviewer comment (Evaluation):
From the empirical results (Table 1), the strategy of combining two adversarial attacks does not perform significantly better than the Eq. (16). This raises doubt about the effectiveness of the newly proposed attack method.

Author response:
We agree that the difference in performance cannot be explained by the theory in the first version of our paper. This is the main reason why we remove it in the new version of our paper. Instead, we attribute the success of Eq 16. (in the old version, now newly named Improved Targeted Attack, ITA) to the introduction of the target and the misalignment between human vision and diffusion models. Eq 15 (in the old version) now only serves as an easy-to-implement variant of Eq. 15 which sometimes shows superiority over SDEdit.

Reviewer comment (Presentation):
Some references are wrongly denoted, e.g. Figure 5.3.

Author response:
Minor: Thanks for your correction, we’ve updated to the correct references.

Reviewer comment (Theory):
The methodology exists many errors.

Author response:
We agree that the theoretical framework cannot work in explaining the result of our proposed methods. Hence, we replace it with the intuitive process about how we yield our proposed methods and try to provide some empirical observations on why our methods outperform. Again, we apologize for our mistaken introduction of the theoretical part in the first version of our paper and sincerely hope that the reviewer can give us more advice on the updated version.

Reviewer comment (Experiments):
The experimental verification are not sound.

Author response:
We highly understand the concern about our experimental verification. We’ve conducted more experiments including cross-model transferability, more ablation studies,denoising countering and so on. We’ve also added more visualization to side-prove our views and numerical results and added more details in experiment setups. We hope the new version of our paper can somewhat relieve the concern.

Reviewer comment (Presentation):
The pseudo-code of algorithm 1 seems redundant and demonstrates nothing. It literally equals to its description: “To optimize J_{ft}, we first finetune the model on x and obtain the finetuned model θ(x). Then, we minimize J_{ft} over x′ on the finetuned model θ(x).”

Author response:
We thank the reviewer for the advice. We have merged Algorithm 1 to original Algorithm 2 to become the new Algorithm 1 in the new version of our paper.

Reviewer comment (Experiments):
The target image adopted by Liang & Wu (2023), visualized in “Figure E.1” (Is it mislabeled in Figure 8?), is the only target image used in the experiments. Did the authors try using different target images? Will the target image affect the effectiveness of this method?

Author response:
Yes, it does. Our new experiments in Appendix B.1 show that picking natural images as the target image yields trivial performance. Section 3.2 also demonstrates new experiments to show that it would be better to pick images with tight patterns and high contrast as the target image.

Reviewer comment (Presentation):
There are issues in the document layout. The labels of figures are mismatched to those in the texts.

Author response:
We have fixed this problem in the new version of our paper.

Reviewer comment (Presentation):
The offset problem needs to be clarified. The authors claim that “The result in Figure 5.3 implies that offset takes place in 30% - 55% of pixel pairs in Δ_z_t and Δ_ε_θ, which means that maximizing J_q pulls Δ_z_t to a different direction of Δ_ε_θ and interferes the maximization of J_p.” Could the authors further explain it and Figure 3 in Section 4.1?

Author response:
We have removed the offset rate in the new version of our paper since we find that it cannot explain empirical observations.

Reviewer comment (Experiments):
Will the target image affect the effectiveness of this method? Would the authors use other images as targets and test the effectiveness?

Author response:
Yes, it does. Our new experiments in Appendix B.1 show that picking natural images as the target image yields trivial performance. Section 3.2 also demonstrates new experiments to show that it would be better to pick images with tight patterns and high contrast as the target image.

Reviewer comment (Writing):
Is that a typo in “In this tractable form, z′_t and z_t sampled from q_φ(v_t|x′) and q_φ(v_t|x′), respectively” below Equation 10?

Author response:
No, the old version of our paper reformulates the intermediate variable z_t as v_t in the forward process. However, all these parts are removed in the new version.

Reviewer comment (Theory):
How does maximizing J_q pull Δ_z_t to a different direction of Δ_ε_θ and interfere the maximization of J_p? Could the authors further explain Figure 3 in Section 4.1?

Author response:
We remove this theory since we find that some observations cannot be explained by the theory.

Reviewer comment (Theory):
The paper formulates an adversarial optimization problem particularly tailored for the latent diffusion models (LDM). The analysis guides the algorithm design to some degree. However, due to the lack of clarity and various approximations being introduced without proper justification, the theoretical results become less convincing.

Author response:
We sincerely thank the reviewer for the insightful and detailed review, which greatly helps us in the improvement of the new version of our paper. As stated in our official comment visible to all reviewers, we remove the main part of the theory in the new version of our paper and replace it with the process of how we yield our methods intuitively. We believe this can solve most of the theoretical problems raised by the review. Also, we conduct novel ablation studies on target selection to answer the question about the empirical effectiveness of our method.

Following are our point-by-point answers of the questions raised in the review. Although most of the theoretical problems are cleared by removing the old theory from the paper, we still respond to them based on the old version of our paper. We hope the reviewer can give opinions about the new version of our paper, which would be very valuable to us.

Reviewer comment (Theory):
If my previous understanding is correct, my next question is why should the adversary care about the latent variables in the intermediate steps of the diffusion process instead of, say, the final step of the inverse process before the decoder?

Author response:
This is because the training and the sampling process is separate. Existing adversarial attacks usually attack the training process because attacking sampling process is very memory-expensive (imaging doing backprop throughout UNet for 25 or 50 times back to the input). Under this condition, considering the fact that the exact goal is to fail the sampling process and that the sampling process is done by step-by-step sampling of intermediate latent variables, it will be better if the attacker tries to make every intermediate latent variable out-of-distribution. This will interfere with every step of the sampling rather than just the final step.

Reviewer comment (Theory):
Based on the text, Eq. (3) should be equivalent to $\mathbb E_{z \sim p_{\theta}(z|x)}[- \log p_\theta(z|x')]$. My question is that a slightly different formula $\mathbb E_{z \sim p_{\theta}(z|x')}[- \log p_\theta(z|x) + \log p_{\theta}(z|x')]$ also seems appropriate (swapping order in the KL-divergence). Why should we prefer one to the other?

Author response:
This is an interesting idea. But as our original theory cannot well explain some observations, we would leave the discussion in the future work.

Reviewer comment (Writing):
Section 3.2 uses the notation $\mathcal N(\mathcal E(x), \sigma_\phi)$ instead of $\mathcal N(f_{\mathcal E}(x), \sigma_{\mathcal E})$ from Section 2.1. Do they refer to the same quantity?

Author response:
Yes, this is a clerical error.

Reviewer comment (Writing):
Related to the previous question, it is mentioned that “the variance of $z_{t-1}$ is estimated by sampling and optimizing over multiple $z_{t-1}$.” It is very unclear what “sampling” and “optimizing” refer to here.

Author response:
Sampling means that we sample a z_t from q(z_t|x) and optimizing means that we optimize the function based on $z_{t-1}$ which is sampled from p(z_t|z_{t-1}). This is a Monte Carlo approximation since we sample z_t for multiple times and take every z_t as a fixed value to conditionally sample $z_{t-1}$.

Reviewer comment (Writing):
$x^{(eval)}$ is mentioned in Section 3.4 but was never defined.

Author response:
This is a clerical error. $x^{eval}$ should be x from the dataset. We use these x to evaluate how well the fine-tuned model learns the dataset.

Reviewer comment (Theory):
I do not largely understand the purpose of the “offset” problem in Section 4.1. In my understanding, most of the discussion around the offset can be concluded by simply expanding the second term on the first line of Eq. (13). So the problem that prevents optimizing just the norm of $\Delta z_t$ and the norm of $\Delta \epsilon_\theta$ directly is the last term in the equation above (the dot product or the cosine similarity). I might be missing something here so please correct me if I’m wrong.

Author response:
We agree that the goal can be simply expanded then optimized. That is the main reason for removing our theory. We try to optimize this goal but find that it is empirically weaker than both Eq.16 (named ITA in the new version) and Eq. 15 (named ITA+ in the new version) in the old version of our paper. We do not include the result in the new version of the paper since we remove the introduction of this goal. Our theory cannot explain this phenomenon so that it is removed.

Reviewer comment (Theory):
It is also unclear to me how the last line of Eq. (13) is reached and what approximation is used.

Author response:
The approximation is about merging the first two terms of the middle line in Eq.13. The sum expectation of $\Vert\delta z_t\Vert^2_2$ approximates $\Vert\mathcal{E}(x)-\mathcal{E}(x’)\Vert^2_2$. However, we rethink the approximation and believe it is not appropriate.

Reviewer comment (Experiments):
In theory, there is nothing preventing one from optimizing Eq. (13) as is. The issue seems to be empirical, but I cannot find the empirical results showing the failure of optimizing Eq. (13) directly and not using the target trick.

Author response:
We try to optimize this goal and it performs similarly to the goal we discuss in the first point. Since it does not work well and is not related to our revised method, we do not include the result in the new version of our paper.

Reviewer comment (Writing):
In the first paragraph of page 7, it is mentioned that “meanwhile, since the original goal is to maximize the mode of the vector sum of…” I think instead of “mode,” it should be “magnitude” or the Euclidean norm.

Author response:
Yes, it does mean magnitude.

Reviewer comment (Experiments):
After inspecting the generated samples in Figure 11-15, my hypothesis is that the major factor contributing to the empirical result is the target pattern and the usage of the targeted attack. The pattern is clearly visible on the generated images when this defense is used, and this pattern hurts the similarity scores. This raises the question of whether the contribution comes from the theoretical formulation and optimization of the three objectives or the target. I would like to see an ablation study on this finding: (1) the proposed optimization + untargeted and (2) the prior attacks + targeted.

Author response:
We understand that one major concern is whether the choice of target or our optimization is the key to our results. We admit that this could be an issue due to our lack of related ablation studies in the first version of our paper. Therefore, we add visualization results of choosing different targets and compare our optimization with the target version of Anti-DB. Results show that both the choice of the target and the optimization goal is the key to our strong empirical results. Please refer to Appendix B.1 for more details.

1. We agree that targeted attack is the main contribution, which has now been the main topic of our method. We give out experiments on how the contrast and the pattern repetition of the target image impacts the targeted attack and discuss some empirical experience to pick the target image in the Section 3.2 of our new paper. For the recommended ablation studies, (1) The untargeted version of our attack should be Anti-DB, included in our baselines. We provide quantitative comparison and visualization to compare Anti-DB with our attacks. We also give a visual comparison between the targeted version of Ant-DB(please refer to section 3.1 for detailed definition) and our attacks under the same targets(Please refer to Appendix B.1).  As for (2) the prior attacks + targeted, we conduct a case study on it in Appendix A.2. Empirically, it is much weaker than our proposed attacks.

Reviewer comment (Experiments):
The choice of the target $\mathcal T$ is ambiguous. While the target pattern is shown in the Appendix, there is no justification for why such a pattern is picked over others and whether other patterns have been experimented with.

Author response:
As previously mentioned, Section 3.2 of the new version of our paper now discusses the issues about target selection. Also, Figure 4 shows the visualization results of choosing targets within the domain of natural images. We also give a visualization in Appendix B.3 to give intuitive explanations about the empirical success of our method.

Reviewer comment (Theory):
What are the approximations made on the fourth line of Eq. (22) and in Eq. (23)?

Author response:
Both share the same approximation. We exchange the sequence of the expectation.

Reviewer comment (Evaluation):
Why are MS-SSIM and CLIP-SIM used as metrics for SDEdit whereas CLIP-IQA score is used for LoRA? The authors allude to this briefly, but it still largely remains unclear to me.

Author response:
SDEdit conducts minor modification to the image and does not change the structure of the image and only modify details. As shown in Figure 1, all outputs of SDEdit are structurally similar to the original input data. Hence, we use MSSSIM as the structural similarity metric to measure the attack performance. Just as mentioned, we also want to measure the semantic difference (e.g. style difference), thus CLIP-SIM is used to assess the similarity of the images in the latent space of CLIP(trained by aligning text and images). We believe that CLIP-SIM is a good metric to evaluate the high-level semantic distortion of our adversarial attacks. For the same purpose, we use CLIPIQA, a clip-based, non-reference image quality metric to evaluate our text-to-image results. We’ve made a mistake by claiming both CLIP-SIM and MSSSIM both measure the structural differences in our original paper. Thanks for your correction, we’ve updated our explanation of the metrics. Please see “SDEdit & Metrics” under section 5.1 for the updated details.

Reviewer comment (Reproducibility):
For the results reported in Table 1, how many samples or runs are they averaged over? Based on the experiment setup, 100 images are used for training the model in total for each dataset, and they are grouped in a subset of 20. So my understanding is that there are 100/20 = 5 runs where 100 images are generated in each run? Is this correct?

Author response:
Yes, it is correct.

Reviewer comment (Experiments):
Have the authors considered any “adaptive attack” where the adversary tries to remove the injected perturbation on the images (e.g., denoising, potentially via another diffusion model) before using them for fine-tuning?

Author response:
For most common purification methods(e.g. JPEG compression, adding gaussian noise) , we have evaluated them in Section 5.4. Results show our methods are robust to those methods. However, we do not think new potential adaptive purification methods should be discussed in this work. Currently, no specific adaptive purification has been designed to remove the adversarial attacks on LDM. We highly agree with the academic significance of such adaptive purifications. But such purification may help the malicious individual to overcome the adversarial noise and steal copyrighted content from artists and organizations. Since these adversarial attacks on LDM are for social goodness, we do not think releasing such adaptive purification is socially responsible. Such technical attempts should be stopped by rules. Therefore, though we’ve explored some potential methods for pure academic purposes, we would not release them in our paper.