Kati8/ICLR_2023_2025 huggingface dataset:
authors: Liu Yuezhang
authors: Xue-Xin Wei
reviews: 
Comments: Theoretically, we illustrate how a purification system with randomness can cause gradient masking, which can not be addressed by the standard expectation-over-time (EOT) method. Experimentally, we verify that gradient masking indeed happens under previous evaluations of diffusion models. This paper studies an interesting phenomenon in the adversarial purification based on diffusion models. However, I am still concerned about the experiments.
Strengths: 1. The idea of studying the connection between Diffusion-based adversarial purification and gradient masking is novel.
2. The theoretical result provide some new insights.
Weaknesses: 1. I am not fully convinced by the experiments that the diffusion-based adversarial purification causes the gradient masking. Here, the problem is whether there is measure of the gradient masking phenomenon.
2. The datasets are limited. Can the authors provide further experiments to verify their theoretical findings, i.e. performing experiments on other datasets, like Imagenet and MNIST?
Recommendation: 5: marginally below the acceptance threshold
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 2 fair
Clarity: 3 good
reviews: 
Comments: This paper theoretically reveals the phenomenon that purification system with randomness provides false sense of robustness. A method named "randomness replay" is further proposed to better estimate the robustness. Experiments on Diffusion Models verify the proposals.
Strengths: - The theoretical analysis is sound and easy to understand (e.g., illustration in Figure 1).
- The paper is easy to read.
Weaknesses: - The structure of the paper is somewhat confusing. Diffusion models and gradient masking appear and mentioned in Abstract and Introduction, but most of the theoretical analysis are irrelevant with them. 
- The experimental results shown in Tables seem weak. It will be better if more results are provided.

- Overall, this work is based on the DiffPure framework and present its one limitation. Moreover, attack on a system with randomness has been sudied. Therefore, I think the novelty is limited.

- Writing issue:
   - "Since the successful rate of applying the attack is the cdf of χ2(n), and n is the dimensionality of the data, which is typically very high for images (i.e., for CIFAR-10, n = 32 × 32 × 3 = 3072)."
Recommendation: 5: marginally below the acceptance threshold
Reviewer confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 3 good
Clarity: 3 good
reviews: 
Comments: Besides generating stunning images, diffusion models have been shown to be very suitable defense for adversarial robustness, which later became an area of research called "Adversarial Purification". This paper first theoretically distinguishes the role of individual components: randomness (gradient masking) of synthesis process and learned generative prior. Then they experimentally (on CIFAR-10) show that previous adversarial purification methods rely on gradient masking. Finally they propose new method RevPure that better controls the effect of randomness, and thus provides better robustness. Interestingly, authors also found that denoising process in diffusion models is orthogonal to original perturbation.
Strengths: 1) Solid theoretical analysis of the key mechanisms in diffusion-based adversarial purification methods. Illustrations and theorems are clear and interesting.
2) Easy-to-follow writing with good introduction, motivation, related work, and theoretical methodology.
3) Interesting observation that sequential denoising processes in diffusion models transform an adversarial example in the direction orthogonal to the original adversarial perturbation.
4) Interesting finding that forward process of diffusion models is unnecessary for adversarial purification, although it was observed previously in DensePure.
Weaknesses: 1) The closest works (DiffPure, DensePure) validated their experiments not only on CIFAR-10, but on large-scale datasets such as ImageNet. To make the method closer to real world, ImageNet experiments are necessary.
2) Only one architecture (WideResNet-28) is studied, while other works studied diverse set of architectures such as ViTs, ResNets, etc.
3) Effect of generation quality of the model (FID) on the robustness is only studied using two models (EMA DDPM and Non-EMA DDPM). You can't conclude anything from two points. More diverse set of models (maybe checkpoints from different epochs) should be studied.

Minor weaknesses:
1) $\chi^2$ should be properly introduced.
Recommendation: 5: marginally below the acceptance threshold
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 4 excellent
Clarity: 3 good
reviews: 
Comments: In this work, the authors revealed that diffusion models gain robustness from randomness, which causes gradient masking. They showed that randomness challenges the traditional expectation-over-time (EOT) method, leading to an overestimate of robustness. Addressing this, they proposed "randomness replay" for a more accurate robustness measure. They also found that robustness in diffusion models doesn't require a forward process; a reverse-only process is sufficient. Importantly, they demonstrated that diffusion models improve robustness by altering perturbed samples in a direction orthogonal to adversarial perturbations, thus weakening the strong attacks by removing the adversarial projections.
Strengths: (1) The experimental results support the claim that randomness in the diffusion model can cause gradient masking, which cannot be addressed EOT method. The simple example in section 4.2 provides good intuition behind the claim.

(2) The exploitation of the purification power of diffusion by removing the adversarial projection is novel and insightful for future work.
Weaknesses: (1) The study, while insightful, falls short of providing fundamental theoretical analysis. The example in section 4.2 is overly specific, leaving a gap in the broader understanding. A more general, perhaps probabilistic, theoretical exploration of how randomness leads to gradient masking and how randomness replay enhances robustness would significantly strengthen the work. Additionally, a theoretical validation of the diffusion model's ability to remove adversarial projections would be a valuable addition.

(2) The presentation of the theorems requires clearer explanations for each notation. For instance, in Theorem 1, the precise meaning of each symbol is unclear without delving into the proof. Providing meaningful explanations alongside the statement of each theorem would enhance comprehension.
Recommendation: 5: marginally below the acceptance threshold
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 3 good
Clarity: 3 good