Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an anomaly detection (AD) method based on the diffusion model. Training a diffusion model on unlabeled data (mostly assumed to be normal), the reconstruction error is proposed as an anomaly score (similar to using autoencoders for AD). In addition, using the diffusion model to "diffuse" an input (i.e. adding noise by partially applying the concatenated diffusion model maps) and afterwards reconstructing it, the model is also used for mitigating adversarial perturbations. An experimental evaluation on MVTec-AD is presented showing that the proposed method achieves a similar anomaly detection performance as existing state-of-the-art methods while demonstrating an improved adversarial robustness.
strength_and_weaknesses: *Strengths*
+ The paper presents a coherent framework for adversarial attacks on anomaly detectors (targeting both anomalies to appear normal and normal data points to appear anomalous) and evaluates several existing methods w.r.t. their adversarial robustness, finding that many existing methods suffer from not being adversarially robust.
+ The experiments show that the proposed method achieves strong adversarial robustness while maintaining a comparable state-of-the-art detection performance on MVTec-AD.

*Weaknesses*
- I find the main weakness of the paper to lie in the experimental evaluation, which only considers *one* dataset (MVTec-AD). This strongly limits evidence for some of the (fairly general) made claims in the paper. Defect detection on MVTec-AD, where anomalies are rather subtle and low-level, is quite distinct from other visual anomaly detection tasks (e.g. detecting novel classes).
This is (implicitly) acknowledged in the paper: "Note that $k$ should be chosen such that the amount of Gaussian noise is dominating the adversarial perturbations and anomaly signals while the high-level features of the input data are still preserved for reconstruction."
Yet, there is no broader discussion on this point, which I think would be critical.
clarity,_quality,_novelty_and_reproducibility: + The paper is written well and easy-to-follow.
+ Overall, the paper is placed well into the existing literature.
+ There are some other first works on using diffusion models for AD, but this class of models has not yet been explored much.
+ The paper seems to present all relevant details for reproducibility in the main paper + appendix

---

*Additional Comments*
* Why do Table 5 and Table 6 only consider 4 classes (Bottle, Grid, Toothbrush, Wood)?
* p.1: "Recently, deep learning (DL) based anomaly detection methods have achieved remarkable improvement over traditional anomaly detection strategies." You might reference recent reviews here for further reading, e.g. Ruff et al. (2021) or Pang et al. (2021).

Ruff, Lukas, et al. "A unifying review of deep and shallow anomaly detection." Proceedings of the IEEE 109.5 (2021): 756-795.

Pang, Guansong, et al. "Deep learning for anomaly detection: A review." ACM Computing Surveys (CSUR) 54.2 (2021): 1-38.
summary_of_the_review: I enjoyed reading this paper and think the presented contributions w.r.t. adversarially robust anomaly detection are interesting and relevant, but the paper is strongly limited in only evaluating on a single dataset which is rather specific. Claims such as in the Conclusion ("We empirically show that our method provides outstanding adversarial robustness while also maintaining strong anomaly detection performance on benchmark datasets.") are simply too far-fetched in my opinion for the provided evidence, which should be corrected for (either by adding more datasets or limiting claims to MVTec-AD).
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper focuses on anomaly detection models and proposes an adversarially robust anomaly detector based on the diffusion model. In detail, they first add gaussian to the image through the forward process, and then reconstruct it through the reverse process;  the Multiscale Reconstruction Error Map is computed as the anomaly score. In the experimental part, they compare the proposed method with SOTAs to show the advantage and combine with randomized smoothing to reach some certified robustness. 

strength_and_weaknesses: Strength: 
This paper is well-written and easy to follow. The proposed method takes advantage of the smoothing property of the diffusion model and reaches robust anomaly detection. Experiments on widely-used anomaly detection benchmarks are conducted, and the performance is compared with SOTAs to better illustrate the performance of the proposed method. 

Weakness: 
- Some experimental results are not very convincing, especially in table 1, where most of the SOTAs fail under PGD attack, but not sure if this is due to the lack of parameter tuning. 
- The idea of applying the diffusion model is not very novel and is similar to the idea in the paper Diffusion Models for Adversarial Purification which also uses the diffusion model to remove adversarial perturbations. 
- The comparison with defense-enabled anomaly detectors is not very convincing, and the comparison with more methods should be added. 
- This paper does not involve other robust anomaly detection methods in the experiment.
- The experiment of defending against the adaptive attack needs more clarification. 

clarity,_quality,_novelty_and_reproducibility: The illustration of methodology is clear, but the explanation of experimental results is not so clear. The reason why certain defense-enabled anomaly detectors are chosen and why compare the performance against adaptive attack in such a way need a better explanation. The overall quality is relatively good. The methodology is reasonable and the experiments are well-designed to show the advantages. However, the novelty is not obvious, because there already exist methods leveraging the diffusion model to remove the perturbations which is also the key to calculating the anomaly reconstruction score in this paper. The authors provide detailed algorithms and experimental settings which may help reproduce.

summary_of_the_review: This paper leverages the diffusion model to create a robust anomaly detector and provides experimental results to support their claim. It is well-written and the illustration of the methodology is clear. The novelty is not obvious, and the experimental results need a better explanation otherwise it will confuse the reader.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper tries to construct an adversarially robust anomaly detector (note that this is different from constructing an classifier because we only need to do a binary classification of anomaly or not) directly using the diffusion models. The idea is to directly use the reconstruction score in diffusion model to indicate whether this is an anomaly or not -- and since reconstruction is resilient to adversarial perturbations, we don't need extra work
strength_and_weaknesses: 1. The use of diffusion models directly as an anomaly detector (and achieves adversarial robustness) seems novel
2. The paper is clearly written, and easy to follow
3. There seems to have systematic experiments to validate the claim

On the other hand, my main objection at this point is that it seems to be a direct application of diffusion models without any modification (so it is like -- ``surprise, diffusion models also have these nice properties!''). For example, Algorithm 1 seems to be just the reverse process in diffusion models (inlining the sampling algorithm?)


clarity,_quality,_novelty_and_reproducibility: The paper is clearly written and easy to follow
summary_of_the_review: This paper seems a solid work, but at this point it reads to me more like ``nice properties of diffusion models -- it can be directly applied as an anomaly detector that also achieves robustness''. If so, please clarify this in the paper. If not -- please clearly discuss what are the revisions to the existing diffusion models.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper develops an adversarially robust anomaly detection through the diffusion model, called FreeRAD. This paper makes an interesting finding that leveraging the diffusion model for anomaly detection can achieve adversarial robustness for free. Experimental results show that the model can still achieve good performance on adversarial samples assembled by PGD and BPDA.
strength_and_weaknesses: Strength

- The paper has an interesting finding.
- The proposed implementation trick can achieve real-time anomaly detection.

Weakness
- On the clean data, FreeRAD cannot beat the baseline CFA.
-  I think more attacking approaches can be incorporated for evaluation. 
- Currently, a few diffusion model-based anomaly detection approaches have been developed. I am curious whether the existing diffusion model-based anomaly detection models are also robust to adversarial attacks.  By further examining the following approaches, we may know whether the adversarial robustness is from the diffusion model or the implementation ideas developed in this paper.

   - Wyatt, Julian, et al. "AnoDDPM: Anomaly Detection With Denoising Diffusion Probabilistic Models Using Simplex Noise." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2022.
   - Wolleb, J., Bieder, F., Sandkühler, R., Cattin, P.C. (2022). Diffusion Models for Medical Anomaly Detection. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. 
clarity,_quality,_novelty_and_reproducibility: This paper is well-written and shows an interesting finding. 
summary_of_the_review: I like the interesting finding in this paper but believe more experimental results could make this paper stronger.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper aims to obtain the adversarial robustness for diffusion model. The presentation is clear, and the idea seems interesting. The reviewers find the experimental evaluation is weak, and the novelty is very limited. It is better to improve the paper based on the comments of the reviewers.
Decision: Reject