Paper: D3AD: DYNAMIC DENOISING DIFFUSION PROBABILISTIC MODEL FOR ANOMALY DETECTION (ICLR-2024)

Reviewer comment (Novelty):
The paper overclaims about the domain adaptation technique. DDAD [1] also uses a similar objective to fine-tune the feature extractor. I would rather appreciate that the authors lower the tone.

Author response:
Our domain adaptation approach differs from DDAD[1]. Unlike DDAD, which minimizes a cosine distance between the reconstruction and a randomly sampled training example, our objective function minimizes a cosine distance between the extracted features of an input images and its reconstructions. This not only learns intricacies of the target domain but also mitigates artifacts arising from using a pre-trained VAE for encoding and decoding images into latent.

Reviewer comment (Theory):
The authors formulate reference $z_{\hat{T}}$ as a multiplication of the latent without any noise addition. However, I do not find any support for why such guidance on the latent should work. The authors may elaborate on why this helps in detail via further analysis.

Author response:
We've conducted comprehensive analytical experiments, now detailed in the appendix (Fig 14), to empirically support our claim. Notably, our model applies noising and denoising to latents, which are inherently robust compared to pure images. However, surprisingly, even with no noise and just scaling, latents are perturbed sufficiently for faithful reconstruction, underscoring their resilience, see appendix figures 7 and 8.

Reviewer comment (Writing):
I do not understand how the binary search function $\psi$ can choose the right time on the bin. Further details should be provided on how this works or at least introduce related concepts.

Author response:
Thank you for pointing out these concerns. We've revised the explanation of binary search function and bin construction in section 4.1 to provide clarity. The use of binary search denotes mapping the average distance of a datapoint to its k-neighbours into the fitting bin. For instance, an average distance of 4.5 would be placed in bin 1 which is defined to cover the range (1,5].

Reviewer comment (Presentation):
I am also confused about how the method constitutes a "predefined histogram" in Figure 3.

Author response:
We have updated the wording from ‘predefined histogram’ to ‘predefined bins’.

Reviewer comment (Experiments):
The authors claim their methods show **state-of-the-art** performance but I doubt this is true. For example, [2] scores I-AUC of 97.6, P-AUC of 98.4, and PRO of 94.9 while this method scores 96.0/97.9/94.1. I don't feel that this method outperforms DDAD even since DDAD scores 99.3 I-AUC in the VisA dataset, which is somehow not shown in Table 1. The authors should compare all the baselines in previous conferences (e.g. ICML, CVPR, ICCV) to support this bold claim.

Author response:
We appreciate your observation. Our comparison in Table 1 includes all published methods available up to the submission deadline. [2] was published after this cutoff, and as such, not featured, therefore we maintain SOTA performance. Our segmentation performance, Table 2, is intentionally confined to relevant diffusion-based methods to offer additional insight showing SOTA segmentation performance. We have now augmented this table with I-AUROC.

Reviewer comment (Presentation):
I also feel like three metrics least (I-AUROC, P-AUROC, p_PRO) should be compared as a whole in the major table (e.g. Table 1).

Author response:
As detailed in our methods section, we chose to emphasize the PRO metric in Table 1. This decision is based on PRO's ability to treat anomalies across scales equally, showcasing our approach's efficacy in handling a variety of anomalies, figures 1, 7 and 10. We have further augmented the appendix with Pixel AUROC in table 7 as requested.

Reviewer comment (Experiments):
I am curious why the authors dropped the MvTec dataset for comparison since most anomaly detection algorithms are compared in the dataset. It is hard to assert that the method shows state-of-the-art performance without comparing the MvTecAD dataset in my opinion.

Author response:
We addressed this in the general response, here a short summary: We focused on the VisA[3] dataset as it is more challenging however, we now performed a preliminary study on MVTec[4] which is detailed in the appendix. Comparison to other reconstruction-based methods is provided in the general response. We are pleased to report that our method demonstrates comparable performance out-of-the-box.

Reviewer comment (Novelty):
Even addressing this issue, I am confused about the gain of this paper since the proposed model uses a latent diffusion model, unlike the other diffusion-based methods.

Author response:
The utilization of latent diffusion models for anomaly detection is a novel and advantageous aspect of our approach. This choice is motivated by computational efficiency compared to standard diffusion models. Latents, being a more robust representation than raw images, play a pivotal role in enabling the reconstruction of large missing components, figure 7, while mitigating background artifacts which is a common challenge in reconstruction-based models. Furthermore, our introduction of a dynamic step mechanism is the primary contribution, addressing a limitation of standard diffusion models.

Reviewer comment (Theory):
The paper provides a well-structured framework, but the explanation of each component lacks clarity and precision. In addition, the paper lacks verification of the validity of some arguments.

Author response:
$\textbf{W1:}$ We respectfully disagree as we adhere to standard convention in notation from well-established diffusion model literature. We are unable to identify specific arguments which lack validity. Could you please specify these points for further clarification?

Reviewer comment (Theory):
The description of some methods used in the paper is obscure. For example, the paper does not provide a clear and rigorous mathematical explanation of the proposed DIC method, but only a short description of it.

Author response:
$\textbf{W2:}$ To address this concern, could you please specify which part of our DIC method explanation you find obscure? We are open to understanding your expectations for further rigor.

Reviewer comment (Evaluation):
Some of experimental results in the paper lack detailed discussions. For example, the paper lacks careful analysis of qualitative results and does not link experimental results to the proposed methods logically.

Author response:
$\textbf{W3:}$ Due to space constraints, an in-depth discussion had not been included, however, in light of your feedback, we've added more qualitative figures 7 and 8 in the appendix material as well as a discussion on linking results to the various components of the framework.

Reviewer comment (Writing):
The paper is imprecise and unpolished. There are a number of grammatical errors in the text, so careful checking and revision is recommended.

Author response:
$\textbf{W4:}$ Once again, we're unable to pinpoint specific shortcomings in the writing. We would greatly appreciate your guidance in identifying precise areas of concern to address and improve.

Reviewer comment (Theory):
How did you determine an noiseless and only scaled input $x_{\hat T}$ is optimal for faithful reconstruction?

Author response:
$\textbf{Q1:}$ We've conducted comprehensive analytical experiments, now detailed in the appendix (Fig 14), to empirically support our claim. Notably, our model applies noising and denoising to latents, which are inherently robust compared to pure images. However, surprisingly, even with no noise and just scaling, latents are perturbed sufficiently for faithful reconstruction, underscoring their resilience, see appendix figures 7 and 8.

Reviewer comment (Experiments):
Why did you chose |B|=10?

Author response:
$\textbf{Q2:}$ The choice was made empirically, and an ablation will be added in camera ready.

Reviewer comment (Writing):
What is meaning of “unnoised”? It would be better to replace “unnoised” with “noiseless”.

Author response:
$\textbf{Q3:}$ Thank you for the suggestion, we have reworded to ‘noiseless’.

Reviewer comment (Evaluation):
Could you please elaborate the qualitative effect of DIC based on Figure 6?

Author response:
$\textbf{Q4:}$ Figure 6 demonstrates that a dynamically computed time step (DIC Mask) provides the most similar anomaly prediction mask to the ground truth (GT) in comparison to masks from fixed noising time steps as shown in remaining sequence -- from 100\% to 25\% of $T=80$. We have updated this description in main paper.

Reviewer comment (Experiments):
How did you localize and detect anomalies of varying scales? The main body lacks a detailed description of the solution to this problem and the corresponding experimental analyses.

Author response:
$\textbf{Q5:}$ As described in the methods section and illustrated in figure 2, we first estimate the anomaly severity by initially using a KNN to compare a test image to the training data. This severity is translated into the scaling level needed for perturbing the latent of the test image. The diffusion model then reconstructs the scaled (and noiseless) latent which is decoded into an image reconstruction. A comparison of the input and reconstruction results in localisation of anomalies of varying severity.

Reviewer comment (Experiments):
D3AD has not been validated on the most widely used dataset, MVTec[1], and I'm curious about its performance on this dataset.

Author response:
$\textbf{W1:}$ We addressed this in the general response, here a short summary: We focused on the VisA[1] dataset as it is more challenging however, we now performed a preliminary study on MVTec[2] which is detailed in the appendix. Comparison to other reconstruction-based methods is provided in the general response. We are pleased to report that our method demonstrates comparable performance out-of-the-box.

Reviewer comment (Evaluation):
The impact of the dynamic noise amount on the reconstruction results is not clear from this paper, and it appears that the reconstruction quality is not very good based on Figure 3.

Author response:
$\textbf{W2:}$ We addressed this in general response, here a short summary: We've updated figure 3 and introduced two new figures 7 and 8 in the appendix which clearly showcases compelling reconstructions achieved with our Dynamic Implicit Conditioning (DIC) mechanism. Figure 8 present a qualitative comparison of reconstruction against current SOTA in reconstruction-based method DRAEM [3] and shows significant superiority. These enhancements emphasize the robustness and effectiveness of DIC in diverse scenarios.

Reviewer comment (Novelty):
The novelty of this work is very limited. The proposed conditioned denoising and domain adaptation are similar to DDAD[2]. Besides, the anomaly scoring paradigm is similar to RD4AD[3]. As a result, the framework of D3AD looks like a combination of existing works.

Author response:
$\textbf{W3:}$ While we leverage DDAD's [4] conditioning mechanism, we don't claim novelty in this regard. Our contribution lies in extending this mechanism by introducing dynamic time step computation based on an early estimate of an anomaly’s severity using k-NN. Additionally, our domain adaptation approach differs as well. Unlike DDAD, which minimizes a cosine distance between the reconstruction and a randomly sampled training example, our objective function minimizes a cosine distance between the extracted features of an input images and its reconstructions. 

Furthermore, our method diverges significantly from RD4AD[5] in anomaly scoring. RD4AD employs a teacher encoder and a student decoder and measures the distance between the representations from these two blocks. Our feature distance $f_{map}$, in contrast, computes a distance between the feature representations (extracted from an encoder) of an input image and its reconstruction, see figure 4. Additionally, our approach deviates from the conventional teacher-student model by introducing a second distance measure $l_{map}$. This measures the disparity between the latent z and its reconstruction z_hat generated by the diffusion model.

Reviewer comment (Theory):
Is there more detailed experimentation or theoretical basis to explain why noise is not needed during inference?

Author response:
$\textbf{A2:}$ Thank you for highlighting this ambiguity. We have taken your suggestions into account and composed a new Figure 15 which presents noising vs scaling paradigm on different perturbance levels. This clarifies the impact of scaling vs noising as seen in reconstructions and consequently anomaly maps. Note the capsules category row 2, T = 320, noising results in higher degradation of the image resulting in a reconstruction which hallucinates an additional green capsule (red circle), whereas an only scaled image retains more of the input image while perturbing pixels enough for a faithful reconstruction in comparison.

Reviewer comment (Experiments):
Are there more visual results showcasing the reconstruction of different types of anomaly regions?

Author response:
$\textbf{W2:}$ We addressed this in general response, here a short summary: We've updated figure 3 and introduced two new figures 7 and 8 in the appendix which clearly showcases compelling reconstructions achieved with our Dynamic Implicit Conditioning (DIC) mechanism. Figure 8 present a qualitative comparison of reconstruction against current SOTA in reconstruction-based method DRAEM [3] and shows significant superiority. These enhancements emphasize the robustness and effectiveness of DIC in diverse scenarios.

Reviewer comment (Experiments):
How does D3AD perform on MVTec, which is one of the most widely used datasets?

Author response:
$\textbf{W1:}$ We addressed this in the general response, here a short summary: We focused on the VisA[1] dataset as it is more challenging however, we now performed a preliminary study on MVTec[2] which is detailed in the appendix. Comparison to other reconstruction-based methods is provided in the general response. We are pleased to report that our method demonstrates comparable performance out-of-the-box.