DiffGAD: A Diffusion-based Unsupervised Graph Anomaly Detector

Authors: Jinghan Li, Yuan Gao, Jinda Lu, Junfeng Fang, Congcong Wen, Hui Lin, Xiang Wang
Primary area: learning on graphs and other geometries & topologies
Keywords: Graph Anomaly Detection, Diffusion Models

Abstract:
Graph Anomaly Detection (GAD) is crucial for identifying abnormal entities within networks, garnering significant attention across various fields. Traditional unsupervised methods, which decode encoded latent representations of unlabeled data with a reconstruction focus, often fail to capture critical discriminative content, leading to suboptimal anomaly detection.
To address these challenges, we present a Diffusion-based Graph Anomaly Detector (DiffGAD). At the heart of DiffGAD is a novel latent space learning paradigm, meticulously designed to enhance the model's proficiency by guiding it with discriminative content. This innovative approach leverages diffusion sampling to infuse the latent space with discriminative content and introduces a content-preservation mechanism that retains valuable information across different scales, significantly improving the model’s adeptness at identifying anomalies with limited time and space complexity. 
Our comprehensive evaluation of DiffGAD, conducted on six real-world and large-scale datasets with various metrics, demonstrated its exceptional performance. Our code is available at https://github.com/fortunato-all/DiffGAD

Decision: accept (poster)
Reviewer scores: 6, 8, 6, 6, 5

Reviews and discussion:
<thread>
creation date:
1730434611547

thread_type:
official_review

<thread_object>
creator: reviewer_eprr

<content>
summary:
The paper presents a graph anomaly detection method, DiffGAD, which aims to capture critical discriminative content in reconstruction. By leveraging diffusion sampling, DiffGAD infuses the latent space with discriminative content. To evaluate its effectiveness, the authors conduct experiments on seven datasets and compare its performance against other anomaly detection methods.

strengths:
1. This paper presents a clear motivation.
2. The logical structure of the paper is clear.

weaknesses:
1. The paper states that ‘the latent space constructed by the AE-based method (Ding et al., 2019) tends to represent all samples for the Books dataset (Sánchez et al., 2013) into the same point’ and 'VAE constructs the latent space within a constrained distribution (e.g., the Gaussian distribution), leading to a uniform latent distribution.' This argument should be further supported through experimental validation.
2. Authors should add the latest baselines to demonstrate the effectiveness of DiffGAD.
3. Sec. 4.3 should further analyze why DIFFGAD does not achieve optimal performance.

questions:
1. The GAE model detects anomalies by calculating reconstruction error. Wouldn’t it be more effective to focus on reconstructing common attributes rather than distinguishable features, as anomalies are typically identified by their deviations from common patterns?
2. Why add noise to the features? Can’t the original features capture the general patterns?
3. In the visualization results, normal nodes and abnormal nodes are not clearly distinguished.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730461529475

thread_type:
official_review

<thread_object>
creator: reviewer_2tpm

<content>
summary:
This paper proposes DiffGAD, an unsupervised graph anomaly detector based on diffusion models, designed to address the limited capability of capturing discriminative content in graph anomaly detection.

The main contributions include:

Pioneering the application of diffusion models in graph anomaly detection by adapting them from generative tasks, presenting DiffGAD to enhance the model's discriminative ability.
Proposing a generative paradigm guided by discriminative content to extract and refine discriminative features into the latent space, and designing a content preservation strategy to improve the reliability of the guidance process.
Demonstrating the effectiveness of DiffGAD through extensive experiments on 6 real-world datasets and 13 baseline methods.

strengths:
1、The authors innovatively adapt diffusion models from generative tasks to the field of graph anomaly detection, introducing DiffGAD, which brings new perspectives to graph anomaly detection.

2、A novel latent space learning paradigm is introduced by combining unconditional and conditional diffusion models to capture and refine discriminative content, offering a fresh approach to address the challenge of extracting discriminative features in graph anomaly detection.

3、When addressing anomaly detection, the method incorporates diffusion sampling and a content preservation mechanism, effectively injecting and retaining discriminative content across different scales.

4、In the experimental section, six different real-world and large-scale datasets are used for evaluation, covering various types of data scenarios, such as social networks (Weibo, Reddit), commercial networks (Disney, Books, Enron), and large-scale financial networks (Dgraph), ensuring the reliability and generalizability of the experimental results.

weaknesses:
1、Although the paper mentions that graph encoders (e.g., GCN in Graph AE) may limit the model's ability to represent complex graph structures and relationships, it only briefly notes this issue without analyzing how this limitation impacts DiffGAD’s performance in specific experiments or real-world scenarios. For example, when handling highly heterogeneous or dynamically changing graph structures, the encoder may fail to accurately capture critical information, potentially leading to reduced anomaly detection accuracy.

2、For the key hyperparameter λ in the model, simply showing the impact of different values on performance through experiments is insufficient. There is a lack of theoretical justification explaining why the optimal value differs across datasets and the intrinsic connection between these values and data characteristics.

3、In the introduction, the logical connection between the research motivation and the subsequent presentation of DiffGAD's innovations could be clarified. It would be helpful to explicitly highlight the specific limitations of traditional methods in capturing discriminative content and how DiffGAD’s unique design (such as the generative paradigm guided by discriminative content and content preservation strategy) directly addresses these issues.

questions:
The paper mentions that graph encoders (such as GCN in Graph AE) might limit the model’s ability to represent complex graph structures and relationships, but it does not detail how this limitation manifests in practical applications or to what extent it impacts DiffGAD’s performance. For instance, when processing graphs with highly heterogeneous node attributes or complex topologies (e.g., multi-layer nested structures, frequently dynamic graphs), would the current encoder lead to critical information loss? How does this information loss impact the accurate extraction of discriminative content and, subsequently, anomaly detection accuracy? The authors could quantify this impact by designing targeted experiments. For example, they could construct synthetic graph datasets with varying levels of heterogeneity and topological complexity to compare DiffGAD’s performance when using the current encoder versus a hypothetical ideal encoder (with stronger representational power). Metrics such as anomaly detection accuracy and recall could provide insight into performance differences.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730549144844

thread_type:
official_review

<thread_object>
creator: reviewer_wf4y

<content>
summary:
This article presents DiffGAD, a new method based on Diffusion Models (DM), for dealing with the problem of anomaly detection in graph data. A research framework based on Diffusion Models is constructed, which encodes the graph data into the latent space via an encoder, then adds noise to preserve the general content and samples from unconditional and conditional diffusion models, and finally transforms the reconstructed embedding back into the graph space via a decoder to compute the reconstruction error. Finally, through experiments on multiple real-world datasets, the authors conclude the effectiveness of DiffGAD on graph anomaly detection tasks.

strengths:
1、The method combines graph neural networks (GNNs) with diffusion models, an innovative fusion of techniques that takes advantage of both the strengths of GNNs in graph structural analysis and the sophistication of diffusion models in generative modeling.
2、 DiffGAD uses both conditional and unconditional diffusion models to reconstruct the graph, and this combination improves the model's sensitivity and ability to recognize anomalies.

weaknesses:
1、the experimental part of the dataset settings, the dataset selected in this paper are relatively small feature dimensions, have you considered the use of high-dimensional features of the dataset for the experiment?
2、Comparison methods on although there are based on deep learning, but the latest is 2022, it is recommended to increase the comparison experiments, and the current advanced methods in the field, to show the advantages of the project method.

questions:
1、How do the two DMs achieve concurrent sampling without adding extra parameters?
2. In 3.3, the minimum perturbation is used to change the potential embedding z0 to zt. How is this “minimum perturbation” defined and quantified?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730578758140

thread_type:
official_review

<thread_object>
creator: reviewer_fvc2

<content>
summary:
The paper introduces DiffGAD, a diffusion based model designed for graph anomaly detection (GAD). It employs a latent space learning paradigm and incorporates discriminative content to enhance profiency. The authors present experimental results on six large real world datasets to demonstrate model performance. Code is made available as well.

strengths:
1. The use of diffusion models for GAD is an innovative application
2. A generous number of meaningful datasets are used for experimentation
3. The figures used in the paper manage to illustrate the problem and approach well.

weaknesses:
1. A specific graph autoencoder (AE) is used by the model. This may limit the adaptability of the model. (See Q1)
2. Hyperparameters like λ affect the performance of DiffGAD. If this turns in to an "art" to get the best performance, then it may be problematic for real world scenarios. Clarification from the authors about some guidelines to selecting λ could be helpful.

questions:
1. Did the authors explore any alternatives to the currently used AE?
2. Follow-up: a less comlex alternative could give an efficiency boost at some performance cost. Would be interesting to see. (not required for acceptance)
3. Are there any particular type of datasets that are more "conducive" to DiffGAD than others?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730699915743

thread_type:
official_review

<thread_object>
creator: reviewer_psko

<content>
summary:
The paper proposes a Diffusion-based Graph Anomaly Detector for unsupervised identifying abnormal entities in networks. The contributions can be summarized as follows: 1: The author made the first attempt to transfer the diffusion models to the graph anomaly diffusion tasks. The model consists of an auto-encoder framework and two diffusion models to conduct unconditional diffusion and conditional diffusion. 2: To guide the training of diffusion models in latent space, the authors propose to use a discriminative content-guided generation paradigm to distill the discriminative content in latent space; and a content-preservation strategy to enhance the confidence of the guidance process. 3: The authors conduct experiments on 7 real-world datasets and make comparisons with 13 baseline methods.

strengths:
- The first attempt to transfer the generative diffusion models to the GAD task is a great try, bringing new perspectives to the anomaly detection community.
- The idea of distilling the discriminative content based on a linear combination of the two different diffusion models is interesting and easy to follow.

weaknesses:
- Some parts of the paper need further explanation. For example, in the introduction, the authors mention that some researchers utilize encoders to map graph data into a latent space, but there is a lack of essential discussion about why they are doing that.
- The motivation behind some of the model designs is ambiguous. For instance, in the selection of the encoder and decoder, the conditions that a good encoder and decoder should satisfy are unclear. Instead of directly utilizing the GAE framework, it seems more important to discuss the criteria for selecting a good encoder and decoder and add related experiments to support your claims. I did not see any discussions related to that.

questions:
- Why did you choose to use GAE as the encoder and decoder instead of considering other models, such as VGAE, graph transformers, etc.? Is it possible to use these models as the encoder and decoder?
- What are the conditions that a good latent space should satisfy? Or, what constitutes a good latent space for performing the diffusion process to conduct anomaly detection tasks?
- Regarding the training process, it looks like you first train a satisfactory encoder and decoder, then fix the parameters, and train the diffusion models. Why did you not choose to construct a global loss and train the GAE and diffusion models together?
</content>
</thread_object>

</thread>
<thread>
creation date:
1733313701660

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
Dear Area Chair and Reviewers,

We truly appreciate your efforts and valuable suggestions in reviewing our paper. We are glad that most reviewers reached a positive consensus on our work's motivation, presentation, novelty, and experimental effectiveness. Since we received a decent number of reviews, we provide a summary on the reviewers’ major feedback and our corresponding actions:

> **Strengths**

- **Motivation**: 
  - "The first attempt...is a great try, bringing new perspectives to the community."  (Reviewer $\color{red}{\text{pSKo}}$)
  - "This combination improves the model's sensitivity and ability to recognize anomalies." (Reviewer $\color{purple}{\text{wf4Y}}$)
  - "Offering a fresh approach to address the challenge." (Reviewer $\color{green}{\text{2tPm}}$)
  - "This paper presents a clear motivation." (Reviewer $\color{orange}{\text{epRR}}$)
- **Novelty**: 
  - "The idea is interesting and easy to follow." (Reviewer $\color{red}{\text{pSKo}}$)
  - "The use of diffusion models for GAD is an innovative application." (Reviewer $\color{blue}{\text{fVC2}}$)
  - "An innovative fusion of techniques." (Reviewer $\color{purple}{\text{wf4Y}}$)
  - "Innovatively adapt diffusion models from generative tasks to the field of graph anomaly detection." (Reviewer $\color{green}{\text{2tPm}}$)
- **Presentation**:
  - "The figures used in the paper manage to illustrate the problem and approach well." (Reviewer $\color{blue}{\text{fVC2}}$)
  - "The logical structure of the paper is clear." (Reviewer $\color{orange}{\text{epRR}}$)
- **Promising results**:
  - "Ensuring the reliability and generalizability of the experimental results." (Reviewer $\color{green}{\text{2tPm}}$)

> **Responses**

- **Reviewer  $\color{red}{\text{pSKo}}$**: 

  The concerns are focused on latent space construction, we address this from the following 2 aspects: 

  - Further explanation: We explain why some researchers utilize encoders to map graph data into a latent space (Introduction). We also explain why we not choose to construct a global loss and train the GAE and diffusion models together from the perspective of theoretical and empirical analysis.
  - More experimental exploration: We explore the impact brought from the different encoder & decoder in our method and discuss the criteria for selecting a good encoder & decoder/latent space. (Appendix C.5)

- **Reviewer $\color{blue}{\text{fVC2}}$**: 

  - Adaptability of the model: We try 4 alternative AE architectures and explore their applicable scenarios. (Appendix C.5)
  - Guidelines for selecting hyper-parameter $\lambda$: We explore the relationship between $\lambda$ and the heterophily of anomalies to offer better guidance. (Appendix C.6)

- **Reviewer $\color{purple}{\text{wf4Y}}$**:

  - Further illustration: We explain the sampling process and the perturbation and we also modify the expression. (Section 3.1 & 3.3)
  - More experimental evaluation: We compare DiffGAD with 2 latest methods.

- **Reviewer $\color{green}{\text{2tPm}}$**:

  - Explore the limitation: We conduct more evaluations on heterogeneous graphs.
  - Clarify the motivation: We modify the logic in the Introduction according to your suggestion.
  - Guidelines for selecting hyper-parameter $\lambda$: We explore the relationship between $\lambda$ and the heterophily of anomalies. (Appendix C.6)

- **Reviewer $\color{orange}{\text{epRR}}$**:

  The concerns are focused on our experiments, we address this from the following 3 aspects: 

  - More experimental validation: We further explain the visualization results and improve the visualization expression (Appendix B). We also further explain why add noise to the features.
  - More experimental evaluation: We compare DiffGAD with 2 latest methods.
  - More experimental analysis: We further analyze the performance of DiffGAD according to your suggestion. (Appendix C.1)

We are pleased that most of the concerns have been clarified, and reviewers expressed satisfaction with our paper and acknowledged our efforts. Reviewer  $\color{red}{\text{pSKo}}$ and $\color{orange}{\text{epRR}}$ consider that we have largely addressed their concerns and **raise their scores from 5 to 6 and from 3 to 5, respectively**! Reviewer  $\color{blue}{\text{fVC2}}$ and $\color{green}{\text{2tPm}}$ **appreciate our efforts  for the detailed and thorough response, and keep supporting us**. Reviewer $\color{purple}{\text{wf4Y}}$ acknowledge our approach and keep supporting us.

Once again, we deeply appreciate the time and expertise you have shared with us. Your encouraging feedback motivates us to continue advancing this work for the broader community, and we are more than happy to add clarifications to address any additional recommendations and reviews from you！

Best regards,

Authors of Paper 6534
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes a diffusion-based graph anomaly detector (DiffGAD) to identify abnormal entities in networks.  DiffGAD encodes graph data into a latent space via an encoder, adds noise and samples using unconditional and conditional diffusion models, and transformers embeddings back to the graph space via a decoder.  The authors demonstrate DiffGAD's effectiveness compared to baseline deep anomaly detectors on a variety of datasets.

Reviewers leaned positively towards this work, and appreciated the adaptation of diffusion models to the GAD space and the breadth of experiments authors conducted.  There were a few points of concern reviewers raised:

- Clarification around conditions of what a good latent space is for anomaly detection and how this influences the autoencoder choice (pSKo, fVC2, 2tPm)

- Some newer baselines were missed in the initial draft; reviewers addressed some of these concerns, but folding them in and presenting the comparisons with the most recent work is important (wf4Y, epRR)

- Concerns around hyperparameter (\lambda) sensitivity (2tPm, fVC2)

I encourage the authors to fold these into the revision.