Paper: Consistency Training with Learnable Data Augmentation for Graph Anomaly Detection with Limited Supervision (ICLR-2024)

Reviewer comment (Theory):
as you said, GAD has low homophily distribution, whether the homophily-aware neighbourhood aggregation you propose will be effective for abnormal nodes, I deeply doubt it.

Author response:
**2. As you said, GAD has low homophily distribution, whether the homophily-aware neighbourhood aggregation you propose will be effective for abnormal nodes, I deeply doubt it.**


We sincerely apologize for any confusion that may have arisen, and we appreciate the opportunity to clarify this aspect of our work. In our study, we use the term "homophily" as a conceptual label, and it is important to note that "heterophily" could be interchangeably used in this context. Our model's primary goal with the homophily-aware neighborhood aggregation is to differentiate between the distinct distributions of homophily (or equivalently, heterophily) that are observed in the edges surrounding normal and anomalous nodes. Specifically, edges in the neighborhood of normal nodes typically exhibit high homophily (or low heterophily), whereas those around anomalous nodes tend to show low homophily (or high heterophily). The cross-entropy loss function in our model facilitates inverse predictions for normal and anomalous nodes. This means that the loss function strategically guides the edge representations surrounding a normal node to diverge from those around an anomalous node, aligning with their distinct homophily (or heterophily) distributions. Consequently, edges with high homophily around normal nodes share similar representations, while those with low homophily around anomalous nodes form analogous representations. This approach is critical for enhancing the discriminatory power of our model between normal and anomalous nodes, aligning with the core objectives of our research.

Reviewer comment (Writing):
I don't find the definition of Vhq in Section 2, whiat is the meaning of Vun?

Author response:
**3. I don't find the definition of Vhq in Section 2, what is the meaning of Vun?**

Thank you for highlighting the need for clarity regarding the definition of $V_{hq}$ and $V_{un}$ in our paper. We appreciate the opportunity to elucidate this aspect of our work. $V_{un}$ is defined as the set of unlabeled nodes.
In the paragraph preceding Eq.(3), $V_{hq}$ represents the set of high-quality nodes, defined as $V_{hq}=\\{v \mid v\in V_{un} \land \exists \mathbf{p}_v[k]\ge\tau\\}$. This set comprises unlabeled nodes for which there exists at least one predicted score in their predictions greater than the threshold $\tau$. We have clarified this definition in our paper.

Reviewer comment (Experiments):
please compare your learnable augmentation with other augmentation methods

Author response:
**4. Please compare your learnable augmentation with other augmentation methods.**

Thank you for your insightful suggestion.
In Appendix E.4, we benchmark our learnable data augmentation against several stochastic augmentation methods. Specifically, we substitute our augmentation module with widely-recognized techniques in the consistency training framework, including *DropNode* (Feng et al., 2020a), *DropEdge* (Rong et al., 2020), and *Dropout* (Srivastava et al., 2014). We administer Dropout to both input features, denoted as *DropInput*, and intermediate features, referred to as *DropHidden*. It is noteworthy that DropHidden serves as a non-learnable analogue to our augmentation module. For a fair comparison, we determine the optimal drop rate for each method within the interval of $(0, 0.5]$, with a step size of $0.1$. We list the optimal drop rate in Table 6. For your ease of reference, we also reported these optimal rates in the table below.

| Methods| Amazon | YelpChi | T-Finance | T-Social |
| --- | --- | --- | --- | --- |
| ConsisGAD (DropInput) | 0.1 | 0.1 | 0.1 | 0.4 |
| ConsisGAD (DropNode) | 0.2 | 0.1 | 0.1 | 0.5 |
| ConsisGAD (DropEdge) | 0.2 | 0.1 | 0.1 | 0.3 |
| ConsisGAD (DropHidden) | 0.3 | 0.1 | 0.2 | 0.1 |

The outcomes, depicted in Figure 9 through the three metrics, underscore the preeminence of our proposed learnable data augmentation over traditional stochastic alternatives.
A salient observation is that among conventional augmentation methods, employing DropHidden in consistency training usually yields more substantial and stable performance improvements across all datasets. This empirical insight substantiates our decision to integrate learnable data augmentation at intermediate states and suggests the potential exploration of applying such augmentation to graph topology (Xia et al., 2022). For your easy reference, we also summarize the numerical results in the following table.
| Dataset    | Metric   | ConsisGAD | ConsisGAD (DropInput) | ConsisGAD (DropNode) | ConsisGAD (DropEdge) | ConsisGAD (DropHidden) |
| --- | --- | --- | --- | --- | --- | --- |
| Amazon     | AUROC    | **93.91±0.58** | 89.16±0.53 | 91.47±0.73 | 92.63±0.45 | 92.71±0.38 |
|            | AUPRC    | **83.33±0.34** | 74.01±1.30 | 77.84±1.03 | 80.22±0.89 | 81.03±1.00 |
|            | Macro F1 | **90.03±0.53** | 86.22±0.93 | 85.58±0.68 | 88.03±0.81 | 87.78±0.91 |
| YelpChi    | AUROC    | **83.36±0.53** | 81.68±0.21 | 79.85±0.26 | 80.70±0.47 | 80.27±0.39 |
|            | AUPRC    | **47.33±0.58** | 42.14±0.52 | 40.14±0.67 | 43.54±0.27 | 43.50±0.21 |
|            | Macro F1 | **69.72±0.30** | 68.22±0.37 | 66.31±0.47 | 67.34±0.29 | 67.32±0.44 |
| T-Finance  | AUROC    | **95.33±0.30** | 95.17±0.23 | 95.07±0.40 | 95.13±0.49 | 94.58±0.45 |
|            | AUPRC    | **86.63±0.44** | 84.87±0.61 | 84.78±0.31 | 85.43±0.47 | 85.59±0.43 |
|            | Macro F1 | **90.97±0.63** | 90.16±0.33 | 90.02±0.66 | 90.16±1.02 | 90.31±1.05 |
| T-Social   | AUROC    | **94.31±0.20** | 94.23±0.16 | 94.04±0.34 | 94.10±0.25 | 94.13±0.32 |
|            | AUPRC    | **58.38±2.10** | 55.32±2.00 | 54.00±2.11 | 55.33±2.39 | 53.58±2.41 |
|            | Macro F1 | **78.08±0.54** | 76.86±0.98 | 76.35±0.71 | 77.75±0.57 | 77.48±1.03 |

Reviewer comment (Theory):
please give complexity analysis

Author response:
**5. Please give complexity analysis.**

As illustrated in Algorithm 1, our model is composed of two primary components: consistency training and the training of the learnable data augmentation module. These components operate iteratively in each iteration of the process. This approach potentially incurs a higher computational cost compared to the standard training procedures of graph neural networks. In this part, we give a detailed complexity analysis and analyze the possibility of using it on large graphs.

At the outset, we focus on analyzing the complexity of our backbone GNN model, as outlined in Equation (4). This model is a critical component in Algorithm 1 for calculating node embeddings. Considering a target node $v$, in the first GNN layer, the function $\delta(\cdot,\cdot;\theta_{\delta})$ is implemented as $\text{MLP}(\mathbf{h}^{l-1}_v||\mathbf{h}^{l-1}_u)$. A typical example of this MLP is $\sigma(\mathbf{W}(\mathbf{h}^{l-1}_v \|\| \mathbf{h}^{l-1}_u)+\mathbf{b})$. The computational complexity in this first layer is $O(2dd_X+\bar{N}d+2d)$, where $d_X$ represents the dimension of the input feature vector, $d$ the intermediate dimension of the embeddings, and $\bar{N}$ the average node degree. Each subsequent layer contributes an additional complexity of $O(2d^2+\bar{N}d+2d)$. Given $L$ total GNN layers, the overall complexity of the GNN backbone sums up to $O(2dd_X+\bar{N}d+2d+(L-1)(2d^2+\bar{N}d+2d))=O(2dd_X+2Ld^2+2Ld+L\bar{N}d-2d^2)$.
In every iteration of Algorithm 1, we sample a batch of labeled and unlabeled nodes for subsequent computations. These batches are of sizes $B$ (for labeled nodes) and $\mu B$ (for unlabeled nodes), respectively. We can split the whole process into three step as follows.
- We begin by selecting high-quality nodes from the sampled batch of unlabeled nodes. In line 6, the GNN backbone introduces a complexity of $O(2dd_X+2Ld^2+2Ld+L\bar{N}d-2d^2)$, as previously analyzed. The prediction step in line 7 has a complexity of $O(Kd+2K)$, where $K$ is the number of classes. Lines 8-10 involve checking the high-quality nodes with a complexity of $K$. Overall, for a batch size of $B$, lines 5-10 collectively result in a complexity of $O(\mu B(2dd_X+2Ld^2+2Ld+L\bar{N}d-2d^2+Kd+3K))$.
- Each high-quality node is augmented and predicted, forming the basis for the consistency and diversity loss calculations. For line 12, the complexity of the Sharpen function in Algorithm 2 involves $O(8\lfloor \xi d \rfloor d+d)$ across its steps. Consequently, line 12 in Algorithm 1 incurs a complexity of $O(8\lfloor \xi d \rfloor d+d^2+3d)$. Line 13, similar to line 7, involves a complexity of $O(Kd+2K)$. Line 14 includes forming both the consistency and diversity losses, with complexities of $O(\mu BK^2)$ and $O(2\mu Bd)$, respectively. Summing up, lines 11-14 entail a complexity of $O(\mu B(8\lfloor \xi d \rfloor d+d^2+3d + Kd+2K) + \mu BK^2 + 2\mu Bd)$.
- The consistency training involves complexity calculations similar to the earlier sections. Lines 17 and 18, as previously analyzed, involve a complexity of $O(2dd_X+2Ld^2+2Ld+L\bar{N}d-2d^2+Kd+2K)$. Lines 19 and 20, pertaining to the loss calculations, have complexities of $O(BK^2)$ and $O(\mu BK^2)$, respectively. Thus, the total complexity for lines 16-20 over $B$ iterations is $O(B(2dd_X+2Ld^2+2Ld+L\bar{N}d-2d^2+Kd+2K)+BK^2+\mu BK^2)$.

In each iteration of Algorithm 1, we aggregate the complexities from the previous sections to derive the overall complexity. This can be expressed as $O(2B(\mu+1)dd_X + B(2\mu L+2L-\mu-2)d^2 + (2\mu BL+\mu BL\bar{N}+\mu BK+8\mu B\lfloor \xi d \rfloor+5\mu B+K\mu B+2BL+BL\bar{N}+BK)d + 5\mu BK+2\mu BK^2+2BK+BK^2)$.

Particularly in our anomaly detection scenario, where typically $K = 2$ due to the binary classification nature, and $L$ generally ranges from 1 to 3 as the number of GNN layers, the complexity of our model is predominantly influenced by the feature dimension $d_X$, the hidden dimension $d$, and the average node degree $\bar{N}$. This indicates that with appropriately set hyper-parameters, our model shows promising potential for application on large-scale graphs. To enhance the clarity of our paper, we have included this detailed complexity analysis in Appendix A of our revised manuscript.


We hope that these clarifications and additional details address your concerns and strengthen the paper's contributions. Your insightful feedback has been instrumental in refining the quality and coherence of our manuscript. Should there be any more comments or issues from your side, we remain open and willing to make the requisite modifications.

Reviewer comment (Novelty):
The paper should acknowledge related works that are pertinent to the proposed learnable data augmentation, such as [a] and [b]. It is crucial to cite and discuss the distinctions between these works and the proposed approach, providing readers with a clear understanding of the novel contributions made by this study.

Author response:
**1. The paper should acknowledge related works that are pertinent to the proposed learnable data augmentation, such as [a] and [b]. It is crucial to cite and discuss the distinctions between these works and the proposed approach, providing readers with a clear understanding of the novel contributions made by this study.**

We acknowledge the importance of referencing and discussing relevant related works. In Appendix F of our revised manuscript, we have included a discussion comparing our work with the studies of [a] and [b]. For your convenience, we summarize the distinctions between these two works and our proposed learnable data augmentation:

- **JOint Augmentation Optimization (JOAO) [a]**: This work learns a sampling distribution to sample augmentation pairs for graph contrastive learning. It instantiates the learning process as a min-max optimization process, iteratively optimizing the GNN encoder and the learnable sampling distribution. Our work is different from JOAO in two aspects:
  - **Evaluation of augmentation quality**: JOAO is based on the adversarial training framework and assumes that a good augmentation will cause a high contrastive learning loss. Intuitively, increasing contrastive loss is to increase the distribution diversity of augmentations. However, the label consistency is overlooked, which may cause excessive noise injected into augmentations and lead to suboptimal performance, as indicated in prior studies [c,d]. In contrast, our work assumes that a good augmentation will increase the diversity of a node without changing its label (i.e., high label consistency). To this end, we formulate differentiable distribution diversity and label consistency metrics to evaluate augmentation quality and to guide the training of our augmentation module.
  - **Learning objectives**: The learning objective of JOAO is to pick the optimal augmentation pair from a pre-defined pool of augmentation types for graph contrastive learning. It requires domain knowledge to manually construct and configure the augmentation pool for selection. By contrast, our work introduces a novel learnable augmentation module through learnable masking. Our module learns to synthesize node-specific augmentations, evolving dynamically with the dataset and GNN encoder. 

- **Graph Contrastive learning with Adaptive augmentation (GCA) [b]**: GCA introduces a general graph contrastive learning framework and generates augmented graphs by adaptively dropping edges and node features. The drop rates of edges and node features are based on node centrality scores, incorporating topological and semantic priors in a graph. The node centrality scores include degree centrality, eigenvector centrality, and PageRank centrality. Our work is different from GCA in two aspects:
  - **Evaluation of augmentation quality**: GCA assumes that a good augmentation should drop unimportant edges and features while keeping important ones. The importance is measured through node centrality scores. In this way, edges whose ending nodes have low centrality scores are more likely to be dropped, and feature dimensions appearing less frequently on important nodes are more likely to be masked. However, this heuristic approach may not guarantee high consistency and diversity in augmentations. On the other hand, our proposed model does not rely on node importance metrics but focuses on maximizing distribution diversity and label consistency in augmentations.
  - **Learning objectives**: The adaptive augmentation module in GCA, when presented with an input graph, remains fixed and non-learnable throughout the training process. This is due to the static nature of node centrality scores, resulting in unchanging dropping rates for edges and node features. Moreover, this module operates independently of the specific GNN encoder employed. On the contrary, our learnable augmentation module is dynamic, updating continuously throughout training to adapt to the dataset and the evolving GNN encoder.

Reviewer comment (Experiments):
The paper predominantly explores the concept of applying learnable data augmentation for graph anomaly detection. While this is valuable, investigating its applicability in broader graph learning tasks, such as node classification with contrastive learning, could significantly expand its scope. For example, how about its benefits to generic graph contrastive learning tasks, compared to existing contrastive techniques?

Author response:
**2. The paper predominantly explores the concept of applying learnable data augmentation for graph anomaly detection. While this is valuable, investigating its applicability in broader graph learning tasks, such as node classification with contrastive learning, could significantly expand its scope. For example, how about its benefits to generic graph contrastive learning tasks, compared to existing contrastive techniques?**

Thank you for your insightful suggestion regarding the potential applicability of our learnable data augmentation in generic graph contrastive learning tasks. We acknowledge the value of this idea. However, our current model is not straightforwardly applicable to such tasks due to the specific requirements of our label consistency metric. In graph contrastive learning, which focuses on pre-training a GNN encoder without specific label information, our method faces challenges, as it depends on label predictions for unlabeled nodes to measure label consistency.

One potential approach to address this could involve clustering nodes in the input graph and then using these clusters as proxy labels to assess label consistency. Nevertheless, due to time constraints, we have not been able to explore this adaptation in the current study. We appreciate your suggestion and agree that expanding the applicability of our augmentation approach represents an exciting and valuable direction for future research. This would indeed enhance the versatility and impact of our learnable augmentation module and the proposed metrics for label consistency and distribution diversity. We are eager to explore this avenue in our future research, aiming to further broaden the scope of our work.

Reviewer comment (Experiments):
While consistency training might usually be deployed on unlabeled data, I wonder if it would be beneficial to utilize labeled data for consistency training as well. Specifically, labeled data has exact labels, which might provide effective information for consistency training the model in dealing with the task of graph anomaly detection.

Author response:
**3. While consistency training might usually be deployed on unlabeled data, I wonder if it would be beneficial to utilize labeled data for consistency training as well. Specifically, labeled data has exact labels, which might provide effective information for consistency training the model in dealing with the task of graph anomaly detection.**

Thank you for highlighting the potential benefits of utilizing labeled data for consistency training. In our current settings, both labeled and unlabeled data are used for consistency training, but labeled samples are treated as unlabeled, using their predicted labels instead of exact labels.

Encouraged by your suggestion, we explore the impact of using exact labels for labeled nodes in consistency training. The results of this additional experiment are detailed in Appendix E.9 of our revised manuscript. For your convenience, we provide a summary in the table below:

| Dataset    | Metric   | ConsisGAD (Predicted Label) | ConsisGAD (Exact Label) |
|------------|----------|-----------------------------|-------------------------|
| Amazon     | AUROC    | 93.91±0.58                  | 93.82±0.55              |
|            | AUPRC    | 83.33±0.34                  | 83.16±0.46              |
|            | Macro F1 | 90.03±0.53                  | 89.88±0.30              |
| YelpChi    | AUROC    | 83.36±0.53                  | 82.91±0.36              |
|            | AUPRC    | 47.33±0.58                  | 47.23±0.98              |
|            | Macro F1 | 69.72±0.30                  | 69.44±0.29              |
| T-Finance  | AUROC    | 95.33±0.30                  | 95.09±0.27              |
|            | AUPRC    | 86.63±0.44                  | 86.94±0.13              |
|            | Macro F1 | 90.97±0.63                  | 91.32±0.18              |
| T-Social   | AUROC    | 94.31±0.20                  | 93.87±0.30              |
|            | AUPRC    | 58.38±2.10                  | 56.54±1.86              |
|            | Macro F1 | 78.08±0.54                  | 77.42±0.67              |

Our findings indicate that using exact labels in consistency training does not significantly improve performance on most datasets. This lack of improvement may stem from the fact that labeled data has already been utilized in the cross-entropy loss, and their reuse in the consistency training does not contribute significantly to performance enhancement. Notably, a decrease in performance is observed in the T-Social dataset. We hypothesize this could be attributed to the model overfitting to label noise, which could be mitigated by the use of predicted labels.

We hope these revisions to our submission adequately address your concerns and enhance its overall contributions. Your thoughtful feedback has been crucial in elevating the quality and coherence of our manuscript. Should you have additional comments or concerns, we remain receptive and prepared to undertake any further modifications.

[a] Graph Contrastive Learning Automated. Yuning You, Tianlong Chen, Yang Shen, Zhangyang Wang. ICML 2021.

[b] Graph Contrastive Learning with Adaptive Augmentation. Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, Liang Wang. WWW 2021.

[c] Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets. Yogesh Balaji, Tom Goldstein, and Judy Hoffman. arXiv preprint arXiv:1910.08051, 2019.

[d] Robustness May Be at Odds with Accuracy. Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. ICLR, 2018.

Reviewer comment (Evaluation):
Regarding consistency training, the proposed model employs thresholds to identify "high-quality nodes" from the unlabeled data for training. This step, akin to semi-supervised learning, assigns synthetic labels to unlabeled data for consistency training. The performance of these "high-quality nodes" is pivotal for downstream iterative training, emphasizing the significance of their accuracy. Hence, the authors should analyze the actual quality of these "high-quality nodes" and the accuracy of their predicted labels. This aspect holds significant importance in evaluating the reliability and effectiveness of the model.

Author response:
Thank you for your thorough review and insightful comments on our submission. We appreciate the time you invested in evaluating our work and offering constructive feedback. Below, we address your comments to clarify and improve our manuscript.

**1. How about the correctness of the "high-quality nodes"?**

Thanks for highlighting the significance of evaluating the quality of "high-quality nodes" in our consistency training approach. Considering your valuable suggestion, we have conducted a detailed experiment to assess this aspect and include the results in Appendix E.6 of our revised manuscript. In this experiment, we visualize Macro F1 scores of "high-quality nodes" at each epoch during training and compare these with scores from the test set. For your reference, we report the scores with an interval of 10 epochs in the following table. For more details, please kindly refer to the Appendix E.6.

|                    | Epoch 10 | Epoch 20 | Epoch 30 | Epoch 40 | Epoch 50 | Epoch 60 | Epoch 70 | Epoch 80 | Epoch 90 | Epoch 100 |
| :----------------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: | :-------: |
|       Amazon       |          |          |          |          |          |          |          |          |          |           |
| High-quality Nodes |  90.79   |  88.89   |  91.06   |  92.18   |  90.91   |  90.76   |  90.76   |  91.43   |  89.99   |   89.29   |
|     Test Nodes     |  87.56   |  88.13   |  90.10   |  90.96   |  90.98   |  90.92   |  90.04   |  90.60   |  90.07   |   90.43   |
|       Diff.        |   3.23   |   0.76   |   0.96   |   1.23   |  -0.07   |  -0.17   |   0.72   |   0.83   |  -0.08   |   -1.14   |
|      YelpChi       |          |          |          |          |          |          |          |          |          |           |
| High-quality Nodes |  67.23   |  69.23   |  68.09   |  69.11   |  68.49   |  68.98   |  69.21   |  70.02   |  69.28   |   68.76   |
|     Test Nodes     |  66.72   |  67.56   |  68.49   |  67.76   |  68.13   |  68.12   |  67.42   |  67.96   |  68.25   |   68.42   |
|       Diff.        |   0.51   |   1.68   |  -0.40   |   1.35   |   0.36   |   0.86   |   1.79   |   2.06   |   1.03   |   0.33    |
|     T-Finance      |          |          |          |          |          |          |          |          |          |           |
| High-quality Nodes |  90.82   |  90.27   |  90.15   |  90.12   |  88.50   |  91.33   |  91.11   |  91.48   |  91.14   |   91.09   |
|     Test Nodes     |  89.28   |  90.22   |  89.39   |  90.44   |  91.16   |  91.05   |  90.90   |  90.87   |  90.62   |   90.39   |
|       Diff.        |   1.54   |   0.05   |   0.76   |  -0.31   |  -2.66   |   0.28   |   0.21   |   0.62   |   0.52   |   0.71    |
|      T-Social      |          |          |          |          |          |          |          |          |          |           |
| High-quality Nodes |  75.68   |  76.35   |  76.92   |  75.82   |  76.09   |  77.53   |  75.48   |  74.07   |  76.11   |   73.59   |
|     Test Nodes     |  72.14   |  73.56   |  76.07   |  74.17   |  74.23   |  76.10   |  73.13   |  71.52   |  73.81   |   72.58   |
|       Diff.        |   3.53   |   2.80   |   0.85   |   1.65   |   1.87   |   1.43   |   2.35   |   2.54   |   2.30   |   1.01    |

The result demonstrates that "high-quality nodes" consistently achieve higher Macro F1 scores than test nodes across various datasets. On Amazon, YelpChi, T-Finance, and T-Social, the average performance improvement of "high-quality nodes" over test nodes is 1.01%, 0.83%, 0.71%, and 0.64%, respectively. Generally, as depicted in Figure 10, both "high-quality nodes" and test nodes exhibit a gradual increase in performance, with the former driving the improvement of the latter. An exception lies in the T-Social dataset, on which we observe some performance fluctuations. This is possibly due to the much smaller batch size and training data relative to the whole dataset size. Overall, our analysis showcases the accuracy of the predicted labels and the robustness of our selection criteria for "high-quality nodes".

Reviewer comment (Experiments):
The paper introduces essential hyperparameters, including $\alpha$ and $\xi$. While the authors provide their specific settings in the experiments, the impact of these parameters on the performance of the model remains unexplored. The authors should perform a sensitivity analysis for the crucial hyperparameters in the proposed model.

Author response:
**2. The authors should perform a sensitivity analysis for the crucial hyperparameters in the proposed model.**

Taking your valuable advice, we have performed a sensitivity analysis on hyper-parameters and included the results in Appendix E.7 of our revised manuscript. We vary the weights of the label consistency loss $\alpha$ among \{0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0\} and the drop ratio $\xi$ among \{0.0, 0.1, 0.2, 0.3, 0.4, 0.5\}, and measure the corresponding model performance across the Amazon, YelpChi, and T-Finance datasets. For your ease of reference, we have summarized the results in the tables below. Additionally, we have included in Appendix E.7 a sensitivity analysis of the normal threshold $\tau_n$ and anomalous threshold $\tau_a$, providing a more comprehensive understanding of our model's behavior under different hyper-parameter settings.

[Full table content as in rebuttal with alpha and xi results]

For the weight of the label consistency loss $\alpha$, our results indicate that an $\alpha$ value of around 1.0 yields optimal performance for the Amazon and YelpChi datasets, whereas a value of around 5.0 is preferable for the T-Finance dataset. Overall, the model performance remains stable with a moderate $\alpha$ value. If $\alpha$ is set to an excessively high value, the performance will be greatly impaired. This is likely due to the diminished diversity in generated augmentations, which is crucial for effective consistency training. 

Regarding the drop ratio $\xi$, our results reveal that maintaining $\xi$ within a range of 0.1 to 0.3 offers consistent benefits across all datasets. A smaller $\xi$ reduces ConsisGAD to its backbone model, limiting its ability to harness unlabeled data effectively. Conversely, a higher $\xi$ could lead to significant information loss, making it hard for the model to discern valuable patterns.

Reviewer comment (Experiments):
The proposed GNN backbone shows promise for anomaly detection by capturing distinctions in the homophily distribution of nodes in graphs. However, it raises intriguing questions about its performance on graphs with varying levels of heterophily. Specifically, how well does the proposed GNN backbone perform on graphs with low or high heterophily when applied to generic multi-class node classification tasks?

Author response:
**3. Can the proposed GNN backbone effectively handle generic multi-class node classification tasks?**
Pursuant to your advice, we have extended our experiments to include graphs with low and high heterophily and discussed the results in Appendix E.8 of our revised manuscript. In this experiment, we adhere to the experimental procedure of a recent work, the Feature Selection Graph Neural Network (FSGNN) [a], and use their public repository for experimentation. Our evaluation encompasses three homophily graphs and six heterophily graphs, with homophily ratios ranging from 0.11 to 0.81. This range provides a comprehensive spectrum for performance assessment of our model. For your easy reference, we list the dataset statistics in the following table:

[Full table content as in rebuttal with homophily ratio datasets and results table]

Reviewer comment (Experiments):
The authors should perform a sensitivity analysis for the crucial hyperparameters in the proposed model.

Author response:
**2. The authors should perform a sensitivity analysis for the crucial hyperparameters in the proposed model.**

Taking your valuable advice, we have performed a sensitivity analysis on hyper-parameters and included the results in Appendix E.7 of our revised manuscript. We vary the weights of the label consistency loss $\alpha$ among \{0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0\} and the drop ratio $\xi$ among \{0.0, 0.1, 0.2, 0.3, 0.4, 0.5\}, and measure the corresponding model performance across the Amazon, YelpChi, and T-Finance datasets. For your ease of reference, we have summarized the results in the tables below. Additionally, we have included in Appendix E.7 a sensitivity analysis of the normal threshold $\tau_n$ and anomalous threshold $\tau_a$, providing a more comprehensive understanding of our model's behavior under different hyper-parameter settings.

[Full table content as in R2 above]

For the weight of the label consistency loss $\alpha$, our results indicate that an $\alpha$ value of around 1.0 yields optimal performance for the Amazon and YelpChi datasets, whereas a value of around 5.0 is preferable for the T-Finance dataset. Overall, the model performance remains stable with a moderate $\alpha$ value. If $\alpha$ is set to an excessively high value, the performance will be greatly impaired. This is likely due to the diminished diversity in generated augmentations, which is crucial for effective consistency training. 

Regarding the drop ratio $\xi$, our results reveal that maintaining $\xi$ within a range of 0.1 to 0.3 offers consistent benefits across all datasets. A smaller $\xi$ reduces ConsisGAD to its backbone model, limiting its ability to harness unlabeled data effectively. Conversely, a higher $\xi$ could lead to significant information loss, making it hard for the model to discern valuable patterns.

Reviewer comment (Experiments):
Can the proposed GNN backbone effectively handle generic multi-class node classification tasks?

Author response:
It is important to note that our proposed GNN backbone, ConsisGAD(GNN), is specifically tailored to exploit the homophily distribution differences between normal and anomalous nodes. Such a phenomenon is characteristic in the graph anomaly detection task (i.e., the imbalanced binary classification task) where normal nodes have high homophily distribution while anomalous nodes have low homophily distribution. This prominent distribution discrepancy lays down the foundation of our GNN backbone. When it comes to multi-class node classification tasks, the homophily-aware neighborhood aggregation (Equation (4)) in our GNN backbone will model the neighbor label distribution within a node's neighborhood. If this neighborhood label distribution exhibits distinguishable patterns for different classes, we would expect our backbone model to function well. 

From the results, we observe that our GNN backbone model performs comparably to existing baselines in the three homophily graphs. Our GNN backbone is built upon the Message-Passing Neural Network (MPNN) framework, which allows handling homophily information naturally. For heterophily graphs, our backbone model yields promising results in the Wisconsin, Texas, and Cornell datasets, but encounters challenges in the Chameleon, Squirrel, and Actor datasets. Our further investigations in Figure 15 reveal that, in Wisconsin, Texas, and Cornell, nodes of different classes have distinct neighbor label distribution within their neighborhood, which allows our backbone model to distinguish different classes effectively. However, such distinct patterns are absent in Chameleon, Squirrel, and Actor, explaining the struggle of our backbone model on them. Notably, despite a low accuracy on the Actor dataset, our model achieves a new state-of-the-art result, underscoring its potential in handling heterophily graphs in generic multi-class node classification tasks. We recognize the opportunity for further enhancements, particularly on datasets where distinct neighborhood patterns are less pronounced. Future research may explore synergies between our GNN backbone model and other advanced techniques to elevate performance on heterophily graphs.

We hope these revisions and additional analyses address your concerns and strengthen the contribution of our work. We appreciate the constructive guidance you have provided, which has substantially improved the manuscript. If you have additional comments or concerns, we welcome your input and are ready to make any necessary adjustments. 

[a] Simplifying approach to node classification in graph neural networks. Sunil Kumar Maurya, Xin Liu, and Tsuyoshi Murata. Journal of Computational Science, 2022.