ARC: A Generalist Graph Anomaly Detector with In-Context Learning

Authors: yixin  liu
shiyuan  li
yu  zheng
qingfeng  chen
chengqi  zhang
shirui  pan
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 22:55:33

Abstract: Graph anomaly detection (GAD), which aims to identify abnormal nodes that differ from the majority within a graph, has garnered significant attention. However, current GAD methods necessitate training specific to each dataset, resulting in high training costs, substantial data requirements, and limited generalizability when being applied to new datasets and domains. To address these limitations, this paper proposes ARC, a generalist GAD approach that enables a ``one-for-all'' GAD model to detect anomalies across various graph datasets on-the-fly. Equipped with in-context learning, ARC can directly extract dataset-specific patterns from the target dataset using few-shot normal samples at the inference stage, without the need for retraining or fine-tuning on the target dataset. ARC comprises three components that are well-crafted for capturing universal graph anomaly patterns: 1) smoothness-based feature **A**lignment module that unifies the features of different datasets into a common and anomaly-sensitive space; 2) ego-neighbor **R**esidual graph encoder that learns abnormality-related node embeddings; and 3) cross-attentive in-**C**ontext anomaly scoring module that predicts node abnormality by leveraging few-shot normal samples. Extensive experiments on multiple benchmark datasets from various domains demonstrate the superior anomaly detection performance, efficiency, and generalizability of ARC.

Main paper: 
 introduction: 
 Graph anomaly detection (GAD) aims to distinguish abnormal nodes that show significant dissimilarity from the majority of nodes in a graph. GAD has broad applications across various real-world scenarios, such as fraud detection in financial transaction networks [1] and rumor detection in social networks [2] . As a result, GAD has attracted increasing research attention in recent years [3, 4, 5, 6, 7, 8] . Conventional GAD methods employ shallow mechanisms to model node-level abnormality [9, 10, 11] ; however, they face limitations in handling high-dimensional features and complex interdependent relations on graphs. Recently, graph neural network (GNN)-based approaches have emerged as the go-to solution for the GAD problem due to their superior performance [4, 6] . Some GNN-based GAD approaches regard GAD as a supervised binary classification problem and use specifically designed GNN architectures to capture anomaly patterns [6, 12, 13, 14] . Another line of approaches targets the more challenging unsupervised paradigm, employing various unsupervised learning objectives and frameworks to identify anomalies without relying on labels [4, 15, 16, 17] . Despite their remarkable detection performance, the existing GAD approaches follow a "one model for one dataset" learning paradigm (as shown in Fig. 1 The data requirements pose a challenge when applying GAD to scenarios with sparse data, data privacy concerns, or high label annotation costs. ❸ Poor generalizability. On a new-coming dataset, existing GAD methods require hyperparameter tuning or even model architecture modifications to achieve optimal performance, which increases the cost of applying them to new data and domains.Given the above limitations, a natural question arises: Can we train a "one-for-all" GAD model that can generalize to detect anomalies across various graph datasets from different application domains, without any training on the target data? Following the trend of artificial general intelligence and foundation models, a new paradigm termed "generalist anomaly detection", originating from image anomaly detection, is a potential answer to this question [18] . As shown in Fig. 1 (c ), in the generalist paradigm, we only need to train the GAD model once; afterward, the well-trained generalist GAD model can directly identify anomalies on diverse datasets, without any re-training or fine-tuning. Considering the diversity of graph data across different domains and datasets, the labels of few-shot normal nodes are required during the inference stage to enable the model to grasp the fundamental characteristics of the target dataset. Compared to conventional paradigms, the generalist paradigm eliminates the need for dataset-specific training, resulting in fewer computations, lower data costs, and stronger generalizability when applying GAD models to new datasets.Nevertheless, due to the unique characteristics of graph data and GAD problem, it is non-trivial to design a generalist GAD approach. The challenge is three-fold: C1 -Feature alignment. Unlike image data, which are typically represented in a consistent RGB feature space, the feature dimensionality and semantic space can vary significantly across different graph datasets. Substituting features with unified representations generated by large language models may be a potential solution [19] ; however, this approach is limited to specific feature semantics and cannot address more general cases [20] . C2 -Representation encoding. As the core of a generalist GAD model, a GNN-based encoder is expected to learn dataset-agnostic and abnormality-aware node embeddings for anomaly detection. However, in the absence of universal pre-trained foundation models [18] for graph data, crafting a potent encoder for a generalist GAD model presents a challenge. C3 -Few-shot sample-guided prediction. Existing GAD methods typically focus on single dataset settings, where dataset-specific knowledge is embedded in the model through training, enabling it to predict abnormality for each node independently. In contrast, a generalist GAD model should derive such knowledge from a small number of normal nodes. In this case, how to effectively utilize the few-shot normal samples during inference remains an open question.To tackle these challenges, we introduce ARC, a generalist GAD approach based on in-context learning. ARC comprises three meticulously designed modules, each targeting a specific challenge. To address C1, we introduce a smoothness-based feature Alignment module, which not only standardizes features across diverse datasets to a common dimensionality but also arranges them in an anomalysensitive order. To deal with C2, we design an ego-neighbor Residual graph encoder. Equipped with a multi-hop residual-based aggregation scheme, the graph encoder learns attributes that indicate high-order affinity and heterophily, capturing informative and abnormality-aware embeddings across different datasets. Last but not least, to solve C3, we propose a cross-attentive in-Context anomaly scoring module. Following the in-context learning schema, we treat the few-shot normal nodes as context samples and utilize a cross-attention block to reconstruct the embeddings of unlabeled samples based on the context samples. Then, the reconstruction distance can serve as the anomaly score for each unlabeled node. In summary, this paper makes the following contributions:• Problem. We, for the first time, propose to investigate the generalist GAD problem, aiming to detect anomalies from various datasets with a single GAD model, without dataset-specific fine-tuning. • Methodology. We propose a novel generalist GAD method ARC, which can detect anomalies in new graph datasets on-the-fly via in-context learning based on few-shot normal samples. • Experiments. We conduct extensive experiments to validate the anomaly detection capability, generalizability, and efficiency of ARC across multiple benchmark datasets from various domains. 
related work: 
 In this section, we offer a brief review of pertinent related works, with a more extensive literature review available in Appendix A.Anomaly Detection. Anomaly detection (AD) aims to identify anomalous samples that deviate from the majority of samples [21] . Mainstream AD methods focus on unsupervised settings and employ various unsupervised techniques to build the models [22, 23, 24, 25, 26, 27, 28, 29] . To enhance the generalizability of AD methods across diverse datasets, RegAD [30] considers few-shot setting and trains a single generalizable model capable of being applied to new in-domain data without re-training or fine-tuning. WinCLIP [31] utilizes visual-language models (VLMs, e.g., CLIP [32] ) with well-crafted text prompts to perform zero/few-shot AD for image data. InCTRL [18] , as the first generalist AD approach, integrates in-context learning and VLMs to achieve domain-agnostic image AD with a single model. However, due to their heavy reliance on pre-trained vision encoders/VLMs and image-specific designs, these approaches excel in AD for image data but face challenges when applied to graph data.In this paper, we focus on the node-level AD on graphs and refer to it as "graph anomaly detection (GAD)" following [6, 33, 34] . While shallow methods [9, 10, 11] show limitations in handling complex real-world graphs [4] , the advanced approaches are mainly based on GNNs [35] . The GNN-based approaches can be divided into supervised and unsupervised approaches [3, 5, 7] . Supervised GAD approaches assume that the labels of both normal and anomalous nodes are available for model training [7] . Hence, related studies mainly introduce GAD methods in a binary classification paradigm [6, 12, 13, 14, 36, 37] . In contrast, unsupervised GAD approaches do not require any labels for model training. They employ several unsupervised learning techniques to learn anomaly patterns on graph data, including data reconstruction [4, 34, 38] , contrastive learning [15, 39, 40] , and other auxiliary objectives [16, 17, 41, 42] . Nevertheless, all the above methods adhere to the conventional paradigm of "one model for one dataset". Although some GAD approaches [43, 44] can handle cross-domain scenarios, their requirement for high correlation (e.g., aligned node features) between source and target datasets limits their generalizability. Differing from existing methods, our proposed ARC is a "one-for-all" GAD model capable of identifying anomalies across target datasets from diverse domains, without the need for re-training or fine-tuning.In-Context Learning (ICL). ICL enables a well-trained model to be effectively (fine-tuning-free) adapted to new domains, datasets, and tasks based on minimal in-context examples (a.k.a. context samples), providing powerful generalization capability of large language models (LLMs) [45, 46, 47] and computer vision (CV) models [18, 48, 49, 50] . Two recent approaches, PRODIGY [51] and UniLP [52] attempt to use ICL for GNNs to solve the node classification and link prediction tasks, respectively. However, how to use ICL to deal with the generalist GAD problem where only normal context samples are available still remains open. 
problem statement: 
 Notations. Let G = (V, E, X) be an attributed graph with n nodes and m edges, where V = {v 1 , • • • , v n } and E are the set of nodes and edges, respectively. The node-level attributes are included by feature matrix X ∈ R n×d , where each row X i indicates the feature vector for node v i . The inter-node connectivity is represented by an adjacency matrix A ∈ {0, 1} n×n , where the i, j-th entry A ij = 1 means v i and v j are connected and vice versa. 𝐙 [1] 𝐙 [2] 𝐑 [1] 𝐑 [2] 𝐇𝐇 𝑘 𝐇 𝑞 𝐖 𝑞 𝐊 𝐐 ෩ 𝐇 𝑞 Figure 2:The overall pipeline of ARC, the proposed generalist GAD approach.Conventional GAD Problem. GAD aims to differentiate abnormal nodes V a from normal nodes V n within a given graph G = (V, E, X), where V a and V n satisfyV a ∪ V n = V, V a ∩ V n = ∅, and |V a | ≪ |V n |.An anomaly label vector y ∈ {0, 1} n can be used to denote the abnormality of each node, where the i-th entryy i = 1 iff v ∈ V a and y i = 0 iff v ∈ V n .Formally, the goal of GAD is to learn an anomaly scoring function (i.e., GAD model)f : V → R such that f (v ′ ) > f (v) for ∀v ′ ∈ V a and ∀v ∈ V n .In the conventional GAD setting of "one model for one dataset", the GAD model f is optimized on the target graph dataset D = (G, y) with a subset of anomaly labels (in supervised setting) or without labels (in unsupervised setting). After sufficient training, the model f can identify anomalies within the target graph G during the inference phase.Generalist GAD Problem. In this paper, we investigate the generalist GAD problem, wherein we aim to develop a generalist GAD model capable of detecting abnormal nodes across diverse graph datasets from various application domains without any training on the specific target data. Formally, we define the generalist GAD setting, aligning it with its counterpart in image AD as introduced by Zhu et al. [18] . Specifically, let T train = {D(1)train , • • • , D (N )train } be a collection of training datasets, where eachD (i) train = (G (i) train , y (i)train ) is a labeled dataset from an arbitrary domain. We aim to train a generalist GAD model f on T train , and f is able to identify anomalies within any test graph datasetD (i) test ∈ T test , where T test = {D (1) test , • • • , D (N ′ )test } is a collection of testing datasets. Note that T train ∩ T test = ∅ and the datasets in T train and T test can be drawn from different distributions and domains. Following [18] , we adopt a "normal few-shot" setting during inference: for each D (i) test , only a handful of n k normal nodes (n k ≪ n) are available, and the model f is expected to predict the abnormality of the rest nodes without re-training and fine-tuning.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA]• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [NA] Justification: There is no societal impact of the work performed. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
arc: a generalist gad approach: 
 In this section, we introduce ARC, a generalist GAD approach capable of identifying anomalies across diverse graph datasets without the need for specific fine-tuning. The overall pipeline of ARC is demonstrated in Fig. 2 . Firstly, to align the features of different datasets, we introduce a smoothnessbased feature alignment module (Sec. 4.1), which not only projects features onto a common plane but also sorts the dimensions in an anomaly-sensitive order. Next, to capture abnormality-aware node embeddings, we propose a simple yet effective GNN model termed ego-neighbor residual graph encoder (Sec. 4.2), which constructs node embeddings by combining residual information between an ego node and its neighbors. Finally, to leverage knowledge from few-shot context samples for predicting node-level abnormality, we introduce a cross-attentive in-context anomaly scoring module (Sec. 4.3). Using the cross-attention block, the model learns to reconstruct query node embeddings based on context node embeddings. Ultimately, the drift distance between the original and reconstructed query embeddings can quantify the abnormality of each node.Graph data from diverse domains typically have different features, characterized by differences in dimensionality and unique meanings for each dimension. For example, features in a citation network usually consist of textual and meta-information associated with each paper, whereas in a social network, the features may be the profile of each user. Therefore, in the first step, we need to align the features into a shared feature space. To achieve this, we introduce the feature alignment module in ARC, consisting of two phases: feature projection, which aligns dimensionality, and smoothnessbased feature sorting, which reorders features according to their smoothness characteristics.Feature Projection. At the first step of ARC, we employ a feature projection block to unify the feature dimensionality of multiple graph datasets [20] . Specifically, given a feature matrixX (i) ∈ R n (i) ×d (i) of D (i) ∈ T train ∪ T test ,the feature projection is defined by a linear mapping:EQUATIONwhere X(i) is the projected feature matrix for D (i) , d u is a predefined projected dimension shared across all datasets, andW (i) ∈ R d (i)×du is a dataset-specific linear projection weight matrix. To maintain generality, W (i) can be defined using commonly used dimensionality reduction approaches such as singular value decomposition [53] (SVD) and principal component analysis [54] (PCA).Smoothness-Based Feature Sorting. Although feature projection can align dimensionality, the semantic meaning of each projected feature across different datasets remains distinct. Considering the difficulty of semantic-level matching without prior knowledge and specific fine-tuning [19, 20] , in this paper, we explore an alternative pathway: aligning features based on their contribution to anomaly detection tasks. Through analytical and empirical studies, we pinpoint that the smoothness of each feature is strongly correlated with its contribution to GAD. Building on this insight, in ARC, we propose to sort the features according to their contribution as our alignment strategy. From the perspective of graph signal processing, Tang et al. [6] have demonstrated that the inverse of the lowfrequency energy ratio monotonically increases with the anomaly degree. In other words, high-frequency graph signals tend to play a more significant role in detecting anomalies. Similar findings have also been observed from the standpoint of spatial GNNs [37, 55] , where heterophily information has been shown to be crucial in discriminating anomalies. Motivated by these findings, can we develop a metric to gauge the contribution of each feature to GAD based on its frequency/heterophily? Considering its correlation to frequency [56] and heterophily [57, 58, 59, 60] , in this paper, we select featurelevel smoothness as the measure for contribution. Formally, given a graph G = (V, E, X) with a normalized feature matrix X, the smoothness of the k-th feature dimension is defined as:EQUATIONwhere a lower s k indicates a significant change in the k-th feature between connected nodes, implying that this feature corresponds to a high-frequency graph signal and exhibits strong heterophily.To verify whether smoothness can indicate the contribution of features in GAD, we further conduct empirical analysis (experimental setup and more results can be found in Appendix B). Concretely, we sort the raw features of each dataset based on the smoothness s k and divide them into 5 groups according to the percentile of s k . Then, we train different GAD models using each group of features separately, and the performance is shown in Fig. 3 and 8 . On both datasets, a model-agnostic observation is that the features with lower s k are more helpful in discriminating anomalies. The consistent trend demonstrates the effectiveness of s k as an indicator of the role of features in GAD.In light of this, given the projected features of different datasets, we can align their feature spaces by rearranging the permutation of features based on the descending order of s k w.r.t. each projected feature. For all datasets, the feature in the first column is the one with the lowest s k , which deserves more attention by ARC; conversely, features with less contribution (i.e. higher s k ) are placed at the end. In this way, the GNN-based model can learn to filter graph signals with different smoothness levels automatically and predict anomalies accordingly. During inference, the smoothness-related information remains transferable because we adhere to the same alignment strategy.Once the features are aligned, we employ a GNN-based graph encoder to learn node embeddings that capture both semantic and structural information for each node. The learned embedding can be utilized to predict the abnormality of the corresponding node with the downstream anomaly scoring module. A naive solution is directly employing commonly used GNNs, such as GCN [61] or GAT [62] , as the graph encoder. However, due to their low-pass filtering characteristic, these GNNs face difficulty in capturing abnormality-related patterns that are high-frequency and heterophilic [6, 37] . Moreover, most GNNs, including those tailored for GAD, tend to prioritize capturing node-level semantic information while disregarding the affinity patterns of local subgraphs [17] . Consequently, employing existing GNN models as the encoder may overemphasize dataset-specific semantic knowledge, but overlook the shared anomaly patterns (i.e. local node affinity) across different datasets.To address the above issues, we design an ego-neighbor residual graph encoder for ARC. Equipped with a residual operation, the encoder can capture multi-hop affinity patterns of each node, providing valuable and comprehensive information for anomaly identification. Similar to the "propagation then transformation" GNN architecture in SGC [63] , our graph encoder consists of three steps: multi-hop propagation, shared MLP-based transformation, and ego-neighbor residual operation. In the first two steps, we perform propagation on the aligned feature matrix X ′ = X [0] for L iterations, and then conduct transformation on the initial and propagated features with a shared MLP network:EQUATIONwhere l ∈ {0, • • • , L}, X [l] is the propagated feature matrix at the l-th iteration, Z [l] is the transformed representation matrix at the l-th iteration, and Ã is the normalized adjacency matrix [61, 63] . Note that, unlike most GNNs that only consider the features/representations after L-iter propagation, here we incorporate both the initial features and intermediate propagated features and transform them into the same representation space. After obtainingZ [0] , • • • , Z [L], we calculate the residual representations by taking the difference between Z [l] (1 ≤ l ≤ L) and Z [0] , and then concatenate the multi-hop residual representations to form the final embeddings:EQUATIONwhere R [l] is the residual matrix at the l-th iteration, H ∈ R n×de is the output embedding matrix, and || denotes the concatenation operator.Discussion. Compared to existing GNNs, our graph encoder offers the following advantages. Firstly, with the residual operation, the proposed encoder emphasizes the difference between the ego node and its neighbors rather than ego semantic information. This approach allows for the explicit modeling of local affinity through the learned embeddings. Since local affinity is a crucial indicator of abnormality and this characteristic can be shared across diverse datasets [17] , the learned embeddings can offer valuable discriminative insights for downstream prediction. Second, the residual operation performs as a high-pass filter on the graph data, aiding ARC in capturing more abnormality-related attributes, i.e., high-frequency signals and local heterophily. Moreover, unlike existing approaches [15, 17] that only consider 1-hop affinity, our encoder also incorporates higher-order affinity through the multi-hop residual design, which enables ARC to capture more complex graph anomaly patterns. More discussion and comparison to the existing GNNs/GAD methods are conducted in Appendix C.To utilize the few-shot normal samples (denoted by context nodes) to predict the abnormality of the remaining nodes (denoted by query nodes), in ARC, we devise an in-context learning module with a cross-attention mechanism for anomaly scoring. The core idea of our in-context learning module is to reconstruct the node embedding of each query node using a cross-attention block to blend the embeddings of context nodes. Then, the drift distance between the original and reconstructed embeddings of a query node can serve as the indicator of its abnormality.Specifically, we partition the embedding matrix H into two parts by indexing the corresponding row vectors: the embeddings of context nodes H k ∈ R n k ×de and the embeddings of query nodes H q ∈ R nq×de . Then, a cross-attention block is utilized to reconstruct each row of H q through a linear combination of H k :EQUATIONwhere Q ∈ R nq×de and K ∈ R n k ×de are the query and key matrices respectively, W q and W k are learnable parameters, and Hq is the reconstructed query embedding matrix. Note that, unlike the conventional cross-attention blocks [64, 65, 66] that further introduce a value matrix V, our block directly multiplies the attention matrix with H k . This design ensures that Hq is in the same embedding space as H q and H k . Thanks to this property, given a query node v i , we can calculate its anomaly score f (v i ) by computing the L2 distance between its query embedding vector Hq i and the corresponding reconstructed query embedding vector Hq i , i.e., f(v i ) = d(Hq i , Hq i ) = de j=1 Hq ij -Hq ij 2 .Context Node Embedding Query Node Embedding Rec. Query Node Embedding Discussion. The design of cross-attentive in-context anomaly scoring follows a basic assumption: normal query nodes have similar patterns to several context nodes, and hence their embeddings can be easily represented by the linear combination of context node embeddings. Consequently, given a normal node, its original and reconstructed embeddings can be close to each other in the embedding space. In contrast, abnormal nodes may display distinct patterns compared to normal ones, making it difficult to depict their corresponding abnormal query embeddings using context embeddings. As a result, their drift distance s i can be significantly larger. Fig. 4 provides examples for the scenarios of (a) single-class normal and (b) multi-class normal 3 . In both cases, the drift distance (➞) can be a significant indicator for distinguishing anomaly (5) from normal nodes (1~4). Interestingly, if the attention matrix assigns uniform weights to all context nodes, then our scoring module becomes a one-class classification model [22] . This property ensures the anomaly detection capability of ARC even without extensive training. A detailed discussion is conducted in Appendix D.2.Model Training. To optimize ARC on training datasets T train , we employ a marginal cosine similarity loss to minimize the drift distance of normal nodes while maximizing the drift distance of abnormal nodes. Specifically, given graph data with anomaly labels, we randomly select n k normal nodes as context nodes and sample an equal number of normal and abnormal nodes as query nodes.Then, given a query node v i with embedding Hq i , reconstructed embedding Hq i , and anomaly label y i , the sample-level loss function can be written by:EQUATIONwhere cos(•, •) and max(•, •) denote the cosine similarity and maximum operation, respectively, and ϵ is a margin hyperparameter. Detailed algorithmic description and complexity analysis of ARC can be found in Appendix E. 
experimental setup: 
 Datasets. To learn generalist GAD models, we train the baseline methods and ARC on a group of graph datasets and test on another group of datasets. For comprehensive evaluations, we consider graph datasets spanning a variety of domains, including social networks, citation networks, and e-commerce co-review networks, each of them with either injected anomalies or real anomalies [7, 15, 17] . Inspired by [52] , we train the models on the largest dataset of each type and conduct testing on the remaining datasets. Specifically, the training datasets T train comprise PubMed, Flickr, Questions, and YelpChi, while the testing datasets T test consist of Cora, CiteSeer, ACM, BlogCatalog, Facebook, Weibo, Reddit, and Amazon. For detailed information, please refer to Appendix F.1.Baselines. We compare ARC with both supervised and unsupervised methods. Supervised methods include two conventional GNNs, i.e., GCN [61] and GAT [62] , and three state-of-the-art GNNs specifically designed for GAD, i.e., BGNN [67] , BWGNN [6] , and GHRN [37] . Unsupervised methods include four representative approaches with distinct designs, including the generative method DOMINANT [4] , the contrastive method CoLA [15] , the hop predictive method HCM-A [16] , and the affinity-based method TAM [17] . For detailed information, refer to Appendix F.2.Evaluation and Implementation. Following [7, 17, 68] , we employ AUROC and AUPRC as our evaluation metrics for GAD. We report the average AUROC/AUPRC with standard deviations across 5 trials. We train ARC on all the datasets in T train jointly, and evaluate the model on each dataset in T test in an in-context learning manner (n k = 10 as default). For the supervised baselines, we follow the same training and testing procedure (denoted as "pre-train only"), since no labeled anomaly is available for fine-tuning. For the unsupervised baselines, we consider two settings: "pre-train only" and "pre-train & fine-tune". In the latter, we additionally conduct dataset-specific fine-tuning with a few epochs. To standardize the feature space in baseline methods, we utilize either learnable projection or random projection as an adapter between the raw feature and the model input layer. We utilize random search to determine the optimal hyperparameters for both the baselines and ARC. Since our goal is to train generalist GAD models, we do not perform dataset-specific hyperparameter search, but instead use the same set of hyperparameters for all testing datasets. More implementation details can be found in Appendix F.3.Performance Comparison. We have the following observations. ❶ ARC demonstrates strong anomaly detection capability in the generalist GAD scenario, without any fine-tuning. Specifically, ARC achieves state-ofthe-art performance on 5 out of 8 datasets and demonstrates competitive performance on the remainder. On several datasets, ARC demonstrates significant improvement compared to the best baseline (e.g., ↑21.1% on Cora, ↑18.8% on CiteSeer, and ↑36.6% on Amazon). ❷ Simply pre-training the dataset-specific GAD methods typically results in poor generalization capability to new datasets. Specifically, the AUROC of the majority of "pre-train only" approaches is close to random guessing (50%) or even lower. ❸ With dataset-specific fine-tuning, the baseline methods achieve better performance in the majority of cases. However, the improvement can be minor or even negative in some cases, demonstrating the limitations of fine-tuning. Additionally, we observe that their performance is sometimes lower than the reported results from training models from scratch [4, 15, 17] , indicating potential risk of negative transfer within the "pre-train & fine-tune" paradigm. ❹ Unsupervised baselines (except HCM-A) generally outperform the supervised ones, highlighting the difficulty of training a generalist GAD model using the binary classification paradigm.Effectiveness of #Context Nodes. To investigate how the number of context nodes n k affects the performance of ARC during inference, we vary n k within the range of 2 to 100. The results are shown in Fig. 5 (more results are in Appendix G.2). From the figure, we observe that the performance of ARC increases as more context nodes are involved, demonstrating its capability to leverage these labeled normal nodes with in-context learning. Furthermore, we can conclude that ARC is also label-efficient: when n k ≥ 10, the performance gain from using more context nodes becomes minor; moreover, even when n k is extremely small, ARC can still perform well on the majority of datasets. From the results, we can conclude that all three components significantly contribute to the performance. Among them, the in-context anomaly scoring module has a significant impact, as the performance of w/o C is close to random guessing on most datasets. The residual graph encoder also has a significant impact on the final performance. Notably, Weibo dataset is an exception where the GCN encoder performs better. A possible reason is that the Weibo dataset exhibits different anomaly patterns compared to the others. Efficiency Analysis. To assess the runtime efficiency of ARC, we compare the inference and fine-tuning time on the ACM dataset.G C N G A T B G N N B W G N N G H R N D O M I . C o L A H C M -A T A M A R C D O M I . C o L A H C M -A T A MAs depicted in Fig. 6 , ARC demonstrates comparable runtime performance with the fastest GNNs (e.g., GCN and BWGNN), and significantly outperforms the unsupervised methods in terms of efficiency. Additionally, we observe that dataset-specific finetuning consumes more time compared to inference. Visualization. To investigate the weight allocation mechanism of the cross-attention module in ARC, we visualize the attention weights between context nodes and query nodes in Fig. 7 (additional results are in Appendix G.4). From Fig. 7 (a), it is evident that ARC tends to assign uniform attention weights to normal nodes, leading to reconstructed embeddings that closely resemble the average embedding of the context nodes. Conversely, anomalies are reconstructed using a combination of 1 or 2 context nodes, suggesting that their embeddings are farther from the center. This allocation aligns with the case of "single-class normal" in Fig. 4 (a). Differently, in Fig. 7 (b), we observe that each normal query node is assigned to several context nodes following two fixed patterns, corresponding to the case of "multi-class normal" in Fig. 4(b) . In summary, the cross-attention module enables ARC to adapt to various normal/anomaly distribution patterns, enhancing its generalizability. 
conclusion: 
 In this paper, we take the first step towards addressing the generalist GAD problem, aiming to detect anomalies across diverse graph datasets with a "one-for-all" GAD model, without requiring dataset-specific fine-tuning. We introduce ARC, a novel and well-crafted in-context learning-based generalist GAD approach, capable of identifying anomalies on-the-fly using only few-shot normal nodes. Extensive experiments on real-world datasets from various domains demonstrate the detection prowess, generalizability, and efficiency of ARC compared to existing approaches. One limitation is that ARC can only use normal context samples during inference but cannot directly utilize abnormal context samples, even when they are available. A potential future direction could involve developing generalist GAD methods that utilize context samples containing both anomalies and normal instances. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [NA]Justification: The paper does not use existing assets.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: Anomaly Detection. The objective of anomaly detection (AD) is to identify anomalous samples that deviate from the majority of samples [21] . Due to the difficulty of collecting labeled anomaly data, mainstream AD methods mainly focus on unsupervised settings. To capture anomaly patterns without guidance by annotated labels, existing studies employ several advanced techniques to learn powerful AD models, such as one-class classification [22, 23] , distance measurement [24, 69] , data reconstruction [25, 70, 71] , generative models [26, 72, 73] , and self-supervised learning [27, 74] . For example, DeepSVDD [22] introduces a fully deep one-class classification objective for unsupervised anomaly detection, optimizing a data-enclosing hypersphere in output space to extract common factors of variation and demonstrating theoretical properties such as the v-property. AnoDDPM [72] , a simplex noise-based approach for anomaly detection, enhances anomaly capture with a partial diffusion strategy [75] and multiscale simplex noise processing, facilitating faster inference and training on high-resolution images. While effective, these approaches are tailored to identify abnormal samples within a predetermined target dataset (i.e., the dataset for training), restricting their generalizability to new domains.Cross-Dataset Anomaly Detection. Recently, some advanced AD methods aim to transcend dataset limitations, enhancing their generalizability across diverse datasets. A research line aims to address the AD problem under domain or distribution shifts [76, 77, 78, 79] ; however, these approaches require domain relevance between the source and target datasets, thus limiting their generalizability. To enable the model to better understand the patterns in target datasets, a viable approach is to incorporate a few-shot setting, allowing access to a limited number of normal samples from the target datasets. Under the few-shot setting, RegAD [30] is a pioneering approach that trains a single generalizable model capable of being applied to new in-domain data without re-training or finetuning. WinCLIP [31] utilizes visual-language models (VLMs, e.g., CLIP [32] ) with well-crafted text prompts to perform zero/few-shot AD for image data. InCTRL [18] , as the first generalist AD approach, integrates in-context learning and VLMs to achieve domain-agnostic image AD with a single model. However, due to their heavy reliance on pre-trained vision encoders/VLMs and imagespecific designs, these approaches excel in anomaly detection for image data but face challenges when applied to graph data.Anomaly Detection on Graph Data. Based on the granularity of anomaly samples within graph data, existing AD approaches can be primarily categorized into three classes: node-level [4, 6] , edge-level [80, 81, 82] , and graph-level [83, 84, 85, 86, 87, 88] AD. Due to its broad real-world applications, node-level AD receives the most research attention [3] . In this paper, we focus on the node-level AD and refer to it as "graph anomaly detection (GAD)", following the convention of most previous papers [6, 33, 34] .Early GAD methods aimed to detect anomalies through shallow mechanisms. For example, AMEN [9] detects anomalies by utilizing the attribute correlations of nodes in each ego-network on the attribute network. In addition, residual analysis is another common method to measure the anomalies of nodes on an attribute network. In particular, Radar [10] characterizes the residuals of attribute information and their coherence with network information for anomaly detection. Further, ANOMALOUS [11] proposes joint learning of attribute selection and anomaly detection based on CUR decomposition and residual analysis. Despite the success of these methods on low-dimensional attribute graph data, they do not work well when graphs have complex structures and high-dimensional attributes [89, 90, 91, 92] due to the limitations of their shallow mechanisms.To overcome the limitations of shallow approaches, recently, GNN-based methods have become the de facto solution for GAD tasks. Existing GNN-based GAD approaches can be divided into two research lines: supervised GAD and unsupervised GAD [3, 5, 7] . Supervised GAD approaches assume that the labels of both normal and anomalous nodes are available for model training [7] . Hence, related studies mainly focus on improving graph convolutional operators, model architectures, and supervised objective functions, to leverage labels to learn node-level anomaly patterns [6, 12, 13, 14, 36, 37] . For example, the spatial GNN has been redesigned mainly in terms of message passing and aggregation mechanisms. In particular, considering that GNN-based fraud detectors fail to effectively identify fraudsters in disguise, CARE-GNN [12] uses a label-aware similarity metric and introduces reinforcement learning to aggregate selected neighbors with different relationships to counteract disguises. Concurrently, spectral GNNs, relate graphical anomalies to high-frequency spectral distributions. Tang et al. [6] analyzed anomalies for the first time from the perspective of the spectral spectrum of the graph and proposed the Beta wavelet GNN (BWGNN), which has spectrally and spatially localized band-pass filters to better capture anomalies. In addition, anomalies are usually associated with high-frequency components in the spectral representation of a graph. Therefore, GHRN [37] prunes inter-class edges by emphasizing the high-frequency components of the graph, which can effectively isolate the anomalous nodes and thus obtain better anomaly detection performance.In contrast, unsupervised GAD approaches do not require any labels for model training. Similar to unsupervised AD for image data, unsupervised GAD approaches employ several unsupervised learning techniques to learn anomaly patterns on graph data, including data reconstruction [4, 34, 38] , contrastive learning [15, 39, 40] , and other auxiliary objectives [16, 17, 41, 42, 93] . For example, DOMINANT [4] is a reconstruction-based approach that employs a graph convolution autoencoder to reconstruct both the adjacency and attribute matrices simultaneously, assessing node abnormality through a weighted sum of the reconstruction error terms. Similarly, ComGA [34] is a communityaware attribute GAD framework based on tailored GCNs to capture local, global, and structural anomalies. CoLA [15] , the first contrastive self-supervised learning framework for GAD, samples novel contrast instance pairs in an unsupervised manner, utilizing contrastive learning to capture local information. HCM-A [16] integrates both local and global contextual information, employs hop count prediction as a self-supervised task, and utilizes Bayesian learning to enhance anomaly identification. TAM [17] optimizes the proposed anomaly metric (affinity) on the truncated graph end-to-end, considering one-class homophily and local affinity. Nevertheless, all the above methods adhere to the conventional paradigm of "one model for one dataset".Although some GAD approaches [43, 44] can handle cross-domain scenarios, their requirement for high correlation (e.g., aligned node features) between source and target datasets limits their generalizability. Differing from existing methods, our proposed ARC is a "one-for-all" GAD model capable of identifying anomalies across target datasets from diverse domains, without the need for re-training or fine-tuning.In-Context Learning. In-context learning (ICL) can be effectively adapted to new tasks based on minimal in-context examples, providing a powerful generalization capability of large language models (LLMs) [45, 46, 47] . For example, Brown et al. [45] demonstrate the remarkable ability of language models to perform diverse tasks with minimal training examples. Through few-shot learning, these models exhibit robustness and adaptability across various domains, exhibiting their potential as general-purpose learners in natural language processing (NLP). Further, given that pre-training objectives are not specifically optimized for ICL [94] , Min et al. introduced MetaICL [95] as a solution to bridge the divide between pre-training and downstream ICL utilization. This method involves continuously training the pre-trained LLMs across diverse tasks using demonstration examples, thereby enhancing its few-shot capabilities. In contrast, supervised in-context fine-tuning [94] suggests constructing self-supervised training data aligned with the ICL format to utilize the original corpora for warm-up in downstream tasks. They converted raw text into input-output pairs and investigated four self-supervised objectives, such as masked token prediction and classification tasks.ICL has also generated research attention in the field of computer vision (CV), where it has been widely used in vision tasks by designing specialized discretization tokens as prompts [18, 48, 49, 50] . For instance, Chen et al. [48] proposed to use a unified interface to represent the output of each task as a sequence of discrete tokens, the neural network can be trained for a variety of tasks using a single model architecture and loss function, thus eliminating the need for customization for specific tasks. Furthermore, given a shot prompt as a task description, the sequence output adapts to the prompt to generate task-specific results. Differently, Amir et al. [96] proposes an innovative method for in-context visual prompting by framing a wide range of vision tasks as grid in-painting problems. This approach leverages image in-painting to generate visual prompts, guiding models to complete various vision tasks by filling in missing parts of an image based on contextual information, thereby enhancing adaptability and performance across different visual challenges.Very recently, few studies applied ICL to graph learning and GNNs. As an initial attempt to use ICL for GNNs, PRODIGY [51] leverages a prompt graph-based framework to conduct few-shot ICL for node-level classification and edge-level prediction tasks. Further, UniLP [52] introduces ICL to resolve conflicting connectivity patterns caused by distributional differences between different graphs. Nevertheless, these methods require context samples in multiple classes for ICL during inference. Experimental Setup. To verify whether smoothness can indicate the contribution of features to GAD, we conducted motivated experiments. We consider three mainstream GAD methods: DOMINANT [4] , CoLA [15] , and TAM [17] . For each method, we conduct the experiments with hyper-parameters reported in the original article across all datasets. At the data pre-processing phase, we first calculate s k based on the raw features and sort them in descending order. According to the new order, we divide the features into 5 groups of feature subsets, denoted by the percentile of 80%-100%, 60%-80%, • • • , 0%-20%. Subsequently, the five feature subsets are sequentially used as inputs for different methods, and the methods are trained to obtain their AUROC values. We statistic the results of 5 random experiments for each method and reported the average AUROC.Results. Additional experimental results are shown in Fig. 8 . As we can witness in the figure, in most cases, features with lower s k are observed to be more helpful in distinguishing anomalies on most of the datasets. This consistent model-independent trend indicates the effectiveness of s k as an indicator of the contribution of each feature in GAD.In this section, we discuss the connections and differences between the ego-neighbor residual graph encoder in ARC (R-ENC for short) and the existing GNNs and GAD methods. R-ENC v.s. Radar [10] . Radar, one of the representative shallow GAD methods, also leverages a residual-based mechanism for anomaly detection. Our proposed R-ENC differs from Radar in the following key aspects:• Radar calculates the residual on the original feature space, which inevitably suffers from high complexity and less representative capability on high-dimensional graph data. In contrast, R-ENC computes residuals based on the output embeddings of a GNN, which enables more efficient processing and enhances the representative power of the model. This approach allows R-ENC to better capture the underlying structure and anomalies within the graph data.• In Radar, the summation of residual R directly serves as the anomaly score for each node, where the irrelevant information within several residual entries may hinder the model's accuracy and effectiveness. Differently, in ARC, we specifically use R-ENC to generate the embedding of each node, and employ another learnable scoring module to calculate the anomaly score based on these embeddings. In this case, our method can effectively filter out noise and irrelevant features, allowing the scoring module to focus on the most informative aspects of the node embeddings.• Radar employs the Laplacian matrix to calculate the residual, meaning that only first-order residual information is considered. In contrast, our R-ENC incorporates multi-hop residual aggregation, enhancing its ability to detect subtle anomalies by considering both local and global graph structures through high-order residuals.R-ENC v.s. ResGCN [97] . ResGCN is a GNN-based GAD approach with a residual-based mechanism. Similar to Radar, ResGCN also uses the summation of residual R as the anomaly score. However, the inclusion of irrelevant information within several residual entries can impair the model's accuracy and effectiveness. Moreover, ResGCN employs a two-branch design, with the node representation by GCN and residual information by MLP calculated by two different network modules.Compared to R-ENC with simpler designs, ResGCN has lower operational efficiency.R-ENC v.s. CoLA [15] . CoLA learns the anomaly patterns by maximizing the agreement between the embedding of each node and its neighboring nodes. Specifically, CoLA employs a bilinear module to compute the agreement score. Such an agreement can also be modeled by the first-order residual in R-ENC, which is computed by the difference between the ego embedding and the 1-hop aggregated embeddings. Compared to CoLA, a significant advantage of R-ENC is its ability to capture not only first-order residuals but also high-order residuals. This makes R-ENC a more robust and comprehensive encoder for graph anomaly detection, addressing a wider range of anomaly patterns across different datasets.R-ENC v.s. TAM [17] . TAM utilizes local affinity, i.e., the feature-level or embedding-level similarity between a node and its neighbors, as the indicator of each node's abnormality. R-ENC, with its residual operation, can also capture local affinity. Specifically, by computing the first-order residual between an ego node and its 1-hop neighbors, the local affinity can be indicated by the negative summation of the first-order residuals since they are highly correlated. Again, R-ENC can not only capture the first-order affinity but also the high-order affinity with the multi-hop residual operator.R-ENC v.s. Heterophily-aware GNNs [37] . Existing studies indicate that a key solution to handle the GAD problem is to enable heterophily-aware graph convolutional operation with high-pass filtering [37] . A feasible filter is graph Laplacian, whose normalized and self-loop added version can be written by L = I -Ã. For R-ENC, its first-order residual can also be viewed as a Laplacianbased graph convolution. Specifically, if we simplify the MLP-based feature transformation as a weight matrix W, the ego information can be written by Z [0] = XW, while the representation Z [1] = ÃXW. Then, the first-order residual can be written by: R [1] = Z [1] -EQUATION)That is to say, the first-order residual in R-ENC can be regarded as Laplacian-based high-pass filtering (note that the negative sign can be fused into the learnable weight W). Such a nice property enables ARC to capture high-frequency graph signals and heterophily information through residual-based embeddings, thereby enhancing its capability to detect anomalies in diverse and complex graph structures.Dataset with single-class normal. In this type of dataset, the normal samples share the same pattern or characteristics. For example, in a network traffic monitoring system dataset, normal behavior might be defined by regular patterns of data packets exchanged between a specific set of IP addresses. Any deviation from this single, well-defined pattern, such as an unexpected spike in data volume or communication with unknown IP addresses, can be flagged as anomalous.Dataset with multi-class normal. In this type of dataset, the normal samples are divided into multiple classes, each with distinct patterns or characteristics. For example, in a corporate email communication network dataset, normal data might be defined by regular patterns of email exchanges within specific departments, such as HR, IT, and Finance. Any deviation from these well-defined patterns, such as a sudden spike in emails between normally unconnected departments or an unusual volume of emails from an individual employee to external addresses, can be detected as anomalous.In this subsection, we first introduce the basic definition of one-class classification (OC) model, and then discuss the connection between one-class classification and cross-attentive in-context anomaly scoring module (C-AS for short).One-class classification. The core idea of OC model is to measure the abnormality of each sample according to the distance between its representation h and a center representation c [22] . Here c can be a fixed random representation vector or dynamically adjusted as the mean of all samples' representation vectors. Formally, the anomaly score by OC model can be written by:EQUATION)where ϕ(•) is a neural network model, as defined in Deep SVDD [22] . Intuitively, a normal sample tends to have a similar representation to the majority of samples, and hence the distance between its representation and c should be closer.C-AS as an OC model. In C-AS, we use a cross-attention block to calculate the weighted sum of context embeddings H k into Hqi for a query node q i . For an initialized model, we assume that the parameters W q and W k are random enough, making Q and K become uniform noise matrices.In this case, each entry in the attention matrix T = Softmax QK ⊤ √ de can be 1 n k , indicating that the attention matrix assigns uniform weights to all context nodes for all query nodes. Then, all the reconstructed query embeddings are equal to the average embedding of context nodes:EQUATIONSince the average context embedding is the center embedding of a group of few-shot normal samples, we can naturally define the center embedding c = 1 n k 1 T H k . Recalling that we define the anomaly score as the L2 distance between Hqi and H qi , then for all query nodes, the anomaly scoring can be rewritten by:EQUATION)That is to say, the C-AS module serves as an OC model under random initialization. Note that in practice, the attention matrix cannot be so ideal, but it can still assign relevantly average weights for the context embeddings. Such merit ensures that ARC can perform like an OC model, effectively detecting anomalies in the case of single-class normal (Fig. 4 (a)) even without costly training.C-AS goes beyond OC model. Thanks to its inherent mechanism, the OC model can effectively handle single-class normal scenarios. However, in the case of multi-class normals (Fig. 4 (b)), a single center is not sufficient to model multiple normal class centers. Unlike the OC model, C-AS can address this issue through cross-attention. Specifically, the cross-attention block learns to reconstruct a query embedding by assigning higher weights to several (but not all) context embeddings that are close to the query node. This way, for a query node, the cross-attention block can automatically learn the center of its corresponding normal class, rather than simply using the average context embedding. The awareness of multiple normal classes ensures that ARC can handle both single-class and multi-class normal cases. [l] via Eq. ( 4) [L] ] via Eq. ( 4)(i) ∈ T train do 7 Obtain X (i) ′ , E (i) , V (i) , y (i) from D (i) 8 for l = 1 : L do 9 Z (i),[l] ← Propagate and transform X (i) ′ = X (i),[0] via Eq. (3) 10 R (i),[l] ← Calculate residual of Z (i),11 end 12 H (i) ← Concatenate [R (i),[1] || • • • ||R (i),13 H (i) q , H (i)k ← Randomly split query and context node sets and indexing from H (i) 14EQUATIONq via Eq. ( 6)16Update model parameters via gradient descent. The algorithmic description of the feature alignment in ARC, the training process of ARC, and inference process of ARC are summarized in Algo. 1, Algo. 2, and Algo. 3, respectively.In the testing phase, the time complexity consists of two main components: feature alignment and model inference. For feature alignment, the overall complexity isO(ndd u + d u m + d u log(d u )),where m = |E| is the number of edges. Here, the first term is used for feature projection, while the second and third terms are used for smoothness computation and feature reordering, respectively. The model inference is divided into two main parts: embedding generation and anomaly scoring. The complexity of node embedding generation is O(L(md u + nd u h + nh 2 )), where the first term is used for feature propagation and the rest of the terms are used for residual encoding by MLP. The anomaly scoring, on the other hand, mainly involves cross-attention computation with time complexity of O(n q n k h + n q h), where n q is the number of query nodes and n k is the number of context nodes. [1] || • • • ||R [L] ] via Eq. ( 4) 8 H q , H k ← Separate query and context node sets and indexing from H 9 Hq ← Calculate cross attention from H q , H k via Eq. ( 5) 10 d ← Computing the L2 distance between Hq and H q 11 Return d as the anomaly scores f (•) for query nodes G via Algo. 1 2 Obtain X ′ , E, V from G 3 for l = 1 : L do 4 Z [l] ← Propagate and transform X (i) ′ = X [0] via Eq. (3) 5 R [l] ← Calculate residual of Z [l] via Eq. (4) 6 end 7 H ← Concatenate [RIn total, we considered 12 benchmark datasets. We divide the datasets into 4 groups: ❶ citation network with injected anomalies, ❷ social network with injected anomalies, ❸ social network with real anomalies, and ❹ co-review network with real anomalies. Within each type, we consider the largest dataset as one of the training datasets, and the rest datasets as the testing datasets. The detailed statistics of the datasets are shown in Table 3 . These datasets are selected from different domains and with injected or real anomalies to ensure that our proposed ARC model learns extensive anomaly patterns. The diversity of the above data can maximally ensure that ARC can effectively adapt to new and unseen graphs. Specifically, the detailed descriptions for the datasets are given as follows:• Cora, CiteSeer, PubMed [98] , and ACM [99] are four citation networks. In these datasets, nodes represent scientific publications, while edges denote the citation links between them. Each publication is characterized by a bag-of-words representation for its node attribute vector, with the dimensionality determined by the size of the respective dictionary.• BlogCatalog and Flickr [4, 100] stand as typical social blog directories, facilitating user connections through following relationships. Each user is depicted as a node, with inter-node links symbolizing mutual following. Node attributes encompass the personalized textual content generated by users within social network, such as blog posts or shared photos with tag descriptions.• Amazon and YelpChi [101, 102] are datasets about the relationship between users and reviews. Amazon is designed to identify users paid to write fake reviews for products, and three different graph datasets are derived from Amazon using different types of relations to construct adjacency matrix [17, 103] . YelpChi aims to identify anomalous reviews on Yelp.com that unfairly promote or demote products or businesses. Based on [101, 104] , three different graph datasets derived from Yelp using different connections in user, product review text, and time. In this work, we focus on Amazon-UPU (users who have reviewed at least one of the same product) and YelpChi-RUR (reviews posted by the same user). • Facebook [105] is a social network in which users can build relationships with others and share their friends. • Reddit [106] serves as a forum posts network sourced from the social media platform Reddit, where users labeled as banned are identified as anomalies. Textual content from posts is transformed into vectors to serve as node attributes. • Weibo [106] dataset encompasses a graph of users and their associated hashtags from the Tencent Weibo platform. Within a defined temporal window (e.g., 60 seconds), consecutive posts by a user are labeled as potentially suspicious behavior. Users engaging in a minimum of five such instances are classified as "suspicious". The raw feature vector includes the location of a micro-blog post and bag-of-words features. • Questions [107] dataset originates from Yandex Q, a platform dedicated to questionanswering. Users represent the nodes, while the connections between them signify the presence or absence of a question-and-answer interaction within a one-year timeframe. The node features are derived from the average of the FastText embeddings of the words in the user description, with an additional binary feature indicating users without description.Anomaly Injection. For the datasets with injected anomalies, we use the strategy introduced in [4, 15] to inject anomalous nodes. Specifically, we inject a set of anomaly combinations for each dataset by perturbing the topology and node attributes, respectively [108] . In terms of structural perturbation, this is done by generating small cliques of otherwise unrelated nodes as anomalies. The intuition for this strategy is that small cliques in the real world are a typical substructure of anomalies, with much more closely linked within the cliques than the mean [109] . Thus for a dataset, we can specify the size of the cliques (i.e., the number of nodes) p and its amount q for anomaly generation. Specifically, randomly sample p nodes from the graph making them fully connected and labeled as anomaly nodes. We iteratively repeat the above process q times to inject a total of p × q anomalies. Finally, we control the number of injected anomalies according to the size of the dataset. In particular, we fix p = 15 and q = 10, 15, 20, 5, 5, 20 on BlogCatalog, Flickr, ACM, Cora, Citeseer, and Pubmed, respectively. On the other hand, for attribute perturbations, we base the schema introduced by [110] . Specifically, for each perturbation target node v i , k nodes are randomly sampled in the graph and their distance from the target node is computed. Then, the node v j with the largest deviation from the target node v i is selected, and the attribute X i of the node v i to X j . We set the number of anomalies of the attribute perturbation to p × q to maintain the balance of different anomalies. In addition, we set k = 50 to ensure that the perturbation magnitude is large enough.In our evaluation, we provide a comprehensive comparsion of ARC with various supervised and unsupervised GAD methods. On the supervised side, two classic GNNs are included as well as 3 state-of-the-art (SOTA) models specifically tailored for the GAD task. For supervised models, it is assumed that the labels of both normal and abnormal nodes can be used for model training. Therefore, the main binary classification task is used to identify the anomalies:• GCN [61] , as a seminal model in the field of GNN, is known for its ability to process graph-structured data using neighborhood aggregation, facilitating efficient node feature extraction and representation learning. • GAT [62] incorporates the attention mechanism into the GNN framework to achieve dynamic weighting of node contributions. It optimizes its attention according to different downstream tasks to achieve high-quality node representations. • BGNN [67] is a GNN that combines gradient boost decision trees (GBDT) with GNN for graphs with tabular node features. It utilizes the GBDT to handle heterogeneous features while the GNN considers the graph structure and significantly improves performance on a variety of graphs with tabular features.• BWGNN [6] has spectral and spatial localized band-pass filters to better handle the "rightshift" phenomenon in anomalies, i.e., the distribution of spectral energy is concentrated at high frequencies rather than at low frequencies.• GHRN [37] is a heterophily-aware supervised GAD method based on graph spectra. By emphasizing the high-frequency components of the graph, the method can effectively cut down inter-class edges, thus improving the overall performance of anomaly detection.For the unsupervised alternative, we consider 4 representative SOTA GAD methods, each of them belonging to a sub-type: data reconstruction, contrastive learning, hop-based auxiliary goal, or affinity-based auxiliary goal:• DOMINANT [4] combines GCN and deep auto-encoder, and its learning objective is to reconstruct the adjacency matrix and node features jointly. It aims to identify structural and attribute anomalies based on reconstruction errors.• CoLA [15] is a contrastive self-supervised learning for anomaly detection on graphs with node attributes. The framework captures the relationship between each node and its neighborhood substructure in an unsupervised manner by sampling novel pairs of contrasting instances and leveraging the local information of the graph.• HCM-A [16] uses hop-count prediction as a self-supervised task to better identify anomalies by modeling both local and global context information. In addition, HCM-A designs two new anomaly scores and introduces Bayesian learning to train the model to capture anomalies.• TAM [17] is designed based on one-class homophily and local affinity. The learning target of TAM is to optimize the proposed anomaly metric (i.e. affinity) end-to-end on the truncated adjacency matrix.Hyper-parameters. We select some key hyper-parameters of ARC through random search within specified grids. Specifically, the random search was performed within the following search space:• Hidden layer dimension: {64, 128, 256, 512, 1024}• Number of MLP layers: {1, 2, 3, 4}• Propagation iteration: {1, 2, 3, 4, 5}• Dropout rate: {0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8}• Learning rate: floats between 10 -5 and 10 -2 • Weight decay: floats between 10 -6 and 10 -3Implementation Pipeline. We employ a fixed set of hyper-parameters to build a generalist GAD model for all datasets. First, we train all the methods (including baselines and ARC) on the training set T train with full labels. Then, the methods are evaluated on each dataset from T test respectively.For feature projection, we employ the PCA algorithm to map the raw features into a fixed space with d u = 64. When the original feature dimension is smaller than the predefined projection dimension d u , we use a random projection (e.g., Gaussian random projection) to upscale the feature into a higher dimensionality and then unify the dimensions into d u with the projection strategy. For the baselines that require fine-tuning, we further conduct dataset-specific tuning at this stage.Metrics. Following [7, 17, 68] , we employ two popular and complementary evaluation metrics for evaluation, including area under the receiver operating characteristic Curve (AUROC) and area under the precision-recall curve (AUPRC). A higher AUROC/AUPRC value indicates better performance. We report the average AUROC/AUPRC with standard deviations across 5 trials.Computing Infrastructures. We implemented the proposed ARC using PyTorch 2.1.2, PyTorch Geometric (PyG) 2.3.1, and DGL 0.9.0. All experiments were performed on a Linux server with an Inter Xeon microprocessor E-2288G CPU and a Quadro RTX 6000 GPU.In terms of AUPRC, Table 4 gives comprehensive comparative results with consistent observations with the AUROC results. Specifically, we have the following observations. ❶ ARC still demonstrates strong anomaly detection in generalist GAD scenarios without any fine-tuning. Specifically, ARC achieves state-of-the-art performance on five of the eight datasets and demonstrates competitive performance on the remaining datasets. On several datasets, ARC showed significant improvement over the best baseline (e.g., ↑ 131.1% on Cora, ↑ 98.8% on Citeseer). ❷ GAD methods that only pre-train specific to a dataset usually result in poor generalization to new datasets. Specifically, existing methods perform very erratically on different datasets, which can be attributed to capturing only specific anomaly patterns. ❸ Using dataset-specific fine-tuning, baseline methods can achieve better performance in most case. However, in some cases the improvement can be small or even negative, demonstrating the limitations of fine-tuning.For all test sets, we varied n k in the range of 2 to 100 and the results are shown in Fig. 5 and Fig. 9 . From the figure, we observe that in most cases the performance of ARC increases with the involvement of more context nodes, which indicates its ability to utilize these labeled normal nodes for context learning. Moreover, even if n k is very small, ARC can still perform well on most datasets.To assess the effectiveness of the key design in the ARC, we conducted an ablation study with three variants of the ARC, 1) w/o A: using random projection to replace smoothness-based feature alignment; 2) w/o R: using GCN to replace ego-neighbor residual graph encoder; and 3) w/o C: using binary classification-based predictor and loss to replace cross-attentive in-context anomaly scoring.As can be seen from Table 5 and Table 6 , the two metrics AUROC and AUPRC of ARC achieved the best in all datasets except Weibo dataset. A possible explanation is that the Weibo dataset exhibits a particular pattern of anomalies. In addition, all three key designs provide significant improvements in performance.The attention weights between context nodes and query nodes in other datasets are shown in Fig. 10 .As can be seen in Fig. 10 , for most of the datasets, the "single-class normal" case in Fig. 4 (a) is met: ARC tends to assign a uniform attention weight to normal nodes. This results in reconstructed embedding that are very similar to the average embedding of context nodes; in contrast, reconstructing anomalies using a combination of a few context nodes results in their embedding being farther from the center. Moreover, corresponding to the "multi-class normal" case in Fig. 4 (b): in Fig. 10 (f), it is observed that each normal query nodes that follow two fixed patterns. In summary, the crossattention module enables ARC to adapt to various normal/abnormal distribution patterns, conferring it generality. Guidelines:Normal Anomaly Query Nodes• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.Specific definitions see Appendix D.1 


Decision: This paper proposes a graph anomaly detection approach that identifies anomalies on-the-fly using only few-shot normal nodes. The reviewers appreciated the clear presentation, noting the high originality and generality of the proposed approach, as well as the extensive experiments conducted. Given that several reviewers championed the paper, I recommend acceptance.

Reviews: review: paper_summary: paper_summary: The paper introduces an ARC method designed to detect anomalies across diverse graph datasets without the need for dataset-specific training. ARC leverages in-context learning with few-shot normal samples during inference, comprising three main components: a smoothness-based feature alignment module, an ego-neighbor residual graph encoder, and a cross-attentive in-context anomaly scoring module. The model demonstrates superior performance, efficiency, and generalizability compared to existing GAD methods, as validated by extensive experiments on multiple benchmark datasets.
strength_weakness: strength_weakness: 1. The paper's claim that it is "capable of detecting abnormal nodes across diverse graph datasets from various application domains without any training on the specific target data" is a bit ambitious. The method section did not provide a solid theoretical analysis to support this claim.

2.  The statement in Line 102 should be moved to the introduction section to declare the scope of this work.

3. The datasets used in the experiments are somewhat similar. It is recommended to include more diverse datasets from different backgrounds.

4. The model relies on the availability of a few-shot normal samples for in-context learning. In some practical situations, obtaining these samples may be challenging, particularly in highly anomalous or unknown environments.
questions: questions: see the weaknesses above
limitations: limitations: yes
paper_summary: paper_summary: This paper proposed AARC, a generalist GAD approach to detect anomalies across various graph datasets on the fly. It consists of feature alignment module, residual graph encoder and in-context anomaly scoring module.
strength_weakness: strength_weakness: 1. The smoothness-based feature sorting is done in the raw input feature. In this case, after unify the feature dimensionality with the feature projection, how do you smooth the new feature $\hat{X}^{(i)}$ with smoothness-based feature sorting? In another word, how do you aligns dimensionality and smooth feature at the same time?
2. In line 232 and 233, the authors mention that "In the first two steps, we perform propagation on the aligned feature matrix $X'=X^{[0]}$ for $L$ iterations, and then conduct transformation on the raw and propagated features with a shared MLP network". This is confusing. It's unclear weather the input feature is aligned feature matrix or the raw feature. Besides, the aligned feature matrix should be $\tilde{X}$ rather than $X$.
3. Equation 3 can be written as $Z^{[l]}=MLP(\tilde{A}X^{[l-1]})=\sigma(\tilde{A}X^{[l-1]}W)$, where $W$ is the weight matrix of MLP and $\sigma$ is the activation function. In this case, what's the difference between equation 3 and traditional GNN. The claim of equation 3 to capture the high-frequency signals and local heterophily is not convincing.
4. The design of cross-attentive in-context anomaly scoring is based on the assumption that the normal query have similar patterns to several context nodes and their embeddings can be easily represented by linear combination of context node embeddings. However, in many cases, given the limited number of  normal nodes, especially for the test graph, it's highly likely that some unsampled normal nodes cannot be represented by linear combination of other normal nodes. For instance, the sampled normal nodes from class 1 to class 5, while there exist some unsampled normal nodes from class 6. In this case, these unsampled normal nodes may not be represented by the linear combination of other normal nodes and these nodes tend to have large reconstruction error, thus being labeled as the abnormal nodes. How do you mitigate this issue?
questions: questions: 1. The smoothness-based feature sorting is done in the raw input feature. In this case, after unifying the feature dimensionality with the feature projection, how do you smooth the new feature $\tilde{X}^{(i)}$ with smoothness-based feature sorting? In another word, how do you aligns feature dimensionality and smooth feature at the same time?
2. The design of cross-attentive in-context anomaly scoring is based on the assumption that the normal query have similar patterns to several context nodes and their embeddings can be easily represented by linear combination of context node embeddings. However, in many cases, given the limited number of  normal nodes, especially for the test graph, it's highly likely that some unsampled normal nodes cannot be represented by linear combination of other normal nodes. For instance, the sampled normal nodes from class 1 to class 5, while there exist some unsampled normal nodes from class 6. In this case, these unsampled normal nodes may not be represented by the linear combination of other normal nodes and these nodes tend to have large reconstruction error, thus being labeled as the abnormal nodes. How do you mitigate this issue?
3. For the supervised baseline methods, what is the available label information for the test graphs. Based on the setting in the paper, it seems that only a small percentage of normal nodes are available for training for test graphs. In this case, how do you train these supervised baseline methods? What is the training procedure for these baseline methods?
limitations: limitations: The authors list limitations in conclusion.
paper_summary: paper_summary: This paper investigates the research problem of "generalist graph anomaly detection (GAD)", targeting to address the cost and generalizability issues of the conventional GAD paradigm. This paper proposes ARC, a "one-for-all" GAD model that is pre-trained on a group of datasets and able to detect anomalies on new datasets on-the-fly. To verify the performance of ARC, experiments on GAD benchmark datasets are executed. The main contributions of this paper are: 1. Addressing a new research problem (generalist GAD); 2. Developing a novel method; 3. Conducting experiments for GAD performance validation.
strength_weakness: strength_weakness: Methodology details: Some specific details of the method design are not clearly given. In specific: 

1. For the feature projection, what if the original feature dimension is smaller than the predefined projected dimension? 

2. Why doesn't ARC use the value-based cross attention? Is there any disadvantage?

Experiment details: Although the experiments are extensive, I think more discussions should be given for some notable results. In specific: 

1. Why unsupervised methods perform better than the supervised methods? 

2. Why Figure 5 (b) has a dropping trend at nk=20?
questions: questions: The questions listed in Weaknesses are expected to be answered. Meanwhile, I have some extra questions:

1. Why "pre-train & fine-tune" settings only include unsupervised methods? 

2. Will ARC be time-costly when the number of context nodes become large?
limitations: limitations: From my perspective, the authors have addressed the limitations and the paper seems doesn't have potential negative impact.
paper_summary: paper_summary: This paper proposes a new framework for graph anomaly detection called ARC. ARC uses in-context learning to detect anomalies across various graph datasets without requiring retraining or fine-tuning. It leverages few-shot normal samples during inference to achieve superior performance in anomaly detection tasks.
strength_weakness: strength_weakness: However, I have several concerns about this paper.

1. In the residual GNN component, a shared MLP is used for feature transformation, which is not vey commonly seen in other GNNs. Could you provide more intuition behind for explanation?

2. In attention mechanism, "value" is an important component in the computational process. However, the in-context learning module doesn't include the computation of "value". Can you give more detailed intuition for this design?
questions: questions: I still have some minor questions.

1. How to define the projected dimension $d_u$? 

2. Considering the cost of fine-tuning (figure 6), will it require a lot of epochs to get the fine-tuned results for the baseline methods?
limitations: limitations: The limitations are upfront in the paper.
paper_summary: paper_summary: The paper produces a "one-for-all" generalist graph anomaly detection model. The proposed model first use feature projection and smoothness-based ranking to align the features of multiple datasets; then it employs a residual graph neural network to extract representations; finally, a cross-attention module is used to calculate the anomaly scores of query nodes. The paper is well motivated and with sufficient experiments for effectiveness validation.
strength_weakness: strength_weakness: 1. The methodology details of the proposed method are not precise: specifically, which type of feature projection method is used in the experiment should be given.
2. The definitions of the concept of single-class normal and multi-class normal are missing, which hinders the readers to understand the design motivation in Sec. 4.3. 
3. The figure is not readable enough. More annotations can be provided in Figure 2, especially for the R and C blocks.
questions: questions: 1. In Eq. 6, the anomaly label is denoted by a bold letter y. Does that mean it's a vector? However, should it be a value actually?
2. What is the reason for the unstable anomaly detection performance under different number of context nodes?
limitations: limitations: I believe the authors have addressed the limitation, and I haven't found any negative societal impact of this work.

Comments: title: Reply to Authors' response
Reply to the further questions raised by Reviewer HCAf
Reply to authors' rebuttal
comment: Thanks for the rebuttal. After reading all the comments, I decided to keep my rating of accept.
Finally, we express our sincere appreciation for your thorough review and insightful questions. We want to emphasize that this paper introduces a new paradigm in graph anomaly detection (GAD), known as generalist GAD, which presents a greater challenge than conventional GAD problems. Additionally, the proposed method is fundamentally different from traditional GAD models, incorporating well-crafted techniques for data alignment and in-context learning. We hope that this frontier discovery of “one-for-all” GAD models will inspire the development of innovative and more generalized solutions within the community.

We kindly ask you to review our explanations with an open mind, considering how our method diverges from conventional GAD approaches. Your expertise is invaluable, and we would be grateful if you could reassess our work in light of these clarifications.

Given this new perspective, we respectfully request that you consider providing a fair evaluation that reflects the true nature and potential impact of our research. Your impartial assessment would be greatly appreciated.

Thank you for your time and consideration.
We genuinely appreciate your thoughtful questions. Before we address your subsequent questions, could you confirm if our response has addressed your concerns regarding **Q1-Q7**? We want to ensure that all your concerns have been adequately addressed before we discuss any new questions. If there are any aspects you feel need further clarification or improvement, please do not hesitate to let us know.

The responses to your new questions are given as follows.

**Q8: Generalization capability of ARC**

**A8:** From a theoretical perspective, ARC is equipped with an in-context learning (ICL) technique, which enhances its generalization capability on new datasets. Specifically, the ICL mechanism enables ARC to distinguish between normal and abnormal samples by using a set of normal samples from each given testing dataset, rather than fitting to the anomaly patterns of a specific training dataset. Following the assumption that normal samples are more similar to each other than anomalies, the cross-attention-based ICL module can learn to make accurate predictions by leveraging inter-sample similarity on new datasets.

From an experimental perspective, we have made every effort to demonstrate the generalizability of ARC through experiments on datasets from a wide range of domains. Specifically, in the original manuscript, we included 8 datasets from 3 domains: **citation network, social network, and co-review network**. During the review process, we further expanded our experiments to cover 4 additional domains: **co-author network, co-purchase network, transaction record network, and work collaboration network**. It is important to note that our model had **never seen** data from these 4 new domains during training, providing strong empirical evidence of ARC’s generalization capabilities on diverse and unseen tasks. To the best of our knowledge, we have covered datasets from most of the mainstream domains in graph anomaly detection (GAD), including **4 datasets for training and 12 datasets for testing**, which exceeds the evaluation scope of the vast majority of GAD studies.

**Q9: Results discussion of experiments of pseudo normal trick**

**A9:** Regarding your question about the performance of ARC with a limited number of pseudo samples, we would like to clarify the following points:
**Optimal number of pseudo samples.** The statement “Why does the performance of ARC achieve the best performance with limited number of pseudo samples (e.g., #Pseudo Normal = 2) on some datasets, such as Amazon, Facebook and Weibo?” is not accurate. As mentioned in our response to Reviewer RwhU, ARC achieves its best performance on these datasets with #Pseudo Normal = 10. This is because, in real-world scenarios, the number of normal samples typically far exceeds the number of anomalies. Therefore, with a sample size of 10, the probability of selecting true normal samples is higher, which is why we set the default sample size to 10 in all our experiments. This demonstrates that ARC can achieve competitive performance with a small number of shot samples.

**Impact of sample size on performance.** The statement "the advantage of having a larger number of normal samples reduces the impact of anomalies within the pseudo normal samples" was intended to illustrate the intuitive feasibility of the "pseudo normal trick." However, this does not imply that increasing the number of pseudo normal samples will always enhance ARC’s performance. On the contrary, when the sample size is very small (e.g., $n_k$=2 or 10), the probability of selecting true normal samples is higher. As the sample size increases, the likelihood of including false normal samples also increases (especially when the anomaly ratio of the dataset is relevantly large), introducing more noisy psuedo samples and potentially degrading performance. To sum up, the performance of ARC with the "pseudo normal trick" is influenced by the "purity" of the sampled pseudo examples, which in turn depends heavily on the anomaly ratio of the dataset and experimental randomness. Therefore, it is reasonable that different datasets exhibit varying preferences for the pseudo sample size. 

Last but not least, we'd like to clarify that **the exploration of the pseudo-label trick is merely an additional exploratory experiment**. We aimed to investigate the possibility of applying ARC to a purely unsupervised (zero-shot) scenario. Surprisingly, the results show that the performance is still acceptable, indicating ARC's potential. However, our main focus remains on the few-shot setting, where reliable content samples are available for ICL. We hope this experiment demonstrates the potential of ARC and inspires future work in this direction, but it is not the main contribution of this paper.
Dear Authors,
Thanks for the detailed responses, which fully addresses my previous concerns. I decide to maintain my positive scoring for this paper.
Thanks for the detailed reply. I am still concerned about the generalization of the proposed method as the generalization capability is the major contribution in this paper. I agree with the comment by reviewer **RwhU** that "The method section did not provide a solid theoretical analysis to support this claim." when dealing with the new dataset. In addition, the experimental results are not sufficient to demonstrate the capability of the proposed method on the unseen tasks.

In addition, I have one more question regarding your response to Reviewer **RwhU** about the results of few-shot samples. Why does the performance of ARC achieve the best performance with limited number of pseudo samples (e.g., 
#Pseudo Normal = 2) on some datasets, such as Amazon, Facebook and Weibo? As you mention during the rebuttal that "the advantage of having a larger number of normal samples reduces the impact of anomalies within the pseudo normal samples.", while the performance of ARC tends to decrease on these datasets when the number of pseudo normal samples increases. Do you have any explanation or thoughts to this observation?
Thanks for the authors' rebuttal, which addresses all my concerns. I would like to raise my rating in response to the authors' efforts during the rebuttal.
Dear Reviewer RwhU,

We sincerely appreciate your time and expertise in reviewing our submission. Acknowledging the demands on your schedule, we are mindful not to intrude on your time. However, we would be grateful if you confirm that our rebuttal adequately addresses your concerns.

Thank you in advance for your consideration.

Authors
We are grateful to Reviewer HCAf for the valuable feedback. 
Please find our responses to your new questions below. We hope that our response addresses your concerns.

**Q6: Impact of unseen normal class problem**

**A6:** We appreciate you once again for highlighting the unseen normal class problem. We agree that a deeper discussion on the impact of this issue is necessary. As shown in Fig. 5 and Fig. 9, the performance gaps between $n_k=2$ (where normal classes should not be fully covered) and $n_k=100$ (where normal classes should be covered) are generally within the range of $0.5$% - $5$% in terms of AUROC. This indicates that the unseen normal class problem may be a factor affecting the performance of ARC. However, we would like to note that the unseen normal class problem is not a fatal issue of ARC, since the performance when $n_k=2$ remains generally acceptable and won't collapse to indistinguishable prediction (e.g., AUROC=50% and AUPRC=0%). 

We want to express our appreciation to the reviewer once again for identifying the potential unseen normal class problem in ARC. We will definitely discuss the potential limitations caused by this issue in the revised manuscript and provide possible strategies (in A4) for users/readers to mitigate this problem.

**Q7: Codes of propagation in ARC**

**A7:** Thank you for your detailed observation regarding the implementation of the Ego-Neighbor Residual Graph Encoder in the ARC model.

The reason the adjacency matrix is not directly used in the forward function (Lines 21-39 in model.py) is due to our decoupled architecture in Eq.(3), i.e., the propagation is directly conducted on the features without learnable parameters. In this case, we can move the propagation step into the codes of data preprocessing to enhance the running efficiency (since we can only run it once). Specifically, the codes of propagation are in the "propagated" function (Lines 143-148 in utils.py), which is called in Lines 53-57 in main.py before the training of the encoder. Then, the propagated features (saved in Dataset.graph.x_list) are subsequently used in the forward function (Lines 21-39 in model.py) for downstream residual encoding. 

We appreciate your suggestion, and to prevent potential misunderstandings, we will incorporate more comments in the publicly released code for interpretation.
I thank the authors for their thoughtful rebuttal. Most of my concerns have been properly addressed, but some remain:
- For the answer to question 4, though the experimental results in Table 1 on the Cora dataset show that the proposed methods significantly outperform the other baseline methods, it can be also observed that some baseline methods can outperform ARC on other datasets, such as Facebook, Weibo. Specifically, TAM (Unsupervised - Pre-Train & Fine-Tune) outperforms ARC by around 7% on Facebook dataset. It's not a consistent observation throughout all evaluated datasets. Thus, I am not fully convinced by the statement that this issue does not affect the performance a lot.
- I look at the code of ARC. The forward function in ARC class (Line 21 -Line 39 in model.py file) corresponds to the implementation of Ego-Neighbor Residual Graph Encoder, I find that **the adjacency matrix is not used for representation learning, which is not consistent with Equation 3**.