ARC: A Generalist Graph Anomaly Detector with In-Context Learning
Authors: Yixin Liu
Shiyuan Li
Yu Zheng
Qingfeng Chen
Chengqi Zhang
Shirui Pan
Venue: NeurIPS 2024 poster (2024)
Keywords: Graph Anomaly Detection
Graph Neural Networks
In-Context Learning
Artificial general intelligence
Primary area: graph_neural_networks

Abstract: Graph anomaly detection (GAD), which aims to identify abnormal nodes that differ from the majority within a graph, has garnered significant attention. However, current GAD methods necessitate training specific to each dataset, resulting in high training costs, substantial data requirements, and limited generalizability when being applied to new datasets and domains. To address these limitations, this paper proposes ARC, a generalist GAD approach that enables a ``one-for-all'' GAD model to detect anomalies across various graph datasets on-the-fly. Equipped with in-context learning, ARC can directly extract dataset-specific patterns from the target dataset using few-shot normal samples at the inference stage, without the need for retraining or fine-tuning on the target dataset. ARC comprises three components that are well-crafted for capturing universal graph anomaly patterns: 1) smoothness-based feature **A**lignment module that unifies the features of different datasets into a common and anomaly-sensitive space; 2) ego-neighbor **R**esidual graph encoder that learns abnormality-related node embeddings; and 3) cross-attentive in-**C**ontext anomaly scoring module that predicts node abnormality by leveraging few-shot normal samples. Extensive experiments on multiple benchmark datasets from various domains demonstrate the superior anomaly detection performance, efficiency, and generalizability of ARC.

ARC: A Generalist Graph Anomaly Detector with In-Context Learning
Yixin Liu Shiyuan Li Yu Zheng**, Qingfeng C Chengqi Zhang', Shirui Pan of Griffith University, Guangxi University, La Trobe University, 'The Hong Kong Polytechnic University
yixin. lidgr.iiuggi.fithh.oo www.fffithh.......e.mmmmmm au, shiy li@alu.. gxu. edu. en, yu.zheng@lattrob.. edu qingfengōgxu. ewaeigguueeuu.coom cn, chengqi zangiiioaaan@plyyu edu. hk, s. s.pan@griffth. edu au
Abstract
Graph anomaly detection (GAD), which aims to identify abnormal nodes that differ from the majority within a graph, has garnered significant attention. However, current GAD methods necessitate training specific each dataset, resulting in high training costs, substantial data requirements, and limited generalizability when being applied to new datasets and domains. To address these limitations, this paper proposes ARC, generalist GAD approach that enables a "one-for-all" GAD model detect anomalies across various graph datasets on-the-fly. Equipped with in- context learning, ARC can directly extract dataset-specific patterns from the target dataset using few-shot normal samples the inference stage, without the need for retraining or fine-tuning on the target dataset. ARC comprises three components that are well-crafted for capturing universal graph anomaly patterns: smoothness- based feature Alignment module that unifies the features of different datasets into a common and anomaly- sensitive space; 2) ego-neighbor Residual graph encoder that learns abnormality-relan in- in-Context anomaly scoring module that predicts node abnormality by leveraging few-shot normal samples. Extensive experiments multiple benchmark datasets from various domains demonstrate superior anomaly detection performance, efficiency, and generalizability of ARC. The source code of ARC is available at https///ghtuu..ommm//xxm////2//
node embeddings; and 3) cross-attentive
Introduction
Graph anomaly detection (GAD) aims distinguish abnormal nodes that show significant dis- similarity from the majority of nodes in a graph. GAD has broad applications across various real-world scenarios, such fraud detection in financial transaction networks [ and rumor detec- tion in social networks (2). As result, GAD has attracted increasing research attention in recent Conventional GAD methods employ shallow mechanisms model node-level however, they face limitations in handling high nigh-dimensional features and com- plex interdependent relations on graphs. Recently, graph neural network (GNN)-based approaches have emerged as the go-to solution for the GAD problem due to their superior performance [4] 6]. Some GNN-based GAD approaches regard GAD as a supervised binary classification problem and specifically designed GNN architectures to capture anomaly patterns 12, [3
years 4 abnormality 9
5
8
10
[4]. Another line
of approaches targets the more challenging unsupervised paradigm, employing various unsupervised
learning objectives and frameworks to identify anomalies without relying labels 4 15
[6 17].
"Equal Contribution. Corresponding Authors.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Nerual Nade
Altucmal Nede
Unisheee Node
Training Stage
Inference Stage
Training Stage
Inference Stage
Taah
Sepervice GAD Mode
Modei
GAD
Irap
Generalia GAD
GAD Modid
(a) Supervised GAD Parndigm
Mood Model
Lnuperical GAD Model
GAD
Model
Unsupervised Paradigm
(6) Generraiist GAD Paradign (ours)
Figure 1: Sketch maps of (a) supervised, (b) unsupervised, and (c) generalist GAD paradigms.
Despite their remarkable detection performance, the existing GAD approaches follow "one model for one dataset" learning paradigm (as shown in Fig. (a) and (b)). necessitating dataset-specific training and ample training data to construct detection model for each dataset. This learning paradigm inherently comes with the following limitations: 1 Expensive training cost. For each dataset, we need to train specialized GAD model from scratch, which incurs significant costs for model training, especially when dealing with large-scale graphs. @ Data requirements. Training a reliable GAD model typically needs sufficient in-domain data, sometimes requiring labels well. The data requirements pose a challenge when applying GAD to scenarios with sparse data, data privacy concerns, or high label annotation costs. @ Poor generalizability. On new-coming dataset, existing GAD methods require hyperparameter tuning or even model architecture modifications to achieve optimal performance, which increases the cost of applying them to new data and domains.
Given the above limitations, a natural question arises: Can we train a "one-for-all" GAD model that can generalize to detect anomalies across various graph datasets from different application domains, without any training on the target data? Following the trend of artificial general intelligence and foundation models, new paradigm termed "generalist anomaly detection", originating from image anomaly detection, is a potential answer to this question [88. As shown in Fig.[ (c), in the generalist paradigm, we only need to train the GAD model once; afterward, the well-trained generalist GAD model can directly identify anomalies on diverse datasets, without any re-training or fine-tuning. Considering the diversity of graph data across different domains and datasets, the labels of few-shot normal nodes are required during the inference stage to enable the model to grasp the fundamental characteristics of the target dataset. Compared to conventional paradigms, the generalist paradigm climinates the need for dataset-specins training, resulting in fewer computations, lower data costs, and stronger generalizability when applying GAD models to new datasets.
Nevertheless, due to the unique characteristics of graph data and GAD problem, it is non-trivial to design a generalist GAD approach. The challenge is three-fold: C1 Feature alignment. Unlike image data, which are typically represented in a consistent RGB feature space, the feature dimensionality and semantic space can vary significantly across different graph datasets. Substituting features with unified representations generated by large language models may be potential solution [19]; however, this approach is limited to specific feature semantics and cannot address more general cases [20]. C2 Representation encoding. As the core of generalist GAD model, a GNN- -ased encoder is expected to learn dataset-agnostic and abnormality-awan node embeddings for anomaly detection. However, in the absence of universal pre-trained foundation models [8] for graph data, crafting a potent encoder for generalist GAD model presents a challenge. C3 Few-shot cmmple-guided prediction. Existing GAD methods typically focus on single dataset settings, where dataset-species knowledge embedded in the model through training, enabling it to predict abnormality for each node independently. In contrast, generalist GAD model should derive such knowledge from a small number of normal nodes. In this case, how to effectively utilize the few-shot normal samples during inference remains an open question.
To tackle these challenges. we introduce ARC, a generalist GAD approach based on in-context learning. ARC comprises three meticulously designed modules, each targeting a specific challenge. To address CI, we introduce smoothness- features across diverse datasets to a common dimensionality but also arranges them in an anomaly- sensitive order. To deal with C2, we design an ego-neighbor Residual graph encoder. Equipped with multi-hop residual- based aggregation scheme, the graph encoder learns attributes that indicate high-order affinity and heterophily, capturing informative and abnormality different datasets. Last but not least, to solve C3, we propose cross-attentio in in-Context anomaly scoring module. Following the in-context learning schema, we treat the few-shot normal nodes as context samples and utilize cross-attention block to reconstruct the embeddings of unlabeled
feature Aligument module, which not only standardizes
embeddings across
2

---

samples based on the context samples. Then, the reconstruction distance can serve as the anomaly score for each unlabeled node. In summary, this paper makes the following contributions:
Problem. We, for the first time, propose to investigate the generalist GAD problem, aiming to detect anomalies from various datasets with single GAD model, without dataset-specific fine-tuning. Methodology. We propose a novel generalist GAD method ARC, which can detect anomalies in new graph datasets on-the-fly via in-context learning based on few-shot normal samples.
Experiments. We conduct extensive experiments to validate the anomaly detection capability, generalizability, and efficiency ARC across multiple benchmark datasets from various domains.
2
Related Work
In this section, we offer a brief review of pertinent related works, with more extensive literature review available Appendix A
Anomaly Detection. Anomaly detection (AD) aims to identify anomalous samples that deviate from the majority of samples [2T]. Mainstream AD methods focus on unsupervised settings and employ
various unsupervised techniques to build the models 22) 23 24 25
26
27
28 29 To enhance
the generalizability of AD methods across diverse datasets, RegAD 30 considers few-shot setting and trains a single generalizable model capable of being applied to new in-domain data without re-training or fine-tuning. WinCLIP [3I] utilizes visual- visual-language models (VLMs, e.g., CLIP (32))
with well-crafted text prompts to perform zero/few-shot AD for image data. InCTRL [18] as the first generalist AD approach, integrates in-context learning and VLMMs to achieve domain-agnostic image AD with a single model. However, due to their heavy reliance on pre-trained vision Encoders/VIMMs and image-specific designs, these approaches excel in AD for image data but face challenges when applied to graph data.
Graph Anomaly Detection (GAD). this paper, we focus on the node-level AD on graphs and refer to it as "graph anomaly detection (GAD)" following 66 33 33). While shallow methods [9] I0 I1] show limitations in handling complex woal-world graphs 4. the advanced upproaches are mainly
based on GNNs [35] The GNN-bbsee approaches can be divided into supervised and unsupervised
Supervised GAD approaches assume that the labels of both normal and anomalous nodes are available for model training (7]. Hence, related studies mainly introduce
approaches
S 7
36 37]. In contrast, unsupervised GAD approaches do not require any labels for model training. They employ several unsupervised learning techniques to learn anomaly patterns on graph data, including data reconstruction [4] 34 38).
GAD methods a binary classification paradigm 62 [3
contrastive learning [5 39 40), and other auxiliary objectives [6
42). Nevertheless, all the above methods adhere to the conventional paradigm of "one model for one dataset".. Although some GAD approaches [43 44 handle cross- cross-domain scenarios, their requirement for high correlation (e.g., aligned node features) between source and target datasets limits their generalizability. Differing from existing methods, proposed ARC is "one-for-all" GAD model capable of identifying anomalies across target datasets from diverse domains, without the need for re-training or fine-tuning.
In- Learning (ICL.. ICL enables a well-trained model to be effectively fine-tuning-fres adapted to new domains, datasets, and tasks based on minimal in-context examples (a.k.a. context samples), providing powerful generalization capability of large language models (LLMs) [45 46 47
and computer vision (CV) models [T8 48 49 50]. Two recent approaches, PRODIGY [5]] and UniLP [52] attempt to use ICL for GNNs to solve the node classification and link prediction tasks, respectively. However, how to use ICL to deal with the generalist GAD problem where only normal context samples are available still remains open.
3 Problem Statement
Notations. Let G (V, E, X) be an attributed graph with n nodes and m edges, where V (uz,... , vn and E are the set of nodes and edges, respectively. The node-level attributes are included by feature matrix X E Rg where each row X; indicates the feature vector for node u;. The inter-node connectivity is represented by an adjacency matrix A E {0, 1jnxa, where the i, j-th entry Ag = 1 means e and y connected and vice versa.
3

---

Prop
Shared Waightt
Feature Projection
43 A E A E
Q
Z
R(AI
X
W
0.82
0.30
Smmottmess Based Feature Sarting
0.28
Prop
Shured
0.23
Woightt
0.21
MLP
R|a
Smoothness-Rase Feature Alignment
Ego-Neighhor Residual Graph Encoder
Cross- Attentive Cn-Conext Anomaly Scoring
Figure 2: The overall pipeline of ARC, the proposed generalist GAD approach.
Conventional GAD Problem. GAD aims differentiate abnormal nodes V. from normal nodes Vn within a given graph 9 = E X), where V. and Vn satisfy V. UV, = V. V. n V; 0, and [Val < /. An anomaly label vector
1]"h {0, can be used denote the abnormality of each 0 iff
node, where the i-th entry y; = iff v E Va and to learn an anomaly scoring function (i.e.. GAD model) f f + R such that f(u') f(v) for Ve' E V, and Vo E Vn. In the conventional GAD setting of "one model for one dataset", the GAD model optimized on the target graph dataset D (G, y) with subset of anomaly labels (in supervised setting) or without labels (in unsupervised setting). After sufficient training, the model can identify anomalies within the target graph during the inference phase.
Vn. Formally, the goal of GAD
Generalist GAD Problem. In this paper, we investigate the generalist GAD problem, wherein we aim develop a generalist GAD model capable of detecting abnormal nodes across diverse graph datasets from various application domains without any training on the specific target data. Formally, we define the generalist GAD setting, aligning it with its counterpart in image AD introduced by ((A) be a collection of training datasets, where each ((() (gl() (rrain Strain) is a labeled dataset from an arbitrary domain. We aim to train a generalist GAD model of on Trrain, and f is able to identify anomalies within any test graph dataset D(i) Trest where Trest (D
Zhu et al. 18]. Specifically, let Terain = (D(L) (rriin.
(8)
tests D ((N)) is collection testing datasets. Note that Terain O (Tesst = 0 and the datasets in Terain and Trest can be drawn from different distributions and domains. Following [18), we adopt a "normal few-shot" setting during inference: for each D(2) only a handful of n; normal nodes (ng < n) are available, and the model f is expected to predict the abnormality of the rest nodes without re- re-training and fine-tuning.
ARC: A generalist GAD approach
In this section, we introduce ARC, generalist GAD approach capable of identifying anomalies across diverse graph datasets without the need for specific fine-tuning. The overall pipeline of ARC is demonstrated Fig. Firstly, to align the features of different datasets, introduce smoothness- based feature alignment module (Sec. (.1]). which not only projects features onto common plane order. Next, to capture abnormality-awar
but also sorts the dimensions in an moomaly-sensitie node embeddings. we propose simple yet effective GNN model termed ego-neighbor residual graph encoder (Sec. 4.2). which constructs node embeddings by combining residual information between an ego node and its neighbors. Finally, to leverage knowledge from few-shot context samples for predicting node-level abnormality, we introduce cross-altentive in-context anomuly scoring module (Sec. 4.3) Using the cross-attention block, the model learns to reconstruct query node embeddings based on context node embeddings. Ultimately, the drift distance between the original and reconstructed query embeddings can quantify the abnormality of each node.

---

4.1 Ainoothness-Bsser Feature Alignment
Graph data from diverse domains typically have different features, characterized by differences in dimensionality and unique meanings for each dimension. For example, features in citation network usually consist of textual and meta-inforrmatio associated with each paper, whereas in a social network, the features may be the profile of each user. Therefore, in the first step, need to align the features into a shared feature space. To achieve this, we introduce the feature alignment module in ARC, consisting of two phases: feature projection, which aligns dimensionality, and smoothness- based feature sorting, which reorders features according to their smoothness characteristics.
Feature Projection. At the first step of ARC, we employ a feature projection block to unify the feature dimensionality multiple graph datasets [20]. Specifically, given feature matrix X R= x of D( TTrain Trest the feature projection is defined by linear mapping:
X ( E I xd= = Proj x (1) = ( (0) w(())
(1)
where (() is the projected feature matrix for D(O, du is a predefined projected dimension shared across all datasets, and ( Fd xdu is a datasct-specifie linear projection weight matrix. To
maintain generality. w can be defined using commonly used dimensionality reduction approaches
such as singular value decomposition [53] (SVD) and principal component analysis [54] (PCA).
Samoothnes--Bssee Feature Sorting, Although feature projection can align dimensionality, the semantic meaning of each projected feature across different datasets remains distinet. Considering the difficulty of semantic-level matching without prior knowledge and specific fine-tuning [19] (0).
in this paper, we explore an alternative pathway: aligning features based on their contribution to anomaly detection tasks. Through analytical and empirical studies, we pinpoint that the smoothness of each feature strongly correlated with its contribution to GAD. Building on this insight, in ARC, we propose to sort the features according to their contribution as our alignment strategy.
From the perspective of graph signal processing, Tang et al. [6] have demon- strated that the inverse of the low- frequency energy ratio monotonically in- creases with the anomaly degree. In other words, high-frequenne graph signals tend to play a more significant role in detect- ing anomalies. Similar findings have also been observed from the standpoint of spa- tial GNNs [37 551, where heterophily in- formation has been shown to be crucial in discriminating anomalies. Motivated by these findings, can we develop met- ric to gauge the contribution of each feature to GAD based on its frequencylheterophhiyy Considering
75
AUROC (%)
50
25-
high 5,
low 52
high 5p low 5g
Percentile w.r.t. Smoothness
Percentile wird. Smoothness
(a) Cora
(b) Facebook
Figure AUROC on data with 5 groups of features.
its correlation to frequency (56) and heterophily [57 58 59 60), in this paper, we select feature- level smoothness as the measure for contribution. Formally, given graph G (V, E, X) with normalized feature matrix X, the smoothness of the k-th feature dimension is defined as:
X
(x - X X₂x)²
E
where lower sæ indicates significant change in the k-th feature between connected nodes, implying that this feature corresponds to a high-frequency graph signal and cxhibits strong heterophily.
To verify whether smoothness can indicate the contribution of features in GAD, we further conduct empirical analysis (experimental setup and more results can be found in Appendix (B). Concretely, we sort the raw features of each dataset based on the smoothness 83 and divide them into 5 groups according to the percentile of 84. Then, we train different GAD models using each group of features separately, and the performance is shown in Fig. 3 and 8 On both datasets, model-agnostic observation is that the features with lower 8g are more helpful discriminating anomalies. The consistent trend demonstrates the effectiveness of sy an indicator of the role of features in GAD.
5

---

In light of this, given the projected features of different datasets, we can align their feature spaces by rearranging the permutation features based on the descending order of 8 w.r.t. each projected feature. For all datasets, the feature in the first column is the one with the lowest 8k, which deserves more attention by ARC; conversely, features with less contribution (i.e. higher 8t) are placed the end. In this way, the GNN-based model can learn to filter graph signals with different smoothness levels automatically and predict anomalies accordingly. During inference, the smoothness- information remains transferable because we adhere to the same alignment strategy.
4.2 Ego-Neighhor Residual Graph Encoder
Once the features are aligned, we employ a GNN-based graph encoder to learn node embeddings that capture both semantic and structural information for each node. The learned embedding can be utilized to predict the abnormality of the corresponding node with the downstream anomaly scoring module. A naive solution is directly employing commonly used GNNs, such as GCN [6] or GAT [62], the graph encoder. However, due to their low low-pass filtering characteristic, these GNNs face difficulty in
capturing abnormality-related patterns that are high-frequency and heterophilic [6 37). Moreover, most GNNs, including those tailored for GAD, tend to prioritize capturing node-level semantic information while disregarding the affinity patterns of local subgraphs [17]. Consequently, employing existing GNN models as the encoder may overemphasize dataset-specific semantic knowledge, but overlook the shared anomaly patterns (i.e. local node affinity) across different datasets.
To address the above issues, we design an ego-neighbor residual graph encoder for ARC. Equipped with residual operation, the encoder can capture multi-hop affinity patterns of each node, providing valuable and comprehensive information for anomaly identification. Similar to the "propagation then transformation" GNN architecture in SGC [63], our graph encoder consists of three steps: multi-hop propagation, shared MLP-based transformation, and ego-neighbor residual operation. In the first two steps, we perform propagation on the aligned feature matrix X' x for L iterations, and then conduct transformation on the initial and propagated features with a shared MLP network:
x Ax
z MLP (x').
where I E (......... L}. X!I) is the propagated feature matrix at the 1-th iteration, z is the trans- formed representation matrix the (-th iteration, and A is the normalized adjacency matrix [6 63). Note that, unlike most GNNs that only consider the features/repressenatioos after L-iter propagation, here we incorporate both the initial features and intermediate propagated features and transform them into the same representation space. After obtaining ZiI,.... .Z(4), we calculate the residual representations by taking the difference between Z (1 < 1 < I.) and Z and then concatenate the multi-hop residual representations form the final embeddings:
R [1] Z
Z [0]
H = R [1
14
where R is the residual matrix at the (-th iteration, H gaxd is the output embedding matrix, and denotes the concatenation operator.
Discussion.
Compared to existing GNNs, graph encoder offers the following advantages. Firstly, with the residual operation, the proposed encoder emphasizes the difference between the ego node and its neighbors rather than ego semantic information. This approach allows for the explicit modeling of local affinity through the learned embeddings. Since local affinity is a crucial indicator of abnormality and this characteristic can be shared across diverse datasets [17. the learned embeddings can offer valuable discriminative insights for downstream prediction. Second, the residual operation performs as high-pass filter on the graph data, aiding ARC capturing more abnormality attributes. i... high-frequency signals and local heterophily. Moreover, unlike existing approaches [15) [7] that only consider 1-hop affinity, our encoder also incorporates higher-order affinity through the molti-hop residual design, which enables ARC to capture more complex graph anomaly patterns. More discussion and comparison to the existing GNNs/GAD methods are conducted in Appendix C
4.3 Cross-Attentive In-Context Anomaly Scoring
To utilize the few-shot normal samples (denoted by context nodes) to predict the abnormality of the remaining nodes (denoted by query nodes), in ARC, we devise an in-context learning module with a cross-attention mechanism for anomaly scoring. The core idea of our in-context learning module

---

is to reconstruct the node embedding of each query node using a cross cotsentttttion block to blend the embeddings context nodes. Then, the drift distance between the original and reconstructed embeddings of a query node can serve as the indicator of its abnormality.
Specifically, we partition the embedding matrix H into two parts by indexing the corresponding row vectors: the embeddings of context nodes H2 E R=xx xd. and the embeddings of query nodes H, E R Then, cross-attention block is utilized to reconstruct each row of H, through linear combination of H4:
QK vdge
Q W. K K
H, Softmax
Hg-
(5)
where Q Xd. and K Rx are the query and key matrices respectively, Wg and W learnable parameters, and H. is the reconstructed query embedding matrix. Note that, unlike the con- ventional cross-attention blocks [64] 665 66) that further introduce a value matrix V, our block directly multiplies the attention matrix with H This design ensures that Hg is in the same embedding space as H, and Hμ. Thanks to this property, given query node u, we can calculate its anomaly score fwx) by computing the L2 distance between its query embedding vector Hgg and the corresponding
reconstructed query embedding vector Hq,, i.e., f(v)) d(Hq Hq.) = J=
(Hqu
Hqus
Discussion. The design of cross-attentive in-context anomaly scor- ing follows a basic assumption: normal query nodes have similar patterns to several context nodes, and hence their embeddings be easily represented by the linear combination of context node embeddings. Consequently, given normal node, its original and constructed embeddings can be close to each other in the embedding space. In contrast, abnormal nodes may display distinet patterns compared to normal ones, making it difficult to depict their corre- sponding abnormal query embeddings using context embeddings. As result, their drift distance s; can be significantly larger. Fig. 4 provides examples for the scenarios of (a) single-class normal and (b) multi-class norma In both cases, the drift distance (-) can be
(a) Casc 1
(b) Case II
Figure 4: Toy examples of query embeddings (o), recon- structed query embeddings (o), and context embeddings
a significant indicator for distinguishing anomaly (5) from normal nodes (1-4). Interestingly, if the attention matrix assigns uniform weights to all context nodes, then our scoring module becomes one-class classification model [22]. This property ensures the anomaly detection capability of ARC even without extensive training. A detailed discussion is conducted in Appendix D.2]
Model Training. To optimize ARC training datasets Terain, we employ marginal cosine similarity loss to minimize the drift distance of normal nodes while maximizing the drift distance of abnormal nodes. Specifically, given graph data with anomaly labels, we randomly select n, normal nodes as context nodes and sample an equal number of normal and abnormal nodes as query nodes. Then, given a query node e with embedding Hq;, reconstructed embedding Hq;, and anomaly label y,, the sample-level loss function can be written by:
cos Hqj, Hq:
if y; = if y;
(6)
max 00 cos Hog Hq:) -e):
where cos(:.-) and max(-,)) denote the cosine similarity and maximum operation, respectively, and e margin hyperparameter. Detailed algorithmic description and complexity analysis of ARC can be found in Appendix
5
Experiments
5.1
Experimental Setup
Datasets. To learn generalist GAD models, we train the baseline methods and ARC on a group of graph datasets and test another group of datasets. For comprehensive evaluations, consider
"Specific definitions Appendix D.I
7

---

Table 1: Anomaly detection performance in terms of AUROC (in percent, mtdn Highlighted
are the results ranked first, second, and third. "Rank" indicates the average ranking over 8 datasets.
Method
Cora
CitSeee
ACM
BlogCataing Facebook
Weibo
Reddit
Amazon
Rank
Supervised Pre Train Only
GCN GAT
60.27-8.11
60.19
56.
29.51=4.00
76.61=17
50043=44.41
46.63+5..7
50.06224s
51.59.3.4
48.79
50.40)a2..0
51.88
53.0627.08
51.78
50.52-11:22 17:22
10.0
BONN
12.15=11.87
12.32-11.82
41.00::1449
47. 47.6T=8.02
54.74=25.2
32,75
50..7=33.44
52.26+53:11
11.1
BWGNN
54.061527
52.61_2.
67.59
18.34
45.84
53.38.1.m
48.97a5.74
55.26. 10.00
GHRN
39 8998.77
56.0M499.19
65.65
57. 65..4.
41.81=s.00
51.87=:4.15 18
46.22
19. 19.4131
9.8
Unsupervised
Train Only
DOMINANT
66.53 16.53=115
69 89:17=22.2
70.08=2.54
74 26+00.05
51.01 51.01=0.78 78
9/2 88
50.06=4..0
48. 24
5.8
CoLA
63 63.29
62.84=9.52
66.83
50 50.01e33.2
12.99=11.05
16.27=0..4
52.81 52.81=s..
47. 77
9.5
HCM-A
28
18.12-0.50
63,70
55. +0.007
35.4++15.7
65.52=112 ss
48. 18.799=2.7 75
43. 03:0.7
11:1
TAM
62.02:239
72.27
74.43
29.86
65 28.88...00
71.54:20.14
55.43as..
26.016:2.19
5.6
Unsupervised Pre Train & Fine-Tune
DOMINANT
72 82
71669-032
71.31=0.12
21.1++0.0
19 00.92=0.ar as
92,21=0.0 10
32.11=s5.0
59 00.06622..0
3.6
CoLA
07.62:425
70.75=x42
69.11
62. a9
64.70:11 as
31.5526.02
58.12ass.7
22.51
5.4
HCM-A
36.452495
65.54=4.7
57.69=8.50
55. 10
36.57=10.72
71.89+2.M t9
19.15 19.15+2272
42.20=aous
10.1
TAM
62.56121
76.54-120 1:33
86, 29:
57. 19..8
76.26
71.7320.36
56.62
S7. 1..00
3.4
Ours
ARC
87 45+0.4
90.95+0.59
79.88-0.28
74.76=0.00 08
67.5 1.
88,85e0.14
60. 00.01:0.:0
80.67+1.8
1.5
graph datasets spanning a variety of domains, including social networks, citation networks, and e-commerce co-review networks, each of them with either injected anomalies or real anomalies [7] [7]. Inspired by [52), we train the models on the largest dataset of each type and conduct testing on the remaining datasets, Specifically, the training datasets Ttrain comprise PubMed, Flickr. Questions, and YelpChi, while the testing datasets Treat consist of Cora, CiteScer, ACM, BlogCatalog, Facebook, Weibo, Reddit, and Amazon. For detailed information, please refer to Appendix F.
Baselines. We compare ARC with both supervised and unsupervised methods. Supervised methods include two conventional GNNs, i.e., GCN 61 and GAT [62], and three state-of-the-art GNNs specifically designed for GAD, i.e., BGNN [67, BWGNN (6) and GHRN [37]. Unsupervised methods include four representative approaches with distinct designs, including the generative method
DOMINANT 4 the contrastive method CoLA [15], the hop predictive method HCM-A [16. and
the affinity-based method TAM [7]. For detailed information, refer to Appendix
Evaluation and Implementation. Following 17 688) we employ AUROC and AUPRC as evaluation metrics for GAD. We report the average AUROC/AUPRC with standard deviations across 5 trials. We train ARC on all the datasets in Terain jointly, and evaluate the model on each dataset in Trest in an in-context learning manner (ng = 10 as default). For the supervised baselines, we follow the same training and testing procedure (denoted as "pre-train only"), since no labeled anomaly is available for fine-tuning. For the unsupervised baselines, we consider two settings: "pre-train only" and "pre-train & fine-tune". In the latter, we additionally conduct dataset-specific fine-tuning with a few epochs. To standardize the feature space in baseline methods, we utilize either learnable projection or random projection as adapter between the raw feature and the model input layer. We utilize random search to determine the optimal hyperparameters for both the basclines and ARC. Since our goal to train generalist GAD models, we do not perform dataset-specific hyperparameter search, but instead use the same set of hyperparameters for all testing datasets. More implementation details can be found in Appendix F3
5.2 Experimental Results
Performance Comparison. Table shows the comparison results of ARC with base- line methods in terms of AUROC. Results in AUPRC are provided in Appendix G.1 We have the following observations. ARC demonstrates strong anomaly detection capabil- ity in the generalist GAD scenario, without any fine-tuning. Specifically, ARC achieves state-of- the-ar art performance on out of datasets and demonstrates competitive performance on the remainder. On several datasets, ARC demon- strates significant improvement compared to the best baseline (e.g., 121.1% 1% on Cora, 118.8 on CiteSeer, and 136.6% on Amazon). @ Simply
(3)
La) AUROCC.5)
S ReanT
36
do
Number of Contest Nodev
Number of Contout Modes an
(a) Cora
(b) Facebook
Figure 5: Performance with varying nig-
8

---

pre-training the -ptaset- GAD methods typically results in poor generalization capability to new datasets. Specifically, the AUROC of the majority of "pre-train only" approaches is close to random guessing (50%) or even lower. @ With dataset-specific fine-tuning, the baseline methods achieve better performance the majority of cases. However, the improvement can be minor or even negative in some cases, demonstrating the limitations of fine-tuning. Additionally, we observe that their performance
sometimes lower than the reported results from training models from scratch 4 15 [77, indicating potential risk of negative transfer within the "pre-train & fine-tune" paradigm. 4 Unsupervised baselines (except HCM-A) generally outperform the supervised ones, highlighting the difficulty of training generalist GAD model using the binary classification paradigm.
Effectiveness of #Context Nodes. To investigate how the number of context nodes n4 affects the performance of ARC during inference, we vary ng within the range of 2 to 100. The results are shown Fig. 3 (more results are in Appendix G.2). From the figure, we observe that the performance of ARC increases as more context nodes are involved, demonstrating its capability to leverage these labeled normal nodes with in-context learning. Furthermore, we can conclude that ARC is also label-efficient: when ni 2 10, the performance gain from using more context nodes becomes minor; moreover, even when vg is extremely small, ARC can still perform well on the majority of datasets.
Ablation Study. To verify the effective- ness of each component of ARC, we make corresponding modifications to ARC and designed three variants: 1) w/o A: using random projection to replace smoothness- based feature alignment; 2) w/o R: us-
Table AUROC of ARC and variants.
Cora
Blag.
Face.
Wei.
ARC
87.45
90.95
79.88
74. 76
67. 56
88 85
60.04
80
wlo A
50.65
63.35
79.29
73.86
62.
89.08
54.60
64.76
37.41
weo C
47.39
53.98
54.24
80.46
48.3
42.54
51.03
89.02
ing GCN to replace ego-neighbor residual graph encoder; and 3) w/o C: using binary classification-basee predictor and loss to replace cross- attentive in-context anomaly scoring. The results are demonstrated in Table 2 (full results are in Appendix G.3). From the results, we can conclude that all three components significantly contribute to the performance. Among them, the in-context anomaly scoring module has a significant impact, as the performance w/o is close to random guessing on most datasets. The residual graph encoder also has significant impact the final performance. Notably, Weibo dataset is an exception where the GCN encoder performs better. A possible reason is that the Weibo datasct exhibits different anomaly patterns compared to the others.
Run Time (s) 60
10²
To assess the runtime efficiency ARC, we compare the inference and fine-tuning time the ACM dataset. As depicted in Fig. [ ARC demonstrates comparable runtime performance with the fastest GNNs (e.g.. GCN and BWGNN), and significantly outperforms the unsupervised methods in terms of efficiency. Additionally, observe that dataset-specifie fine- tuning consumes more time compared to inference.
Efficiency Analysis.
Infasenee
spech)
16
10
Figure 6: Time comparison.
Visualization. To investigate the weight allocation mechanism of the cross-attention module in ARC, we visualize the atten- tion weights between context nodes and query nodes in Fig 7 (additional results are in Appendix G.4). From Fig. [a) is evident that ARC tends to assign uniform attention weights to
Normal
Normal
Quary Nodes
Quary Nodes
normal nodes, leading to reconstructed embeddings that closely resemble the average embedding of the context nodes. Con- versely, anomalies are reconstructed using a combination of 1 or 2 context nodes, suggesting that their embeddings are far-
Anomaly
Anomaly
ther from the center. This allocation aligns with the case of "single-class normal" in Fig. 4 a). Differently, in Fig. (b), we observe that each normal query node is assigned to several context nodes following two fixed patterns, corresponding to
Context Nodes
Contex: Nodes
(a) Cora
(b) Amazon
Figure 7: Visualization results.
the case of "multi-class normal" in Fig. 4b.. In summary, the cross-attention module enables ARC
adapt to various normal/anomaly distribution patterns, enhancing its generalizability.

---

6 Conclusion
In this paper, we take the first step towards addressing the generalist GAD problem, aiming to detect anomalies across diverse graph datasets with a "one-for-all" GAD model, without requiring dataset-specific fine-tuning. We introduce ARC, novel and well-crafted in-context learning-bases generalist GAD approach, capable of identifying anomalies on-the-fly using only few-shot normal nodes. Extensive experiments on real-world datasets from various domains demonstrate the detection prowess, generalizability, and efficiency of ARC compared to existing approaches. One limitation is that ARC can only use normal context samples during inference but cannot directly utilize abnormal context samples, even when they are available. A potential future direction could involve developing generalist GAD methods that utilize context samples containing both anomalies and normal instances.
Acknowledgments and Disclosure of Funding
This research was partly funded by Australian Research Council (ARC) under grants F1210100097 and 54240101547 and the CSTRO National Science Foundation (US) Al Research Collaboration Program.
References
[1] Ranran Li, Zhaowei Liu, Yuanqing Ma, Dong Yang, and Shuaijie Sun. Internet financial fraud detection based on graph learning. Jeee Transactions on Computational Social Systems, 2022.
[2] Tian Bian, Xi Xiao, Tingyang Xu, Peilin Zhao, Wenbing Huang. Yu Rong. and Junzhou Huang. Rumor detection on social media with bi-directional graph convolutional networks. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 549-556, 2020.
[3] Xiaoxiao Ma. Jia Wu, Shan Xue, Jian Yang, Chuan Zhou, Quan Z Sheng, Hui Xiong, and Leman Akoglu, comprehensive survey on graph anomaly detection with deep learning. IEEE Transactions on Knowledge and Data Engineering, 35(12): 12112-12333 2021.
[4] Kaize Ding, Jundong Li. Rohit Bhanushali, and Huan Liu. Deep anomaly detection on attributed networks. In Proceedings of the 2019 SIAM International Conference Data Mining, pages 594 602. SIAM, 2019.
[5] Kay Liu, Yingtong Dou, Yue Zhao, Xucying Ding, Xiyang Hu, Ruitong Zhang, Kaize Ding, Canyu Chen, Hao Peng, Kai Shu, et al. Bond: Benchmarking unsupervised outlier node detection on static attributed graphs. Advances in Neural Information Processing Systems. 55:27021-270033 2022.
[6] Jianheng Tang, Jiajin Li, Ziqi Gao, and Jia Li. Rethinking graph neural networks anomaly detection. In International Conference on Machine Learning, pages 21076 11076-21089, PMLR, 2022.
[7] Jianheng Tang, Fengrui Hua, Ziqi Gao, Peilin Zhao, and Jia Li. Gadbench: Revisiting and benchmarking supervised graph anomaly detection. Advances in Neural Information Processing Systems, 36, 2024.
[8] Jinyu Cai, Yunhe Zhang, Zhoumin Lu, Wenzhong Guo, and See-kiong Ng. Towards effective federated graph anomaly detection via self-boosted knowledge distillation. ACM Multimedia, 2024.
[9] Bryan Perozzi and Leman Akoglu. Scalahle anomaly ranking of attributed neighborhonds. In Proceedings of the 2016 SIAM International Conference on Data Mining, pages 207-215. SIAM, 2016.
[10] Jundong Li, Harsh Dani, Xia Hu, and Huan Liu. Radar: Residual analysis for anomaly detection in attributed networks. In IJCAI, volume 17, pages 2152-2158, 2017.
[11] Zhen Peng, Minnan Luo, Jundong Li, Huan Liu, Qinghua Zheng, et al. Anomalous: A joint modeling approach for anomaly detection on attributed networks. In IJCAI, volume 18, pages 3513-3519, 2018.
[12] Yingtong Dou, Zhiwei Liu, Li Sun, Yutong Deng, Hao Peng, and Philip S Yu. Enhancing graph neural network-based fraud detectors against camouflaged fraudsters. In Proceedings of the 29th ACM international conference on information & knowledge management, pages 315-324, 2020.
[13] Ao Li, Zhou Qin, Runshi Liu, Yiqun Yang, and Dong Li. Spam review detection with graph convolutional networks. In Proceedings of the 28th ACM International conference on information and knowledge management, pages 2703-27 22-22211. 2019.
[14] Yang Liu, Xiang Ao, Zidi Qin, Jianfeng Chi, Jinghua Feng, Hao Yang, and Qing Hc. Pick and choose: a gnn-based imbalanced learning approach for fraud detection. In Proceedings of the web conference 2021, pages 3168-3177. 2021.
[15] Yixin Liu, Zhao Li, Shirui Pan, Chen Gong, Chuan Zhou, and George Karypis. Anomaly detection on attributed networks via contrastive self- supervised learning. IEEE transactions on neural networks and learning systems, 33(6):2378 229922222222222 2021.
10

---

[16] Tianjin Huang, Yulong Pei. Vlado Menkovski, and Mykola Pechenizkiy. Hop-count based self-supervised anomaly detection on attributed networks. In Joint European conference on machine learning and knowledge discovery in databases, pages 225-241. Springer, 2022.
[17] Hezhe Qiao and Guansong Pang. Truncated affinity maximization: One-class homophily modeling for graph anomaly detection. In Advances in Neural Information Processing Systems, volume 36, 2023.
[18] Jiawen Zhu and Guansong Pang. Toward generalist anomaly detection via in-context residual learning with few-shot sample prompts. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[19] Hao Liu, Jiarui Feng. Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. One for all: Towards training one graph model for all classification tasks. In International Conference on Learning Representations, 2024.
[20) Haihong Zhao, Aochuan Chen, Xiangguo Sun, Hong Cheng, and Jia Li, All in one and one for all: A simple yet effective method towards cross-domain graph pretraining. arXiv preprint 24X1v:2402.09834 2024.
[21] Guansong Pang, Chunhua Shen, Longbing Cao, and Anton Van Den Hengel. Deep learning for anomaly detection: A review. ACM computing surveys (CSUR), 54(2):1-38, 2021.
[22] Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. In International conference on machine learning, pages 4393 4402. PMLR, 2018.
[23] Sachin Goyal, Aditi Raghunathan, Moksh Jain, Harsha Vardhan Simhadri, and Prateek Jain. Droce: Deep robust one-class classification. In International conference on machine learning, pages 3711 PM 2020.
[24] Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schölkopf, Thomas Brox, and Peter Gehler. Towards total recall in industrial anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2022.
[25] Chong Zhou and Randy C Paffenroth, Anomaly detection with robust deep autoencoders. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 665 674, 2017.
[26] Thomas Schlegl, Philipp Seeböck, Sebastian M Waldstein, Ursula Slhmidt-Erfurthh and Georg Langs. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In International conference on information processing in medical imaging, pages 146-157. Springer, 2017.
[27] Vikash Sehwag, Mung Chiang, and Prateek Mittal, SSD: A unified framework for self-supervised outlier detection. In International Conference on Learning Representations, 2021.
[28] Jinyu Cai and Jicong Fan. Perturbation learning based anomaly detection. In Advances in Neural Information Processing Systems, pages 14317-14330, 2022.
[29] Yunhe Zhang, Yan Sun, Jinyu Cai, and Jicong Fan. Deep orthogonal hypersphere compression for anomaly detection. In Proceedings of the Twelfth International Conference on Learning Representations, 2024.
[30] Chaogin Huang, Haoyan Guan, Aofan Jiang, Ya Zhang, Michael Spratling, and Yan-Feng Wang. Regis- tration based few-shot anomaly detection. In European Conference on Computer Vision, pages 303-319. Springer, 2022.
[31] Jongheon Jeong. Yang Zou, Taewan Kim, Dongqing Zhang, Avinash Ravichandran, and Onkar Dabeer. Winelip: sero-ffew-shot anomaly classification and segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19606-19616, 2023.
[32] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8788778 8763. PMLR, 2021.
[33] Yu Zheng, Ming Jin, Yixin Liu, Lianhua Chi, Khoa T Phan, and Yi-Ping Phoebe Chen. Generative and contrastive self-supervised learning for graph anomaly detection, IEEF Transactions on Knowledge and Data Engineering, 35(1 22:10220-1222 2021.
[34] Xuexiong Luo, Jia Wu, Amin Beheshti, Jian Yang. Xiankun Zhang, Yuan Wang, and Shan Xue. Comga: Community- aware attributed graph anomaly detection. In Proceedings of the Fifteenth ACM International Conference Web Search and Data Mining, pages 657-665, 2022.
[35] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A compre- hensive survey on graph neural networks. IEEE transactions neural networks and learning systems. 32(1):4-22 24, 2020.
11

---

[36]
Mingguo He, Zhewei Wei, Hongteng Xu, et al. Bernnet: Learning arbitrary graph spectral filters via bernstein approximation. Advances Neural Information Processing Systems, 34:14239-14251, 2021.
[37] Yuan Gao, Xiang Wang. Xiangnan He, Zhenguang Liu, Huamin Feng, and Yongdong Zhang. Addressing heterophily in graph anomaly detection: A perspective of graph spectrum. In Proceedings of the A ACM Web Conference 2023, pages 15588-1388, 2023.
[38] Haoyi Fan, Fengbin Zhang, and Zaoyong Li. Anomalydae: Dual autoencoder for anomaly detection on attributed networks. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5685-5689. IEEE, 2020.
[39] Jingcan Duan, Siwei Wang, Pei Zhang. En Zhu, Jingtao Hu, Hu Jin, Yue Liu, and Zhibin Dong. Graph anomaly detection via multi-scale contrastive learning networks with augmented view. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 7459-7467, 2023.
[40] Bo Chen, Jing Zhang, Xiaokang Zhang, Yuxiao Dong. Jian Song, Peng Zhang. Kaibo Xu, Evgeny Kharlamov, and Jie Tang. Gecad: Graph contrastive learning for anomaly detection. IEEE Transactions on Knowledge and Data Engineering. 2022.
[41] Tong Zhao, Chuchen Deng, Kaifeng Yu. Tianwen Jiang. Daheng Wang, and Meng Jiang. Error- bounde graph anomaly loss for gnns. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 1873-1882 2020.
[42] Junjun Pan, Yixin Liu, Yizhen Zheng, and Shirui Pan. Prem: A simple yet effective approach for node-level graph anomaly detection. In 2023 IEEE International Conference on Data Mining (ICDM), pages 1253- 1258. IEEE, 2023.
[43] Kaize Ding, Kai Shu, Xuan Shan, Jundong Li, and Huan Liu. Cross-domain graph anomaly detection. I.F. Transactions on Neural Networks and Learning Systems, 33(6):2406-2455, 2021.
[44] Qizhou Wang, Guansong Pang, Mahsa Salchi, Wray Buntine, and Christopher Leckie. Cross-domain graph anomaly detection via anomaly aware contrastive alignment. In Proceedings the AAAI Conference on Artificial Intelligence, volume 37, pages 4676-4684, 2023.
[45] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901 2020.
[46] Bean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lene, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716-23727 2022.
[47] Yaru Hao, Haoyu Song, Li Dong, Shaohan Huang, Zewen Chi, Wenhui Wang. Shuming Ma, and Furu interfaces. arXiv preprint arXiv:2206. 2.XXx:2200606333, 2022.
Wei. Language models are
[48] Ting Chen, Saurabh Saxena, Lala Li, Tsung-Yi Yi Lin, David J Fleet, and Geoffrey E Hinton. A unified sequence interface for vision tasks. Advances in Neural Information Processing Systems, 35:31333-31346, 2022.
[49] Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. Ofa: Unifying architectures, tasks, and modalities through simple sequence- to-sequence learning framework. In International Conference on Machine Leurning, pages 23318-23344 PMLR, 2022.
[50] Alexander Kolesnikov, André Susano Pinto, Lucas Beyer, Xiaohua Zhai, Jeremiah Harmsen, and Neil Houlsby. Uvim: A unified modeling approach for vision with learned guiding codes. Advances in Neural Information Processing Systems, 35:26295-26338 2022.
[51] Qian Huang, Hongyu Ren, Peng Chen, Gregor Kržmane, Daniel Zeng, Perey Liang, and Jure Leskovee. Prodigy: Enabling in-context learning over graphs. Advances in Neural Information Processing Systems, 36, 2024.
[52] Kaiwen Dong. Haitao Mao. Zhichun Guo, and Nitesh V Chawla. Universal link predictor by in-context learning, arXiv preprint arXiv:2402.07738 2024.
[53] Gilbert W Stewart. On the early history of the singular value decomposition. SIAM review, 35(4)5551-66. 1993.
[54] Hervé Abdi and Lynne J Williams. Principal component analysis. Wiley interdisciplinary reviews: computational statistics, 2(4):433-459 2010.
[55] Yuan Gao, Xiang Wang, Xiangnan Hc, Zhenguang Liu, Huamin Feng, and Yongdong Zhang. Alleviating structural distribution shift in graph anomaly detection. In Proceedings the Sixteenth ACM International Conference on Web Search and Data Mining, pages 357-365, 2023.
[56] Yushun Dong Kaize Ding, Brian Jalaian, Shuiwang Ji, and Jundong Li. Adagnn: Graph neural networks with adaptive frequency response filter. In Proceedings of the 30th ACM international conference on information & knowledge management, pages 392 401, 2021.
12

---

[57] Sitao Luan, Chenqing Hua, Qincheng Lu, Jiaqi Zhu, Mingde Zhao, Shuyuan Zhang, Xiao-Wer Wen Chang. and Doina Precup. Revisiting heterophily for graph neural networks. Advances in neural information processing systems, 35:1362-1375. 2022.
[58] Yizhen Zheng, He Zhang, Vincent Lee, Yu Zheng, Xiao Wang, and Shirui Pan. Finding the missing-half: Graph complementary learning for homophily-promm and heterophily-prone graphs. In International Conference Machine Learning, pages 42492 -2005.. PMLR, 2023.
[59] Yixin Liu, Yizhen Zheng, Daokun Zhang, Vincent CS Lee, and Shirui Pan. Beyond smoothing: Unsuper- vised graph representation learning with edge heterophily discriminating. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 4516-4524, 2023.
[60] Xin Zheng, Yi Wang. Yixin Liu, Ming Li, Miao Zhang. Di Jin, Philip S Yu, and Shirui Pan. Graph neural networks for graphs with heterophily; A survey. arXiv preprint arXiv:2202.07062, 2022.
[61] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference Learning Representations, 2017.
[62] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.
[63] Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Simplifying graph convolutional networks. In International conference on machine learning, pages 6861-6871. PMLR, 2019.
[64] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez. Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, Advances in neural information processing systems, 30, 2017.
[65] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[66] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- image pre- training with frozen image encoders and large language models. In International conference on machine learning, pages 19730-19742. PMLR, 2023.
[67] Sergei Ivanov and Liudmila Prokhorenkova. Boost then convolve: Gradient boosting meets graph neural networks. In International Conference on Learning Representations, 2021.
[68] Guansong Pang, Anton van den Hengel, Chunhua Shen, and Longhing Cao. Toward deep supervised anomaly detection: Reinforcement learning from partially labeled anomaly data. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 1298-1300 1908,1308, 2021.
[69] Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier. Padim: a patch distribution modeling framework for anomaly detection and localization. In International Conference on Pattern Recognition, pages 475-489. Springer, 2021.
[70] Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj. Reconstruction by inpainting for visual anomaly detection. Pattern Recognition, 112: 107706, 2021.
[71] Yiru Zhao, Bing Deng, Chen Shen, Yao Liu, Hongtao Lu, and Xian-Sheng Hua. Spatio-temporal autocncoder for video anomaly detection. In Proceedings of the 25th ACM international conference on Multimedia, pages 1933 1941, 2017.
[72] Julian Wyatt, Adam Leach, Sebastian M Schmon, and Chris G Willcocks. Anoddpm: Anomaly detection with denoising diffusion probabilistic models using simplex noise. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 650-656, 2022.
[73] Thomas Schlegl, Philipp Seeböck, Sebastian M Waldstein, Georg Langs, and Ursula Schmidt-Erfurth f-anogan: Fast unsupervised anomaly detection with generative adversarial networks. Medical image analysis, 54:30 44, 2019.
[74] Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, and Tomas Pfister. Cutpaste: Self-supervised learning for anomaly detection and localization. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 9664-9674, 2021.
[75] Yixin Liu, Thalaiyasingam Ajanthan, Hisham Husain, and Vu Nguyen. Self-supervision improves diffusion models for tabular data imputation. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, 2024.
[76] Choubo Ding, Guansong Pang, and Chunhua Shen. Catching both gray and black swans: Open-set supervised anomaly detection. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 7388-7398. 2022.
[77] Yiwei Lu, Frank Yu, Mahesh Kumar Krishna Reddy, and Yang Wang. Few-shot scene scene-adaptive anomaly detection. In Computer Vision -Cconn-CCV/- 2020: 16th European Conference, Glasgow, UK, August 23 28, 2020, Proceedings, Part V 16, pages 125- 141. Springer, 2020.
13

---

[78] Tri Cao. Jiawen Zhu, and Guansong Pang. Anomaly detection under distribution shift. In Proceedings of the IEEE/CVF International Conference on Computer Vision. pages 651 2023.
[79] Abhishek Aich, Kuan-Chuan Peng, and Amit K Koy-Chowdhurg Cross-domain vidco anomaly detection without target domain adaptation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2579-2591, 2023.
[80] Wenchao Yu, Wei Cheng, Charu C. Aggarwal, Kai Zhang, Haifeng Chen, and Wei Wang. Netwalk: A flexible deep embedding approach for anomaly detection in dynamic networks. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2672-2681, 2018.
[81] Li Zheng, Zhenpeng Li, Jian Li, Zhao Li, and Jun Gao. Addgraph: Anomaly detection in dynamic graph using atlention-base temporal gen. In IJCAI, volume 3. page 7, 2019.
[82] Yixin Liu, Shirui Pan, Yu Guang Wang, Fei Xiong, Liang Wang. Qingfeng Chen, and Vincent CS Lee. Anomaly detection in dynamic graphs via transformer. IEEE Transactions on Knowledge and Data Engincering, 35(12): 12094, 2021.
[83] Rongrong Ma, Guansong Pang, Ling Chen, and Anton van den Hengel. Deep graph-level anomaly detection by glocal knowledge distillation, In Proceddings of the fifteenth ACM international conference on web search and data mining, pages 704-714, 2022.
[84] Yixin Liu, Kaize Ding, Qinghua Lu, Fuyi Li, Leo Yu Zhang, and Shirui Pan. Towards self self-interpretable graph-level anomaly detection. Advances in Neural Information Processing Systems, 36, 2024.
[85] Yili Wang, Yixin Liu, Xu Shen, Chenyu Li, Kaize Ding, Rui Miao, Ying Wang. Shirui Pan, and Xin Wang. Unifying unsupervised graph-level anomaly detection and out-of-distributiin detection: A benchmark. arXiv preprint arXiv:2406. 15533, 2024.
[86] Yixin Liu, Kaize Ding, Huan Liu, and Shirui Pan. Good-d: unsupervised graph distt--dittiiuttion detection. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, pages 339-347, 2023.
[87] Luzhi Wang, Dongxiao He, He Zhang, Yixin Liu, Wenjic Wang, Shirui Pan, Di Jin, and Tat-Seng Chua. Goodat: Towards test-time graph out-of-distriution detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 15537-15545, 2024.
[88] Jinyu Cai, Yunhe Zhang, Jicong Fan, and See- Kis-Kiong Ng. Lg fgad: An effective federated graph anomaly detection framework. In Proceedings of the International Joint Conference on Artificial Intelligence. 2024.
[89] Xin Zheng, Yixin Liu, Zhifeng Bao, Meng Fang, Xia Hu, Alan Wee-Chung Liew, and Shirui Pan. Towards data-centric graph machine learning: Review and outlook. arXiv preprint arXiv:2309. 10979, 2023.
[90] Yixin Liu, Kaize Ding, Jianling Wang. Vincent Lee, Huan Liu, and Shirui Pan. Learning strong graph neural networks with weak information. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1559-1571, 2023.
[91] Shiyuan Li, Yixin Liu, Qingfeng Chen, Geoffrey Webb, and Shirui Pan. Noise-resilient unsupervised graph representation learning via multi-hop feature quality estimation. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, 2024.
[92] Zhao Li, Yixin Liu, Zhen Zhang, Shirui Pan, Jianliang Gao, and Jiajun Bu. Cyclic label propagation for graph semi-supervised learning. World Wide Web, 25(2):033-721 2022.
[93] Yizhen Zheng, Shirui Pan, Vincent Lee, Yu Zheng, and Philip S Yu. Rethinking and scaling up graph contrastive learning: An extremely efficient approach with group discrimination. Advances in Neural Information Processing Systems, 10800-00000000 2022.
[94] Mingda Chen, Jingfei Du Ramakanth Pasunuru, Todor Mihaylov, Srini Iyer, Veselin Stoyanov, and Zomitsa Kozareva. Improving in-context sho- learning via supe-vid training. arXiv preprint arXiv:2205.01703, 2022.
[95] Sewon Min, Mike Lewis, Luke 7ettlemoyer, and Hannanch Hajishirzi. Metaicl: Learning to learn in context. arXiv preprint arXiv:2110.1544 2021.
[96] Amir Bar, Yossi Gandelsman, Trevor Darrell, Amir Globerson, and Alexci Efros. Visual prompting via image inpainting, Advances in Neural Information Processing Systems, 25:25005-25017. 2022.
[97] Yulong Pei, Tianjin Huang, Werner van Ipenburg, and Mykola Pechenizkiy. Resgen: Attention-based deep residual modeling for anomaly detection on attributed networks. Muchine Learning, 111 11: 2022.
[98] Prithviraj Sen, Galileo Namata, Mustafa Bilgic. Lise Getoor, Brian Galligher, and Tina Eliassi Rad. Collective classification in network data. Al magazine, 29(3):93-93, 2008
14

---

[99] Jie Tang, Jing Zhang, Limin Yao. Juanzi Li. Li Zhang, and Zhong Su. Ametminer: extraction and mining of academic social networks. In Proceedings the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 990-998, 2008.
[100] Lei Tang and Huan Liu, Relational learning via latent social dimensions. In Proceedings of the 15th ACM SIGKDD international conference Knowledge discovery and data mining, pages 117-826, 2009.
[101] Shebuti Rayana and Leman Akoglu. Collective opinion spam detection: Bridging review networks and metadata. In Proceedings of the 21th acm sigkdd international conference on knowledge discovery and data mining, pages 085-994, 2015.
[102) Julian John McAuley and Jure Leskovec. From amatcurs to connoisscurs: modeling the evolution of user expertise through online reviews. In Proceedings of the 22nd international conference on World Wide Web, pages 897-908, 2013.
[103] Shijic 7hang, Hongzhi Yin, Tong Chen, Qune Viet Nguyen Hung. Huang, and Lizhen Cui. Gen-based user representation learning for unifying robust recommendation and fraudster detection. In Proceedings of the 43rd international ACM SIGIR conference research and development in information retrieval, pages 689-698, 2020.
[104] Arjun Mukherjee, Vivek Venkataraman, Bing Liu, and Natalie Glance. What yelp fake review filter might be doing? In Proceedings of the international AAAI conference on web and social media, volume 7, pages 409 418, 2013.
[105] Zhiming Xu, Xiao Huang. Yue Zhao, Yushun Dong. and Jundong Li. Contrastive attributed network anomaly detection with data augmentation. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 444-457. Springer, 2022.
[106] Srijan Kumar, Xikun Zhang, and Jure Leskovee. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1269-1278, 2019.
[107] Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova. critical look at the evaluation of gnns under heterophily; Are we really making progress? arXiv preprint arXiv:2302.1 11640, 2023.
[108] Kaize Ding, Jundong Li, and Huan Liu. Interactive anomaly detection on attributed networks. In Proceedings of the twelfth ACM international conference on web search and data mining, pages 357-365, 2019.
[109] David B Skillicorn. Detecting anomalies graphs. 2007 IEEE Intelligence and Security Informatics. pages 209-216 IEEE, 2007.
[110] Xiuyao Song, Mingxi Wu, Christopher Jermaine, and Sanjay Ranka. Conditional anomaly detection. TEEE Transactions knowledge and Data Engineering, 19(5):6316655 2007.
15

---

A Detailing Related Work
Anomaly Detection. The objective of anomaly detection (AD) is to identify anomalous samples that deviate from the majority of samples [21]. Due to the difficulty of collecting labeled anomaly data, mainstream AD methods mainly focus on unsupervised settings. To capture anomaly patterns without guidance by annotated labels, existing studies employ several advanced techniques to learn powerful
AD models, such as one-class classification [22 23 distance measurement [24] [69 data reconstruc-
tion [25 70 71). generative models [26] 72
73 and self-supervised learning (2 74) For example,
DeepSVDD [22] introduces a fully deep one-class classification objective for unsupervised anomaly detection, optimizing a data-enclosing hypersphere in output space extract common factors of variation and demonstrating theoretical properties such as the e-property. AnoDDPM [72), a simplex noise-based approach for anomaly detection, enhances anomaly capture with a partial diffusion strategy [75] and multiscale simplex noise processing, facilitating faster inference and training on higl-resolution images. While effective, these approaches are tailored to identify abnormal samples within a predetermined target dataset (i.e., the dataset for training), restricting their generalizability to new domains.
Anomaly Detection. Recently, some advanced AD methods aim to transcend dataset limitations, enhancing their generalizability across diverse datasets. A research line aims address the AD problem under domain or distribution shifts 776 77 78 [7)]: however, these approaches require domain relevance between the source and target datasets, thus limiting their generalizability. To enable the model to better understand the patterns in target datasets, viable approach is to incorporate a few-shot setting, allowing access to a limited number of normal samples from the target datasets. Under the few-shot setting, RegAD [30 is pioneering approach that trains a single generalizable model capable of being applied to new in-domain data without re-training or fine- tuning. WinCLIP [11 utilizes visual-language models (VLMs. e.g., CLIP (32) with well-crafted text prompts to perform zero/few-she AD for image data. InCTRL [8]. as the first generalist AD approach, integrates in-context learning and VLMs to achieve domain-agnostic image AD with single model. However, due to their heavy reliance on pre-trained vision encoders/VLLMs and image- specific designs, these approaches excel in anomaly detection for image data but face challenges when applied to graph data.
Anomaly Detection on Graph Data. Based on the granularity of anomaly samples within graph data, existing AD approaches can be primarily categorized into three classes: node-Ievel 4 6]. edge-level 80 81 82, and graph-level
K3 84
85
86
88
AD. Due to its broad real-world 3. In this paper, we focus on the
applications, node-level AD receives the most research attention
node-level AD and refer to it as "graph anomaly detection (GAD)", following the convention of most
previous papers 6
Early GAD methods aimed to detect anomalies through shallow mechanisms. For example, AMEN 9 detects anomalies by utilizing the attribute correlations of nodes in each ego-network on the attribute network. In addition, residual analysis is another common method to measure the anomalies of nodes on an attribute network. In particular, Radar (10) characterizes the residuals of attribute information and their coherence with network information for anomaly detection. Further, ANOMALOUS (I]) proposes joint learning of attribute selection and anomaly detection based on CUR decomposition and residual analysis. Despite the success of these methods on www-dimeniional attribute graph data, they do not work well when graphs have complex structures and high tigh-iimensional attributes g9 90 91 [2
due to the limitations of their shallow mechanisms.
To overcome the limitations of shallow approaches, recently, GNN-based methods have become the de facto solution for GAD tasks. Existing GNN-based GAD approaches can be divided into two research lines: supervised GAD and unsupervised GAD 5 7 Supervised GAD approaches assume that the labels of both normal and anomalous nodes are available for model training [7]. Hence, related studies mainly focus on improving graph convolutional operators, model architectures, and supervised objective functions, to leverage labels to learn node-level anomaly patterns (6 12
B 36. 37]. For example, the spatial GNN has been redesigned mainly in terms of message passing and aggregation mechanisms. In particular, considering that GNN-based fraud detectors fail to effectively identify fraudsters in disguise, CARE-GNN [12 uses a label-aware similarity metric and introduces reinforcement learning to aggregate selected neighbors with different relationships to counteract disguises. Concurrently, spectral GNNs, relate graphical anomalies to seg-ffreuenccy spectral distributions. Tang et al. [6] analyzed anomalies for the first time from the perspective
16

---

of the spectral spectrum of the graph and proposed the Beta wavelet GNN (BWGNN), which has spectrally and spatially localized band-pass filters better capture anomalies. In addition, anomalies are usually associated with high-frequency components in the spectral representation of a graph. Therefore, GHRN (37] prunes inter-class edges by emphasizing the high-frequency components of the graph, which can effectively isolate the anomalous nodes and thus obtain better anomaly detection performance.
In contrast, unsupervised GAD approaches do not require any labels for model training. Similar to unsupervised AD for image data, unsupervised GAD approaches employ several unsupervised 34 (38.
learning techniques to learn anomaly patterns on graph data, including data reconstruction
contrastive learning [TS] 39 40), and other auxiliary objectives [16] 17 41 42 93]. For example,
approach that employs a graph convolution autoencoder to reconstruct both the adjacency and attribute matrices simultaneously, assessing node abnormality through a weighted sum of the reconstruction error terms. Similarly, ComGA [4] is community- aware attribute GAD framework based on tailored GCNs to capture local, global, and structural anomalies. CoLA [15,, the first contrastive self-supervised learning framework for GAD, samples novel contrast instance pairs in an unsupervised manner, utilizing contrastive learning to capture local information. HCM-A [6] integrates both local and global contextual information, employs hop count prediction as a suff-sueee task, and utilizes Bayesian learning to enhance anomaly identification. TAM [17] optimizes the proposed anomaly metric (affinity) the truncated graph end-to-end, considering one-class homophily and local affinity. Nevertheless, all the above methods adhere to the conventional paradigm of "one model for one dataset"..
DOMINANT 4 is a reconstruction-base
43 can handle cross-domain scenarios, their requirement for high correlation (e.g.. aligned node features) between source and target datasets limits their generalizability. Differing from existing methods, our proposed ARC is a "one-for-all" GAD model capable of identifying anomalies across target datasets from diverse domains, without the need for re-training or fine-tuning.
Although some GAD approaches
In-Context Learning, In-context learning (ICL) can be effectively adapted to new tasks based on minimal in-context examples, providing a powerful generalization capability of large language models (LLMs) 44 46 47]. For example. Brown et al. 45 demonstrate the remarkable ability of language models to perform diverse tasks with minimal training examples. Through few-shot learning, these models exhibit robustness and adaptability across various domains, exhibiting their potential as general-purge -prrpose learners natural language processing (NLP). Further, given that pre-training objectives are not specifically optimized for ICL (94), Min et al. introduced MetalCL [95] as a solution to bridge the divide between pre-training and downstream ICL utilization. This method involves continuously training the pre-trained LLMs across diverse tasks using demonstration examples, thereby enhancing its few-shot capabilities. contrast, supervised in-context fine-tuning (94) suggests constructing self-supervised training data aligned with the ICL format to utilize the original corpora for warm-up in downstream tasks. They converted raw text into input-output pairs and investigated four self-supervised objectives, such as masked token prediction and classification tasks.
ICL has also generated research attention in the field of computer vision (CV), where it has been widely used in vision tasks by designing specialized discretization tokens as prompts [18 48 49 50]. For instance, Chen et al. [48] proposed use unified interface to represent the output of each task as sequence of discrete tokens, the neural network can be trained for a variety of tasks using single model architecture and loss function, thus eliminating the need for customization for specific tasks. Furthermore, given shot prompt as a task description, the sequence output adapts to the prompt to generate task-specific results. Differently. Amir et al. [96) proposes an innovative method for in-context visual prompting by framing a wide range of vision tasks as grid in in-painting problems. This approach leverages image in- painting to generate visual prompts, guiding models to complete various vision tasks by filling in missing parts of an image based contextual information, thereby enhancing adaptability and performance across different visual challenges.
Very recently, few studies apply ICL to graph learning and GNNs. As an initial attempt to use ICL for GNNs, PRODIGY (51] leverages a prompt graph- framework to conduct few-shot ICL for node-level classification and edge-level prediction tasks. Further, UniLP [52 introduces ICL to resolve conflicting connectivity patterns caused by distributional differences between different graphs. Nevertheless, these methods require context samples in multiple classes for ICL during inference.
17

---

DDMMINANT IN TTM
DOMINANT TAM
DOMINANT
I TEM
AUROC (%)
AUROC (%)
AUROC (%)
50
50
50
0
82-1006
40-25
20-425
3-275
00-935
0 93-1000
40-055
20425
high
Jow 5t
high
Jow
high
low
Percentile w.r.t Smoothness
Percentile w.r.. Smoothness
Percentile w.r.t Smoothness
(a) CiteSeer
(b) PubMed
(c) BlogCatalog
DOMINANT TAM
DOMINANT TAM
AUROC (%)
50
AUROC
50
00-255
80-10WS
60-005
20-435
0-302
high low
high low
Percentile w.r.l Smoothness
Percentile w.r.l Smoothmess
(d) Flickr
(e) ACM
Figure 8: AUROC on data with 5 groups of features split by 8k-
Then, how to leverage one-class context samples with ICL in the scenario of generalist GAD remains an open question.
B Motivated Experiments for Smoothness-Based Feature Sorting
Experimental Setup. To verify whether smoothness can indicate the contribution features GAD, we conducted motivated experiments. We consider three mainstream GAD methods: DOMINANT [4) CoLA [15), and TAM [17]. For each method, we conduct the experiments with hyper-parameters reported in the original article across all datasets. At the data pre-processing phase, we first calculate based on the raw features and sort them in descending order. According to the new order, we divide the features into 5 groups of feature subsets, denoted by the percentile of 100%, 60%-80%, ..., 0%-20%. Subsequently, the five feature subsets are sequentially used as inputs for different methods, and the methods are trained to obtain their AUROC values. We statistic the results of 5 random experiments for each method and reported the average AUROC.
witness in the figure, in most cases, features with lower are observed to be more helpful in distinguishing anomalies on most of the datasets. This consistent modellindepenndnt trend indicates the effectiveness of s as an indicator of the contribution of each feature in GAD.
Results. Additional experimental results shown in Fig. 8 As
C Discussion of Ego-Neighbor Residual Graph Encoder
In this section, we discuss the connections and differences between the ego-neighbor residual graph encoder in ARC (R-ENC for short) and the existing GNNs and GAD methods.
R-ENC v.s. Radar [10]. Radar, one of the representative shallow GAD methods, also leverages saiduall-asee mechanism for anomaly detection. Our proposed R-ENC differs from Radar in the following key aspects:
Radar calculates the residual on the original feature space, which inevitably suffers from high complexity and less representative capability on high-dimensional graph data. In contrast, R- R-ENC computes residuals based on the output embeddings of GNN, which enables more efficient processing and enhances the representative power of the model. This
18

---

approach allows R-ENC to better capture the underlying structure and anomalies within the graph data.
In Radar, the summation of residual R directly serves as the anomaly score for each node, where the irrelevant information within several residual entries may hinder the model's s accuracy and effectiveness. Differently, ARC. we specifically use R-ENC to generate the embedding of each node, and employ another learnable scoring module to calculate the anomaly score based on these embeddings. In this case, our method can effectively filter out noise and irrelevant features, allowing the scoring module to focus on the most informative aspects of the node embeddings.
Radar employs the Laplacian matrix to calculate the residual, meaning that only first-order residual information is considered. In contrast, our R-ENC incorporates multi-hop residual aggregation, enhancing its ability to detect subtle anomalies by considering both local and global graph structures through high-order residuals.
R-ENC v.s. ResGCN (97]. ResGCN is GNN-based GAD approach with a residual- residual-based mech- anism. Similar to Radar, ResGCN also uses the summation of residual R as the anomaly score. However, the inclusion irrelevant information within several residual entries can impair the model's accuracy and effectiveness. Moreover, ResGCN employs two-branch design, with the node repre- sentation by GCN and residual information by MLP calculated by two different network modules. Compared to R-ENC with simpler designs, ResGCN has lower operational efficiency.
R-ENC v.s. CoLA [5]. CoLA learns the anomaly patterns by maximizing the agreement between the embedding of each node and its neighboring nodes. Specifically, CoLA employs bilinear module to compute the agreement score. Such an agreement can also be modeled by the first-order residual in R R-ENC, which is computed by the difference between the ego embedding and the 1-hop aggregated embeddings. Compared to CoLA, significant advantage of R-ENC is its ability to capture not only first-order residuals but also high-order residuals. This makes R-ENC more robust and comprehensive encoder for graph anomaly detection, addressing wider range of anomaly patterns across different datasets.
R-ENC v.s. TAM [17]. TAM utilizes local affinity, i.e., the feature-level or embedding-level similarity between a node and its neighbors, as the indicator of each node's abnormality. R-ENC, with its residual operation, can also capture local affinity. Specifically, by computing the first-order residual between an ego node and its 1-hop neighbors, the local affinity can be indicated by the negative summation of the first-order residuals since they are highly correlated. Again, R-ENC can not only capture the first-order affinity but also the high-order affinity with the multi-hop residual operator.
R-ENC v.s. Heterophily- the GAD problem is to enable heterophily-awaree graph convolutional operation with high-pass filtering (37]. feasible filter is graph Laplacian, whose normalized and self-loop added version can be written by I A. For R-ENC, its first-order residual can also be viewed as Laplacian- based graph convolution. Specifically, if we simplify the MLP-based feature transformation as a weight matrix W, the ego information can be written by Z = XW, while the representation z! = AXW. Then, the first-order residual can be written by:
GNNs 37.. Existing studies indicate that key solution to handle
R n 0 AXW- XW-
= -LXW.
That is to say, the first-order residual in R-ENC can be regarded as Laplacian-based high-pass filtering (note that the negative sign can be fused into the learnable weight W). Such a nice property enables ARC to capture high-frequency graph signals and heterophily information through residual-based embeddings, thereby enhancing its capability to detect anomalies in diverse and complex graph structures.
D Discussion of Cross-Attentive In- In-Context Anomaly Scoring
D.1 Definitions of Single-Clas Class and Multi-Class Normal
Dataset with single-class normal. In this type of dataset, the normal samples share the same pattern or characteristics. For example, in a network traffic monitoring system dataset, normal behavior
19

---

might be defined by regular patterns of data packets exchanged between a specific set of IP addresses. Any deviation from this single, well-defined pattern, such as an unexpected spike in data volume or communication with unknown IP addresses, can be flagged as anomalous.
Dataset with multi-class normal. In this type of dataset, the normal samples divided into multiple classes, each with distinct patterns or characteristics. For example, in a corporate email communication network dataset, normal data might be defined by regular patterns email exchanges within specific departments, such as HR, IT, and Finance. Any deviation from these well-defined patterns, such as a sudden spike in emails between normally unconnected departments an unusual volume of emails from an individual employee to external addresses, can be detected as anomalous.
D.2 ARC as One-Class Classification model
In this subsection, we first introduce the basic definition of one-class classification (OC) model, and then discuss the connection between one-class classification and cross-attentive in-context anomaly scoring module (C-AS for short).
One-class classification. The core idea of OC model is to measure the abnormality of each sample
according the distance between its representation h and center representation [22]. Here can be a fixed random representation vector or dynamically adjusted as the mean of all samples' representation vectors, Formally, the anomaly score by OC model can be written by:
((x) / (x.) clI² = llh c|l²
(8)
where ob-) is neural network model, as defined in Deep SVDD [22]. Intuitively, normal sample tends to have a similar representation to the majority of samples, and hence the distance between its representation and should be closer.
C-AS as an OC model. In C-AS, we use cross-attention block to calculate the weighted sum of context embeddings H2 into Hg, for a query node 9:- For an initialized model, we assume that the parameters Wg and W; are random enough, making Q and K become uniform noise matrices.
can be - indicating that the attention matrix assigns uniform weights to all context nodes for all query nodes. Then, all the reconstructed query embeddings are equal to the average embedding of context nodes:
In this case, each entry in the attention matrix T = Softmax
QK VT.
H 11
H 12k
Since the average context embedding the center embedding of a group of few-shot normal samples, we can naturally define the center embedding с = nu H Recalling that we define the anomaly
score as the 1.2 distance between H., and H then for all query nodes, the anomaly scoring can be rewritten by:
f(v:) - d(Hg, Hq.) d(Hq; c) |H, cl
(10)
That is to say, the C-AS module serves as an OC model under random initialization. Note that in practice, the attention matrix cannot be so ideal, but can still assign relevantly average weights for the context embeddings. Such merit ensures that ARC can perform like an OC model, effectively detecting anomalies in the case of single-class normal (Fig. 4 (a)) even without costly training.
C-AS goes beyond OC model. Thanks to its inherent mechanism, the OC model can effectively handle single-class normal scenarios. However, in the case of multi-class normals (Fig. (b)), single center is not sufficient to model multiple normal class centers. Unlike the OC model, C-AS can address this issue through pross-attentie
Specifically, the cross-ate attos-attention block learns to reconstruct query embedding by assigning higher weights to several (but not all) context embeddings that are close to the query node. This way, for a query node, the cross-attention block can automatically learn the center of its corresponding normal class, rather than simply using the average context embedding. The awareness of multiple normal classes ensures that ARC can handle both single-class and multi-class normal cases.
20

---

Algorithm 1: Smoothhess-basee Feature Alignment
Input: Graph G.
Parameters: Projected dimension du.
Extract X, &, and V from G
2 X E gaxd, f- Calculate projected features by linear projection via Eq.
for k 1 d, do
Sk Calculate feature-level smoothness of the k-tth column of X via Eq.
5 end
X' Rearrange the permutation of features of X based the descending order of
7 Return G (V),
Algorithm 2: The Training algorithm of ARC
Input: Training datasets Ttrain- Parameters Number of epoch E; Propagation iteration: L. Initialize model parameters
2 for D() E Terain do
Align features in G() via Algo.
end
5 for e = 1: do
for D(1) E Teaaso do
Obtain X g(i,, v()) y() from D(
for 1 11 L do
(+),19
Propagate and transform X ( via Eq. 3
R@M
Calculate residual of (0.0 via Eq.
10
11
end
(i)) +-
Concatenate [R 10.01I1- |R [.[44] via Eq. 44)
12
13
H
(0)
H Randomly split query and context node sets and indexing from H
14
H <- Calculate cross attention from H
H (4)
k
via Eq.
Calculate loss L from (())) 1/y
(6) via Eq. (6
15
16
Update model parameters via gradient descent.
end
18
end
E
Algorithm and Complexity
Algorithmic deseription
The algorithmic description of the feature alignment in ARC, the training process ARC, and inference process of ARC summarized in Algo. Algo. 2 and Algo. respectively.
E.2 Complexity Analysis
In the testing phase, the time complexity consists of two main components: feature alignment and model inference. For feature alignment, the overall complexity is O(ndd, + d,m + d,logg(d,)) (d,)). where = |E| is the number of edges. Here, the first term is used for feature projection, while the second and third terms are used for smoothness computation and feature reordering, respectively. The model inference is divided into two main parts: embedding generation and anomaly scoring. The complexity of node embedding generation (LLmm,, + rud,,h + nd??)), where the first term is used for feature propagation and the rest of the terms are used for residual encoding by MLP. The anomaly scoring, on the other hand, mainly involves cross-attention computation with time complexity of Ongmahh + nyh), where ng is the number of query nodes and ny is the number of context nodes.
21

---

Algorithm 3: The Inference algorithm of ARC
Input: Test dataset DD with few-shot normal nodes (Uk "1 Mkine
Parameters: Well-trained model weight parameters.
Align features in G via Algo.
2 Obtain X EE V. from
3 for | 1: L do
Z Propagate and transform X X via Eq.
R -- Calculate residual of l via Eq.
end
H f- Concatenate |I... [ via Eq.
8 Hg. H₃ + Separate query and context node sets and indexing from H
H Calculate cross attention from H.. H2 via Eq. (5)
10 <- Computing the 1,2 distance between H, and Hg
Return d as the anomaly scores f(-) for query nodes
Table 3: The statistics of datasets.
Dataset
Train
Test
#Nodes
#Edges
#Features
Avg. Degree
#Anomaly
% &Anomaly
Citation network with injected anomalies
Cora
2,708
5,429
1,433
3.90
150
5.53
CiteSeer
3,327
4,732
3,703
2.77
150
4.50
ACM
16,484
71,980
8,337
8.73
597
3.62
PubMed
19,717
44,338
500
4.50
600
3.04
Social network with injected anomalies
BlogCatalog Flickr
5,196
171,743
8,189
66.11
300
5.77
7,575
239,738
12.047
63.30
450
5.94
Social network with real anomalies
Facebook
1,081
55,104
S76
50.97
25
2.31
Weibo
8,405
407,963
400
48.53
868
10.30
Reddit
10,984
168,016
64
15.30
366
3.33
Questions
48,921
153,540
301
3.13
1,460
2.98
Co-review network with anomalies
Amazon
10,244
175,608
25
17.18
693
6.76
YelpChi
23,831
49,315
32
2.07
1,217
5.10
Details of Experimental Setup
F.1
Description of Datasets
In total, we considered 12 benchmark datasets. We divide the datasets into 4 groups: citation network with injected anomalies, social network with injected anomalies. social network with real anomalies, and 4 co-review network with real anomalies. Within each type, we consider the largest dataset one of the training datasets, and the rest datasets as the testing datasets. The detailed statistics of the datasets are shown in Table 3 These datasets are selected from different domains and with injected or real anomalies to ensure that our proposed ARC model learns extensive anomaly patterns. The diversity of the above data can maximally ensure that ARC can effectively adapt to new and unseen graphs. Specifically, the detailed descriptions for the datasets are given as follows:
Cora, CiteSeer, PubMed [98], and ACM [99 are four citation networks. these datasets, nodes represent scientific publications, while edges denote the citation links between them. Each publication characterized by bag-of -oodds representation for its node attribute vector, with the dimensionality determined by the size of the respective dictionary.
BlogCatalog and Flickr [ 100] stand as typical social blog directories, facilitating user connections through following relationships. Each user is depicted as a node, with inter-node links symbolizing mutual following. Node attributes encompass the personalized textual content generated by users within social network, such blog posts or shared photos with tag descriptions.
22

---

Amazon and YelpChi [101] 102 are datasets about the relationship between users and reviews. Amazon designed to identify users paid to write fake reviews for products, and
three different graph datasets are derived from Amazon using different types of relations
to construct adjacency matrix [17 103]. YelpChi aims to identify anomalous reviews on
Yelp.com that unfairly promote or demote products or businesses. Based on [TOT] [04],
three different graph datasets derived from Yelp using different connections in user, product review text, and time. In this work, we focus on Amazon-UFF UPU (users who have reviewed at least one of the same product) and YelpChi-RUR (reviews posted by the same user).
Facebook [105] is social network in which users build relationships with others and share their friends.
Reddit [106] serves as forum posts network sourced from the social media platform Reddit, where users labeled as banned are identified as anomalies. Textual content from posts transformed into vectors to serve as node attributes.
Weibo [106] dataset encompasses a graph of users and their associated hashtags from the Tencent Weibo platform. Within a defined temporal window (e.g., 60 seconds), consecutive posts by a user are labeled potentially suspicious behavior. Users engaging in minimum five such instances are classified as "suspicious". The raw feature vector includes the location of micro-blog post and bag-of- words features.
Questions [107] dataset originates from Yandex Q. a platform dedicated to question- answering. Users represent the nodes, while the connections between them signify the presence or absence of a question-and-annwed interaction within a one-year timeframe, The node features are derived from the average of the FastText embeddings of the words in the user description, with an additional binary feature indicating users without description.
Anomaly Injection. For the datasets with injected anomalies, we use the strategy introduced in [5 to inject anomalous nodes. Specifically, we inject a set of anomaly combinations for each dataset by perturbing the topology and node attributes, respectively [088]. In terms of structural perturbation, this is done by generating small cliques of otherwise unrelated nodes as anomalies. The intuition for this strategy that small cliques in the real world are a typical substructure of anomalies, with much more closely linked within the cliques than the mean [109). Thus for a dataset, we can specify the size of the cliques (i.e., the number of nodes) p and its amount q for anomaly generation. Specifically, randomly sample p nodes from the graph making them fully connected and labeled as anomaly nodes. We iteratively repeat the above process q times to inject total of p x q anomalies. Finally, we control the number of injected anomalies according to the size of the dataset. In particular, we fix p- = 15 and y = 10, 15, 20, 5, 5, 20 on BlogCatalog, Flickr, ACM, Cora, Citeseer, and Pubmed, respectively. On the other hand, for attribute perturbations, we base the schema introduced by (110). Specifically, for each perturbation target node u, & nodes are randomly sampled in the graph and their distance from the target node is computed. Then, the node v, with the largest deviation from the target node is is selected, and the attribute X; of the node v; to X,. We set the number of anomalies of the attribute perturbation to p g maintain the balance of different anomalies. In addition, set h = 50 to ensure that the perturbation magnitude is large enough.
F.2 Deseription of Baselines
In our evaluation, we provide a comprehensive comparsion of ARC with various supervised and unsupervised GAD methods. On the supervised side, two classic GNNs are included as well statc-of-the-art (SOTA) models specifically tailored for the GAD task. For supervised models, it is assumed that the labels of both normal and abnormal nodes can be used for model training. Therefore, the main binary classification task is used to identify the anomalies:
as seminal model in the field of GNN, is known for its ability to process graph-structured data using neighborhood aggregation, facilitating efficient node feature extraction and representation learning.
GCN [61,
GAT [62] incorporates the attention mechanism into the GNN framework to achieve dynamic weighting of node contributions. It optimizes its attention according different downstream tasks to achieve high-quality node representations.
BGNN 67 a GNN that combines gradient boost decision trees (GBDT) with GNN for graphs with tabular node features. It utilizes the GBDT to handle heterogencous features
23

---

Table 4: Anomaly detection performance in terms of AUPRC (in percent, mean Highlighted
are the results ranked first, second, and third. "Rank" indicates the average ranking over datasets.
Method
Corn
CiteSeer
BlogCotalog
Facebook
Weibo
Amazon
Rank
Supervised Pre Train Only
GCN
6:40-1140
5.27=1.12
7. 7.111107
1.5+0.11
3.39=
6.96442.4
GAT
6. B.40ge 84
6.08
4.70
12.81=2.08
3. 3.14
33.34
3.73
16.74
7.3
BONN
1.90 1.00122
3.91=1.0
3.48a133
5.73e147
3.81+2.12
30,26-29.05
3.52+0.00
7.51
10.5
BWGNN
26:0 8o
6.35-077
7.14
8.99
2.54+40.03
12: 12.13
3. 3.09
13.12211.89
GHRN
U.56+2240 40
7.79-2.01
5.61=0.71
10.94=2.0
2.4140.02
28. 28.533.7.38
3.24+0.3
7.51+2.0:
8.4
Uasupervised Pre-Train Only
DOMINANT
12. 72
13.85+2.54
15. 05
35, 25.22-0.55
2.96#0.00
81.47=0.2
3. 3.49=00.4
6. 61
6.1
CoLA
8.33_..m
1.31
6.04
1.90
7.50ga 16
3.77
11.06:1445
9.0
HCM-A
3.78
4.18-0.76
4.01
6 8.8920.34
2.08+0.00
21.91=1175
3. 3.184022
5.87
12.1
TAM
11.182075
11.0020.44
23 23.20a2.a0
10.57
8.40
16.4620.c
3.94
10.76a3.10
5.8
Unsapervised Pre- Train Fine- Tume
DOMINANT
2577
23.02
22. 22.741o.s
35.79 w.a
1.56.
17.60.144
3.84
1.48
4.0
CoLA
13.91
19.5
8.48.0.51
10.43-1.22
15.19=1100
8.03
4.07+.
7.27
6.8
HCM-A
4.76_0.51
6.8210.24
2,2511..
27.20 27.2015.53
3. 3.1011
5.6410.0
11.9
TAM
13.62
18.00
58 04:46
13.90=0.5a
11. 11:33:000
16.4720cs
3.93+2.09
11 1.856
4.1
Ours
ARC
49.3
45. 11.77.1.25
10.6216.10
36.0610.11
8.38.12.3
64.18
4.48.
44.25.7.41
1.9
while the GNN considers the graph structure and significantly improves performance on a variety of graphs with tabular features.
BWGNN [6] has spectral and spatial localized band-pass filters to better handle the "right- shift" phenomenon in anomalies, i.e., the distribution of spectral energy is concentrated at high frequencies rather than at low frequencies.
GHRN [37 is heterophily- supervised GAD method based on graph spectra, By emphasizing the tighffrequency components of the graph, the method can effectively cut down inter-class edges, thus improving the overall performance of anomaly detection.
For the unsupervised alternative, we consider representative SOTA GAD methods, each of them belonging to sub-type: data reconstruction, contrastive learning, hop-based auxiliary goal, or affinity-based auxiliary goal:
DOMINANT combines GCN and deep auto-encoder, and its learning objective is to reconstruct the adjacency matrix and node features jointly. It aims to identify structural and attribute anomalies based on reconstruction errors.
CoLA 15 contrastive self-supervised learning for anomaly detection on graphs with node attributes. The framework captures the relationship between each node and its neigh- borhood substructure in an unsupervised manner by sampling novel pairs of contrasting instances and leveraging the local information of the graph.
uses hop-count prediction as a self-supervised task to better identify anomalies by modeling both local and global context information. In addition, HCM-A designs two new anomaly scores and introduces Bayesian learning to train the model to capture anomalies.
HCM-A [16]
TAM [17] is designed based on one-class homophily and local affinity. The learning target of ('M is to optimize the proposed anomaly metric (i.e. affinity) end-to-end on the truncated adjacency matrix.
F33 Details of Implementation
Hyper-parameters. We select some key hyper-pann paremeeerreeerss of ARC through random search within specified grids. Specifically, the random search was performed within the following search space:
Hidden layer dimension: (64, 128, 256, 512, 1024)
Number of MLP layers: 21, 3, 4)
Propagation iteration: [1, 3, 4, 5)
Dropout rate: (0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8)
Learning rate: floats between 10-5 and 10-2
24

---

Table 5: Performance of ARC and its variants in terms of AUROC.
Variant
Cora
CiteSeer
ACM
BlogCatalog
Facebook
Weibo
Reddit
Amazon
ARC w/o
80.65+0.71
83.35+0.64
19.29
73. 13
62.80=2.00 os
89.69=0.17
54.60==192
64.76=2
ARC w/o
37.4411
31.5210.7
61.8311.16
29.301220
20.38
97.72.0.50
52.94
50.15.0.24
ARC w/o C
47.3920.12 42
53.9820.72
54.2421.52
60.4621.25
48.8620 o7
42.8423.01
51.0320.56
69.0220.97
ARC
87.4520.74
90.9510.5e
79.88
76..0.0
67.5611.00
88,85
60.04....0
80.67
Weight decay: floats between 10-6 and 10-3
Implementation Pipeline. We employ fixed set of hyper-parameters to build a generalist GAD model for all datasets. First, we train all the methods (including baselines and ARC) on the training set Thrain with full labels. Then, the methods are evaluated each dataset from Thest respectively. For feature projection, we employ the PCA algorithm to map the raw features into a fixed space with d, = 64. When the original feature dimension is smaller than the predefined projection dimension du. we use a random projection (e.g., Gaussian random projection) to upscale the feature into a higher dimensionality and then unify the dimensions into d,, with the projection strategy. For the baselines that require fine-tuning, we further conduct dataset-specifie tuning at this stage.
Metrics. Following 7 17 688 we employ two popular and complementary evaluation metrics for evaluation, including area under the receiver operating characteristic Curve (AUROC) and area under the precision-recall curve (AUPRC), A higher AUROC/AUPRC value indicates better performance. We report the average AUROC/AUPRC with standard deviations across trials.
Computing Infrastructures. We implemented the proposed ARC using Py Torch 2.1.2, Py PyTorch Geometric (PyG) 1, and DGL 0.9.0. All experiments were performed on a Linux server with an Inter Xcon microprocessor E-2288G CPU and Quadro RTX 6000 GPU.
G Supplemental Experiments
G.1 Performance Comparison in Terms of AUPRC
In terms of AUPRC, Table gives comprehensive comparative results with consistent observations with the AUROC results. Specifically, we have the following observations. 1 ARC still demonstrates strong anomaly detection in generalist GAD scenarios without any fine-tuning. Specifically, ARC achieves state-of- oftte-of-thharr the-art performance on five of the eight datasets and demonstrates competitive performance on the remaining datasets. On several datasets, ARC showed significant improvement over the best baseline (e.g.. f 131.1% on Cora, t 98.8% on Citeseer). GAD methods that only pre-train specific to dataset usually result in poor generalization to new datasets. Specifically, existing methods perform very erratically on different datasets, which can attributed to capturing only specific anomaly patterns. @ Using dataset-specific fine-tuning, baseline methods can achieve better performance in most case. However, in some cases the improvement can be small or even negative, demonstrating the limitations of fine-tuning.
G.2 Effectiveness of Context Sample Number
For all test sets, we varied n& in the range of 2 to 100 and the results are shown in Fig. and Fig. 9 From the figure, we observe that most cases the performance of ARC increases with the involvement of more context nodes, which indicates its ability to utilize these labeled normal nodes for context learning. Moreover, even ny is very small, ARC can still perform well on most datasets.
G.3 Detailed Results of Ablation Study
To assess the effectiveness of the key design in the ARC, we conducted an ablation study with three variants of the ARC, 1) w/o A: using random projection to replace smoothness-base feature alignment; 2) w/o R: using GCN to replace ego-neighbor residual graph encoder; and 3) w/o C: using binary classification-bant based predictor and loss to replace cross-attentive in-context anomaly scoring. As can be seen from Table 5 and Table the two metries AUROC and AUPRC of ARC achieved the best in all datasets except Weibo dataset. A possible explanation is that the Weibo dataset exhibits
25

---

47.5
80.0
40.8
75.0
36.50
91
CIPRCCC (8) AUPRO
AUROC (%)
(30 Sada
AUROC (15)
AUPRC (
40.7
AUROC
79.5
74.8
36.25
90
42.5
40.6
AL AURRC ALPRI
AUROC AUPRC
74.6
AUROC AUPRC
36,00
89
79.0
3
15
3
16
Number of Context Nodes
Number of Context Nodes II
Number of Context Nodes
(a) CiteSeer
(b) ACM
(c) BlogCatalog
89.0
4.6
AUROC (%)
AUROC (5)
AUPRC (%)
AUROC
4.4
88.5
(9)
AUPRC
AUROC AUPRO
4.2
AUPRC
58
Number Context Nodes
Number of Context Nodes
Number of Context Nodes
Weibo
(e) Reddit
(I) Amazon
Figure Performance with varying T& the rest of six datasets.
Table 6: Performance of ARC and its variants in terms of AUPRC.
Variant
Cora
CiteSeer
ACM
BlogCatalog
Facehook
Weibo
Reddit
Amazon
ARC w/o A
28. 11.77
29 2s
20.1310.41
34.23.0.47
4.15_a.3
67.3610.46
3.6211.16
10.3311.01
ARC
6.18
8.09-030
4.95
6.3220.44
1.50-0.1
92.07.1.0
3.59.0.07
6.92...19
ARC w/o
8.21
8.93=0.67
16. 86.0.91
26.87
4.95=150
12.21#1.18
3.74=0.10
13.09+1a1
ARC
69
25.77
40.6240.10
36.06=0.18
8.38-2.99
64.18
4.48+0.28
44.25=7.41
particular pattern of anomalies. In addition, all three key designs provide significant improvements in performance.
G.4 Visualization
The attention weights between context nodes and query nodes in other datasets are shown in Fig. 10 As can be seen in Fig. TO) for most of the datasets, the "single-class normal" case in Fig. (a) is met: ARC tends assign uniform attention weight normal nodes. This results in reconstructed embedding that are very similar to the average embedding of context nodes; in contrast, reconstructing few context nodes results in their embedding being farther from
anomalies using combination the center. Moreover, corresponding to the "multi-class normal" case in Fig. 4 (b): in Fig. (1), is observed that each normal query nodes that follow two fixed patterns. In summary, the cross- attention module enables ARC to adapt to various normal/abnormal distribution patterns, conferring generality.
26

---

Normal
Normal
Normal
Query Nodes
Query Nodes
Query Nodes
Anomaly
Anomaly
Anomaly
Context Nodes
Context Nodes
Context Nodes
(a) CiteSeer
(b) ACM
(c) BlogCatalog
Norma
Normal
Normal
Query Nodes
Query Nodes
Query Nodes
Anomaly
Anomaly
Anomaly
Context Nodes
Context Nodes
Context Nodes
(d) Facebook
(c) Weibo
(f) Reddit
Figure 10: Attention visualization results for more datasets.
27

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately match the experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of this paper have been discussed in Section 6 "Conclusion",
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
28

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We display the experimental instruction in the paper, provide the hyper- parameter search space, and upload the source code for reproduction the proposed
method.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
29

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: Yes, all the datasets are included along with the uploaded source code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips //ipss.cc/pill wttpcccpppppppppeeeeeeeeeeeeeeee httpa///guddeee/eeeeabiiissssss /Cc////oddiiissseeeeeeee ssioiiliccyyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
All the experimental details are given in Section 5 and Appendix F.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All the experimental results acquired by multiple trails of experiments, and we report the average and standard deviation results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
30

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, we provide the computing infrastructures in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
This research conforms with the Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no socictal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
31

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
32

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
33


Reviews:
[{"review_id": "mKBqJqdiGG", "reviewer": "Reviewer_RwhU", "summary": "The paper introduces an ARC method designed to detect anomalies across diverse graph datasets without the need for dataset-specific training. ARC leverages in-context learning with few-shot normal samples during inference, comprising three main components: a smoothness-based feature alignment module, an ego-neighbor residual graph encoder, and a cross-attentive in-context anomaly scoring module. The model demonstrates superior performance, efficiency, and generalizability compared to existing GAD methods, as validated by extensive experiments on multiple benchmark datasets.", "questions": "see the weaknesses above", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Graph anomaly detection is very challenging. This paper untangles this problem into three folds and solves them one by one, making the paper technically sound.\n\n2. The proposed ARC model is a “one-for-all” GAD model that is capable of identifying anomalies across target datasets from diverse domains, without the need for re-training or fine-tuning.", "weaknesses": "1. The paper's claim that it is \"capable of detecting abnormal nodes across diverse graph datasets from various application domains without any training on the specific target data\" is a bit ambitious. The method section did not provide a solid theoretical analysis to support this claim.\n\n2.  The statement in Line 102 should be moved to the introduction section to declare the scope of this work.\n\n3. The datasets used in the experiments are somewhat similar. It is recommended to include more diverse datasets from different backgrounds.\n\n4. The model relies on the availability of a few-shot normal samples for in-context learning. In some practical situations, obtaining these samples may be challenging, particularly in highly anomalous or unknown environments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate Reviewer RwhU for the perception of our contributions and thank the reviewer for the insightful feedback. The detailed responses are provided below.\n\n**Q1: Ambitious statement of the learning target**\n\n**A1:** Thank you for raising this valuable comment. While we cannot guarantee that the proposed method will perform well on every new dataset, it is capable of generating reliable predictions through in-context learning on unseen datasets. Our experiments also demonstrate the generalizability of the proposed method. To clarify the capabilities of ARC, we will carefully revise the statement to ensure it is more precise. \n\n**Q2: Declare the scope of this work**\n\n**A2:** Thank you for the insightful suggestion! In the revised manuscript, we will further emphasis the scope of this work (node-level generalist GAD) in the introduction section.\n\n**Q3: Diversity of datasets**\n\n**A3:** We appreciate the reviewer’s valuable suggestion. We recognize the importance of dataset diversity in the generalist GAD setting, and we have included datasets from three different domains (Citation network, Social network, and Co-review network) with both real and injected anomalies. To further evaluate the generalizability of ARC, we have added four additional datasets from four new domains as test datasets for ARC. These new datasets exhibit significant diversity compared to the training datasets, as they come from diverse, new, and unseen domains. The statistics are as follows.\n\n\n|**Name**|**\\#Nodes**|**\\#Edges**|**\\#Dim.**|**Anomaly**|**Anomaly\\_Type**|**Domain**|\n|:-|-:|-:|-:|-:|:-|:-|\n|Tolokers|11,758|519,000|10|21\\.80%|Real|Crowd-sourcing Service network|\n|T-Finance|39,357|21,222,543|10|4\\.60%|Real|Finance network|\n|CoAuthor CS|18,333|163,788|6,805|3\\.27%|Injected|Co-author network|\n|Amazon Photo|7,650|238,162|745|5\\.88%|Injected|Co-purchase network|\n\n\nThe AUROC comparison on the four new datasets is shown in the following table. As seen in the table, the proposed method ARC demonstrates competitive performance on all new datasets, showcasing its strong generalizability to unseen domains. Additionally, ARC exhibits excellent scalability on large datasets (e.g., T-Finance). We hope that these additional experiments on unseen domains and datasets address your concerns about dataset diversity.\n\n|method|CoAuthor CS|Amazon Photo|Tolokers|T-Finance|\n|:-|:-|:-|:-|:-|\n|DOMINANT(PT)|0.6061|0.4739|0.4812|OOM|\n|CoLA(PT)|0.7026|0.5608|0.5236|0.5180|\n|TAM(PT)|0.6995|0.5835|0.5051|OOM|\n|DOMINANT(PTFT)|0.7363|0.6060|0.4744|OOM|\n|CoLA(PTFT)|0.7927|0.6421|**0.5838**|0.2381|\n|TAM(PTFT)|0.6996|0.5728|0.5081|OOM|\n|**ARC**|**0.8273**|**0.7555**|0.5712|**0.6410**|\n\n**Q4: Requirement on few-shot samples**\n\n**A4:** Thank you for the thoughtful comment. In-context learning is a critical mechanism in ARC to ensure the model can receive basic patterns of test datasets. Since we only require a few normal samples, the annotation cost of ARC can be low. In parctice, it is easier to find some normal data in a real-world graph because they are more numerous and their patterns are more representative. Also, according the results in Fig.5, ARC can also work well when the shot number is quite small, which further shows its label efficiency.\n\nIn cases where labeled samples are not available, we can introduce a \"pseudo normal trick\". This involves randomly sampling nodes from the test graph to serve as \"pseudo normal samples\". Given the assumption that the number of normal samples is significantly larger than that of anomalies, most of the pseudo normal samples will actually be true normal samples. The advantage of having a larger number of normal samples reduces the impact of anomalies within the pseudo normal samples. We can then use these pseudo normal samples as context nodes in ARC.\n\n|*#Pseudo Normal*|**Cora**|**CiteSeer**|**ACM**|**BlogCatalog**|**Facebook**|**Weibo**|**Reddit**|**Amazon**|\n|:-|:-|:-|:-|:-|:-|:-|:-|:-|\n|2|0.8434|0.8991|0.7947|0.7422|0.6842|0.8775|0.5793|0.6321|\n|10|0.8665|0.9195|0.7962|0.7357|0.7049|0.8817|0.5843|0.6459|\n|30|0.8665|0.9206|0.7985|0.7425|0.6938|0.8376|0.5927|0.6412|\n|50|0.8673|0.9208|0.7980|0.7417|0.6949|0.8352|0.5923|0.6380|\n|100|0.8691|0.9213|0.7992|0.7409|0.6877|0.8264|0.5926|0.6420|\n|200|0.8690|0.9212|0.7989|0.7422|0.6843|0.7969|0.5938|0.6439|\n\nThe experimental results of using different numbers of pseudo normal samples are shown in the table above. Surprisingly, ARC demonstrates competitive performance with the pseudo normal trick. It is important to note that no labeled data is used at all in this case. However, for several datasets (e.g., Weibo and Amazon), the performance is still inferior to that of the few-shot counterpart, highlighting the importance of in-context normal samples in generalist GAD.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qZm3IUK5CR", "reviewer": "Reviewer_HCAf", "summary": "This paper proposed AARC, a generalist GAD approach to detect anomalies across various graph datasets on the fly. It consists of feature alignment module, residual graph encoder and in-context anomaly scoring module.", "questions": "1. The smoothness-based feature sorting is done in the raw input feature. In this case, after unifying the feature dimensionality with the feature projection, how do you smooth the new feature $\\tilde{X}^{(i)}$ with smoothness-based feature sorting? In another word, how do you aligns feature dimensionality and smooth feature at the same time?\n2. The design of cross-attentive in-context anomaly scoring is based on the assumption that the normal query have similar patterns to several context nodes and their embeddings can be easily represented by linear combination of context node embeddings. However, in many cases, given the limited number of  normal nodes, especially for the test graph, it's highly likely that some unsampled normal nodes cannot be represented by linear combination of other normal nodes. For instance, the sampled normal nodes from class 1 to class 5, while there exist some unsampled normal nodes from class 6. In this case, these unsampled normal nodes may not be represented by the linear combination of other normal nodes and these nodes tend to have large reconstruction error, thus being labeled as the abnormal nodes. How do you mitigate this issue?\n3. For the supervised baseline methods, what is the available label information for the test graphs. Based on the setting in the paper, it seems that only a small percentage of normal nodes are available for training for test graphs. In this case, how do you train these supervised baseline methods? What is the training procedure for these baseline methods?", "limitations": "The authors list limitations in conclusion.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The experimental results show that the proposed method outperform most of the baseline methods across various datasets.\n2. The presentation of this paper is good and the paper is easy to follow.", "weaknesses": "1. The smoothness-based feature sorting is done in the raw input feature. In this case, after unify the feature dimensionality with the feature projection, how do you smooth the new feature $\\hat{X}^{(i)}$ with smoothness-based feature sorting? In another word, how do you aligns dimensionality and smooth feature at the same time?\n2. In line 232 and 233, the authors mention that \"In the first two steps, we perform propagation on the aligned feature matrix $X'=X^{[0]}$ for $L$ iterations, and then conduct transformation on the raw and propagated features with a shared MLP network\". This is confusing. It's unclear weather the input feature is aligned feature matrix or the raw feature. Besides, the aligned feature matrix should be $\\tilde{X}$ rather than $X$.\n3. Equation 3 can be written as $Z^{[l]}=MLP(\\tilde{A}X^{[l-1]})=\\sigma(\\tilde{A}X^{[l-1]}W)$, where $W$ is the weight matrix of MLP and $\\sigma$ is the activation function. In this case, what's the difference between equation 3 and traditional GNN. The claim of equation 3 to capture the high-frequency signals and local heterophily is not convincing.\n4. The design of cross-attentive in-context anomaly scoring is based on the assumption that the normal query have similar patterns to several context nodes and their embeddings can be easily represented by linear combination of context node embeddings. However, in many cases, given the limited number of  normal nodes, especially for the test graph, it's highly likely that some unsampled normal nodes cannot be represented by linear combination of other normal nodes. For instance, the sampled normal nodes from class 1 to class 5, while there exist some unsampled normal nodes from class 6. In this case, these unsampled normal nodes may not be represented by the linear combination of other normal nodes and these nodes tend to have large reconstruction error, thus being labeled as the abnormal nodes. How do you mitigate this issue?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful to Reviewer HCAf for providing insightful feedback. The detailed responses are provided below.\n\n**Q1: Details of feature alignment with sorting**\n\n**A1:** Thank you for the thoughtful comment. In ARC, the smoothness-based feature sorting is performed on the projected features rather than the raw features (please see Line 209 and Algorithm 1 for details). In this context, the projected features of different datasets can be aligned in the same order. We understand that Equation (2) may mislead readers into thinking that $s_k$ is calculated from raw features; but actually, it can be calculated from projected features as well. We will further emphasize this to prevent such misunderstandings.\n\n**Q2: Confusing statement of \"raw features\"**\n\n**A2:** We appreciate the reviewer for pointing out the confusing sentence! This is a wrong statement. The right process is that \"the MLP transformation is conducted on the **aligned** and propagated features\", rather than the **raw** features. Here we denote the raw, projected, and aligned features as $\\mathbf{X}$, $\\tilde{\\mathbf{X}}$, and $\\mathbf{X}'$, respectively, so $\\mathbf{X}^{[0]}=\\mathbf{X}'$. Algo. 1 and Algo. 2 can better demonstrate the operation of this part. We thank the reviewer again for carefully finding this issue and will definitely fix it in the later version.\n\n**Q3: How the encoder capture high-frequency signals**\n\n**A3:** Thanks for the insightful comment. In ARC, Eq.(3) is similar to a traditional GNN and can be viewed as low-pass filters. However, with the first sub-formula in Eq.(4), i.e., $\\mathbf{R}^{[l]} = \\mathbf{Z}^{[l]} - \\mathbf{Z}^{[0]}$, the final residual representation $\\mathbf{R}^{[l]}$ can capture high-frequency signals. Specifically, if we omit the non-linear transformation in MLP, then $\\mathbf{R}^{[l]}=\\tilde{\\mathbf{A}}^l\\mathbf{X}\\mathbf{W} -  \\mathbf{X}\\mathbf{W}=(\\tilde{\\mathbf{A}}^l-\\mathbf{I})\\mathbf{X}\\mathbf{W}$. Here, $(\\tilde{\\mathbf{A}}^l-\\mathbf{I})$ can typically serve as a high-pass filter according to [\\*1, \\*2]. Especially when $l=1$, $(\\tilde{\\mathbf{A}}-\\mathbf{I})$ can serve as Laplacian graph filter, a classic high-pass filter. A detailed discussion is also given in the final paragraph of Appendix C. We hope this answer can address the reviewer's concern. \n\n*[\\*1] Zhu, Meiqi, et al. \"Interpreting and unifying graph neural networks with an optimization framework.\" WebConf'21.* \n\n*[\\*2] Bo, Deyu, et al. \"Beyond low-frequency information in graph convolutional networks.\" AAAI'21.*\n\n**Q4: Few-shot setting meets unseen normal class**\n\n**A4:** Thanks for raising this valuable question! Theoretically, the unseen classes may lead to confusing predictions; however, in practice, we find that this issue does not affect the performance a lot. Taking the performance on Cora dataset (which has 7 normal classes) as an example (see Fig. 5 (a)), when shot number $n_k$ is smaller than 7, which means the labeled samples must not cover all classes, the performance is still acceptable and significantly better than the baselines in Table 1. Potential reasons are: 1) the difference between anomalies and normal samples is significantly larger than the intra-class difference of normal samples; 2) the normal samples belonging to unseen classes can be easily represented by the few-shot normal samples.\n\nHowever, to mitigate the potential negative impact of this issue, we can also employ the following extra strategies: 1) Increase the number of few-shot normal samples as much as possible to cover all normal classes; 2) Enhance the diversity of the few-shot normal samples; 3) Integrate active learning techniques into ARC, which allows the model to select the most confusing sample to be annotated.\n\n\n**Q5: Implementation details of supervised baselines**\n\n**A5:** Thanks for the valuable question. As the reviewer mentioned, the supervised baselines require both labeled normal samples and anomalies for training. However, in the setting of generalist anomaly detection, only few-shot normal samples are available, making it difficult to fine-tune the supervised baselines on the testing datasets. Hence, we only pre-train them on the training datasets and directly apply the pre-trained models to the testing datasets. Note that, in the datasets for pre-training, both labeled anomalies and normal samples are available, ensuring effective pre-training for supervised baselines and ARC. We will add the implementation details to the revised manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PDHYgGAxQW", "reviewer": "Reviewer_bLtA", "summary": "This paper investigates the research problem of \"generalist graph anomaly detection (GAD)\", targeting to address the cost and generalizability issues of the conventional GAD paradigm. This paper proposes ARC, a \"one-for-all\" GAD model that is pre-trained on a group of datasets and able to detect anomalies on new datasets on-the-fly. To verify the performance of ARC, experiments on GAD benchmark datasets are executed. The main contributions of this paper are: 1. Addressing a new research problem (generalist GAD); 2. Developing a novel method; 3. Conducting experiments for GAD performance validation.", "questions": "The questions listed in Weaknesses are expected to be answered. Meanwhile, I have some extra questions:\n\n1. Why \"pre-train & fine-tune\" settings only include unsupervised methods? \n\n2. Will ARC be time-costly when the number of context nodes become large?", "limitations": "From my perspective, the authors have addressed the limitations and the paper seems doesn't have potential negative impact.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality: the proposed method has great originality. The three components in ARC are fresh and novel. The authors further discuss the difference and connection between ARC and other method, which is sufficient to show the originality of this paper.\n\nQuality: This paper is of high quality. The research problem and proposed method are both solid. Comprehensive experiments and discussions are given in the paper (some are in the appendix).\n\nClarity: The presentation of this paper is good, making it easy to read. Most of the equations, description, figures, and tables are good to understand. \n\nSignificance: I'm not an expert of GAD domain, but I think this paper make good significance in the community. Considering the significant difference between the proposed generalist GAD paradigm and the existing one, I think think paper is an important step towards AGI.", "weaknesses": "Methodology details: Some specific details of the method design are not clearly given. In specific: \n\n1. For the feature projection, what if the original feature dimension is smaller than the predefined projected dimension? \n\n2. Why doesn't ARC use the value-based cross attention? Is there any disadvantage?\n\nExperiment details: Although the experiments are extensive, I think more discussions should be given for some notable results. In specific: \n\n1. Why unsupervised methods perform better than the supervised methods? \n\n2. Why Figure 5 (b) has a dropping trend at nk=20?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate Reviewer bLtA for the positive review and constructive comments. We provide our responses as follows.\n\n**Q1: If the original feature dimension is smaller than the predefined projected dimension**\n\n**A1:** Thanks for the thoughtful comment! When the original feature dimension is smaller than the predefined projection dimension, we use a random projection (e.g., Gaussian random projection) to upscale the feature into a higher dimensionality and then unify the dimensions into $d_u$ with the projection strategy. We will add these implementation details to the manuscript.\n\n**Q2: The reason for using non-value-based cross attention**\n\n**A2:** Thanks for the valuable question. As shown in the Discussion in Sec. 4.3 of the paper, in ARC we follow a basic assumption: normal query nodes have similar patterns to several context nodes, and hence their embeddings can be easily represented by the linear combination of context node embeddings. Therefore, the node anomaly score is obtained based on the error before and after the reconstruction. In this case, it is necessary to ensure that the reconstructed features are in the same embedding space as the original features, so we discard the value-based attention. We will add this explanation to the paper.\n\n**Q3: Why unsupervised methods perform better than the supervised methods**\n\n**A3:** We appreciate your insightful question. The possible reasons are two-fold. Firstly, most supervised GAD methods rely on the binary classification paradigm. This means that supervised GAD is trained based on specific anomalies in the training data, which may result in overfitting the anomaly patterns of the trained dataset. In addition, supervised GAD cannot be fine-tuned on the test set using \"normal\" samples, which also limits its generalizability to new-coming datasets.\n\n**Q4: Why Figure 5 (b) has a dropping trend at $n_k$=20?**\n\n**A4:** Thanks for your careful review. As in-context node information is variable, the performance of anomaly detection can be affected by the quality of context node information. In our experiments, we sample normal nodes randomly, so there is randomness due to sample-specific sampling bias. This randomness leads to instability in the model's performance at a particular number of context nodes. Despite this, we can still observe that the overall trend of performance is gradually increasing with increasing $n_k$.\n\n**Q5: Why \"pre-train & fine-tune\" only includes unsupervised methods**\n\n**A5:** Thanks for your insightful comment. To fine-tune the supervised GAD methods on the target dataset, they require both labeled normal samples and anomalies. However, the labeled anomalous samples are not available in our test scenario for fine-tuning by supervised methods. On the other hand, unsupervised methods are label-free and thus can be fine-tuned on the testing dataset.\n\n**Q6: Does large number of context nodes cause ARC to become time-costly?**\n\n**A6:** Thanks for the valuable question. As can be seen from the section Complexity Analysis in Appendix E.2 of the paper, the model inference mainly consists of two parts, node embedding generation and anomaly scoring, $\\mathcal{O}(L(md_u + nd_uh + nh^2))$ and $\\mathcal{O}(n_q(n_k+1)h)$, respectively. It is worth noting that $n_k \\ll n$, thus complexity mainly depends on the node embedding generation. Therefore, a larger number of context nodes does not make ARC time-costly. \n\nIn addition, we compare the test times (in second) of ARC for datasets with different numbers of context nodes, and the results are shown in the table below. We can see that the effect of different numbers of context nodes on ARC is almost negligible, indicating the high running efficinecy of ARC when $n_k$ is large.\n\n|$n_k$|**200**|**400**|**1000**|**2000**|\n|:-|:-|:-|:-|:-|\n|ACM|0\\.0301|0\\.031|0\\.0319|0\\.0344|\n|Facebook|0\\.0071|0\\.0075|0\\.0072|0\\.0072|\n|Reddit|0\\.0224|0\\.0224|0\\.0236|0\\.025|\n|citeseer|0\\.011|0\\.0101|0\\.0102|0\\.0102|", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1wSOEns6l0", "reviewer": "Reviewer_VWTk", "summary": "This paper proposes a new framework for graph anomaly detection called ARC. ARC uses in-context learning to detect anomalies across various graph datasets without requiring retraining or fine-tuning. It leverages few-shot normal samples during inference to achieve superior performance in anomaly detection tasks.", "questions": "I still have some minor questions.\n\n1. How to define the projected dimension $d_u$? \n\n2. Considering the cost of fine-tuning (figure 6), will it require a lot of epochs to get the fine-tuned results for the baseline methods?", "limitations": "The limitations are upfront in the paper.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "Overall, this paper explores an interesting research topic with a well-designed method. The experimental results are also promising. \n\n1. Methodology design. The design of the proposed method ARC is reasonable, where each component address a challenge in generalized graph anomaly detection.\n\n2. Promising results. The experimental results show that indeed the ARC model can achieve better performance on a series of datasets.", "weaknesses": "However, I have several concerns about this paper.\n\n1. In the residual GNN component, a shared MLP is used for feature transformation, which is not vey commonly seen in other GNNs. Could you provide more intuition behind for explanation?\n\n2. In attention mechanism, \"value\" is an important component in the computational process. However, the in-context learning module doesn't include the computation of \"value\". Can you give more detailed intuition for this design?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate Reviewer VWTk for the valuable feedback and acknowledge our technical contributions and the effectiveness of the proposed method. We address the concerns raised by the reviewer as follows.\n\n**Q1:Why Use a shared MLP in the encoder**\n\n**A1:** Thanks for the insightful comment! According to the empirical analysis in the previous studies [\\*3,\\*4], GNNs benefit mainly from propagation and require only a small number of MLPs to increase their nonlinear capabilities. Therefore, we increase the nonlinear capability of the propagating features for different neighborhoods by a shared MLP. In addition, the design of the shared MLP allows the model parameters to be drastically reduced, especially when the number of propagations is large.\n\n*[\\*3] Wu F, Souza A, Zhang T, et al. Simplifying graph convolutional networks[C]//International conference on machine learning. PMLR, 2019: 6861-6871.* \n\n*[\\*4] Nt H, Maehara T. Revisiting graph neural networks: All we have is low-pass filters[J]. arXiv preprint arXiv:1905.09550, 2019.* \n\n**Q2: Reason for value-based attention mechanism**\n\n**A2:** Thanks for the valuable question! As shown in the Discussion in Sec. 4.3 of the paper, in ARC we follow a basic assumption: normal query nodes have similar patterns to several context nodes, and hence their embeddings can be easily represented by the linear combination of context node embeddings. Therefore, the node anomaly score is obtained based on the error before and after the reconstruction. In this case, it is necessary to ensure that the reconstructed features are in the same embedding space as the original features, so we discard the value-based attention. We will add this explanation to the paper.\n\n**Q3: How to define the projected dimension?**\n\n**A3:** Thanks for your insightful comment. The projection size of our ARC model is flexible and can be determined based on a balance between computational efficiency and the ability to capture the underlying anomaly-related features. Specifically, we perform a grid search over a range of dimensions and select the one that yields the best loss on a training set. This approach ensures that the chosen dimension is optimal for capturing the necessary information while maintaining computational feasibility.\n\n**Q4: Will it require a lot of epochs to get the fine-tuned results for the baseline methods?**\n\n**A4:** We appreciate your insightful question. The cost of fine-tuning for baseline methods indeed varies depending on the complexity of the model and the dataset. In our experiments, we observed that baseline methods generally require a significant number of epochs to achieve optimal performance, which contributes to their high training costs. In contrast, our ARC model leverages in-context learning with few-shot normal samples, allowing it to adapt to new datasets without extensive retraining or fine-tuning. This results in a more efficient and cost-effective approach to graph anomaly detection.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wG6q00wbI0", "reviewer": "Reviewer_B6wn", "summary": "The paper produces a \"one-for-all\" generalist graph anomaly detection model. The proposed model first use feature projection and smoothness-based ranking to align the features of multiple datasets; then it employs a residual graph neural network to extract representations; finally, a cross-attention module is used to calculate the anomaly scores of query nodes. The paper is well motivated and with sufficient experiments for effectiveness validation.", "questions": "1. In Eq. 6, the anomaly label is denoted by a bold letter y. Does that mean it's a vector? However, should it be a value actually?\n2. What is the reason for the unstable anomaly detection performance under different number of context nodes?", "limitations": "I believe the authors have addressed the limitation, and I haven't found any negative societal impact of this work.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The setting of generalist graph anomaly detection is interesting and novel. The research question in this paper has high practice value.\n2. The discussion of feature smoothness and abnormality is insightful, which may also inspires future studies.\n3. The proposed method is carefully designed with three powerful modules. Each of them plays a critical role in capturing anomaly-aware knowledge and making precise prediction.\n4. The experimental results show that the proposed method has significant performance advantage over the baselines.\n5. The visualization experiment is interesting, which show the working mechanism of the attention module.", "weaknesses": "1. The methodology details of the proposed method are not precise: specifically, which type of feature projection method is used in the experiment should be given.\n2. The definitions of the concept of single-class normal and multi-class normal are missing, which hinders the readers to understand the design motivation in Sec. 4.3. \n3. The figure is not readable enough. More annotations can be provided in Figure 2, especially for the R and C blocks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate Reviewer B6wn for the insightful feedback and acknowledge our contributions. The responses to the reviewer are as follows.\n\n**Q1: Which type of feature projection method is used in the experiment should be given**\n\n**A1:** Thanks for your valuable suggestion. In our experiments, we used a dimensionality reduction method based on principal component analysis (PCA). We will re-emphasize this in the manuscript.\n\n**Q2: The definitions of the concept of single-class normal and multi-class normal**\n\n**A2:** We appreciate your suggestion. We will add the following definitions in the revised manuscript:\n\n**Dataset with single-class normal:** In this type of dataset, the normal samples share the same pattern or characteristics. For example, in a network traffic monitoring system dataset, normal behavior might be defined by regular patterns of data packets exchanged between a specific set of IP addresses. Any deviation from this single, well-defined pattern, such as an unexpected spike in data volume or communication with unknown IP addresses, can be flagged as anomalous.\n\n**Dataset with multi-class normal:** In this type of dataset, the normal samples are divided into multiple classes, each with distinct patterns or characteristics. For example, in a corporate email communication network dataset, normal data might be defined by regular patterns of email exchanges within specific departments, such as HR, IT, and Finance. Any deviation from these well-defined patterns, such as a sudden spike in emails between normally unconnected departments or an unusual volume of emails from an individual employee to external addresses, can be detected as anomalous.\n\n**Q3: More annotations can be provided in Figure 2**\n\n**A3:** Thanks for the valuable comment! We will add more annotations into Figure 2, including the annotations of $\\mathbf{X}$, $\\mathbf{R}$, and $\\mathbf{H}$ in the \"R\" block, as well as the annotations of $\\mathbf{Q}$, $\\mathbf{K}$, $\\mathbf{H}_q$, and $\\tilde{\\mathbf{H}}_q$ in the \"C\" block.\n\n**Q4: In Eq. 6, the anomaly label is denoted by a bold letter y. Does that mean it's a vector? However, should it be a value actually?**\n\n**A4:** Thanks for your careful review. In fact, $\\mathbf{y}_{i}$ is the label of sample (node) i, which is a specific value, and $\\mathbf{y}$ is a vector consisting of the labels for all samples.\n\n**Q5: The reason for the unstable performance under different number of context nodes**\n\n**A5:** Thanks for the valuable question! The instability in anomaly detection performance with varying numbers of context nodes can be attributed to several factors: \n\n**Contextual Information Variability:** The number of context nodes directly influences the amount of contextual information available for anomaly detection. With too few context nodes, the model may lack sufficient information to accurately distinguish between normal and anomalous nodes. Conversely, with too many context nodes, the model might include irrelevant or noisy information, which can degrade performance.\n\n**Dataset Characteristics:** Different datasets have varying structures and anomaly distributions, which can affect how context nodes contribute to anomaly detection. The optimal number of context nodes may vary depending on the specific characteristics of each dataset.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all the reviewers for their valuable and insightful comments. We are glad that the reviewers find that the studied problem is novel and significant (Reviewer RwhU, bLtA, VWTk, and B6wn), the proposed method is novel and well-motivated (Reviewer bLtA, VWTk, and B6wn), the empirical studies are adequate and reasonable (Reviewer HCAf, bLtA, VWTk, and B6wn), and the writing is smooth and has a good storyline (Reviewer HCAf, and bLtA). 

To the best of our efforts, we provided detailed responses to address the concerns raised by each reviewer in the following. Meanwhile, we carefully revised the paper according to the reviewers’ comments. We will incorporate all the feedback in the final version.

Specifically, the main modifications we made are as follows.
﻿
* We added additional experiments to validate the effectiveness of the proposed method on a wider range of datasets from different domains (see the reply A3 to Reviewer RwhU for details).
* We added additional experiments to discuss the correlation between the time complexity of the proposed method and the number of context samples (see the reply A6 to Reviewer bLtA for details). The discussion shows that the complexity of the method is almost independent of the number of context samples.
* We added additional experiments discussing the robustness of the model in the purely unsupervised scenario, i.e., zero-shot case (see the reply A4 to Reviewer RwhU for details).
* We illustrated the implementation details of the proposed methodology as well as the design intuition of the model (see the replies to Reviewer HCAf, bLtA, VWTk, and B6wn for details).
* We illustrated the implementation details of the baselines (see the replies to Reviewer HCAf and bLtA for details).

Meta-review: This paper proposes a graph anomaly detection approach that identifies anomalies on-the-fly using only few-shot normal nodes. The reviewers appreciated the clear presentation, noting the high originality and generality of the proposed approach, as well as the extensive experiments conducted. Given that several reviewers championed the paper, I recommend acceptance.
Decision: Accept (poster)