Towards Effective Federated Graph Anomaly Detection via Self-boosted Knowledge Distillation

Authors: jinyu  cai
yunhe  zhang
zhoumin  lu
wenzhong  guo
see-kiong  ng
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 04:48:52

Abstract: Graph anomaly detection (GAD) aims to identify anomalous graphs that significantly deviate from other ones, which has raised growing attention due to the broad existence and complexity of graph-structured data in many real-world scenarios. However, existing GAD methods usually execute with centralized training, which may lead to privacy leakage risk in some sensitive cases, thereby impeding collaboration among organizations seeking to collectively develop robust GAD models. Although federated learning offers a promising solution, the prevalent non-IID problems and high communication costs present significant challenges, particularly pronounced in collaborations with graph data distributed among different participants. To tackle these challenges, we propose an effective federated graph anomaly detection framework (FGAD). We first introduce an anomaly generator to perturb the normal graphs to be anomalous and train a powerful anomaly detector by distinguishing generated anomalous graphs from normal ones. We subsequently leverage a student model to distill knowledge from the trained anomaly detector (teacher model), which aims to maintain the personality of local models and alleviate the adverse impact of non-IID problems. Additionally, we design an effective collaborative learning mechanism that facilitates the personalization preservation of local models and significantly reduces communication costs among clients. Empirical results of diverse GAD tasks demonstrate the superiority and efficiency of FGAD.

Main paper: 
 introduction: 
 Anomaly detection [5, 35] is a fundamental research problem in machine learning, which has been extensively explored in various domains, e.g., images [2, 17] and time-series data [1, 6] . In the real world, graph-structured data [21-23, 34, 44] is commonly available due to its exceptional ability to represent complicated relationship information among entities [29, 30, 54] . This is particularly evident in domains like social networks and medical applications. In return, graph anomaly detection (GAD) [7, 32] , which aims to identify graphs that exhibit significant deviations from other normal graphs, has raised broad attention in recent years. With the advancement of graph neural networks (GNNs) [4, 15, 47] , GAD has made remarkable strides and demonstrated promising performance in detecting anomalies across many real-world scenarios with natural graph-structured data, e.g., social networks and bioinformatics.In realistic collaborative efforts among different companies and organizations, a common demand is an attempt to share their knowledge to detect anomalies more accurately. Although existing GAD approaches [11, 31, 39, 50] simplify coordination with centralized training, it introduces a critical privacy leakage risk as it typically requires all participants to provide their own data to train a global model, as shown in Figure 1(a) . Graph data may encompass sensitive information that the participants are not willing to share, e.g., private relationships in social networks, which will hinder collaborations. Therefore, an urgent imperative emerges to investigate approaches that facilitate collaboration between GAD models distributed to different participants while protecting their privacy.As the emerging technique in machine learning, federated learning (FL), as shown in Figure 1 with the server rather than their local data, which will prevent the leakage of sensitive information in participants. Classical FL methods, such as FedAvg [33] and FedProx [20] , have become the paradigm of collaborative learning across various domains [16, 45] .To facilitate collaborative training of GNN models for graph data across clients, federated graph learning (FGL) [10, 24, 51] has also been widely studied in recent years. FGL methods [38, 46] integrate GNNs with FL methods to collaboratively learn representations for complicated graph data distributed in various clients, and have demonstrated superiority in many downstream tasks, e.g., graph classification. Hence, an intuitive approach to address the above issue is to integrate the existing advancements in FL and FGL with general anomaly detection techniques, e.g., deep one-class classification (DeepSVDD) [37] . However, this solution may encounter the following challenges:(1) The graph data distributed in various clients often exhibit significant heterogeneity and non-IID property [12, 46] , e.g., containing different graph structures or feature dimensions. These factors place a higher demand on maintaining the validity of the local models for their own data, e.g., personalization. (2) It is difficult to learn a universal hypersphere as the decision boundary for highly heterogeneous graph data under the federated learning setting. Besides, such non-IID graphs across clients hardly conform to the assumption in DeepSVDD that their latent distribution could follow a universal hypersphere. (3) Existing collaborative learning mechanisms, e.g., FedAvg [33] ,require transmitting all network parameters of each client in a single communication round, which brings substantial communication costs in applications.These challenges naturally lead us to a research question: Can we design an FL-based collaborative GAD framework to detect anomalous graphs with non-IID properties effectively?In this paper, we propose an effective federated graph anomaly detection (FGAD) framework, as shown in Figure 2 , to answer this research question. To improve the anomaly detection capability in the local model, we introduce an anomaly generator that perturbs normal graphs to be anomalous, and unsupervised train a classifier to identify anomalies from normal graphs. The generated anomalous graphs are encouraged to be diverse but resemble normal ones through iterations, so that more robust decision boundaries can be learned in a self-boosted manner. To alleviate the adverse impact of non-IID problems, we propose to preserve the personalization of each client by leveraging knowledge distillation. Specifically, we introduce a student model to distill the knowledge from the trained teacher anomaly detector. The student model only takes the normal graphs as the input, with the aim of aligning its predicted distributions with that of the teacher model. Moreover, we further design an effective collaborative learning mechanism, in which the student and teacher models share the same backbone network to streamline the capacity of local models. Furthermore, we engage only the parameters of the student head rather than the entire model in collaborative learning, which allows the teacher model to preserve the personalization of a client. In this way, we not only alleviate the adverse impact of non-IID property, but also reduce the communication costs between clients and server during collaborative learning. The contributions of this paper are:• We, for the first time, explore the GAD problem on non-IID graphs under federated learning setup, and propose an effective federated graph anomaly detection (FGAD) method. • We introduce a self-boosted distillation module, which not only promotes the detecting capability by identifying self-generated anomalies, but also maintains the personalization of local models from knowledge distillation to alleviate non-IID problems. • We propose an effective collaborative learning mechanism that streamlines the capacity of local models and reduces communication costs with the server. • We establish a comprehensive set of baselines for federated graph anomaly detection. Extensive experiments also validate the effectiveness of our FGAD. 
related works 2.1 graph anomaly detection: 
 Graph anomaly detection (GAD) [3, 26, 32, 42, 52] refers to detecting abnormal graphs that significantly differ from other normal ones, which have received growing attention in recent years owing to the ubiquitous prevalence of graph-structured data in real-world scenarios, e.g., social networks [25, 28] . There are many works that advance the research on GAD. For instance, Zhao et al. [53] investigated graph-level anomaly detection issues by integrating graph isomorphism network (GIN) [47] with deep one-class classification (DeepSVDD) [37] . Qiu et al. [36] leveraged neural transformation learning to develop a more robust GAD model to overcome the performance flip issue. Ma et al. [31] utilized knowledge distillation to capture more comprehensive normal patterns from the global and local views for detecting graph anomalies.Although these GAD methods have achieved remarkable success, they primarily rely on a centralized training paradigm. In real-world collaborative scenarios, the graph data is often distributed across various clients from data owners, which necessitates the transmission of local graph data to a central server during their practical collaborations. Unfortunately, this process can potentially expose sensitive information and pose severe privacy risks. Additionally, the inherent non-IID property in the graph data distributed across diverse clients presents yet another formidable challenge. As a result, effective solutions to address these challenges remain an open research problem. Federated learning (FL) approaches [12, 19, 49] , such as FedAvg [33] , FedProx [20] , provide a promising solution for collaboratively training models with data distributed in different clients, while preserving their privacy. In FL, clients only share their network parameters rather than data with the central server, which mitigates the privacy leakage risk and enables clients to share and leverage knowledge from others. As an emerging technique, FL has not only made remarkable advancements in handling image [8, 18, 48] and time series data [27, 41] , but also raised attention to graph data [43, 51] , where collaborative efforts are significantly more challenging due to the complex structural information and heterogeneous characteristic of graphs compared to other data types. Federated graph learning (FGL) [51] aims to facilitate the collaboration of GNNs distributed in multiple remote clients to meet the requirement of handling complicated non-IID graph data that widely exist in many real-world scenarios, e.g., social networks, medical, and biological data. For example, Xie et al. [46] studied the federated learning issue on non-IID graphs by integrating clustered federated learning with GIN, which achieves effective collaborations for distributed GINs. Tan et al. [38] designed a structural knowledge-sharing mechanism to facilitate the federated graph learning process. Although existing FGL methods have been validated for many tasks, such as graph classification, their effectiveness in addressing the intricate unsupervised graph anomaly detection remains an ongoing area to be explored. While it is possible to extend these FL/FGL [20, 33, 38, 46] methods to address GAD tasks by integrating them with classical anomaly detection methods like DeepSVDD [32, 37] , it is imperative to acknowledge some significant challenges, e.g., the adverse impact of the non-IID problem across different clients and the communication costs of transmitting complex GNN model parameters during collaborative learning. 
methodology 3.1 preliminary and problem formulation: 
 Notation: Let 𝐷={𝐺 1 , . . . , 𝐺 𝑁 } denote a graph dataset which consists of 𝑁 graphs, and each graph 𝐺 𝑖 ={𝑉 𝑖 , 𝐸 𝑖 } in the graph set comprises a node set 𝑉 𝑖 and edge set 𝐸 𝑖 . Typically, assume the number of nodes in a graph 𝐺 𝑖 is 𝑛 𝑖 =|𝑉 𝑖 |, an adjacency matrix A 𝑖 ∈{0, 1} 𝑛 𝑖 ×𝑛 𝑖 is used to represent the topology of graph 𝐺 𝑖 . Besides, let x 𝑣 ∈R 𝑑 denote the attribute vector for node 𝑣 ∈𝑉 𝑖 , X 𝑖 ∈R 𝑛 𝑖 ×𝑑 is used to represent the attribute matrix of graph 𝐺 𝑖 .Graph Neural Networks: Graph neural networks (GNNs), which iteratively learn representations with neighborhood aggregation and message propagation, is a widely used paradigm of learning representation for graph-structured data in many downstream tasks. In this paper, we leverage the graph isomorphism network (GIN) [47] , a widely used GNN backbone, to learn graph representation for anomaly detection tasks. Generally, in each layer of a GIN, the node representation is updated by aggregating its neighborhood information. For instance, in the 𝑘-th layer of GIN, the learned aggregated features a (𝑘 ) 𝑣 for node 𝑣 can be formulated as:EQUATIONwhere AGGREGATE(•) indicates the aggregation function, and Ñ (𝑣) represents the neighbor node set of node 𝑣. Then, the node feature h(𝑘 ) 𝑣 in the 𝑘-th layer is obtained by combing the node feature learned in the (𝑘 -1)-th layer with the aggregated feature:EQUATIONwhere 𝜎 (•) denotes the activation function, e.g., LeakyReLU. Particularly, the initial feature h(0) 𝑣 for node 𝑣 is set as h (0)𝑣 =x 𝑣 . Consequently, we can obtain the representation for a graph 𝐺 based on the learned features of all nodes within 𝐺 as follows:EQUATIONwhere 𝐾 is the number of GIN layers, and CONCAT(•) denotes the concatenate operation that stacks the graph representation learned across all 𝐾 layers. R (•) denotes the readout function that obtains the graph-level representation by aggregating the node features within a graph, and we choose sum-readout in this paper. Note that for convenience, we use GIN(•) to simply represent a GIN model containing the above three operations in the following sections.Problem Formulation: The objective of the GAD under the FL setup is to facilitate collaboration among clients, which allows each participant to enhance their GAD models by leveraging knowledge from others without exposing private data. Given 𝐶 clients, the collective graph dataset is denoted as 𝐷={𝐷 𝑐 } 𝐶 𝑐=1 , where each client possesses its own graph set 𝐷 𝑐 . A prevalent paradigm in GAD [32] is that all graphs within the client, i.e., ∀𝐺 𝑖 ∈𝐷 𝑐 , are deemed as "normal". The model is trained to capture this normality so that the trained model can identify when an "anomalous" graph G deviates significantly from the distribution of 𝐷 𝑐 by some pre-defined assumptions, e.g., the hypersphere decision boundary in DeepSVDD [37] . On the contrary, in this paper, we attempt to develop an anomaly detector that can adaptively learn decision boundaries rather than relying on the strong assumption of the shape of the latent distribution. This can be regarded as solving: minimizeEQUATIONwhere 𝑐 ) . However, tackling this problem presents the following challenges: 1) GAD is generally an unsupervised task in which only normal graphs (w/o labels) are accessible. How will we produce highquality anomalous graphs for training local anomaly detectors? 2) In the context of FL-based GAD, how can we alleviate the adverse impact of the prevalent non-IID problem across clients? 3) Transmitting all network parameters following conventional FL methods may limit the scalability given the complexity of GIN. Therefore, how do we reduce communication costs in collaborative learning while maintaining the validity of local models?The first challenge here is to produce anomalous graphs without using any supervised information. To resolve this, we propose a graph anomaly generator, denoted as G w 𝑎 (•), to generate anomalous graphs by perturbing the graph structure of normal graph 𝐺. For each client, we aim to generate an anomalous graph set D𝑐 ={X 𝑐 , Ã𝑐 } in an unsupervised manner by feeding it with normal graph set 𝐷 𝑐1 . To ensure diversity in the generated anomalous graphs, we utilize variational graph auto-encoder (VGAE) [14] to build the anomaly generator. Specifically, we first learn a latent Gaussian distribution N (𝝁 𝑐 , 𝝈 2 𝑐 ), which can be determined as:EQUATIONwhere GIN 𝝁 (•) and GIN 𝝈 (•) denote two distinct GINs in anomaly generator, and 𝝁 𝑐 and 𝝈 𝑐 explicitly parameterize the following inference model:EQUATIONwhere 𝑞(Z(𝑖) 𝑐 |X 𝑐 , A 𝑐 )=N ( Z(𝑖) 𝑐 |𝝁 (𝑖 ) 𝑐 , diag(𝝈 (𝑖 ) 2 𝑐)), and it allows us to sample from a wide range in the latent space thereby facilitating the diverse anomalous graph generation. Here, we employ the reparametrization trick [13] to address the obstacle of gradient propagation in the sample operation. Consequently, the generated adjacency matrix can be calculated by: Ã𝑐 = T ( Z⊤ 𝑐 Z𝑐 ), Z𝑐 = 𝝁 𝑐 + 𝜖 exp(𝝈 𝑐 ), 𝜖 ∼ N (0, 1),where T : R→[0, 1] represents the element-wise transformation operations such as Sigmoid(•), and 𝜖 represents a random Gaussian noise that follows the standard normal distribution N (0, 1).Intuitively, allowing the generated graphs to closely resemble normal graphs while remaining as anomalies is beneficial in training a robust and powerful anomaly detector, as it forces the model to distinguish those subtle deviations from the normal patterns. Therefore, we propose to optimize the anomaly generator by minimizing the following objective:EQUATIONwhere A𝑐 denotes the presence (1) or absence (0) of an edge between nodes 𝑖 and 𝑗, ℓ 𝑐 g is the binary-cross entropy loss function. Subsequently, we can train an anomaly detector with the normal and generated anomalous graph sets for the local client as follows: (9) where Ã𝑐 =G w 𝑎 (X 𝑐 , A 𝑐 ), 𝑙 ce (•) is the cross-entropy loss, and 𝑓 w 𝑔 (•) denotes the GIN backbone that learns graph representation by feeding with graph data. Proj(•) is the MLP-based projection head that maps the graph representation learned from 𝑓 w 𝑔 (•) into the predicted logits. Notably, we simply set pseudo labels for the normal graphs and the generated anomalous graphs, i.e., 𝑦 𝑐 =1 and ỹ𝑐 =0.ℓ 𝑐 ad = 𝑙 ce (𝑦 𝑐 , Proj(𝑓 w 𝑔 (X 𝑐 , A 𝑐 ))) + 𝑙 ce ( ỹ𝑐 , Proj(𝑓 w 𝑔 (X 𝑐 , Ã𝑐 ))),Hence, we can train an anomaly detector in an unsupervised manner by minimizing the following objective function:EQUATIONwhere ℓ 𝑐 g attempts to generate anomalous graphs that closely resemble normal ones, while ℓ 𝑐 ad aims to identify those generated anomalous graphs. Therefore, we produce diverse anomalous graphs to learn a powerful anomaly detector in such a self-boosted style, and the two objectives mutually improve each other during training.However, in the context of federated learning, the graph data across different clients are often heterogeneous and exhibit non-IID property. Such characteristics can potentially affect the anomaly detection performance of local models, i.e., the second challenge. To alleviate the adverse impact of the non-IID problem, we propose a graph knowledge distillation framework, which is designed to preserve the personalization of the local model during collaborative learning. Specifically, we regard the previously pre-trained anomaly detector as the "teacher" model, and introduce a "student" model that aims to distill the knowledge from the teacher model and achieve collaboration between clients.The network architecture of the student model is similar to the teacher model, which consists of a GIN backbone and a projection head. Since the purpose of the student model is to mimic the predictions of the teacher model for normal data, only normal graphs are considered in the knowledge distillation. The predicted logits of the teacher and student models are computed as follows: (11) where 𝑓 w 𝑔 (•), Proj t|w t (•) and 𝑓 w 𝑔 ′ (•), Proj s|w s (•) are the backbone networks and projection heads of teacher and student models respectively. Note that Proj t|w t (•) is actually the same as the projection head Proj(•) in Eq. (9) . Subsequently, the student model distills the knowledge from the teacher model by matching its predicted logits with those of the teacher model, described as follows:Q 𝑐,t = Proj t|w t (𝑓 w 𝑔 (X 𝑐 , A 𝑐 )), Q 𝑐,s = Proj s|w s (𝑓 w 𝑔 ′ (X 𝑐 , A 𝑐 )),EQUATIONwhere 𝐾𝐿(•, •) denotes the Kullback-Leibler divergence, which is applied to measure the discrepancy between the distribution of the predicted logits from teacher and student models. softmax(•) is the softmax function, i.e., softmax(𝑞 𝑖 /𝜏)= exp(𝑞 𝑖 /𝜏 )𝑗 exp(𝑞 𝑗 /𝜏 ) , and 𝜏 is the temperature factor that controls the smoothness of the distillation.Based on the design of the self-boosted graph knowledge distillation module, the objective function of all clients is defined as follows:EQUATIONwhere 𝜆 and 𝛾 are the two trade-off parameters. In federated learning, let W (𝑐 ) ={wEQUATIONs } denote the parameter set of the 𝑐-th client, the conventional solution achieves collaboration by uploading the network parameters to the server and subsequently distribute the aggregated network parameters to each client. However, there are several problems with this solution. First, the high parameter complexity of a GIN-based backbone can limit the scalability of the model during the parameter aggregation process. Second, the transmission of all network parameters may introduce non-IID problems, and affect the performance of local models trained on different graph data across clients.To address these issues, we propose an effective collaborative learning mechanism, which is described in Figure 2 . Specifically, We let the teacher and student models share the same GIN backbone for learning graph representation, i.e.,EQUATIONwhere Z 𝑐 denotes the learned graph representation that is shared as the input to the projection heads of teacher and student. This operation not only reduces the complexity of the local model, but also simplifies the knowledge distillation of the student model. We upload only the parameter set w (𝑐 ) s of the student head for collaboration instead of uploading all the network parameters, i.e., the parameter aggregation in the server is formalized as follows:EQUATIONwhere ws denotes the aggregated parameters in the server. The proposed collaborative learning mechanism not only streamlines the capacity of local models, but also significantly reduces the communication costs, which addresses the third challenge. To facilitate the understanding of the proposed FGAD method, we summarize its detailed training process in Algorithm 1. The collaboration between clients via the student model is performed in the following steps:• Each client performs graph knowledge distillation independently, updating its network parameters, and uploads the network parameters of the student head to the server. • The server then aggregates the network parameters following Eq. ( 15), and distributes the aggregated network parameters to each client.To demonstrate the efficiency of FGAD, we further conduct theoretical and empirical analysis (refer to Appendix 4-5). Generate anomalous graph set D with Eqs. ( 5), ( 6), (7); Compute loss items ℓ 𝑐 ad , ℓ 𝑐 g , ℓ 𝑐 kd with Eqs. ( 8), ( 9), (end for 9:Back-propagation and update each local model via minimizing Eq. ( 13);  
experiment 4.1 experimental setup: 
 Datasets. We evaluate the performance of FL-based graph anomaly detection on non-IID graphs through two distinct experimental setups: (1) single-dataset and (2) multi-dataset scenarios.• Single-dataset: We distribute a single dataset across multiple clients, each of which possesses a unique subset of the dataset. This setup allows us to assess the effectiveness when clients collaborate on a shared dataset. We employ three social network datasets, including IMDB-BINARY, COLLAB, and IMDB-MULTI, to conduct this experiment. • Multi-dataset: We broaden our evaluation by considering various datasets distributed in multiple clients, and each of them holds a specific dataset. We consider social network data (SO-CIALNET) as well as molecular (MOLECULES), biochemical (BIOCHEM) and mix data types (MIX). This allows us to thoroughly assess FL-based graph anomaly detection across a spectrum of data types and collaboration scenarios.Note that for all datasets, we regard the graphs in the first class as normal and graphs in other classes as anomalous. We allocate 80% of the normal graphs data for training, and subsequently construct the testing data by combining the remaining normal data with an equal number of anomalous graphs. Refer to Appendix 2 for more information and construction details of each dataset.Network Structure. We employ a 3-layer GIN [47] as the backbone network for our method, with the aggregated dimension in each layer set to 64. Additionally, we adopt the 4-layer and 3-layer fully connected networks for the teacher head and student head, respectively. The network structure of the teacher head is set to 256-192-128-64-2, while for the student head is 192-128-64-2. Refer to our code for further details and reproducibility 2 .Baseline Methods. We compare the proposed FGAD method with several state-of-the-art baseline methods. We include two federated learning methods: FedAvg [33] and FedProx [20] , as well as two federated graph learning methods: GCFL [46] and FedStar [38] . Note that in order to adapt these baseline methods to the graph anomaly detection task, we integrate them with DeepSVDD [37] to construct an end-to-end graph anomaly detection model. Besides, we regard the self-training strategy without the FL setting as one of 2 Code is available at https://github.com/wownice333/FGAD. the baselines. To ensure a fair comparison with FGAD, we employ the same GIN network structure as FGAD in all baseline methods.Implementation Details. We use GIN [47] as the graph representation learning backbone for FGAD and all baselines. The number of GIN layer 𝐾 is set to 3, and the dimensions of the hidden layer of GIN and projection head of student and teacher models are all set to 64. We utilize Adam [13] as the optimizer and fixed the learning rate 𝛼 = 0.001. For all datasets, we first pretrain the anomaly generator and teacher model for 10 epochs, thereafter, jointly train with knowledge distillation and collaborative learning for 200 epochs. The implementation of FGAD is based on PyTorch Geometric [9] library, and the experiments are run on NVIDIA Tesla A100 GPU with AMD EPYC 7532 CPU.Evaluation Metrics: We use Area Under the Curve (AUC) and Area Under the Precision-Recall Curve (AUPRC) as the evaluation metrics in the experiment. Each method is executed 10 times to report their means and standard deviations.Additionally, refer to Appendix 3 for more experimental settings, including the training details, trade-off parameter setting, and baseline setting.In this section, we conduct comprehensive experiments including two types of non-IID graph scenarios, i.e., the single-dataset and multi-dataset distributed in multiple clients, to validate the effectiveness of the proposed method. • Discussion: The Self-train strategy discards collaborative training and fails to leverage the knowledge from other clients to learn more robust local GAD models. FedAvg and FedProx require the transmission of all network parameters of the local models, which introduces severe non-IID problems in collaborative learning. Consequently, these three baselines yield sub-optimal performance in most cases. Although GCFL incorporates a specific design to alleviate non-IID challenges, such as utilizing clustered FL for collaborative learning, it still necessitates the transmission of all network parameters and does not effectively address non-IID problems, as validated by the experimental results. On the other hand, FedStar achieves runner-up performance in most cases, which may primarily be attributed to the introduced structural embedding that helps to preserve the personalization of local models. Compared with the baseline methods, FGAD considers enhancing the detecting capability of local models in a self-boosted manner, and introduces an effective collaborative learning mechanism by leveraging knowledge distillation. This allows FGAD to learn more powerful local GAD models, mitigate the adverse effects of non-IID problems, and reduce communication costs among clients.We employ t-SNE [40] to visualize the learned embeddings for intuitive comparison. Figure 3 shows the embedding visualization for AIDS, one of the constituents of MOLECULES. We include results from FedAvg, GCFL, and FedStar for a comprehensive analysis. It's evident that the learned embeddings by FedAvg and GCFL exhibit poor discriminative properties, with both normal and anomalous graphs appearing entangled in the latent space. Although the visualization result of FedStar shows some separation between normal and anomalous graphs, decision boundaries remain blurred. Conversely, the learned embeddings of FGAD are clearly more discriminative compared to the other baseline methods. The visualization of FGAD reveals distinct boundaries between the embeddings of normal and anomalous graphs, supporting its effectiveness.To validate the effectiveness of each component in the proposed FGAD method, we derive three variants from FGAD and perform a systematic evaluation. Specifically, we illustrate the construction details of the three variants as follows:• FGAD_v1: This variant only considers local training in each client and abandons the collaborative learning between clients. • FGAD_v2: This variant drops the proposed collaborative learning mechanism and follows the parameter aggregation mechanism of the classical FedAvg method. • FGAD_v3: This variant drops the knowledge distillation module, i.e., removes the student model and only takes the teacher anomaly detector in collaboration.Table 3 shows the experimental results of FGAD and its three variants on two datasets, yielding the following observations. FGAD_v1 demonstrates a performance decline compared to FGAD, which is primarily due to the fact that FGAD_v1 exclusively focuses on In this section, we analyze the impact of the two hyper-parameters on anomaly detection performance. Specifically, we vary the values of 𝜆 and 𝛾 within the range of [1𝑒 -3 , . . . , 1𝑒 4 ] and Figure 4 presents the experimental results on IMDB-BINARY and MOLECULES datasets. Based on the observations in the figure, we draw several conclusions. Firstly, FGAD tends to yield sub-optimal performance when the values of 𝜆 and 𝛾 are set too low, e.g., 1𝑒 -4 and 1𝑒 -3 . This emphasizes the significant role of both loss terms in the FGAD framework and suggests their effectiveness. Secondly, we can observe that excessively high values of 𝜆 and 𝛾 also have an adverse impact on performance, because they may obscure the primary objective of optimizing the anomaly detector. Finally, it is worth noting that FGAD exhibits relatively stable performance both in AUC and AUPRC across a wide range of 𝜆 and 𝛾 values, demonstrating its robustness. 5 . Note that we also include FedAvg as a baseline method for comparative analysis. It can be observed that FGAD consistently achieves remarkable performance improvement compared to FedAvg in all cases, and exhibits stability against changes in the number of clients. However, when the number of clients increases to certain large values, the average performance degrades, and the performance variance between different clients becomes more significant in both FGAD and Fe-dAvg. This is primarily due to the gradually increasing discrepancy between the graph data distributed across different clients, which causes more severe non-IID problems. Nevertheless, FGAD still exhibits relatively smaller performance fluctuations compared with FedAvg, which fully demonstrates the scalability of the proposed FGAD method.Other Experiments. We further analyze the impact of the number of layers and latent dimensions in GIN, as well as the justification of backbone sharing. Refer to Appendix 6, 7, and 8 for details due to the limitation of the paper length. 
conclusion: 
 In this paper, we studied a challenging GAD problem with non-IID graph data distributed across multiple clients, and proposed an effective federated graph anomaly detection (FGAD) method to tackle this issue. To enhance the detecting capability of local models, we proposed to unsupervised train a classifier in a self-boosted manner by distinguishing the normal and anomalous graphs generated from an anomaly generator. Besides that, in order to alleviate the adverse impact of non-IID problems among clients, we introduce a student model to distill knowledge from the teacher anomaly detector and engage only the student model in collaborative learning so that the personalization of local models could be preserved. Furthermore, we improved the collaborative learning mechanism that streamlines the capacity of local models and reduces the communication costs during collaborative learning. Comparative experiments with state-of-the-art baselines in the graph anomaly detection tasks on single/multi datasets (from diverse domains) demonstrated the superiority of FGAD. We believe that this work will pave the way for future research on collaborative GAD under the FL setting. 
 
 Appendix: Note that we slightly abuse notations X 𝑐 , Ã𝑐 , and A 𝑐 to represent the relevant attribute and adjacency matrices in the 𝑐-th client. 


Decision: The authors have proposed a federated graph anomaly detection framework that uses an anomaly generation module to perturb the normal graphs and then trains an anomaly detector by distinguishing between generated anomalous and normal graphs. The framework also uses a student model to distill knowledge from the trained anomaly detector, thus reducing the impact of non-IID data. Finally, it contains also a collaborative learning mechanism that has the goal of preserving the personalization of local models. While some reviewers are keeping some concerns about the paper, the authors were able in the original submission and also in the rebuttal to show the value of the proposed framework, for example adding further comparisons with SOTA approaches and reporting statistical significance of the results by reporting a t-test in the rebuttal (this should be included in the potential final version). Again, the ablation reported seems comprehensive.

Reviews: review: paper_summary: paper_summary: This paper proposes a federated graph anomaly detection (FGAD) framework to enable collaborative learning among clients while preserving data privacy. The key components of FGAD include an anomaly generator to produce diverse anomalous graphs, a self-boosted distillation module to maintain the personalization of local models, and an effective collaborative learning mechanism to reduce communication costs. Experiments on various datasets demonstrate the superiority of FGAD over state-of-the-art baselines in detecting anomalies in non-IID graph data.
strength_weakness: strength_weakness: Pros:
1. FGAD provides a privacy-preserving solution for collaborative graph anomaly detection by leveraging federated learning.
2. The self-boosted distillation module enhances the detecting capability of local models by generating diverse anomalous graphs and preserving personalization.
3. The collaborative learning mechanism streamlines the capacity of local models and significantly reduces communication costs.
4. FGAD demonstrates superior performance compared to state-of-the-art baselines on both single-dataset and multi-dataset scenarios.
5. Extensive experiments, including ablation studies and parameter analysis, validate the effectiveness and robustness of FGAD.
limitations: limitations: Cons:
1. Statistical testing is desired. It is highly recommended that the authors conduct a Statistical testing. Otherwise, the statistical significance of the results are not clear.
2. No code is provided. It is strongly encouraged that the authors release the code for reproduction. The authors could upload code to website such as https://anonymous.4open.science/
paper_summary: paper_summary: This paper proposes an effective federated graph anomaly detection framework (FGAD). It conducts federated learning across multiple clients, not sharing local data, thereby preventing leakage of sensitive information and protecting client privacy.
strength_weakness: strength_weakness: 1.The viewpoints presented in this paper are highly innovative.
2.The experiments conducted are comprehensive and thorough.
limitations: limitations: 1.Why only upload the parameter set of the student head for collaboration instead of uploading all the network parameters? Even though the student models in each client share the parameters of the GNN portion with the teacher model, the parameters across different clients are still different. Similarly, w_g is different for different clients, so why not upload?
2. Personally, I believe that the ablation section is not sufficiently comprehensive. Ablation experiments for the anomaly generator should also be included.
3. A minor detail: should "a node set and edge set" be adjusted to "a node set and an edge set" or "node set and edge set"?
4. After reading this paper, the feeling is that although the paper is innovative, its relationship with multimodal and multimedia aspects is not significant.
paper_summary: paper_summary: In this paper, the authors propose a federated graph anomaly detection (FGAD) framework to address the challenges posed by the heterogeneity and non-IID property of graph data in graph federated learning. The FGAD framework utilizes an anomaly generator to create anomalous graphs and unsupervised trains a local teacher model. Then a student model is used to distill knowledge from the trained teacher model, ensuring that its prediction distribution of normal graphs aligns with that of the teacher model, thereby mitigating the impact of non-IID problems. Finally, a collaborative learning mechanism is employed to globally optimize the parameters of the student head.
strength_weakness: strength_weakness: 1. The completeness of the method proposed in this paper is well, as it is the first to tackle graph anomaly detection on non-IID graphs with a federated learning setup and introduces a self-boosted distillation module to preserve the model personalization and alleviate non-IID problems.
2. This paper completed comparative experiments of multiple baseline methods in single-dataset and multi-data scenarios, as well as some ablation experiments of its own.
limitations: limitations: 1. The backbone used in both the teacher and student models is solely GIN. There is a lack of experiments using other GNNs to demonstrate the generalizability of the method.
2. The experimental setup for multi-dataset scenario is unclear. The paper doesn’t specify how the multi-domain datasets are distributed to multiple clients.
3. The paper mentions that all normal graphs are simply regarded as one class, while the generated anomalous graphs are regarded as abnormal. Under this setting, is it possible that a perturbed graph from one class might be considered a normal graph in another class?
4. There is no clear explanation in the experimental section about how the abnormal data is obtained.
paper_summary: paper_summary: This paper studies a valuable topic in graph anomaly detection. The authors propose an effective federated graph anomaly detection framework. They design a student model to distill the knowledge from the trained teacher anomaly detector. Then they align the student model’s predicted distributions with that of the teacher model. Moreover, the authors further design an effective collaborative learning mechanism and achieve good performance.
strength_weakness: strength_weakness: This paper investigates the challenging GAD issue on non-IID graphs under federated learning setup, and propose an effective federated graph anomaly detection (FGAD) method.
limitations: limitations: 1. What is non-IID property in federated graph anomaly detection? The authors should provide more explanations in Introduction Section and clearly articulate the advantages of the model in solving the problem of non-IID.
2. The authors should provide more explanations of Eq.(8), e.g., A_c^{ij}.
3. The anomalous graphs in training are generated by perturbing graph structure. Why not use other methods? e.g. perturbing node features. Does this fit the real-life scenario?
4. The compared methods in experiments are old. Why doesn't the author compare the proposed method with [1]? The experiment results cannot confirm the effectiveness of the proposed model.

[1] Ma R, Pang G, Chen L, et al. Deep graph-level anomaly detection by glocal knowledge distillation[C]//Proceedings of the fifteenth ACM international conference on web search and data mining. 2022: 704-714.