FedSSP: Federated Graph Learning with Spectral Knowledge and Personalized Preference
Authors: Zihan Tan
Guancheng Wan
Wenke Huang
Mang Ye
Venue: NeurIPS 2024 poster (2024)
Keywords: Federated Learning; Graph Learning
Primary area: safety_in_machine_learning

Abstract: Personalized Federated Graph Learning (pFGL) facilitates the decentralized training of Graph Neural Networks (GNNs) without compromising privacy while accommodating personalized requirements for non-IID participants. In cross-domain scenarios, structural heterogeneity poses significant challenges for pFGL. Nevertheless, previous pFGL methods incorrectly share non-generic knowledge globally and fail to tailor personalized solutions locally under domain structural shift. We innovatively reveal that the spectral nature of graphs can well reflect inherent domain structural shifts. Correspondingly, our method overcomes it by sharing generic spectral knowledge. Moreover, we indicate the biased message-passing schemes for graph structures and propose the personalized preference module. Combining both strategies, we propose our pFGL framework $\textbf{FedSSP}$ which $\textbf{S}$hares generic $\textbf{S}$pectral knowledge while satisfying graph $\textbf{P}$references. Furthermore, We perform extensive experiments on cross-dataset and cross-domain settings to demonstrate the superiority of our framework. The code is available at https://github.com/OakleyTan/FedSSP.

FedSSP: Federated Graph Learning with Spectral Knowledge and Personalized Preference
Zihan Tan Guancheng Wan Wenke Huang!* Mang Ye!.
1 National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, Hubei Key Laboratory of Multimedia and Network Communication Engineering, School of Computer Science, Wuhan University, Wuhan, China. 2 Taikang Center for Life and Medical Sciences. Wuhan University, Wuhan, China
{zihantan, guanchengwar. wenkehuang, yewaneemmmaaiieeeeeeeeee @whu, edu.
Abstract
Personalized Federated Graph Learning (pFGL) facilitates the decentralized train- ing of Graph Neural Networks (GNNs) without compromising privacy while accommodating personalized requirements for non-IlI) participants. In cross- domain scenarios, structural heterogeneity poses significant challenges for pFGL. Nevertheless, previous pFGL methods incorrectly share non-generic knowledge globally and fail to tailor personalized solutions locally under domain structural shift. We innovatively reveal that the spectral nature of graphs can well reflect inherent domain structural shifts. Correspondingly, sharing generic spectral knowledge. Moreover, we indicate the biased message- passing schemes for graph structures and propose the personalized preference module. Combining both strategies, we propose our pFGL. framework FedSSP which Shares generic Spectral knowledge while satisfying graph Preferences. Fur- thermore, We perform extensive experiments on cross-dataset and cross-domain settings to demonstrate the superiority our framework. The code is available htt.g/tww...mmmmaaaee/aaa////a//
method overcomes it by
Introduction
have demonstrated their superiority in modeling graph data which frequently emerges in variety of scenarios [73 71], as exemplified by graph clustering 45
Graph Neural Networks (GNNs) [56 63 48
46 [44, graph contrastive learning (64] 80], anatomy detection [75] 88), knowledge
67 and on. However, large amounts of graph data are generated by edge devices in reality, which brings in privacy concerns and the challenges of data silos 889 24 To address these difficulties, federated learning (FI.) has recently been applied It allows models on various clients to collaborate without sharing
graph (79 78
38 structural inference [66] 68
65
to graph learning 18
20
25
27 and makes federated graph learning (FGL) a promising direction. Nonetheless, the non-lID problem remains major challenge in FGL, graph data from different clients usually vary significantly. In such scenarios, a single global model struggles to adapt well to the local data of each client with inconsistent data distributions [60 [58). To tackle these challenges, personalized federated graph learning (pFGL) has emerged, offering customized GNNs for each client to achieve satisfying local performance [1]
local data 26
15
23
However, PFGL still encounters substantial challenges from structural heterogeneity 29 especially in domain shift tasks, for instance, between social networks [94] (95] and molecular structures [59 55
There are two significant drawbacks previous algorithms as Fig. demonstrates. For global
Eqqual contribution. Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

(a) Global Collaboration
(b) I Lcal A Application
(e) Spectral Biases
Biased Messag
Paninx
Mode
Client
Unsuibiled
M.C
Preferonrs
Unpromising Collaboration
Client
Unsuitable Feature
Problem illustration. We illustrate the challenges of the cross-domain scenario. (a) Considering the domain structural shifts, clients struggle with knowledge conflict caused by non-generic sharing which arises from the shifts, thus leading to unpromising global collaboration. (b) The aggregated pesssag-passingg scheme suffers from inconsistent preferences that remain unsatisfied of specific datascts in this scenario. Consequently, it leads to unsuitable features of graphs in local applications. (c) The heat map of Jensen- Shannon divergence of algebraic connectivity [17 and eigenvalues distributions among six datasets from three different domains. Spectral characteristics exhibit significant biases across domains but are more similar within same domain.
Figure 1:
collaboration, the considerable domain structural shifts inevitably lead to non-generic knowledge, thus resulting in knowledge conflict. Both current methods suffer from conflict and are trapped in unpromising collaboration. Specifically, [6] share non-generic structural encoding and struggle with structural knowledge conflict, while strategy [77] mitigating conflicts by limiting the potential for collaboration. The key to addressing knowledge conffict is pursuing a way to share generic knowledge that benefits all clients. Based on this observation, we raise the question: 1) how to address the knowledge conflict under domain structural shift by extracting and sharing generic knowledge?
For local applications, each client owns its specific dataset with distinct structural characteristics in this cross-dataset scenario. Due to the GNN passagg-passing nature, distinct graph structures stored in different clients prefer different message-passing schemes. Consequently, the scheme provided by aggregated GNN exhibits biases from the optimum when applied locally, thus leading to unsuitable features. Both current methods neglect the preferences of various clients for specific graph structures. This deficiency leads us consider: 2) how design personalized plans deal with inconsistent preferences of specific graph datasets from various clients?
To address problem 1), given that structural shifts make it hard to directly achieve generic sharing at the structural level, we propose to explore the structure shifts from another spectral perspective since previous works have demonstrated the strong correlation between graph structure and spectra 81 32). The major advantage of spectra is the detailed propagation and processing of graph signals on the graph structure, thus facilitating the discovery of generic knowledge in several certain processes unaffected by structural shifts. To validate our assumption, we first conduct experiments and explicitly reveal the domain spectral biases that directly reflect domain structural shifts on spectra as Fig. [ demonstrates. To tackle these spectral biases directly to overcome structural shifts, we design Generic Spectral Knowledge Sharing (GSKS) share generic spectral knowledge extracted from spectral encoders. It enables clients to benefit from others through collaboration without knowledge conflict. Conversely, other components containing non-generic knowledge are retained locally. Correspondingly. clients can customize powerful graph convolutions for their local graph characteristics while benefiting from generic knowledge without conflict. Through this strategy, we promote the sharing of generic spectral knowledge and the personalizing of non-generic knowledge, thus achieving effective collaboration against knowledge conflict.
Moreover, we attempt to achieve target 2) and design suitable personalized plans for each client graph structure locally. Specifically,, we explore the gressage-passinggg nature of GNN [5] 16 62). From the spectral perspective, spectral encoders strongly affect message transmission. Therefore, when aggregated spectral encoders are applied distinct graph structures locally, they tend deviate from the optimal message-passe passing scheme for the client [49. Consequently. GNNs extract inappropriate frequency messages which lead to unsuitable features. To meet the inconsistent graph preferences, we innovatively configure learnable preference for each client and propose Personalized Graph Preference Adjustment (PGPA). These personalized preference modules apply adjustments to the feature extracted with the participation of global spectral encoders. It allows the feature to suit the specific graph structures of each client. Moreover, to address the issue of over-reliance when applying the preference module independently, a regularization term is introduced. Combining both strategies
2

---

for effective global collaboration and personalized local application, we propose our pFGL framework FedSSP. In conclusion, our key contributions are:
We are the first to reveal domain structural shifts through spectral biases, as well as consider the inconsistent preferences of distinct datasets from various clients.
We propose FedSSP, which innovatively overcomes knowledge conflicts from a spectral perspective and implements personalized graph preference adjustments for each client.
We conduct experiments in various cross-dataset and cross-domain settings, proving that our approach outperforms several current state-of -ftthofft methods and achieves optimal results.
2 Related Work
2.1 Spectral GNNs
Spectral GNNs 4 12 are based spectral graph filters set in the spectral domain, providing
powerful models for graph neural networks 776 generally be categorized into two types: those with fixed filters and those with learnable filters. Fixed filter spectral GNNs, such as APPNP [19], utilize personalized PageRank (PPR) (53) to construct their filtering functions. GNN-LF/HF (96] designs filter weights from the perspective of graph optimization functions. Learnable filter spectral GNNs include subelass that approximates arbitrary filters using various types of orthogonal polynomials, including Bernstein (22). Chebyshev [21]. and Jacobi [74). Another subclass parameterizes the filters by neural networks, including LanczosNet 40 and Specformer (3). The robust modeling capability of spectral graph neural networks data inspires us to leverage this foundation to tackle the issue of structural heterogeneity across domains.
87 86 47 72
IO 70
8 69 7 Spectral GNNs can
2.2 Personalized Federated Learning
collaborative learning on local data, introducing methods like FedAvg [51] to address this. Yet. it struggles with non-IID data
Federated learning [84 34 28
14
82 facilitates
across clients. Several techniques aim to address the challenge 333 35 30], but achieving a global
model that suits all local data remains difficuut
[29]. Personalized Federated Learning (pFL) has
attracted increasing attention for its ability to address the non-lll problem [3
39 60
Research
has approached improvements from various aspects. In FedPHP aggregates the global model and old personalized models locally to preserve historical information [36], FedALA achieves personalized aggregation through personalized masks [91] and
methods,
APPLE uploads only core models and employs directed relationship vectors for downloading (50 In
model-splittin-baan baasl approaches, FedRol) 6 learns with global feature extractor and two heads for both global and personalized tasks. FedCP decouples features suitable for global and local heads through conditional policy scheme [92]. Moreover, methods based on regularization and knowledge distillation have also been utilized to enhance pFL. However, pFL methods lack targeted strategy designs for graphs, making them not particularly suited for pFGL scenarios.
2.3 Federated Graph Learning
Recent studies have utilized the FL framework for distributed training of GNNs without sharing graph data [20 41 28, 99. Current Federated Graph Learning (FGL) research can be categorized into two types: aatra-graph and gater- FGL. In inter-graph FGL, each client has distinct graphs, and they jointly participate in federated learning to either improve GNN modeling of local data or achieve model that can generalize across different datasets (77 6T]. Intra-graph FGL, on the other hand, aims to deal with challenges such as missing link prediction [1. subgraph community detection [93] I.. and node classification [25] 37]. However, most FGL methods lack specific design considerations for our scenario. More precisely, there is general absence of consideration for the heterogeneity of graph-level structures and the personalized needs of different clients brought about by structural characteristics. In this paper, we focus on inter-graph FGL, taking into account spectral domain biases and the uniqueness of graph structures that result in client-specific preferences, to customize a personalized optimal model for each client specifically for graph classification tasks.
3

---

3 Preliminary
3.1 Graph Signal Filter
Assume that we have a graph G EV, where V represents the node set with |V] and & is the edge set. The corresponding adjacency matrix is defined as E {0, 1jnxn, where Aub = there is edge between nodes and b, and Aas 0 otherwise. The normalized graph Laplacian matrix is defined as L In D 2----11-- AD 1/2, where I, denotes the n x identity matrix and D is the diagonal degree matrix. We assume is undirected. L is real symmetric matrix, whose spectral decomposition can be written L = UAUT, where the columns of U are the eigenvectors and A = diag( Ag, >g........ An) are the corresponding eigenvalues ranged in [0, 20. The Graph Fourier transform of a signal
Raxd is defined as म U E Raxd The inverse transform defined as Uz [57] The i-th column of U denotes frequency component corresponding to the eigenvalue A Let x U x. where Ux represents the eigenvector corresponding to X, be the frequency component of x at À frequency. We can use function g [0. 2] + R to filter each frequency component by multiplying g(A). By defining = dAag(A), g implements filtering on A thus ultimately implementing filtering the graph signal The whole process is defined follows:
Ug(A)U**.
By defining ((..)) = Ei= on i. where y often set to be a polynomial of degree for parameter- izing the filter, the filtering process can be rewritten as follows:
Ug(A)U* x = =g(()
3.2 Federated Learning and Personalized Federated Learning
Traditional FL leverages isolated data of distributed clients and collaboratively learns models M for a generalizable global model without leaking privacy. Specifically, the goal is to minimize:
min fy( min Σ w,M,(O,,
where (-) denotes the global objective. is computed as the weighted sum of the local objectives, with N heing the number of clients and wl; being the weights. The local objective M:(-)) is often defined as the expected error over all data under local data D.
In the context of personalized federated learning, the global objective takes more flexible form:
N
min (p) = min (,
w;,)(0.).
(4)
θ
where fo(-) the global objective for the personalized algorithms, and θ 11.00.... 02. .......................... 0N
is the matrix with all personalized models. In this work, we aim to obtain the optimal θ arg mine (e), which equivalently represents the set of optimal personalized models o,, e [N].
Methodology
4.1 Generic Spectral Knowledge Sharing (GSKS)
Motivation. Current methods suffer from knowledge conffict arising from non-generic sharing under domain structural shifts. Since structural shifts impede the direct generic sharing at the structural level, are the first to reveal and resolve knowledge confficts from the spectral perspective. To explicitly address the spectral biases that reflect structural shifts Fig. I we base our pFGL strategy on spectral GNNs and further propose Generic Spectral Knowledge Sharing (GSKS). Effective collaboration that overcomes spectral bias and structural shift is achieved, thereby addressing knowledge conflict. Details of GSKS are presented in Fig. (a).
Eigenvalue filtering. Aiming at more expressive representations of frequency information, the cigenvalues are firstly mapped from scalars to meaningful vectors for subsequent learning of frequency

---

Client
Filters ((A)
$
Graph Ease
A.
Preference Modiule
AAA
4
Mown
Shardd Globally
Bases delogrian
Shared Globally
40
Fillers ((A)
Muun
Rigens clunn)
Graph
Preference
Client
Blsses
Module
(a) G Generic Spectral Knowledge Sharing
(b) Personalized Graph Preference A djjustment
Figure 2: Architecture illustration of FedSSP. The left box (a) refers to Generic Spectral Knowledge Sharing (GSKS), where address knowledge conflict and promote effective global collaboration by sharing generic spectral knowledge extracted from spectral encoders d' and of while retaining non-generic in other components. The right box (b) represents Personalized Graph Preference Adjustment (PGPA), where we leverage preference module guided by L C.GPA for satisfying inconsistent preferences and achieving suitable feature of datasets locally. These two boxes correspondingly refer to the two core strategies of our framework FedSSP.
interrelation in the multi-head attention module as follows:
if q is even, sc (8 X) = c)) if q is odd,
[ii(B3/ea/4)
(5)
where e keeps eigenvalues within a suitable numerical range to distinguish different eigenvalues for trigonometric functions. q E Z [0.d -d] is the index for dimension d while 3 controlling the importance of À with defaulted value 10000. Moreover, H" denotes parameters of the eigenvalue encoder e², by which eigenvalues are mapped from scalars to vectors for richer frequency information. Consequently, they are more expressive for filtering by the attention module and decoder through R Rd. The initial representations are the concatenation of eigenvalues and their encodings: X' [concal()), 5 (0=;Ai)), concal (An of (0"; ; )'') E R=x(d+1), Then the multi-head attention module is leveraged. After stacking multiple transformer blocks, spectral decoder dd for eigenvalue decoding can learn new eigenvalues from the expressive representations of spectra:
Am = (Attention (Q0 Kel V0%));
(6)
Where the representations learned by the m-th head are fed into vid, while ifd denotes a combination of liner and optional activation. Am E Rax1 is the filtered eigenvalues by the m-th head. The whole process in Eq. (6) acts as a spectral filter g for the origin eigenvalues in Eq.
To address the challenge of knowledge conflict, we attempt to explore the functionality of each module. The eigenvalue encoder 6f captures multi-scale representations of eigenvalues and provides meaningful vectors of distinct frequencies. Since the mapping from eigenvalues to vectors by de is independent of the domain characteristics, 6 of de is considered to contain generic knowledge. In contrast, as the spectral biases we reveal in Figure demonstrate, biases exist in eigenvalue distribution across domains. In contrast, spectral characteristics within the same domain are more similar. Therefore, the attention module learns the non-generic magnitudes and relative dependencies specific to the spectral characteristics of each client. Correspondingly, the eigenvalue decoder focuses on decoding the most suitable message-passing scheme and client-specific frequency components from the representation processed by the attention module. Attention module and decoder together formed g in Eq. ( aiming at designing personalized filtered eigenvalue that guides cessage-passing at a personalized suitable frequency. Therefore, 6 shared in our strategy to achieve generic spectral knowledge sharing and effective collaboration unaffected by knowledge conflict.
Specifically, client i uploads its update of 0;. the f-th iteration (t 2 0), the central server distributes global spectral encoder weights Ar, Accordingly, client i updates local GNN weights including 0 with their dataset D; and send these updates as A = 0 of, to the central server. Then the server aggregates the received local updates and modifies the global weight g1+1 follows:
A0t (i E [I..N]),
--11-
N
notably, aggregation based on sample size unsuitable in this scenario for effective collaboration across various domains and datasets. Since clients here possess specific datasets with significant
5

---

quantitative variance, those with larger datasets tend to dominate the collaboration. Thereby prevent- ing them from benefiting from the spectral and frequency knowledge of clients with fewer samples. Correspondingly, clients with fewer graphs are almost entirely dominated by knowledge that does not originate from their local data. To address the problem, we leverage direct average of spectral encoder weights from all clients to achieve fair collaboration and cross-datase knowledge sharing.
Personalized graph convolution constructing. After getting M filtered eigenvalues, filter encoder: of (01; B3) Ra + E is leveraged to construct the bases for personalized graph convolution. To avoid confusion and distinguish from the mentioned filter g on eigenvalues in Eq. (I) filter here in filter encoder refers to the filtering on feature message-passing passing through bases in graph convolution. New bases are first reconstructed and concatenated along the channel dimension. Specifically, by
defining Am = diag(Am). they are fed into filter encoder of as follows:
Bm = UAm UAMUU, Vm E (L,...,,M}
(8)
while B,, E R=x* is the m-th new basis and å E Raxd is for feature filtering in graph convolution ultimately. The original hases B., are initially constructed from the eigenvectors U and the filtered eigenvalues Am. with Φ (01; B) responsible for the learnable transformation of bases from original to new. This transformation facilitates learning more suitable schemes for graph massage-passing and processing at various frequencies. Filter encoders Φ (01; B) in clients encapsulate knowledge of various frequency components which affects how much graph signal varies from nodes their neighbors for better graph convolution construction. Due to restrictions on the sample size for certain clients, they are unable to adequately learn message-passing techniques for handling specific frequency components. As a solution, the filter encoder is shared among clients, enabling them to fully acquire the graph signal processing methods for frequencies that are challenging to learn locally. Specifically, collaboration on filter encoder can aid &f (6'; ; B) of each client in learning how to construct suitable graph convolution from various message-passing schemes. Therefore, we design client i to upload the weights 0f of its of the same way of Eq. 7 thereby achieving a comprehensive understanding of different frequency messages in graphs. Subsequently, it enables the construction of powerful personalized graph convolutions follows:
where B = [B, .,..... Bul] E R
z'y- = ( Te CoCo))) + Fer
(10)
where v, is the node e representation from the previous layer, while vi, represents the output of the current layer. @Conv denotes weights of graph convolution, and refers to the optional activation. Ultimately, the representations of all nodes within a graph are aggregated by an average pooling layer form the overall feature representation of graph Gi dataset D, of client as follows:
M
Tus WE (L. (PI].
(11)
where hy is defined as the average of all node features in graph Gi namely the graph feature, while V refers to the node quantities in graph / here. By sharing generic spectral knowledge and retaining client-specific knowledge we successfully achieve effective collaboration that overcomes spectral bias, thereby domain structural shift from the spectral perspective.
4.2 Personalized Graph Preference Adjustment (PGPA)
Motivation. Due to the GNN message-ppssingg nature, distinct graph structures prefer different message-passing schemes, especially when meeting the specificity of datasets in cross-dataset and cross-domain scenarios. Consequently, The spectral encoders under global collaboration fail to satisfy the inconsistent local preferences of graphs. Correspondingly, graph convolutions tend to learn biased meessage-passingg schemes, thereby extracting unsuitable graph features. Our approach provides personalized adjustments address this challenge based on client preference. Moreover, we solve the over-reliance issue that arises from the process of satisfying various preferences. Details of GSKS are presented in Fig. (b).
To satisfy the various preferences and make the graph features more suitable for graphs, we propose learnable preference module that adjusts to features extracted by client i to satisfy local graph

---

structure preference explicitly. The module includes learnable parameters matched in size with the feature space, thus acting as a refined tool to flexibly satisfy the preferences of each client during local training. Considering local model M including feature extractor F(BF; ()) classification head C(OC, h)), and preference module P(6), where 9 represents graphs contained dataset 'D of client. The whole graph feature-extracting process can be defined as follows in our strategy:
= F(00;
:' h + ó,
(12)
by integrating the original feature with preference adjustments 6, h' becomes the ultimately suitable feature that satisfies client preference. Now we leverage adjusted feature h' for z' C(09;h') instead of the original unsuitable representation h. Specifically, the local loss for client is:
L (,, _ = (=), C2E(2, Ur).
(13)
where the Cross-entropy (CE) loss measures the difference between the predicted probability and the true label. Nevertheless, the preference module learns not only the client-specifie preference but also aspects that should be handled by the feature extractor J. without a guide for preference. As result, the local feature extractor FF might overly rely on adjustments provided by the preference module during training, thereby hindering its capability. Correspondingly. this over-reliance can negatively impact federated collaboration. When the capability of F is degraded, the shared spectral encoders fail to convey beneficial knowledge to others, leading to unpromising collaboration.
Therefore, it is essential to guide the preference module to focus solely on the aspects of client preferences, rather than interfering with the feature extraction guided by collaboration. We achieve this by guiding the output of the feature extractor to align more closely with global graph features. Consequently, the PGPA module is directed to concentrate on client preferences. To implement this, we first calculate the mean of local graph features in each iteration:
(14) where μ denotes the momentum we introduce for bringing graph modeling patterns from previous batches to the current batch in the same local epoch. h and h!th represent the local mean graph feature of the previous batches and the current batch. is necessary to distinguish between mean and prototype. In this scenario, clients own various datasets, thus the class information is client-specific. Correspondingly, the mean h; which represents the average modeling for graphs in client i is unrelated to class information. After local training, client i uploads its mean to the server for global consensus aggregation. Based on our exploration of Eq. 77 a direct average is leveraged here:
h4 (1 - 11) h + 1 -.
T hg N
(15)
where h, refers to the global graph modeling consensus calculated from all samples across all clients. Accordingly, we employ the Mean Squared Error (MSE) to measure the distance between the local graph feature mean and the global graph mean obtained from the previous round. This measurement serves as a regularization term to encourage the local graph feature to align closer to the global modeling consensus, thus guiding the preference module to focus on preference and correspondingly address the over-reliance issue. Specifically, local feature extractors are encouraged to extract certain frequency messages in graphs that reflect the global modeling consensus, making the PGPA only responsible for client-specific preference. The local loss of client is now defined as:
L. (eg,n) ( P. ( [CE(2), v:) MSE(h, hy).
(16)
By implementing MSE(h,, hg). we explicitly align local graph modelings with the global consensus, thus guiding the preference module P to focus on preference and addressing the considered issue of over-reliance by forcing the preference module to focus on client-specific graph preference.
5
Experiments
5.1 Experimental Setup
We perform experiments on graph classification tasks in various cross-dataset and cross-domain scenarios validate the superiority of our framework FedSSP.
Datasets. Follow the settings in GT,, we use 15 public graph classification datasets from four different domains, including Small Molecules (MUTAG, BZR, COX2, DHFR, AIDS,
7

---

NCII), Bioinformatics (PROTEIN, OHSU, Peking 1), Social Networks(IMDB (et---rR(MDDB-BINAARR BINARY, IMDB- MULTI), and Computer Vision (Letter-low, Letter-high, Letter-med) [52]. Node features are available in some datasets, and graph labels are either binary multi-class. We create six different settings: (1) cross-dataset setting based on seven small molecules datasets (SM); (2)-(6) both cross-dataset and cross-domain settings based on datasets from two different domains(BIO-SM) SM-CV) and three different domains(BIO BBIO-SM-NN, BIO-SN-CV, CHEM-SN-CV)
Baselines. We compare ours with several SOTA federated approaches. (I)Local as the first baseline; Av) [T]; Fed [3 which address heterogeneity issues in FL; (4)APPLE (50] and
F5) 192]. two state-of-the the-the-art aare-artppp pFL (6 [93], (7)GCFL 77 (8)FGSSL [25] and Fe)arr [6], four state-of-the-arr FGL methods.
Implementation Details. The experiments are conducted using NVIDIA GeForce RTX 3090 GPUs the hardware platform, coupled with Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz. For each setting, every client holds its unique graph dataset, among which 10% are held out for testing, 10% for validation, and 80% for training. We leverage the AdamW optimizer [31] for local GNNs with learning rate 0,001, the default parameter of = le 8, and (31, (2) (0.99, 0.999), as suggested by [54] (55. The number of communication rounds is 200 for all FL methods. We report the results with an average of over 5 runs of different random seeds.
5.2 Experimental Results
Performance Comparison We show the federated graph classification results of all methods under six different non-IID settings, including one cross-dataset setting (SM), two cross-double-domain set- tings (BIO-SM, SM-CV) cross-multi-domain settings (BIO-SM-SN, BIO-SN-CV, SM-SN-CV). We summarize the final average test accuracy in Tab. IT These results indicate that FedSSP outperforms all other baselines in five out of the six settings. Notably, traditional FL algorithms such as FedAvg and FedProx failed to outperform self- self-training due to the strong cross-datasets and cross-domain non-IID challenge of this scenario. Correspondingly, algorithms such as FedStar and FedCP which designed specifically for pFGL pFL scenarios perform better here.
Table 1: Comparison with the state-of-the of-the-ar art methods on one cross-dataset and five cross- dommin settings. Best in bold and second with underline. In each setting, each client owns unique dataset.
single-domain
double-domain
Multi-Domain
Methods
SM
SM-BIO
SM-CV
SM-BIO-SN
BIO-SN-CV
SM-SN-CV
Local
77.33 + 1.15
72.32 + 1.86
82,24 + 1.73
71.13 + 1.32
72.59 + 2.70
77.83 + 0.54
FedAvg [ASTAT17]
74.12 + 2.10
67.82 + 1.63
81.21 4: 1.00
67.31 4 2.56
70.93
2.91
75.33 + 1.06
FedProx [urXiv18]
69.35 + 3.36
67.27 + 4.17
70.02 4 2.27
63.89 + 4.33
69.32
1.75
67.15 + 2.25
FedSage [NeurIPS21]
75.61 + 1.16
72.60 - 3.18
76.23 0.49
70.81 0.88
69.69
1.11
73.36 + 0.86
GCFL [NeurIPS2]]
77.71 + 1.53
72.05 + 2.20
72.64 0..44:0.1 0.71
70.43 1.39
67.91 + 2.15
71.79 + 0.21
APPLE [UCA122]
74.29 = 1.89
70.40 + 2.13
76.07 + 2.55
71.07 : 1.64
72.52 + 1.03
72.33 + 0.42
FedCP [KDD23]
77.58 2.00
75. + 1.77
81.59 + 0.40
71.32 +: 1.23
73.74 1- 2.53
78.17 + 1.78
FGSSL [UCAI23]
77.90 + 0.85
72.47 + 2.15
82,60 + 0.48
68.13 + 1.71
73.44 - 1.33
77.90 + 0.62
FedStar [AAA123]
78.63 + 2.11
72.71
1.22
78.84 1: 1.07
72.60
2.45
69.51
3.24
75.94 0.40
FedSSP (ours)
73.66
2.34
84.29
0.68
72.37
2.18
2.70 79.12 1.23
Convergence Analysis Fig.3] shows the curves of the average test accuracy with standard deviation during the training process across five random runs of three settings (SM, SM-CV, SM-SN-CV) representing single-domain, double-domain, and multi-domain scenarios, including the results of various baselines. As is shown, traditional FL. methods such as FedAvg or FedProx own higher standard deviations and are more unstable while methods designed specifically for pFGL scenarios such as GCFL and FedStar are more stable with lower standard deviations.
5.3
Ablation Study
Effects Key Components Mechanism of FedSSP To better understand the impact of specific design components the overall performance of FedSSP, we conducted an ablation study in which
8

---

FesssP
75
Fastig
Fes
FesProx
FedSage
Fettaar
G2FL
(a)
single-domain (SM)
(b) double-domain(SMMCY
(c) Multi-domain (SM-SN-CV)
Figure 3: Test accuracy curves of FedSSP and six other methods along the communication rounds on our three different SeM, SM-CV, SM-SN-CV). The y-axis range is from 65 to 85 for all settings.
we varied these components single, double, and multi-domain settings(SM, SM-CV, SM-SN-CV). As shown in Tab. 2 compared to FedAvg, GSKS significantly enhances accuracy when applied independently. Correspondingly, as a further exploration of the nature of GNN message passing, PGPA achieves noticeable success in adjusting client preferences.
Effects of Key Component Mechanism of GSKS Tab. 3 discuss the key component of GSKS. We demonstrated the impact of different sharing strategies. Specifically, sharing only non-generic spectral GNN components or all components often fails to outperform self-training, while GSKS successfully dominates all the strategies. Accordingly, the results fully validate the effectiveness of GSKS sharing universal knowledge and promoting effective collaboration in this scenario.
Table Ablation study of key components of FedSSP on single-domain, double-domain, and multi- domain settings (SM, SM-CV, SM-SN-CV).
Table 3: Ablation study of key components of GSKS single-domain, double-domain, and multi-domain settings (SM, SM-CV, SM-SN-CV).
Setting
Setting
GSKS
PGPA
Ours
Other
SM
SM-CV
SM-SN-CV
SM
SM-CV
SM-SN-CV
X
X
74.12
81.21
75.33
X
77.33
82.24
77.83
/
X
77.83
82.78
78.54
V
74.12
81.21
75.33
X
/
74.59
81.33
76.12
X
77.21
81.64
78.17
79.62
84.29
79.12
/
77.83
82.78
78.54
5.4 Hyper-parametts Study
We compare the graph classification performance under different values of PGPA parameter momentum μ. number of attention heads, and hidden dimension. Where Fig. 4 shows the results when these T can affect the strength of PGPA while performance is not influenced much unless they are set to extreme values. All studies of and f here outperform the baseline. We also test the performances under different values of attention heads and hidden dimensions. For results in Tab. [ heads for the multi-head attention mechanism while 128 for the hidden dimension.
are fixed at different scales and values. It indicates that the choosing of
we set up 4
300
3000
0.5
(a) PGPA Parameter
(b) Momentum 1
(c) Attention Heads
(d) Hidden Dimension
Figure 4: Analysis hyper-parameter in FedSSP. Graph classification results under different attention heads, and hidden dimensions. Colors green, blue, and yellow refer to performance single, double, and multi-domain settings (SM, SM-CV, SM-SN-CV). dashed lines of corresponding colors represent baseline test accuracy for each setting, which includes only the GSKS strategy.

---

Discussion
Even though FedSSP has achieved significant success in cross-domain federated graph learning collaborations, it still faces certain limitations a spectral GNN-based approach. Compared to spatial GNNs, while spectral GNNs have the advantage of overcoming structural heterogeneity from the spectral domain, many spectral GNNs require eigenvectors and eigenvalues, which adds to the computational overhead of data preprocessing and subsequent storage burden.
Furthermore, we notice a similar approach in DBE [90] which employs static global consensus in FL to separate personalized and global information. Nevertheless, it inevitably struggles to handle scenarios where the peesagg-passing of multiple GNNs is continuously updated. It merely provides a static anchor point, making it difficult to establish a global graph modeling consensus that could guide the local GNNs in capturing graph signals. Instead, we align the GNN backbone with dynamic global graph modeling to avoid the preference module from overly extracting features that should be captured by the GNN itself, which could lead to decreased GNN performance and hinder global collaboration. This approach allows for real-tima time adjustment of pressage-passingg across different client GNNs, focusing the preference module solely on personalization. Additionally, address issues such as sample size disparity between domains and dominance of large domains in model parameter aggregation, adopt direct averaging strategy in dynamic global aggregation instead of conventional weighted averaging to mitigate these concerns.
7 Conclusion
In this paper, we pioneer an innovative exploration of cross-domain personalized Federated Graph Learning. To achieve this goal, we achieve improvements in two aspects: seeking effective global collaboration and suitable local application, thus proposing novel framework FedSSP. For global collaboration, GSKS is leveraged to facilitate the sharing of generic spectral knowledge, overcoming knowledge conffict by domain structural shift from spectral perspective. For local applications, we design PGPA to satisfy inconsistent preferences of specific datasets contained in clients. By integrating these two strategies, FedSSP outperforms various state-of-the-art methods various cross-dataset and cross-domain pFGL scenarios in graph classification.
8 Acknowledgment
This work is supported by National Natural Science Foundation of China under Grant 16662,, 62176188, 623B2080), Key Research and Development Project of Hubei Province (2022BAD175), and the Luojia Undergraduate Innovation Research Fund Project of Wuhan University. The numerical calculations this paper have been supported by the super-computing system in the Supercomputing Center of Wuhan University.
References
[1] Jinheon Baek, Wonyong Jeong, Jiongdao Jin, Jaehong Yoon, and Sung Ju Hwang. Personalized subgraph federated learning. In ICML,, pages 1396-1415, 2023.
[2] Deyu Bo, Yuan Fang, Yang Liu, and Chuan Shi. Graph contrastive learning with stable and scalable spectral encoding. In NeurlPS, volume 36, 2023.
[3] Deyu Bo, Chuan Shi, Lele Wang, and Renjie Liao. Specformer: Spectral graph neural networks meet transformers. arXiv preprint arXiv:2303. 000088 2023.
[4] Deyu Bo, Xiao Wang, Yang Liu, Yuan Fang. Yawen Li, and Chuan Shi. A survey on spectral graph neural networks. arXiv preprint arXiv:2302.056.. 2023.
[5] Shaofei Cai, Liang Li, Jincan Deng, Beichen Zhang, Zheng-Jun Zha, Li Su, and Qingming Huang. Rethinking graph neural architecture search from message-passing. In CVPR, pages 6657-6666, 2021.
[6] Hong- You Chen and Wei-Lun Chao. On bridging generic and personalized federated learning for image classification. arXiv preprint arXiv:2107.00778 2021.
10

---

[7] Jinsong Chen, Kaiyuan Gao, Gaichao Li, and Kun He. Nagphormer: A tokenized graph transformer for node classification large graphs. In ICLR. 2023.
[8] Jinsong Chen, Siyu Jiang, and Kun He. Ntformer: A composite node tokenized graph trans- former for node classification. CoRR, abs/2406. 19249, 2024.
[9] Jinsong Chen, Boyu Li, Qiuting Hc, and Kun He. PAMT: A novel propagation based approach via adaptive similarity mask for node classification. IEEE Trans. Comput. Soc. Syst., 11(5):5973- 5983, 2024.
[10] Jinsong Chen, Hanpeng Liu, John E. Hopcroft, and Kun He. Leveraging contrastive learning for enhanced node representations tokenized graph transformers. NeurIPS, 2024.
[11] Mingyang Chen, Wen Zhang, Zonggang Yuan, Yantao Jía, and Huajun Chen. Fede: Embedding knowledge graphs in federated setting. In ICCGG, pages 80-88, 2021.
[12] Zhiqian Chen, Fanglan Chen, Lei Zhang, Taoran Ji, Kaiqun Fu, Liang Zhao, Feng Chen, Lingfei Wu, Charu Aggarwal, and Chang-Tien Lu. Bridging the gap between spatial and spectral domains: A unified framework for graph neural networks. ACM Computing Surveys, pages
1 1-42, 2023.
[13] Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. In NeurlPS, pages 3557- 3568, 2020.
[14] Xiuwen Fang and Mang Ye. Robust federated learning with noisy and heterogeneous clients. In CVPR, pages 10072-10081, 2022.
[15] Xiuwen Fang, Mang Ye, and Xiyuan Yang. Robust heterogeneous federated learning under data corruption. In ICCV, pages 0020-5030, 2023.
[16] Jiarui Feng, Yixin Chen, Fuhai Li, Anindya Sarkar, and Muhan Zhang. How powerful are k-hop message passing graph neural networks. In NeurlPS, pages 4776-4790, 2022.
[17] Miroslav Fiedler. Algebraic connectivity of graphs. Czechoslovak mathematical journal, 1973.
[18] Xingbo Fu, Binchi Zhang, Yushun Dong, Chen Chen, and Jundong Li. Federated graph machine learning: A survey of concepts, techniques, and applications. arXiv preprint 18Xiv:2207.11812, 2022.
[19] Johannes Gasteiger, Aleksandar Bojchevski, and Stephan Günnemann. Predict then propagate: Graph neural networks meet personalized pagerank. arXiv preprint arXiv: arXiv: /10005597, 2018.
[20] Chaoyang Hc, Keshav Balasubramanian. Emir Ceyani, Carl Yang, Han Xie, Lichao Sun, Lifang He, Liangwei Yang, Philip S Yu, Yu Rong, et al. Fedgraphon: A federated learning system and benchmark for graph neural networks. In ICLR, 2021.
[21] Mingguo He, Zhewei Wei, and Ji-Rong Wen. Convolutional neural networks graphs with chebyshev approximation, revisited. In NeurlPS, pages 7264-7276, 2022.
[22] Mingguo He, Zhewei Wei, Hongteng Xu, et al. Bernnet: Learning arbitrary graph spectral filters via bernstein approximation. In NeurlPS, pages 14239-14251, 2021.
[23] Ming Hu, Yue Cao, Anran Li, Zhiming Li, Chengwei Liu, Tianlin Li, Mingsong Chen, and Yang Liu. Fedmut: Generalized federated learning via stochastic mutation. In AAAI, pages 2024.
[24] Ming Hu, Peiheng Zhou, Zhihao Yue, Zhiwei Ling, Yihao Huang, Anran Li, Yang Liu, Xiang Lian, and Mingsong Chen. Fedcross: Towards accurate federated learning via multi-model cross- aggrrggggg... In ICDE, pages 2137-2150. 2024.
[25] Wenke Huang, Guancheng Wan, Mang Ye, and Bo Du. Federated graph semantic and structural learning. In IJCAI, pages 3830-3838, 2023.
[26] Wenke Huang, Mang Ye, Bo Du, and Xiang Gao. Few-shot model agnostic federated learning. In ACM MM, pages 7309-7311 7016-711122220 2022.
[27] Wenke Huang. Mang Ye, Zekun Shi, Guancheng Wan, He Li, and Bo Du. Parameter disparities dissection for backdoor defense in heterogeneous federated learning. In NeurlPS, 2024.
[28] Wenke Huang, Mang Ye, Zekun Shi, Guancheng Wan, He Li, Bo Du, and Qiang Yang. Federated learning for generalization, robustness, fairness: A survey and benchmark. TPAMI, 2024.
11

---

[29]
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Ar- jun Nitin Bhagoji. Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings. et al. Advances and open problems in federated learning. arXiv preprint arXiv:191..0000 1912.0497, 2019.
[30] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J Reddi, Sebastian U Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for on-device federated learning. In ICML, pages 5132-5143, 2020.
[31] D Kinga, Jimmy Ba Adam, al. A method for stochastic optimization. In ICLR, 2015.
[32] Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létourneau, and Prudencio Tossou. Rethinking graph transformers with spectral attention. In NerulPS, pages 21618-21629, 2021.
[33] Qinbin Li, Bingsheng He, and Dawn Song. Model-contrastive federated learning. CVPR, pages 10713-10722, 2021.
[34] Qinbin Li, Zeyi Wen, Zhaomin Wu, Sixu Hu, Naibo Wang, Yuan Li, Xu Liu, and Bingsheng He. survey on federated learning systems: vision, hype and reality for data privacy and protection. IEEE TKDE, pages 3347-3366, 2021.
[35] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. arXiv preprint arXiv: 181.06611 2020.
[36] Xin-Chun Li, De-Chuan Zhan, Yunfeng Shao, Bingshuai Li, and Shaoming Song. Fedphp:
Federated personalization with inherited private models. In ECML, 2021.
[37] Xunkai Li, Zhengyu Wu, Wentao Zhang, Yinlin Zhu, Rong- Hua Li, and Guoren Wang. Fedgta: Topology-aware averaging for federated graph learning. In VLDB, 2024.
[38] Ke Liang, Lingyuan Meng, Meng Liu, Yue Liu, Wenxuan Tu, Siwei Wang, Sihang Zhou, Xinwang Liu, Fuchun Sun, and Kunlun He. A survey of knowledge graph reasoning on graph types: Static, dynamic, and
TPAMI, 2024.
[39] Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B Allen, Randy P Auerbach, David Brent, Ruslan Salakhutdinov, and Louis-Philippe Morency. Think locally, act globally: Federated learning with local and global representations. arXiv preprint urXiv:2001.01522 2020.
[40] Renjie Liao, Zhizhen Zhao, Raquel Urtasun, and Richard S Zemel. Lanczosnet: Multi-scale deep graph convolutional networks. In ICLR, 2019.
[41] Rui Liu and Han Yu. Federated graph neural networks: Overview, techniques and challenges. arXiv preprint arXiv:2202.0723: 07256, 2022.
[42] Weiming Liu, Xiaolin Zheng, Jiajie Su, Mengling Hu, Yanchao Tan, and Chaochao Chen. Exploiting variational domain-invariant user embedding for partially overlapped cross domain recommendation. In SIGIR, pages 312-321, 2022.
[43] Yang Liu, Deyu Bo, and Chuan Shi. Graph condensation via eigenbasis matching. arXiv preprint vrXiv:2310009200 2023.
[44] Yue Liu, Ke Liang, Jun Xia, Xihong Yang, Sihang Zhou, Meng Liu, Xinwang Liu, and Stan Z Li. Reinforcement graph clustering with unknown cluster number. In ACMMM, pages 3528-3537, 2023.
[45] Yue Liu, Xihong Yang, Sihang Zhou, and Xinwang Liu. Simple contrastive graph clustering. TNNLS, 2023.
[46] Yue Liu, Xihong Yang. Sihang Zhou, Xinwang Liu, Zhen Wang, Ke Liang, Wenxuan Tu, Liang Li, Jingcan Duan, and Cancan Chen. Hard sample aware network for contrastive deep graph clustering. In AAAI, pages 8914-8922 2023.
[47] Zewen Liu, Yunxiao Li, Mingyang Wei, Guancheng Wan, Max SY Lau, and Wei Jin. Epilearn: python library for machine learning in epidemic modeling. arXiv preprint arXiv:2406.06016 2024.
[48] Zewen Liu, Guancheng Wan, B Aditya Prakash, Max SY Lau, and Wei Jin. A review of graph neural networks in epidemic modeling. In ACM SIGKDD, pages 6577-6587, 2024.
[49] Andreas Loukas and Pascal Frossard. Building powerful and equivariant graph neural networks with structural peessag--passing NeurIPS, 2020.
12

---

[50] Jun Luo and Shandong Wu. Adapt to adaptation: Learning personalization for cross-silo federated learning. In IJCAI, pages 2166-2175 2022.
[51] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication--fficcee learning of deep networks from decentralized data. In AISTATS, pages 1273-1282, 2017.
[52] Christopher Morris, Nils M Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. Tudataset: A collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663 2020.
[53] Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd, The pagerank citation ranking: Bring order to the web. Technical report, 1998.
[54] Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. Recipe for general, powerful, scalable graph transformer. NeurlPS, pages 14501-14515, 2022.
[55] Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. Self-supervised graph transformer on large-scale molecular data. In NeurIPS, pages 12559-12571, 2024.
[56] Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski. and Stephan Günnemann. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:7811.0586. 2018.
[57] David Shuman, Sunil K. Narang, Pascal Frossard, Antonio Ortega, and Pierre Vandergheynst. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing. Mag., 2013.
(58) Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar. Federated multi-task learning. In NeurlPS, 2017.
[59] Ruoxi Sun, Hanjun Dai, and Adams Wei Yu. Does gnn pretraining help molecular representa- tion? In NeurlPS, pages 12096-12109, 2022.
[60] Canh T Dinh, Nguyen Tran, and Josh Nguyen. Personalized federated learning with moreau envelopes. In NeurlPS, pages 21394-21405, 2020.
[61] Yue Tan, Yixin Liu, Guodong Long, Jing Jiang, Qinghua Lu, and Chengqi Zhang. Federated learning on non-iid graphs via structural knowledge sharing. In AAAI, pages 9953-9961, 2023
[62] Clement Vignac, Andreas Loukas, and Pascal Frossard. Building powerful and equivariant graph neural networks with structural message-passing In NewrlPS, pages 14143-14155, 2020.
[63] Guancheng Wan, Zewen Liu, Max S. Lau, B. Aditya Prakash, and Wei Jin. Epidemiology- aware neural ode with continuous disease transmission graph. arXiv preprint arXiv:2410.00049, 2024.
[64] Guancheng Wan, Yijun Tian, Wenke Huang, Nitesh Chawla, and Mang Ye. S3GCL: Spectral, swift, spatial graph contrastive learning. In ICML, 2024.
[65] Aoran Wang and Jun Pang. Iterative structural inference of directed graphs. NeurlPS, 35:8717- 8730, 2022.
[66] Aoran Wang and Jun Pang. Active learning based structural inference. In ICMI,, pages 36224-366 -3245,, 2023.
[67] Aoran Wang and Jun Pang. Structural inference with dynamics encoding and partial correlation coefficients. In ICIR, 2024.
[68] Aoran Wang, Tsz Pan Tong, and Jun Pang. Effective and efficient structural inference with reservoir computing. In ICML, pages 36391-36410, 2023.
[69] Binwu Wang, Jiaming Ma, Pengkun Wang, Xu Wang, Yudong Zhang, Zhengyang Zhou, and Yang Wang. Stone: ppatio-temporal ood learning framework kills both spatial and temporal shifts. In SIGKDD, pages 2948-2222 2959, 2024.
[70] Binwu Wang, Pengkun Wang, Yudong Zhang, Xu Wang, Zhengyang Zhou, Lei Bai, and Yang Wang. Towards dynamic spatial-temporal graph learning: A decoupled perspective. In AAAI, pages 9089-9097, 2024.
[71] Binwu Wang, Pengkun Wang, Yudong Zhang, Xu Wang, Zhengyang Zhou, and Yang Wang. Condition-guided urban traffic co-prediction with multiple sparse surveillance data. TVT, 2024.
13

---

[72] Binwu Wang, Yudong Zhang, Jiahao Shi, Pengkun Wang, Xu Wang, Lei Bai, and Yang Wang. Knowledge expansion and consolidation for continual traffic prediction with expanding graphs IEEE TTTS, 2023.
[73] Binwu Wang, Yudong Zhang, Xu Wang, Pengkun Wang, Zhengyang Zhou, Lei Bai, and Yang Wang. Pattern expansion and consolidation evolving graphs for continual traffic prediction. In SIGKDD, pages 2223-2232, 2023.
[74] Xiyuan Wang and Muhan Zhang. How powerful are spectral graph neural networks. In ICML, pages 23341-23362, 2023.
[75] Yili Wang, Yixin Liu, Xu Shen, Chenyu Li, Kaize Ding, Rui Miao, Ying Wang, Shirui Pan, and Xin Wang. Unifying unsupervised graph-level anomaly detection and out-of-distributtin detection: A benchmark. arXiv preprint arXiv:2406. 15523, 2024.
[76] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Yu Philip. A comprehensive survey on graph neural networks. IEEE TNNLS, pages 4-24, 2020.
[77] Han Xie, Jing Ma. Li Xiong, and Carl Yang. Federated graph classification over non-iid graphs NeurIPS, 34:1 044118839-8882 2021.
[78] Siheng Xiong, Yuan Yang, Faramarz Fekri, and James Clayton Kerce. Tilp: Differentiable
learning of temporal logical rules on knowledge graphs. In ICLR, 2023.
[79] Siheng Xiong, Yuan Yang, Ali Payani, James C Kerce, and Faramarz Fekri. Teilp: Time prediction over knowledge graphs via logical reasoning. In AAAI, volume 38, pages 16112- 16119, 2024.
[80] Dongkuan Xu, Wei Cheng, Dongsheng Luo, Haifeng Chen, and Xiang Zhang. Infogel: Inforretion-awwree graph contrastive learning. In NewrIPS, pages 30414-30425, 2021.
[81] Beining Yang. Kai Wang. Qingyun Sun, Cheng Ji, Xingcheng Fu, Hao Tang, Yang You, and Jianxin Li. Does graph distillation see like vision dataset counterpart? In NeurlPS, volume 36, 2023.
[82] Mang Ye, Xiuwen Fang, Bo Du, Pong C Yuen, and Dacheng Tao. Heterogeneous federated learning: State-of-the-art and research challenges. ACM Computing Surveys, 2023.
[83] Mang Ye, Wenke Huang, Zekun Shi, He Li, and Du Bo. Revisiting federated learning with label skew: An overconfidence perspective. SCIS, 2024.
[84] Mang Ye, Wei Shen, Eduard Snezhko, Vassili Kovalev, Pong C Yuen, and Bo Du. Verti- cal federated learning for effectiveness, security, applicability: A survey. arXiv preprint arXiv:2405.1 /495, 2024.
[85] Chengxuan Ying. Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Yan Liu. Do transformers really perform badly for graph representation?, 2021.
(86) Guibin Zhang, Yanwei Yue, Zhixun Li, Sukwon Yun, Guancheng Wan, Kun Wang, Dawei Cheng, Jeffrey Xu Yu, and Tianlong Chen. Cut the crap: An economical communication pipeline for llmebased multi-agent systems. arXiv preprint arXiv.2410.02506, 2024.
[87] Guibin Zhang, Yanwei Yue, Xiangguo Sun, Guancheng Wan, Miao Yu, Junfeng Fang, Kun Wang, and Dawei Cheng. G-designer: Architecting multi-agent communication topologies via graph neural networks. arXiv preprint arXiv:2410. 1782, 2024.
[88] Guixian Zhang, Shichao Zhang, and Guan Yuan. Bayesian graph local extrema convolution with long-tail strategy for misinformation detection. In SIGKDD, pages 1-21, 2024.
[89]
Huanding Zhang. Tao Shen, Fei Wu, Mingyang Yin, Hongxia Yang. and Chao Wu. Federated graph learning a position paper. arXiv preprint arXiv:2105.11099 2021.
[90] Jianqing Zhang, Yang Hua, Jian Cao, Hao Wang, Tao Song, Zhengui XUE, Ruhui Ma, and Haibing Guan. Eliminating domain bias for federated learning in representation space. In NeurlPS, volume 36, 2023.
[91] Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Fedala: Adaptive local aggregation for personalized federated learning. In AAAI, pages 11237-11244, 2023.
[92] Jianqing Zhang, Yang Hua, Hao Wang, Tao Song. Zhengui Xue, Ruhui Ma, and Haibing Guan. Fedep: Separating feature information for personalized federated learning via conditional policy. In SIGKDD, pages 3249-3261, 2023.
14

---

[93] Ke Zhang, Carl Yang, Xiaoxiao Li, Lichao Sun, and Siu Ming Yiu. Subgraph federated learning with missing neighbor generation. In NeurIPS, volume 34, pages 6671-6682, 2021.
[94] Yanfu Zhang, Hongchang Gao, Jian Pei, and Heng Huang. Robust self-supervised structural
graph neural network for social network prediction. In WWW, pages 1352-1361, 2022.
[95] Liwang Zhu, Qi Bao, and Zhongzhi Zhang. Minimizing polarization and disagreement in social networks via link recommendation. In NeurlPS, volume 34, pages 2072-2084, 2021.
[96] Meiqi Zhu, Xiao Wang, Chuan Shi, Houye Ji, and Peng Cui. Interpreting and unifying graph neural networks with an optimization framework. In WWW, pages 1215-1226, 2021.
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the contributions and scope of this paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
We discuss the limitations in Sec. 6
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
15

---

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We fully disclose all the information needed to reproduce the main experi- mental results in this paper and our code. We convinced that the obtained results can be reproduced.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
16

---

(c) If the contribution is a new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
Code is accessible in this paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/eetails
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: Details are included in Sec. 5.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
17

---

Justification:
Statistical significance of the experiments is considered and included in the main claims of the paper.
Sec. [.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide sufficient information on the computer resources.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: http//urr....c.pplleeeeeeteeeeee
Answer: [Yes]
Justification: The research conducted in the paper strictly adheres the NeurIPS Code of Ethics, ensuring that all aspects of the work are in compliance with the guidelines provided.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
18

---

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The research presented in this paper foundational. It not directly tied to any specific applications or deployments.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (c.g., dep.lymeln of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
This paper poses no such risks
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: This paper does not existing assets.
The answer NA means that the paper does not existing assets.
19

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the datasets..paerssttt paperswi thcode. com/datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
20

---

Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
21


Reviews:
[{"review_id": "Hwmj8o4Ofb", "reviewer": "Reviewer_LD7P", "summary": "FedSSP tackles structural heterogeneity well in personalized Federated Graph Learning. When it comes to cross-domain scenarios, the structural heterogeneity becomes more negative than usual. It is crucial to mitigate the impact of domain structural shifts. FedSSP proposes two strategies to address these challenges from two directions. It firstly overcomes knowledge conflict by generic spectral encoder weights to seek better collaboration across various datasets. Besides, considering the uniqueness of clients in this scenario, it then satisfies the special preferences of the graphs by feature adjusting.", "questions": "Please refer to the weaknesses.", "limitations": "The experiment should include performance under varying conditions to ensure the method has no limitations in this aspect, such as different numbers of training rounds.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Studies on cross-domain federated graph learning are crucial for the applications of federated learning in the real world. This would help increase the generalizability in practical applications. This study promotes the generalizability of federated graph learning and demonstrates superior performance in various simulated settings, proving its advantage and broad applicability in practical applications.\n- The authors present a novel perspective and a spectra-based solution to tackle the issue of structural heterogeneity among clients. The proposed approach effectively addresses the variations in graph structures across clients, thereby enhancing the overall coherence and performance of the federated learning framework.\n- The generic spectral knowledge sharing and preference modules work harmoniously and complement each other effectively. After addressing knowledge conflicts through spectral bias mitigation, personalized adjustments act as further personalized optimizations, enhancing generic message passing. They together enable adaptive collaboration across diverse datasets and domains.", "weaknesses": "The client scale considered in the experiments is quite small. Cross-device federated learning typically involves a significantly larger client scale to ensure the robustness and generalizability of the results. Therefore, it would be beneficial to consider expanding the client scale in experiments to better reflect practical scenarios and to validate the scalability of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer LD7P:\n\nThank you for your encouraging comments on the coordination of our methods and the overall significance of our work. We hope that our responses below will address your concerns and reinforce your positive evaluation.\n\n### Weakness\n\n**W1: The client scale in the experiments is too small to reflect practical scenarios.**\n\nA1: For performance analysis under larger client scales, our method is extended to scenarios where each client possesses a portion of a dataset and different clients may have different datasets.\n\nTo demonstrate the performance of FedSSP under varying client scales, we conducted experiments in SM cross-dataset settings with client scales ranging from 7 to 77 where each of the seven small molecule datasets was split into 1-11 segments. The results in the following Table denote that FedSSP consistently outperformed the other methods across all scenarios.\n\n*Table: **Performance under varying client scales***\n\n| Client Number | 7    | 21   | 35   | 49   | 63   | 77   |\n|:---:| :--:|:---:|:---:|:---:|:---:|:---:|\n| FedAvg       | 74.12 | 71.92 | 70.75 | 71.16 | 73.66 | 74.03 |\n| FedStar   | 78.63 | 76.37 | 74.18 | 73.20 | 75.81 | 78.28 |\n| FedSSP     | **79.62** | **76.74** | **74.86** | **74.31** | **76.25** | **79.44** |\n\n### Limitations\n\n**L1: The experiment lacks performance evaluation under varying conditions, such as different numbers of training rounds.**\n\nA2: To demonstrate the performance of FedSSP under varying local training rounds, we conducted experiments in SM cross-dataset scenarios under different local training rounds. The results in the following Table denote that FedSSP consistently outperforms other methods in all four settings of local training rounds.\n\n*Table: **Performance under varying local training rounds ***\n\n| Local Training Round | 1  | 2 | 3 | 4 |\n|:---:| :--:|:---:|:---:|:---:|\n| FedAvg        | 74.12 | 75.56 | 74.75 | 73.81 |\n|FedStar    | 78.63 | 79.35 | 79.42 | 79.04 | \n| FedSSP       | **79.62** | **80.78** | **80.13** | **79.56** |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "31DzO0l4P9", "reviewer": "Reviewer_jSYo", "summary": "This work proposed a personalized federated graph learning framework for federated graph classification. The framework includes strategies for sharing generic knowledge and satisfying personalized preferences. The authors evaluated six different cross-dataset and cross-domain settings and showed good performance.", "questions": "Q1) I have some confusion regarding the two spectral bias metrics used in the motivation. Could the authors specifically point out the relationship between these two spectral bias metrics and graph structure?\nQ2) Could the authors add experiments on the scalability of the method, such as performance with varying numbers of clients?\nQ3) The global consensus in the method is similar to the prototype. If it differs from prototype, authors should provide distinctions and explanations.", "limitations": "Yes", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The writing in this paper is coherent, making the proposed concepts and methodologies easy to follow. The logical flow and concise explanations enhance overall readability and comprehension.\n2. The methods are thoughtfully designed with a deep understanding of the specific challenges inherent in cross-domain scenarios. They effectively address these targeted issues, ensuring good performance and adaptability across diverse datasets and domains.", "weaknesses": "1. In the motivation, the authors do not clearly explain the relationship between the two spectral bias metrics and the graph structure.\n2. The scalability of the method is not well analyzed in the experiment. \n3. Absence of a detailed discussion on the necessity of the shared filter encoder in the proposed method, particularly in relation to its specific architecture.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer jSYo:\n\nWe sincerely appreciate your time and effort in reviewing our paper, and we are grateful for your positive feedback on our writing and the effectiveness of our methodology. We hope that our responses below will address your concerns and lead to an updated score.\n\n### Weakness\n\n**W1 & Q1: How spectral bias metrics are related to graph structure?**\n\nA1: Eigenvalues and the Fiedler value are significantly associated with the graph structure. In the first place, eigenvalues and eigenvectors can be employed for graph clustering and partitioning. For instance, the k smallest non-zero eigenvalues and their corresponding eigenvectors can be utilized for k-means clustering of the graph. Furthermore, the Fiedler value can assess the connectivity of the graph [1]. The Fiedler value and its corresponding eigenvector, known as the Fiedler vector, also play a crucial role in community identification [2].\n\n**W2 & Q2: Validation of the scalability of the proposed method under varying numbers of clients.**\n\nA2: Theoretically, our method can be extended to scenarios where each client possesses a portion of a dataset and different clients may have different datasets. Therefore, we validated the scalability of our method based on these scenarios.\n\nWe conducted experiments in SM cross-dataset settings under varying client scales. Specifically, we performed experiments with the client scales ranging from 7 to 77 where each of the seven small molecule datasets was split into 1-11 segments. The results in the following Table denote that FedSSP consistently outperforms the other methods across all scenarios.\n\n*Table: **Performance under varying client scales***\n\n| Client Number | 7    | 21   | 35   | 49   | 63   | 77   |\n|:---:| :--:|:---:|:---:|:---:|:---:|:---:|\n| FedAvg       | 74.12 | 71.92 | 70.75 | 71.16 | 73.66 | 74.03 |\n| FedStar   | 78.63 | 76.37 | 74.18 | 73.20 | 75.81 | 78.28 |\n| FedSSP     | **79.62** | **76.74** | **74.86** | **74.31** | **76.25** | **79.44** |\n\n**W3: Absence of a detailed discussion on the necessity and architecture of the shared filter encoder.**\n\nA3: Filter encoders encapsulate knowledge of various frequency components which affects how much graph signal varies from nodes to their neighbors for better graph convolution construction. This strategy benefits clients by enabling them to acquire knowledge of transmitting graph signals at different frequencies from other GNNs, thereby promoting the construction of graph convolutions for individual clients. Furthermore, the functionality is discussed in detail on line 253 of our paper.\n\n### Questions\n\n**Q3: How does the prototype differ from global consensus?**\n\nA4: Our paper focuses on federated graph learning in cross-dataset and cross-domain scenarios, which denotes that each client possesses its unique class information, thus rendering the concept of prototypes inapplicable here. The global consensus in this paper is derived from the feature mean of graphs across all clients in each round, without the class information contained in prototypes. Global consensus represents the knowledge for graph data modeling by all participating GNNs and provides standardized global graph processing knowledge, which the prototypes cannot achieve in this scenario. In addition, this difference is discussed in detail on line 253 of our paper. We hope this provides the necessary clarification.\n\n[1] Fiedler M. Algebraic connectivity of graphs[J]. Czechoslovak mathematical journal, 1973, 23(2): 298-305.\n\n[2] Von Luxburg U. A tutorial on spectral clustering[J]. Statistics and computing, 2007, 17: 395-416.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "G3MzPuaUs9", "reviewer": "Reviewer_3kee", "summary": "The paper introduces a novel method FedSSP designed to address the current limitations of personalized Federated Graph Learning methods. The author highlighten that existing methods fail to deal with domain structural shift and ignore the uniqueness of datasets in cross-dataset scenarios. To address the limitations, FedSSP innovatively utilizes spectral GNNs and shares key layers in them to facilitate generic communication. Meanwhile, the authors propose preference module for the preferences derive from different graph datasets.", "questions": "Please refer to the Weakness part.", "limitations": "Authors should discuss communication cost for this potential limitation.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Addressing problems in pFGL from a spectral perspective is novel and interesting, setting a new direction for future research in federated graph learning.\n2. The motivation is clear and compelling. The sharing strategy tackles the issue of knowledge conflict and spectral bias. Preference adjustment is employed to cater to the structural characteristics and preferences of individual datasets, thus ensuring that the graph features extracted are highly relevant and suitable.\n3. The experiment is comprehensive, covering a wide range of scenarios to thoroughly evaluate the proposed methods. The ablation studies are thoroughly conducted, providing clear insights into the contributions of each component and demonstrating its effectiveness.", "weaknesses": "1. Since the method of spectral encoder sharing essentially belongs to the personalized layer methods, I believe the authors need to discuss the differences between this method and other personalized layer methods in this paper, which they did not, such as APPLE.\n2. How the node-level method FGSSL is used here in the experiments? What is the core model of APPLE here in the federated graph learning scenario? Could the authors provide the implementation details of the baselines?\n3. The author changed the model architecture here but has not compared its communication cost with baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer 3kee:\n\nWe deeply appreciate your positive feedback regarding the innovative aspects of our methods and the comprehensiveness of our experiments. Thank you for your time and effort in reviewing our paper. We hope that our responses below will address your concerns and further affirm the quality of our research.\n\n### Weakness\n\n**W1: Insufficient discussion on differences between spectral encoder sharing and other personalized layer methods like APPLE.**\n\nA1: Following the implementation of the personalized layer method FedPer in previous FGL research, the implementation of APPLE here shares the graph convolution parameters in GNNs. Due to the lack of targeted solutions for cross-dataset and cross-domain scenarios, direct sharing of all graph convolution parameters inevitably struggles with structural conflict.\n\nInstead, collaboration under our proposed solution overcomes the conflict from the spectral perspective. Based on our investigation of domain spectral biases, we achieved effective cross-domain collaboration by exploring components unaffected by structural bias in spectral GNN.\n\n**W2: Lack of implementation details for node-level method FGSSL and core model APPLE in federated graph learning.**\n\nA2: FGSSL primarily includes two strategies: federated graph structure distillation (FGSD) to enhance node feature modeling, and federated node semantic contrast (FNSC) to encourage the neighborhood modeling and structural processing of each client to approach the global level.\n\nThe node features before GNN average pooling is leveraged for similarity matrix calculation in FGSD. Specifically, in our graph-level task, each client possesses multiple graphs, each graph having its feature similarity matrix. Based on this, they converge towards the inherent adjacency relationships provided by the global model. For FNSC, since there are no unified node labels across clients in the scenario, we implemented contrastive learning using an unsupervised strategy to replicate feature optimization of FGSSL in graph-level scenarios.\n\nIn addition, the core model part in APPLE here refers to the graph convolution parameters as we illustrated in A1.\n\n**W3: No comparison of communication cost with baselines.**\n\nA3: Since our strategy only shares knowledge of the eigenvalue encoder and filter encoder, it is certain that our communication cost is significantly lower than all baselines. Whether it is the traditional approach of sharing all parameters, the sharing of structure processing channels in FedStar, or the sharing of graph convolution parameters in APPLE, our communication cost is lower. If accepted, our final version will include a discussion on communication costs.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1fstH8RwPf", "reviewer": "Reviewer_cuMU", "summary": "This paper focuses on cross-domain Federated Graph Learning, where graph data stored in clients exists negative domain structural shifts. Authors observe the presence of spectral biases as a reflection of structural shifts. Thereafter, this work proposes the Generic Spectral Knowledge Sharing (GSKS), which allows clients to share certain components containing generic spectral knowledge. Moreover, it proposes Personalized Graph Preference Adjustment (PGPA) to satisfy preferences of clients for more suitable local applications. Extensive experiments on different scenarios demonstrates the superiority of the proposed methods.", "questions": "I expected a more detailed comparative analysis of FGL baselines mentioned in the weakness. Could the authors provide a discussion on it?", "limitations": "This work solves the problems in cross-domain settings well. I wonder whether it is generalizable enough to be extended to other non-iid problems, such as each client having a portion of the same graph dataset. It’s necessary to add relevant experiments to demonstrate that this method is not limited.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. Well-motivated. In particular, it first observes the spectral biases as a reflection of structural shifts and further provides a targeted solution to address the biases. Moreover, the authors acknowledge the unique preferences of the dataset in this scenario may lead to the global message passing being not fully applicable to local data and accordingly propose appropriate solutions for the particularity of datasets. \n\n2. Very easy to follow. The Figure is well presented. In particular, the authors provide a well-defined motivation visualization, helping with the understanding of specific challenges and the framework details. In addition, the precise framework presentation aids in the comprehension of each module’s role.\n\n3. Structural heterogeneity is a common challenge in federated graph learning. The innovative solution presented in this paper offers a new perspective for future advancement by promoting improvements in spectral GNNs instead of traditional ones. It is not necessarily limited to cross-domain scenarios but can also bring new ideas for addressing structural heterogeneity in other non-IID federated graph learning contexts.", "weaknesses": "1. Lacking of comprehensive comparative analysis of existing FGL methods. For instance, FGSSL appears in the performance table but is not comparatively analyzed in the introduction. The authors are supposed to clarify the shortcomings of this method in the given scenario.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer cuMU:\n\nThank you for your thorough review and the kind words about our well-motivated study and handling of structural heterogeneity. We sincerely appreciate your time and effort. We hope that our responses below will address your concerns.\n\n### Weakness\n\n**W1: Lacking comprehensive comparative analysis of existing FGL method FGSSL.**\n\nA1: In scenarios with strong structural heterogeneity across datasets and domains, alignment to global structural knowledge in FGSSL is inevitably negatively impacted. Even with adjustments to align client neighbor modeling knowledge to the global GNN, it remains challenging to aggregate a highly generalizable global model under domain structural bias. Furthermore, even though its node-level strategies are beneficial for better graph modeling, the effectiveness of the method decreases under the influence of structure-biased message passing. Specifically, clients are inevitably influenced by the message-passing schemes of GNNs trained on graph data with significantly different domain structures, thus inevitably failing to model their local graph data accurately.\n\n\n### Limitations\n\n**Uncertainty about generalizability to other non-iid problems**\n\nA2: Theoretically, our method can be extended to scenarios where each client possesses a portion of a dataset and different clients may have different datasets. Therefore, we validated the scalability of our method in this scenario and conducted the following experiments for demonstration. \n\nSpecifically, we conduct experiments where each of the seven small molecule datasets is split into 1-11 segments. Different from cross-domain and cross-dataset settings in this paper, it simulates non-iid scenarios where different clients may have portions of the same dataset. It denotes that FedSSP consistently outperformed the other methods across all scenarios.\n\n*Table: **Extended non-iid scenarios** where each client possesses a portion of a small molecule dataset*\n\n| Client Number | 7    | 21   | 35   | 49   | 63   | 77   |\n|:---:| :--:|:---:|:---:|:---:|:---:|:---:|\n| FedAvg       | 74.12 | 71.92 | 70.75 | 71.16 | 73.66 | 74.03 |\n| FedStar   | 78.63 | 76.37 | 74.18 | 73.20 | 75.81 | 78.28 |\n| FedSSP     | **79.62** | **76.74** | **74.86** | **74.31** | **76.25** | **79.44** |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper focuses on structural heterogeneity in personalized Federated Graph Learning. Specifically, it effectively addresses the domain structural shift from a spectral perspective, tackling issues such as knowledge conflict and spectral bias. The framework includes Generic Spectral Knowledge Sharing (GSKS), which shares generic spectral knowledge across clients within certain components, while applying Personalized Graph Preference Adjustment (PGPA) to meet the preferences of clients for more personalized results. The experimental settings are appropriate to demonstrate the superiority of the proposed methods, although some limitations (e.g., number of clients, non-i.i.d. scenarios) were highlighted by the reviewers. The reviewers initially noted both strong points and concerns.

Strong points:

- All reviewers praise the clear motivation. Alleviating knowledge conflicts from a spectral perspective is a novel and important challenge in personalized Federated Graph Learning.

- The method is thoughtfully designed, and its performance, as well as the harmony between modules, is commendable.

- Comprehensive experiments demonstrate the effectiveness of the proposed method.

Weak points:

- Analysis of the scalability with respect to the number of clients, performance in non-i.i.d. settings, and communication cost is required.

- Additional explanation of the motivation experiments (Figure 1(c)) and justifications for sharing the filter encoder are needed.

- A more detailed discussion of related works is necessary.

The authors have adequately addressed the reviewers’ concerns. Please be sure to incorporate the rebuttal into the revised version, particularly the clarification of the motivation behind sharing the filter encoder and the detailed explanation of the motivation experiment shown in Figure 1. Moreover, the methodology section should be more refined (e.g., where is the parameter for the eigenvalue encoder (i.e., $\theta^e$) used? How is $B$ calculated?). Lastly, it would be beneficial to include an experiment that effectively demonstrates that generic knowledge has less spectral bias than other knowledge, similar to the motivation experiment in Figure 1(c). Despite some limitations, this paper has strong points that address important challenges from a novel perspective (i.e., spectral), and it holds significant potential to provide valuable insights for future work in graph federated learning.
Decision: Accept (poster)