EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
Authors: Zitong Shi
Guancheng Wan
Wenke Huang
Guibin Zhang
He Li
Carl Yang
Mang Ye
Venue: ICML 2025 poster (2025)
Keywords: Federated Learning
Graph Learning
Sparsification
Primary area: deep_learning->graph_neural_networks

Abstract: Federated Graph Learning (FGL) has gained significant attention as a privacy-preserving approach to collaborative learning, but the computational demands increase substantially as datasets grow and Graph Neural Network (GNN) layers deepen. To address these challenges, we propose $\textbf{EAGLES}$, a unified sparsification framework. EAGLES applies client-consensus parameter sparsification to generate multiple unbiased subnetworks at varying sparsity levels, reducing the need for iterative adjustments and mitigating performance degradation. In the graph structure domain, we introduced a dual-expert approach: a $\textit{graph sparsification expert}$ uses multi-criteria node-level sparsification, and a $\textit{graph synergy expert}$ integrates contextual node information to produce optimal sparse subgraphs. Furthermore, the framework introduces a novel distance metric that leverages node contextual information to measure structural similarity among clients, fostering effective knowledge sharing. We also introduce the $\textbf{Harmony Sparsification Principle}$, EAGLES balances model performance with lightweight graph and model structures. Extensive experiments demonstrate its superiority, achieving competitive performance on various datasets, such as reducing training FLOPS by 82\% $\downarrow$ and communication costs by 80\% $\downarrow$ on the ogbn-proteins dataset, while maintaining high performance.

Federated Graph Learning (FGL) has emerged as a pivotal field decentralized machine learning, harnessing the EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
Zitong Shi"
Guancheng Wan"
Wenke Huang" *1
Guibin Zhang? 2 He Li
Carl Yang?
Mang Ye
Abstract
One hup
Non- conn-cosccssss Pruning
Federated Graph Learning (FGL) has gained significant attention privacy-preserving ap- proach to collaborative learning, but the computa- tional demands increase substantially as datasets grow and Graph Neural Network (GNN) layers deepen. To address these challenges, we pro- pose EAGLES, unified sparsification frame- work. EAGLES applies client-consensus param- eter sparsification to generate multiple unbiased subnetworks at varying sparsity levels, reducing the need for iterative adjustments and mitigating performance degradation. In the graph structure domain, we introduced dual-expert approach: graph sparsification experl multi-criteria node-level sparsification, and a graph synergy pert integrates contextual node information to pro- duce optimal sparse subgraphs. Furthermore, the framework introduces a novel distance metric that leverages node contextual information to measure structural similarity among clients, fostering ef- fective knowledge sharing. We also introduce the Harmony Sparsification Principle, EAGLES balances model performance with lightweight graph and model structures. Extensive experi- ments demonstrate its superiority, achieving com- petitive performance on various datasets, such as reducing training FLOPS by 82% L and com- munication costs by 80% I on the ogbn-proteins dataset, while maintaining high performance. The code is anonymously available at this link.
Pao--oo Rop
Network
Network
hop
Network
Network
"Tmmpuinng resources
Cg Network
Purformance Decroased
rapidly Increasing
Degrnn-haved
Single Criterio
"annot save alone"
0.5.
Information Overliting
Sinueurn
C2 Cg
Figure 1: Problem Illustration. The message- pa-ssagssnnn mechanism in GNNs leads to significant increase in computational demand as dataset sizes grow and the number of GNN layers deepens, Existing studies attempt to reduce computational requirements through model pruning or graph sparsification. However, model pruning approaches often overlook the variability in parameter im- portance across different clients, while certain graph sparsification techniques rely single pruning criterion and fail to account for data heterogeneity, thus limiting their effectiveness.
combined strengths of Federated Learning (FI.) systems (Li et al., 2020; Yang et al., 2016; 2019; Mammen, 2021) and Graph Neural Networks (GNNs) (Luo & Wu, 2022: Lee et al., 2019; Gao & Ji, 2019; Wan al., 2024b). This approach allows multiple clients to collaboratively train without sharing their raw data, ensuring data privacy while effectively extracting knowledge from graph-structured information. It has already been widely applied in various domains such as social networks (Goldenberg, 2021; Barnes, 1969), healthcare (Schrodt et al., 2020; et al., 2022b), finance (Cardoso al., 2020: Saha al.. 2022), and rec- ommendation systems (Gao al., 2023; Wang et al., 2021; Ma et al., 2025). However, as graph data scales continue to expand, the computational cost of message passing grows rapidly, posing significant challenges to the computational resource demands during the training phase of GNNs.
1. Introduction
'Equal contribution 'National Engineering Research Center for Multimedia Software, School of Computer Science. Wuhan University, Wuhan, China 'National University of Singapore. Sin- gapore Depparmenn of Computer Science, Emory University, USA. Correspondence to: Mang Ye <yemang@ whu.edu.en>.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
To address this inefficiency and obtain a better-performing

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
model, numerous effective methods have been proposed.
ROC-AUC
ROC-AUC
These approaches can be mainly categorized into four types:
0%
graph simplification optimization (Chen et al., 2021b;a;
70%
10%
70%
10%
Sohrabi et al., 2021) and model parameter tuning (Ma, 2024;
Yi et al., 2024; Liang et al., 2024; Nguyen et al., 2024), graph-based knowledge distillation (Zhu et al., 2021; Wang et al., 2024b; Zhu et al., 2024), graph structure completion (Back et al., 2023; Chen et al.. 2022; Liang et al., 2024). The former two groups focus on simplifying graph struc- tures and adjusting model parameters. Graph simplification optimization typically reduces the training burden through sampling or pruning. However these strategies often based on a single selection strategy (Li et al., 2022a; Parchas et al., 2018; Fung et al., 2011), which leads to the neglect of criti- cal information inherent within the graph structure. We refer to this phenomenon as structure information overfitting. In model parameter tuning, some methods (Ma, 2024; Yi et al., 2024) consider lightweighting models by pruning redundant parameters. However, these approaches necessitate iterative adjustments to the pruning rate and are not fully suitable for federated systems, due to the fact that the importance of parameters may vary across different client models for the same layer. We present radar charts comparing advanced centralized methods: Dspar and ACE-GLT with our method, as shown in Figure 2. The results indicate that advanced methods relying solely on single approach and disregard- ing federated characteristics also face a significant risk of model performance degradation. Based on the observations, we raise the following question: 1) How can we stream- line models to reduce the training burden while preserving critical graph information?
60%
20%
60%
20%
05per
50%
30%
50%
30%
40%
40%
Weight Sparsity (%)
Graph Sparsity (%)
Figure 2: ROC-AUC Comparison on Ogbn-Proteing using 4- layer DeeperGCN: The radial axis represents ROC-AUC. (left) ACE-GLT focuses on parameter-level sparsification, while (right) Dspar specializes in graph-level sparsification. For a detailed explanation of the ROC-AUC metric, please refer to Appendix B.
sparsification approach, applied at both the graph level and the model parameter space. At graph level, we deploy two types of expert models at the client to solve structure in- formation overfitting, referred to as sparsification experts and synergy experts. Each sparsification expert specializes in a particular sparsification criterion, such as jaccard sim- ilarity, spectral sparsification, and others. They perform sparsification on the one-hop subgraph of each node based on specific expertise. To better integrate the sparsified sub- graphs from multiple experts, synergy expert models node features employing hard concrete distribution to learn struc- tural information and dynamically select and integrate key information from different sparsified versions for each node. By the combined effect of the two types of expert, we re- duce graph size, enabling more efficient message passing and alleviating structure information overfitting. While at the parameter space, we allow sub-networks with various pruning rates to share weight parameters and thus avoid the need for iterative training. This approach produces multiple sparsified networks in single pass, reducing parameters and achieving a more economical communication overhead.
Graph-based knowledge distillation aggregates by trans- mitting logits or graph embeddings instead of model parameters. However, neither logits nor graph embeddings effectively capture structural information, and both introduce additional computational overhead. As regards the graph structure completion, certain methods endeavor to produce missing neighbors and their corresponding features for peripheral nodes, aiming to enhance the trainability of these damaged low-quality nodes. However, these generative methods encounter even greater computational overhead compared to graph-based knowledge distilla- tion, particularly in large-scale graphs. Although both methods aim to enhance model performance or mitigate heterogencity, they all come with substantially higher computational costs, leading to a strong dependence on more hardware resources. This leads us consider: II) How can effectively mitigate structural heterogeneity while minimizing additional computational demands?
To tackle II,, we utilize the node contextual knowledge learned by the graph synergy expert, applying it to a novel transmission distance metric for the purpose of assess struc- tural similarity among clients' data. This approach encour- ages structurally similar clients share more knowledge. While parameter sparsification is performed, we consider the varying importance of parameters at corresponding positions across clients. To achieve this, we introduce a consensus- based parameter sparsification strategy. This strategy in- cludes layer-wise alignment of the personalized parameter sparsification masks. With aggregation adjustments and parameter consensus sparsification, we more accurately ac- count for unbiased knowledge sharing, effectively mitigat- ing the performance degradation caused by heterogeneity. Additionally, we introduce the Harmony Sparsification Principle as the guiding principle for the above process. Thereby we establish an efficient, economical unified and effective sparsification framework. Our principal contribu-
To simultaneously overcome the aforementioned ques- tions, we introduce FAGLES: Towards Effective, Efficient, And Economical Federated Graph Learning via UnifiEd Sparsification. To address I), we first introduce a unified
2

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
tions are summarized as follows:
2.3. Graph Sparsification & Model Pruning
Graph sparsification and model pruning are widely studied techniques for reducing computational complexity while maintaining model performance. Graph sparsification aims to retain the most important structural components of a graph by removing less critical edges nodes (Gong et al., 2021; Müller et al., 2022: Liu et al., 2023c; Batjargal et al., 2019 Wan & Schweitzer, 2021). On the other hand, model pruning focuses on removing redundant model parameters to create more lightweight architectures (Yi et al., 2024; Qiu et al., 2022; Wu et al., 2023a). For instance, DSpar selectively retains the connections of high-degree nodes to sparsify the graph structure (Liu al., 2023c; Cai et al., 2024b). PR-FL employs an approach where model pruning is performed first, followed by the gradual recovery of model size during the training process (Ma, 2024; Cai et al., 2024a). However, as mentioned earlier, both methods face different limitations. We aim to overcome these shortcomings and obtain performner lightweight model.
We identify that existing sparsification methods lack unified framework, considering only graph level or pa- rameter level sparsification. Moreover, they fail to ensure effectiveness while achieving efficiency and economy. We are the first to introduce unified sparsification in FGL. This approach performs both graph and parameter space, minimizing computational and communication demands while maintaining improving model performance.
We conducted extensive experiments on both small and large graphs, evaluating our method across various metrics to confirm its superiority. For instance, under the Ogbn-Proteins + DeeperGCN setting, it achieves state- of-the-art (SOTA) performance while reducing training FLOPS by 82% | and communication bytes by 80%! 4.
2. Related Work
2.1. Federated Graph Learning
Federated Graph Learning (FGL.) combines the dual ad- vantages of federated systems and Graph Neural Networks (GNNs) (Wu et al., 2020; Wan al., 2025b; Shi et al., 2024; Wan et al., 2025a), enabling the training of graph-structured data while preserving data privacy (He et al., 2021; Jin et al., 2022; Wang et al., 2020; Wu et al., 2021; Zhang et al., 2022; Wan et al., 2024a; 2025c). While numerous studies have concentrated on improving model performance, as datasets continue to grow, reducing computational resource con- sumption have gradually become pressing challenges (Liu et al., 2023b; Chen et al., 2023a; 2024). Existing methods primarily focused on sparsification within the parame- ter space of traditional FL (Zeng et al., 2022; Jiang et al., 2023; Fang et al., 2025). To the best of our knowledge, we are the first to propose a unified framework that simultane- ously sparsifies both the graph structure and parameter space. More importantly, our approach successfully mitigates data heterogeneity and maintain or enhances model performance.
3. Preliminary
Notations. Following the general paradigm of federated graph learning, there are K participants (indexed by k), each client C2 possesses its own private data, represented as G. Ex) Here, V/- {u:) denotes the set of nodes, containing [v]] N₂ nodes, while & = {Ema represents the set of edges. The adjacency matrix for the k-th client's graph G& is denoted as A = (A where Aij = 1 if there is an edge between nodes e; and Vj. and A.j = otherwise. Additionally, each client has a feature matrix Xx, where each row corresponds to the features of specific node in VR. We define the global model at the beginning of the (-th communication round as M', and the model of the k-th local client as M with parameters 0z-
3.1. Graph & Parameters Sparsification
For the graph sparsification task, we define the sparsified set of edges as Eg. and the sparsified graph representation of the k-th client as G = [Vk, Eg). The one-hop subgraph of node U; is denoted as Gi... We define the parameter matrix of the k-thh local model's 1-th layer as w! Its sparsified version is denoted as (1).s The sparsification rates for the graph and weights S and sw are defined as follows: Definition 1. (Graph Sparsification Ratio). For sparsified graph Go, we calculate its corresponding graph sparsifica- tion rate as follows:
2.2. Mixture of Experts
The Mixture of Experts (MoE) has emerged as effective approach in scaling deep learning models. By dynamically selecting and combining the outputs of specialized expert networks. MoE enables the model to perform more efficiently and accurately across different tasks or inputs. In graph learning, moE has also been widely applied, including large-scale graph learning (Wang et al., 2024a; Gupta et al., 2022; Jacobs et al., 1991), multi-task learning (Liu et al., 2023a; Chen et al., 2023b; Ma et al., 2018; Shazeer et al., 2017; Bi et al., 2025a;b;a), and graph classification (Hu et al., 2021; Yao et al., 2019). In our work, we leverage MoE to achieve multi-criteria sparsification at the graph level, alleviating data heterogeneity and enabling structurally similar clients to share more knowledge.
G =²= (V,"". S E
Definition 2. (Model Sparsification Ratio). For a sparsi- fied model M with L layers, we calculate its parameter sparsification rate as follows:
5)
W
M = (W()) w w()), sw
(2)
ΣE W
3

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
where represents the cardinality (i.e,, the number of elements) of set, and denotes denotes the sparsified portions. Our objective is to maintain or improve the performance of the local model while maximizing 8 and SW.
Heterogeneous-Awaae Graph Sparsification, which corre- spond to section 4.2, section 4.3, respectively. We use dynamic rollback pruning for layer-wise parameter spar- sification, performing row-wise pruning within each layer to obtain subnetworks at different sparsity levels. Based on task requirements, we select and retrain suitable sub- networks. Subsequently, we leverage sparsification and synergy experts to remove redundant edges and unify client structures. Ultimately, we upload both synergy expert and backbone parameters, using Optimal Transport to assign higher weights to clients with similar structures, allowing the local model to absorb more knowledge from similar data distributions. The framework of the method is shown in Figure 3.
3.2. Sparsification Criterion Design
Current sparsification research in FGL lacks consideration for data structure heterogeneity and lacks unified frame- work to simultaneously address both graph level and param- eter space. These constraints prevent deep sparsification, as illustrated in Figure 2. We argue the conditions that an ideal sparsification architecture should satisfy as follows:
Ideal Sparsification Architecture for FGL: Given sparsification task 9. an optimal FGL architecture to achieve this should fulfill the following specifica- tions: Effectiveness: The sparsification structure must effectively mitigate data heterogeneity. Completeness: The sparsification structure should comprehensively account for redundant components (i.e., graph and parameter space). Scalability: The sparsification ar- chitecture should be capable of efficient operation on large-scale datasets.
4.2. Consensus-Informed Parameter Sparsification Motivation. GNNs in both centralized and federated dis- tributed environments face significant computational de- mands and extended training times, particularly on large datasets (Hamilton et al., 2017; Chen et al., 2017). However, traditional model sparsification approaches rely on multiple iterative cycles with preset pruning rates, directly conflict- ing with the aim of reducing computational load. Moreover, existing pruning methods in FL fail to effectively consider unbiased knowledge sharing, leading to notable efficiency losses.
To fulfill the specifications above, we introduce the Harmony Sparsification Principle to guide the criteria for designing sparsification architectures in FGL::
Principle 1. (Harmony Sparsification Principle). Given graph G {V.E} and L-layer model M (W()), W(2) W(L),, ,, along with two ideal sparsifiers Fg E 10.1)117 and F, E {0, 13IMI, responsible for graph and parameter sparsiftcation, respectively. Ge and M should satisfy the following condition:
Dynamic Mask Thresholds. Traditional lottery ticket net- works require manually setting sparsity levels, selecting a fixed proportion of parameters with the smallest absolute values for pruning. We define the threshold vector for the I- th layer kig, corresponding to the parameter matrix W in this layer. The parameter mask mi for the entry located at the i-th row and j-th column of W is computed as follows:
Σ NG + E A(M.M) WEE(N
arg min E Ju
d'E(A)
( 0, otherwise,
1. if W 00
())
m (1)
(4)
arg max E. Jq
Σ 2'(9.0)+ 2'c(0)
Σ 2'E(Z)
z (M,
where w!() denotes the weight in the i-th row and j-th ij column, and (i()) denotes the i-th element of the vector x. Dynamic threshold optimization. In the forward pass, we define simple binary step function BS(x) generate binary mask based whether the input exceeds a threshold. However, this discrete value is non-differentiable during the backward pass. To address this and facilitate effective threshold tuning, we employ the straight-through estimator (STE) method for optimization, whose effectiveness has been demonstrated in prior works (Bengio et al., 2013; Jang et al., 2016; Huh et al., 2023; Wu et al., 2023b; Yin et al., 2019). Specifically, in the backward pass, we approximate the gradient of BS as the gradient of an identity function:
(3)
where E denotes the mathematical expectation, {N}
represents metrics for evaluating differences in graph struc-
Jaccard similarity, spectral sparsification, etc.), ture Jaccard
(A) represents metrics for assessing model parameters in terms of inference performance (e.g., Top-k Accuracy, ROC-AUC, etc.), {[} evaluates metrics related to the graph message-passing load (e.g., node load, betweenness centrality, etc.), and {2} includes metrics for evaluating reductions in computational resources (e.g., training flops, communication bytes, etc.).
4. Methodology
4.1. Overview
d BS(x) dx
0. 0 < |z|| < 0.5.
(5)
The proposed EAGLES can be decomposed into two com- ponents: Consensus- Bonsed Parameter Sparsification and
1, otherwise.
This approach allows gradients to bypass the non-
4

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
Consensus -ansensuu-Informeed Parameter Sparsification
Heterogeneity- Aware Graph Sparsification
Weight vector
Threshold
vector
STE Mapping
Hard Concrete
Graph Expert (GSyt)
Graph Sparsification
Export (GSE)
1.2
0.8
Uplate
Dapul be
Inpur X
Consensus
EE Fuston Exsation
10010
network
Eq
Consensus
Dieatt
C2
Preservation
Upload
Weight
e
1.2
Sparse Subgraph
Equation (20)
Input
nanpeetationn Hoterr
obtain
network
Po.8
0.8
0.0
Sparse Datavets
1.000
Sparie Weight
Stractura
Paraamterr Sparefficatoon Flow
Graph Sparsification Flow
CR network
Asgrccation Squatten
Adiastment
Propagation Updaat
Upload Flow
Figure 3: Architecture illustration of EAGLES. (Left) Consensus Informedd Parameter Sparsification depicted, where the sparsification threshold vector is optimized using the Straight-Thouugh Estimator (STE), (Right) Hotcrogencity-awar graph sparsification is shown, with the Graph Synergy Expert optimized in reverse via the Hard Concrete distribution. (Middle) The server adjusts aggregation weights according to structural differences. Best viewed in color. Zoom in for details.
differentiability of BS during the backward pass, enabling effective updates to the threshold parameter Nig The loss function for this part is simply defined as follows:
aggregation increases (Chen et al., 2017; 2018; Chiang et al., 2019; Rong et al., 2019; Velickovic et al., 2019). There is limited research on sparsification within feder- ated distributed settings. Clients in FGL often exhibit substantial structural differences, and poorly configured pruning methods can intensify this structural heterogeneity, Moreover, most existing sparsification methods rely on single criterion. This reliance tends to overly preserve specific structural information aligned with that criterion, overlooking other aspects and resulting in structure information overfitting
Para Σ (exp(=ng)) (
Consensus Parameter Mask Alignment. Client-specific sparsification masks can lead to progressive overfitting to local data, a challenge that becomes even more pronounced during model pruning. Building on this, we propose fiientconseensu-buseee parameter pruning scheme. After applying dynamic masking on each client, the resulting mask vector (in boolean form) is compressed into bit stor- age, with each 8-bit segment occupying a single byte. This approach significantly reduces communication overhead while enabling efficient selection of consensus parameters.
Graph Sparsification Expert. The Graph Sparsification Expert (GSE) is the central module for graph sparsification. Each GSE applies unique sparsification criterion, evalu- ating each edge within the one-hop subgraph of each node according to its specific scoring standard. Specifically, we concatenate the feature matrix X; of node U, the feature matrix X, of its neighboring node vy, and the weight leayl of the edge to form the input feature matrix for the GSE:
C [1, 0,0,,00,* 1,1,0] :
$
$
Cx 1,1,110,, 1,00,000,,000001
:
Final Mask Vector
X, X, E NxD. leayl E RIx1
Cx [1,1, m 1,0,0]
(7)
2OSE X, X,. leiyl INxDl E Nxx22+11)
(8)
This approach promotes pruning stability at macro level while preserving a degree of generalization in the local models. To further reinforce deep pruning, we employ a dynamic rollback strategy. Specifically, we designate the pirfreit- subnetwork within an accuracy range of +3% around each multiple of ten as the optimal subnetwork for that pruning point. Before progressing to the next prun- ing point, we roll back to the optimal subnetwork from the previous point, ensuring stable and effective pruning depth.
Here, leay| is treated as a vector to enable broadcasting, and X, serves as the input matrix for the GSE to compute the edge score oj corresponding
to eij. This process is repeated No, times, where No, denotes the number of neighbors of node Uú, ultimately yielding an ordered set of edge scores:
O2, {o1, o2....,ON...
(9)
The graph sparsity rate is a predefined hyperparameter, based on which we determine the final sparsified subgraph for this GSE:
O", = {oR, Ok41y ON., }.
4.3. Heterogeneity-Awarr Graph Sparsification Motivation. The information aggregation mechanism of GNNs imposes substantial computational demands, with time complexity growing explosively as the depth of
(10)
G (V, J. E = e e eN.,
Assuming that t different GSEs have each produced a spar- sified version for vi we obtain a set of sparsified graphs:
5

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
G. -( -e: go,Ca,
(11)
where Var denotes the variance, À serves a balancing coefficient regulate the trade-off between the variance of importance scores and load scores, and is a small constant added to avoid division by zero. The primary node pre- diction task uses the standard negative log-likelihood loss:
Graph Synergy Expert. To integrate the sparsification results from different experts, we introduce a Graph Synergy Expert (GSyH), driven by gating mechanism (Greff et al., 2016; Li et al., 2015; Srivastava et al., 2015). However, the activation or deactivation of multiple sparsification experts poses a discrete binary challenge, making it non-differentiable in the backward pass. We propose optimizing the GSyE employing hard concrete distribution to address this issue. The optimization of 9ü is essentially an optimization problem over E This module leverages the GSyE at each node to learn its contextual information, thereby determining the expert assignments most suitable for each node. On a micro level, the GSyE optimizes subgraph Synergy by integrating structural details. On global macro level, nodes with similar contextual characteristics across clients tend to preserve more similar and refined structures, promoting consistency from the node level. The specific process is follows:
DNLL = 1
log(p(m((x:) (m))..
(18)
The final loss function comprises three components and is formulated as:
Suo X1 CNLL + A2 LasyE + X3 Lpare:
(19)
where X controls the relative weight of each loss compo- nent. Losse and Cnow are defined Equation (17) and Equation (6), respectively. À3 is applied only during pre- training for subnetwork training, and is set λ - 0 during retraining.
Transport-based Similarity. During each communication
round, local clients upload their GNN parameters along with Waate from the GSyE. The matrix Wgan encodes structural information derived from each client's 's data, as detailed in Section 4.3. The OT distance between clients C, and Cj is
(12) where N is the number of nodes, and D represents the dimensionality of the node features. serves as the input for sampling from the hard concrete distribution (Louizos et al.,
z= = X W gate: X E RNXD,
then defined as:
dey = min ΣT mm C.n:
(20)
2017; Maddison et al., 2016; Schlichtkrull et al., 2020).
(eT(ab m.n.
where a and denote the distributions derived from the unfolded W gate matrices of clients C, and Cj. respectively, Here, T(a, b) is the set of feasible transport plans that satisfy supply and demand constraints. Tym represents the amount of mass transported from the m-th element of dis- tribution to the n-th element of distribution b, while C,nn denotes the unit transportation cost between these elements.
xx)) = a- a--0 [og(2) log(1 - 2) + 92
(13)
The term a denotes the Sigmoid function, a is a mixing co- efficient, and 7 is the temperature parameter controlling the smoothness of the distribution. 9I and f/2 are random noise variables sampled for the reparameterization trick, which ensures differentiability of the binary gating mechanism.
Structure-Similarte Personalized Aggregation. We lever- age the OT distance between clients, formulated as:
= HardStee( (=)) 1, if (z) > 0.5.
(14)
0, otherwise.
dij min(d)
morm ij
(21)
max d) min(d)+ +
is a threshold function used to determine whether an ex- pert is activated. Furthermore, we obtain the control matrix:
where d represents the set of distance values between all pairs of clients. The similarity and personalized aggregation weight between clients Cj and C, are defined follows:
وزj if By E Gi.
My
(15)
0, otherwise.
S
Based on this, we define the calculation methods for expert importance scores and expert load:
Sij = 1 dorm
WiJ
(22)
KK
S +
No,
I.mp..
The parameter update rule for client C. is formulated as:
Ieme
(16)
K
N..
W 0z.
(23)
The optimization objective of the GSyE module and the procedure for optimizing W gate are formulated as follows:
4.4. Discussion
Our method employs unified sparsification framework to significantly reduce computational demands and communication overhead in federated systems. The parameter sparsification module identifies subnetworks across all sparsity levels in a single pass, albeit with a pretraining step inherited from conventional lottery
Var(Iimp.) (Iimp )2
Var(ILoa.)
LOSEE x
+(1 (1-A).
(Ihme y2 )
+
dC
Oz' (x)
dz
OW gate
Oz
(1)(2)
dz
OW gate
(17)
6

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
Table 1:
Node classification performance comparison across multiple metrics with various methods from state-of -hte-off-hee-at federated systems and centralized environments. All reported results represent the mean over five runs. Green arrows 14 indicate advancements in the given metrie, while red arrows
14 denote regressions. The best and second-best results are highlighted in bold and underlined, respectively.
Please refer to section 5.2 for additional analysis. Additional experimental results on more datasets can be found in Appendix D.
Puhmed (GCN)
Ogho-Arsiv (GruphSAGE))
Oghn-Prooeins: - (DeeperGCN)
Proteins
Methods
Effe. Eiti. Econ
Top-1
Max Training Communication
Top-1
Max Training Communication ROC-ADC. (t) FLOPS (4) BYTES (L)
Training Communication
Accuracy (1) FLOPS (4)
BYTES (4)
Accuracy (1)
FLOPS (L)
BYTES (J)
FedAvg [ASTATI7]
X X
x
85.65
(x(2449F9)
1x(6.19F9)
63.99
(x(1,88R10)
1x(6.1489)
81.54
1x(1.8199)
1x 2689)
FedProx (srXiv18]
X X X
84.51
1.00ga.00s
1.00
64.05po.08
1.00k 10.0z
81.12,0.42
1.0K-g
1,00x
FedSage+ [NourIPS21]
V V x
64.2310.24
1.00x.o. 1000
00
APPLE [UCA122]
X X
X
83.23
82
1.00
1,0K
63.21
1.00
1.00k o0.
80.02 1.52
1.00--0 00.
1,00x
FedCP [KDD23]
V X
X
85.82
1,03x 03.
4o.12
3.22xy2:20x
1.22x 22
81.49 10.00
3.86x
.6x
1,28x y0.00 28e
FedTAD [UCA124]
/ X
86.15 60
4,22x 22
1.05x 10.05x
65.03
1.04
35.33x (34.38e
1,13x 18
82.02
31,21x
20.
1.15x 1Ge
FedSSP [NeuriPS24]
/ X
X
86.32 07
3,21x B1w
2.45x414444
64.34
27.
(2.89x220.50
2.33x
BB
81.98 ro.
25.78x 78.
2.89x 1 1.80
PGGP [AAA124]
X X
84.55 11:10
1.15x 15a
1.32x 10.
63.78
5.78x 75
1,44x 4di
80.56 Lo 98
11,23x 23v
7.65x yo .5e
FedGTA [VLDB 24]
V X x
86.45 80
1,00x n0w
1.00x 000
64.13
1.00x00000
1.00xeo 00w
81.78 24
1.00x-0 000
1.00% yo o0e
PruneFL. (
x V
80,45.6.11
0.67x
1.00x 10.000.00x
60.88
0. 0.55x
1.00k
78.89 _8.89_2..0
0.06x 34x
1.00s 00
FedTiny [ICDCS23]
x V=
g2.84
0.77xgo.28z 20k
0.80x 1o, 20x
0.51x j2.98
40x
0.69x Siz
79.92..1.0
0.55x 48xx
0.59x 41s
FedDIP [ICDM23)
x V
83.30
0.59x
0.56x jo6xoo.4e 44x
63.85
0.58x p..8x
42
81.21 jo.3s
0.56x .4.4
0.42% Lo. 58
DSpar [TM).223]
X X x
84.14
1.00
jo oox
63.44 1o.55
1.000
1o.00z
1.00x 00s
80.86
1.00g.0 0ox
1.00x jo one
ACE-GLT (CVPR24)
V / x
70.68
0.55x
go. 65
1.00k to. 0ox
64.04
0.36%
1o.04x
1.00x nox
81.58
0.59x o..
1.00x Lo. one
EAGLES
86.97
0.48% 02.
0.37x 177xjo..c..
65.37
1..35
0.32x 1o, 10.68,
0.48x o. .2.
82.10+0.50 0.18x Lo, 82. 0. 1.20x s0.
networks. This pretraining, however, enables substantial parameter sparsification, greatly reducing communication costs during federated updates. At the graph sparsification level, a multi-expert approach is introduced to effectively mitigate structure information overfitting, a common issue in single-standard sparsification methods.
bone, while for large-scale datasets, we select a four-layer GraphSAGE and DeeperGCN as the backbones (Defferrard et al., 2016; Li et al., 2018: 8; Zhang et al., 2018). Please see Appendix E for more details.
Evaluation Details. We utilize range of metrics to comprehensively evaluate the fulti-faceted performance of each method: ① Top-1 Accuracy (Top-I-AC) @ Receiver Operating Characteristic-Aree Under the Curve (ROC- AUC) (Bradley, 1997). @ Max Training FLOPS (TF). O Communication Bytes (CB). We compared our method with traditional FL methods: (1) FedAvg (McMahan et al.,
5. Experiments
In this section, we omprehensively evaluate our proposed EAGLES by addressing the following key questions.
Q1: Superiority. Does EAGLES maintain or surpass baseline performance?
2017); (2) FedProx (Li et al., 2020); (3) FedSage+ (Zhang
Q2: Efficiency and Economical. Is EAGLES capable of reducing computational resource requirements and com- munication bytes?
et al., 2021); six state-of-the-an FGL methods: (4) APPLE (Luo & Wu, 2022); (5) FedCP (Zhang et al., 2023); (6) FedTAD (Zhu et al., 2024); (7) FedSSP (Tan et al., 2024);
Q3: Resilience. What is the performance of EAGLES under varying sparsification rates and different numbers of clients?
(8) FGGP (Wan et al., 2024a); (9) FedTGA (Li et al., 2024);
two widely used graph sparsification methods: (10) DSpar (Liu et al., 2023c); (11) ACE-GLT (Wang et al., 2023);
Q4: Sensitivity. How does EAGLES perform with differ- ent hyper-parameter settings?
and three recent pruning methods for traditional FI.: (12)
FedDIP (Long et al., 2023); (13) PruneFL (Jiang et al., 2022); (14) FedTiny (Huang et al., 2023). Additional
The answer of Q1-Q3 are illustrated in 5.2-5.4, and the analyses of Q4 can be found in Appendix D.
evaluation details are provided in Appendix B.
5.1. Experiment Setup
5.2. Superiority
Datasets and Split. To comprehensively evaluate EA- GLES across different datasets and tasks, we selected Cora (Mammen, 2021). Pubmed (Shchur et al., 2018), and Photo (McAuley et al., 2015) from small to medium-scale scauu datasets, and Arbnn,rxiv, Tgbn-Proteinss and large-scale datasets (Hu et al., 2020). Detailed descriptions for these datasets can be found in Appendix A.
To address Q1, we compare EAGLES against state-of-the- art methods in federated and centralized settings across multiple metrics, as shown in Table 1. Most existing methods excel in specific metrics but fail to achieve overall superiority: 1 FedCP, FedTAD, and FedSSP improve model generalization but neglect the redundancy in graph data and model parameters, which hampers training quality and limits potential gains. @ Graph sparsification and model pruning methods, such as DSpar, reduce computational
Products from
Backbone and Parameter Configurations. For small to cediumm-ccale datasets, we use a two-layer GCN as the back-
7

---

Ogbn-Arxiv
Pubmed
Ogbn-Arxiv
Pubmed
66
09
06
64
96
60
70
06
85
+ibnnom
52
+AACE-GET
62
EAAGLS
Weight Sparsity (%)
30 40
10
Weight Sparsity (%)
20
30
40
50
60
70
80
50
10
Graph Sparsity (%)
20
30
40
50
0
10
Graph Sparsity (%)
20
30
40
50
Figure 4: Node classification performance comparison of accuracy on the Pubmed and Ogbn-arxiv datasets, illustrating results for random pruning, EAGLES, and the advanced stal-sparsiiicatiin approach ACE-GLT. Please see details in section 5.4.
costs but often compromise model performance due to simplistic criteria like single-factor sparsification, leading
100x
100%
0.75
0.75x
0.5x
0.50x
to structural overfitting. @ FedTinn adopts non-recoverable pruning without considering parameter importance across clients, resulting in significant performance degradation. In contrast, EAGLES achieves superior performance across all metrics, effectively minimizing Training FLOPS and Communication YTES while maintaining or even enhancing model performance.
0.5x
0.25x
10
Weight Sparsity
30
40
50 60
70
TF
Weight Sparsity (%)
20
30
40
60
80
(a) Ogbn-proteins
(b) Ogbn-Arxiv
Figure 5: Three-dimensional bar chart delincates the influence of weight sparsity on three key metries: AC (Top-1 Accuracy), CB (Communication BYTES), and TF (Training FLOPS). For an in-depth analysis, please refer to section 5.4.
of-the-art dual-sparsification method for centralized systems, and random sparsification node classification tasks for Pubmed and Ogbn-Arxiv. For weight sparsity analysis, graph sparsity is fixed at 30%; for graph sparsity analysis, weight sparsity is fixed at 50%. Observations: @ EAGLES consistently outperforms the baseline under most conditions, while ACE-GLT falls short. This discrepancy likely arises from ACE-GLT's limited consideration of data heterogeneity in federated systems and the varying importance of parameters across clients.
5.3. Efficiency
To address Q2, we examine Training FLOPS and Commu- nication BYTES, summarized in Table 1, with metric de- tails provided in Appendix B. Methods like FedSSP and Fed- TAD improve local or global generalization but at the cost of higher computational and communication overhead, partic- ularly in large-scale graphs. Sparsification approaches, such recovery, prevent-
as PruneFL and ACE-GLT, involve recovvey ing reductions in communication bytes. Moreover, PruneFL. lacks graph-level sparsification, and both methods overlook client-specific parameter importance, limiting their contribu- tions to FGL. FedTiny and FedDIP avoid weight recovery by pruning during training, reducing both Training FL.OPS and Communication YTTES, but at the expense of model per- formance. In contrast, EAGLES employs consensus-buue based parameter sparsification to identify subnetworks at each pruning rate, leveraging these subnetworks during retraining to simultaneously reduce dataset size and ensure both effi- eiency and effectiveness. Furthermore, the reduced param- eter size of the subnetworks significantly lowers communi- cation bytes, achieving a more economical training process.
Figure presents two three-dimensional bar charts il- lustrating three key metrics- (Top-1 Accuracy), CB (Communication BYTES), and TF (Training FLOPS) on the Ogbn-Arxiv and Proteins datasets. Baseline met- rics are normalized to 1.00x, with all results reported as percentages relative to this baseline. We observe that Obs. @ as weight sparsity increases, both TF and CB steadily de- crease, while AC remains stable, consistently near or above the baseline. Collectively, these observations provide strong evidence of the resilience of EAGLES.
6. Conclusion
5.4. Resilience
In this paper, we are pioneers in addressing the challenge of reducing computational resource demands in federated graph learning. We introduce EAGLES: Towards Effective, Efficient, And Economical Federated Graph Learning via UnifiEd Sparsification. Previous works rely on sparsifica- tion in single aspect, we are the first to propose unified sparsification, overcoming issues such as parameter space sparsification without federated characteristics and graph sparsification constrained by single criteria. Our method's 's superiority is demonstrated across multiple metries on sev- eral datasets. We hope this work offers novel perspective for future research focused on reducing computational re- source consumption in federated graph systems.
To address Q3, we conducted extensive experiments. Ta- ble 2 and Table 4 present the Top-1 Accuracy of EAGLES across various sparsification rates. Observations: 1 The op- timal graph sparsification rate is approximately 30%, while parameter sparsification shows greater flexibility. For exam- ple, under the Ogbn- Proteins DeeperGCN configuration, parameter sparsification rate of 81.79% (rounded to 80% in the table) results in 0.4% performance improvement over the baseline. We present ablation experiments on the number of clients in Figure 6a and Figure 6b.
Figure compares EAGLES with ACE-GLT, a state-
8

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
Acknowledgement
the ACM International Conference on Multimedia, pp. 5537-5546, 2024a.
This research is supported by National Key Research and Development Project of China (2024FFC3004400, the National Natural Science Foundation of China (Grants 62361 166629, 62176188, 623B2080), the Wuhan Univer- sity Undergraduate Innovation Research Fund Project. The supercomputing system at the Supercompuing Center of Wuhan University supported the numerical calculations in this paper. Carl Yang was not supported by any funds from China.
Cai, J., Zhang, Y., Fan, J., and Ng. S.-K. Lg-fgad: An effective federated graph anomaly detection framework. In Proceedings of the International Joint Conference Artificial Intelligence, pp. 3760-3767 2024b.
Cardoso, J. V. d. M., Ying, J., and Palomar, D. P. Algorithms for learning graphs in financial markets. arXiv preprint 1.Xiv:2012.15411 2020.
Chen, F., Li, P., Miyazaki, T., and Wu, C. Fedgraph: Fed- erated graph learning with intelligent sampling. IEEE Transactions on Parallel and Distributed Systems, 33(8): 1775-1786, 2021a.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Chen, F., Long, G., Wu, Z., Zhou, T., and Jiang, Per- sonalized federated learning with graph. arXiv preprint arXi::2203.00829 2022.
References
Chen, J., Zhu, J., and Song, L. Stochastic training of graph convolutional networks with variance reduction. arXiv preprint arXiv: 1e1111100.00000 10568, 2017.
Back, J., Jeong, W., Jin, J., Yoon, J., and Hwang, S. J. Per- sonalized subgraph federated learning. International conference on machine learning, pp. 1396-1415. PMLR, 2023.
Chen, J., Ma, T., and Xiao, C. Fastgen: fast learning with graph convolutional networks via importance sampling. arXiv preprint /801. 2018.
Barnes, J. A. Graph theory and social networks: A technical comment on connectedness and connectivity. Sociology, 3(2):215-23) 232, 1969.
Chen, J., Gao, K., Li, G., and He, K. Nagphormer: A tokenized graph transformer for node classification in large graphs. In Proceedings of the Eleventh International Conference on Learning Representations, 2023a.
Batjargal, D., Khan, K. U., and Lee, Y.-K. Em-fgs: Graph sparsification via faster semi-metri edges pruning, Ap- plied Intelligence, 49:3731-3748, 2019.
Bengio, Y., Léonard, N., and Courville, A. Estimating propagating gradients through stochastic neurons for con- ditional computation. arXiv preprint arXiv:1308.344 2013.
Chen, J., Liu, H., Hopcroft, J. E., and He. K. Leveraging contrastive learning for enhanced node representations in tokenized graph transformers. I Proceedings of the 38th Annual Conference on Neural Information Processing Systems, 2024.
Bi, J., Wang. Y., Chen, H., Xiao, X., Hecker, A., Tresp, V., and Ma, Y. Llava steering: Visual instruction tun- ing with 500x fewer parameters through modality linear representation 2025a. URL https::///// //rxps: org/ arg/abs/2412.1 12359.
Chen, M., Shlezinger, N., Poor, H. V., Eldar, Y. C., and Cui, Communication-efficieee federated learning. Pro- ceedings of the National Academy of Sciences, 118(17): e20247891 18, 2021b.
Bi, J., Wang, Y., Yan, D., Xiao, X., Hecker, A., Tresp, V., and Ma, Y. Prism: Self-pruning intrinsic selection method for training-free multimodal data selection, 2025b. URL https:/ /tte://arxxiee /arxivv. org/ abs////// 2502. 12119.
Chen, Z., Shen, Y., Ding, M., Chen, Z., Zhao, H., Learned- Miller, E. G., and Gan, C. Mod-squad: Designing mix- tures of experts modular multi-task learners. In Pro- ceedings of the IEEF/CVF Conference on Computer Vi- sion and Pattern Recognition, pp. 11828-11837, 2023b.
Bradley, A.
The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern recognition, 30(7): 1445 145-115111 1159, 1997.
Chiang, W.-.., Liu, X., Si, S., Li, Y., Bengio, S., and Hsieh, C.-J. Cluster-gen: An efficient algorithm for training deep and large graph convolutional networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 257-266. 2019.
Cai, J., Zhan, Y., Lu, Z., Guo, W. and Ng, S.-K. To- wards effective federated graph anomaly detection via self-bon toff-oooste knowledge distillation. In Proceedings of

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
Defferrard, M., Bresson, X., and Vandergheynst, P. Con- volutional neural networks graphs with fast localized spectral filtering Advances in neural information pro- cessing systems, 29, 2016.
Hu, W., Fey, M., Zitnik, M., Dong, Y., Ren, H., Liu, B., Catasta, M., and Leskovec, J. Open graph benchmark: Datasets for machine learning graphs. Advances in neural information processing systems, 33:22118 18-22133, 2020.
Fang, X., Easwaran, A., Genest, B., and Suganthan, P. N. Adaptive hierarchical graph cut for multi-gre gualliigaaularity out- of-distribution detection. IEEE Transactions on Artificial Intelligence, 2025.
Huang, H., Zhang, L.,, Sun, C., Fang, R., Yuan, X., and Wu, D. Distributed pruning towards tiny neural networks in federated learning. In 2023 IEEE 43rd International Conference on Distributed Computing Systems (ICDCS), pp. 190-201, IEEE, 2023.
Fung. W. S., Hariharan, R., Harvey, N. J., and Panigrahi, D. A general framework for graph sparsification. In Proceedings of the forty-third annual ACM symposium on Theory of computing, pp. 71-80. 2011.
Huh, M., Cheung, B., Agrawal, P., and Isola, P. Straight- ening out the straigh-tthrough estimator: Overcoming optimization challenges in vector quantized networks. International Conference on Machine Learning, pp.
Gao, C., Zheng, Y., Li, N., Li, Y., Qin, Y., Piao, J., Quan, Y., Chang. J., Jin, D., He, X., al. survey of graph neural networks for recommender systems: Challenges, meth- ods, and directions. ACM Transactions Recommender Systems, 1(1):1-51, 2023.
14096-14113. PMLR, 2023.
Jacobs, R. A., Jordan, M. I., Nowlan, J., and Hinton, G. B. Adaptive mixtures of local experts. Neural computation, 3(1):79-87, 1991.
Gao, H. and Ji, S. Graph u-nets. In international conference on machine learning, pp. 2083-2092. PMLR, 2019.
Jang, E., Gu, S., and Poole, B. Categorical repa- rameterization with gumbel-softmax, arXiv preprint arXiv: arXiv:6611.00 111111111440 144, 2016.
Goldenberg, D. Social network analysis: From graph theory to applications with python. arXiv preprint arXiv:2102.10014, 2021.
Jiang, Y., Wang. S., Valls, V., Ko. J., Lee, W.-H., Leung, K. K., and Tassiulas, L. Model pruning enables efficient federated learning on edge devices. IEEE Transactions on Neural Networks and Learning Systems, 34(12); 10386, 2022.
Gong, Y. Huang, L., and Chen, L. Eliminate devia- tion with deviation for data augmentation and a gen- eral multi-modal data learning method. arXiv preprint (rXiv220010885:: 2021.
Jiang, Z., Xu, Y., Xu, H., Wang, Z., Liu, J., Chen, Q., and Qiao, C. Computation and communication efficient federated learning with adaptive model pruning. IEEE Transactions on Mobile Computing, 23(3):2003-2021, 2023.
Greff, K., Srivastava, R. K., Koutník, J., Steunebrink, B. R., and Schmidhuber, J. Lstm: search space odyssey. IEEE transactions on neural networks and learning systems, 28
(10):2222 2222,, 2016.
Jin, J., Ren, J., Zhou, Y., Lyu, L., Liu, J., and Dou, D. Accelerated federated learning with decoupled adaptive optimization. In International Conference on Machine Learning, pp. (0298-10322) PMLR, 2022.
Gupta, S., Mukherjee, S., Subudhi, K., Gonzalez, E., Jose, D., Awadallah, A. H., and Gao, Sparsely activated mixture-of cocuure-of-xxpeee efpeeeesssseeeeeeeeeeeeeeee--ese are robust multi-task learners. arXiv 2022.
preprint ar arXiv:2204.07660
Lee, J., Lee, I,, and Kang. J. Self-attention graph pooling. In International conference machine learning, pp. 3734- 3743. pmlr, 2019.
Hamilton, W., Ying, 7., and Leskovec, J. Inductive repre- sentation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Li, J., Zhang, T., Tian, H., Jin, S., Fardad, M., and Za- farani, R. Graph sparsification with graph convolutional networks. International Journal of Data Science and Analytics, pp. 1-14, 2022a.
He, C., Balasubramanian. K., Ceyani, E., Yang, C., Xie. H., Sun, L.., He, L., Yang, L., Yu, P. S., Rong, Y., et al. Fed- graphnn: A federated learning system and benchmark for graph neural networks. arXiv preprint 2rXiv:2104.07145 2021.
Li, M. M., Huang, K., and Zitnik, M. Graph representation learning in biomedicine and healthcare. Nature Biomedi- cal Engineering 63133:355-1136 2022b.
Hu, F., Wang. L., Wu, S., Wang, L., and Tan, T. Graph clas- sification by mixture of diverse experts. arXiv preprint arXiv:2103 2033. 55622, 2021.
Li, Q. Han, Z., and Wu, X.-M. Deeper insights into graph convolutional networks for semi-supervised learning. In
10

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
Proceedings of the AAAI conference on artificial intelli- gence, volume 32, 2018.
multi-gate mixture-of-expers.. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 1930-1939, 2018.
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., and Smith, V. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2:429-450. 2020.
Ma, Y., Dai, W., Huang, W., and Chen, J. Geometric knowledge-guided localized global distribution alignment for federated learning. In CVPR, 2025.
Li, X., Wu, Z., Zhang, W., Zhu, Y., Li, R.-H., and Wang, G. Fedgta: Topology-aware averaging for federated graph learning. arXiv preprint vrXiv:2401.11755 2024.
Maddison, C. J., Mnih, A., and Teh, W. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint arXiv: (Tt:::6111,0007 2016.
Li, Y. Tarlow, D., Brockschmidt, M., and Zemel, R. Gated graph sequence neural networks. arXiv preprinl arXiv:1511.05440 2015.
Mammen, P. M. Federated learning: Opportunities and challenges. arXiv preprint arXiv:2 2021.
McAuley, J., Targett, C., Shi, Q., and Van Den Hengel, A.
Liang, W., Zhao, Y., She, R., Li, Y., and Tay, W. Fedsheafhn: Personalized federated learning on graph- structured data. arXiv preprint ar Xii:24405.16066 2024.
Image-based recommendations on styles and substitutes. Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pp. 43-52, 2015.
Liu, L., Song, X., Sun, B., Gong, G., and Li, W. Mmoe- gat: A multi-gate mixture-of-expers boosted graph at- tention network for aircraft engine remaining useful life prediction. In Asia Simulation Conference, pp. 451-465. Springer, 2023a.
McMahan, B., Moore, E., Ramage, D., Hampson, S., and learning of deep networks from decentralized data. In Artificial intelli- gence and statistics, pp. 1273-1282. PMLR, 2017.
y Arcas, B. A. Communication--eficcee
Liu, W., Chen, C., Liao, X., Hu, M., Yin, J., Tan, Y., and Zheng, L. Federated probabilistic preference distribution modelling with compactness co-clustering for privacy- preserving multi-domain recommendation. In IJCAL, pp. 2206-2214, 2023b.
Müller, J. U., Weinmann, M., and Klein, R. Unbiased gradient estimation for differentiable surface splatting via poisson sampling. In European Conference on Computer Vision, pp. 281 281-299. Springer, 2022.
Nguyen, B., Sani, L., Qiu, X., Liò, P., and Lane, N. D. Sheaf hypemeeworrs for personalized federated learning. arXiv preprint 2rXiv:2405.20882. 2024.
Liu, Z., Zhou, K., Jiang, Z., Li, L.., Chen, R., Choi, S.-H., and Hu, X. Dspar: An embarrassingly simple strategy for efficient gnn training and inference via degree- based spar- sification. Transactions Machine Learning Research, 2023c.
Parchas, P., Papailiou, N., Papadias, D., and Bonchi, F. Uncertain graph sparsification. IEEE Transactions on Knowledge and Data Engineering, 30(12):2435-24M) 2018.
Long, Q. Anagnostopoulos, C., Parambath, S. P., and Bi, D. Feddip: Federated learning with extreme dynamic pruning and incremental regularization. In 2023 IEEE International Conference on Data Mining (ICDM). pp- 1187-1192. IEEE, 2023.
Qiu, X., Fernandez-Marques. J., Gusmao, P. P., Gao, Y., Parcollet, T., and Lane, N. D. Zerofl: Efficient on-device training for federated learning with local sparsity. arXiv preprint arXiv:2208. 025007 2022.
Louizos, C., Welling, M., and Kingma, D. P. Learning sparse neural networks through /.0 regularization. arXiv preprint arXiv: 1712:1712..03122 2017.
Rong. Y., Huang, W., Xu, T., and Huang. Dropedge: Towards deep graph convolutional networks on node clas- sification. arXiv preprint arXiv:1907.10003 2019.
Luo, J. and Wu, S. Adapt to adaptation: Learning per- sonalization for cross-silo federated learning. In IJCAI: proceedings of the conference, volume 2022, pp. 2166. NIH Public Access, 2022.
Saha, S., Gao, and Gerlach, R. A survey of the application of graph- based upproaches in stock market analysis and prediction. International Journal of Data Science and Analytics, 14(1):1-15, 2022.
Ma, C. Federated learning based on pruning and recovery. arXiv preprini arXiv:2403. 200e22443335444 15439, 2024.
Schlichtkrull, M. S., De Cao, N., and Titov, I. Interpreting graph neural networks for nlp with differentiable edge masking. arXiv preprint 20Xiv:2010.00577 2020.
Ma, J., Zhao, Z., Yi, X., Chen, J., Hong, L., and Chi, E. H. Modeling task relationships in multi-task learning with

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
Schrodt, J., Dudchenko, A., Knaup-Gregori, P., and Ganzinger, M. Craph-rrepresentation of patient data: systematic literature review. Journal of medical systems, 44(4):86, 2020.
Wan, G., Shi, Z., Huang, W., Zhang. G., Tao, D., and Ye, M. Energy-based backdoor defense against federated graph learning. In International Conference Learning Representations, 2025c.
Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q. Hinton, G., and Dean, J. Outrageously large neural networks: The sparsely-gated mixture-of-expprrs layer. arXiv preprint /tt1::700.0653. 2017.
Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., and Khazaeni, Y. Federated learning with matched averaging. 2020.
arXiv preprint arXiv:2002.0644
Wang. H., Jiang, Z.,, You, Y., Han, Y., Liu, G., Srinivasa, J., Kompella, R., Wang. Z., et al. Graph mixture of experts: Learning on large-scale graphs with explicit diversity modeling. Advances in Neural Information Processing Systems, 36, 2024a.
Shchur, O., Mumme, M., Bojchevski, A., and Günnemann, S. Pitfalls of graph neural network evaluation. Relational Representation Learning Workshop, NeurIPS 2018, 2018.
Shi, J., Zheng, S., Yin, X., Lu, Y., Xie, Y., and Qu, Y. Clip-guided federated learning on heterogeneity and long- tailed data. In Proceedings of the AAAI Conference on Ar- tificial Intelligence, volume 38, pp. 14955-14963, 2024.
Wang, S., Hu, L.., Wang, Y., He, X., Sheng, Q. Z., Orgun, M. A., Cao, L., Ricci, F,, and Yu, P. S. Graph learning based recommender systems: A review. arXiv preprint arXiv:2/05.063 2021.
Sohrabi, F., Attiah, K. M., and Yu, W. Deep learning for distributed channel feedback and multiuser precoding in fdd massive mimo IEEE Transactions on Wirelexs Communicationss 2007):044-4057, 2021.
Wang, X., Chen, K., Wang. K., Wang, Z., Zheng, K., and Zhang. J. Fedkg: A knowledge distillation- based feder- ated graph method for social bot detection. Sensors, 24 (11):3481, 2024b.
Srivastava, R. K., Greff, K., and Schmidhuber, J. Highway networks. arXiv preprint arXiv: 1505 2015.
Wang, Y., Liu, S., Chen, K., Zhu, T., Qiao, J., Shi, M., Wan, Y., and Song, M. Adversarial erasing with pruned elements: Towards better graph lottery tickets. In ECAI 2023, pp. 2583-2590. IOS Press, 2023.
Tan, Z., Wan, G., Huang, W., and Ye, M. Fedssp: Federated graph learning with spectral knowledge and personalized preference. arXiv preprint arXiv:2410.20111 2024.
Velickovic, P., Fedus, W., Hamilton, W. L., Liò, P., Bengio, Y., and Hjelm, R. D. Deep graph infomax. ICLR, 2(3):4, 2019.
Wu, C., Wu, F., Cao, Y., Huang, Y., and Xie, X. Fedgnn: Federated graph neural network for privacy-preserving recommendation. arXiv preprint arXiv:2102.04923, 2021.
Wan, G. and Schweitzer, H. Edge sparsification for graphs via meta-leaming. In 2021 IEEE 37th International Con- ference on Data Engineering (ICDE), pp. 2733-2738. IEEE, 2021.
Wu, T., Song, C., and Zeng, P. Efficient federated learning on snource-constrainet edge devices based on model pruning. Complex & Intelligent Systems, 96)16999-7013, 2023a.
Wan, G., Huang, W., and Ye, M. Federated graph learn- ing under domain shift with generalizable prototypes. In Proceedings of the AAAI Conference Artificial Intelli- gence, volume 38, pp. 15429- 15437, 2024a.
Wu, X.-M., Zheng, D., Liu, Z., and Zheng, W.-S, Estima- tor meets equilibrium perspective: rectified straight through estimator for binary neural networks training. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 17055-17064, 2023b.
Wan, G., Tian, Y., Huang, W., Chawla, N. V., and Ye, M. S3gel: Spectral, swift, spatial graph contrastive learn- ing. In forty-first International Conference on Machine Learning, 2024b.
Wu, Z.., Pan, S., Chen, F., Long, G., Zhang, C., and Philip, S. Y. comprehensive survey on graph neural networks. IEEE Iransactions on neural networks and learning sys- tems, 32(1):4-24, 2020.
Wan, G., Huang, Z., Zhao, W. Luo, X., Sun, Y., and Wang, W. Rethink graphode generalization within coupled dy- namical system. In Forty-second International Confer- ence on Machine Learning. 2025a.
Yang. Q., Liu, Y., Chen, T., and Tong, Y. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1-19, 2019.
Wan, G., Liu, Z., Shan, X., Lau, M. S., Prakash, B. A., and Jin, W. spidemiology-awarc neural ode with continuous disease transmission graph. In Forty-second International Conference on Machine Learning, 2025b.
Yang, Z., Cohen, W., and Salakhudinov, R. Revisiting supe-supe learning with graph embeddings. In
12

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
International conference on machine learning, pp. 40-48. PMIR, 2016.
Yao, L., Mao, C., and Luo, Y. Graph convolutional networks for classification. In Proceedings of the AAAI confer- ence on artificial intelligence, volume 33, pp. 7370-7377, 2019.
Yi, L., Shi, X., Wang, N., Zhang, J., Wang, G., and Liu, X. for federated
Fedpe: Adaptive model learning on mobile devices. IEEE Transactions on Mobile Computing, 2024.
Yin, P. Lyu, J., Zhang, S., Osher, S., Qi, Y., and Xin, Understanding braight-through estimator in train- ing activation quantized neural nets. arXiv preprint arXiv: 056622, 2019.
Zeng, H., Zhou, T., Wu, X., and Cai, Z. Never too late: Trac- ing and mitigating backdoor attacks in federated learning. In Proc. of the International Symposium on Reliable Dis- tributed Systems (SRDS), pp. 69-81, IEEE, 2022.
Zhang. J., Hua, Y., Wang, H., Song, T., Xue, Z., Ma, R., and Guan, H. Fedep: Separating feature information for personalized federated learning via conditional policy. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3249-3261, 2023.
Zhang, K., Yang, C., Li, X., Sun, L., and Yiu, S. M. Sub- graph federated learning with missing neighbor genera- tion. Advances in Neural Information Processing Systems, 34:6671-6682 2021.
Zhang, K., Wang. Y., Wang. H., Huang, L.., Yang, C., Chen, X. and Sun, L. Efficient federated learning on know]- edge graphs via privavy-preserving relation embedding aggregation. arXiv preprint 100ii::20030095555 2022.
Zhang, M., Cui, Z.,, Neumann, M., and Chen, Y. An end-to- end deep learning architecture for graph classification. In Proceedings of the AAAI conference on artificial intelli- gence, volume 32, 2018.
Zhu, Y., Li, X., Wu, Z., Wu, D., Hu, M., and Li, R.-H. Fedtad: Topology-awan data-free knowledge distilla- tion for subgraph federated learning. arXiv preprint arXiv:2404. 14061, 2024.
Zhu,2 Z., Hong, J., and Zhou, J. Data-free knowledge distil- lation for heterogeneous federated learning. Interna- tional conference on machine learning, pp. 12878-12889.
PMLR. 2021.
13

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
A. Datasets Details
All datasets are used for node classification tasks. For the small to medium group Cora, pubmed and Amz-Photo, we split training. validation and test set as 60%. 20%, 20%. we manually split the data into 60% for training. 20% for validation, and 20% for testing, while for the latter group, we utilized the official dataset splits. The official (training, validation, test) splits for OGBN-Arivv, OGBN-Proteins. and OGBN-Product are (53.7%, 17.6%, 28.7%), (65.3%, 13.9%, 20.8%), and (8.03%, 4.01 %, 87. ),), respectively. The statistics of the datasets used in our experiments are provided in Table 5.
Citation Network (Cora and pubmed): The Citation Network datasets Cora and pubmed are standard benchmarks for graph neural networks in machine learning. The Cora dataset contains 2.708 scientific publications divided into seven categories. Each publication is linked by citations and represented by a binary word vector from vocabulary of 1,433 unique words. The pubmed dataset focuses on diabetes and includes 19,717 publications across three categories: Diabetes Mellitus, Experimental Diabetes Mellitus Type 1, and Diabetes Mellitus Type 2. It has 44,338 citation links, with each publication represented by a TF-IDF weighted word vector from a dictionary of 500 unique terms (Yang et al., 2016).
Amz-purchase (Photo): The Amz-purchase Photo dataset, derived from Amazon's co-purchase network, is widely utilized for classification tasks in recommendation systems and market analysis. In this dataset, nodes represent products, while edges indicate co-purchase relationships, which capture products often bought together. Each product is associated with specific features, and the main task is to predict its category. This dataset supports research on consumer behavior, product categorization, and recommendation algorithms (McAuley et al., 2015).
OGBN-ARXIV The OGBN-ARXIV dataset from the Open Graph Benchmark (OGB) supports node property prediction tasks within an academic citation network. This dataset comprises 169,343 nodes, each representing computer science paper from the arxiv repository with directed edges indicating citation relationships. Each node includes a I 288dimensiionl feature vector derived from embeddings of the paper's s title and abstract. The main task is to predict the paper's s subject area across 40 classes. The dataset is partitioned by publication year: papers published up to 2017 are used for training, those from 2018 for validation, and those from 2019 onward for testing (Hu et al., 2020).
OGBN-PROTEINS The OGBN PROTEINS dataset from the Open Graph Benchmark (OGB) supports node property prediction tasks within a protein-protein association network. This undirected weighted graph contains 132,534 nodes, each representing a protein from one of eight species. Edges indicate biologically meaningful associations such as physical interactions, co-expression, or homology. Each edge has an 8-dimensional feature vector where each dimension represents the confidence score (from 1) of a specific association type. The primary task to predict 1 112 different protein functions in multi-label binary classification setup. The dataset is split by species, allowing models to be evaluated on their generalization across species (Hu et al., 2020).
OGBN-PRODUCTS The OGBNUPRODUCTS dataset from the Open Graph Benchmark (OGB) supports node prop- erty prediction tasks within an Amazon product co-purchasing network. This undirected unweighted graph includes approximately 2.4 million nodes, each representing a product, and 61.9 million edges indicating frequent co-purchase relationships. Each node has 100-dimensiona. feature vector derived from the product description. The primary task is to predict the product category among 47 top-level classes in multi-class classification setup. The dataset is split by sales rank, with the top 8% of products for training, the next 2% for validation, and the remaining 90% for testing, simulating a realistic scenario where popular products have labeled data while predictions are required for less popular items (Hu et al., 2020).
B. More Evaluation Details
The metrics used in the experiments in this paper calculated follows:
Top-1 Accuracy Top-1 Accuracy is a common evaluation metric used in classification tasks, particularly for multi-class problems. It represents the proportion of predictions where the model's highest confidence label (the top-1 prediction) matches the true label. This metric is calculated as the ratio of correctly classified samples to the total number of samples, providing a straightforward measure of the model's ability to correctly identify the primary class for each input.
T
1 (truee [j. 1] 3preed [J, {
Accuracy = T
(24)
N
im1
14

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
Table 2: Node classification performance of EAGLES evaluated across numerous values of 59 and SW on pubmed, Ogbn-Arxiv and SeoneProoeinns The reported performance represents the average over five runs and is measured within a target sparsity range of +3%. Please refer to appendix D for additional analysis.
Purameter
Graph Spursity:h pubmed
Graph Sparsity:: St OgbnnArxiv
Graph Sparsity:: 9 Ogbn-Proteinns
Sparsity
10
30
50
10
30
50
10
30
50
.7.77
15.50
f0.06
84.98 84.98.o.07
10.07
jo.55
63.32.j.o7
64.01
Lo.02
62.89
81.89
82.13
81.92fo.as
81.88
10
84,65 11.00
85.51
po.14
1o
84.88 g0.77
63.23
1o.76
63.48.0.81
63.48
po.51
63.97
10.02
(2.47
f0.84
82.17
81.90
81.9770.43
20
84,66.0.00 09
85.79
1.08
84.27
63.11
1o.88
63.88
1o.11
64.11 10.12
62.66
82.97
82.62
82.22
10.68
81.70+0.16
30
85.6640.01
y..9
86.76f1.11
84.24.1.41
64.42-0.44
64.12-0.13
64.05yo.0s
63.92 10.07
82.82
22
83.76
82.69
82.61p1.07
40
85. 85.12,j..s 55
g4.55
86.1440.40
84.38
j1.27
64.95
64.59
64.35
63.66
82.62
42.29
1..51
82.34,1.00
50
85.81 85.88
p..97
85.71
65.09
64.64
63.21 jo.78
81.47.o.07
82.18pa.64
83.2441.70
81.44.0.11
84.89 70.76 85,44_...2
85.86
82.78.2..87
g1.6
63.02..o.o7
60.85
64.08
82.63
82.25
42.14
82.52,0.95
70
83.88
1:77
85.78
p0.13
85.67
po.02
83.19.pa ja.46
p4.19
64.20
65.37
1..38
63.34
1o.65
po.58p
81.29
po.25
81.55
81.27.0.22
80
82.28 ja J3.37
83.89
u1.70 mi 83,98 11.07
82.17.3.44
j1.28 60.41 مي 62.96
11.9661.000
60.13
60.13
ja.s6
81.94
80.50
82.10
80.87 60.67
Vanilla
85.65
63.99
81.54
Table 3: Statistics of datasets used in experiments.
Dataset
#Nodes
#Edges
#Classes
#Average Degree
#Metric
Cora
2,708
5,278
7
2.25
Accuracy
pubmed
19,717
44,324
3
8.00
Accuracy
Amz-Photo
7,650
287,326
8
31.12
Accuracy
Ogbn-Arixy
169,343
1,166,243
40
13.77
Accuracy
Ogbn-Proteins
132,534
39,561,252
2
597.00
ROC-AUC
Ogbn-PProuuee 2,449,029
61,859,140
47
50.52
Accuracy
ROC-AUC The ROC-AUC is widely adopted metric for evaluating binary classifiers and can be extended to multi-label settings. The ROC curve plots the true positive rate (TPR) against the false positive rate (FPR) across various thresholds, illustrating the model's performance over a range of decision boundaries. The AUC (Area Under the Curve) quantifies the likelihood that a randomly chosen positive instance ranks above a randomly chosen negative instance. An AUC of 1.0 indicates perfect discrimination, while an AUC of 0.5 suggests performance no better than random guessing.
1 T Σ
ROC-AUC
TPR, (FPR, (t)) dt,
T 1=1
(25)
TP
FP
TPR TP + FN'
FPR
FP + TTN
where TP (True Positives) denotes correctly classified positive samples, FN (False Negatives) represents positive samples
misclassified as negative, FP (False Positives) indicates negative samples misclassified as positive, and TN (True Negatives) is the count of correctly classified negative samples. These values are essential for calculating the True Positive Rate (TPR) and False Positive Rate (FPR), which are used to plot the ROC curve and compute the Area Under the Curve (ROC-AUC).
MAX Training FLOPS The calculation of MAX Training FLOPS primarily addresses two main components: message- passing operations and feature transformation. The message-passing aspect accounts for the computational load of propagating information across nodes and edges, while the feature transformation component captures operations within masked linear layers and activation functions such as ReLU and BatchNorm (with Mixture of Experts (MoF) modules in the case of EAGLES). This approach provides a comprehensive assessment of the FLOPs required for both propagation and transformation processes within the model.
Communication BYTES Communication cost is calculated based on the total bytes transmitted by clients to the server (upload bytes) and the bytes required for the server to broadcast the aggregated global model back to each client (download bytes). The model size serves as the basis for calculating both upload and download bytes, with each client transmitting its model to the server and receiving the updated global model after aggregation. This metric reflects the overall data transfer required synchronize model updates across the network.
15

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
Table 4: Node classification performance of EAGLES evaluated across numerous values of 5 and SW on Cora, photo and Ogbn-Products. Please refer to appendix D for additional analysis.
Purameter
Graph Sparsity& % Coru
Gruph Spursity & Photo
Graph Spursity
Ogbn-Produces
Sparsity: %t.
0
10
30
50
10
30
50
10
30
50
74.89.o.00 06
7o.
10.77
73.55
(1.40
92.03+o.01
85.51
1o.14
85.75
84,88 10.77 73.82.0.04
jo.0s
10.66.0.20
72.75
10
76.08
75.48p0.53
po.16 73.60
92.86y0.84
92.68
92.18 p2.52p0.00
73.65
73.89 po.0s
13. 73.82 j0.01
76.23
75.5
74.77.0.18
73.46.11.4
92.4040.3s
70.11
92.54
92.28
73.88.0.02
20.25
74.13
74.022o.16
30
75.55 co 74.97p0.02
p4.60
72.27 12.0s
92.044o.02
92.37
92.42
92.25
73,81 Lo.0s
73.99
f4.22
74.15po.29
40
16.14
75.15
75.10po.15
72.58 j2.37
92.46,0.44
92.41 10.39 92.31 10.29
92.26,0.24
74.23
70.85
74.98
74.16,0.30
75.30/0.35
p0
p0.43
po.52
43.09.1..00
41.80
92.26ya.24
20.09 91.95 10.07
91,86,0,16
74.33
74.78p0.92
74.46
74.09p0.23
60
oo. 75,26ro.3.
8.
14.31jo.0a
22.75
92.40
92.08 fo 92.49 40.47
92.14
74.07 73.99
73.76.o. jo.10
73,94,0.05
70
75.23
75.0440.00
74,88 10.07
72.69 12:26
92.23
92.0990.uz 00 po.
92.77.0.75 20.75
73.
33.64.0.222
73.93
73.82, 10.04
80
g3.94.1.01 75.3
72.46.2.59
j1.44
f1.88
92.27 70.26 92.23
10.52 33.52.0.34
73.84,0 13.02
y2.65 72.99 22.99
Vanilla
74.95
92.02
73,86
Table 5: Detailed hyper- parameter configurations.
Dataset
#Model
#Round
# #Weight Decay
#learning rate
#Optimizer
Cora
GCN
200
2e-4
0.01
Adam
pubmed
GCN
500
2e-4
0.01
Adam
Amz-Photo
GCN
800
2c-4
0.01
Adam
Ogbn-Arixv
GraphSAGE
800
le-6
0.01
Adam
Ogbn-Proteing
DeeperGCN
300
5e-6
0.01
Adam
-ebn-Products
GraphSAGE
800
lc-6
0.01
Adam
In Table 1, the reported FLOPS and BYTES values for each sparsification baseline method correspond to the scenarios where the highest accuracy is achieved. And the details of the baseline employed for comparison are as follows:
FedAvg (MeMahan et al., 2017): FedAvg a foundational algorithm in federated learning, allowing multiple clients to collaboratively train global model without sharing their raw data. Each client locally computes model updates based on its data, and the server aggregates these updates through periodic averaging to form the global model. This process maintains data privacy and minimizes communication costs, making it efficient for distributed environments.
FedProx (Li et al., 2020): FedProx extends FedAvg by introducing a proximal term in the objective function, which helps address issues related to client heterogeneity. By regularizing local updates toward the global model, FedProx stabilizes convergence, particularly in federated learning environments with non-iid data distributions and clients with diverse computational capabilities.
APPLE (Luo & Wu, 2022): The Adaptive Personalized Cross-Silo Federated Learning (APPLE) framework addresses non-IID data challenges by enabling clients to personalize models through selective integration of others' model information, APPLE employs three components: Core Model Sharing, where clients share a core model with the server for distribution; Directed Relationship (DR) Vector, assigning weights to received core models to enhance relevant integration; and Dynamic Penalty Mechanism that balances global and local objectives, allowing clients to tailor the model to both global trends and local needs.
FedSage+ (Zhang et al., 2021): FedSage+ is federated learning framework designed for graph neural networks (GNNs), addressing challenges of distributed graph data and heterogeneity. The framework leverages Dynamic Neighbor Sampling to efficiently sample and aggregate information from local graph structures, reducing computational overhead while maintaining performance. FedSage+ employs three key components: Local Subgraph Training, where clients independently train on sampled subgraphs to preserve data privacy; Federated Aggregation, combining client models via weighted averaging (e-g., FedAvg); and Model Sparsification, reducing communication costs by pruning less important model parameters. This design achieves efficient learning while balancing global knowledge and local data diversity.
16

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
FedCP (Zhang et al., 2023): The Federated Conditional Policy (FedCP) method addresses client data heterogeneity in federated learning by separating global and personalized information within feature representations. It employs Con- ditional Policy Network (CPN) generate sample-specific policies that partition features into global and personalized components, These components are then processed by shared global head and a client-specific personalized head, respectively. This approach enables more fine-grained personalization, enhancing model performance across diverse client data distributions.
FedTAD (Zhu et al., 2024): FedTAD is a topology-awee -wwrre data-free knowledge distillation method designed for subgraph federated learning. addresses subgraph heterogeneity by decoupling node and topology variations, which correspond to differences in label distribution and structure homophily. FedTTAD enhances reliable knowledge transfer from local models to the global model by leveraging topology-aware node embeddings to measure class-wise knowledge reliability. This approach improves the performance of the global graph neural network without requiring raw data exchange, making it suitable for scenarios with data privacy constraints.
FedSSP (Tan et al., 2024): FedSSP is a federated graph learning framework that addresses structural heterogeneity across clients by sharing generic spectral knowledge and accommodating personalized preferences. introduces a global spectral knowledge-sharing mechanism capture common structural patterns and personalized preference module to adjust local message-passing passing schemes. This combination enhances model performance in cross-domain scenarios by effectively balancing global collaboration and local adaptation.
FGGP (Wan et al., 2024a): Federated Graph Learning with Generalizable Prototypes (FGGP) addresses the challenge of domain shift in federated graph learning. It decouples the global model into two fevels-feature extractor and classifier-ande and bridges them via prototypes, which serve semantic centers derived from the feature extractor. At the classifier level, FGGP leverages clustered prototypes to capture domain-specific information and enhance class discriminability. At the feature extractor level, it employs contrastive learning to obtain more robust prototypes, thereby improving the generalization ability of the feature extractor across diverse domains.
FedGTA (Li et al., 2024): FedGTA (Federated Graph Topology Augmentation) is a federated learning framework tailored for graph data, tackling challenges of graph heterogeneity and sparsity. The framework introduces Topology Augmentation, dynamically enhancing local graph structures by generating virtual neighbors or predicting missing edges based on node feature similarity and local connectivity. FedGTA employs three key components: Local Graph Enhancement, where clients augment their subgraphs to address sparsity; Federated Aggregation, combining model updates and augmentation summaries to improve global learning; and Personalized Topology Rules, allowing clients to adapt global trends to their unique graph structures. This approach enhances performance in distributed, heterogeneous graph scenarios while ensuring data privacy.
DSpar (Liu et al., 2023c): DSpar offers a straightforward strategy for improving the efficiency of Graph Neural Network (GNN) training and inference by selectively sparsifying graph structures based on node degree. By prioritizing digrer-degree nodes, DSpar reduces the number of edges while preserving essential structural information, thus significantly lowering computational demands without compromising model accuracy. This approach is particularly suitable for large-scale graphs, where computational efficiency is critical.
ACE-GLT (Wang et al., 2023): Adaptive Compression and Efficient Gradient Learning Transmission (ACE-GLT) is a centralized sparsification method that adaptively compresses gradients during transmission. This technique effectively reduces communication costs while maintaining model performance, making especially advantageous in environments with bandwidth constraints or high communication costs.
PruneFL (Jiang et al., 2022): PruneFL employs an adaptive, distributed pruning strategy within federated learning to optimize performance on resource-constrained edge devices. Initially, an aggressive pruning phase reduces model size on chosen client, followed by further pruning during training across clients. This iterative approach lowers both communication and computational demands while preserving model accuracy, making it highly suitable for deployment in edge environments with limited resources.
17

---

EAGLES: Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
Fed'Tiny (Huang al., 2023): FedTiny is a distributed pruning framework tailored to produce compact neural networks for federated learning devices with limited memory and computational capacity. It employs an adaptive batch normalization selection to address data heterogeneity and progressive pruning module that incrementally adjusts pruning policies at each layer. This design effectively minimizes computational and memory overhead while preserving model accuracy, making it well-suited for deployment in resource-constrained edge environments.
C. Ablation with Different Number of Clients
In this section, we conduct ablation experiments the number of clients. Figure 6a and Figure 6b respectively show the system performance for 10, 30, 50, 70, and 100 clients Ogbn-Arxiv and Pgbn-Proteins As the number of clients increases, overall performance tends to decline. However, EAGLES consistently outperforms the baseline in most cases. For instance, with 100 clients on Ogbn-Proteins, EAGLES surpasses the baseline by 2.13%, demonstrating its resilience. In contrast, on Ogbn-Arxiv with 100 clients, EAGLES shows a 0.26% decrease compared to the baseline. This may be attributed the smaller size of Ogbn-Arxiv, where splitting the dataset among 100 clients results in fewer training nodes per client, limiting the model's ability to effectively learn patterns inherent the dataset graph.
Ogbn-Arxiv
Ogbn- Proteinste
85
64
FedAvg
FedAvg EAGLES
Top-1 Accuracy
EAGLES
82
60
ROC-AUC
79
56
76
52
73
48
10
30
50
70
100
70
30
50
70
100
Number Clients
Number Clients
(a) Ablation Analysis of the number of clients on the Ogbn-
(b) Ablation Analysis of the number of clients on the Ogbn- Proteins. Please refer appendix C for additional analysis.
ArXiv. Please refer appendix C for additional analysis.
D. Sensitivity Analysis
In this section, we address Q4 by conducting extensive ablation studies. Table 2 and Table 4 present a sensitivity analysis across various sparsification rates. Obs. When fixing the row and observing the column data, we observe that overall model performance follows a trend of initially increasing and then decreasing, with optimal performance achieved when so approximately 30%. Obs. Fixing the column and analyzing the row data reveals that parameter sparsification can be effectively deepened, achieving optimal performance in the range of 50% to 70% for sw although slight variations may arise depending the dataset. Obs. Simultaneous application of graph-level and parameter-level sparsification leads to significant improvements in model performance; for instance, in Ogbn-Arxiv with (Se, SW) (30%, 70%), accuracy increases by 1.38% 1, while in Ogbn-Proteins with (S sw) = (30%, 50%), accuracy improves by 1.70% 1.
Figure 7a and Figure 7b illustrate the ablation analysis of two key parameters in our method: X2 and the number of sparsifica- tion experts N Obs. We observed that A2 exerts minimal influence on model performance, reaching its peak within the range of 0.3 to 0.4. Obs. G The classification accuracy exhibits increasing trend with the growth of N In our experiments, set No to 3, which prevents the introduction of excessive experts while still achieving relatively superior performance.
85
Ag Abalation
66
Ag Abalation
85
No Abalation
66
N" Abalation
84
65
65
83
Accurasy
64
BOCC-UCC (UC))
83
Acturay
84
63
9 63
81
62
81
61
Abalation of Ag
Abalation of N
(a) Ablation Analysis of Xa the Ogbn-Proteins Proteins and Ogbn-
(b) Ablation Analysis of the number of esparsification experts
ArXiv, Please refer appendix D for additional analysis.
N the Ogbn- Proteins and Ogbn-ArXiv. Please refer
appendix for additional analysis.
18

---

EAGLES:
Towards Effective, Efficient, and Economical Federated Graph Learning via Unified Sparsification
E. Implementation Details
The experiments are conducted using NVIDIA GeForce RTX 3090 GPUs as the hardware platform, coupled with an Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz. The deep learning framework employed is Pytorch, version 2.0.1, alongside CUDA version 11.7. For the small datasets Cora, pubmed and Ama-photo we utilized 2-layer GCN, For the larger datasets ogbn-arxiv and default number of clients is set to 10.
we employed 4-layer GraphSage and applied 4-layer DeeperGCN to ogbn-proteins. The
F. Quantitative analysis of heterogeneity reduction
We compute the overall similarity hetween two graphs by equally weighting structure similarity and feature similarity (50% each). The computation methods for each are as follows:
E, Ej1 E, + B3 +
F,
Sureet = 1
Steature
Swail = 0.5 S + 0.5 Steature
where E, and E, denote the number of edges in the two graphs, and is included to prevent division by zero. The average node feature vector for graph is defined as F. M Evev, Fu; the feature vectors are truncated to the same dimension dmin = min(d;, d,) during computation.
Employing the above calculation, we present heatmap visualizations for 10 clients on the PubMed and Ogbn-arxiv datasets. These visualizations illustrate the client subgraphs both at initialization and after five rounds communication. Notably, an overall increase isomorphism among clients can be observed, suggesting improved structural alignment through communication. The results are shown below.
044839008E N S7 379035930
E Estaneess SED CN 35 ES4004
20003320 to 3080
Fiee communccation runds
ameessm Entoaee 353000a00
Five communication rounds
os
47001N061 (1977169977
A35E 85 01 894 11807
amcaassenee mase samness
Sonoass 3011 309 1I00NEE
378 panorze aNAVEEESSEEE 20315ce13
nce
Pubmed
Oghn-andv
(a) Quantitative analysis of heterogeneity reduction Pubmed.
(b) Quantitative analysis of heterogeneity reduction Ogbn- arxiv.
19


Reviews:
[{"review_id": "JgWaPyWDFW", "reviewer": "Reviewer_PxDJ", "summary": "This work introduces EAGLES, a framework for Federated Graph Learning (FGL) that reduces computational demands while maintaining high performance. By unifying graph and model sparsification, it simplifies graph structures and prunes model parameters efficiently. EAGLES uses multi-criteria experts to sparsify graphs and integrates results using a synergy expert, ensuring better knowledge sharing across clients with diverse data.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate for taking the time to review our manuscript and hope our response will address your concerns and contribute to an improved score.\n\n> ` W1: How the GSyE processes and combines the outputs of various sparsification experts.`\n\nIn our method, GSyE (Graph Synergy Expert) is used to fuse and refine different subgraphs generated by multiple GSE (Graph Sparsification Expert, aiming to alleviate *structure information overfitting* during the graph sparsification process. The specific steps are as follows:\n\n1. Multiple GSE generate various versions of subgraphs based on different criteria (Eq. (11)).\n2. For the target node *v*, its node features $\\mathbf{X}$ are projected using a learnable matrix $\\mathbf{W}_{gate}$ to obtain gating scores $\\boldsymbol{z}$, which are then used to generate continuous approximations $\\psi(\\boldsymbol{z})$ of binary gates, thereby enabling gradient-based optimization (Eq. (13)).\n3. The HardStep function (Eq. (14)) is applied to hard-threshold $\\psi(\\boldsymbol{z})$ to either 0 or 1, determining whether to activate the corresponding expert's recommendation for the edge $e_{ij}$.\n4. For each edge $e_{ij}$, if at least one expert recommends retaining it, the edge is marked as a candidate edge; otherwise, it is pruned directly.\n\n\n\n> `Q1: How the hard concrete distribution is optimized?`\n\nThe hard concrete distribution is applied to the raw gating scores $\\boldsymbol{z}$ (Eq. (12)) to produce continuous probabilities $\\psi(\\boldsymbol{z})$. The HardStep function further binarizes $\\psi(\\boldsymbol{z})$ into discrete gates—0 and 1. During backpropagation, we employ the Straight-Through estimator [1] to approximate gradients and address the optimization problem.\n\n[1]: Bengio, Y.; Léonard, N.; and Courville, A. (2013). *Estimating or propagating gradients through stochastic neurons for conditional computation*. arXiv preprint arXiv:1308.3432.\n\n\n\n> `Q2: How dynamic masking thresholds are computed? + Q3: How client-specific masks are aligned?`\n\nIn consensus-informed parameter sparsification, dynamic masking thresholds are computed through a layer-wise adaptive process. For the $l$-th layer’s parameter matrix $W^{(l)}$, a threshold vector $\\kappa\\_{0}^{(l)}$ is dynamically optimized using straight-through estimators (STE) to bypass non-differentiability during backpropagation. Specifically, parameters $W^{(l)}$ are pruned if their absolute values fall below $\\kappa\\_{0}^{(l)}$, where thresholds are updated via a loss function (Eq. (6)) to maximize sparsity while maintaining performance. Client-specific masks are aligned through a consensus mechanism where overlapping “1”s in binary masks across clients form a unified sparse subnetwork, enabling parameter sharing and communication efficiency.\n\n\n\n> `Q4: How the rollback pruning strategy ensures pruning stability?`\n\nThe rollback pruning strategy ensures pruning stability by designating, at each predefined pruning checkpoint (for example, every 10% increment in pruning rate), the highest-performing subnetwork within an acceptable accuracy range (±3%). Before moving on to a deeper level of pruning, the method reverts (rolls back) to this optimal subnetwork. This rollback step prevents the accumulation of errors from continuous pruning and mitigates sudden drops in performance, thereby maintaining overall model stability and ensuring effective deep pruning.\n\n\n\n> `Q5: How the framework balances trade-offs between structural similarity, computational efficiency, and model performance in the optimization process (Equation 3)?`\n\nOur weighted loss function combines a structural similarity term (enforcing alignment of sparse subnetworks via mask consensus), a computational cost penalty (promoting parameter sparsity to reduce FLOPs), and a task-specific performance loss (e.g., cross-entropy for accuracy). Hyperparameters $λ\\_1$ and $λ\\_2$ dynamically adjust the trade-offs: higher $λ\\_1$ prioritizes mask consistency across clients, while $λ\\_2$ controls sparsity-intensity.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "Yes", "essential_references_not_discussed": "Yes", "other_strengths_and_weaknesses": "Strengths:\n\n- EAGLES introduces a unified sparsification framework that simultaneously sparsifies graph structures and model parameters. It uses multi-criteria graph sparsification experts and a synergy expert to reduce graph size while preserving critical structural information.\n\n- By addressing key challenges like data heterogeneity, computational inefficiency, and communication overhead, EAGLES provides a scalable and economical solution for Federated Graph Learning. Consensus-based parameter sparsification further ensures efficient pruning without iterative adjustments, addressing computational and communication overhead.\n\n- The evaluation shows resilience under varying sparsification rates and client distributions, making it effective for large-scale federated graph learning.\n\nWeaknesses:\n\n- While the paper introduces the Graph Synergy Expert to integrate sparsified subgraphs from multiple experts, it would benefit from a more detailed step-by-step explanation of how the GSyE processes and combines the outputs of various sparsification experts.\n\nFor instance, describing how the hard concrete distribution is optimized and how the gating mechanism selects and integrates key structural information for each node would enhance clarity.\n\n- The consensus-based parameter sparsification strategy is described briefly, but its implementation could use more depth. A detailed breakdown of how the dynamic masking thresholds are computed, how client-specific masks are aligned, and how the rollback pruning strategy ensures pruning stability would be valuable.\n\n- Elaborating on how the trade-offs between structural similarity, computational efficiency, and model performance are balanced in the optimization process (Equation 3) would strengthen the framework.", "other_comments_or_suggestions": "N.A", "questions_for_authors": "The paper introduces the Graph Synergy Expert (GSyE) to integrate sparsified subgraphs from multiple experts, but a more detailed, step-by-step explanation of its process would enhance clarity. Specifically, describing how the GSyE optimizes the hard concrete distribution and how the gating mechanism selects and integrates key structural information for each node would provide valuable insights. Including examples, pseudocode, or visualizations could further illustrate the functionality and importance of this component.\n\nThe consensus-based parameter sparsification strategy is briefly mentioned, but its implementation could benefit from greater depth. A detailed explanation of how dynamic masking thresholds are computed, how client-specific masks are aligned, and how the rollback pruning strategy ensures pruning stability would make the approach more comprehensible.\n\nAdditionally, elaborating on how the framework balances trade-offs between structural similarity, computational efficiency, and model performance in the optimization process (Equation 3) would offer a stronger understanding of its effectiveness.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Rc3Y3L7dA0", "reviewer": "Reviewer_jagF", "summary": "EAGLES introduces a framework designed to reduce computational and communication costs in federated graph learning by jointly sparsifying both model parameters and graph structures. It employs client-consensus pruning to generate subnetworks at different sparsity levels and utilizes a mixture of experts for graph sparsification. This approach achieve substantial reductions in FLOPs and communication overhead across various datasets, all while preserving accuracy. Results show improved performance over baselines in node classification tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Dear Reviewer jagF\n\nWe sincerely appreciate your detailed review and invaluable feedback. In the response below, we provide a thorough reply to address your concerns and offer a clearer explanation of our method.\n\n> ` W1: what $W_{gate}$ refers to in eq (12) lacks the necessary explanation in the text.`\n\n$W\\_{\\text{gate}}$ is a learnable weight matrix in the Graph Synergy Expert (GSyE) module that performs gating. It projects the node feature matrix $X$ into a latent space for Hard Concrete sampling, determining which graph sparsification experts to activate. In essence, $W\\_{\\text{gate}}$ generates gating vectors $z$ that, after thresholding, indicate the significance of each edge in the final sparsified subgraph. We will introduce $W_{\\text{gate}}$ in the revised manuscript.\n\n\n\n> ` W2: Performance on heterophilic graphs (e.g., arXiv) remains unvalidated.`\n\nWe conducted experiments on heterophilic graphs using ogbn-arxiv-TA [1] from the HeTGB (Heterophilic Text-attributed Graph Benchmark):\n\n|   Methods   | Top-1 Accuracy |   Max Training FLOPS    |     Communication      |\n| :---------: | :------------: | :---------------------: | :--------------------: |\n| **FedAvg**  |     64.24      |      1x (1.78E10)       |      1x (6.16E9)       |\n| **PruneFL** |     61.37      |          0.58x          |         1.00x          |\n| **FedTAD**  |     64.92      |         32.42x          |         1.11x          |\n| **EAGLES**  |   **65.89**    | **0.33x  (↓0.68x**) | **0.48x** (↓**0.52x**) |\n\nThe results show that on heterophilic graphs using ogbn-arxiv-TA, EAGLES also demonstrates superiority.\n\n[1]: Li, S.; Wu, Y.; Shi, C.; and Fang, Y. (2025). *HeTGB: A Comprehensive Benchmark for Heterophilic Text-Attributed Graphs*. arXiv preprint arXiv:2503.04822.\n\n\n\n> ` W3 & S1: Whether the proposed method can speed up the experiments was not explored, and supplement experiments on training time to further verify the efficiency of the method.` \n\nWe measured the time required for clients to reach the target accuracy on the ogbn-arxiv dataset across different methods, and the accuracy achieved by different methods at the same number of epochs on the ogbn-proteins dataset. The results are presented below:\n\n**TIME TO REACH TARGET ACCURACY**\n\n|   Methods   | Time to reach 70% accuracy | Time to reach 80% accuracy | Time to reach 90% accuracy |\n| :---------: | :------------------------: | :------------------------: | :------------------------: |\n| **FedAvg**  |          54.23 s           |          223.42 s          |          376.23 s          |\n| **FedTiny** |          35.28 s           |          149.08 s          |          281.23 s          |\n| **FedDIP**  |          36.44 s           |          155.68 s          |          278.62 s          |\n| **PruneFL** |          40.52 s           |          177.21 s          |          319.53 s          |\n| **EAGLES**  |        **14.04 s**         |        **85.94 s**         |        **162.79 s**        |\n\n**ROC-AUC UNDER THE SAME EPOCH**\n\n|   Methods   | EPOCH: 50 | EPOCH: 150 | EPOCH: 300 |\n| :---------: | :-------: | :--------: | :--------: |\n| **FedAvg**  |  70.34   |   80.38    |   81.49    |\n| **FedTiny** |   71.36   |   77.68    |   79.90    |\n| **FedDIP**  |   70.98   |   78.59    |   81.33    |\n| **PruneFL** |   69.22   |   76.23    |   78.69    |\n| **EAGLES**  | **73.97** | **81.85**  | **82.32**  |\n\nThe results show that EAGLES can train the model to a target accuracy in a shorter amount of time, and also achieve higher performance at the same number of epochs. This further verifies that our proposed method can accelerate model training.\n\n\n\n> `Q1: Can the authors provide additional ablation experiments regarding $λ_2$ and $λ_3$ in Eq(19)`\n\nWe conducted ablation experiments on $λ_2$ and $λ_3$ on the ogbn-arxiv dataset, and the results are presented below:\n\n**Ogbn-arxiv** (fix $λ_3$ = 1e-6)\n\n|    dataset     | $λ_2$=0.1 |    $λ_2$=0.05    |    $λ_2$=0.2     |\n| :------------: | :----: | :-----------: | :-----------: |\n|   **Pubmed**   | 86.97  |     86.23 (↓0.74)     | 86.96 (↓0.01) |\n|   **photo**    | 92.31  | 92.75 (↑0.44) | 91.85 (↓0.46) |\n| **Ogbn-arxiv** | 65.37  | 64.92 (↓0.45) | 65.33 (↓0.04) |\n\n**Ogbn-arxiv** (fix $λ_2$ = 0.1)\n\n|    dataset     | $λ_3$=1e-6 |    $λ_3$=1e-5    |    $λ_3$=1e-7    |\n| :------------: | :-----: | :-----------: | :-----------: |\n|   **Pubmed**   |  86.97  | 84.22 (↓2.75) | 86.89 (↓0.08) |\n|   **photo**    |  92.31  | 90.56 (↓1.75) | 92.15 (↓0.16) |\n| **Ogbn-arxiv** |  65.37  | 64.38 (↓0.99) | 65.42 (↑0.05) |\n\nIncreasing $λ_2$ enforces stricter GSyE constraints, balancing expert contributions and enhancing subgraph homogeneity. In heterogeneous scenarios, raising $λ_2$ from 0.05 to 0.2 leads to a clear accuracy boost. Conversely, while a larger $λ_3$ speeds up sparse parameter identification, it may cause significant accuracy loss; a smaller $λ_3$ permits finer-grained sparsification.", "claims_and_evidence": "The claims are largely supported by experiments across six datasets (Cora, Pubmed, OGB benchmarks) and comparisons with 14 baselines. Evidence includes:\n\n1. Table 1 shows EAGLES outperforms FedAvg/FedProx in accuracy (e.g., +1.32% on Pubmed) while reducing FLOPs (52%) and communication (63%).\n\n2. Ablation studies (Table 2, Figure 5) validate the impact of sparsity levels.\n\n3. Theoretical grounding via the Harmony Sparsification Principle (Eq. 3) aligns with empirical results.", "methods_and_evaluation_criteria": "1. Parameter Sparsification: Dynamic threshold optimization with STE and consensus masking (Eq. 4-7) is novel and suitable for federated settings.\n\n2. Graph Sparsification: Dual experts (GSE/GSyE) with hard concrete distribution (Eq. 13-17) effectively address structural heterogeneity.\nEvaluation：\n\n3. Metrics (Top-1 Accuracy, ROC-AUC, FLOPs, communication bytes) are standard and comprehensive.", "theoretical_claims": "The theoretical claims are basically correct. However, what $W_{gate}$ refers to in eq (12) lacks the necessary explanation in the text.", "experimental_designs_or_analyses": "The framework is validated across 6 datasets with diverse backbones (GCN, GraphSAGE) and compared against 14 baselines, including the state-of-the-art federated graph learning method (FedTAD) and pruning approaches like ACE-GLT.", "relation_to_broader_scientific_literature": "The contributions of the paper are related to the broader scientific literature of following areas. \n\n1. FGL: Improves FedAvg/FedProx by addressing graph/parameter redundancy.\n\n2. MoE: Adapts mixture-of-experts to graph sparsification (novel).\n\n3. Pruning: Unifies model/graph pruning, unlike DSpar/ACE-GLT.", "essential_references_not_discussed": "This manuscript compares and discusses quite a few baseline methods.", "other_strengths_and_weaknesses": "Strengths:\n\n1. The paper creatively bridges federated learning and graph sparsification, addressing both computational and structural challenges in FGL. This dual focus (parameter + graph sparsification) is novel and addresses a critical gap in federated graph learning literature.\n\n2. The framework’s ability to handle large-scale graphs (e.g., ogbn-proteins with 132,534 nodes) demonstrates real-world applicability.\n\nWeaknesses:\n\n1. Experiments focus on homophilic graphs (e.g., Cora, OGB). Performance on heterophilic graphs (e.g., arXiv) remains unvalidated.\n\n2. Whether the proposed method can speed up the experiments was not explored.", "other_comments_or_suggestions": "It is suggested to supplement experiments on training time to further verify the efficiency of the method.", "questions_for_authors": "Can the authors provide additional ablation experiments regarding $\\lambda_2$ and $\\lambda_3$ in Eq (19)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Hciq3kM037", "reviewer": "Reviewer_Fguv", "summary": "The paper introduces EAGLES, a unified sparsification framework designed to enhance FGL by addressing computational and communication challenges. EAGLES optimizes both graph structures and model parameters through client-consensus parameter sparsification, which generates multiple unbiased subnetworks at various sparsity levels. The method also employs a dual-expert approach with graph sparsification and synergy experts, which improve the efficiency of message passing and reduce data overfitting. The comprehensive experimental results validate the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Dear Reviewer Fguv\nWe sincerely thank you for taking the time to evaluate our work and have adressed your concerns as follows:\n\n>  ` W1: Additional empirical studies addressing the significant computational challenges faced by FGL will further strengthen this analysis.`\n\nIn the theoretical model, the message passing mechanism in GNNs causes the neighborhood size to expand exponentially with the number of layers. For a graph with an average degree of $d$, 1-hop neighborhood covers $d$ neighbors, a 2-hop neighborhood covers $d^2$ neighbors, and an $L$-hop neighborhood covers $d^L$ neighbors [1].\n\nWe measured the k-hop receptive fields (k=1,2,3,4) for the amz-photo and Ogbn-arxiv datasets. The results are as follows:\n\n|   datasets    | 1-hop | 2-hop  |  3-hop  |  4-hop  |\n| :-----------: | :---: | :----: | :-----: | :-----: |\n| **amz-photo** | 32.13 | 802.86 | 2519.35 | 4681.62 |\n\nThe results show that the receptive fields exhibit a clearly super-linear growth trend, confirming that GNNs indeed face significant computational challenges.\n\n[1]: Xu, K.; Hu, W.; Leskovec, J.; and Jegelka, S. (2019). How Powerful are Graph Neural Networks? arXiv preprint arXiv:1810.00826.\n\n\n\n> `W2: Supplementary experiments addressing the omitted DropEdge.`\n\nWe conducted experiments on DropEdge on the PubMed, with some of the results presented below (we adopted the original 0.8 retention rate for GCN).\n\n**Pubmed**\n\n|   Methods    | Top-1 Accuracy | Max Training FLOPS | Communication BYTES |\n| :----------: | :------------: | :----------------: | :-----------------: |\n|  **FedAvg**  |     85.65      |    1x (2.49E9)     |     1x (6.19E9)     |\n| **DropEdge** |     85.78      |   0.90x (↓0.10x)   |   1.00x (↓0.00x)    |\n|  **EAGLES**  |   **86.97**    | **0.48x (↓0.52x)** | **0.37x (↓0.63x)**  |\n\n> `W3: A typo on page seven in Section 5 where \"comprehensively\" is misspelled as \"omprehensively.\"`\n\nThank you for your careful reading. We have corrected the typo and carefully proofread the manuscript to fix similar issues.\n\n\n\n> `W4: Validate the proposed method on social network graph datasets. `\n\nWe conducted experiments on the Flickr dataset to validate the effectiveness of the proposed method on social network graph datasets:\n\n**Flickr** \n\n|   Methods   |  Top-1 Accuracy   | Max Training FLOPS | Communication BYTES |\n| :---------: | :---------------: | :----------------: | :-----------------: |\n| **FedAvg**  |       50.15       |    1x (8.49E9)     |     1x (4.67E9)     |\n|  **FGGP**   |   49.78(↓0.37)    |   1.23x (↑0.23x)   |   1.33x (↑0.33x)    |\n| **PruneFL** |   47.45(↓2.70)    |   0.77x (↓0.23x)   |   1.00x (↓0.00x)    |\n| **FedDIP**  |   50.33(↑0.17)    |   0.67x (↓0.33x)   |   0.83x (↓0.17x)    |\n| **EAGLES**  | **50.89** (↑0.74) | **0.48x (↓0.52x)** | **0.37x (↓0.63x)**  |\n\n\n\n> `S1: The manuscript specifies the split ratios for each dataset but does not describe the splitting strategy. `\n\nThank you for pointing that out. We will include additional details on the splitting strategy in the revised manuscript.\n\n> `Q1: How does the proposed method perform in scenarios where clients have vastly different computational capabilities (e.g., edge devices versus more powerful systems)?`\n\nClients with limited resources can opt for higher sparsity to reduce memory and computation, while more capable machines may choose lower sparsity for better accuracy. Additionally, consensus-based parameter masks and a multi-expert graph sparsification framework ensure that all clients benefit from an efficient, robust model.\n\n\n\n> `Q2: Can the proposed approach still be effective under other non-iid partitioning methods, such as Metis?`\n\nWe conducted experiments on Cora and ogbn-arxiv, and the results are shown below:\n\n**Cora** \n\n|   Methods   |  Top-1 Accuracy   | Max Training FLOPS | Communication BYTES |\n| :---------: | :---------------: | :----------------: | :-----------------: |\n| **FedAvg**  |       70.62      |    1x (6.72E8)     |     1x (6.02E9)     |\n|  **FGGP**   |   69.58 (↓1.04)   |   1.42x (↑0.42x)   |   1.18x (↑0.18x)    |\n| **PruneFL** |   67.94 (↓2.68)   |   0.57x (↓0.43x)   |   1.00x (↓0.00x)    |\n| **FedDIP**  |   70.38 (↓0.24)   |   0.61x (↓0.39x)   |   0.59x (↓0.41x)    |\n| **EAGLES**  | **71.27** (↑0.65) | **0.48x (↓0.52x)** | **0.37x (↓0.63x)**  |\n\n**Ogbn-arxiv** \n\n|   Methods   |    Top-1 Accuracy     | Max Training FLOPS | Communication BYTES |\n| :---------: | :-------------------: | :----------------: | :-----------------: |\n| **FedAvg**  |         55.30         |    1x (1.58E10)    |     1x (6.14E9)     |\n|  **FGGP**   |     55.09(↓0.21)      |   5.66x (↑4.66x)   |   1.45x (↑0.45x)    |\n| **PruneFL** |     52.34 (↓2.96)     |   0.69x (↓0.31x)   |   1.00x (↓0.00x)    |\n| **FedDIP**  |     55.32 (↑0.02)     |   0.48x (↓0.52x)    |   0.59x (↓0.41x)    |\n| **EAGLES**  | **56.89** **(↑1.59)** | **0.35x (↓0.65x)** | **0.47x (↓0.53x)**  |\n\nThe results show that under Metis partitioning, EAGLES still exhibits superior performance.", "claims_and_evidence": "The paper provides a relatively clear explanation of its claims. FGL faces significant computational challenges when handling large-scale graph data. Figure 1 effectively illustrates this phenomenon. However, additional empirical studies could further corroborate this analysis and strengthen the claims presented in the paper.", "methods_and_evaluation_criteria": "The proposed methodology and evaluation criteria align well with the problem of optimizing federated graph learning. The dual-expert sparsification approach appears to be a reasonable solution, and the chosen evaluation metrics (FLOPS and communication costs) are directly applicable to the problem at hand.", "theoretical_claims": "The theoretical section of the manuscript is relatively detailed. In particular, the Harmony Sparsification Principle and its impact on federated graph learning are interesting and well-reasoned, providing concrete theoretical guidance and practical reference for the design of sparsification frameworks.", "experimental_designs_or_analyses": "Extensive experiments across six datasets and multiple backbones (GCN, GraphSAGE, DeeperGCN) strengthen validity. Ablation studies on sparsity rates and client numbers (Figures 4–7) convincingly demonstrate resilience.", "relation_to_broader_scientific_literature": "EAGLES builds on federated learning (FedAvg, FedProx) and graph sparsification (DSpar [1]). The integration of MoE for graph pruning is novel, advancing prior work on MoE [2]. \n\n[1] Liu Z, Zhou K, Jiang Z, et al. DSpar: An Embarrassingly Simple Strategy for Efficient GNN Training and Inference via Degree-based Sparsification. arXiv preprint arXiv:2307.02947, 2023. \n\n[2] Shazeer N, Mnih A, Ranzato M, et al. Outrageously large neural networks: The sparsely gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538, 2017.", "essential_references_not_discussed": "The key contribution of the paper is the unified sparsification approach for FGL, but it only references a graph sparsification technique, DSpar, that sparsifies graph structures based on node degree. However, there is also a relevant method, DropEdge, introduced by [3], which applies random edge dropout to improve deep graph convolutional networks for node classification. This technique is particularly important for reducing computational costs while preserving graph structure and can be considered an essential reference for addressing graph sparsification challenges in the context of FGL, especially in comparison to the single-criterion sparsification discussed in the paper.\n\n[3] Rong Y, Huang W, Zhang Y, et al. DropEdge: Towards Deep Graph Convolutional Networks on Graphs with Sparse Edge Features. arXiv preprint arXiv:2006.10616, 2020.", "other_strengths_and_weaknesses": "**Strengths:**\n- This paper introduces the first unified framework for both graph and parameter sparsification in FGL.\n- The motivation behind this paper is explained with great clarity.\n- This paper presents a novel use of Optimal Transport (OT) to measure client similarity, which is an interesting approach.\n\n**Weaknesses:**\n- There is a typo on page seven in Section 5 where \"comprehensively\" is misspelled as \"omprehensively.\"\n- Experiments focus primarily on academic citation and biological networks. There is no validation on social network graphs. Including relevant experiments would strengthen the generalizability and applicability of the proposed method.", "other_comments_or_suggestions": "The manuscript specifies the split ratios for each dataset but does not describe the splitting strategy. The authors should include details on the splitting approach in the manuscript.", "questions_for_authors": "1.How does the proposed method perform in scenarios where clients have vastly different computational capabilities (e.g., edge devices versus more powerful systems)?\n\n2.In the code, the authors only perform data partitioning using the Louvain method. Can the proposed approach still be effective under other non-iid partitioning methods, such as Metis?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ca6ApK8T6b", "reviewer": "Reviewer_h7Za", "summary": "This paper introduces a unified framework that jointly considers graph-level and parameter-level sparsification. It incorporates dual experts and consensus-based sparsification to ensure a stable sparsification process. Extensive experiments demonstrate that the proposed method is effective, efficient, and economical.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Dear Reviewer h7Za\n\nWe sincerely thank you for your insightful feedback and have provided detailed responses to your questions.\n\n> ` W1: Without quantitative analysis of heterogeneity reduction.`\n\nWe provide a quantitative analysis of heterogeneity reduction at [this link](https://anonymous.4open.science/r/Appendix-67CF/README.md).\n\n> `W2: Lack an analysis of computational complexity & S1: How its sparsification techniques scale with increasing data size or client count.`\n\nWe analyzed the computational complexity from three aspects:\n\n1. **Parameter Sparsification Module**:\n   - Forward/Backward FLOPs are $O(s_p \\cdot d)$, where $s_p$ is the parameter sparsity rate and $d$ is the parameter dimension.\n   - Communication costs are reduced to $O(s_p \\cdot d)$ via bit-wise mask compression (Section 4.2).\n   - Mask alignment requires $O(K \\cdot L)$ operations per round, but with small constants for $K$ (clients) and $L$ (layers), its impact is negligible.\n\n2. **Graph Sparsification Module**:\n   - With $T$ GSEs, the local computation is $O(T \\cdot |E|)$, where $|E|$ is the number of edges.\n   - The message passing process has a complexity of $O(s_g \\cdot |E| \\cdot d)$, with $s_g$ as the graph sparsification rate.\n   - The gating mechanism adds $O(N \\cdot D)$ operations, but since $D$ is typically small, its overhead is minimal.\n\n3. **OT-based Similarity Computation**:\n   - Standard OT complexity is $O(n^3)$ for $n$-node graphs, but we reduce this to $O(n \\log n)$ using the sliced Wasserstein distance.\n\nOverall, the computational complexity of EAGLES is given by:\n$$\nO\\Big(s_p \\cdot d + T \\cdot |E| + s_g \\cdot |E| \\cdot d + n \\log n\\Big)\n$$\nIgnoring smaller constants, this simplifies to:\t\n$$\nO(d + |E| + n \\log n)\n$$\nIn summary, EAGLES scales linearly with the data size, and the number of clients has minimal impact on the computational complexity.\n\n\n\n> `W3: However, the impact of expert count (Figure 7b) is under-discussed.`\n\nAs shown in Figure 7b (Appendix D.3), performance improves consistently as the number of experts increases, reaching its peak around 4 experts due to the benefit of richer structural perspectives. Beyond this point, the marginal gains diminish. While Section 5.4 briefly touches on this point, we agree that a more in-depth analysis would further strengthen the discussion.\n\n\n> `Q 1.1: Could the authors clarify how the OT method adapts to the federated setting when client graphs have significant structural variations?  `\n\nIn FL settings with significant structural variations, our OT adaptation relies on two key mechanisms. First, the graph synergy expert encodes node contextual features in the $W_{\\text{gate}}$ matrix, forming a structure-aware semantic space via hard concrete distribution sampling. This enables OT to assess similarity based on learned semantics instead of raw topology. Second, by treating each client’s structural distribution as a probability measure over this space, we derive client-specific transport plans and similarity weights (Eqs. 20 and 22), which automatically assign lower weights to structurally dissimilar clients. Importantly, only the compact $W_{\\text{gate}}$ parameters are transmitted, preserving privacy while allowing the server to compute OT plans with $O(n \\log n)$ complexity through entropic regularization.\n\n\n\n> `Q 1.2: Would this method still function efficiently if the number of clients increased substantially?`\n\nEAGLES remains efficient even as the number of clients grows significantly. Our framework reduces communication and computation through parameter sparsification (dynamic mask consensus) and OT-based similarity aggregation, minimizing redundant interactions. Experiments (Appendix Fig. 6a & 6b) show that scaling to 100 clients on Ogbn-Proteins and Cora results in only a minor performance drop, while achieving an 18% reduction in Training FLOPS and a 20% reduction in Communication Bytes compared to baselines.\n\n\n\n> `Q 2.1: How does $W_{gate}$ impact parameter sparsification when the number of GSEs increases?`\n\n$W_{\\text{gate}}$ is a learnable gating parameter. Additional GSEs introduce richer structural diversity, and by integrating sparsified subgraphs obtained from multiple criteria through $W_{\\text{gate}}$, the robustness of graph sparsification is enhanced. This reduction in structural redundancy allows for the allocation of different gradient update weights to model parameters during backpropagation, thereby influencing parameter sparsification.\n\n\n\n> `Q 2.2: Would $W_{gate}$ lead to an excessive amount of additional parameters?`\n\nThe gating parameter matrix $W_{\\text{gate}}$ (Eq. (12)) is designed as a lightweight mapping layer with low dimensionality. Specifically, its parameter size is $D \\times T$, where $D$ is the input feature dimension and $T$ denotes the number of experts. Since the number of experts is typically a small constant, $W_{\\text{gate}}$ scales linearly with $D$, thereby not introducing an excessive number of parameters.", "claims_and_evidence": "The claims are supported by extensive experiments across datasets (Cora, Ogbn-Proteins) and metrics (FLOPS, ROC-AUC). Reductions in computational costs (82%↓ FLOPS) and communication (80%↓ bytes) are validated against baselines like FedAvg and ACE-GLT. However, claims about mitigating structural heterogeneity rely on qualitative arguments (e.g., \"similar clients share knowledge via OT distance\") without quantitative analysis of heterogeneity reduction.", "methods_and_evaluation_criteria": "The methods are well-suited for FGL challenges. Parameter sparsification avoids iterative pruning via dynamic masking, and graph sparsification addresses structural overfitting through multi-criteria experts. Evaluation on diverse datasets (small to large-scale) and metrics (FLOPS, ROC-AUC) is comprehensive.", "theoretical_claims": "The manuscript’s mathematical formulation is generally free from notable errors; however, it lacks an analysis of computational complexity, which would provide a clearer understanding of the scalability and practical applicability of the proposed methods.", "experimental_designs_or_analyses": "Experiments are thorough, covering multiple datasets, sparsity levels, and baselines. Ablation studies (Table 2) validate parameter-graph sparsity interplay. However, the impact of expert count (Figure 7b) is under-discussed.", "relation_to_broader_scientific_literature": "EAGLES makes a significant contribution by introducing a unified sparsification framework. The dual-expert approach, which builds upon MoE methods, adapts them for federated graph learning, an area that has seen limited exploration.", "essential_references_not_discussed": "The authors discuss and compare a wide range of related methods.", "other_strengths_and_weaknesses": "Strengths:\n(1) The paper effectively identifies a critical challenge in federated graph learning (FGL): the high computational cost and communication overhead when training GNNs on large-scale federated datasets. By introducing EAGLES, a unified sparsification approach, the authors provide a clear solution that addresses both graph and parameter sparsification, ensuring efficiency without sacrificing model performance.\n(2) The extensive set of experiments conducted across various benchmark datasets, including ogbn-proteins and Pubmed, demonstrates the practical effectiveness of the proposed method. The substantial reductions in training FLOPS and communication costs, achieved while maintaining or even improving model accuracy, provide strong empirical evidence of the method’s efficiency and scalability.\n\nWeaknesses:\n(1)While the method demonstrates significant improvements in computational efficiency, a clear computational complexity analysis would help contextualize the performance gains.", "other_comments_or_suggestions": "The computational complexity of EAGLES could be better articulated, particularly regarding how its sparsification techniques scale with increasing data size or client count. This would provide a clearer view of the system’s scalability in large federated environments.", "questions_for_authors": "(1)Could the authors clarify how the Optimal Transport (OT) method adapts to the federated setting when client graphs have significant structural variations? Would this method still function efficiently if the number of clients increased substantially?\n(2)How does $W_{gate}$ impact parameter sparsification when the number of GSEs increases? Would it lead to an excessive amount of additional parameters?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Three reviewers gave it a score of 4. The paper makes a clear and compelling case for the need to jointly consider parameter-level and data-level sparsification in FGL systems, a perspective that is both original and timely given the increasing deployment of GNNs in resource-constrained federated environments. The paper is backed by strong motivation and a novel problem setting and the authors provided additional analysis on computational complexity and clarified certain architectural details.Therefore, this paper can make it a valid contribution to ICML, and the decision is to recommend Acceptance.
Decision: Accept (poster)