Multi-order Orchestrated Curriculum Distillation for Model-Heterogeneous Federated Graph Learning
Authors: Guancheng Wan
Xu Cheng
Run Liu
Wenke Huang
Zitong Shi
Pinyi Jin
Guibin Zhang
Bo Du
Mang Ye
Venue: NeurIPS 2025 poster ()
Keywords: Federated Learning
Graph Learning
Knowledge Distillation
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Federated Graph Learning (FGL) has been shown to be particularly effective in enabling collaborative training of Graph Neural Networks (GNNs) in decentralized settings. Model-heterogeneous FGL further enhances practical applicability by accommodating client preferences for diverse model architectures. However, existing model-heterogeneous approaches primarily target Euclidean data and fail to account for a crucial aspect of graph-structured data: topological relationships. To address this limitation, we propose **TRUST**, a novel knowledge distillation-based **model-heterogeneous FGL** framework. Specifically, we propose Progressive Curriculum Node Scheduler to progressively introduce challenging nodes based on learning difficulty. In Adaptive Curriculum Distillation Modulator, we propose an adaptive temperature modulator that dynamically adjusts knowledge distillation temperature to accommodate varying client capabilities and graph complexity. Moreover, we leverage Wasserstein‑Driven Affinity Distillation to enable models to capture cross-class structural relationships through optimal transport. Extensive experiments on multiple graph benchmarks and model-heterogeneous settings show that **TRUST** outperforms existing methods, achieving an average 3.6\% $\uparrow$ performance gain, particularly under moderate heterogeneity conditions. The code is available for anonymous access at https://anonymous.4open.science/r/TRUST-NeurIPS2025.

Multi-order Orchestrated Curriculum Distillation for Heteeeneteo Federated Graph Learning
Guancheng Wan't, Xu Cheng Run Liu', Wenke Huang', Zitong Shi',
Pinyi Jin', Guibin Zhang", Bo Ye
Wuhan University 2NUS
(guanchengwan, yemang)
Abstract
Federated Graph Learning (FGL) has been shown to be particularly effective enabling collaborative training of Graph Neural Networks (GNNs) in decentralized settings. hodee--hetroggeneuus FGL further enhances practical applicability by ac- commodating client preferences for diverse model architectures. However, existing modee--hteroggeeous approaches primarily target Fuclidean data and fail to account for crucial aspect of graph strucuutr data: topological relationships. To address this limitation, we propose TRUST, a novel knowledge distillation-baa -baeed model- heterogeneous FGL framework. Specifically, we propose Progressive Curriculum Node Scheduler to progressively introduce challenging nodes based on learning difficulty. In Adaptive Curriculum Distillation Modulator, propose adaptive temperature modulator that dynamically adjusts knowledge distillation temperature accommodate varying client capabilities and graph complexity. Moreover, we leverage Wasserstein-Driieer Affinity Distillation to enable models to capture cross- class structural relationships through optimal transport. Extensive experiments multiple graph benchmarks and model- hheele--hterogeeeouss settings show that TRUST outperforms existing methods, achieving an average 3.6% performance gain, particularly under moderate heterogeneity conditions. The code is available for anonymous access at /tttt:://ggthbb.... httpchuw..coo/uuannnnnneneeeeeee
Introduction
Federated Learning (FL) 20
has emerged as a distributed machine learning paradigm that enables multiple clients to collaboratively train a global model without sharing their privacy- sensitive data, thereby preserving data confidentiality. Traditional locally computed model updates (e.g., gradients or weights) from participating clients under the coordination of central server, eliminating the need for direct data exchange. prominent branch of FL. is Federated Graph Learning (FGL) [53,
18
methods operate by aggregating
42 which specializes in handling graph-structured data. In addition to inheriting rrivacy-reesevvng benefits FL, FGL usually leverages Graph Neural Networks (GNNs) 21
6 41
30 to capture topology information in graph data, offering a flexible and expressive framework for modeling graph-structured information.
45
Although many existing FGL works have made significant progress in improving the performance of the global model, these studies are often based on the assumption that client models follow the same architecture. This assumption rarely holds in real-world scenarios, where computational resources and task requirements vary across participants and they prefer to design private models independently rather than agreeing on unified model architecture [54), ultimately restricting their real-worke
Equal Contribation.
Corresponding Author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

applicability. This challenge is formally termed model-heterogeeneous federated learning. To address this challenge, recent works have proposed several solutions. For instance, pFedHR [4 generates personalized model for each client through model reassembly to transfer knowledge between clients. DESA [15] leverages synthetie global data to distill knowledge from other client models. FedTGP (56] employs server-side maintained global prototypes to bridge heterogeneous models. However, these methods are primarily tailored for traditional data types like images and do not generalize to non-Euclidean graph-structured data. This gap motivates our core research question:
How can we design a madel-heterogenenoe FI. framework specifically tailored for graph data?
Some previous model-heterogene heterogeneous
FL methods leverage knowledge dis- tillation (KD) 13
Constant Learning Difficulty
Prac medd
9 50 45|
to trans-
Pighostt:
Traditinal ET.
fer knowledge between clients [6
24
[4] For example, Fed [4 in-
troduces small identical proxy mod- els for clients to bridge the global ar- chitecture discrepancy, and then they leverage KD to transfer knowledge be- tween large private and small proxy models. However, in the model- heterogeneous FGL scenario, there are significant differences in model architectures and computational capa- bilities between clients, and the graph data itself has high-order information such as complex non-Eur -on-Euclidean topol- ogy, multi-hop path pattern, and com- munity structure. Traditional knowl- edge distillation methods maintain a constant "distillation difficulty" (fixed task complexity) [10] (58) throughout the training process. This one-size-fits-de all approach creates dual dilemmas: For lightweight models, the early introduction of higher-order topology information causes learning bottlenecks and slow convergence. Conversely, for large-scale models with strong expressive ability, may be too easy for them to exploit their full capacity, resulting in inadeuuate capture of global structure information. This fixed diff-iiffiuulty paradigm cannot take into account the requirements brought about by model heterogeneity, preventing progressive learning and ultimately leading to degraded FGL Performance. This naturally raises the following question: I) How can we design a KD strategy that dynamically adjusts the task difficulty to accommodate heterogeneous architectures and complex graph topologies?
Nodie dificaly
Femer space
suece
Printt
Ande
Aunitation
mustiasiion Afficaly
mikk
High (enpernennt Leam dace
Fall eapaare aracianol anfermatie
Focur on aujer chenre
(ID Fixed Distilation Temperature
(TI) Missing cross-clase comparisons
Figure 1: Problem Illustration. We describe three challenges mndee--heeroggenouu encounter:: constant learning difficulty and fixed distillation temperature limit client-specific adaptation and topology preservation. And the lack of cross-class comparison impedes the capture of deep structural graph relationships
In addition to the KD task itself, current KL-divergence based KD approaches maintain static temperature parameters (controlling label distribution smoothness), which govern transfer intensity. This rigidity prevents adaptation to varying graph complexity, label sparsity, and noise patterns. To be specific, KD) at low temperatures will neglect small probability events, blurring subtle but important structural information like multi-hop paths and community features. In contrast, high temperatures lose fine-graine grained node-level information. This inflexibility in distillation signaling hinders client differentiation and deep topological knowledge transfer, ultimately limiting the capacity of the proxy model. This leads us to think that: II) How can we devise an adaptive KD scheme that dynamically calibrates temperature to harmonize global topology transfer with f fine-grained node details? Furthermore, the mentioned KD methods rely KL-divergence loss, which only compares probability distributions within the same class. This approach lacks cross-class comparison mechanisms, making proxy models fail to capture topological relationships across differrnt categories. This limitation leads to the third problem: III) How can we enable cross-class comparison to fully leverage deep structural information during the KD phase?
To address these challenges, inspired by curriculum learning 22 511, we propose MulTi-oRder Orchestrated CUrriculum DiSTillation (TRUST), novel modee--hetroogeneous FGL framework. For Problem D) we introduce the Progressive Curriculum Node Scheduler (PCNS), which progressively schedules node samples for each client from easy (nodes with typical class representations) to difficult (nodes situated near class boundaries that may confuse proxy models), thereby enabling heterogeneous

---

models to first absorb low-order semantic cues and then incrementally acquire high-order topological knowledge. For Issue [D] we propose the Adaptive Curriculum Distillation Modulator (ACDM), module that dynamically calibrates the distillation temperature during training- -hini mechanism allows the framework to fluidly shift emphasis between capturing global structures and preserving fine-grained node details. For Problem ITЛ) in addition to the KI.-Divergence loss, we introduce Wasserstein- Affinity Distillation (WDAD), which leverages class prototypes from private models and computes cross-class relational distances via the Wasserstein metric, thereby enabling comprehensive cross-class topological knowledge transfer. The contributions are as follows.
Problem Identification. We are the first to systematically study model-heterogeneous FGL and formally characterize three core challenges in KD-based methods: the need for dynamic task difficulty, adaptive distillation signaling, and cross-class relational transfer.
Practical Solution. We develop curriculum-guide distillation framework that progressively schedules node difficulty, dynamically adjusts distillation strength, and enables knowledge transfer across classes, effectively reconciling heterogeneous architectures with complex graph topologies. Experimental Validation. We conduct extensive experiments on multiple graph benchmarks under diverse architecture heterogeneity settings. Empirical results demonstrate that TRUST consistently outperforms state-of-the-ae baselines.
Preliminaries
2.1 Notations
Graph Neural Networks. Given a graph g = (V,E), where V denotes the set of nodes and represents the set of edges. For every node U V, U; is associated with a k-dimensional feature vector T; The feature vectors of all nodes are represented collectively as the feature matrix X E R The topology of the graph g is encoded in the adjacency matrix A E RNXN where A u) if (v, u) E & and A (u, u) = 0 otherwise. GNN models operate through iteratively updating node representations using a mnessage-passingg
mechanism, where each node aggregates information from its local neighborhood and updates its own state. Let N₁ denote the neighborhood nodes of 04 and for the 1-th layer of GNN model, the representation of h of node U, can be computed as:
h Update [1!- Aggregate (E1h |0y E N..)).
(-1
where in denotes the representation of node v at layer 1.
Knowledge Distillation Traditional knowledge distillation methods employ the Kullback-Leible (KL) Divergence loss to align the output distributions of student and teacher models:
Σ (i) log p (1)
Lx ,p²)
(2)
(1)1)*
where p" and p denote the class probability distributions predicted by the teacher and student model respectively. These are computed via softmax function and distillation temperature т:
h
p
p
Model Heterogeneous Federated Graph Learning framework. Let S denotes the central server and C denotes the k-th client with K clients total. Each client holds its own graph g = (V* 8*). In a model heterogeneous setting, each client trains model w parameterized training data and then uploads it to the server, and the client models {w" w² ..., w } do not share identical architectures. Mathematically, the learning objective can be formulated as:
8 on its own
|
min
(", w² (g")),
(4)
M
where |D*] and |V] represents the number of samples of the client k and the total number of samples across all clients respectively, y denotes ground truth labels, and L(.,.) is the empirical loss.
2.2 Model Heterogencous FGL with proxy model
we leverage proxy models to bridge heterogeneous models. our framework, each client maintains both its private model and proxy model. The proxy model is small model with an identical architecture across all clients. The training process consists of three key phases: (1) Forward Distillation: First, each client distills knowledge from its private model to its
Inspired by FedType 49
3

---

Dis. Di(s) D.is)
mott
model classifir
Difficito Mensure
Curicaton Schedater
Reluionsh@rr
P
CR WW
00
7
OOOO00OO
Score mode dificcllt
LAND
Graanally inpaalice difficsle
Pruny login
Progressive Curriculum Node Schedtuler
Epocc
Adaptive Curriculum Distillation Modulator
Private mndeet lngits
lover tomppeattire
Cross-class comparision
Wasserstein Driven Affinity Distillation
Complex gouph with small musil capirity
Facus category inforaution
LKLL
LwD lex a- Lorn + 8- Iea.
Heterngmernce privaie moddl
Forward KD
Overall Framework
Prexy mode!
incress Mespeenature
Rackword KD
Simple with 9ums Buee high cupacity
Mus-E Fl
Copture graned
6 Global model
information
Figure 2: Architecture overview of TRUST, which includes three core components: (1) PCNS is liffiuultyppooreessive progressive curriculum learning module. (2) ACDM provides capability- aware dynamic temperature adjustment. (3) WDAD employs Wasserstein- based cross- cross-class distillation.
proxy model. (2) Global Aggregation: Then the weights of all proxy models transmitted to the server where they are aggregated through weighted averaging to form a global model. (3) Backward Distillation: Next, the updated global model is distributed back to proxy models, which conducts knowledge distillation to transfer global knowledge to private models using a conformal model. The details of backward distillation are given in Section Elin Appendíx. This eyclic process continues until the global model converges. However, FedType is originally tailored for Euclidean data types as it does not fully exploit unique topological information of graph data. In order to generalize to graph data, our approach specifically addresses the three key challenges identified in Section I
3 Methodology
3.1 Framework Overview
In this section, present overview of TRUST. TRUST adds three key components to the knowledge distillation process: (1) At the client side, during knowledge transfer from private to proxy models, we propose PCNS gradually introduce challenging nodes proxy models. (2) At the same time, distillation temperature automatically calibrates throughout the process to adjust to both client model capabilities and local graph complexity. (3) After forward propagation,
introduce WDAD loss, which incorporates cross-class comparison through optimal transport theory, preserving topological information that conventional distillation methods typically ignore. These components work synergistically to address the unique challenges of model-heterogeneousr federated learning on graph data, while maintaining the privecy-preeervig benefits of the federated paradigm. The framework is illustrated in Figure 2
3.2 Progressive Curriculum Node Scheduler
Motivation. As established in Sec. I graph-structures data contains complex topological relation- ships. Therefore, premature exposure to this intricate high-order information during initial training phases results in client models struggling to learn generalizable patterns from highly complex nodes, and nodes situated near class boundaries with ambiguous class representations may even lead to misleading learning signals. Therefore, we need to design a progressive learning framework to introduce complex graph knowledge step by step.

---

Curriculum Learning, Drawing inspiration from human cognitive development, Curriculum Learn- ing (CL) enables models learn from data in a structured manner, transitioning from simpler to more complex samples, rather than processing all data uniformly in each epoch [2]. Prior works have proved that CL's effectiveness in improving model convergence, generalization, and final performance
[23]. To implement this approach, we need to (1) make a formal definition of sample "difficulty" and (2) schedule data samples based on the proposed definition.
Difficulty Measurer. In GNN, each layer aggregates neighboring nodes to update node representa- tions, as formalized in Equation I Intuitively, GNN models are more proficient at learning nodes whose neighbors belong to the same class because they share certain features. By contrast, for cross-class nodes connected to neighbors with divergent labels, GNN models receive conflicting gradient updates during aggregation, resulting in impaired learning. Therefore, we can quantify node learning difficulty through the neighborhood label distribution, where for node vs the node difficulty is formally defined as the entropy of the distribution:
Pa(vs) [y., eln E U.U (c:))
[m
(5)
(1(
Σ Pc(t)) log(P.(v))).
CEC
where Pc(#:) computes proportion of class e in neighborhood Mt U {vi), Yn denotes the label of node n, and C' represents the set of labels.
Notably, the neighbors of node e; may include samples from both the training and test datasets, and theoretically, the labels of test dataset neighbors are unavailable during the training phase, which means we need a locally pretrained GNN to generate pseudo-labels for unlabeled nodes. However, this approach introduces a fundamental challenge in decentralized FGL settings: locally pretrained GNN models may be unreliable due to the lack of global knowledge, potentially resulting in pseudo- label inaccuracies. Therefore, our difficulty measurer must exhibit robustness potential label noise. Intuitively, we can implement this by evaluating the alignment between node representations and class prototypes. For a node v; with pseudo-label ye. if its node representation h; demonstrates high similarity with prototype of class y/. it is less likely to be assigned a wrong label because its features exhibit strong class-typical characteristics. But if it's the opposite, we can identify it as a "difficult" node because it has ambiguous node representation and the pretrained model has low confidence in the predicted pseudo-label. We can summarize this difficulty measurer as:
Pv:
hes
VV
VEVy:
(6)
exp(h, Pv:)
D2(1:) =
max.cc exp(h, p.)
where Vy, is subset of V with all nodes belonging to class yi in it, and py. denotes the prototypes of
class yi (mean embedding of nodes in V.,)) Combining these two difficulty measurer, the overall node difficulty score can be formalized as follows:
D(v)) = (v:) + a D2(v:),
where a balances the weight of D2(e)).
Curriculum scheduler. Once data samples are sorted by node difficulty in ascending order, each client implements a curriculum scheduler to gradually expose proxy model to more complex samples. The scheduler regulates the proportion of sorted training data used at each epoch through pacing function. For simplicity, we adopt a linear function:
t
(8) where À denotes the available proportion of data samples at epoch 0, and 7' is the epoch when full training data is first utilized. f(t) monotonically increases from À to I over T' epochs. Notably, after ft) reaches 1, the model should still continue training for several additional epochs to ensure complete assimilation of challenging knowledge patterns.
f(t) min(1, A + (1- A) -)-
T
Incorporating the proposed curriculum scheduler, we can reformulate the KL divergence loss as:
ix(lp" .p''')))==))))))
B
1
Σ
Σ
ps
(1) log
D (1)
2 (i)
(9)
1=1
5

---

where, for a given client k, the nodes are pre-sorted as U1, U...... Up in ascending order of their difficulty (())). At epoch t, the number of nodes selected from the beginning of this sorted list is B: = -f((t) [ []. In Equation 9 B corresponds to this B3 at epoch t.
3.3 Adaptive Curriculum Distillation Modulator
Motivation. In model-heterogeneeee FGL, each client adjusts local model architecture based on its available computational resources, and each client operates subgraph with varying complexity. Fixed temperature fails to account for differences in client model capabilities and topological knowl- edge complexity, limiting the ability of proxy models learn intricate graph structures. To address this issue, we need to propose an ACDM that dynamically calibrates temperature.
Adaptive Temperature Modulator. To dynamically adjust distillation signals during training, intuitively if the current task difficulty is too low for a client model, increasing the difficulty can fully exploit model capability. Conversely, if the task exceeds the model's current capacity, reducing the difficulty prevents ineffective training, which resembles an adversarial process. Inspired by Generative Adversarial Networks (GANs) [25 g implement this mechanism by converting the constant temperature value into a learnable parameter Oremp. which is optimized in the opposite direction of the client model parameters. In this way, Ozemp controls the difficulty of the loss minimization process, thereby enabling indirect dynamic adjustment of the distillation difficulty. Mathematically, the learning objective can be formulated as:
aL mmddel 7 @0
aL
mmodel
Oummp = 0emp + 7) 30
(10)
where mmodal denotes the client model parameters (the proxy model more precisely in this framework), Oramp is the temperature parameter, and 7 denotes the learning rate.
Curriculum Distillation Modulator. While the Adaptive Temperature Modulator provides dynamic adjustment, we decide to prevent excessive interference the early training stage since the model is still initializing and has limited learning capacity. Therefore, we incorporate curriculum learning to progressively increase the influence of the modulator. Specifically, we scale the loss T. by a factor Jr:
0(p
Premp = Gxemp +
(11)
00
Here, μ is determined by the pacing function, which smoothly increases from to I over the training epochs. We implement the pacing function as a cosine scheduler to progress more smoothly:
cos( min(t.
(12)
2
where denotes the current epoch.
Module Pipeline. Having established the optimization objective of ACDM, we now formalize its overall pipeline. The module can be conceptualized as a network layer denoted as hemp parameterized by ftemp. Each training epoch executes the following steps: (1) we first compute the scaling factor using Equation 12] (2) During forward propagation, lemp takes as input and outputs Peemp to compute the distillation temperature T for current epoch. (3) In backward propagation, 4emp updates Otemp via gradient descent using Equation IT] This cyclic process continues throughout the entire training phase. Notably, rather than directly use Aremp as the distillation temperature т, we constrain it to a reasonable range via sigmoid function. Therefore, KL-divergence loss is reformulated as:
B
log 7 (
tRa(p" p
ΣΣ
B
(1)
he
(13)
T
Tixin + Tiaax sigmoid connoidd@rcmaa (temp);
where Twan and Tmaa are the upper bound and lower bound for distillation temperature.
3.4 Wasserstein-Driven Affinity Distillation
Motivation. As shown in Equation 2 Kl.-divergence only compares intra-class probability distri- butions. For graph data with complex topoloiiaal relationships, this approach may fail to capture

---

Table 1: Comparison with the ftate-of-the-ars methods on five real-world datasets under moderate heterogeneity. For each dataset, we report local and global accuracy(%) (with red/green markers indicating regression/improvement over FedAvg). The best and recond-best results marked with bold and underline, respectively. Additional results under more settings are in Appendix D
CiteSeer
PubMaol
gpobal
(ASTXT11)
05,45
88.10
82.09
83,85
$4.10
FoNNENA (NertPS31)
Nt. M0.40ou
64.97
06, 22
82.87
90.33
82-37
10.34
FL.
(AAAEE)
75,
64.70
52.61 co
67. g4.30
85, 10p
83.46y)
91.8
50.51
66,8
64.04,0om
JCVPE11)
81 52
05 70
$1.58
00..0 03.51:
88 15
82.30
91.
53.51
80. 10.00
Ralleye (KMML24
82.23
22.00
58.20
64.24.1
87.09 0k
84
91.64:
20.30
88, 14.00..00
AdRFE (CCE22)
$1,0300 o7
64.0 M6.0laarr
50 00.40.1
66 48
65,85
82 87.8 Pa
91.1 91:31
84 26
00, 00.440o.o oo
82.54 (2.54.)
PGL
PodGTA (VLD824)
27.73
64.48,0.1
79.00...05
00.38
85.02
82.077o.c
01.80,..75
63.89-0.3
Moldl Molch-Herngsooen
POL
THUST
28.90-2.64 64 75 82,
77.48/..v 1o
00.00
84.87-9.49
02.10-2..0
87.07 79
91.42/1.00 om
subtle but important cross-class structural dependencies. To address this limitation, we propose a cross-class comparison mechanism.
Cross-class Relationships. We quantify cross-class relationships (CR) by computing cosine similarity between prototypes of different categories. Following Equation 6 prototypes can be derived from the mean node embeddings for each category. Drawing inspiration from (37], for implementation efficiency we decide to instead adopt the classifier weight vectors of the private model to compute class prototypes. For categories with certain connections, their classifier weight vectors also share certain similarity. Let W E Rexn denotes the classifier weight matrix, where e is the number of classes and n the feature dimensions. The CR metric is computed as:
ub; |oo,112" where w; represents the weight vector for class in W. wi; denotes the Lg-normalized weight vector of w., and CR captures the pairwise similarity between all classes.
W;
CR = WWT.
(14)
Wasserstein-Driven Affinity Distillation Loss. To quantify the probability distribution difference between p² and y², based on Wasserstein distance 44, we define WDAD loss
Lwo(p", T. .p = min) Σ Cugliy + q.y log qu)
(15)
where Qi and Cty represents the transmitted probability mass and transmission cost from teacher (private model in forward distillation) category to student (proxy model) category j respectively, and 17 is regularization hyper-parameter parameter. Ziş is constrained by:
Σ Σa==na = Σ 1u = pj 45 2 0.
(16)
Intuitively, if two categories similar in the feature space, the transmission cost Cj should be lower. Therefore, we can leverage CR to compute &j using a Gaussian kernel:
Gy = exp(-x(1- CR.,)),
(17)
where & is hhper--paaee parameter adjusting the sensitivity to CR. Overall, WDAD loss explicitly intro- duces cross-class relationships through transmission cost Cty- By minimizing this loss, we enforce consistency between private and proxy models in their probability allocation, particularly for cate- gories that are similar in the feature space.
Moreover, rather than completely replace İxL with LWD. decide to employ weighted combina- tion of LwD and f.g. to ensure smooth transition between the two objectives. Therefore, our final optimization objective can be formulated as:
L = Lex + LWD + 3- LRI.,
(18)
where TAr denotes the cross-entropy function for classification tasks, and a and /3 are two hyper- parameters regulating the weight of LWD and LRL. respectively.
4 Experiment
In this section, we comprehensively evaluate TRUST through four axes: Q1 (Superiority), Q2 (Resilience). Q3 (Effectiveness)), Q4 (Sensitivity).
7

---

4.1 Experimental Setup
Datasets. To effectively evaluate the performance of our approach, we employed five benchmark
graph datasets of various scales and distributions. including Cora [31], CiteSeer [7], PubMed [38..
CS, and Photo. Detailed descriptions and splits for these datasets can be found in Appendix
Moreover, the implementation details and parameter settings can be found in Appendix C.3
Counterparts. We compare TRUST against several traditional FL methods: (1) FedAvg [ASTAT17]
[32]. (2) FedNOVA [NeurIPS20] [47], (3) FedProto AAA122] [3), (4) MOON [CVPR 21 (6))
FedType [ICML24] 49];
; two popular FGL approaches: (6) A [ICDE24] 28 (7) FedGTA
[VLDB24] (29 Detailed descriptions can be found in Appendix C.2
Cera
Cera
CLO
Cora
17.
10.2
lccll
Bathere
34,77)
62.1
gltal
Figure 3: We report the performance of different methods under varying data heterogeneity levels on Cora, CiteSeer, and PubMed, with both local and global accuracy. The red color denotes the performance of TRUST. (First): severe heterogeneittya=0.3).. (Second): mild heterogeneity(c=11. (Third): Results Cora under different heterogeneity levels with a set to 0.3, 0.5, and 1.0.
4.2 Superiority
To answer Q1, we conducted systematic experimental evaluations in a variety of subgraph data heterogeneity environments, which we control by partitioning the graph using Dirichlet distribution with parameter a. Smaller values of lead to more extreme subgraph distributions. We compared the performance of TRUST with existing approaches based on two metrics: local accuracy and global accuracy. Local accuracy measured as the average classification accuracy across all clients, reflecting the model's effectiveness on decentralized subsets. Global accuracy is evaluated on the aggregated model at the server. For most existing methods, the global model is updated via knowledge distillation. In contrast, both TRUST and FedType update the global model by simply averaging the parameters of the proxy models. The results are summarized Tab. and Figure
From the table, several key observations can be made (Obs.): Obs. OExisting approaches show suboptimal performance in heterogeneous FGL scenarios. For instance, when a is set 0.3, most existing methods achieve accuracy on the Cora dataset that is either lower than or comparable to FedAvg. Notably, when a is reduced to 0.1, the performance of these methods drops significantly, with the average global accuracy consistently falling below 35%.
Obs. @ TRUST demonstrates remarkable robustness across various graph data heterogeneity scales. Under moderate heterogeneity conditions (a = 0.5), TRUST exhibits clear advantages. As shown in Tab. 1. TRUST consistently outperforms both FL. and FGL baseline methods across different datasets in terms of both local and global accuracy. For example, TRUST achieves a global accuracy of 75.32% on the Cora dataset, surpassing the best baseline method, FedType (72.96% by 2.36%, and outperforming FedAvg (64.52% by significant 10.80% margin. Furthermore, as shown in Figure 3 TRUST consistently outperforms all baselines across various heterogeneity levels. In highly heterogeneous environments, TRUST achieves varying degrees of performance improvement over all baselines In mildly heterogeneous settings, demonstrates an average accuracy gain of 2.25%.
4.3 Resilience
To address Q2, we evaluate the performance of each method on the Cora dataset across varying levels of data heterogeneity, where is set to 0.3, 0.5, and F.Qure Third) illustrates that TRUST
8

---

203
[ex
03
ww0.---000 HBVD
Mm
VELO
(a) Cora Local
(b) Cora Global
(c) Citeseer Local
(d) Citeseer Global
Figure 4: Ablation Study of the key components SCNS,ACDIS and WDAD on Cora and Citeseer
datasets. For an in-depth analysis, please refer to Sec. 4.4
achieves robust performance gains across varying levels of data heterogeneity, outperforming other algorithms by average of 5.78%, and at 0.5 on the Cora dataset, it even surpasses FedAvg by 10.80%. This demonstrates that TRUST can effectively identify heterogeneous graphs and maintain superior performance, under challenging conditions with extreme data heterogeneity.
4.4 Effectiveness
To address Q3, conduct ablation study the key components of the framework: ACDM, and WDAD. Experimental results on Cora and CiteSeer are presented in Figure
PCNS,
The ablation study demonstrates that all three components contribute substantially to performance improvement, PCNS has the most pronounced individual effect; its removal results 2.9% drop in accuracy (Citeseer Global: 67.43% - 64.51 1)), confirming its effectiveness in progressive node scheduling. ACDM provides cataset-depende benefits, yielding a 1.93% gain on Citeseer Global through dynamic temperature modulation. WDAD consistently contributes 0.8-1.6% improvements by enabling cross-class knowledge transfer.
When PCNS, ACDM, and WDAD are combined, the model achieves the best performance, effectively distilling both structural and semantic knowledge into gellreenerllizable student model.
4.5 Sensitivity
To address Q4, we conduct analyses on hyperparameters of TRUST. Specifically, we analyze the model's performance under different values of À and T, as defined in Equation 8 We evaluate all combinations of À {0.25, 0.5, 0.75) and T {20, 40, 80, 100). Results shown in Figure demonstrate that the choice of hyperparameters À and T has a minimal impact on the performance of TRUST, proving the robustness of TRUST.
85.0
75.0
85.0
82
72.
82.0
68.0
78.0
78.
66.0
73.0
62.0
70.0
63.0
70.0
60.0
100
60
80
0.75
0.25
0.75
0.25
0.75
0.25
0.5
20
40
(a) Cora local
(b) Cora global
(c) CiteSeer local
(d) CiteSeer global
Figure 5: Analysis on hyper-parameter in TRUST.Nodk classification results are evaluated on Cora and CiteSeer datasets under various hyperparameter combinations, testing both global and local accuracy. All experiments are conducted under the setting of a = 0.5.
Conclusion
In this paper, we propose TRUST address model heterogeneity Federated Graph Learning. Based on knowledge distillation to bridge heterogenous client models, we integrates three key strategies to effectively handle complex topological information in graph-structured data. first employ PCNS to progressively introduce complex samples based on learning difficulty. Then we propose ACDM for dynamic temperature adjustment. We further propose WDAD that captures cross-class structural relationships. Comprehensive experiments across five benchmark datasets demonstrate TRUST's

---

superior capability in resolving model heterogeneity challenges while preserving graph topological properties. The framework establishes sfute-of-the-ar for heterogeneous FGL systems.
Acknowledgement
This work is supported by National Natural Science Foundation of China under Grant (62361 62225113, 623B2080), the Major Project of Science and Technology Innovation of Hubei Province (2024BCA003, 2025BFA002), and the Innovative Research Group Project of Hubei Province under Grants 2024AFA017. The supercomputing system at the Supercomputing Center of Wuhan University supported the numerical calculations in this paper.
References
[1] A. Angelopoulos, S. Bates, J. Malik, and M. I. Jordan. Uncertainty sets for image classifiers using conformal prediction, 2022.
[2] Y. Bengio, J. Louradour, R. Collobert, and J. Weston. Curriculum learning. In Proceedings of the 26th Annual International Conference Machine Learning, ICML. '09, page 41-48, New York, NY, USA, 2009. Association for Computing Machinery.
[3] Cai, Zhang. J. Fan, and S.-K. Ng. Lg-fgad: An effective federated graph anomaly detection framework. In Proceedings of the International Joint Conference Artificial Intelligence (IJCAI), pages 3760-3769, 2024.
[4] M. Cuturi. Sinkhorn distances: Lightspeed computation of optimal transportation distances, 2013.
[5] X. Fu, Z. Chen, Y. He, S. Wang, B. Zhang, C. Chen, and J. Li. Virtual nodes can help: Tackling distribution shifts in federated graph learning. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025.
[6] X. Fu, B. Zhang, Y. Dong, C. Chen, and J. Li. Federated graph machine learning: A survey of concepts, techniques, and applications. arXiv preprint arXiv:2207.118. 2022.
[7] C. L. Giles, K. D. Bollacker, and S. Lawrence. Citeseer: An automatic citation indexing system. In Proceedings of the third ACM conference Digital libraries, pages 89-98, 1998.
[8] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial networks, 2014.
[9] J. Gou, B. Yu, S. J. Maybank, and D. Tao. Knowledge distillation: A survey. IJCV, pages 1789-1819, 2021.
[10] Q Guo, X. Wang, Y. Wu, Z. Yu, Liang, X. Hu, and Luo. Online knowledge distillation via collaborative learning. In CVPR, pages 11020-1 1029, 2020.
[11] W. Hamilton, Z. Ying, and J. Leskovec. Inductive representation learning on large graphs, In NeurIPS, 2017.
[12] C. He, K. Balasubramanian, E. Ceyani, Yang, H. Xie, L. Sun, L. He, L. Yang. S. Yu, Y. Rong, P. Zhao, J. Huang, M. Annavaram, and S. Avestimehr. Fedgraphnn: A federated learning system and benchmark for graph neural networks, 2021.
[13] G. Hinton, O. Vinyals, and Dean. Distilling the knowledge neural network. arXiv preprint arXiv: 1503.0253), 2015.
[14] G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network, 2015.
[15] C.-Y. Huang, K. Srinivas, X. Zhang, and X. Li. Overcoming data and model heterogeneities in decentralized federated learning via synthetic anchors, 2025.
[16] W. Huang, M. Ye, and B. Du. Learn from others and be yourself in heterogeneous federated learning. In CVPR, 2022.
10

---

[17] W. Huang, M. Ye, and B. Du. Learn from others and be yourself in heterogeneous federated learning. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10133-1014; 2022.
[18] W. Huang, M. Ye, Z. Shi, and B. Du. Generalizable heterogeneous federated cross-correlation and instance similarity learning. TPAMI, 2023.
[19] W. Huang, M. Ye, Z. Shi, H. Li, and B. Du. Rethinking federated learning with domain shift: A prototype view. In CVPR, 2023.
[20] W. Huang, M. Ye, Z. Shi, G. Wan, H. Li, B. Du, and Q. Yang. A federated learning for generalization, robustness, fairness: A survey and benchmark. arXiv, 2023.
[21] T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
[22] S. Kotz, N. Balakrishnan, and N. L. Johnson. Continuous multivariate distributions, Volume 1: Models and applicatioons volume 1. John Wiley & Sons, 2004.
[23] H. Li, X. Wang, and W. Zhu. Curriculum graph machine learning: A survey, 2024.
[24] M. Li, X. Zhang, Q Wang, T. LIU, R. Wu, W. Wang, F. Zhuang, H. Xiong, and D. Yu. Resource- aware federated self-super learning with global class representations. In The Thirty-eighth Annual Conference
Neural Information Processing Systems, 2024.
[25] Q. Li, B. He, and D. Song, Adversarial collaborative learning on non-iid features. arXiv, 2021.
[26] Q. Li, B. Hc, and D. Song. Model-contrastive federated learning. In CVPR, pages 10713-10722 2021.
[27] X. Li, K. Huang, W. Yang. S. Wang, and Z. Zhang. On the convergence of fedavg non-iid data. arXiv preprint arXiv:1907.022 2019.
[28] X. Li, Z. Wu, W. Zhang, H. Sun, R.-H. Li, and G. Wang. Adafgl: A new paradigm for federated node classification with topology heterogeneity. In 2024 IEFE 40th International Conference on Data Engineering (ICDE), pages 2379-2392. IEEE, 2024.
[29] X. Li, Z. Wu, W. Zhang, Y. Zhu, R.-H. Li, and G. Wang. Fedgta: Topology-aware averaging for federated graph learning. Proceedings of the VLDB Endowment, 17(1):11-50, 2023.
[30] Z. Liu, G. Wan, B. A. Prakash, M. S. Lau, and W. Jin. A review of graph neural networks epidemic modeling. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Datu Mining, pages 6577-6587, 2024.
[31] A. K. McCallum, K. Nigam, J. Rennie, and K. Seymore. Automating the construction of internet portals with machine learning. Information Retrieval, 3(2):127-163, 2000.
[32] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas. Communication-effcienn learning of deep networks from decentralized data. In AISTATS, pages 1273-1282, 2017.
[33] S. Mehta and A. Aneja. Securing data privacy in machine learning: The fedavg of federated learning approach. In 2024 4th Asian Conference on Innovation in Technology (ASIANCON), pages 1-5. IEEE, 2024.
[34] M. Morafah, V. Kungurtsev, H. Chang, C. Chen, and B. Lin. Towards diverse device heteroge- neous federated learning via task arithmetic knowledge integration, 2024.
[35] C. Pan, J. Xu, Y. Yu, Z. Yang, Q. Wu, C. Wang, L. Chen, and Y. Yang. Towards fair graph federated learning via incentive mechanisms, 2023.
[36] H. Papadopoulos, K. Proedrou, V. Vovk, and A. Gammerman. Inductive confidence machines for regression. In Machine Learning: ECML 2002: 13th European Conference Machine Learning Helsinki, Finland, August 19-23, 2002 Proceedings 13, pages 345-356. Springer, 2002.

---

[37] V. Papyan, X. Y. Han, and D. L. Donoho. Prevalence of neural collapse during the ter- minal phase of deep learning training. Proceedings of the National Academy of Sciences. 11 117/40):24652-2466) Sept. 2020.
[38] Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. Eliassi-Rad Collective classifica- tion in network data. Al magazine, 29(3):93 93, 2008.
[39] Y. Tan, G. Long, L. Liu, T. Zhou, Q. Lu, J. Jiang, and C. Zhang. Fedproto: Federated prototype learning across heterogeneous clients. In AAAI, 2022.
[40] Z. Tan, G. Wan, W. Huang, and M. Ye. Fedssp: Federated graph learning with spectral knowl- edge and personalized preference. In Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[41] G. Wan, W. Huang, and M. Ye. Federated graph learning under domain shift with generalizable prototypes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 15429-15437, 2024.
[42] G. Wan, Z. Huang, W. Zhao, X. Luo, Y. Sun, and W. Wang. Rethink graphode generalization within coupled dynamical system. In Forty-second International Conference on Machine Learning, 2025.
[43] G. Wan, Z. Liu, Shan, M. S. Lau, A. Prakash, and W. Jin. Epidemiology neural ode with continuous disease transmission graph. In Forty-second International Conference Machine Learning, 2025.
[44] G. Wan, Z. Shi, W. Huang, G. Zhang, D. Tao, and M. Ye. Energy-based backdoor defense against federated graph learning. In International Conference on Learning Representations. 2025.
[45] G. Wan, Y. Tian, W. Huang, N. V. Chawla, and M. Ye. S3gel: Spectral, swift, spatial graph contrastive learning. In Forty-first International Conference on Machine Learning, 2024.
[46] J. Wang, Q. Li, L. Lyu, and F. Ma. pfedclub: Controllable heterogeneous model aggregation for personalized federated learning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[47] Wang, Q. Liu, H. Liang, G. Joshi, and H. V. Poor. Tackling the objective inconsistency problem in heterogeneous federated optimization. In NeurlPS, pages 7611-7623, 2020.
[48] Wang, X. Yang, S. Cui, L. Che, Lyu, D. Xu, and F. Ma. Towards personalized federated learning via heterogeneous model reassembly, 2023.
[49] Wang, C. Zhao, L. Lyu, Q. You, M. Huai, and Ma. Bridging model heterogeneity in federated learning via uncertainty-basee asymmetrical reciprocity learning, 2024.
[50] L. Wang and K.-J. Yoon. Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks. IEEE TPAMI, 2021.
[51] X. Wang, Y. Chen, and W. Zhu. A survey on curriculum learning, 2021.
[52] C. Wu, F. Wu, L. Lyu, T. Qi, Y. Huang, and X. Xie. A federated graph neural network framework for privacy-preservee personalization. Nature Communications, 13(1), June 2022.
[53] H. Xie, Ma. L. Xiong, and C. Yang. Federated graph classification over non-iid graphs. NeurlPS, 34:1844 1889-1882,, 2021.
[54] M. Ye, X. Fang, B. Du, P. C. Yuen, and Tao. Heterogeneous federated learning: State-of-the- art and research challenges, 2023.
[55] I.. Yi, H. Yu, C. Ren, G. Wang, Liu, and X. Li. Federated model heterogeneous matryoshka representation learning, 2024.
12

---

[56] J. Zhang, Y. Liu, Y. Hua, and J. Cao. Fedtgp: Trainable global prototypes with adaptive- margin- enhanced contrastive learning for data and model heterogeneity in federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, 2024.
[57] K. Zhang, C. Yang, X. Li, L. Sun, and S. M. Yiu. Subgraph federated learning with missing neighbor generation, 2021.
[58] B. Zhao, Q. Cui, R. Song, Qiu, and Liang. Decoupled knowledge distillation. CVPR, 2022.
13

---

A
Notations
We present a comprehensive review the commonly used notations and their definitions in Tab.
Table 2: Notation and Definitions
Notation
Definition
K
The number of clients.
D
The dataset of k-th client.
w
The model of k-th client. Graph data.
V
The node set of G. The edge set g. The label set of g.
C
X
The feature matrix of G.
A
The adjacency matrix of g. Node in V.
24
Feature vector of node i.
Label of node i.
N
The neighborhood nodes of ey.
The representation of v at the 1-th layer of GNN.
((e)
The node difficulty of V;-
V.
The node subset consisting of all nodes labeled as The prototype class c.
Pe
The available proportion of data samples epoch 0. The epoch when full training data first utilized.
The learning rate.
The scaling factor of Adaptive Temperature Modulator. The class probability distribution predicted by the teacher model.
p"
ps
The class probability distribution predicted by the student model. The distillation temperature.
CR
The pairwise similarity matrix between all classes.
Cij
The probability mass transferred from teacher category to student category The transmission cost from teacher category to student category j-
Ji
The hyper-paramett adjusting the sensitivity to cross-class similarity in Gaussian kernel.
LcE
The cross-entropy function for classification tasks.
LuD
The proposed Wasserstein-Driven Affinity Distillation loss.
LAI.
The Kullback-Leiblrr (KL) Divergence loss.
B Related Work.
Federated Graph Learning. With recent advances in Federated Learning (FL) for vision and language tasks 20 18), Federated Graph Learning (FGL) further extends FL graph-structured data [41 (2). Existing FGL-related researches are mainly focused processing graph-structured data. Based on how graphs are distributed across clients, these methods can fall into three categories: graph-level, subgraph-level and node-level [12]. Graph-level FGL. methods consider settings where
clients possess completely disjoint graphs while in subgraph-Ievel FGL settings, each client holds a subgraph that is part of larger global graph [57 52). In node-level FGL, each agent possesses the ego-networks of one or multiple nodes 335
[2] However, these methods often assume homogeneous
model architectures cross clients [17
54 40), which is impractical constraint that degrades
performance in real-world scenarios. To tackle this problem, we propose model-hhetroogenouus method that transfers knowledge between clients in model-agnostic manner through novel knowledge distillation framework, overcoming limitations existing FGL approaches.
Model-heterogeneoue Federated Learning, Mooeel-heterogenoor federated learning entails learn- ing from others without disclosing information about local model architectures. Recent works on
federated learning can be categorized into three approaches: data-level,
14

---

model-level and server-level [54]. Data-level methods, such as TAKFL 34 and DESA [15] distill knowledge via external public data distributions. Model-Jevel methods, such as pFedHR 48 and pFedClub [46], share partial model structures or reassembled components to other clients. Server- level methods, such as FedMRI. [55] and FedTyyp [49], deploy proxy models as intermediaries to bridge heterogeneous models. While effective for Euclidean data, these methods fail to preserve graph topological features during knowledge transfer. Our work breaks this limitation by uniquely integrating curriculum learning and Wasserstein distance to capture complex structural information, improving graph representation in mode--heetrrogeneous FGL.
Knowledge Distillation. Knowledge Distillation (KD) is machine learning method designed for model compression and knowledge transfer [13 [4]. Traditional KD transfers knowledge from large teacher model to compact student model through softened outputs or intermediate representations [3]. Recently, researches have demonstrated its effectiveness facilitating model collaboration in federated learning (FL.,, particularly in scenarios involving heterogeneous models or data distributions (54) For instance, TAKFL
introduces KD FL framework distill knowledge from heterogeneous clients and then integrate the separately distilled knowledge with task arithmetic. Similarly, FedMKD (24] combines KD and attention mechanisms to work with model heterogeneity in FL. However, existing KD methods typically rely KL-Divergens minimization, which only performs intra-category comparisons between teacher and student models. By contrast, our work proposes noval Wasserstein knowledge distillation framework which introduces Wasserstein- distance to enable cross-class comparison, explicitly modeling graph-structure relationships during knowledge transfer and maintaining topological properties of the original graph data.
C Experimental Details.
C.1 Dataset Details.
To assess the effectiveness of TRUST, we conduct experiments on five real-world graph datasets: Cora, CiteSeer, PubMed, Amazon-Photo and CoAuthor-CS. Each dataset is split into training, validation, and test sets in a fixed 20%%440%/40% ratio. The key statistics of these datasets are summarized in Tab. A detailed description is provided below:
Cora, CiteSeer, and PubMed. These three citation network datasets are standard benchmarks in graph-based machine learning, especially for tasks like node classification and link prediction. In these datasets, nodes correspond to academic papers, while edges represent citation links. Each node is assigned class label, and its feature vector is constructed from textual information such as words in the title or abstract. These datasets exhibit sparsity and high dimensionality, making them well-suited for evaluating the effectiveness and scalability of graph neural networks (GNNs).
CoAuthor-CS. This dataset represents a co-authorship network in the field of computer science, where nodes correspond to research papers, and edges denote co-authorship relations. Each paper is associated with a topic category, and features are extracted from the paper's ' title and abstract.
This dataset is commonly used to evaluate node classification and community detection algorithms. Amazon-Photo This dataset is built from the Amazon product catalog, where nodes represent product images and edges indicate co-purchase relationships. Each photo is categorized into specific class, and node features are derived from image metadata. Amazon- Photo serves as a benchmark for testing graph-based learning models in visual domains.
Table 3: Statistics datasets used in experiments.
Dataset
#Nodes
#lidges
#Classes
#Features
Cora
2,708
5,278
7
1,433
Citeseer
3,327
4,552
3,703
Pubmed
19,717
44,324
3
500
Coauthor-CSS
18,333
327.576
15
6,805
Amz-Photo
7,650
287.326
8
745
C.2 Counterpart Details.
This section provides comprehensive overview of the bascline approaches employed in study.
15

---

FedAvg [ASTATI7]. A foundational algorithm in Federated Learning, FedAvg operates by allowing clients independently train models on their local datasets and subsequently transmit their model updates to a central server. The server performs a weighted aggregation of these updates to refine the global model, which is then redistributed to the clients for further local training. By transmitting only model parameters instead of raw data, FedAvg reduces communication costs and enhances privacy. However, it struggles with performance degradation in scenarios where client data distributions are highly non-IID [27 33
FedNova [NeurIPS20]. FedNova refines the FedAvg framework by introducing normalization to local updates before aggregation. Unlike standard averaging methods, FedNova ensures that each client's 's contribution to the global model is proportional to the amount of data it possesses. This approach addresses the issue of unequal client influence, leading to more balanced and efficient convergence. FedNova is particularly beneficial in federated environments where data distributions are skewed across clients.
introduces prototype-based federated learning to address both data and model heterogeneity across clients. Unlike pradient-based approaches, the framework exchanges class prototypes (mean feature representations) between server and clients, enabling knowledge transfer while accommodating different model architectures and non-IID data dis- tributions. Through prototype aggregation and local regularization, FedProto achieves superior communication efficiency and convergence guarantees while preserving privacy [39], The method demonstrates strong performance on image datasets while requiring significantly fewer communi- cated parameters than traditional FL approaches.
FedProto (AAA122).
Moon [EVPR21].MOON adopts model-contrastive approach to address data heterogeneity in federated learning. The framework utilizes similarities between model representations to correct local training through model-level contrastive learning, providing an effective solution for collaborative training with deep learning models on image datasets while preserving data privacy. FedType [ICML24). FedType[49] introduces novel uncertainty-based asymmetrical reciprocity learning framework to address model heterogeneity in federated learning. The approach employs small identical proxy models as secure intermediaries for information exchange, eliminating the need for public data while ensuring privacy protection. Through bidirectional knowledge distillation with dynamic conformal prediction, FedType achieves superior performance across diverse model architectures and datasets, demonstrating significant improvements in communication efficiency and model security compared to existing methods.
introduces a novel paradigm for federated node classification with topology heterogeneity, addressing the critical challenge of structural divergence among clients in federated graph learning. The framework employs a decoupled two-step approach: first obtaining a federated knowledge extractor through collaborative training, then performing personalized propagation optimized by local topology. By incorporating adaptive mechanisms that automatically balance homophilous and heterophilous propagation based on quantified structural characteristics, AdaFGL achieves saute-of-the-al performance across 12 benchmark datasets while minimizing communication overhead and privacy risks (28).
AdaFGL [ICDE24)
FedGTA [VLDB24], FedGTA is tailored for large-scale graph federated Icarning, tackling issues of slow convergence and suboptimal scalability. Unlike prior methods that focus on either optimization strategies or complex local models, FedGTA integrates topology-ao aware local smoothing with mixed neighbor feature aggregation improve learning efficiency [29]. By leveraging graph structures in aggregation, it enhances scalability and performance in federated graph learning.
C,3 Implementation Details.
The experiments are conducted on NVIDIA GeForce RTX 3090 GPUs, paired with dual Intel(R) Xcon(R) Gold 6240 CPUs @ 2.60GHz (36 cores per socket, Turbo Boost up to 3,90GHz). The deep learning framework used is PyTorch (v2.5.1) with CUDA 12.1.
The experimental setup involves 10 clients. To simulate real-worle model heterogeneity, each client maintains a private model whose architecture is randomly selected from GCN, GAT, or GraphS AGE. All private models are configured with three layers, a hidden dimension of 64, and a dropout rate of 0.3. To facilitate collaboration, each client is equipped with additional small proxy model that serves as a communication bridge. This proxy model employs a standardized GCN architecture with 3 layers to ensure compatibility across clients. On the server side, we implement global model
16

---

Table 4: Comparison with the state-of-the-art methods three selected real-world datasets. The alpha is set to 0.3 and 1.0. The best and second-best results are highlighted with bold and underline, respectively.
Canaery
Nethvdh
appa
dghe
kel
pholl
glokal
boaflg (ASUNET)
16,30
66 16
61:43
04.14
10.52
64.64
74.21
12.30
T1L07
0110
41.42
$4.08
INERONA (Se1
68.36
655 M4
24.22
76 18.77
72 72.05
73.77
11.11
6110
46
64.55ma
03 66g
90 25
$4.90
03 23
71
76, 19.33
To.
56 50.6
22
65
90
64.90
71
71.28
10 16.6..
90 1i
81 10p0
Felbax (KNL24
$6,23
80.72
09.37
16.20
8476
ICE
77.00
that also adopts GCN backbone and hidden dimension of 32 while sharing the remaining configurations with the client models.
To simulate data heterogeneity, client data is partitioned using Dirichlet distribution 22 drawing Pk ~ Dir(a) to allocate a fraction p of class c to client k. Each client's ss subgraph is split into training, respectively. We use the Adam optimizer with learning rate of 5 x 10-3 and a weight decay of 4 x 10 for training. The number of communication rounds is set to 200.
validation, and test sets with ratio of 0.6/0.2/0.
For PCNS, in the Difficulty Measurer, we set a 0.5. In the Curriculum Scheduler, the hyperparam- eters À and T are selected via grid search over (0.25, 0.5, 0.75) and (20, 40, 80, 100), respectively. For ACDM, the model parameters 0 are optimized using Adam (learning rate: 0.01, weight decay: 5 x 10-1), while the temperature parameters Oremp are optimized using SGD (momentum: 0.9, weight decay: 4 x 10-4). The conformal mode parameters in backward distillation follow the configuration in FedType. In WDAD, we set rj 0.05, 1.0, with loss weights a 0.025 (Wasserstein) and B = 0.01 (KL divergence).
Regarding baseline implementations, all selected FL baselines, except for FedAvg, support model heterogeneity and operate similarly to their applications in computer vision tasks. However, for FGL methods, the best of our knowledge. our work is the first to explore model heterogeneity in FGL settings. Consequently, the chosen FGL basclines are originally designed for model homogeneity only. To include them (along with FedAvg) heterogenous settings, we remove direct parameter sharing between architecturally different models and maintain all other original components and hyperparameters. For evaluation, we distill global model from local private models and assess its accuracy.
D Additional Experimental Results.
D.1
Comparison with More FGL Baselines.
To further validate the efficacy of our approach, we compare TRUST against two additional FGL baselines, FedTAD and FedSSL, on the Cora, Citeseer, Pubmed, and CS datasets. The corresponding results are presented in Tab.
Table 5: Local and gobal accuracy results of TRUST and two additional FGL baselines the Cora, Citeseer, Pubmed and CS datasets under moderate heterogeneity(a= = 0.5).
Cora
Citeseer
Pubmed
CS
Methods
local
global
local
global
local
global
local
global
FedTAD
80.27
65.70
83.24
66.37
87.88
83.30
90.72
82.93
FedSSL
79.53
64.79
82.32
68.15
88.07
83.58
91.97
80.73
TRUST
83.90
75.32
85.84
67.43
89.06
84.57
92.19
87.07
17

---

As shown in Tab. 5 TRUST achieves the best performance on 7 out of the 8 evaluation metrics. The only exception the global accuracy on the Citeseer dataset, where FedSSL attains the best result. These comprehensive results further confirm the effectiveness of TRUST against state-of-the-art FGL methods.
D.2 Comparison under Extreme Data Heterogeneity.
We provide additional local and global accuracy results under the a settings of 0.3 and 1.0 in Tab. 4 which is previously visualized in Figure 3 We also experiment more extreme level of data heterogeneity, where a 0.1. Results are presented in Tab.
Table 6: Comparison with state-of-the-art methods on three datasets under a 0.1. The best and second-best results are highlighted with bold and underline, respectively.
Cora
Citeseer
Pubmed
Methods
local
global
local
global
local
global
FedAvg
93.14
35.04
86.33
54.98
97.61
79.99
FedNOVA
93.51
36.50
86.47
56.17
97.66
79.92
FedProto
92.59
30.84
87.51
49.48
97.69
78.45
MOON
93.32
31.75
86.61
51.56
97.56
77.16
FedType
94.06
40.88
87.64
60.03
97.48
79.36
AdaFGL
85.44
36.13
88.22
56.91
92.88
79.69
FedGTA
85.43
35.40
88.79
56.17
92.86
80.02
TRUST 94.60
42.52
88.44
64.04
97.76
82.65
The results in Tab. 6 demonstrate that TRUST maintains superior performance even under extreme data heterogeneity, achieving the best performance on 5 out of the evaluation metrics, with only the local accuracy on the Citeseer dataset showing slightly lower performance. These findings strongly validate the robustness and effectiveness of our approach.
E
Backward Distillation in FedType.
Backward distillation leverages knowledge distillation transfer knowledge from proxy models to private models. However, since proxy models are smaller than private models, conventional distillation approaches may lead performance degradation. To address this, FedType introduces an Uncertainty-baser Behavior Imitation Learning method that selectively transfers high-confidence knowledge rather than complete logits. For every node t; we construct prediction set S, which guarantees inclusion of the true label with high probability (e.g., 95% confidence). To compute S,, we need to train a conformal model denoted as op using the validation dataset denoted as D' following Split Conformal Prediction [3, 1. After training the conformal model, FedType proposes a dynamic conformal prediction with Regularized Adaptive Prediction Sets (RAPS) to calculate the prediction set:
S = (vl p(()) - p:(v)) + o( A) (o:((v)) Kezg) ) S T),
p((y) Σ pl(y') (15000>20000
d X A' A ir <0, otherwise,
(19)
X
d(0)) = (V((()) ((())
where py() denotes the probability of class y for data sample v; predicted by conformal model, is a randomization factor for prediction set construction. Furthermore, p,(3) denotes the total probability mass of labels more probable than y. g( A) is piccewise calibration function where A A(p', D') A(p--1, 10g=-11"" D') represents the accuracy difference between epoch t and 1. Moreover, (v) denotes the label ranking of y based on predicted possiblity pi: Kreg is regularization hyper-parametter, and (-)+ is the positive part operator.
After constructing the prediction set, FedTTpe computes the knowledge transfer weight 7ps which plays crucial role in determining the amount of information transferred from the proxy model the
18

---

private model. This weight is inversely proportional to the size of the prediction set St When S; contains fewer labels, which indicates higher confidence the prediction, we assign larger γ7 to encourage the proxy model to transfer such confident knowledge to the private model. Conversely, larger prediction sets result in smaller transfer weights. Therefore, Y can be defined as:
|s, n C.//S, U Cil. S: n L.)/
if |f > [c
(20)
i < [cc)
where S; and C; are similarly computed using Equation 19 but with conformal model trained by proxy model and private model respectively. This formulation ensures adaptive knowledge transfer based on the proxy model's 's confidence level.
During backward distillation, we specifically enhance the probability alignment for labels within the prediction set through the following loss function:
exp(w((u=)l))
Liaekward
Σ
log
4f
1=1
DES,
(21)
Σ
exp(w((u))l))
Σ
exp(w((e))ll)
VES;
S'
where S, = - S and represents labels with low confidence in the prediction.
F
Complexity Analysis.
In this section, we present the complexity analysis of our proposed method TRUST. We begin with FedAvg baseline. Let T denote the communication rounds. On the client side, each client executes E epochs of local training per round with model parameter size d. When combined with GNN message passing, the computational complexity is (L- d) where L and | represent the number of GNN layers and edges respectively. On the server side, model aggregation across K clients yields O(K d) complexity. Therefore, the total complexity is: O(T (F. T. (8) d K kd))..
For FedType, knowledge distillation introduces an additional O(N C*) cost per epoch, where N and C' are the number of nodes and classes respectively. The total complexity therefore becomes: O (E (L [8| d N C*) K dd).
For TRUST, it extends the knowledge distillation framework with three key components:
PCNS first requires calculating node difficulties via neighborhood distribution entropy and proto- type alignment. For neighborhood distribution entropy. it iterates through all edges (O(E))). For prototype alignment, we pre-compute prototypes for all classes and calculate prototype similarity for every node (O(N + C)). Node sorting then adds O(N log N).
ACDM involves only lightweight operations: temperature scaling ((1) and cosine scheduling O(1) per epoch, as indicated in Equation 13
WDAD leverages Sinkhorn algorithm to solve the loss function quickly. This reduces optimal transport complexity from O(C!) to O(k C²), where k is the iteration count (in our experiments this is set to 10).
Notably, WDAD's Olk C²) complexity remains manageable in practice since C is bounded (e.g.. C=7 for Cora and C=3 for Pubmed), Therefore, the total complexity is: O (E (L- [8] N. C+ C²) + N log N + K dd).
As shown above, this represents only two additive terms compared to FedType: O(N log N) and O(T- E k- C2).
G Discussion on Limitations.
Although TRUST achieves significant success in addressing key challenges in model-heterogenee Federated Graph Learning (FGL))such as dynamic task difficulty adjustment, adaptive distillation signaling, and cross-class relational knowledge transfer-it still has some limitations. One notable challenge is the computational complexity involved dynamically adjusting task difficulty and distillation strength, which could become bottleneck in large-scale settings.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the contributions and scope of this paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
We discuss the limitations in Appendix G
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
20

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We fully disclose all the information needed to reproduce the main experi- mental results in this paper and our code. We are convinced that the obtained results can be reproduced.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
21

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
Code is accessible in this paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https
/ "pipsscc/puullc//uuu /ttpe//ggu..eeeoooouuuuumbibbi ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Details are included in Appendix Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Statistical significance of the experiments is considered and included Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
22

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide sufficient information on the computer resources.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:pppr./.aooss.cooooooo///aaa
Answer: [Yes]
Justification: The research conducted the paper strictly adheres to the NeurIPS Code of Ethics, ensuring that all aspects of the work are in compliance with the guidelines provided.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The research presented in this paper is foundational. It is not directly tied to any specific applications or deployments.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
23

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system Iearns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses such risks
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: This paper does not existing assets.
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the patasets.paperesiii paperswi thcode. com datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information not available online, the authors are encouraged to reach out to the asset's 's creators.
24

---

13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code//modll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation,
or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "769cSrAPS1", "reviewer": "Reviewer_hwGA", "summary": "This paper proposes TRUST, a knowledge-distillation-based framework for model-heterogeneous Federated Graph Learning (FGL). It introduces three key components: a Progressive Curriculum Node Scheduler to gradually increase node difficulty, an Adaptive Curriculum Distillation Modulator to dynamically adjust distillation temperature, and a Wasserstein-Driven Affinity Distillation loss to transfer cross-class relationships. Experiments show the effectiveness of the proposed method.", "questions": "1. Please refer to the weaknesses section.\n2. Some baselines compared in the experiments are originally not designed for heterogenous models. For these algorithms, how do clients communicate in model-heterogenous settings?\n3. The adversarial update rule (Eq. 10) is introduced without clear motivation. Why should temperature compete with model parameters?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. The problem illustration and framework overview are clear and intuitive, and the methodology is also explained with clarity, making the work easy to follow.\n2. The paper presents a multi-order knowledge transfer framework tailored for model-heterogeneous FGL. By leveraging progressive node scheduling, cross-class relational transfer and client-specific distillation temperature, TRUST effectively addresses limitations of topological knowledge learning in model-heterogeneous FGL.\n3. The experimental validation are comprehensive. Extensive experiments on 5 graph datasets confirm the superiority of the proposed method. The authors also presented a clear reporting of average performance improvements for each algorithm and dataset, which is useful for reviewing.\n\nWeaknesses:\n1. It may be beneficial to add more critical hyperpamaters to sensitivity analysis, such as \\alpha that balances D_{1} and D_{2}. Such analysis would strengthen reproducibility.\n2. The contribution of individual components (PCNS, ACDM, WDAD) is shown via accuracy drops, but their synergistic interactions are not deeply analyzed. For instance, how PCNS’s node scheduling interacts with ACDM’s temperature adaptation remains unclear.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive review and insightful questions. We are grateful for your recognition of our work and we address each of your comments below:\n\n> `Weakness 1`: Need more hyperparameter sensitivity analysis.\n\nWe appreciate this valuable feedback. Regarding hyperparameter $\\alpha$, which balances the weight between two difficulty measurers $D_{1}$ and $D_{2}$, our analysis reveals:\n|Metric|$\\alpha$=0.5|$\\alpha$=0.75|$\\alpha$=1|$\\alpha$=1.25|$\\alpha$=1.5|\n|--------|-------------|------------|------------| ------------| ------------|\n|local (acc)|83.57|**84.13**|83.42|83.34|82.84|\n|global (acc)|74.80|74.82|**75.18**|71.33|71.33|\n\nAs shown above, $\\alpha$=0.75 and $\\alpha$=1 provide optimal performance for local and global accuracy respectively. Values that are too small ($\\alpha$=0.5) compromise robustness when pseudo-labels are incorrectly predicted, while larger values ($\\alpha$>1.0) underestimate neighborhood distribution impact on difficulty, which results in introducing nodes with vague class representations too early. Importantly, the model maintains stable performance across a reasonable range of $\\alpha$ values.\n\n\n> `Weakness 2`: Limited discussion of synergistic interactions of components.\n\nWe thank the reviewer for this insightful observation. We agree that analyzing the interplay between components is crucial to understanding our design. For example, we conducted experiments demonstrating the complementary relationship between PCNS and ACDM:\n\n|Method|cora (local acc)|cora(global acc)|citeseer (local acc)|citeseer (global acc)|\n|--------|-------------|-------------|------------|------------|\n|PCNS|82.09|72.33|84.09|61.85|\n|ACDM|82.61|71.02|83.35|61.23|\n|PCNS+ACDM|**82.79**|**73.51**|**84.54**|**63.76**|\n\nAs shown in the table, the combined approach achieves the best performance by leveraging their complementary strengths: PCNS progressively introduces challenging nodes based on neighborhood distributions and prototype misalignment, while ACDM dynamically adjusts the distillation temperature to match the current difficulty level. This synergistic interaction enables more effective knowledge transfer of complex topological information.\n\n\n> `Question 1`: Implementation details of baselines?\n\nFor FL methods, all baselines except FedAvg support model heterogeneity, and they work the same way as they do in computer vision tasks. But for FGL methods, to the best of our knowledge, we are the first to explore model heterogenity in FGL settings. Consequently, existing FGL methods (and also FedAvg) are originally designed for homogeneous models. To adapt them for our heterogeneous setting, we remove direct parameter sharing between architecturally different models. For evaluation, we distill a global model from local private models and assess its accuracy.\n\n\n> `Question 2`: Reasons for competition of temperature with model parameters?\n\nWe are grateful that you asked this question, because we have the opportunity to explain to you once again an important feature of TRUST. When the proxy model learns well (low L), $\\theta_{temp}$ increases to raise $\\tau$, softening the possibility distributions and making the task harder to prevent underutilization of model capacity. When the model struggles (high L), $\\theta_{temp}$ decreases to lower $\\tau$, sharpening the task focus on high-confidence knowledge. In this way, $\\tau$ self-adjusts to client capabilities without manual tuning.\n\n\n> `Formatting Concerns`\n\nWe will correct the relevant typo in the revised version."}, {"review_id": "NYBGTo4X76", "reviewer": "Reviewer_SdVL", "summary": "The authors propose a novel framework named TRUST  to solve the model-heterogeneous issue in FGL.  Following the FedType pipeline, three key modules collaboratively transfer structural knowledge and reconcile heterogeneous model architectures. In the experiments section, five real world datasets under moderate and high heterogeneity validate the method’s effectiveness.", "questions": "1. Does the categories ‘FL’ and ‘FGL’in Table 1 both refer exclusively to model-homogeneous methods? This is not explained explicitly in the paper. \n\n2. In Section 3.4, the theoretical formulation of WDAD loss is presented, but the practical computational aspects remain unclear. Could the authors provide more specific details about how this loss is actually computed during training?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n1. The paper provides a comprehensive analysis of model heterogeneity in FGL. This is an important contribution, as previous research in this area has been limited.\n\n2. The paper tackles model heterogeneity using curriculum learning and dynamic temperature adjustment. This provides valuable insights into applying knowledge distillation to federated graph learning while preserving structural attributes.\n\n3. The paper introduces a cross-class mechanism, which ensures intricate topological knowledge transfer. This represents an advancement over conventional KL-divergence approaches, which are limited to intra-class possibility distribution alignment and may neglect some information.\n\n4. The experimental results verify the effectiveness of the proposed method, making the paper cohesive and solid.\n\n Weaknesses:\n\n1. The baseline methods primarily focus on conventional FL approaches (e.g., FedAvg, FedProto) and a limited set of FGL methods (e.g., AdaFGL), and the implementation details of these baselines are insufficient. The authors should incorporate more state-of-the-art FGL methods to strengthen the experimental validation.\n\n2. This paper lacks discussions on the computational complexity of the proposed framework. Such an analysis would make a good complement to the contribution.\n\n3. The use of the terms \"inter-class\" and \"cross-class\" appears to be inconsistent in the paper. Maintaining consistent terminology would enhance clarity and precision.", "was_revised": true, "final_justification": "The author's rebuttal has resolved my primary concerns. After reviewing the comments from other reviewers, I will maintain my score to support this paper", "rebuttal": "We sincerely appreciate your thoughtful review and valuable feedback on our work. We address your concerns below and hope these clarifications will help you re-evaluate and update the score:\n\n> `Weakness 1 and Question 1`: Need more evaluation and implementation details of FGL baselines.\n\nWe thank the reviewer for this constructive suggestion. To thoroughly evaluate TRUST's performance, we have conducted comprehensive experiments comparing against other state-of-the-art FGL methods across five benchmark datasets:\n\n|Methods|Cora-local|Cora-global|Citeseer-local|Citeseer-global|PubMed-local|PubMed-global|CS-local|CS-global|Photo-local|Photo-global|\n|---------|---------|---------|---------|-----------|---------|---------|-------|--------|--------|--------|\n|FedTAD|80.27|65.70|83.24|66.37|87.88|83.30|90.72|82.93|**93.04**|84.04|\n|FedSSL|79.53|64.79|82.32|68.15|88.07|83.58|91.97|80.73|88.01|83.52|\n|TRUST|**83.90**|**75.32**|**85.84**|**67.43**|**89.06**|**84.57**|**92.19**|**87.07**|91.42|**91.50**|\n\nFrom the results, we can see that TRUST achieved the best performance on 90% of the evaluation metrics, only falling short on the local accuracy of the Photo dataset, where FedTAD achieved the best performance. The results confirms TRUST's effectiveness against current state-of-the-art methods.\n\nWith regard to implementation details, all FL baselines we choose except FedAvg support model heterogeneity, and they work the same way as they do in computer vision tasks. But for FGL methods, to the best of our knowledge, we are the first to explore model heterogenity in FGL settings. So all FGL baselines chosen only support model homogeneity. To include them (and also FedAvg) in heterogenous settings, we remove direct parameter exchange between architecturally different models and maintain all other original components and hyperparameters.\n\n\n> `Weakness 2`: Limited discussions on the computational complexity.\n\nWe strongly agree with this suggestion. In our experiments, it takes about 15 minutes to run 200 epochs on small-sized datasets like Cora, and for mid-sized datasets like Pubmed and Large-sized datasets like ogbn-arxiv, this expands to 1 hour and 8 hours respectively, which is about 25% higher compared to FedType. \n\n> `Weakness 3`: Inconsistent terminology throughtout the paper.\n\nThanks for this valuable suggestion. We will carefully review and standardize all terminology in the revised version.\n\n\n> `Question 2`: Details about computational aspects of WDAD?\n\nFollowing the maximum-entropy principle, we add a regulation term to WDAD loss ($\\eta \\cdot q_{ij}\\log q_{ij}$). Taking advantage of this regulation, it can be solved extremely quickly with the Sinkhorn algorithm. This algorithm exhibits linear convergence and is inherently parallelizable and can be fully vectorized, making it particularly suitable for modern GPU architectures."}, {"review_id": "Y51xyDB77Q", "reviewer": "Reviewer_6U5h", "summary": "This paper proposes TRUST, a novel and generalizable model-heterogeneous FGL framework specially designed for non-Euclidean graph data. The method enables each client to construct a proxy model, which transfers knowledge between the client and a central server, while integrating three key components to handle client heterogeneity and topological learning. Progressive Curriculum Node Scheduler employs curriculum learning to gradually introduce difficult knowledge to proxy models, ensuring stable knowledge assimilation. Adaptive Curriculum Distillation Modulator dynamically adjusts distillation intensity based on client capabilities and graph complexity. Wasserstein-Driven Affinity Distillation incorporates a cross-category comparison mechanism to learn subtle structural information. Experimental results demonstrate that TRUST outperforms existing methods in FGL settings, showing robust performance against model heterogeneity.", "questions": "a) While the method performs well on mid-sized graphs, its computational overhead, particularly for WDAD’s optimal transport calculations, is not rigorously analyzed. Large-scale graphs are absent from experiments. Can the authors provide evaluation metrics for larger datasets?\n\nb) The paper claims robustness to \\lambda and T, but other critical hyperparameters (\\tau_{min} and \\tau_{max} in ACDM) lack sensitivity analysis. How sensitive is performance to these hyperparameters?\n\nc) Several key implemenation details are omitted in the paper, which may hinder reproduction. For example, What are the specific details of proxy model architectures?  How many clients were used for every dataset? Were baseline methods (e.g., AdaFGL) adapted for FGL, or used as originally published?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "- Strengths:\na) The paper is well-motivated and effectively addresses a gap in model-heterogeneous FGL. The methodology is technically sound, with logically derived equations and well-justified reasons.\nb) The paper provides a clear and detailed explaination for the proposed  model-hetergenous FGL framework, and the three core components for knowledge transfer are described in detail.\nc) The framework’s focus on model heterogeneity aligns with real-world needs where clients demand architectural flexibility. TRUST provides a practical solution to this challenge.\nd) The paper formalizes three underexplored challenges of existing FGL methods and proposes an innovative solution. This work innovatively bridges the gap between FGL and model-heterogenous setting. \n\n- Weaknesses:\na) Experiments are limited to mid-sized graphs (e.g., PubMed). Large-scale graphs (e.g., OGB-ARXIV) are absent, which raise concerns about  scalability and computational efficiency.\nb) While the paper provides some implementation details and hyperparameter configurations, several critical aspects remain either omitted or inadequately specified. For example, although the configurations of private models are described, the architectural details of proxy models are not discussed in the paper.\nc) The computational overhead of critical components (e.g., WDAD’s optimal transport calculations) is not quantified, leaving questions about efficiency in resource-constrained environments.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review and encouraging feedback. We are grateful for your positive assessment about importance of our work. We address your questions below:\n\n> `Weakness 1 & Question 1`: Need evaluation metrics for larger datasets.\n\nThank you for the feedback. We conducted experiments on larger datasets like twitch-gamer and ogbn-arxiv, and we report our experimental results below:\n\n|Metric|ogbn-arxiv(local)|ogbn-arxiv(global)|twitch-gamer(local)|twitch-gamer(global)|\n|------|------------|-------------|--------------|------------|\n|FedAvg|68.90|41.58|91.60|50.67|\n|FedGTA|68.32|41.33|86.61|50.84|\n|AdaFGL|68.94|41.61|91.59|49.03|\n|TRUST|**70.05**|**42.38**|**92.56**|**51.92**|\n\nAs shown, TRUST consistently outperformed baselines across all datasets, which demonstrates that our method maintains its advantages even at larger scales. We will include these results in the experiment section in the revised version.\n\n> `Weakness 2 & Question 3`: Need clearer implementation details.\n\nWe thank the reviewer for this important suggestion. For experimental setup, there are 10 clients in our experiments, where each client maintains a private model with architecture randomly selected from GCN, GAT, or GraphSAGE to simulate real-world model heterogeneity. To facilitate collaboration, each client is equipped with an additional small proxy model that serves as a communication bridge. This proxy model uses a standardized GCN architecture with 3 layers to ensure compatibility across all clients. On the server side, we implement a global model that also adopts a GCN backbone while maintaining consistent configurations with client models for fair comparison.\n\nFor baseline implementations, all FL baselines we choose except FedAvg support model heterogeneity, and they work the same way as they do in computer vision tasks. But for FGL methods, to the best of our knowledge, we are the first to explore model heterogenity in FGL settings. So all FGL baselines chosen only support model homogeneity. To include them (and also FedAvg) in heterogenous settings, we remove direct parameter exchange between architecturally different models and maintain all other original components and hyperparameters.\n\nWe will add more implementation details in the revised version.\n\n\n> `Weakness 3`: Limited discussion of computational overhead for WDAD.\n\nWe strongly agree with this suggestion. In our experiments, it takes about 15 minutes to run 200 epochs on small-sized datasets like Cora, and for mid-sized datasets like Pubmed and Large-sized datasets like ogbn-arxiv, this expands to 1 hour and 8 hours respectively, which is about 25% higher compared to FedType.\n\n\n> `Question 2`: Need more parameter sensitivity analysis.\n\nWe strongly agree with you that more critical hyperparameters should be included in the sensitivity analysis. We conducted experiments evaluate the impact of key hyperparameters $\\tau_{min}$ and $\\tau_{max}$, which control the bounds of our adaptive temperature mechanism. The results are shown below:\n\n|Metric|$\\tau_{min}$=0.1|$\\tau_{min}$=0.3|$\\tau_{min}$=0.5|$\\tau_{min}$=1.0|$\\tau_{min}$=2.0|\n|------------|---------------|---------------|--------------|---------------|---------------|\n|local (acc)|82.97|83.88|83.53|**83.90**|83.51|\n|global (acc)|12.16|12.16|74.59|75.32|**75.50**|\n\n|Metric|$\\tau_{max}$=8|$\\tau_{max}$=9|$\\tau_{max}$=10|$\\tau_{max}$=11|$\\tau_{max}$=12|\n|------------|------------|------------|--------------|--------------|--------------|\n|local (acc)|84.97|83.86|83.90|**85.14**|84.42|\n|global (acc)|**75.52**|74.59|75.32|74.59|74.77|\n\nAs shown in the results, when $\\tau_{min}$ is too low, the probability distribution of the teacher model will be close to the original one-hot form, which prevents effective knowledge transfer. However, within a reasonable range of $\\tau_{min}$ and $\\tau_{max}$, the model demonstrates consistent performance. We will incorporate these sensitivity analyses into the revised version to further demonstrate the robustness of our approach."}, {"review_id": "Wr9wm16ctU", "reviewer": "Reviewer_i4hd", "summary": "This paper introduces the TRUST framework where three techniques are used to  advance the knowledge distillation framework as a means to address model heterogeneity in FGL. PCNS progressively introduces challenging samples based on learning difficulty. ACDM enables capability-aware adaptive temperature control, and WDAD employs Wasserstein-based cross-class comparison. As far as I am aware, this is the first work to tackle model heterogeneity in FGL.", "questions": "Please address the concerns in \"weaknesses\".", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. The methodology is clearly presented through illustrations and explanations, demonstrating the framework's  theoretical soundness.\n2. The proposed method is flexible and is compatible with various backbone models.\n3. The inclusion of a thorough ablation study proves that all components contributes meaningfully to overall performance.\n4. The paper is well-structured and easy to follow.\n\nWeaknesses:\n1. In the experimental evaluation , the comparison is limited to only two existing FGL approaches. Expanding this to include more recent state-of-the-art methods would strengthen the validity of the claims.\n2. The authors should conduct computation time and memory usage study.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your constructive feedback and positive assessment of our work. Below we provide detailed responses to your valuable comments:\n\n> `Weakness 1`: Need more recent state-of-the-art FGL methods.\n\nAdding more state-of-the-art FGL baselines is indeed a good suggestion. We have conducted experiments on five datasets to confirm the superiority over other FGL methods. The results are shown below:\n\n|Methods|Cora-local|Cora-global|Citeseer-local|Citeseer-global|PubMed-local|PubMed-global|CS-local|CS-global|Photo-local|Photo-global|\n|---------|---------|---------|---------|-----------|---------|---------|-------|--------|--------|--------|\n|FedTAD|80.27|65.70|83.24|66.37|87.88|83.30|90.72|82.93|**93.04**|84.04|\n|FedSSL|79.53|64.79|82.32|68.15|88.07|83.58|91.97|80.73|88.01|83.52|\n|TRUST|**83.90**|**75.32**|**85.84**|**67.43**|**89.06**|**84.57**|**92.19**|**87.07**|91.42|**91.50**|\n\nFrom the results, we can see that TRUST achieved the best performance on 90% of the evaluation metrics, only falling short on the local accuracy of the Photo dataset, where FedTAD achieved the best performance. The results confirms TRUST's effectiveness against current state-of-the-art methods.\n\n\n> `Weakness 2`: The authors should conduct computation time and memory usage study.\n\nWe strongly agree with this suggestion. In our experiments, it takes about 15 minutes to run 200 epochs on small-sized datasets like Cora, and for mid-sized datasets like Pubmed and Large-sized datasets like ogbn-arxiv, this expands to 1 hour and 8 hours respectively, which is about 25% higher compared to FedType. As for memory usage, in actual experiments, we found that the memory usage of TRUST is similar to that of other baselines,  demonstrating that our architectural enhancements do not incur significant memory overhead."}, {"review_id": "wzXEK0Zhb8", "reviewer": "Reviewer_BwgV", "summary": "This work focuses on Federated Graph Learning and proposes TRUST, a novel knowledge distillation-based model8 heterogeneous FGL framework. \n\nThe authors claim the following contributions:\n- Problem identification - systematically investigate model-heterogeneous FGL. They formally identify three fundamental challenges in knowledge distillation (KD)-based methods.\n- Proposing TRUST which includes:\n    - Progressively schedules node difficulty,\n    - Dynamically adjusts distillation strength throughout training,\n    - Enables effective knowledge transfer across diverse architectures and complex graph topologies.\n- Empirical evaluation.", "questions": "* How many clients were involved in the experiments presented in the paper?\n* How is model heterogeneity defined by the authors? Section 2.1 suggests that each client may use a different architecture, which raises the question of how traditional federated learning methods (e.g., FedAvg)—which assume homogeneous models—are included in the comparisons, given their limitations in such settings.\n* How does TRUST scale with the number of clients? What is the maximum number of clients tested in the experimental evaluation?", "limitations": "NA", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Pros:**\n\n* The paper tackles a timely and important problem in FGL under model heterogeneity, which is highly relevant to real-world applications where client models vary in architecture and capacity.\n* With the exception of the experimental section, the paper is generally well-written and clearly structured.\n* The proposed method demonstrates improved performance compared to existing approaches.\n\n**Cons:**\n\n* The paper lacks a theoretical analysis of the proposed method, such as convergence guarantees or formal justification.\n* The experimental section requires improvement—key implementation details (e.g., the number of clients) are missing and should be included for reproducibility.\n* The authors do not report statistical measures such as standard deviations or confidence intervals, which are important for assessing robustness.\n* The figures are not sufficiently self-explanatory, particularly Figures 1, 2, and 5:\n\n  * Figures 1 and 2 contain excessive detail and color, making them difficult to interpret; simplifying and focusing them would enhance clarity.\n  * Figure 5 could be more informative if visualized as a heatmap or another more interpretable format.\n* From Figure 3, it appears that TRUST’s advantage diminishes as the value of \\$\\alpha\\$ increases. It would be insightful to evaluate performance at more extreme levels of data heterogeneity, such as \\$\\alpha=0.1\\$ or cases where each client has distinct class distributions.\n* The paper does not include a runtime or memory complexity analysis. A comparison to existing methods in terms of computational cost would strengthen the evaluation.", "was_revised": true, "final_justification": "See rebuttal discussion.", "rebuttal": "We thank the reviewer for your thoughtful reviews and are happy to hear that you find the work valuable. We address your concerns below and hope these clarifications will help you re-evaluate and update the score:\n\n> `Cons 1`: The paper lacks of a theoretical analysis of the proposed method.\n\nThank you for this valuable suggestion. We have offered an example line chart in our anonymous GitHub repository demonstrating convergence on the Cora dataset under the setting of $\\alpha$ = 0.5, and we report the detailed loss trajectory during training as follows:\n|Epoch|1|25|50|75|100|125|150|175|200|\n|-|-|-|-|-|-|-|-|-|-|\n|Loss|2.6849|1.1302|0.6986|0.5943|0.5338|0.5010|0.4855|0.4791|0.4722|\n\nFor theoretical analysis, with regard to WDAD, we add a regulation term to WDAD loss ($\\eta \\cdot q_{ij}\\log q_{ij}$) following the maximum-entropy principle. Taking advantage of this regulation, it can be solved extremely quickly with the Sinkhorn algorithm. Convergence of this algorithm has been proved in previous works[1]. Therefore, our analysis majorly focuses on ACDM and PCNS.\n\nWith regard to ACDM, for the adversarial dynamics in Eq. 10 with fixed learning rates $\\eta_{model}$ = 0.01, $\\eta_{temp}$ = 0.001, we establish:\n\n**Theorem 1**. Under $\\beta$-smooth loss ($||\\nabla^2L(\\theta)|| \\leq \\beta$) and bounded gradient variance $\\mathbb{E}[||\\nabla L - \\mathbb{E}[\\nabla L]||^2] \\leq \\sigma^2$, after T iterations:\n\n$\\min_{1≤t≤T} \\mathbb{E}[||\\nabla L(θ_{model}^t)||^2] \\leq \\frac{2[L(θ^0)-L^*]}{η_{model}T} + η_{model} \\beta \\sigma^2 + C \\eta_{temp} \\beta \\sigma$,\n\nwhere $C = O(1)$ captures the timescale separation. Additionally, the gradient norm ratio satisfies empirically:\n\n$\\frac{\\mathbb{E}[||\\nabla L(\\theta_{temp})||]} {\\mathbb{E}[||\\nabla L(\\theta_{model})||]} \\leq 0.1, \\forall t \\geq t_0$.\n\nWith regard to PCNS, for the pacing function $f(t) = \\min(1, \\lambda + (1-\\lambda)t/T)$ with $\\lambda$ = 0.5, $T$ = 50:\n\n**Lemma 1**. With $\\beta$-smooth loss and $\\epsilon_{pseudo}$-bounded pseudo-labels ($P(\\hat y_{pseudo} \\ne y) \\leq \\epsilon_{pseudo}$):\n\n$|L_t - L_{full}| \\leq \\beta D_{max}(1 - f(t))^2 + \\epsilon_{pseudo}$,\n\nwhere:\n\n- $\\beta \\in [10^{-4},10^{-3}]$ measures the maximum ratio of gradient norms between consecutive GNN layers:\n$\\beta = \\max_{v \\in V} \\frac{||\\nabla h_v^{l+1}||}{||∇h_v^{l}||}$,\n\n- $D_{\\max} = \\max_{v \\in \\mathcal{V}} D(v_i)$ represents the maximum node difficulty based on the difficulty measurer,\n\n- $\\epsilon_{pseudo} \\leq 0.1$ assumes that the worst-case validation accuracy across clients is at least 90%, which is plausible for well-pretrained local models:\n$\\epsilon_{pseudo} = 1 - \\min_k acc_{val}(w_{pretrain}^k)$.\n\nWe will include a more detailed discussion of this in the camera-ready version.\n\n\n> `Cons 2, Question 1 and Question 2`: Key implementation details are missing.\n\nFor FL methods, all baselines except FedAvg support model heterogeneity, and they work the same way as they do in computer vision tasks. But for FGL methods, to the best of our knowledge, we are the first to explore model heterogenity in FGL settings. Consequently, existing FGL methods (and also FedAvg) are originally designed for homogeneous models. To adapt them for our heterogeneous setting, we remove direct parameter sharing between architecturally different models and maintain all other original components and hyperparameters. For evaluation, we distill a global model from local private models and assess its accuracy.\n\n\n> `Cons 3`: The authors do not report statistical measures.\n\nWe strongly agree with this suggestion. We report standard deviations on five datasets to assess robustness of our framework. The results (mean ± standard deviation) are shown below:\n\n|Method|Cora-local|Cora-global|Citeseer-local|Citeseer-global|PubMed-local| PubMed-global|CS-local|CS-global|Photo-local|Photo-global|\n|---------------|----------|----|-----------|--------------|------------|-------------|----------|----------|-----------|----------|\n|TRUST|83.90±0.28|75.32±0.54|85.84±0.30|67.43±0.45|89.06±0.09|84.57±0.15|92.19±0.15|87.07±0.05|91.42±0.20|91.50±0.35|\n\nThe small standard deviations demonstrate consistent performance of TRUST across different datasets.\n\n\n> `Cons 4`: The figures are not sufficiently self-explanatory.\n\nWe appreciate this valuable feedback. We add excessive details in order to provide comprehensive visual explanations of our method and its components through color-coding different modules, which has been recognized by some other reviewers like hwGA (\"The problem illustration and framework overview are clear and intuitive, and the methodology is also explained with clarity, making the work easy to follow\"). In the revised version, we will  streamline all figures to focus on the most essential elements and maintain a balance between completeness and readability.\n\n\n> `Cons 5`: Lack performance evaluation at more extreme levels of data heterogeneity.\n\nWe conducted extensive experiments under severe data heterogeneity ($\\alpha$=0.1) to evaluate robustness of our method. Results are shown below:\n\n|methods|Cora-local|Cora-global|Citeseer-local|Citeseer-global| PubMed-local|PubMed-global|\n|------|-----|----|-------|------|-------|-------|\n|FedAvg|93.14|35.04|86.33|54.98|97.61|79.99|\n|FedNOVA|93.51|36.50|86.47|56.17|97.66|79.92|\n|FedProto|92.59|30.84|87.51|49.48|97.69|78.45|\n|MOON|93.32|31.75|86.61|51.56|97.56|77.16|\n|FedType|94.06|40.88|87.64|60.03|97.48|79.36|\n|AdaFGL|85.44|36.13|88.22|56.91|92.88|79.69|\n|FedGTA|85.43|35.40|**88.79**|56.17| 92.86|80.02|\n|TRUST|**94.60**|**42.52**|88.44|**64.04**|**97.76**|**82.65**|\n\nThese results demonstrate the superior performance of TRUST even under extreme data heterogeneity, outperforming baselines on 5 of 6 evaluation metrics, with only the local accuracy on the Citeseer dataset showing slightly lower performance.\n\n> `Cons 6`: The paper does not include a runtime or memory complexity analysis.\n\nWe also strongly agree with this suggestion. In our experiments, it takes about 15 minutes to run 200 epochs on small-sized datasets like Cora, and for mid-sized datasets like Pubmed and large-sized datasets like ogbn-arxiv, this expands to 1 hour and 8 hours respectively, which is about 25% higher compared to FedType. As for memory usage, in actual experiments, we found that the memory usage of TRUST is similar to that of other baselines, demonstrating that our architectural enhancements do not incur significant memory overhead.\n\n\n> `Question 3`: How does TRUST scale with the number of clients? What is the maximum number of clients tested in the experimental evaluation?\n\nIn the paper, we keep the client number at 10 throughout the main experiment. However, we supplement some experiments on the model performance under varying client number settings. The results are shown below:\n\n|client number|5|10|20|25|50|\n|-------|------|------|------|----|----|\n|local acc|**89.26**|89.26|89.70|87.23|85.03|\n|global acc|82.92|84.42|84.49|85.13|**85.95**|\n\nNotably, in federated learning settings, data is typically partitioned among clients using Dirichlet distribution. When the number of clients becomes excessively large (25-50), each client receives only a small data portion, leading to highly fragmented subgraphs where local models cannot adequately capture neighborhood information. This can be reflected in the table where performance suffer a great drop in local accuracy when client number is large. However, within a reasonable range, our method proves robustness when client number changes.\n\n\nAll in all, thank you again for your recognition of our work and your valuable suggestions. All suggested improvements will be carefully incorporated into the revised version and please look forward to TRUST being better presented to academic peers.\n\n[1] Cuturi, Marco. \"Sinkhorn distances: Lightspeed computation of optimal transport.\" Advances in neural information processing systems 26 (2013)."}]

Author rebuttal:

Meta-review: The paper introduces TRUST, a novel framework for model-heterogeneous federated graph learning. The authors assert that they have addressed several key issues found in previous approaches in this area. The paper received five reviews, with high scores of 5, 5, 4, 4, and 3. Overall, the reviewers were impressed with the paper and supported its acceptance.
Decision: Accept (poster)