From Euclidean to Graph-Structured Data: A Survey of Collaborative Learning
Authors: Rémi Bourgerie
Sarunas Girdzijauskas
Viktoria Fodor
Venue: Accepted by TMLR ()

Abstract: The conventional approach to machine learning, that is, collecting data, training models, and performing inference in a single location, faces fundamental limitations, including scalability and privacy, that restrict its applicability. To address these challenges, recent research has explored collaborative learning approaches, including federated learning and decentralized learning, where individual agents perform training and inference locally, with limited collaboration.
Most collaborative learning research focuses on Euclidean data with regular, grid-like structure (e.g., images, text). However, these approaches fail to capture the relational patterns in many real-world applications, best represented by graphs. Learning on graphs relies on message-passing mechanisms to propagate information between connected nodes, making it conceptually well-suited for collaborative environments where agents must exchange information. Yet, the opportunities and challenges of learning on graph-structured data in collaborative settings remain largely underexplored.
This survey provides a comprehensive investigation of collaborative learning from Euclidean to graph-structured data, aiming to consolidate this emerging field. We begin by reviewing its foundational principles for Euclidean data, organizing them along three core dimensions: learning effectiveness, efficiency, and privacy preservation. We then extend the discussion to graph-structured data, introducing a taxonomy of graph distribution scenarios, characterizing associated statistical heterogeneities, and developing standardized problem formulations and algorithmic frameworks. Finally, we systematically identify open challenges and promising research directions.
By bridging established techniques for Euclidean data with emerging methods for graph learning, our survey provides researchers and practitioners with a well-structured foundation of collaborative learning, supporting further development across a wide range of scientific and industrial fields.

Published in Transactions on Machine Learning Research (02/2026)
From Euclidean to Graph-Structured Data: A Survey of Collaborative Learning
Rémi Bourgerie
remibo @kth.
School of Electrical Engineering and Computer Science, and Digital Futures
KTH Royal Institute of Technology
Stockholm, Sweden
Sarunas Girdzijauskas
sarmnsy@ktth.ss
School of Electrical Engineering and Computer Science, and Digital Futures
KTH Royal Institute of Technology
Stockholm, Sweden
Viktoria Fodor
urfodor wkth.
School of Electrical Engineering and Computer Science, and Digital Futures
KTH Royal Institute of Technology Stockholm, Sweden
Reviewed on OpenReview: https: // openreview. net/ oet///ouuurui//////a///ooooooee0
Abstract
The conventional approach machine learning. that is, collecting data, training mod- els, and performing inference in single location, faces fundamental limitations, including scalability and privacy, that restrict its applicability. To address these challenges, recent research has explored collaborative learning approaches, including federated learning and decentralized learning. where individual agents perform training and inference locally, with limited collaboration. Most collaborative learning research focuses on Euclidean data with regular, grid-like structure (e.g., images, text). However, these approaches fail to cap- ture the relational patterns in many real-world applications, best represented by graphs. Learning on graphs relies on pessage-passingg mechanisms propagate information be- tween connected nodes, making it conceptually well-suited for collaborative environments where agents must exchange information. Yet, the opportunities and challenges of learn- ing graph-structured data in collaborative settings remain largely underexplored. This survey provides comprehensive investigation of collaborative learning from Euclidean to graph-structurod data, aiming to consolidate this emerging field. We begin by revicwing its foundational principles for Euclidean data, organizing them along three core dimensions: learning effectiveness, efficiency, and privacy preservation. We then extend the discussion to graph-structured data, introducing a taxonomy of graph distribution scenarios, char- acterizing associated statistical heterogeneities, and developing standardized problem for- mulations and algorithmic frameworks. Finally, we systematically identify open challenges and promising research directions. By bridging established techniques for Euclidean data with emerging methods for graph learning, our survey provides researchers and practition- ers with a well-structured foundation of collaborative learning, supporting further develop- ment across a wide range of scientific and industrial fields. Resources are available

---

Published in Transactions on Machine Learning Research (02/2026)
1 Introduction
Over the past decade, Deep Learning has turned into a success story, largely due to the combination of four factors [Goodfellow et al. 2016): (i) neural networks as universal function approximators (Rosenblatt,
1958 Rumelhart et al., 1986) (īi sample efficient training algorithms (Bottou 2010), (ii) parallelization
schemes that align with hardware architectures (Hooker 2021 and (iv) the availability of massive amounts
of high-quality data.
While the first three factors have been consolidated over the years, the fourth requirement, access to massive dalasels, remains a major obstacle. Indeed, today data ecosystem remains highly fragmented, constrained by privacy legislation (European Union 2016), data ownership concerns, integration complexity, and the inherent coordination difficulties multi--gent systems. This access to data partially addressed for LLMs by scaling on public web data (Navcod al. 2023 Baack 2024), but the fundamental challenge persists: the absence of mechanisms to collect, align, and utilize data across organizational and technical boundaries.
Collaborative learning, often termed under the umbrella of federated or decentralized learning, provides framework to address this challenge by allowing multiple agents to train shared model directly on their private datasets. In this framework, only model updates or aggregated information are exchanged ween agents rather than entire datascts, drastically reducing the amount of data communicatod. Morcover, by keeping data at its source, this framework holds the promise privacy preservation.
Agent
Collaborative System
Domains of Euclidean Data
Training/nffrrence inference on distributed Euclidean datascts
N
Agent
Mensurement
Temporal
Communication
efficiency
Agent
Privacy preservation agents data
Vision
Language
(a) Collaborative Learning on Euclidean Data
Agent
Collaborative System
Domains Graph Data
Training/ifferenee inference on distributed graph datasets
Agent
Communication efficiency
Human
Physical
Privacy preservation agents data
:
Agent
Inter-agent relations
natNve
Systems
Chemistry
(b) Collaborative Learning Graph Data
Figure Collaborative Learning from Euclidean to Graph Data. Collaborative learning refers system of agents that collectively train and infer MI. models on their local private data. Collaborative learning has been extensively studied for Euelidean data (a), comprising highly popular domains like vision, language, temporal sequences, and measurement data. Conversely, collaborative learning on graph data (b) is an emerging field. Graphs canonically represent data structured with relations, arising in domains including human behavior, physical systems, chemistry, and information systems. Compared to centralized learning, collaborative learning offers three main advantages: enabling training and inference on distributed datasets, communication efficiency, and the promise of privacy preservation agents data. Collaborative learning on graph data aligns naturally with Message Passing Neural Networks (MPNNs).
2

---

Published in Transactions on Machine Learning Research (02/2026)
Collaborative learning solutions can be divided into two categories based on data structure: Euclidean dala, which exhibits regular, grid-like structure (e.g., images as 2D grids, text as 1D sequences), and non- Euclidean data, where relationships are defined by irregular structures such as graphs (Bronstein et al. 2021 as illustrated in Figure 1
For Fuclidean data, typical use cases include cross-device scenarios involving numerous agents, each possess- ing limited data insufficient for generating accurate models independently. These agents may include mobile phones (Xu et al.), 2023), autonomous vehicles (Hellström et al. 2022). mobile base stations (Zhang et al., 2022), or Internet of Things (loT) devices (Zhou et al. 2023). Cross-silo applications involve large datasets held by distinct actors who cannot share data to preserve privacy, prevalent in healthcare and finance (Li ett al. 2020). Collaboration typically involves exchanging and aggregating model parameters during training, while inference is generally performed locally by individual agents.
For graph-structured data, two distinct scenarios emerge. The first scenario learning on a. large collection of independent graph instances, typical in computational biology (e.g., predicting molecular properties) (He et al. 2022), involves learning parameters of graph- based model (e.g., GNN) using tools conceptually similar to those for Euclidean data, each graph instance can be treated analogously to a single data sample. The second scenario arises when agents hold a portion of shared global graph, both inference and learning necessitate information exchange among agents, as the graph topology encodes critical relationships spanning agent boundaries. Applications include collaborative autonomous agents in loT systems (Liu et al.
2025), autonomous vehicles (Blumenkamp et al.] 2022 Pan et al. 2023). and privacy-preservin learning
over large-scale social network or recommender system graphs (He al. 2021 Dong et al. 2023 Agrawal
et al.
2024 Han et al. 2024).
While these diverse applications demonstrate broad polential, their implementation presents significant chal- lenges that differ fundamentally between Euclidean and non-Euclidean settings. Modern machine learning (ML) relies predominantly on iterative optimization algorithms (Bottou et al.] 2018), which have long posed difficulties for distributed systems (Bertsekas) 2011 Ram et al.j 2009; Low et al.] 2010 Xing et al.) 2016). These challenges are exacerbated in collaborative learning due to the substantial size of the models that need to be communicated and inherent system discrepancies, driving active research in system architecture and
algorithm design [Kirrouz et al. 2021 Daly et al. 2024
For graph-structured data, the challenges are further amplified by the non-Fuclidean nature of the data. The graph topology is often captured by Message Passing Newral Networks (MPNNs) (Gilmer et al. 2017 Hamilton et al. 2017), where the nodes iteratively exchange and aggregate messages with their neighbors. These MPNNs are inherently distributed iterative algorithms,, which makes them conceptually well-suited for collaborative scttings. However, when each agent holds a partition of the shared global graph and cannot freely share data, the requirement for information to propagate through the graph topology creates novel challenges beyond those in Euclidean settings, leading to rapidly expanding area of research.
1.1 The Objectives of the Survey and Related Work
While existing surveys have extensively covered federated learning on Euclidean data, and to a lesser extent on graph data, none have systematically analyzed their intersection. This survey bridges that gap by providing unified view that maps the design space of collaborative learning from Euclidean to graph-structured data. We outline how different approaches relate to each other and identify research challenges that remain unanswered for the less understood case of graphs. Crucially, while methods for Euclidean data are well- established, graph-structured data introduces unique complexities that draw upon knowledge from multiple domains. Collaborative learning for graph data stands on three foundational pillars, borrowing from distinct research areas as illustrated in Figure]
Diffusion Algorithms on Networks are distributed iterative methods that compute node-level or network-level functions through local message passing. These methods were initially devel- oped for distributed decision problems in networked systems (c.g., multi-agent systems, sensor networks) (Tsitsiklis et al. 1986 Olfati-Saber et al., 2007), and later adopted in large-scale net-
3

---

Published in Transactions on Machine Learning Research (02/2026)
Collaborative Learning
Learning private distributed data ex Finldyg (McMilaan 2017)
Graph Representation Learning
Collaborative Learaing on Graph Data Learning distributed graph structured data ex Fedligge (Zhung 2021)
Learning representations graph-stncctreed data (Kipf Welling 2016)
Algorithms Networks
Computing functions graphs via parameter free message passing Label Propogation (Zhou
2013
2017
Figure Collaborative Learning on graph data and its three pillars. Collaborative learning on graph data relies on three distinct pillars: diffusion algorithm on networks, graph representation learning, and collaborative learning.
work analytics and learning through closely related sandom-walk-basee processes (e.g.: ranking, recommendation, semi-supervised inference on graphs) (Zhou al.) 2003).
Graph Representation Learning employs deep neural architectures to learn representations from graph-structured data (Kipf & Welling 2017). Applications where relational structure is intrinsic (e.g,, social networks, molecular modeling, infrastructure systems) have driven this area, which emerged the intersection of graph signal processing and machine learning (Bruna et al. 2013).
Collaborative Learning, for example federated learning (McMahan et al. 2017 allows training and inference on private distributed datasets through centralized or decentralized approaches. Col- laborative learning is motivated in two key practical cases: when sensitive data cannot be shared between actors (healthcare, finance, user data) and when natively distributed data cannot be col- lected due communication constraints (robotics, IoT). The supporting disciplines span machine learning, distributed optimization, information theory, and communication systems.
Collaborative learning on Graph Data (Zhang et al. 2021) studies learning on private dis- tributed graph-structured data. intersects the three areas above, by combining the message- passing principles of diffusion algorithms on networks, the neural architectures of graph representa- tion learning, and the distributed optimization frameworks of collaborative learning.
This survey connects these three domains to examine how established results can accelerate research in emerging fields and novel applications. Table [ presents representative surveys and foundational papers across these pillars, providing comprehensive details that were necessarily excluded maintain focus. The table also includes works parallel processing of big data, learning. To limit the scope of the survey, focus supervised learning. Interested readers may find references for collaborative unsupervised and self-supervised learning in (Uludag et al.] 2025 Zhang el al.
with partial overlap with collaborative
2025 Ji et al. 2024). for multi-agent reinforcement learning in (Gronaner & Diepold
2022
Chen et al.
2025 and for reinforcement lcarning on graph-structured data in (Nic et al. 2023 Liu et al. 2024).
1.2 The Organization of the Paper
In this survey, we provide a structured overview of lcarning from distributed private datascts, including the basic concepts, the research challenges, and proposed solutions, starting from classical collaborative learning, to learning over distributed private graph datasets. The skeleton of the paper is shown in Figure 3 In the first part of the paper, we discuss collaborative learning over Euclidean data, that is, classical centralized FL and its distributed counterparts. We formulate the learning problem, discuss how the data can be partitioned across the agents, and what heterogeneity aspects need to be considered. Then we discuss
4

---

Published in Transactions on Machine Learning Research (02/2026)
Тopic
Representative Works
Diffusion Algorithms on Networks
(Bettekas & Tsitsiklis
2015
Olfati-Saber al.
2007
Distributed averaging and consensus
Yang et al. 2019
Label propagation
Zhou et al.
2003
Graph Representation Learning
Graph Neural Architectures
Kipf & Welling
2017 Wu et al.
2020
Graph Foundation Models
Wang al. 2025
Collaborative Learning on Euclidean Data
Federated Learning
McMahan et al.
2017)
Kairouz et al.
2021
Zhang et al. 2021
Yang et al.
2019
Liu et al. 2022
Vertical Federated Learning
Liu et al. 2024
Data heterogeneily and personalization
Zhu et al.
2021
Tan et al.
2022
Liu et al. 2024
Privacy and security
Lyu et al.
2020
Daly al.
2024
Parallel training
Xing et al. 2016
Machine Learning wireless networks
Hellström al.
2022
Collaborative Learning on Graph Data
Zhang et al.
2021
(He et al.
2021
Zhang et al, 2021
Federated Learning with GNNs
Liu al. 2025
Fu et
2024
Applications and use cases GNNs in wireless commmnications Parallel training for large GNNs
He
2021
Dong el. al. 2023
He el al.
2021
Lee et al. 2022
Shao al.
2024
Lin et 2023
Liu
2022
Key surveys and seminal works (in bold) on collaborative learning graph data, including the three pillars of Collaborative Learning, Diffusion algorithms Networks, and Graph Repre- sentation Learning.
Table 1:
proposed solutions (i) for effective learning, that is, achieve models with high accuracy despite the various forms of heterogencity, (it) for efficiency, in terms of the use of communication and computation resources, and finally, (iii) solutions for privacy preservation. Since collaborative learning over Fuclidean data is an established research area, we review the key contributions in this part.
In the second part of the paper, we follow the same structure to discuss the emerging research field of collaborative learning on graph-structured data. The ways data (or information) can be distributed are significantly more variod now than in the Euclidean case, so we introduce a taxonomy for data partition. Then, we follow the structure of the first part to find relevant research results and questions that still open for the design of collaborative learning systems graph data. This part is based rigorous survey of all recent papers with kcywords federated graph and decentralized graph learning. The taxonomy of solutions for effective, efficient, and privacy-preservigg collaborative learning for Euclidean and graph data is provided in Appendix A together with the notations and the glossary of key terminology used throughout this survey.

---

Published in Transactions on Machine Learning Research (02/2026)
Collaborative Learning Euclidean Data (Sec. 2)
Problem Definition and Federated Learning Framework (Sec. 2.1)
Data Partition, Heterogemeity and Privacy Challenges (Sec. 2.2)
Improving Learning and Inference Effectiveness (Sec. 2.3)
Improving Learning and Inference Efficiency (See.
Privacy Preserving Collaborative Learning (Sec. 2.5)
Collaborative Learning Graph Data (Sec. 3)
Machine Learning on Graph Data (Sec. 3.1)
Collaborative Learning on Graph Data: Problem Formulation (Sec. 3.2)
Graph Duta, Partition, Heterugeneity and Privacy Challenges (Sec. 3.3)
Improving Leurning und Inference Effeetivvneess (Sec.
Improving Learning and Inference Efficiency (Sec. 3.5)
Privacy for Graph Data (Sec. 3.6)
Techniques
Figure 3: The structure of the survey. Our contributions reside in extending and mapping the concepts found for collaborative learning on Euclidean data to collaborative learning on graph data, thereby consol- idating insights across these separale research areas. Our analysis of the design choices focuses on three directions: improving learning and inference effectiveness, improving learning and inference efficiency, and prieacy-preserven techniques.
2 Collaborative Learning on Euclidean Data
The concept of collaborative learning emerges from multiple taxonomics, which are all cast under the umbrella term of distribuled machine learning as outlined in Figure 4
The term distribuled machine learning originally
(a) Distributed learning
(b) Parallel computing
(c) Collaborative Learning
Data can be centralized
cannot centraizzed
Accelerates computations for taining and/or inference
Allows privacy-preservin training and/or inference
data parallelism
federated learning
model parallelism
decentralized learning
Figure Taxonomies of distributed learning and the scope of our survey. Distributed learning (a) is often used as an umbrella term encompassing various scenarios and techniques. In particular, parallel computing (b) aims to accelerate training and/or inference under the assumption that the data be centralized, and includes both data- pata-parallelism and model-parallelism approaches. Our focus, however, is different scenario that we term collaborative learning (c), which addresses cases where the data remains distributed and private to its owner, covering both federated learning and decentralized learning.
served as a broad label for methods designed to train and/or infer ML models on massive datasets by distributing computations across multiple machines, often in a cluster (Xing et al. 2016 Kairouz et al. 2021). Distribution then largely associated with parallelization, combining data parallelism, where datasets are divided among machines, and model parallelism, where parts of the model are divided among machines and updated in parallel.

---

Published in Transactions on Machine Learning Research (02/2026)
The term collaborative learning refers specifically to scenarios where local data remains strictly private to its owner. For clarity, assume that local data is generated locally at multiple agents, where it persists throughout training and inference phases while remaining exclusively accessible to the data owner, a case commonly termed federated learning or decentralized learning in the literature. This privacy constraint neces- sitates precise problem formulation and distinct design methodologies to address real-world implementation challenges, which we develop in this section. Our focus here on Euclidean data, defined as data where each sample represented as feature vector in an Euclidean space (in practice, some Rd with the standard inner product).
2.1 Collaborative Learning: Problem Formulation and the Federated Learning Framework
Central Server
01
Aggrogate
/
1,
T
0:
6K
Shared
01
0g
Minimize (F))
Minimize Lx
Infer Si
Infer Sk
Private data
S
8x Agent
Agent
Figure 5: Federated learning, the basic approach to collaborative learning represented here for K agents. Agents possess private local datasets Ss- In each round i, they train their local model Fg on their local loss and communicate the model parameter updates (t). The central server aggregates in turn the local models into a global model with parameters f(t), which then shared again with the agents for another round t
Problem definition. Consider set of agents, denoted by K= (1....,A, K), illustrated in Figure 5 where the data of each agent is sampled from an unknown distribution Dr on X' x y where X' is the feature Euclidean domain and y the label domain. Given loss function é X x ) + R+, each agent seeks to learn a mapping F X' + y among a class of function F that minimizes the local population loss
(F) : ((F())),,(((())..
(1)
If the data distributions Dx were known, agents could determine an optimal solution Fg E F, by directly minimizing their respective local loss [1 on J. However, in practice, the true distributions D3 are unknown. Instead, agents rely empirical samples forming local datasets Sk, drawn i.i.d. from Dz.- These datasets allow agents to construct surrogate bx, which represents an empirical approximation of the true loss Ls based on their local data only
(x(F) = SA
Σ ((F(x),))
(2)
(a.y)ESa
This approach provides local estimate Pr E F. However, the local dalasets Ss are typically too small to ac- curately approximate the underlying distribution Dj- As result, the local estimate ft poorly approximates the true optimal function Fr, leading to high estimation error (Valiant) 1984 Bottou & Bousquet (007). To address this issue, key assumption is that although the distributions De may differ across agents, they
7

---

Published in Transactions on Machine Learning Research (02/2026)
share strong similarities. Therefore, collaboration among agents enables them to benefit from the aggregated knowledge contained in the union of local datasets (St KEK-
In the most favorable case, when the local distributions D2 are the same, an effective solution is to collaborate on building a single model common to all agents. The problem can then be framed as identifying F F that minimizes the average of local population losses among all agents
C(F)
(F. K 1=1
(3)
Since the population losses L3 are generally uncomputable, 8 is typically approxinated using local empirical losses. Most existing work focuses on minimizing (4) which is compulationally feasible, rather than the true loss
1 (F)
((F)
Σ K
(4)
This line of work typically assumes that F F" that is, minimizing 4 provides a good approximation of minimizing (3). This assumption holds only when the generalization gap -(())_(((F)) is negligible (Bottou
et al. 2018
To minimize (4. it can be convenient to parametrize Fg with 0 E θ where θ is the domain paramcter. Typically, F can be neural network, parametrized by its learnable weights 0. The problem can be framed as identifying 8", a solution of
1 Σ (Fo).
min ((F), equivalently, 8E
min 6c K
Federated learning framework. The basic approach for collaborative learning on distributed datasets is
the Federaaed Learning (FI.) framework (Konečný al.] 2016 McMahan et al. 2017 Kairouz et al. 2021) The key idea of federated learning is to federate (McMahan et al.) 2017) the training of agents through common round-based protocol coordinated by central server, shown in Figure At the end of training, each agent receives the jointly trained model, which can be used to perform inference independently on local data.
Federated Averaging (FedAvg) protocol. The most popular and simplest implementation of FL is Federated Averaging (FedAug) [McMahan et al. (017)) detailed Algorithm [ It proceeds T communica- tion rounds between leader, referred to the server, and the agents. At the beginning of each round the server randomly selects K agents (line 2) and communicates the latest global model parameters O(t - to them (line 3/. Upon reception, each selected agent initializes its temporary local model Ox - 1) to the received global model 0(t - 1). Then, cach agent k performs E local update steps 7 on its local dataset Sa to minimize its empirical loss (2) using the Stochaslic Gradient Descent (SGD) algorithm (Robbins & Monro 1951). After completing the local updates, each agent sends its resulting local model @x(t) back to the server (line 10). The server then aggregates all received models compute the updatod global model
#(t) via averaging, denoted by 0(t) (11). Under specific conditions, this iterative process converges 6*
solution of (5) (Yu et. al. 2019
Stich 2019
8

---

Published in Transactions on Machine Learning Research (02/2026)
Algorithm Federated Averaging (FedAvg) adapted from (McMahan et al. 2017
Require:
Set of agents K = {l,..., K local datasets Sk, total number of communication rounds T', number of local training iterations E, number of sampled clients Kmans initial global model ((0), learning rate y)
1: for each round t I,..., do Server:
Sample uniformly a random subset K((I) C KC of clients (IC(()) Kmaz)
Distribute global model parameters ((t - to all sampled clients & Kr.
for each agent A K(t) in parallel do
Upon reception of O(t 1), initialize local model parameter 0x(t ( = = 1) for e= E do Update local model using gradient descent
1 0x(t 2) + 0x(t
1 " 1/V
Σ (2,N)SS
((Fo)(c-))22)11
8:
end for end for
10:
Communicate local model (x(+) 0x(t 1/) to the server
Server:
11: Upon reception of models {0x {0x(())) average local models:
0(1) Σ
Tek (n()
kER())
12: Set global model: ((t) + O(t)
13: end for
14: return Final global model ((T")
2.2 Data Partition, Heterogeneity and Privacy Challenges
The basic approach for collaborative learning, as defined in FedAng (Algorithm D. builds on several unre- alistic assumptions. In the sequel, we discuss the characteristics collaborative learning that need to considered to address realistic use cases, that is, data partition, statistical imbalance, system hetero- geneity, and privacy vulnerabilities.
2.2.1 Data Partition
The first deviation from the ideal assumptions outlined in Section 2.1 concerns data-related aspects, notably the partitioning of data across agents and the resulting statistical imbalance.
FI. assumes a significant shared knowledge among the agents through the homogeneity the local datasets. However, in practice, local datasets {5x hee often vary significantly across the agents, with heterogeneity arising from the partition of the data or statistical imbalance. An example of these partitions for two agents is shown in Figure ( with summary provided in Table 2 In the native scenario (Section 2.1 agents collect similar types of features and labels, represented by common feature and label spaces X' x ) However, inconsistency in the features and labels collected is frequent in data collection practices. While these challenges have long been recognized in centralized ML (Pan & Yang) 2009 Zhu 2005), they are amplified in collaborative settings where the dataset S, that would ideally be centrally available, is instead partitioned across agents local datasets {St which stay private to each agent. To formalize, we
In practice proposed the original algorithm McMahan 2017 the dataset is divided batches fixed size and the model is updated with the gradient each batch of data, elided simplicity.
algarithmmMeMahan

---

Published in Transactions on Machine Learning Research (02/2020)
I1
S
S.
S
$2
S
X
N
M
X2
>
X
(a) No partitioning
(b) Horizontal partitioning
(0) Vertical partitioning
(d) Transfer partitioning
Figure 6: The different partitions of Euclidean data for two-agent case, where Sp denotes the datasot of agent k. Compared to no pertitioning (a), horizontal partitioning (b) splits the sample space I, vertical partitioning (c) splits the feature space X, and transfer partitioning (d) splits both, with the source agent 8 holding most the data.
assume the existence of a global dataset
S = {(x', ( Yer
where is the global sample index, and each agent. E K. holds a private partition Ss S. Different partitions exist for S:
Horizontal partition. Under horizontal partition, the index of S is partitioned across agents into local indexes Ig C I such as UkexIz I
S = (x''"")')) ('))
Therefore, agents possess data about distinct samples indexed by Ix, but with the same feature X' and label space V. This the default setting of FL presented in Algorithm and the most studied partition (Kairouz et al. 2021) in the FL. literature. This partition further distinguishes cross-device scenarios, where millions of agents with limited data (e.g., mobile devices) collaborate, from cross-silo scenarios, where few institutions holding vast amounts of data "silos" (e.g., banks, hospitals, e-commerce) collaborate.
Vertical partition. Under vertical partitions, the domains X' and D usually some Euclidean vectorial spaces, are partitioned across agents into local feature Xx and label Y* domains such that X = #bEK& and y = # Consequently, each sample (x y²) is partitioned into (I vt) E X x Vh. to form the local datasets
S = (a ((at.1))) v
Therefore, agents observe the same samples but with distinct views, as represented by different feature and label domains (Liu et al. 2024). This naturally arises in multi-modal scenarios where agents collect features and labels specific to their modality (e.g., a hospital will diagnose respiralory diseases using lung radiography techniques, while another will use respiratory tests)
special case of vertical partition is the transfer partition. agents (c.g., agent E KC) hold datasct from one feature space X,, while another target agent (c.g.. agent / K)) holds dataset from different feature space X,. For simplicity, describe the case involving two agents and
occurs when one or more source
Vk E (8,1), S = ( E Mr x 3k Dikie
Despite differences in feature domains, the agents may share subset of common samples
Ak E {s,t) ) C
These act as anchor points for transferring information between agents. Transfer partitions often involve significant data imbalance, where [Z,] > [Te], consistent with the idea. of "transferring"
10

---

Published in Transactions on Machine Learning Research (02/2026)
substantial knowledge from to t. This setting frequently arises when organizations employ different data acquisition modalities but share overlapping entities. For example, hospital with large archive of traditional diagnostic records may support another institution, using small set of shared patient samples.
2.2.2 Statistical Imbalance
In the native model, local datascts Sy assumed to be of the same size and sampled from shared distribution D on X'x'. However, in practice, the statistical properties of local dalasets can vary significantly from one agent to another, and may evolve over time as agents join or leave the system, leading to different modes of imbalance (Kairouz et al. 2021 Paulik et al. 2021
Quantity imbalance. The most obvious phenomenon arises from quanbity imbalance ([Sa] > [Si)), where the number of samples collected locally varies significantly across agents. For example, a small clinic might manage thousands of patient records, while multinational institution handles millions. This imbalance represents a natural phenomenon observed across many fields (Nisonger 2008): most of the world's data is concentrated among minority of data holders. The significant imbalance has given rise to different regimes known as dross-device, where millions of agents usually corresponding to users devices (e.g., phone) collaborate, and cross-silo, where few agents holding vast silos of data (c.g. an hospital) collaborate.
Distributional shift. In addition, statistical imbalance in local datasets can arise from the distribution itself, where the datasets are drawn. Factors like population bias, dynamic environments, multiple modalities, and varying labeling standards influence shift in the marginal distribution Dz(x,)) y) from one agent to
another, known as distributional shift Sforeno-Tooress et al. 2012
Quilioneo-Canne Caniell et al. 2022 Kairouz
et al. 2021
Zhu et al., 2021 We distinguish:
Covariate shift occurs when the feature Dx(x)) or label distribution (v(v) observed locally, differ across agents but the concept D( I y) or D(y æ) remains common. Covariate shift is ubiquitous phenomenon in statistics dating back to early statistical studies (Snow 1855) and is the most studied
case for FL (Kairouz et al. 2021 Zhu et al. 2021 McMahan et al. 2017 Hsieh et al. 2020 Li et al. 2022). This often arises from population selection bias or non-stationary environments Moreno-- Torres et al., 2012). This iittributionall shift can be observed from two different perspectives. Firstly, differences in observed label distribution Dx(y), known label shift, lead to to differences in observed features distribution (x) D(= y) (v). For example, a hospital located in a polluted area records higher frequency of respiratory disease diagnoses compared to one in a clean environment. Similarly, a difference in observed feature distributions DE(c)), known as feature shift, leads to a difference in label distribution Dx(x)= = D(y (x). For example, one hospital focuses on diagnosing a young population while another focuses on an older population, leading to differences in features collected, yet the diagnostic knowledge remains the same.
Concept shift occurs when the when the conditional distribution D&(y 2)) (or Dx y)) varies across agents. This corresponds to cases where the understanding of the concept varies from agent to another. For example, the standards used for diagnosing hypertension from blood pressure differ between hospitals. This scenario is rather rare (Moreno- Torres et al.) 2012). and challenging for most applications that assume shared mapping F e F from features to labels (Huang et al. 2021).
2.2.3 System Heterogeneity
The native forms of FL, such as FedAug in Algorithm [ work best if the capabilities and tasks the agents are similar. In practice, real-world systems include variety of devices, networks, and tasks, and these system heterogeneities affect the learning and inference performance. We briefly characterize the main types of system heterogeneity that impact collaborative training and inference.
11

---

Published in Transactions on Machine Learning Research (02/2026)
Device and network constraints. Collaborative learning systems must accommodate agents with vastly different hardware and network capabilities. These constraints manifest in two primary dimensions: compu- tational resources and communication infrastructure.
Computational beterogeneityy. Agent computations are performed on diverse physical machines rang- ing from IoT sensors and mobile devices telecom base stations, cloud containers, powerful servers, or even HPC clusters. This diversity creates order-of-magnitude differences in the time required to calculate local model updates (Xie et al. 2020 Even et al. 2024), and in the model sizes the devices can hold. Computing capabilities may also fluctuate over time due to parallel workloads, varying energy availability, or quantity shifts.
Communication heterogeneity. Agents employ different network technologies such as fiber, WiFi, 5G, and low-power loT protocols. These technologies determine both the communication topology (typically star configuration with central base station or mesh with direct pecr-to- to--per links) and the achievable bitrate, which directly affects transmission delays (Hellström et al. 2022 Wang cet al. 2020). For wireless communication, interference and noise introduce temporal variations that degrade link reliability. These network characteristics evolve dynamically as agents join, leave, or move through the environment.
These computational and communication heterogeneities create challenges at multiple levels. At the indi- vidual level, capability variations affect resource consumption such energy cost, computation time, and communication overhead during collaborative learning (Wang et al. 2022 Meng et al. 2025p. At the system level, slower agents, known as stragglers, can become bottlenecks that delay the entire learning process or introduce biasos affecting all participants (Li et al. 2022). Dynamic populations present additional complica- tions: when agents join or leave during training, the system must initialize newcomers and address potential distributional shifts and novel label classes.
Task heterogeneity. The native model assumes that all agents share the same task, represented as a common mapping from the feature domain X' to the label domain y. In practice, however, agents may use their data to perform different tasks, thereby optimizing distinct mappings (Tan et al. 2022. For example, different hospitals may use the same radiography images diagnose different diseases, resulting in heterogeneous local losses Lx Another case of task heterogeneity emerges when models are shared among applications with compound ML models, as exemplified by certain Android implementations (Huang et al. 2021). Furthermore, agents may employ different model architectures F due to limitations in memory or computational resources (Park& & Joe- LoeeWong 2024).
2.2.4 Privacy Vulnerabilities
Collaborative learning techniques were initially motivated by their potential to enable privacy-presevving training between agents (McMahan et al., 2017 Bonawitz et al., 2017), making them well-suited for appli- cations where local data cannot be pooled for regulatory reasons (Yang et al. 2018 European Union 2016).
The design of the original FedAug algorithm (Algorithm 1 supports this motivation, since ensures minimal data collection and anouymization (Bonawitz el al. 2022: Daly el al. 2024): local datasets Sş remain with the agents while only model parameters (t) or gradients are communicated (line 10), limiting what the server learns about the underlying data; and communicated parameters are immediately aggregated into O(t) [line I1 before broadcast (line 2). preventing other agents from directly accessing individual contributions. Despite these design choices, collaborative learning does not provide inherent privacy guarantees. Even in the presence of al allempting to infer sensitive information, collaborative learning in its native form suffers from information leakage at multiple levels:
agents or server, that is, entities that follow the protocol correctly while
The fully trained model parameters ((T) are subject to model inversion attacks that infer train- ing labels (Fredrikson al. 2015) or determine membership of specific samples in the training data (Shokri et al.] (017))
12

---

Published in Transactions on Machine Learning Research (02/2026)
Local model updates (t) - 8(t 8 1)), or equivalently gradients, sent to the server
Phong et al,
2017] are vulnerable to gradient inversion allacks that infer sample properties (Melss et al. 2019 achieve pixel-perfect data reconstruction through gradient matching (Zhu et al. 2019 Geiping et al,
2020; Zhao ct al. 2020). While other agents can also attempt gradient inversion upon receiving the aggregated model ((1), these attacks degrade significantly with the number of agents and batch size
due to aggregation.
Observing the evolution of the aggregated model (0(t)} over training enables attacks beyond single- round inference. Passive observation reveals temporal information, such when samples with specific properties first appear in the training data (Melis et al. 2019). More advanced threat sce- narios involve agents actively manipulating their contributions to extract information about others* training data (Melis et al. 2019
Hitaj et al. 2017).
The Collaborative Learning Trilemma
Maintaining effective and efficient learning under the privacy requirements of collaborative learning (Sec-
tion (22..4), while simultaneously accommodating data and system heterogeneities (Sections 2.2. 2.2.3
poses significant challenges. Specifically, collaborative learning must balance three competing objectives: effective learning (accurate model predictions), efficient learning (constrained communication and compu- tation), and privacy preservation (anonymizing agents data). These objectives create fundamental tensions: effective learning requires extensive information exchange and complex computations that challenge efficiency, while privacy preservation requires cryptographic techniques and noise injection that decrease efficiency and effectiveness, respectively. These tradc-offs form the trilemma illustrated in Figure which serves as the foundation for analyzing the design choices in the following subsections.
Achieving local loss minimization despite inherent challenges:
Effectiveness
Data partitioning
Statistical heterogeneity
Task heterogeneity
Dynamic agent participation
Running training and/or inference despite inherent challenges:
Anonymizing agent data from curious-but- honest agents despite inherent challenges:
Device and network constraints
Balancing anonymization with information loss
Dynamic agent participation
Complex multi-agent protocols
Mitigating increased computational cost
Efficiency
Privacy
Figure The trilemma of collaborative learning: balancing effectiveness (Section
2.3 efficiency
(Section 2.3 and privacy (Section 2.5
2.3 Improving Learning Effectiveness
Learning algorithms are effective if the generated models manage to minimize the learning loss of the agents, according to (1). The native proposition of collaborative learning (Algorithm 1 offers strong theoretical guarantees YYu et al. 2019 Stich 2019) to convergence to the solution of (5), particularly when the local datasets DReX: are i.i.d. However, real-world deployments often diverge from this ideal case, as outlined in Section 2.1
In such circumstances. the commonly used FedAng algorithm becomes suboptimal achieve adequate min- imization of local objectives. This motivates the exploration of alternative strategies aimed at improving
13

---

Published in Transactions on Machine Learning Research (02/2026)
Contal Server
Oentral Serve
dat
Tike
Apont
2
Agont
Provte
Agena
(a) learning
(b) Averaging
Central
Cental Server
Centrll
T
C Inker for T...
Infer Sor Lu
Indee Sor
Traate
Train
Trsie
Trade
Tridee
Treis
Trais
Traiee
Ascri
Agoin
Agret
Duta-ktaed tachniquu
(8) Medit-teavd techniquss
(4) Veriiaal Foderated Learning techniquus
Design principles
learning
Local damn
Model
A Label
Sample representaioon
Local operation
Commenication Op"
Standard operation medifiod
Figure 8: Variations of federated learning that aim at improving the learning effectiveness. In contrast to centrulized learning (a), where data is collected from agents and training occurs on central server, (b) Federated Averaging represents the baseline approach for collaborative learning systems. Methods to improve effectiveness (c) focus on data-based (i) and model-based (ii) techniques, typically applied to horizontal partitions. An alternative to the baseline is Vertical Federated Learning (VFL) (iii) where the inference becomes collaborative.
the effectiveness of the models used by agents to minimize their respective local objectives, that is, data-based, model-based technique and lcarning under vertical partition, as discussed in the following sections and outlined in Figure
2.3.1 Data-based Techniques
Federated learning employs shared global model, computed by the server average of the models over participants in (see step (11) ), to compute optimal solution 0* of the optimization problem 5 This approach performs effectively in its native application of horizontal cross-device scenarios with small, balanced, and similar datasets (Yang et al. 2018 Hard et al.] 2018). However, in horizontal settings, under statistical data heterogeneity (seen in Section 2.2 or unbalanced agent participation (seen Section 2.2.3 the returned local models 9μ (t) show high variance, which famously leads the average model @(t) drift away from A", the solution of (5), phenomenon well-known in FL as client drift (Karimireddy et al., 2020 Li et al. 2022). To mitigate this client drift, variety of methods have been developed that retain shared global model paradigm but modify the FedAvy Algorithm D to achieve tighter convergence to optimal solution 6" These methods, collectively known as data-base basd solutions, range from regularizing the local models to influencing the underlying data distribution through data augmentation and client sampling.
Local regularization. Regularization approaches aim to minimize the discrepancy between the local models (0x (t))ee, returned the server by modifying the local updates 7 Bounding the variance of local gradient updates in 7 across agents has shown theoretical guarantees [T.i et al. 2020) well
14

---

Published in Transactions on Machine Learning Research (02/2026)
empirical evidence Li al. 2020 for FedAug to converge to the optimal solution 8", also in non-i.id. settings. Local regularization methods can be broadly categorized into three approaches (Kairouz et al. 2021):
Gradient control methods reduce the gradient variance across agents and within local iterations by incorporating variance correction terms into the local updates 77. These terms are usually computed
from the previous iterations of the local 0z - 1) and global 0(t- - 1) models. The main approaches
rely on adaptive algorithms (Karimireddy et al., 2020; Khanduri et al. 2021
Kim et al. 2024) or
dual-based optimization (Fan et al. 2022 Gong ct al. 2022 Zhou & Li
2023
Selective update strategies roduce noise by refining which model components updated in either by restricting updates to the least noisy layers (Park & Joe- Wong 2025 by using local
optimizers that ensure smoother gradients (Wang et al. 2020).
Objective modificattion approaches modify the local empirical objective (2) by penalizing divergence
from the global model 0(t-1) 1) (Li et al.) 2020 Yao & Sun 2020 Li al. 2021) or, conversely,
encouraging deviation from the previous local model (4 1) [Li et al. 2021 It can also penalize
divergence between predictions of the global and the local model (Lin et al
2020 typically using
contrastive losses (Lee et al. 2022). Alternatively, sccond-order extensions to SGD provide smoother
local updates (Fallah et al.
2020 T Dinh et al. 2020 Luo et al. 2025
Prototype-based regularization regularize the embedding space, that is, the hidden output of the model F before its prediction head, rather than the model space. It. is achieved using "prototypes', that is, representative embeddings for each label (Snell et al. 2017 Agents construct local proto- types by aggregating embeddings of the same-label samples of Sk. which are aggregated at the server into global protolypes. Local training then incorporates regularization terms to align embeddings
with these prototypes (Tan et al. 2022a) or contrastive losses (Tan et al. 2022b).
Data augmentation. Data augmentation addresses distributional shifts, including quantity, feature, and label shifts, as well as missing labels, by modifying the local datasets St used for local training (sce ( in A Algorithm Augmentation strategies in federated settings can be categorized as follows:
Server-instructed augmentation. A central server may augment local datasets by redistributing samples between agents to balance distributional shifts (Zhao et al.) 2018), by instructing selec- tive down-sampling of overrepresented classes in agents local datascts (Duan et al. (020), or by generating synthetic samples and communicating them to the agents (Jeong et al. 2018).
Independent local augmentation. Agents may augment their local datasets independently. Tech- niques include traditional transformations (Zhang et al.) 2018) or learned generative models (Wul et al. 2020). When labels are missing, contrastive losses can be used to train on the unlabeled local
datasct, augmented with negative samples (Li et al. 2022 He et al. 2025).
Collaborative local data augmentation. Agents may augment their local datascts by collaborating with other agents. This is particularly salient in vertical partitioning, where common samples have to be aligned between agents (Yang al.) 2022 Kang et al. 2022). When not all local samples of the local datasets can be aligned, the pseudo features labels can be estimated with generative models (Kang et al. 2022), or by matching features across agents (Yang et al. 2022
Client selection. Client selection techniques tackle the statistical imbalance seen in horizontal partition settings by modifying the server strategy for sampling agents in line 3 of Algorithm 1 There two approaches:
Total availability. Some works assume total availability of agents. Strategics are designed to select
underperforming clients (Li et al. 2021 Cho et al. 2020), group clients with similar performance
in the same round CCai et al. 2020 Yang al. 2021 ; prioritize clients that contribute to
15

---

Published in Transactions on Machine Learning Research (02/2026)
improving the global accuracy (Wang et al. 2020 These strategies can be further optimized using reinforcement. learning techniques on the server side, such as Q-learning (Wang et al. 2020 multi-armed bandits (Yang et al. 2021).
Partial availability. Other works assume that not all agents are available at the same time. Several participation patterns are sludied, ranging from cyclical patterns (Cho al.
2023
to lemporal
participation modeled by stationary Markov chains (Wang & Ji
2022
Rodio et al,
2023), or a
combination of both using R-separated Markov chains (Sun et al.
2025
Xiang al.
2024 Some
assuming independent agent unavailability (Wang & Ji. 2024) or handle agent turnover with departing and newly arriving agents (Ruan et al., 2021p. The techniques involve maintaining estimate of the
works look at spatio-temporal correlation using Markov chains (Rodio et al. 2023
participation statistics to modify the sampling strategy (line 2 in Algorithm [ (Cho et al.) 2020
Wang & Ji 2022 Ruan et al.] 2021 Ribero et al. 2022 the aggregation (line 11 in Algorit
(Ruan al.) 2021 Ribero et al., 2022: Wang & 2024). and the local update (line 19 Algorithm
I) (Ruan al. 2021) accordingly guarantee biased convergence to 0*. Theoretical results on
correlated participation (Rodio et al.
2023
Cho et al. 2023 establish the trade-off of fast and tight
convergence to 0"
Data-based techniques address client drift under statistical heterogeneity while maintaining the shared global model. They follow three complementary approaches: local regularization, data augmentation, and client selection. These techniques intervene at different stages of the learning pipeline: regularization constrains local gradient updates during training, data augmentation modifies the effective data distribution before training, and client selection filters which distributions reach aggregation.
Trilemma. Each intervention point trades effectiveness through variance reduction against resource efficiency. Regularization requires computing correction terms or maintaining dual vari- ables, data augmentation demands generative models or sample redistribution, and adaptive client selection needs participation tracking and optimized aggregation schemes. The privacy implications vary by technique: while regularization methods preserve privacy, techniques requiring data sharing (server- innterr- augmentation, collaborative augmentation, prototype aggregation) expose local information and necessitate additional protection mechanisms.
2.3.2 Model-based Techniques
In the first. place, FI. aims to minimize the local objectives of ① among all agents. However, the optimal solutions F" of the different local objectives in ① might be different between agents due factors like distributional shifts (Section 2.2 or task heterogeneity (Section 2.2.3 This implies that the convenient approach of looking for a model solution F E F, common to all agents, is suboptimal. Therefore, some efforts have been devoted to allow model solutions that are personalized to each agent, while retaining the main steps of the collaborative learning framework depicted in Section 2.1 With personalization, the optimization problem becomes
K Σ (F
min e. 0x
(6)
K
Personalization is a well-established direction in federated learning (Tan et al. 2022 Sabah et al.) 2024 It can be achieved using two approaches: keeping common architecture while personalizing the model parameters 0g E θ, or personalizing the model architectures (Ox)ee themselves.
Parameter personalization. VI o k, ش 0; G = θ Parameter personalization maintains a common domain for the personalized models but allows models to converge distinct parameters. These tech- niques apply exclusively to horizontal partitions, where all agents maintain similar model architectures. We distinguish:
16

---

Published in Transactions on Machine Learning Research (02/2026)
Local adaptation. Local adaptation extends Algorithm by allowing each agent to modify the received global model parameters (t) into personalized version (x+++ 2)) through additional steps in the local training round. This adaptation can occur after the federated learning process, as in meta-learning, where ((T) is rapidly fine-tuned on local data (Finn et al.) 2017 Jiang et al. 2019). Alternatively, adaptation can be performed in every round by modilying the local model initialization step (line in Algorithm 1. c.g., by interpolating between ((t) and (x(t [0] (Deng et al. 2020), using learnable local masks (Li al., 2021 Dai et al. 2022 Huang et al. 2022 or
applying low-rank adaption methods to ((t) (Sun et al. 2024
Guo et al. 2025
Hypernelwork--bueee A hypernetwork (Klein al. 2015 that is, a network that generales the weights of a target model, can be used produce personalized parameters for each agent. Given an agent representation vector Cky the server-side hypernetwork Ha with trainable parameters a outputs the personalized model (4) = Ho (cx). Instead of maintaining shared model Fo(o: all agents share common hypernetwork with learnable weights a jointly trained across agents by minimizing (hamsiia et al. 2021
K
Σ
En (FHooeta
(7)
K
Training is performed on the server, where agents transmit Valx and the server updates via gradient steps. The representation C2 can be fixed (e.g., encoding an agent type) or updated jointly with a.
Cluster-based. Similar agents are divided into clusters, and learn cluster-specific modcls. Assign- ments can be static (Duan et al. 2021 where clusters operate independently, dynamic, allowing clustering and learning to occur simultaneously. A straightforward approach leverages similarities for clustering. These methods typically alternate between clustering and local updates, framing the problem as an optimization task (Long et al.] 2023). Clustering optimization aims to minimize
inter-cluster similarities (Briggs et al. 2020; Werner et al., 2024 or maximize intra-cluster similar-
ities (Sattler et al.) 2020 Briggs et al. 2020 Long et al.,
2023 Werner et al.] 2024). Similarities
are computed between clients (Briggs
al. 2020), clients and clusters (Ghosh et al.
2020 Man-
sour et al.] 2020 Werner et al.
2024)
clients and the global model (Sattler et al.)
2020 ussing
losses Ghosh et al. 2020 Sattler et al.
2020) gradients (Werner et al. 2024). Techniques include
greedy assignment (Ghosh et al.] 2020 Mansour et al. 2020), recursive bi-partitioning (Sattler et al.
2020), recursive fusion (Briggs el al. 2020). and threshold-based clustering robust. to Byzantine set-
tings
Werner et al. 2024). Assignments can be performed server-side (Sattler et al. 2020 Long
ct al. 2023) or client-side, although this requires sending all cluster models (Ghosh et al. 2020).
Additionally, some studies also address the assignment. of new entrants to clusters (Duan el al.
2021
Similarilies belween agents can be used to favor aggregation between the most similar agents even without hard division into clusters. A similarity matrix Ω [wx,]] gK* measuring the proximity between local objectives, is maintained by the server to build personalized models. These similaritics
can be computed explicitly using metrics on local models 6% (t)kex (Huang et al. 2021 Chen &
Zhang 2022) or by evaluating local empirical objectives on models of other agents (x()))..))
OOnoskko et al. 2021 Zec et al. 2022). Given Ω models can be personalized by averaging the
most similar ones (Onoszko et al. 2021
or using weighted averages (Zec et al. 2022 Huang et. al.
2021]. Alternatively, Ω can be learned as a problem parameter by aadding regularization term
R (f..)((.)))=)))) to (6): (mithh et al. 2017 Huang ct al.) 2021 Shoham et al. 2019 Chen &
Zhang 2022 Chen et al. 2022). This requires alternating optimization, updating 12 and model
parameters in separate steps (Smith et al. 2017 Huang et al. [2021] Chen et al. 2022). While
learning Ω introduces significant communication overhead, decentralized protocols (Zec et al. 2022)
and smart sampling (Zec et al. 2024 help mitigate scaling issues.
Architecture personalization. v 0 # Ox + O2 Agents can utilize different model architectures, for example, neural networks of different sizes. The need for architecture personalization is driven both by
17

---

Published in Transactions on Machine Learning Research (02/2026)
concept statistical imbalance (seen in Section 2.2.2] but also by system heterogeneity constraints like device heterogeneity and application heterogeneity (seen in Section 2.2.3 We distinguish:
Model decoupling. Local models, 02, are divided into a shared component, 0% 0, trained using the FedAug algorithm, and a private component, f% which may vary in architecture across agents. In neural networks, this split lypically layer-wise (depth) (Wang et al. 2016 2023 Liang et. al. 2020:
Arivazhagan et al. 2019 Bui et al. 2019), but may also occur width-wise Diao al. 2021 in
The structure of the split directly influences communication costs. Some strategics reduce these costs by replicating shared layers to private components, especially in CNNs Park & Joe- Wong 2024 or by focusing on the most impactful layers (Park & Joe- Wong, 2025).
hybrid manner (Kang al. 2025
The level of decoupling impacts model performance: greater device heterogeneity benefits
from more private neurons, while noisy local updates favor larger shared component (Liang al.
2020
Collins et al., 2021). The optimal split can be determined through adaptive (Vahidian al,
2021
or even learnable (Deng al.) 2022 strategies. However, incomplete aggregation schemes introduce security risks (Park & Joe- Wong 2024).
Multi-modal personalization. The server maintains M global models, 0 U M where model 6,, corresponds to a data domain m. Agents selectively combine these models based on their local data, addressing feature shifts. Identifying the modality can rely prior knowledge (Chen & statistical estimation (Huang et al.
Zhang 2022
2019). Model combinations vary: agents may
assign distinct models per sample (Huang et al.) 2019) or aggregate outputs from multiple models
when tasks require multi-modal fusion (Chen & Zhang 2022). Training often leverages parameter
personalization, using clustering (Huang et al.) 2019) or agent similarity (Smith et al. 2017 Chen
Zhang 2022 to train these models jointly.
Knowledge distillation. Instead of sharing full model parameters, agents can exchange sample- like information, such as predicted labels or embeddings, which can be shared across agents with potentially different architectures. This information may come from globally shared unlabeled dataset LLi & Wang 2019 Bistritz et al. 2020: He et al. 2020) or, for more communication-fficin approaches, from prototypes, i.e., fixed set of representatives of the same label samples built collaboratively (Tan et al. 2022b Kim et al. 2023). The shared sample information must be aligned across agents, typically using knowledge distillation losses (Hinton et al. 2014), which can be further improved with contrastive losses (Tan et al.) 2022b).. Exchanging these compact data instead of full models can significantly reduce communication and storage costs (Tan et al.] 2022b). Even when global model is not strictly required, maintaining
be useful: it distill knowledge to agents with smaller architectures, rapidly initialize new agents' models from pre-trained model (Li
& Wang 2019
Kim et al.) 2023), or continuously provide shared knowledge through server-side
Training
[e el al. 2020 Lin el al. 2020: Zhu et al. 2021).
Model personalization techniques address the fundamental limitation that a single shared model cannot optimally serve all agents under heterogeneous conditions. They follow two complementary strategies: parameter personalization and architecture personalization. Parameter personalization maintains common architectures while allowing the model parameters diverge through local adaptation, hypernetworks, clustering, similarity-based weighting. Architecture personalization accommodates structural heterogeneity through model decoupling (shared-private splits), maintain- ing multiple global models, the alignment of representations via distillation and prototypes.
Trilemma. Agent-specific adaptation improves local performance but decreases efficiency due increased coordination complexity (similarity computation, cluster maintenance, hypernetwork training) and communication overhead (multiple model transfers, prototype exchange). The techniques that operate purely on model parameters (local adaptation, layer decoupling) limit information leakage while those requiring explicit agent comparison data sharing (similarity cstimation, clustcring, distillation, prototype aggregation) expose agent-specific information.
18

---

Published in Transactions on Machine Learning Research (02/2020)
2.3.3 Training and Inference under Vertical Partitioning
The techniques presented Sections 2.3.1 and 2.3.2 consider horizontal partitioning of the data (Sec- tion 2.2.1 In many domains, however, the data are instead vertically partilioned, with agents observing complementary feature domains of the same samples (Section 2.2.1). In this case, effective learning requires combining the information from these partitions. There are two main lines of solutions for training under vertical partitioning, training with collaborative inference, for isolated inference.
Central Server
04 Oog
RC Del
i=
1,
ac Oeg
Infer for i=1,
Infer for i-1, ....
Infer for 1- n
Eval
zf Model update
01.00
=f
xt2
22
03,99
f
g
Model update
Model update
S
S₂
Sg
Agent
Agent
Agent (active agent)
Local data
Local model
Sample representation
A Label
Local operation
Communication
Figure 9: Training under vertical partitioning with three agents. During inference, the computations of the global model Fa,y are distributed across agents: to produce a prediction ÿ² for input x', each agent k communicates its partial representation 2f = Φx.8, R. (af)) of the sample. Only one or a subset of the agents (here, agent 3) possesses the true label, which can evaluate the training loss É. During training, the gradients with respect to the learnable parameters 02,02,0 03, tha must be communicated across agents in the form of sc āz
Training with collaborative inference. Models be trained under vertical data partitions by collab- oratively optimizing a collaborative inference loss. In previous Sections 2.3.2 and 2.3.1 each agent trains local model On used to perform inference in isolation on its local data. Collaboration occurs solely during training when agents periodically share their local model iterate (t). However, when agents know about the same sample, as seen in the example of vertical data partition (Section 2.2.1). agents might benefit from col- laborating during inference as well, for building a common prediction. This collaborative inference imposes a reevaluation of the problem definition of standard FL in and reevaluation the training protocol. Un- der vertical partition, input features of a sample ( y²) are split across the agents k E K. Each agent k holds #t, a subset of the features x' such x Ukexa forming the local dataset S₁ {at E Xeeiez For sim- plicity, we assume that single agent, agent 4= has access to the labels, hence Sg- ((=t.,,
X x Hie. Since the features of each sample are split across agents, the global model F E F, used to infer on the centrally available sample (x', y²), can be naturally decomposed to process the features locally that each agent holds a local model component Φk,0, mapping from X'. to latent space 2,, parameterized
+ Consider
2AA related setting, known split learning (Cupta & Raskar 2018 Vepakomma al. 2019), partitions the inference model agents for computational offloading, where typically agent holds the data while others provide model layers. While this he viewed an case vertical partition (where agent holds all features), facus on scenarios where multiple agents each possess complementary private data.
19

---

Published in Transactions on Machine Learning Research (02/2026)
by 0g bb while final predictor feue, parameterized by sha E Me, is held by agent kt. Letting F's model parameters be 0 = (f1...., 0x v/x-) with parameter space Q = On x x... x Ox x Va,, the inference of Fo on sample indexed by can be expressed as
0' Fo = Fo((aft......kk))) few- (01,0, (2t)....)........ OK,0x (ak)))
(8)
Therefore, the partition of the global model F results in a collaborative inference described in Figure 9 for performing a prodiction on n², each agent communicates its partial representation Φk.0. (xf) of the sample. Since collaborative inference requires agents exchange intermediate representations and gradients, which may expose sensitive information, practical deployments typically incorporate privacy protection mechanisms to mitigate such leakage (Liu et al. 2024), detailed in Section 2.5
The population loss can be defined for all agents as
F(can...rh)))) (Fo (Fo.,..,0k.vv (Z xx))
(9)
Since the data distribution D unknown, the loss over Sy
(9 is approximated by an empirical counterpart defined
(F = 1
Σ e @Fe.... IEI
xxk)
(10)
Notably, both 9 and 10 are common to all agents, allowing the global loss in this collaborative setting as
3 to be equivalently expressed
E CVFL(FF) = LvFL(F).
(11)
However, unlike in 3 where local losses can be computed independently by each agent, the structure (9) requires collaborative inference to evaluate the loss. As result, computing the objective requires inter- collaborative inference loss. Despite this difference,
agent communication, and we refer (9) and (10) the learning goal remains the same: to approximate minimizer of the population loss O by minimizing its empirical surrogate (10) over parametrized hypothesis class. This leads to the following optimization problem of Vertical Federated Learning, in direct analogy with 5
min 8,E0 0x 2 EM,
(12)
The typical training pipeline follows two steps
(Liu et al. 2024
Privacy-preserving sample aligmment. The formulation in 10 assumes a shared understanding of sample indices I, their features (I'), and labels (y") across agents. However, in practice, each agent only sees its local view, making the computation To address this problem, the first step is to identify the common index set across the agents. It usually achieved through the Privale Set Intersection (PST) protocols (Liang Chawathe 2004 Pinkas et al. 2018). PSI is extended to multiparty setups in (Zhou al. 2021 Lu & Ding 2020).
(10) infeasible without preliminary coordination.
Collaborative training. The optimization problem in solved collaboratively between agents K. = (1...., K}, summarized in Algorithm 2 It proceeds in T rounds, where each round consists of two phases:
Collaborative Inference (lines Each agent k computes intermediate representations #{
&x. (xf)) (line 3) for each sample and sends them to the active agent k* (line 4h which aggregates
them and computes the empirical loss ((F)) on the mini-batch (line 7 updates vik- (line
and sends partial gradients dz;
ac to each agent (line 91.
CCollboratiive Model Update (lines 11 12
Each agent applies the chain rule to compute Ve,
(line 11 and updates its local parameters (line 12
20

---

Published in Transactions on Machine Learning Research (02/2026)
Algorithm 2 Vertical Federated Learning algorithm (VFL) adapted from (Liu et al. 2024)
Require: Set of agents K = {1..... K), shared sample index set Z, learning rates 7), local parameters
((0) = (0) k=1 (bx- (0)), mumber commuiiatiin rounds T, mini-batch size B
1: for each round t == I....,T
, do
Collaborative Inference:
for each agent K in parallel do
3:
Compute local representations: 2 01,0, for al EI Send feg to the active agent k- end for
Active agent k*: Aggregate {zî} and compute empirical predictions for each i E I: 10 {f (2f.....(k)) lEI
Compute partial empirical loss on samples of index I, LVVL. M 전 Eiez (1' Update predictor model: vhx-(() the (1-1) - "n1 V VoL. LVFI.
Compute and send partial derivatives each agent k:
ac
20 iEB:
Collaborative Model Update:
10:
for each agent K. in parallel do
Compute gradients via chain rule: Vo, C = =iEET
divEL
0s
11:
82
59
12:
Update local model: 0& <- 0x 172 V22 LVFL
13: end for
14:
end for
15: return Final parameters (O) (T), 2/ (T)) and sample predictions {i/ hiez
Algorithm 2 can be extended in several ways adapt more realistic scttings with:
To decrease the level of communication required by the collaboration, agents may pre-train their local models on private data prior collaborative training (Feng et al., 2022 Shen
Pre-traiming.
et al. 2025). This approach leads to scalable VFL, but may introduce bias from local data distri- butions (hhnn et al. 2025).
Model averuging. The VFL protocol Algorithm 2 assumes that each agent k E maintains a private local model dx.0, with domain paramcters O₃ which are individually updated basod on practice, certain agents might
the collaborative model update phase (lines g 12 However, use the same model architecture, leading to the same domain space O₂ θ, This occur, for example, when agents (c.g., hospitals) collect the same features related to the same samples (e.g., patients). In such cases, it can be beneficial to introduce a periodic averaging of the models, (x(() = (t) 0. (f)+(((()) in the collaborative model update steps (Fu et al. 2024h.
Unlabeled data. The formulation in 112 assumes full access to supervision through the labels y'. However, in practical scenarios, significant portion of the data may be unlabeled. To handle such cases, the empirical loss in (10) can be extended to include self-supervised objectives, such contrastive losses over unlabeled samples (Li et al. 2022 He et al. 2025). Additionally, the local inference modules Φx.0;
autoencoders, enabling self-supervised training via reconstruction losses (Feng 2022 These extensions apply primarily to the collaborative inference phase (lines 3 4 of A Amgorit.mm and they allow agents to learn useful representations even in the absence of labels.
be instantiated
Training for isolated inference. Transfer techniques aim to train models capable of performing isolated inference on vertically partitioned data while achicving effectiveness comparable to collaborative inference. This addresses the primary limitation collaborative inference, which the communication overhead at
Note that, in practice, this operation be efficiently parallelized using batching over the samples, voluntarily eluded here clarity.
21

---

Published in Transactions on Machine Learning Research (02/2026)
inference time, while still exploiting the knowledge contained in vertically partitioned datasets. Consider transfer partition as defined in Section 2.2.1 involving source agent, and largel agent 6, each aware of their common sample set Ia n Iz, obtained, for example, via PSI protocols. Two main approaches are commonly used depending on the structure of the partition (Lu et al. 2024):
Knowledge distillation. When the feature spaces of the agents exhibit significant overlap, knowl- edge distillation (KD) techniques, also discussed for horizontal partitioning in Section 2.3.2 can be adapted to vertical settings. The objective to transfer the knowledge of the collaborative inference model (ds,o,, 0r,6,, fv,), pre-trained on domains X, and X, by agents and & according to [12,, to student model (Øot, fwf) for agent t, that performs isolated inference domain X, To this end, agent t trains its model in the shared latent space 3, x Z,, using aligned unlabeled samples (Huang ct al. 2023 Ren et al. 2022).
distillation loss that aligns the representations of dot, 0g..., and 0
Transfer learning. When the number of aligned samples limited, and only the source agent has sufficient data to train independent model, transfer learning techniques (Liu et al., 2020 can be employed. The goal is to adapt model pre-trained by agents on its dataset S, and domain X, to the target domain X,, using a shared embedding space Z. The core assumption in this setting is that unaligned samples from s and t still yield semantically similar representations in Z. The process involves two steps. First, agent trains a local encoder Øs.0, : X2 - Z and a prodictor fv. on Ss, yielding represenlations Z, = {z i E I.} = {05.0, (xr),, E I.). Then, agent initializes its own encoder 01,8, : Xr + Z and predictor fw.. The training of parameters 1hr and 0, proceeds by aligning Z, = i E In] & (z/), E Ie} with Z, in the latent space Z using a transfer loss, while keeping vs, 02 frozen. Since Zg and Z, may differ in size, advanced transfer losses are typically required, such as feature matching (Yang et al. 2022). pseudo-label matching (Feng et al. 2022), adversarial methods (Kang et al. 2022
Learning under vertical partitioning methods improves learning effectiveness when agents hold complementary features of shared samples. They follow two paradigms based on inference requirements: collaborative inference and isolated inference. Learning with collaborative inference enables joint optimization over the full feature space, while isolated inference trains models for autonomous deployment through knowledge distillation (when feature spaces overlap) or transfer learning (when source agents have richer data).
Trilemma. Collaborative inference decreases efficiency due to persistent coordination over- head (reprosentation exchange, gradient commmication) but achieves accurate models. Isolated inference avoids the communication overhead at inference, but needs complex representation align- ment. Privacy exposure is severe in both approaches compared to horizontal methods: collaborative inference contimuuuly shares feature-derived embeddings during training and inference, while isolated inference requires sample alignment through PSI protocols and representation sharing during knowledge transfer, necessitating explicit privacy-preserving mechanisms throughout.
2.4 Improving Learning Efficiency
While Section 2.3 explored techniques to enhance the effectiveness of the learning process, these approaches often overlook the constraints of the communication and computing resources introduced in Scction 2.2.3 This section examines stralegies for efficient learning, that is, ways to achieve the fast convergence of the learning models despite the constraints given by the network topology, the heterogeneity of the devices and network connections, and the often costly and unreliable communication links. We discuss model aggregation strategies, and solutions for asynchronous aggregation, and for increased communication efficiency.
22

---

Published in Transactions on Machine Learning Research (02/2026)
Server
Server
Agent
Edge
servers
Agent 4
Agent
Agents
Agont
Agent
Agent
Agent
Agent
(a) Centralized aggregation
(b) Decentralized aggregation
(c) Hierarchical aggregation
Figure 10: Different aggregation patterns for collaborative learning. Centralized aggregalion (a) uses a central server that collects and combines model updates from all agents. Decentralized aggregation (b) allows agents to exchange and aggregate updates over a network with restricted communication links. Hierarchical aggregation (c) relies on multi-level structure where a central server coordinates with edge servers, which in turn oversee the aggregation of updates of their associated agents.
2.4.1 Model Aggregation
The efficiency collaborative learning depends primarily on the model aggregation pattern, defined by the topology and the timing of the model parameter exchange and aggregation. This section discusses aggregation schemes in three aggregation topologies: centralized, decentralized, and hierarchical, shown in Figure 10
Centralized aggregation. Centralized aggregation is the standard strategy for combining local models, employed in Algorithm[] In this setting, central server coordinates training: at each round t= 1,..., T, it collects local model paramcters (t), each obtained after training for E local epochs, aggregates them, and redistribules the result to all agents.
The motivation for centralized aggregation originates in distributed computing, where distributing stochastic gradient computations can thcorotically reduce training time by a factor proportional to the number of agents K (Zinkevich et al.] 2009 McDonald et. al., 2010). However, naive parallelization with aggregation at every epoch (E' 1) incurs significant communication overhead, which can offset the computational benefits depending system cost constraints described in Section 2.2.3
When communication overhead outweighs the gains from faster convergence, simple workaround is one-shol aggregation, where models are aggregated once after local training completes (T = (Medonald et al. 2009 Zinkevich et al. 2010). While highly communicttion-efficiii
(Zhang et al. 2012 this strategy relies on strong assumptions, such as statistical similarity across agents datasets, that become increasingly difficult satisfy as K increases (Li et al. 2020 Khaled al. 2020).
A more balanced alternative is to let. agents perform multiple local updates (E > 1) before periodically aggregating their models. This approach, known as local SGD, leverages the robustness of SGD to noise and relaxed synchronization (Dean et al. 2012: Bottou et al.] 2018). It has proven successful both in practice
and theory, achieving near-linear speedup under mild i.i.d. assumptions Stich 2019
Yu et al. 2019
et al. 2020
Further refinements include hybrid schemes combining onc-shot and local SGD Hou et al. 2022 ), the use of server-side optimizers (Reddi al.] 2021), and dual-based methods such as ADMM (oyyd al. 2011
Zhou & Li 2023
23

---

Published in Transactions on Machine Learning Research (02/2026)
Decentralized aggregation. Decentralized aggregation is the opposite of centralized aggregation: agents aggregate models directly without a central coordinator. Tt is often proposed as a more robust alternative, mitigating the single point of failure inherent to centralized schemes. In some cases, it is also necessitated by system constraints or by governance, trust, and incentive considerations that make a central coordinator undesirable or infeasible. Unlike centralized approaches, both training and aggregation occur over a mesh network, introducing additional challenges related to network topology.
Distributed aggregation builds the early results of distributed averaging (DeGroot 1974 Tsitsiklis et al, 1986 Bertsekas & Tsitsiklis 2015). For simplicity, consider agents connected according to mesh topology modeled as a graph, and holding initial value vectors n; (0), with an average of ((0). At cach step E, the updates follow a linear rule: Ak E K, ( + WHaTT (1) + EiEN(a) (t)( where (Vk) denotes the neighbors of agent k. These updates can be expressed with a Markov chain Il(t + 1) = W (t) where (tt) = (u) and [[(t) = [t)]. Convergence to consensus state occurs when all nodes reach the same value Ak E IC, lim Te (1) = x² Average consensus is achieved if, in addition, x= = =(0) In the literature, we find two algorithmic approaches for distributed averaging (Denantes et al. 2008
Average consensus algorithms assume that all nodes communicate and update their values simulta- neously. The problem is formulated by minimizing the Dirichlet eneryy, expressed as
li(t) ( LII(t) E E (t((t)-T((()) Tt))) JEN(E)
(13)
which quantifies the quadratic disagreement between neighboring nodes, where L is the graph Lapla- cian (Mohar et al. 1991) and (i)) denotes the neighbors of node i in the mesh network. In continuous time, updates towards a minimum of 13] can be expressed as diffusion equation as =t) = - (t)
in (Cortés) 2006 Jiang & Wang. 2009 Olfati-Saber et al. 2007). In discrete time, the update takes
a Markov chain form (1(+1)) W (t) in (Blondel et al. 2005 DeGroot 1974 Nedic et al. 2009)
Olshevsky & Tsitsikiis 2009 For example W(t) is derived directly from (13) with the update rule W w(t)=/-eh with step size > 0 and the identity matrix in (Saber & Murray 2003 DDfati-Saber
& Murray] 2004 Olfati-Saber et al. 2007). For static graphs, convergence is guarantoed when W is doubly stochastic and connected. The convergence rale is exponential, determined by the topol- ogy of the graph (Fiedler 1973), through the second-larged eigenvalue of the symmetric Laplacian
A2(L..) (Fax & Murray 2004 Olfati-Saber & Murray, 2004). Average consensus algorithms have
been
widely applied in multi-agent systems (Vicsek et al. 1995 Jadbabaie et al. 2003 Blondel
et al.
2005
Fax & Murray (004) and adapted to address issues like dynamic topologies (Jabbabaee
et al.
2003
Olshevsky & Tsitsiklis 2009 Moreau 2005) and delays (Blondel et al.
2005 Tsitsiklis
ct al.
1986
Gossip algorithms perform pairwise communication (Kapp et al. 2000 Bawa et al. 2003). Upon a lick of its internal clock, node k communicates with node with probability Pkt and they update their values as Tx(t + 1) = it + 1) = 2 matrix P mirrors the role of adjacency matrices in synchronous algorithms. Gossip protocols are communicaiion-elicient and due to their inherent randomness, they are robust to asynchronous
(()+(((() (Boyd et al., 2005 2006). The probability
execution and partial observability (Jelasity et al. 2005 Denantes et al. 2008). Variations include geographic routing (Dimakis et al. 2008; Bénézit et al. 2010), weighted updates (Bénézit et al.
2010), and memory-angmente protocols Cao et al. 2006 Liu et al. 2013
Distributed averaging combined with local learning methods leads to decentralized aggregation. Compared to centralized aggregation, the challenge is that both the local learning and the model aggregation processes require multiple steps (as represented by respective indíces and t). Early methods combine local gradient updates with a single averaging step (Tsitsiklis et al. 1986 Nedic & Ozdaglar 2009). More advanced techniques incorporate multiple propagation steps (Johansson et al. 2008),, handle link failures (Lobel & Ozdaglar 2008), enforce constraints via projections (Nedic et al.] 2010), or consider non-i.i.d. data (Wang et al.) 2022). Gossip-based methods (Ram et al.) 22099] Ormándi et al.) 2013 Blot et al. (0166) enhance robustness to asynchronous participation and dynamically changing agent population (eegadi et al. 2019
24

---

Published in Transactions on Machine Learning Research (02/2026)
Dual-based techniques, such ADMM (Boyd et al. 2011 also support decentralized learning, where dual variable averaging is performed synchronously (Nsterrvv 2009 (Vanhaesebrouck et al. 2017).
Wei & Ozdaglar 2012 via gossip
The methods above rely on distributed averaging, where each agent's value (x(t) converges to a global consensus л = #(0). However, when the network topology reflects data heterogeneity, aggregation can instead balance global and local consensus. To preserve local information, the objective [3] can be modified to combine global agreement with local smoothness, in (Zhou et al. )003). If this formulation admits closed-form solution, it can be computed in a decentralized
in {II(t)"LI((t) + [=] m
proposed
way using a broad class of diffussion (also known as propagation or smoothing) algorithms (Zhu & Ghahramani
2002 Zhou et al. 2003). These mechanisms integrate naturally with local learning by initializing Ta (t)
(t), as in /Vanhaesebrouck et al. 2017
Hierarchical aggregation. Both centralized and distributed aggregation schemes have limitations in scalability in terms of the number of agents and in terms of the size of the geographic area the agents are spread over. Hicrarchical architectures are proposed to overcome these limitations by introducing multiple layers of aggregation. The lower layers typically follow a star topology with local aggregators. In wireless networks, these are often implemented at the base stations or edge servers. Aggregation at the higher layers may use a central server (Luo al. 2020 Gupta et al. 2016 Wu et al. 2023 Azimi- Silmi--Abarge Abarghouyi & Fischione, 2025), or be performed over a mesh structure {Castiglia et al. 2021 Zhang et al. 2022).
Hicrarchical architectures are often motivated by retransmission constraints, where the wide geographic distribution of agents requires the model parameters to be transmitied over long transmission paths (Gupla) et al. 2016). In wireless networks, the hierarchical structure improves the transmission quality over wireless links Wen et al., 2022) and increases the efficiency of the learning process by localizing certain operations, which conserves communication resources and reduces time (Gupta et al. 2016). Clustering of agents helps to manage the heterogeneity of the system, supports personalization (Castiglia et al., 2021 Wu et al. 2023)) and increases privacy by limiting data traffic to specific administrative regions or social groups (Zhou al.
2023
Model aggregation techniques determine how and when agents exchange and combine their models, directly impacting communication efficiency and convergence behavior. Centralized ag- gregation offers simple coordination, but creates communication bottlenecks and single points of failure. Decentralized aggregation eliminates central coordination through distributed synchronous or asynchronous consensus protocols, naturally handling dynamic topologies. Hierarchical aggre- gation scales to larger systems by introducing multiple aggregation layers that localize communication.
Trilemma. Centralized aggregation allows effective learning with accurate models and fast convergence. Under docentralized aggregation, both the convergence and the model accuracy depend the aggregation topology, and hierarchical aggregation needs complex coordination to ensure efficient learning under non-i.i.d. data. Beyond topology, aggregation timing trades communication efficiency against convergence speed. The privacy exposure depends both on the number of aggregating nodes and the required model updates: fewer aggregators reduce exposure but require trust, while frequent local updates increase data leakage.
2.4.2 Asynchronous Aggregation
The model aggregation approaches discussed above generally assume that all clients are fully synchronized in the learning process, that is, they are all aware of and, if sclected, participate in a given global itcration I. This is often an unrealistic approach. First, due to the high number of agents, which makes it is costly to manage a shared view in the entire system. Second, due to the presence of stragglers, that is, agents with limited computational capacity or slow network connectivity, which can result in the server receiving the updated model (x(t) from the agents (line [1] at different times. The common concept to describe stragglers is staleness, defined for the server the difference between the index of the current round global model
25

---

Published in Transactions on Machine Learning Research (02/2026)
update (line 12 of Algorithm D and the index of the local model (x(tz) received by straggler k. The issue of staleness is considered both for centralized aggregation and for decent learning, with the usual assumption of bounded staleness.
Asynchronous centralized aggregation. In the centralized aggregation scheme of (Xie et al. 2020 the global model is updated (line [1] at cach asynchronoully arriving client update, with weights that consider the staleness of the client. Under a non-i.i.d. data distribution, the models of straggling agents may still hold important updates for the global learning process. Therefore, (Li et al., 2022) suggests that late model updates from stragglers should also be weighted according the importance of their update for the learning process. However, updating the global model after individual client updates has two disadvantages. It increases the communication requirements and contradicts the efforts for privacy-preserving FI. (Nguyen et al. 2022). Therrffore to allow efficient model aggregation despite stragglers, (Chai et al., 2021) introduces a hicrarchical schome, where clients are dynamically clustered according to their update speed, synchronous learning is performed within the clusters, and asynchronous updates across the clusters. The same challenge is addressed in (Wei al., [2015] Nguyen et al.) 2022 Huba al. 2022), suggesting delaying the global model update until given number of asynchronous client updates arrive the central parameter server.
Asynchronous aggregation for decentralized learning. Asynchronous updates with stragglers are also addressed for decentralized learning over graph topologies. The convergence of the learning process is proved in (Lian et al. 2018 assuming bounded staleness, and averaging process that does not take this staleness into account. Instead, the weight matrix for the averaging is optimized according to the expected staleness of the neighboring nodes in Bornsttii et al., 2023). The comprehensive work (Even et al., 2024) suggests update strategies and proves convergence, considering both staleness due to computation delays and the effect of unreliable communication.
The convergence results of the asynchronous distributed learning schemes show that under bounded staleness, the convergence rate and the accuracy of synchronous systems are preserved, though the level of staleness and delayed aggregated updates may decrease the actual convergence speed.
Asynchronous aggregation techniques address the presence of stragglers, that is, agents with delayed updates due to computational or communication constraints, by relaxing synchronization requirements in both centralized and decentralized settings. Two aggregation strategies emerge based update timing: immediate aggregation processes each arriving update independently, maximizing responsiveness but increasing commmncation overhead, while batched aggregation waits for multiple updates before aggregating, reducing communication costs. These strategies can be enhanced through staleness-aware weighting (accounting for update age), hierarchical clustering (grouping agents by speed for mixed synchronous-asyycaoommn operation), or optimized consensus matrices in decentralized settings (adapting weights to expected neighbor delays). Theoretical analysis under bounded staleness assumptions shows preserved convergence guarantees with degraded convergence speed proportional staleness levels, establishing the fundamental tradeoff between synchronization flexibility and optimization tightness.
Trilemma. Immediate aggregation leads to effective learning, but compromises resource ef- ficiency and leads to privacy exposure due to the individual updates and the frequent model exchanges. Batched aggregation at the same time increases the convergence times, and potentially discarding valuable straggler information may docrease model accuracy.
2.4.3 Communication Efficiency
For collaborative learning, model paramelers or gradients must be communicated to and from the parameter server (line 3 and 10 of Algorithm [ or among the agents in a peer-to- teeerr peer or mesh architecture. Since the communication links between the agents and the central server or among the agents may be hetcrogeneous, unreliable, and costly (as described in Section (.2.3), the transmission of the information needs to be or- ganized by taking the state of the communication links and the cost of communication into account. The
26

---

Published in Transactions on Machine Learning Research (02/2026)
communication requirements of collaborative learning differ from traditional data transmission, exact, error-free information is usually not. required for learning, and the model aggregation, (11) needs the function of the inputs, rather than the individual values. These motivate the design of novel, FL-specific communication schemes, including the efficient transmission of information, the organization of medium access wireless nelworks, and client scheduling (Hellström et al. 2022
Transmission efficiency. Transmission efficiency measures the importance of the transmitted information versus the invested resources, such as bandwidth and transmission power. Bandwidth usage and learning efficiency in distributed learning be balanced through quantization, that is, the number of bits repre- senting model parameter, and model sparsification, that is, the number of model parameters transmitted (Oh et al.) 2024 Shlezinger et al. 2021 Xie et al. 2015). Similarly, in wireless networks, power control can achieve a tradeoff between energy efficiency and learning rate (Wang et al. 2020). As randomization can help the learning process, noise and wireless interference are proposed to be tuned dynamically to accelerate learning Zhang el al. 2022). To ensure that important model updates are transmilted successfully, (Liu et al. 2019 suggests importance-aware power and retransmission control.
Wireless communication for model aggregation. FL often assumes that agents communicate through wireless channels. Several works propose to utilize the superposition property of the wireless multiple access channel to allow all nodes in a wireless cell upload the model paramcters at the same time, performing the averaging step of (11 over the air in the analog @Liu et al. 2020 Hellström et al.] 2022) or in the digital [Razavikia et al. 2024 domain, including hierarchical federated learning (Azimi-A barghoouyi Fodor 2024). Over-the-air computation ensures that the communication costs are independent of the number of nodes. fully docentralized learning, the broadcast nature of the medium can be used for efficient model exchange
among neighbors (Pérez Herrera et. al. 2025
Client scheduling. To limit wireless resource usage while ensuring learning accuracy, NNishio & Yonetani 2019 Yang et al.] 2019) suggests scheduling clients with good channel quality for each global update (line 3 in Algorithm!] Dp. Recognizing that some gradient or model updates have higher importance for the learning process, combined importance and channel-aware client selection proposed in several works, with different
importance metrics (Goetz et al., 2019 Ren al. 2020; Liu et al. 2021 Leng el. al. 2022. Typically, client selection methods aim maximize the learning accuracy in one global round, under a time constraint.
Instead, client selection for minimizing the total learning time is proposed in (Chen et al., 2020 Nishio &
Yonetani 2019), balancing the time of global round and the improvements achieved.
Summary: Communication efficiency techniques exploit the unique properties of collaborative learning, the tolerance to noise, and the aggregation-oriente communication, to reduce transmission costs. They cover three complementary mechanisms: transmission efficiency, wireless-specific aggregation, and client. scheduling. These mechanisms target different. bottleneck:: transmission efficiency reduces information volume through quantization and sparsification, wireless aggregation exploits channel superposition to perform computation during transmission, while client scheduling selectively activates agents based on communication conditions.
Trilemma. Improved efficiency typically trades off learning accuracy (e.g., quantization er- ror, gradient sparsity), system complexity (e.g., synchronized wireless access, channel state information), or fairness (e.g., bias toward agents with better connectivity). Wireless over-the-air computation achieves communication costs independent of agent count but requires careful noisc management and synchronization.
27

---

Published in Transactions on Machine Learning Research (02/2026)
2.5 Privacy-preservve Collaborative Learning
collaborative learning in its native form does not provide formal privacy guarantees and is thus subject to privacy vulnerabilities. To address these vulnerabilities, the literature has developed both a rigorous framework for quantifying information Ieakage through differential privacy (DP) and pnivacy-preservng mechanisms that operate at different stages of the collaborative learning process, Considering honest-but-curion agents and servers, the two main lines of blind computations, which apply cryptographic techniques that prevent access to intermediate computations, and DP for collaborative learning, which provides statistical guarantees that limit what can be inferred from the evolution of model parameters.
As we have seen in Section 2.2.4
mechanisms are
Blind computations. If training data is not shared, Algorithms and 2 still require agents to exchange intermediate computations, which may need to be protected from information leakage. These intermediate values, such as local models, gradients, or intermediate representations, still carry information on Sp and can potentially be exploited to infer private information (Fredrikson al.) 2015 Phong et al.) 2017 Chai 20200 VFL scenarios are more subject to this information leakage since they require the exchange of
ct al. intermediate represenlations of samples, whereas horizontal FL scenarios only aggregale models or gradients [Liu et al. 2024p. One approach to prevent such an attack is to run the algorithm using blind computation techniques. They provide cryptographic guarantees that the operations on the private variables of the agents are performed without the possibility of reading them. Notable techniques in the literature (Kairouz et al. 2024] include Trusted Execution Environments (TFFs) (Mo et al. (021), Homomorphic Encryption (HE) (Chai et al., 2020), and Secure Multi-Party Computation (SMPC) (Liu et al., 2024).
2021 Liu et. al.
Differential privacy (DP). Even without access to raw data or intermediate computations, agents can observe the evolution of the global model parameters f(t), which may reveal information about S.. Prior knowledge on the training algorithm can be used to extract. private information on Sy, for example, by querying the final model Fatr) (Fredrikson et al. 2015), or querying the collaborative lcarning algorithm (Melis et al. 2019 Zhu et al. 2019). To quantily and address these risks, the concept of differentiol privacy (DP) (Dwork et al. 2006; Dwork 2006) can be applied. Formally, randomized mechanism M satisfies (c, 6)- if for all neighboring sample sets S and S' differing in one sample and all measurable outcome sets
U, we have:
PIM(S) E U] S (PP(M()) E E
(14)
where e represents the privacy budget and 5 the failure probability. In other words, DP bounds the ability of adversary to distinguish whether specific sample was included in R, with smaller e providing stronger guarantees and 8 allowing for rare violations of this bound. Originally proposed for centralized databases where S represents the database records and M a query mechanism, DP offers rigorous formal guarantees against information Icakage (Doork& & Roth 2014]. Since DP is a general-purpose privacy framework, it can be effectively adapted to any operations that involve the release of aggregate statistics (Duchi el al. 2018 Liu et al.] 2022). In machine learning, DP is widely used to protect the training process, where S represents the training dataset and M the training algorithm (c.g., SGD). Repeated qucries, such the stochastic gradient compulations, are privatized using Differentially Privale Slochastic Gradient Descent (DP-SGD) and its variants (Song et al. 2013 Bassily et al.) 2014 Abadi et al., 2016). These methods ensure that an adversary observing the final model parameters O(T') cannot reliably infer whether any specific sample (z, y)
was included in the training dataset S, through a sequence of randomized operations:
subsampling and shuffling of training data (Kasiviswanathan et al. 2011 Bassily et al. 2014 Abadi
ct al.) 2016
clipping or quantization per-sample of the gradients to bound sensitivity Abadi et al. et al.) 2021
2016 A Andrew
adding calibrated noise scaled to the privacy budget (s, 8) before model update
Dwork et al.
2006
Abadi et al. 2016)
28

---

Published in Transactions on Machine Learning Research (02/2026)
Differential privacy for collaborative learning. Applying DP to collaborative learning introduces new challenges due to the distributed nature of the data and the system. In collaborative settings, the global dataset S is partitioned across agents as S = UReKSR making it natural to protect each agent's data as a whole rather than individual samples. Consequently, agent-level DP (McMahan et al.) 2018 Kairouz et al., 202Th, that is, where neighboring datasets differ by one agent's entire local dataset Sx rather than a single sample (Dwork et al.] 2006), is typically considered. In this setting, the randomized mechanism M corresponds to the full collaborative learning algorithm (e.g,, Algorithm [ To achieve this protection, privacy- privacy-preserving primitives can be integrated at different levels depending the privacy threat model:
Local DP (LDP). A first approach involves applying DP techniques directly to the local training loop (lines @ g Algorithm 1 to privatize the gradient or model parameters 0g (t) transmitted to the aggregator, for example, using DP-SGD and its variants. These techniques are designed for the case when participants aim to protect their data from an "honest-but-curioss" central aggregator.
While LDP provides strong privacy guarantees, it has high communication (Smith et al.
2017 Duchi
& Rogers 2019) and computation demands (Kasiviswanathan et al., 2011 Choi et al.
2018 and
leads to the degradation of model estimates (Duchi et al.) 2013; 2018
Bhowmick et al.
2018 This
limited the practical adoption of LDP techniques [Bonawitz et al. 2022).
Distributed DP. A second approach to protecting the privacy of local data from the central coor- dinator is to use distributed DP techniques. These methods combine mild LDP mechanisms with server-side primitives for aggregating local statistics, which amplify this privacy level (Eyfimievski (et al.) 2003). Privacy amplisication is typically achieved through randomized shuffling and subsamn-
pling of the received LDP records (Bittau et al. 2017 Erlingsson al.] 2019 Cheu al. 2019:
Balle et al. 2020).
Adapting these distributed DP principles to collaborative learning presents specific challenges. The goal is to perform distributed training while achieving agent-level DP. However, naively distributing DP-SGD is not possible, as the server lacks sampling control over local training data and local model updates. Therefore, DP variants of FedAvg (Algorithm 11 have been proposed that balance effective
learning, privacy, and communication efficiency (McMahan et al. 2018 Agarwal et al.
2018 These
variants modify model aggregation (lines 11
12 by incorporating server-side primitives: clipping
(Andrew et al. 2021), scaling (Geyer et al. 2017 shuffling (Geyer et al. 2017
Pihur et al. 2022
and subsampling (Geyer et al., 2017 Pihur et al. 2022 McMahan et al. 2018 Tal aawaar et al. 2024] of
the received gradients or model parameters. These server-side primitives are coupled with mild local
DP mechanisms, including additive noise and gradient quantization (McMahan et al.] 2018 The current state-of tet-hhe-rt technique, Differential Privacy-Follow The Regularized Leader (DP-PTRL) [Karrouz et al. 2021 proposes different approach. It avoids sampling and shuffling by combining tree aggregation (Dwork et al. 2010 Chan et al. 2011 with correlated noise through online estimate
regularization Duchi et al. 2011
Distributed DP tochniques have boen demonstrated for large-scale foderated learning deployments
at Google (Ramaswamy et. al.) 2020: Xu et. al. 2023). Apple (Apple) 2023), and Microsoft. (Ding
let al. 2017
While most existing work focuses on privacy for centralized model aggregation, El Mrini et al. 2024 Cyffers et al.) 2024 Biswas et al.] 2025 extend privacy mechanisms to decentralized architectures with gossip learning. To enable DP in real systems, recent rescarch proposes DP solutions with transparent model aggregation (Talarr et al.) 2024 Daly al. 2024) and inference (Shumailov al., 2025). and verifiable privacy mechanisms (Daly et al. 2024).
29

---

Published in Transactions on Machine Learning Research (02/2026)
Privacy- prracciiig techniques address the gap between keeping data local and achieving true privacy. They represent two complementary paradigms: blind compulations and differential privacy (DP). These paradigns address distinct threat models: blind computation uses cryptographic primitives (TEE, НЕ, SMPC) to prevent adversaries from accessing intermediate values during computation, while DP performs calibrated randomization and provides formal statistical bounds on what can be inferred from observing the model itself, even under full observability of the training process. Distributed DP techniques achieve practical agent-level protection through server-side amplification mechanisms that strengthen mild local protections while maintaining efficiency.
Trilemma. Privacy protection trades off learning effectiveness (e.g., noise-induced accuracy degradation, bias from gradient clipping, reduced statistical efficiency due to subsampling). It also trades off system efficiency, by increasing computational overhead (e.g,, cryptographic operations, agent- and server-side processing), communication cost (e.g., verbose or repeated noisy updates under local mechanisms), and reducing robustness to asynchronicity (e.g., complex multi-agent protocols).
3 Collaborative Learning on Graph-structured Data
We have reviewed the main design choices of collaborative learning on Euclidean data in Section 2 We now extend these concepts to the case of graph-structured data. Since graph-structured data introduces unique characteristics and challenges, we first provide an overview of state-of-the-arl solutions for learning on graph-structured data in Section 3.1 Building on this foundation, we apply the same analytical structure used for Euclidean data to the graph domain. We begin by formulating the collaborative learning problem for graph-structured data in Section
3.2 then examine the different types of heterogeneity that emerge in Section 3.3 Subsequently, we structure our analysis of design choices along the same trilemma identified
in Section 2 and formalized in Figure
7 learning effectiveness in Section [.4] efficiency in Section 335] and Table summarizes the papers surveyed along these three lines.
privacy preservation in Section 3.6
3.1 Machine Learning on Graphs and GNNs
Problem definition. Machine Learning on graphs aims to develop models that operate on graph- structured data to solve tasks defined on graphs and generalize to unseen graph instances. Graphs are a fundamental data structure for modeling relationships between entities, which arise naturally in numerous domains such social networks, molecular chemistry, and networked systems. Unlike traditional data re- siding in Euclidean spaces (o.g., images, text sequences), graphs exhibit irregular, non-Euclidean structure requiring specialized methods to process them elfectively.
Formally, a graph is defined as G = (V,E), where V is the set of vertices, also called nodes, and È CCVX V the set of edges, also called links. Given a fixed indexing of the nodes V = (un....,NN), the graph structure can be conveniently encoded in an equivalen [ matrix form as an adjacency matrix E RNNN, where aij 1 = (vi, U/)) E E.
In practice, graphs often include auxiliary information. Most commonly, nodes are associated with features {H, E X'eev, organized into feature matrix X E Nxd,, where d, denotes the feature dimension. De- pending on the task, label information may also be available: node-level labels for classification (e.g., user characteristics), edge-level labels for link prediction (e.g., friend recommendation), and graph-level labels for graph classification (e.g., molecule property).
A graph sample can then be expressed as (G,y) where G= G represents the graph data its matrix form and y the label associated with the task. When learning on graph data, we consider that we observe n graph data samples drawn from an unknown distribution D, which together form dataset S = {G', y Im TE
A graph G (V,E) porresponds equivalence class of adjacency matrices E NXN, where two matrices equivalent. if they are related by permutation of vertex indices; that is, for any permutation matrix on IT E RN, we have ITATI A.
30

---

Published in Transactions on Machine Learning Research (02/2026)
Coe
Subgraph
Maltiple Gruph Instancse
Grapl, bye
Honoogenoous
Heterogenoous
Homogeucous
Heterogenooes
heterogencous
Type partition
Not upplicable
Nos applicable
No
Yeu
Possible
Yo
Tepolay
Horis.
Horis.
Horie.
Horia.
Horks.
partitioninx
Geamlactty
Suhgraph
Rio
edge-fill
wigenfrre
Suhhgraph
Rigo
Suhgraph
Fige
Not applicable
applicable
Bandling gnipi
Edge
Edge
Vertex
Variex
Vertex
Vertex
Vertex
Vertex
Not applicable
Nos upplicable
Repreentation
Improwing Learning and Infersnen Rifezziveness (Section
Dato-basd teclaniues (Section
8.4.1
Angunticcion
B3h
Regulaciantoor
Subwamping
MY
MAT 21
62 5:12 5531
Model-based tecluniques (Section
0.4.0
ge
Dewondliation
Training and Inference under aligned partitions (Section 3.1.3
Aggnessting nodle repesenntatius
BSS MF
2
Mim S1 1
2002 GD SCAd 10:3
Asgrogatiss, gropi. repessntaatane
Transder techoliqees
P153 BCC
Improwing Communication and Compuration liftidency (Nection 2.4
Aggropation patterma (Section2.cr 2.4.1
Cencralisod (Moddls)
KO0
153 8JN9
Concolland
UNS K220
Agpregation (Modeis)
Decentzallest
0211
H94
1331
(Fowbwidligg))
Decentralless
1PM FS Rihd
1841
6425
1310
Hienuciica (Middaba)
Hicrorhicl (kinbeliicgg)
P3ky 5005 Firl
Asynchronicity (Section
3.5.3
axprrpttins
Stake Messge Pussing
K341
REU 101 kml8 19:3
Communication (Section 8.5-3
Teussiiioo Ellcinncy
2022
Wirdess conomiietti
133 102 [DJ PiM) 1P33
applepttion Clisst strdtiling and topelogy managemenn
PLB
150 AMl 13dd
[22]
Privary (Section 3.6
parateters
subedding
EEST
topology
Table 2: The landscape of collaborative learning techniques on graph data organized by graph partition. The table presents the design choices and techniques employed by different approaches across various graph partitioning schemes. Each row represents a specific technique or design choice, while columns correspond to different graph partitioning strategies categorized by graph type, feature partition, and gran- ularity, Citations are placed in columns according the partitioning approach used by each work, and representative works are marked in bold.
31

---

Published in Transactions on Machine Learning Research (02/2026)
The goal is then to learn model F among class of models FF which minimizes the expected loss
[(F) == B(G.)-
(15)
Graph representation learning. Unlike traditional data residing in Euclidean spaces (e.g., images, lext sequences), graphs exhibit irregular, non- -on-Euclidean structure where nodes have variable numbers of neighbors and no canonical ordering, roquiring permutation equivariant (or invariant) representations (Bronstein al., 2021). Graph representation learning addresses this by learning mappings that embed graph data into fixed- dimensional Euclidean vector spaces Rd: while preserving both topological structure A and node attributes X. Specifically, the goal is to learn mapping that embeds vertices, edges, or full graphs into R such that the embeddings preserve both topological structure and node attributes (sce Figure 11p. This mapping typically consists of a composition of two functions. First, learnable node encoder Onode maps each node v in graph G to an embedding 2. = Φnode (G, v) E RRd that captures both structural and feature information about node v. These node embeddings {žui v E V} then serve as building blocks for edge and graph embeddings, depending on the task:
The edge embeddings can be computed by an encoder dlink (Zu, (e.g., simple concatenation). which can also be learnable (e.g,, via an MLP))
The graph embedding can be computed by an encoder on the multisct of all node representations e G)) (e.g., mean), which can be learnable (e.g., via an attention layer).
@graph (ffze, E
The complete encoder model e thus depends on the task at stake. In most models, only the node encoder is learnable, and &node is parametrized by these learnable weights 0 E O, expressed by the notation de. When labels are available, prodictor fw, parameterized by v E V is added to the output of the encoder such that F is the complete model. The problem is then to identify F or which minimizes
((F.,)) = B(G.))-D(Fy..
(16)
encoder
predictor
Node prediction
G X
Link prodiction
Onode
f coceaa
Clink
Graph prediction
Z
@graph
y
Figure 11: Graph representation learning. The objective is to learn node encoder @node that embed the graph data G = X), consisting of the adjacency matrix A E N and the feature matrix X Nxd, into a node representation matrix Z E R Nxd, These node representations are then used for predictive tasks at the node, link, or graph level, with the use of a predictor fe and, depending on the task, a link drink or graph Øgraph encoder.
Graph Neural Networks (GNNs). Graph Neural Networks represent the dominant node encoder dnode architecture. Two architectural paradigms are commonly distinguished (Wu et al. 2020
Spectral GNNs apply spectral filters the graph. They operate on the full spatial domain and a truncated spectral domain (Hammond et al. 2011)
32

---

Published in Transactions on Machine Learning Research (02/2026)
Spatial GNNs apply spatial filters to the graph. They operate on the truncated spatial domain and the full spectral domain (Kipf & Welling 2017).
Modern GNN architectures predominantly adopt the Message Passing Newral Network (MPNN) framework Gilmer et al. 2017) (see Figure [2], which is a spatial approach to define @node. To encode a node v E V in a representation Zu, an MPNN follows an itcrative process that aggrogates the information of v's L-hop neighborhood. This neighborhood, denoted as G represents the so-called L-hop ego-graph centered on node v. The iterative process is as follows:
Feature encoding:
1 + fEno(Tv)
(17)
Message Passing: for each layer I 0, O,...,,..
m(+1) (1tt2(((1 (())
(message computation)
(18)
m ((+1) s'Agg (1 (12), u ((o))))
(message aggregation)
(19)
(((+)) SUp ((lt1))(( m m+11)
(state update)
(20)
where ((v) denotes the set of 1-hop neighbours of in G, fEnc (1+1) c(l+1) tions (e.g., MLPs), and f(+1) is a permutation-ines invariant operator on multisets (e.g., sum, mean, Agg
JUp are neural func-
attention).
Final Embedding:
2u : 11 = @node (G", v).
(21)
The full encoder dø can be considered to be parametrized by 0, representing all learnable weights across message, aggregation, and update functions and across all layers = O..... L. The framework is modular: different architecture choices for fMrs, SAgg and füp define the most famous GNN architectures (e.g., GCN [Kipf & Welling) 2017), GraphSAGE (Hamilton et al. 2017), GATs (Veličković et al., 2018 Brody et al.
2022)). ). The resulting node embeddings {2u Jee can then be fed to the predictor for solving downstream
tasks.
While the MPNN framework described above represents the modern approach to GNNs, spectral GNNs were the historical approach, motivating the design of architectures from spectral graph theory (Bruna et al. 2013). Given the spectral decomposition of the symmetric normalized graph Laplacian Ż -= I- D AD= T. UAUT, where U NxN is the orthogonal matrix of eigenbasís and A diag(A1, AN) is the diagonal matrix of cigenvalues, spectral GNNs define convolution through spectral filtering operation given by
x = Ug(A)U X,
(22)
where g [0,2] -> R is spectral filter (Hammond al. 2011 However, this requires expensive eigendecom- position and filtering operations, and the decomposition must be recomputed for graph modifications. To address these limitations, practical spectral GNNs approximate spectral filters using learnable polynomial functions of the normalized Laplacian L,
o(i) Mait
Z= -
(23)
with lcarnable cocfficients for (o This polynomial formulation induces itcrative information propaga- tion, with each power I' aggregating features from (-hop neighborhoods, thereby reducing to spatial-based Message Passing Neural Networks (MPNNs). Various polynomial bases have been explored (e.g., Cheb- Nets (Defferrard al. 2016 He al. 2022). BernNet (He et al.] [202]), APPNP (Gasteiger et al. 2019). JacobiNet (Wang Zhang 2022), and GPRGNN (Chien et al. (2021)), each exploiting the recurrence rela- tions of its polynomial basis to efficiently compute g(.)) through 1-hop message passing operations.
33

---

Published in Transactions on Machine Learning Research (02/2026)
Layer
B
Aug
Apy
Apg
Agg
A1 Frp
<E3
<B3
ftp K
Layer 1
C00 Zu
Og OG",
A
(a) Ego graph G
Layer 2
2, = @o
(b) Computational tree T
Figure 12: Computations in a Message Passing Neural Network (MPNN) Φg- To infer the rep- resentation 2o of a node v, the MPNN takes as input the ego-graph G centered on illustrated in (a) for the case of a 2-layer MPNN. The inference process consists of a sequence of hierarchical steps, given by 19 and (20), which together form the computational tree T shown in (b). In practice, these operations can be efficiently parallelized to infer the representations 2, of a batch of nodes.
GNN training. GNNs can be trained to approximate the solution of 16 on empirical samples by mini- mizing the loss
((F
1
Σ ((Fv,o(G).))
(24)
|S|
(G.y)ES
Minimizing (24 is commonly done using the SGD algorithm and neural backpropagation techniques (Rumel-
hart et al. 1986 Hamilton al., 2017); often augmented with self-supervised objectives such link predic-
tion (Hamilton et al. 2017] or with contrastive learning (Veličković et al. 2019 You et al. 2020). In practice,
most approaches minimize the empirical risk (44) as tractable proxy for the true risk
16). assuming that
the resulting model generalizes well, i.e., that the generalization gap is small (Jegelka
2022
Interest for distributed systems A notable characteristic of the MPNN architecture is that both the model parameters and the computed gradients need to be propagated along the edges of the graph. MPNNs share strong similarities with distributed message passing algorithms (Papp & Wattenhofer 2022), which iteratively aggregate messages from their neighbors, update their state, and then pass on the new states to their neighbors again. This suggests that MPNNs are a natural fit for Machine Learning tasks in collaborative systems.
3.2 Collaborative Learning on Graph-structured Data Problem Definition
As pointed out for Euclidean data in Section 2 the term distributed ML often encapsulates two lechniques: parallel training on distributed machines and training private distributed data. Our settings focus on the latter, where the graph data is collected locally by different agents. For the parallel training case, we refer the reader to (Shao et al. 2024).
We now extend the collaborative learning framework introduced in Section 2.1 to learning on graph data. Two main problem settings are distinguished: learning from multiple graph instances and learning from subgraphs, shown in Figure 13
Learning from multiple graph instances. We consider set of agents K. fl...., K......K, where each agent à holds dataset Sg containing ny graphs sampled from unknown local distribution Dr. For
34

---

Published in Transactions on Machine Learning Research (02/2026)
Molecular biolegy
Health monitoring
Social network
Recommender systems
Autonomous vehicles
Agent
Agent
Agent
Agent
Node prediction
Node prediction
Link prediction
- Link prediction
Graph prediction
(a) Multiple graph instances
(b) Subgraph instances
Figure 13: Two main settings of Collaborative Learning on Graph Data. Agents hold either (a) multiple independent graph instances or (b) disjoint subgraphs of single global graph.
example, the dataset can made of numerous molecule graphs known by different labs. Usually, graph- level tasks are considered for this case, for example, predicting a property of a molecule graph (Xie et al. 2021). Each agent aims to minimize the local objective
L (F D ( ((F..o(G).))
(25)
Agents can compute local estimates of the local objectives
(F.. ne (G.))ES
Σ
((Fv.e(G),)) v).
(26)
One can notice that the local objectives in (2). with the difference here that the data distribution D; operates graph data. As all the other aspects are the same as in Euclidean settings, if the distributions D2 are i.i.d., then the FedAug algorithm can be used to effectively identify the optimal GNN parameters, the solution of (3
25
26 formulated in the same form for Euclidean data
Learning from subgraphs. We now consider the subgraph case. We consider a global graph 9 = (V,E) where each agents & K. observes a set of N₂ nodes and the edges connecting these nodes & = {(u, v) E Elu, v E V& V/ C V, forming a subgraph topology G3 EV,, Each agent collects subgraph data Ge related to its observed nodes Vk, and some labels y associated with the task. Learning for recommender systems, large knowledge graphs, social networks, large-scale infrastructures, but also learning in groups of interactive objects, falls into this category. The learning tasks are typically on the node, link, or subgraph level.
Recall that the GNN requires, for each node v, the data from the L-hop neighborhood u as formulated in The local empirical objective for agent io then expressed not on the graph data Gg, but on the graph data supported on G3 expanded by an L-hop neighborhood within the global graph. We denote this expanded topology by GR, and the corresponding data supported on it by Gt
(Fo.F)) e(Fg.o v)-
(27)
35

---

Published in Transactions on Machine Learning Research (02/2026)
If agent & does not observe GR, then it does not have the necessary data to minimize its local loss (27 This makes the collaboration of the agents that hold parts of Ge necessary, phenomenon illustrated in Figure 14
Agent
Agent ++
T
t=t+l
G v
2, = Oe (G",
Computation requiring communication
Fdge computation graph
agent fetching
A
Edge-cut between agenss
Computation requiring communication
Limit of griph expansion of
Inter-agent communication
Figure 14: Collaborative inference in subgraph. In the subgraph setting, inference becomes collabo- rative as described in Algorithm 3 illustrated here for the inference of the red node e's representation based on its I-hop ego-graph G The computational tree T shows how computations can be distributed across multiple agents, requiring the exchange of intermediate results between them, hence the term collaboralive inference.
This need for collaborative inference resembles the setting of vertical partitions described in Section 2.3.3 and leads to the following observation: collaborative learning on subgruph data borrows from the principles of Vertical Federated Learning. If we assume that (G&, y) are sampled from an unknown distribution Dk: the true local objective can be formulated in collaborative form
(F = B(G.))-D, (Fv.e(G)).)).
(28)
An effective solution to minimize the local objectives minimize the average of local objectives across agents
is to collaboratively train a common GNN Fg.0 to 9
Σ KER
FF.F..))
(29)
K
This formulation is analogous to the Horizontal FI. problem formulation 2 with the difference that the local losses La are collaborative, and thus collaborative learning subgroph data borrows from the principles of Horizontal Federated Learning.
A GNN can be trained to minimize 29 in collaborative settings by following two algorithmic directions:
With isolated inference, inference is performed on the local subgraph only. This case borrows directly from the Horizontal FI. Algorithm 1
With collaborative inference, inference requires the information exchange between the agents. This case borrows directly from the Vertical FL Algorithm 2
A possible unified implementation of federated training from subgraph data described in Algorithm 3
following the concepts from (Chen al. 2002 Wu al. 2022 Du & Wu. 22022) Chen et al., 2022 Yao al. 2023 Han et al. 2024). Tt. is round-based algorithm which outputs both the model parameters ((T)
36

---

Published in Transactions on Machine Learning Research (02/2026)
and the node representations (T) hee (line 25 at the end of the training round. The algorithm builds the centralized aggregation of Algorithm 1 where model parameters iterate (b()) and (z(() are aggregated by a central server (line (2ine In the subgraph setting, however, the server also assumnes additional responsibilities: it stores the node representation itcrates (.(v))ev (line 23 and allows the exchange of node embeddings between agents during collaborative inference (lines
and 12
In the algorithm, for agents K, GE is the local subgraph data defined on VA, the set of nodes held by the agent k. G% is the part of the ego-graph data held by agent & and related to We also need to define boundary nodes, that is, nodes that are connected to nodes held by different agents. For each agent k, the set of foreign boundary nodes is the set of nodes held by different agent and connected to a node seen by k: y = {u; 3u E Vv (u, v) E & E V V&J. The set of private boundary nodes is the set of nodes private to that are foreign nodes of other agents: pp,k = Uiexy E V J Vy,}.
A round of Algorithm 3 begins with the broadcast of common model parameters to all agents (line 20, which then initialize their local model parameters accordingly (line Two inference modes are considered:
Isolated inference: each agent performs inference solely on its local subgraph data G& (line 6
Collaborative inference: agents collectively replicate the inference process of message-passinn neural network (MPNN) in distributed fashion. This is executed iteratively over the L. layers of the MPNN. At each iteration 1, agents communicate (line and request (line 12) the 1-th layer embeddings of
their foreign boundary nodes Vy. through the server, then compute the (1+1)-thh embeddings using
the MPNN inference equations 18 19 and 22
The remaining steps are common to both inference modes. Node representations are sent to the server for aggregation (lines 17 23 Local model parameters are updated using gradient. descent based on the computed loss (line and subsequently sent to the server for aggregation (line 11
3.3 Graph Data Partition, Heterogeneity and Privacy Challenges
The homogeneity of graph data collected across agents is central to the problem setup in 25
28 However,
data heterogeneity, which is well-studied for Euclidean data in Section 2.2 also challenges colla boraaiore learn- ing in the graph setting. We now describe graph-specific heterogeneity from two angles: data partitioning and statistical heterogeneity.
3.3.1 Graph Partition Taxonomy
A key feature of distributed learning how the data is partitioned across agents. We subsequently propose characterization of the different. partitions for graph data appearing in the literature. To structure our analysis, we build on the taxonomy for Euclidean data partitions (Section 2.2.1 and adapt it to graph data, following prior efforts in (Zhang et al. 2021 He et al. 2021 Liu et al. 2025 J. We continue to discuss the two
main cases introduced in Section 3.2: (i) agents holding multiple graph instances, and (ii) agents holding a
subgraph as described in Figure 13
We examine the case where agents observe full graph instances as introduced in Section 3.2 To formalize, assume the existence of global dataset S made of full graph instances S = [[G', v')Jizz. We distinguish two ways in which graph instances are typically distributed across agents, illustrated in Figure
Multiple graph instances.
15
Horizontal topology partition analogous to the taxonomy of inter-graph FL Xie et al. 2021 or graph level FL (He et al. 2021). Similar horizontal partitioning seen in Scction 2.2.1 the index I of is partitioned across agents into local indexes In C I such as UREKIa = I, yielding dalasets for agent k:
S ((A')X')) VEI.
iIn practice, this step can efficiently computed parallel using vectorization libraries GNN such as PyTorch Geo- metric (eey al. 2025).
37

---

Published in Transactions on Machine Learning Research (02/2026)
Algorithm 3 Federated training of subgraph GNN
Require: K. {1,..., K} set of agents F0.0: GNN parametrized by to E v,0 E θ global graph topology G = (V,E), (Va subset of nodes V held by each agent, foreign boundary nodes {V,, Jeek known by each agent, private boundary nodes & hee known by each agent, boundary edges {Ex kee known by each agent, learning rate y, initial model parameters (s)(0), 0(0)) and initial node embeddings {z (0))eevv local graph data {GR]Rek held by each agent, number of training rounds T, boolean isolated inference, ollaboralive inference
Training loop for each round & .,... T do Server:
2: Broadcast current model parameters 0(t- (t--1) to all agents
3: for each agent E K. in parallel do
Local Model Initialization
Initialize Vx -t (1- such as (x(1-2) - 0(1-)), (b)(1-2) - ((1-1)
Inference Phase if isolated inference then
5:
Local Inference on Isolated Subgraph
Decompose G₁ into {Gr)ver, and compute 2o 2)) = 1) for each node V4
as
21
end if
if collaborative inference then Collaborative Inference /ee Compute (h, Jeev, as in [17 and set Ve E Vk, ho (t) hh
10:
for 1= O,..,, T- do
Send (11 vE)), to the server
11:
12:
Request {1. (t 1) from the server
13:
Upon reception, compute [8 (19 20 for each E VE
14:
end for
zo 1) t 164 for all u E Vk
15:
16:
end if
17:
Send (t
the server Loss Computation
18:
Compute &x (Pss(t--)) (1-0x(1)2))
Local Model Update
19:
(1) 0g(1 )/ 20 (1-2)) 2
ac.
(a(() + vu(1 11)
ac, 0 (t- 2
20:
Send 0& (t), (t) to the server
21:
end for
Server Aggregation
Server:
22:
B(t) + EkEK N (t),,),,(11) EbEK, (t)
23:
(Z)(1)) + Ukextzu(l- w)ee
24: end for
Final Output
25: return (TT), ((T), (2.
38

---

Published in Transactions on Machine Learning Research (02/2026)
Agent
Agent
A
Multiple graph instunces per agent
Molecular biolugy
Homogeneous heterogenenus graphs Node, link graph task
(a) Horizontal partitioning
Health monitoring
Agent
Agent
Multiple graph instamces per agent
Graph aligned berween agents
Heterngeneous graphs
Graph task only
(b) Vertical topology partitioning
Figure 15: Illustration of the different partitioning schemes for the case of multiple graph instances. Multiple graph instances can be partitioned (a) horizontally, where agents hold unrelated graph samples, or (b) vertically, where graph instances associated with the same label are divided among agents.
That is, agents possess data about distinct graph samples indexed by Ix, for example, laboratories observe distinct sets of protein graphs (Xie et al. 2021 He et al. 2022).
Vertical topology partition For each sample of graph data Gi, the topology G is partitioned into K disconnected components Oj held by different agents. The node features X are split so that agent k observes the features of the nodes G The label, usually graph-label for these cases, is held by agent k", yielding datasets:
S = (Gilez:: =
hiez-
This partitioning typical in multimodal scenarios where the graph G represents one modality k for the same sample i. These modalities are combined to solve task related to the same sample G An example is to diagnose patients based the combination of different physiological observations, represented as graphs and held by different hospitals (Dong et al. 2021). This vertical topology partition is studied for centralized training QQia et al. 2022 but has not yet been implemented in distributed training settings.
Subgraphs of global graph. We now consider setting where agents observe parts of single, usually large-scale, global graph. In such scenarios, agents have only partial local views of the global graph, making learning over isolated graph data a central challenge.
In the most general case, the global, possibly heterogeneous graph, is partitioned across agents, such as each agent & observes a local heterogeneous subgraph G. (Ve, Ex, Ok, RR). That is, each agent observes subset of nodes VA S V of a subset of types O₂ S O, and links En S of subset of relation types RR R. These may induce a restricted type and relation mapping Pk : V₁ - O2 and pk : & Ra. For example, in a citation network, we might have object types O {Author, Paper, Venue) and relation types
R (Writes, PublishedIn, Cites).
A graph is characterized by three main components: its topology, the types of nodes and relations along with their mappings, and the node features and potential labels. Accordingly, partitions can be defined along these dimensions, giving rise to the following (axonomy of graph partitions illustrated in Figures 16
Topology partition. The partition of the graph is done along the graph structure Hg = (V,E), without considering the node types, and is achieved by
Edge-cut: Nodes are divided into (possibly overlapping) sets Utex Vk. inducing local edge sets Ev.xV. ((u,v) v) e E vu,v) E V Va). For example, consider a network of IoT devices related by global graph. Telecom base stations monitor different geographical areas, resulting
39

---

Published in Transactions on Machine Learning Research (02/2026)
lel
Agent
Agent
Agent
Topolegy edge-cut
Autonomous vehicles
Full feature space
(i) Ego-graph partition
(ii) Subgraph partition
(a) Topology partitioning
Agent
Agent
Financlal Network
fully aligned
Splii feature space
Medical Data
8
Agent 2
Agent
(i) topology knowledge
Topelogy-feee
(b) Feature partitioning
Figure 16: Illustration of different subgraph partitioning schemes in homogeneous graph. In topology partitioning (a), the graph topology is divided across agents by edge cuts. Fach agent observes the complete set of node features for the nodes assigned to it. Two cases can be distinguished based on the subgraph size: ego-graph partition (i), where an agent only observes a node together with its direct neighbors, thus forming an ego-graph; and subgraph portition (ii), where an agent holds a larger portion of the graph consisting of multiple, aligned nodes. feature partitioning (b), agents share the same set of nodes but access different subsets of features. Two scenarios arise: in full topology knowledge (i), all agents know the complete set of graph links; whereas in the topology- trre case (ii), only subset of agents has access to link information.
in a different set of devices observed Zeng et al. 2022). Usually, agents also know about their forcign nodes Vf = {e; Bu E V4 vu, E E,0 E V ViJ and the related edges cut Eo == 8 xV, = ((,,,) v) E E E V4 u E Vyn} but not their features.
Verter-cut: Edges are divided into (possibly overlapping) sets Usex &x, inducing node sets VA Vi {u 3B E V, (u, E E or (v,u) E Ex). For example, device records the items browsed by a single user in marketplace, forming an interaction graph private to the device (Wu et al.] 2022 Han et al. 2024). We define the set of verter-cut, vertices which are shared between agents: Veut,k = U E V. 3K * k, U E Vel. The interactions represent the edges private to the agent, and some items represent the vertex cut between agents.
Feature partition. Analogous to vertical partitioning for Fuclidean data. seen in Section 2.2.1 the feature space X is split among agents X = # resulting in vertical feature partitioneing of the features malrix X. Specifically, for each object lype E O, the feature matrices X, is partitioned across feature dimensions: for each node v of object type o, I, (Æ,, where Io,k E X is the part of vector I, observed by agent k.. Feature partition may result topology partition as well, which can be classified as a vertical topology partition, with a vertex cut.
Schema partition. In a heterogeneous graph G, partitioning can be defined along the network schema Tg (O,R) R), by partitioning:
40

---

Published in Transactions on Machine Learning Research (02/2026)
The object types: Partitioning the set of object types O into disjoint subsets {Ox}e, ie., O U3 Oz, induces corresponding partition of the node set in the original heterogeneous graph via the inverse mapping of the type function V - O, namely V4 = O2 : ( = {v
VI 7() E Oz}. Object type based partition often leads to edge cuts.
The relation types: Similarly, partitioning the set of relation types R into disjoint subsets (R ie., R U2 RR. induces a partition of the edge set via the inverse mapping of the relation function p & > R. yielding & -+ p (Rx) te E & p(e) E RR)). Relation type based partition often leads to vertex cut.
Recommender systems
Agent
Agent
Agunt 1
Topology One feature space per
Knowledge Graph
ooo
Full feature space per ugent
Agant
Subgraph partition
(b) Type-based partition
Illustration of different subgraph partitioning schemes in a heterogeneous graph. The subgraph can be partitioned two levels: either by dividing the node set V among agents, or by partitioning the schema graph, partitioning the object types O across agents.
Figure 17:
Graph partition of topology, schema, or features can be combined freely, leading to a high number of possible combinations of data distribution across the agents. In Table 2 we list the combinations that are most prominent in the literature.
Considering homogeneous graphs, lopology partition, resulting horizontal feature partition, is the most frequently studied case. Agents may hold subgraph of the global graph due to geographic limitations, for example, in mobile networks, due to collaborating system providers in IoT systems (Zhang et al. 2021p. The other typical case of topology partition is when agents hold single node and perform inference through the ego-graph. This the typical case of interacting autonomous vehicles, robots, or drones (Pan et al.] 2023). Traffic flow forecasting (Zeng et al. 2022) location- based user recommendation (Zeng
2022), vehicle motion forecasting (Casas et al. 2020),
swarm navigation (Zhou et al.) 2021 Solodova et. al. 2025 2022) prominent tasks.
Gao et al. 2021
Blumenkamp et al.
Vertical feature partition of homogeneous graphs ariscs when agents hold partial information about the nodes, for example, spending habits of customers are known partially by e-commerce sites (Ni
let al.
2021 Chen et al.) 2024 He et al. 2024), and hospitals have partial records of patients (Fu
et al.
2024). In most scenarios, all agents hold a graph (Ni et al. 2021 Chen et al. 2024 He et al.
2024].
but alternatively, some agents may be topology-frree -free, holding only Euclidean data (Mei al.
2019
Cheung et al. 2021 Fu et al. 2024
Key examples of heterogeneous graphs knowledge graphs Chen et al. 2022 Wang al. 2024
and recommender systems (Agrawal et al. 2024 Han et al.) 2024 Liu et al. 2022 Quu et al. 2022
Yan et al.) 2024). Helerogeneous graphs can be distributed without considering the type of nodes, resulting in topology partition. Within that, subgraph partition emerges when companies observe the ego-graphs of group of customers (Han et al. 2024). The requirement to preserve node privacy, on the other hand, may lead to ego-graph partition, where one agent knows about the ego graph of single customer (Agrawal et al. 2024).
41

---

Published in Transactions on Machine Learning Research (02/2026)
Type-based partition in heterogeneous graphs is typical in knowledge graphs, for example, when agent holds a patient-sympton database, and another one symptom-disease database (Peng et al. predictor, leading to agents holding type-
2021), with a common goal of training based subgraphs. Privacy preservation under collaborating agents, for example, in health applications, can require a combination of type-based and ego-graph partition (Sun el al. 2020
3.3.2 Statistical Heterogeneity in Graphs
outlines the main forms of statistical imbalance in Euclidean data, quantily skew and distribu- tional shift, which in turn include covariate shift and concept shift. In the Euclidean setting, data is typically drawn from joint distribution D),,y). In contrast, for graph data, the feature matrix X is structured through an adjacency matrix, which together forms graph sample = (A, The coupling between topology, Teatures, and labels is captured by the joint distribution D(G, y), and brings new dimensions the possible forms of statistical imbalance in the data, illustrated in Figure 18.
Section 2.2.2
Agent
Agent
Agent
number of instances
sige
subgraphs
A
ii Subgraph
(ii) Multiple graph instances
(a) Quantity shift
Agent
Agent
Agent
Agent 2
feature distribution
topology distribution
8
(i) Feature
(ii) Structure
(h) Covariate Shift
Agent 1
Agent
Agent
Agent
Feature-label distribution
topological concept
Homophilous
Heterophilous
Feature
(ii) Structure
(c) Concept Shift
Figure 18: Different types of statistical heterogeneity in partitioned graph data. (a) Quanlity shift refers to differences in dataset size across agents sampling from the same distribution, reflected in varying (i) subgraph sizes or (ii) numbers of graph instances. (b) Covariate shift refers to differences in input graph distributions, reflected in (i) node or edge features or (ii) graph structures. (c) Concept shift refers to
differences in the mapping from input graphs to labels (colored on the figure), reflected in (i) ciature-conacee or (ii) structure concept shifts.
Quantily shiftt can be observed graph data when agents hold different. amounts of graph information. Tn the subgraph case, this can be obscrved when the size of subgraphs, mcasured by the number of odges or nodes, differs across agents. In the full graph instances case, quantity shift be observed when agents hold different numbers of graph samples.
Distributional shifts exhibit greater diversity for graph data, compared to the Euclidcan case, due to the additional information of the graph structure A, as discussed in detail in (Zhang el al. 2024). Below, we characterize graph-specific distributional shifts using the global properties (A, X,y). However, it is important note that MPNNs operate on the collection I-hop ego graphs data (G"), hee), centered cach node rather than the full graph data G (A,X), and thus experience these shifts in practice through changes in local neighborhood distributions D(G").
42

---

Published in Transactions on Machine Learning Research (02/2026)
Covariate shift. Covariate shift refers to differences in the input graph distribution D(G) across agents, while the conditional label distribution D(y G) remains unchanged. Alternatively, one can view it from the label perspective as discussed in Section 2.2.2 where D(y) and D(G | y) differ across agents. We adopt the input perspective, which is more common in the literature, and introduce graph-specific covariate shifts, feature shift and structure shift (Zhang et al. 2024
Feature shift for graph data is analogous to feature shift for Euclidean data seen in Section 2.2.1 It occurs when, given a graph topology A, the node features distribution D(X A) differs (Zhang ct al. 2024h. For example, in superpixel graphs of natural scenes (Monti et al. 2017). one agent may use grayscale images while another uses color images from the same scene. In email networks (Bojchevski & Günnemann 2018). changes in the language used across organizations can lead to different node features.
Structure shift is specific to graph data. Real-world graphs often exhibit structural diversity across regions and nodes. In collaborative scnearios it means that the topology distribution ((4) may differ significantly across agents (Liu et al. 2022).. The factors influencing structural shifts are many, often driven by hidden latent variables of the environment (Wu et al. 2018 Gui et al. 2022) or differences in data domains (Tan et al. 2024). These shifts can be present from the start or emerge when new agents or nodes with distinct structures or labels arrive. While structural shifts can be characterized through the spectrum of A (Tan et al.) 2024), three large categorics are usually considcred: Size shifts occur when graphs or subgraphs differ in the number of vertices, e.g., sentence graphs vary with text length, and it is the most discussed in the literature (Yehudai et al., 2021). Density shifts are present when the edge density, measured by different metrics (node degree, clustering coefficient), differs. This is a structure shift that could prove to be the most critical to GNN generalization capabilities (Bazhenov et al.) (023). Motif shifts occur when the base of the graph is common, but only a portion of the graph differs in terms of typical connection patterns (Wu et al. 2022
Concept shift.
Concept shift ariscs when the conditional distribution D(y G) differs between agents.
Feature concept shift in graph data is analog to concept shift seen in Section 2.2.2 It refers to changes in the concept distribution D(y X), assuming the topology A is fixed. For example, in the case of full graph instances, some superpixel graphs might have a tight correlation between color and class (e.g., blue for water), while others do not (Gui et al. 2022). In the subgraph case, an email network, agents holding emails from respectively older and younger users, may observe different vocabulary-class relations. These shifts are particularly challenging for GNNs since they assume stable feature-label correlations (Duong et al. 2019 Faber et al. 2021
Structure concept shift for graph refers to differences in D(y A) (or D(A v)) ct al. 2023 This structural concept has been historically modeled using using the Stochastic Block Model (Holland) jt al. 1983 Karrer & Newman 2011 Abbe 2018) that assumes (A (ye, yu uniform across the graph. More commonlyy, structure concept shift is connected to the well-studied principle of graph homophily (Zhu et al., 2020 Platonov et al. 2023 Fuchsgruber et al. 2025), which states that nodes tend to connect to nodes of the same labels. Constant homophily level is critical in centralized training, as GNN tends to perform gradually worse on heterophilous graphs, referred to as homophily bias (Tan et al.) (0225a). In the case of distributed graphs, differences in the graphs homophily level between agents, referred to as homophily conftict (Tan et al. 2025a lead to decreased learning accuracy.
While the distinctions between feature shift, structure shift, covariate shift, and concept shift help under- standing the challenges of learning, this calegorization is limiting for evolving graphs, and requires further research. Since MPNNs operate on overlapping L-hop ego graphs, topology A and features X are inherently coupled: when nodes join or leave network, both D(A) and D(X) change concurrently across multiple overlapping neighborhoods (Kim et al., (202)). In the subgraph case, when partition boundaries evolve (c.g., nodes reassigned across agents), agents observe different. local distributions despite unified global graph.
43

---

Published in Transactions on Machine Learning Research (02/2026)
Moreover, these shifts have heterogeneous impacts across ego graphs, since not all nodes affect the distri- bution equally when joining or leaving the network. For example, highly connected nodes, nodes with rare labels, and nodes violating local homophily levels have greater effects.
3.3.3 System Heterogeneity in Collaborative Learning over Graph Data
Collaborative learning over graphs must address device, network, and task heterogeneity, in the Eu- clidean case discussed in Section 2.2.3 However, for the inference process, both under and after learning, graph-structured data introduces new challenges that stem from the interdependence between the system constraints and the graph lopology itself. We discuss these challenges below.
Device and network constraints under collaborative inference. Unlike Euclidean federated Iearn- ing, where inference local (Algorithm [ subgraph inference inherently collaboralive and requires agents exchange node embeddings across subgraph boundaries during message passing (lines II 12 of Algo- rithm While collaborative inference also appears in vertical federated learning (Algorithm 2 graph- structured data differs in that the graph topology itself determines inference dependencies between nodes and communication patterns between agents (Figure 14 resulting in tightly coupled system constraints. Building on the device and network heterogeneities identified in Section 2.2.3 collaborative inference over graphs does not introduce entirely new aspects. Instead, the graph topology modulates and amplifies existing computational and communication heterogeneities:
Computational heterogeneity is amplified by the structure of local subgraphs. Message passing cost scales with topology: computation grows with the number of edges in 9& times the number of layers L, while memory must store boundary embeddings (he I=1.... (line 12). Consequently, ly, dense subgraphs or highly connected agents induce higher computational and memory loads, exacerbating eventual device beterogencity.
Communication heterogeneity is further amplified by the topology of inter-agent boundarics. Incom- ing and outgoing communication volumes scale with the number of foreign and private boundary nodes, Vy and Vp.k which typically correlate with the centrality of boundary nodes. These unbal- anced loads further compound the communictiio heterogeneities identified in Section 2.2.3 where unreliable links can delay or corrupt embedding exchanges. When subgraph boundaries span many agents, syuchronizing boundary node embeddings incurs additional communication overhead and delays.
The joint effects communication and computing heterogeneities induce a partitioning tradeoff. Mini- mizing edge cuts reduces cross-agent communication but may distribute load unevenly, amplifying device heterogeneity, whereas distributing computation more evenly increases cross-agent edges and consequently communication overhead. This trade-off becomes harder to manage the number of agents grows and additional system constraints are introduced. The partitioning dilemma results in system-level bottlenecks. Stragglers caused by uneven computation or communication delays slow both parameter updates and node representation propagation (line [7). Late embeddings prevent message passing from completing, directly degrading inference speed and quality.
These challenges affect the management dynamic graphs. When new nodes join, the inference topol- ogy changes, compounding existing system heterogeneity with the shifting local neighborhood distributions discussed in Scction 3.3.2
Task heterogeneity. Task heterogeneity requires the learning and inference processes to handle multiple distinct tasks within the same GNN framework (Mai et al.] 2024 Meng et al. 2021 Zhang et al. 2023 This heterogeneity can manifest across prediction levels (node, edge, or graph-level tasks) or across different application domains (e.g., citation networks and social networks) (Wang et al. 2025). The modular structure of graph representation learning, and MPNNs in particular, naturally supports this heterogeneity through separate encoding, message passing, and decoding components, as described in Section 3.1 This modularity allows agents to share common encoders while maintaining task-specific prediction heads.
44

---

Published in Transactions on Machine Learning Research (02/2026)
3.3.4 Privacy Vulnerabilities in Collaborative Learning over Graphs
Collaborative learning over graphs inherits the privacy vulnerabilities identified in Section 2.2.4 for model aggregation. The fully trained parameters, local updates, and the evolution of aggregated models remain vulnerable to model and gradient inversion. Additionally, it inherits the privacy vulnerabilities of collaborative inference seen for Euclidean data in Soction 2.3.3] Collaborative inference also presents additional privacy challenges, considering the information held by the topology itself, as discussed below.
Privacy-preserving collaborative inference. Unlike standard collaborative learning where inference is local (Algorithm [], collaborative inference over graphs requires agents to exchange node embeddings across subgraph boundaries during message passing (lines] 11 of Algorithm 3). These embedding exchanges create an attack surface as observing boundary node representations enables inference about the local subgraph structure and feature distributions held by neighboring agents. If intermediary embeddings A and features (00) are not communicated, the final embeddings Z. = -00 (2) shared across agents can still leak sensitive information (Duddu et al. 2020).. Adversaries
zv during distributed message passing as in 18].
perform inference allacks that can extract private node features 2. for nodes held by other agents (Duddu
et al.
2020), reconstruct edge existence (u, v) E & in the subgraphs of other agents Gj (Wu et al. 2022
He
ctt al.
2021
or determine whether specific nodes (Olatunji et al. 2021 Zhang ct al. 2021 or odges
He
et al.
2021
participated in the training.
The vulnerabilities discussed above assume honest- pensst-but-curriue agents that passively infer information from shared embeddings. Beyond passive observation, malicious agents could actively manipulate their local
subgraph structure or leatures to atlack inference oulcomes ZZügner et al. 2020 Sun et. al. 2022) poison
the training process (Dai et al.] 2018: Sun et al. 2022). These malicious threat scenarios fall outside the
scope of this study.
3.4 Improving Inference and Learning Effectiveness
As per Euclidean data, methods for improving the effectiveness of learning have been proposed for graph data. Table 2 shows that the proposed solutions spread over data-based and model-based techniques, as well as methods for vertically aligned data. Several solutions address learning for multiple graph instances, which is closely related to traditional FT..
Methods increase learning effectiveness aim at two objectives: designing a tighter convergence to a global solution ,², 0 of the collaborative training objective (29), tighter individual convergence to the optimal solution of the agent, 1/", ,0,", according to the local objectives 255 28 Effectiveness is challenged by the generalization gap (seen in Scction 3.1), data isolation arising from the various graph data partitions (seen in Section 3.3.1]. , or by statistical helerogeneity (seen in Section 3.3.2). Subsequently, we classify and describe the main strategies that address effectiveness, following the classification seen for Euclidean data in Scction 2.3] that is, data-based, model-basod techniques, and training and inference under aligned partitions.
3.4.1 Data-based Techniques
Data- based techniques manipulate the training data to improve convergence toward the optimal parame- ters 0²,0"" of a jointly trained GNN. For Euclidean data, these techniques typically operate on independent samples through data augmentation, controlled sampling, erogeneity (Section p. However, graph data is more complex than sets of independent samples. First, topology couples nodes, features, and labels, creating high structural diversity: nodes vary by centrality and connectivity, subgraphs differ in density and spectral properties, exhibiting high statistical heterogencity (Section 3.3.2 making stable learning challenging. Second, in subgraph partitions, receplive fields extend beyond local boundaries, creating severe data isolation as neighborhoods are cut across agent boundaries (Soction 3.3.1 Furthermore, depending on the data partition setting and the learning tasks, graph-specific data-based techniques manipulate entities of different granularity, such node features and labels, topology, entire graph instances.
local regularization mitigate statistical het-
45

---

Published in Transactions on Machine Learning Research (02/2026)
Data augmentation. The datasets S5 held locally by agents can be augmented to address three challenges: addressing the generalization gap, statistical heterogeneity, and data isolation. We distinguish methods that augment a graph instance, or augment the dataset with new graph instances.
Methods that augment a graph instance may manipulate the graph topology:
L-hop recovering techniques are applicd both for edge and vertex cut, and exchange raw node and edge information wet agents, that they complete the ego-graphs of the nodes. This simplifies the collaborative inference phase of Algorithm 3 however requires the use of cryptographic primitives
to preserve privacy QQiu et al. 2022 Agrawal et a.. 2024 Consequently, the L-hop expansion prove to be expensive and can therefore be restricted to the 1-hop neighborhood (Agrawal et al. 2024
Node rewiring modifies the existing edge set & to yield a graph (or subgraph) with a connectivity that improves generalization. This is achieved by a collaboratively trained transformer encoder- decoder to modify the spectrum, and thus the connectivity, of each local graph instance (Tan et al. 2024), introducing new frequencies that counter covariate structure shift.
Node addition techniques aim to extend the local graph by adding new virtual nodes and their associated edges using local graph data only. This can be done using generative models trained on the server-side and then communicated to agents to extend local graphs (Peng et al., 2022). Rewiring strategies using pairwise feature similarity have also been explored, allowing agents to infer plausible edges and labels. These are then integrated into a global graph the server and redistributed to other agents (Chen et al. 2024).
Other ways of manipulating the graph instances are the augmentation of features and labels. These tech- niques are also specific to graph data, since their objective is to decrease statistical heterogeneity among neighbors or ego-graphs:
Feature augmentation techniques for graph data manipulate the feature matrix X. Averaging fea- tures of neighbors and synthetic feature generation address homophily bias and feature imbalance (Pei et al. 2023 Lin et al. 2022). Classical methods developed for Euclidean data are used to inject controlled noise [Gao et al. 2024).
Label augmentation has been proposed for various learning tasks. For link prediction, one method uses local link prediction model to identify plausible links, which are treated as virtual edges, These virtual edges can then be shared across agents to enhance their subgraphs in vertex-cut partitions (Liu et al.] 2022). For node classification, label smoothing has been applied to mitigate the effects homophily bias. In this setting, a single round of distributed averaging (as introduced in Section 2.4.1) is used to propagate label information across agents (Mai et al., 2024). An alternative approach uses contrastive learning, where negative node emboddings, chosen to be close to true node embeddings, are used to sharpen representation learning. These embeddings can be sampled from other agents, such as randomly selected agents in ego-graph datasets when the number of agents is large enough GGiaretta & Girdzijauskas 2023).
Other methods augment the dataset with new graph instances. A graph generative model, trained on the server side, can be used to synthesize new graph instances on the agent side based on local graph data
(Guo al. 2023). The existing graph instances and their node labels can be combined (Wang al. 2021
to generate new plausible graph and label instances. These techniques can be applied to full graph instances or to the collection of ego graphs and node label (G%, yo) in the subgraph case (Zhang et al. 2023 Instead of generating new plausible graphs, one can focus on generating non-plausible graphs. This contrastive data augmentattonn is a frequent approach to provide good generalization capabilities to GNN (Veličković et al., 2019 You et al.) 2020). Perturbations include node dropping, edge perturbation, attribute masking, random reindexing of the features, or subgraph sampling (Veličković et al., 2019 You et al. 2020). In collaborative settings, these techniques can be applied to the local graph data for better generalization as in (Chen et al.
2021 2022
Mai et al. 2024..
46

---

Published in Transactions on Machine Learning Research (02/2026)
Local regularization. Local regularization techniques reduce discrepancies between agents in their model parameters or vertex-cut embeddings. For graph data, these methods exploit the topology by using graph perturbations for contrastive learning, regularizing based on structural properties (centrality, connectivity), or aligning shared vertex representations across partitions.
Regularizing local models is ubiquitous for FL as described in Section 2.3.1 For graph data, it can be achieved by modifying the local loss (line 18) of the agents with an additional regularization term. This additional term can make use of the augmented data techniques described above, when integrated in contrastive term against perturbed graphs (Mai et al., 2024), or loss on trimmed subgraphs (Ceyani et al. 2025).
Prototype-basel regularization is challenging since nodes can exhibit high structural varicty yet must be mapped to limited set of prototypes. Solutions, therefore, leverage graph-specific metries to construct structurally relevant prototypes. Prototypes can be stratified according to graph properties
such as node centrality (Tan et al. 2025b Kim et al.) 2025) or label influence (Tan et al. 2025b) within subgraphs, thereby mitigating noise from low-connectivity nodes while amplifying information from rare labels. The locally constructed protolypes are then aligned with global prototypes through
contrastive losses (Kim et al. 2025 Tan et al. 2025b). For full-graph instances, prototype variance can be reduced via accumulation across training iterations (Tan et al. 2024).
Regularizing the verter cut representations is specific to the subgraph case with vertex cut, and so far discussed for KG tasks only (Chen et al. 2022). Regularization terms can be added to the local loss (line 18 to decrease the variance between vertex cut representations, respectively on the agent-side
(line 15 and server-side (line 23 or decrease the similarity of the consecutive updates.
Subsampling strategies. In contrast to data augmentation, which uses more data than available locally, subsampling techniques use less. This subsampling operates at multiple levels: agents are sampled in training rounds (line 2 of Algorithm 1 and line 8 of Algorithm 3) while nodes are sampled during local inference (lines G and 13 of Algorithm 3 For graph data, subsampling serves dual purposes: it can improve learning effectivenoss through regularization or improve efficiency by reducing message-passing costs. The techniques include:
Agent selection for graph data is inspired by agent selection techniques in traditional FL, seen in Section 2.3.1 Tt can be used to mitigate the homophily conflict through server-side weighted aggregation of the models, where the weight assigned to each agent depends on the homophily level of its local dataset. Learning effectiveness is increased if agents with high-homophily graphs receive higher aggregation weights (Tan el al. 2025a). ) As a form of agent selection for vertex-cut subgraphs, Peng et al. 2021) suggests averaging zo(() only if it leads to lower local loss.
Graph subsampling has the potential to improve the model accuracy. For effective sampling in large graphs, (Ceyani et al. 2025) suggests the use of advanced generative models (Bengio et al. 2023 Training on graphs that are perturbed using subsampling techniques like random node dropping or subgraph sampling is classical technique that allows GNN to better generalize Youu et al.
2020 For collaborative learning, the same technique is applied to the local datasets in (Chen et al, 2021 2022). Node and model parameter sampling is proposed in (Shi et al., 2025) to decrease the computational and communication burden under training. In the subgraph collaborative inference case, node subsampling can be applied to the local expanded graph G5- For instance, (Liu et al. proposes to select foreign vertex boundary nodes only if their contribution improves the local loss (26). Subsampling strategics are also designed to balance efficiency and effectiveness, using greedy heuristics (Pan et al. 2023), reinforcement learning (Chen et al. 2021). or Markov chain (Pan et al.] 2023).
2021)
methods
Open challenges of data-basod techniques. In collaborative settings, graph-spocific data properties create fundamental challenges for learning and inference effectiveness that extend beyond the Euclidean case. Some challenges stem from open problems existing also for centralized graph machine learning, while others
47

---

Published in Transactions on Machine Learning Research (02/2026)
emerge specifically from the interaction between graph structure and collaborative dynamics during both training and inference. Below we identify key research directions where graph-specific data properties create unique obstacles for collaborative settings:
Towerds collaborative graph rewiring. In centralized settings, recent research has explored decoupling the observed input graph from the graph used for message passing (Wang al. 2019 Fatemi et al. 2021 Kazi et al. 2022: Topping et al. 2022 Gutteridge et al. 2023). These graph rewiring approaches augment the graph structure to neighbor information more effectively, improving model accuracy. Extending such rewiring to collaborative settings presents uiique challenges: when a global graph is partitioned across agents, rewiring decisions have to take into account conflicting constraints. For instance, adding edges between nodes held by different agents increases inter-agent communication and thus adds communication costs and privacy vulnerabilities.
Towards capluring statistical helerogeneitiess Collaborative GNN learning is challenged by structural diversity inherent to graph domains. Unlike images or text, graphs span vastly different domains, from social networks to molecular structures, that create challenging statistical heterogeneity across and within agents datasets. At the same time, graph properties are typically characterized by clas- sical network analyties metrics (e-g., centrality, degree), which do not integrate the computational scheme of MPNN (Figure 12), nor the feature information. Promising directions for more expressive characterization include adapting GNN-specific metrics from centralized training theory to collab- orative settings. For example, moving tree distances (Chuang & Jegelka, 2022) have been tied to GNN gencralization capabilitics both empirically and thcoretically (Maskey et al. 2025 Southern el al. 2025 This improved characterization could enable more informed dataa-haring strategies, such as identifying which agents should collaborate based on structural similarity rather than simple topological statistics.
Towards graph-specifie model optimization. While global model optimization has been extensively studied for collaborative learning on Euclidean data (Section 2.3.1 ensuring convergence and gen- eralization for GNNs remains poorly understood even in centralized settings (Jegelka 2022 Morris
et. al. 2024h. Collaborative scenarios exacerbate this challenge: agents with topologically dist.int subgraphs (scale-free vs. regular networks, dense vs. sparse regions) may experience fundamentally different optimization landscapes. For instance, MPNNs exhibit representation collapse along high commute-time paths (Di Giovanni et al. 2023). , causing sensitivity to vary with network structure, a problem amplified when agents cannot share data to compensate for these structural differences. De-
veloping convergence guarantees and regularization strategies that account for topology-dependent optimization dynamics in collaborative settings is promising research direction.
Towards collaborative learning on evolving graphs. Most collaborative GNN work assuines static graphs, where topology and features remain fixed during training. However, real-world graphs evolve temporally: nodes and edges appear disappear (e.g., users joining social networks, devices entering IoT systems), causing topology D(A) and features D(X) to shift concurrently (Section 3.3.2). This coupled evolution creates unique challenges for collaborative settings: when a node's neighborhood changes, its ego graph distribution D(G") shifts, but agents holding overlapping ego graphs may observe these changes asynchronously, creating temporal misalignment during collaborative train- ing. Therefore, it is an essential open challenge to establish characterization methods for temporal evolution and benchmarks for dynamic collaborative graph learning (Kmm al. 2025).
48

---

Published in Transactions on Machine Learning Research (02/2026)
Data-based techniques for graph-structured data adapt the techniques seen for Euclidean data (local regularization, data augmentation, and subsampling) to address statistical heterogencity and data isolation in partitioned graphs. However, in contrast to Euclidean data, many of the methods manipulate the graph itself, extending, regulizing or subsampling nodes, links, subgraphs, vertex-cuts or the agents themselves. Open challenges include the statistical characterization of static or dynamic subgraphs, as well the extension of emerging centralized models to collaborative settings.
Trilemma. Efficiency tradc-offs emerge subsampling reduces messago-passing costs but may discard informative boundary nodes, while topology augmentation increases communication overhead through cross-agent coordination. Privacy concerns arise methods requiring structural coordination (L-hop recovery, label smoothing, prototype sharing) expose topology and embeddings.
3.4.2 Model-based Techniques
Model-based solutions with parameter personalization and architecture personalization are proposed to find agent-optimal GNN models despite data and system heterogencity. Most modcl-basod solutions seen for Euelidean data in Section 2.3.2 have been adapted for graph-structured data seen in Table 2 However, graph-structured data and GNNs introduce distinct considerations for model personalization. Techniques for personalizing model parameters can address statistical heterogeneity different levels of gramularity (nodes, edges, subgraphs, entire graphs). Model personalization can utilize the layered cessage-passing architecture of GNNs to decouple model components.
Parameter personalization. Parameter personalization techniques seen in Section 2.3.2 have been
adapted for graph data, using graph-specific quantities to personalize model parameters:
Local adaptation strategics can be extended to GNNs. In (Zhang et al., 2023 the local model is formulated as a learnable interpolation between the global and the local GNN parameters, while (Baek et al.} 2023 Fang al. 2025: Han et al.] 2024) suggest masking the weights of the global models using learnable parameters. meta-learning framework is suggested for node-level tasks on
graph data in (Wang et al. as subgraph partitions.
2022 Han et al. 2024
where the support and query sets are defined
Hypernethwork-based techniques Shamsian et al. 2024), that defines the representation vector of agent k as the mean of node representations within the local subgraph: Ni Ever, (t).
2021 are extended for graph data in (Liang et al.
1
Cluster-based techniques can be used to cluster agents based on gradient similarities (ie et al. 2021). the agents observe a single ego-graph only, then they can be clustered based on their node embeddings. As suggested in (Qu et al. 2023). an effective way to support learning within clusters to construct virtual local graphs.
Similarities- techniques are also adapted specifically graph data. However, defining simi- larities between graph distributions is challenging. The similarity matrix Ω [wa]], can be interpreted defining a latent inder-agent graph, distinct from the local graph data of each agent. Similarity scores, inspired by (Zec et al. 2022 are evaluated for different GNN models on randomly generated subgraphs in (Baek et al. 2023). Liang et al. 2024) use subgraph representations O₁ to compute similarities between agents, which are then diffused over the inter-agent graph using learnable sheaf diffusion (Bodnar al. 2022 algorithms trained on the server.
Architecture personalization. Architecture personalization modifies the model architccture used by each agent. While standard techniques from Section 2.3.2 remain applicable, GNNs compositional message- passing architecture is particularly well-suited for flexible architecture personalization:
49

---

Published in Transactions on Machine Learning Research (02/2026)
Model decoupling. As described in Section 3.1 GNN Fv.0 can be decomposed into an encoder Φ decomposed into sublayers fEne, ((0) f(l) Agg f0g for 1=1,...,I, and a predictor fv. This layorod architecture allows for splitting the model layer-wnise into different components. The split means that one component is shared while another is personalized to the agents (Mai et al. 2024). More precisely, the shared component can either be trained on the agent side and is then aggregated periodically the server, or it is trained on the server (Meng et al., 2021 Zhang et al., 2023 using alternative optimization techniques (Meng et al.
2021 Decoupling can occur between the encoder
Φ and predictor fu (Mai et al.
2024 Tan et al. 2024]. or between the feature encoder fEne and
subsequent layers, where either the early layers are personalized, and later ones shared (Meng et al.
2021), or vice versa (Zhang et al.
2023
Local GNNs have also been pruned width-wise in (hh et al. 2025), where each agent removes local learnable weights based on a layer-specifie, learnable threshold. To mitigate severe perfor- mance degradation during aggressive pruning. a gradual rollback strategy is employed. Aggregating width-wise pruned models remains challenging and has been proposed to be performed at lower bit resolution to promote generalization across agents.
Multimodal personalization. The server can maintain M specialized GNNs paramcters {6 JMM for each graph or subgraph lype. The framework for multimodal personalization (Chen & Zhang, 2022; Smith et al. 2017) seen in Section 2.3.2 has been applied to full graph instances associated when one agent that can collect graphs of different modalities or tasks (He al. (022). In vertically partitioned subgraph settings, predictions from agents local models for the same nodes can be combined using a confidence-basee aggregation scheme known as knowledge voling (Mai et al. 2024).
Knowledge distillation. Instead of sharing knowledge through model parameters, agents can exchange node reprosentations while remaining free to choose their local architectures. This approach leverages a key properly of GNNs: they maintain a shared representation space Rd: across layers, enabling meaningful comparison of node embeddings between agents. To achieve knowledge sharing, agents add an alignment term to their local loss function. This term measures the discrepancy between local and global representations, either for vertex-cut nodes (Fu et al., 2024) or for node prototypes (Tan let al., 2024). In vertical topology-free partitions, this embedding alignment enables a useful form of knowledge transfer: topology-aware agents can distill their structural knowledge to topology- agnostic agents (Fu et al., 2024). The topology-agnostic fcature encoder fEne thereby learns to integrate lopological information indirectly, avoiding communication and synchronization overhead during inference.
Decoupling the message passing operations. An architecture personalization approach specific to GNNs the personalization of the message passing operation itself. In [Lei et al. 2023),, the message computations in (18) are adapted based on edge type. Messages along edges cut Eeut carry only partial information, for instance, when connected to stale embeddings. To compensate for this limited information, these cross-boundary messages are processed using more expressive message and aggregation functions, SMes and Jams: designed to better capture missing dependencies. In rostrast, messages over local edges (i.e., those within G₂) rely on simpler functions (Wu et al. 2019). as they benefit from more complete and up-to-date local information.
Open challenges of model-based techniques. While collaborative GNN learning has adapted many techniques from Euclidean federated learning, recent advances in graph machine learning, from expressive architectures beyond standard MPNNs to foundation models for graphs, present new opportunities and challenges for collaborative settings. Key research directions include:
Towards more advanced lopological structures. Work in collaborative settings has primarily studied standard GNN architectures (e.g, GCN (Kipf & Welling 2017), GAT (Veličković et al.) 2018 However, more expressive extensions of the MPNN framework have boen introduced under the umbrella of topological deep learning(Hajij et al. 2022 Papanarkou et al. 2024) or geometric deep learning (Bronstein et al. 2021). These architectures introduce new collaborative challenges, since
50

---

Published in Transactions on Machine Learning Research (02/2026)
partitioning higher-order structures across agents creates more complex boundary interactions than standard vertex cuts. Therefore, understanding how these structures integrate into the current collaborative learning solutions and physical network architectures is an exciting open research direction.
Towards effective layer pruning. Apart from (Wu al. 2019 Lei al. 2023 most work assumes all GNN layers are equally important for collaborative learning. However, studies have revealed this is generally not true in deep architectures, with GNNs potentially benefiting from aggressive layer pruning strategies (Kummer et al., 2025). While this concept has been exploited to alleviate communication costs in CNN-based collaborative settings (Park Joe- Wong, 2024), its application to GNNs remains unexplored. In partitioned graphs, layer importance may depend explicitly on graph topology and partitioning characteristics. Investigating which layers essent ial for collabo- rative learning, building (Lei et al. 2023) 's finding that boumdaty-procesinng layers require special treatment, could enable more efficient communication strategics.
Towords the utilization of foundation models. Most works focus on training task-specific models from scratch, maintain separate models for different domains. However, this of personalized per- domain per-task architectures might be challenged by the paradigı shift of fowndation models (FMs) currently revolutionizing ML. These generalist pre-trained models, often with billions of parameters, can be effectively adapted to downstream tasks with relatively small amount of new data, but the development FMs on graphs remains an open problem (Wang et al. 2025). Thoir integration inlo collaborative settings, where billion-arrameter models face computational, storage, communication, and privacy constraints, a research avenue.
Model-based techniques for graph-structured data adapt parameter personalization and architecture personalization strategies from Euclidean collaborative learning to address data and system heterogencity. Parameter personalization, however, needs to consider statistical properties observed by an agent, which requires new solutions. GNNs layered message-passing architecture enables fine-grained model decoupling across layers, edge types, and message-passing operations, providing richer personalization opportunities than fully-connected networks. Challenging research questions include the adaptation of advanced topological structures, optimized layer pruning, and the general question of the adaptability of foundation models.
Trilemma. Architecture decoupling improves effectiveness by allowing agents with heterogencous model architectures to collaborate, but increases communication overhead through coordination of multiple model components. Knowledge distillation and embodding alignment methods expose intermediate node representations and topology information, necessitating privacy-protection mechanisms.
3.4.3 Training and Inference under Aligned Partitioning
The vertical and vertex-cut partitioning cases introduced in Section 3.3.1 create a distinct challenge, since multiple agents hold complementary views the same graph entities (nodes or entire graphs). Unlike horizontal partitions, where each agent observes complete but disjoint samples, these partitions split information about the same entities across agents, similarly to vertical federated learning for Euclidean data (Section 2.3.3). The presence of aligned entities across agents induces the need for aggregation mechanisms that go beyond model parameter aggregation (Algorithm I cross-boundary message passing (Algorithm borrowing instead from aggregating cross-agent views (Algorithm 2
Specifically, when multiple agents generate representations for the same node (vertex-cut partitions of homogencous graphs in Figure 16 and heterogeneous graphs in Figure or for the same graph instance (vertical partitions of multiple graph instances in Figure 15). these representations must be combined to support. effective prediction. This section, therefore, organizes existing methods into three families:
51

---

Published in Transactions on Machine Learning Research (02/2026)
techniques for aggregating node representations across agents, transfer learning strategies for cross-agent adaptation without inference-time alignment, and methods for combining graph-level representations.
A Aggregating node representations. In the case of vertex-cut partitions, node representations for the same node are generated at different agents, and therefore need to be combined. Indeed, the server receives (line 17 a collection of node representation fzr, (t 2)) from each agent A E I.. So for a node cut u E cut, the different representations {ze,x(t - k have to be aggregated (line 23 Different choices exist for this aggregation step, which aim to increase the quality of the final node representation (zu (T))
Averaging node representations. The standard approach is to compute (t).), the average on K. of the representations {zv.a (t- t)) k E K}} from the different agents to obtain more generalizable
2022 Wu et al. 2022 2021 Zhang et. al. 2022 Qu el. al. 2023 Han et al. 2024 Agrawal et al., 2024 Ni et al. 2021). The averaging can be restricted to agents
representations (Chen el al.] 2021
with low local loss (Mai et al. 2024). Alternatively, the embeddings (t) can be updated by taking
small step toward 2, (1) using & gradient-based formulation, which provides more stable updates
throughout the training (Wu et al. 2021
2022 Zheng et al. 2023
Personalizing node representations. Personalization techniques return a different node representation (I) for each agent at the end of the training instead of a common one ze (T). The personalization techniques used for the model parameters (x(t 2)) in Section 2.3.2 can be applied to the node representations (4, - - to address statistical shifts for graph data and task heterogeneity. Z,,A (1) constructed by combining (t,u(t 1) and (t) using MLP (Chen et al. 2021 or learnable linear interpolation (Wang et al., 2024
Transfer learning. Rather than aggregating the node representations (z - ke) their difference between agents can be used to design transfer learning. Typically, it is assumed that a model has been trained by "source' agents 8 on a sufficient amount of vertically partitioned subgraph data, and this knowledge is transferred to a target agent t. Aligning these representations while & trains its local model allows adapt the frozen model of to I's domain. This aligument. can be achieved by training against, adversarial model (Shen et al.] 2020), trained to distinguish between source and target domains (Guan et al. 2021
using a knowledge distillation loss (Hinton et al. 2014 Mai et al. 2024
Aggregating graph representations. Lastly, in the case of vertical partition of full graph instances (sce Figure 15 multiple graphs held by different agents can characterize the same item or person. To increase the coneralization capabilities of graph-level tasks, (Dong et al., 2021 suggests averaging the graph embeddings to increase prediction accuracy, considering the specific use case of health applications.
Open challenges. Most work on vertical graph partitions focuses on basic aggregation aligned represen- tations. However, practical implementations come with open challenges, as they often include multi-modal features on one side, and have significant communication privacy constraints on the other side, leading:
Towards transfer learning for multi-modal aligned partitions. Current work on vertical partitions primarily uses simple averaging of aligned node representations. However, cross-organizationn col- laboration typically involves multi-modal heterogeneity: entities align, but features and topology differ (c.g.. banks and social platforms sharing customer data). While transfer learning (Frasca) et al. 2024 Wang et al., (025) and knowledge distillation (Tian et al. 2025) are well-studied for centralized GNNs, their adaptation to collaborative multi-modal scttings remains underexplored. A key challenge is negative transfer: structural diversity across agents means transfer may not al- ways be beneficial, requiring methods to identify when knowledge transfer is appropriate across heterogencous collaborators.
Towards efficient and private aggregation for aligned entities. Current work merges all aligned node representations uniformly, regardless of importance. However, vertex-cut partitions can involve hundreds of aligned nodes, creating substantial communication overhead and privacy exposure. For
52

---

Published in Transactions on Machine Learning Research (02/2026)
graphs, structural heterogencity means nodes vary significantly in their contribution to learning (centrality, boundary position). Additionally, aggregating node representations leaks inference data beyond model or gradient inversion attacks. Developing selective aggregation strategies that priori- tize informative nodes while minimizing leakage remains an open challenge.
Training and inference under aligned partitions adapts aggregation and transfer learning strate- gies from vertical federated learning (Section 2.3.3) to merge complementary views of the same graph eutities held across agents. Unlike horizontal partitions, where agents observe complete but disjoint samples, vertical and vertex-cut, partitions split information about the same entities across agents. For effective learning and inference, local representations are aggregated, or personalized collaboratively. Transfer learning over alignment entitics enables collaborative lcarning and isolated inference. Chal- lenging open questions include multi-modal alignment and increasing elficiency and privacy through sampling.
Trilemma. Merging aligned node representations creates substantial communication overhead, while transfer learning reduces inference-time communication at the cost of potential negative transfer when structural diversity is high. Node representation aggregation exposes topology and inference data be- yond model or gradient inversion attacks, with aligned entities creating greater attack surfaces that require privacy-potection The process of aligning nodes or subgraphs thenselves requires privacy guarantees.
3.5 Improving Inference and Learning Efficiency
While Section 2.4 reviewed general design principles to improve efficiency in collaborative learning under realistic resource constraints for Euclidean data, we now focus on how these principles translate to graph data specifically. The most important difference between the Euclidean and graph-based scenarios is that due to message passing, efficiency now needs to be considered both for training and for inference, and during training, model updates with gradient backpropagation may also communication resources. We discuss aggregation patterns, asynchronous aggregation, and communication efficiency. Table 2 shows that existing work concentrates primarily on aggregation patterns, where solutions depend heavily on the graph partitioning.
3.5.1 Aggregation over a network
The centralized, decentralized, and hierarchical aggregation patterns, introduced in Section 2.4.1 are all applicable for learning with graph data. They can be employed to aggregate the local models 0g (1) dur- ing collaborative training and to aggregate node embeddings in the message passing equation 19 during collaborative inference.
Under collaborative learning with Fuclidean data, the communication graph W for the model aggregation is determined mainly by the physical network topology, that is, by the possible communication links or paths among the agents. This is true well for collaborative learning over multiple graph instances. However, in the subgraph ego-graph partition cases, the computational graph G (or its matrix representation A) has to be taken into account as well, both for inference and for the local model update.
The coupling of the computation graph G and the communication graph given by W typically follows one of two paradigms, as illustrated in Figure 19
Communication graph derived from subgraph partitionèng of G. In this case, the communication topology is consistent with the computational one, mcaning that certain edges in g also represent communication links in W. For example, in a swarm of robots [Blumenkamp et al. 2022 Nazzal et al. 2024). 9 connects physically adjacent robots, thereby defining both computation and commu- nication topologies. In ego-graph settings, the communication graph is identical the computation graph. For edge-cut subgraph partitions, communication links correspond to edges cut between agents, as in ToT systems where agents monitor different subsets of nodes (Pan et. al. 2023 Naz-
53

---

Published in Transactions on Machine Learning Research (02/2026)
Inter-agent communication graph W
Coupled communicution- computational topology
Computational graph f
Subgraph partition of
Node assignment
(a) Communication graph derived from the subgraph partition
Node Inter-node link Agent
Inter-agent link
(b) Arbitrary communication graph
Ilustration of the coupling between computation and communication topologies in the subgraph case. The computational topology, represented by graph G, and the communication topology, represented by matrix W, can be: (a) derived from the partitioning of g between agents, such as an odge-cut partition (sec Section 3.3.1]. or (b) independent from G, representing an arbitrary commuiccation network.
Figure 19:
zal et al. 2024 Zeng et al. 2022
The node-to-agent assignments in G may be given as problem
constraints, for example, determined by agent locations (Nazzal al. 2024 Zeng el al. 2022), or optimized given the problem constraints (Pan et al. 2023 Zeng al. 2022).
Communication and computational graph decoupled. Here, computation and communication are decoupled. Typically, when peer-to-peer communications among all agents are allowed (Giaretta) Gürdzijauskas/ 2023 Qu al.] 2023) (W forming a fully connocted graph), the commuications graph can be further refined adaptively to accommodate learning elficiency (Wang et al. 2025
Typically, agents aggregate their model parameters and embeddings through the same topology. However, distinct communication topologies be employed for model and embedding aggregation, allowing the topology to differ between training and inference (Lei et al. 2023 Giaretta & Girdzijauska 2023 Guo et al.
2024 Wang et al. 2025). In hicrarchical architectures, centralized or decentralized aggregation patterns may occur for model updates and for inference al different levels of the hierarchy (Guo et al. 2024). Due to this increased flexibility, we discuss solutions for embedding aggregation and for model aggregation separately.
Embedding aggregation. In the case of subgraph partitions, embeddings h generated by different agents must be aggregated either to perform message passing (line [3 of Algorithm 3 or for vertex-cut processing (see Section 3.4.3). Embedding aggregation techniques aim to adapt to constrained communica- tion topologies while reducing both the communication cost of transmitting embeddings (line 12 and the computational cost of aggregating them (line 13 during collaborative inference in Algorithm Solutions are proposed for centralized and for decentralized aggregation topologies:
Centralized communications are the default approach inherited from FL algorithms and Algorithm 3 The server collects and redistributes embeddings and sometimes responsible for maintaining them (Liu et al. 2022 Wu
2021 2022 Yan et al. 2024).
Decentrolized communications, leading to truly decentralized inference, means that embeddings are exchanged directly between the agents. Decentralized inference the default for odgo-cut ego-graph partitions, where the communication pattern naturally aligns with the computation pattern (Pan)
& Zhu 2022 Solodova et al. 2025 Guo et al. 2024) However, decentralized inference also appears
54

---

Published in Transactions on Machine Learning Research (02/2026)
in subgraph edge-cut partitions, for example, when agents represent institutions edge servers monitoring subset of nodes (Lei et al. 2023 Pan et al. 2023 Nauzal et al. 2024 Zeng et al. 2022: Wang et al. 2023). In such cases, the communication graph can be inferred from the cut of g (Zeng ct al. 2022 or from the communication coverage area of the scrvers
(Nazaal ct al. 2024 as shown
iure 19
Agent
Agont
Agent
Agent
Agent
Agent
1-1
Z
(i) Pre-commanicating graph Expansion
(iI) Communicating stale embedding
(ii) Truncated message passing
Modifying timing of communication
Agent
Agen 2
Agent
Agent 2
T
Trimming computational
Optimizing graph partitioning
(b) Modifying computation communication topology
Agent
Agont
Agent t-141
Agent 1+1
I=t+1
[-1
A ConnonJoebolky
Pre-uggregating before communicating
(ii) Compressing the embedding space
(c) Modifying the data communicated
Figure 20: Embedding aggregation techniques. Methods focus on modifying: (a) the timing of com- munications, (b) the computation or communication topology, or (c) the data communicated
The embedding aggregation techniques for increased efficiency modify the time of the aggregation, the topology, or the content of the exchanged data, as summarized in Figure 20
The first line of work focuses on modifying the timing of embedding aggregation to reduce the communication cost using:
Pre-commanication. Communicating sufficient information before the start of the training can avoid the need for a collaborative inference. Typically, the expanded subgraph data Gj communicated to relevant agents or the server prior the first training round, so that agents perform isolated
inference Gt. proposed for centralized aggregation (Zhang al. 2022 Chen et al. 2021 2022)
through decentralized protocol (Agrawal et. al. 2024
Qiu et al. 2022). This uphill communication
55

---

Published in Transactions on Machine Learning Research (02/2026)
cost can prove worth it if the algorithm takes many training rounds to converge in the collaborative inference, ultimately outweighing it.
Stale embeddings. Letting node embeddings h. (t), requested normally in every round in the default Algorithm 3 (line 12 become stale reduces the communication cost of collaborative inference. Therefore, (4) is propased to be requested periodically in |Du & Wu 2022). To avoid affecting learning effectiveness due to this staleness, (Du & Wu 2022) derives an optimal sampling rate w.r.t. the tradeoff between a communication budget and the convergence speed of training.
Truncated message passing. Alternatively, to roduce the amount of information to be communicated in each round, (Liu et al. 2021 suggests communicating only a part of the embeddings, specifically only the later layers 1, eliminating layers that have less contribution to the learning process.
The literature focuses on modifying the computation and communication topology to reduce communication overhead. During inference, the representation bn for node is obtained by propagating and aggregating messages along the edges of the imput graph, following the procedures in [88 19 and 20 This sequence of computations induces computational tree T rooted in v, directly derived from its ego-graph G". In the case of subgraph partitioning, edge cuts in the input graph translate into inter-agent communication links in the computational tree. Reducing communication during collaborative inference thus often amounts to modifying To by:
Trimminn the computational tree Trimming the computational tree T can significantly reduce the communication needed. Tree-trimming strategies proposed in the literature (Liu al., 2021 Chen let al.
2021 Pan & Zhu 2022 Pan et al.) 2023) remove selected branches of To leading to node v, thereby reducing both computation and communication, and notably the communication when the removed branches involve inter-agent commuication. Such strategics can be determined locally by each agent. (Liu et al., 2021), applied to its local expanded lopology Gk. assumedd to be known; or centrally by a server with knowledge of the global graph G (Chen et al.) 2021 Pan & Zhu 2022
Pan al. 2023). The communication workload can be balancod, cither the global level, c.g., minimizing the maximum workload across agents (Pan al.]
2023 or at the individual level by
balancing each agent's 's workload (Liu et al., 2021 Chen et al. 2022). While trimming To reduces the computational and communication load, it also discards part of the knowledge contained in the subgraph G", potentially degrading model performance. The impact of the computational tree depth on prediction accuracy has been demonstrated in real-world applications such as robot swarm localization (Zhou et al. 2021 Blumenkamp et al. 2022) The tradc-off between efficiency and accuracy can be balanced by various techniques, including selecting nodes that contribute only marginally loss reduction (Liu al. 2021) or using reinforcement learning to guide branch removal (Chen et al. 2021). In homophilous graphs, shorteut jumps within T can be used to reach target node v in fewer communication steps (Pan & Zhu 2022
Graph partitioning. Another approach is to optimize graph partitioning to minimize communication cost shown by T Partition refinement can be achieved by reassigning nodes dynamically to agents to reduce the number of edge cuts (Nazaal et al. 2024). Node assignment can also account for other system-level constraints such as data acquisition cost or the number of agents (Zeng et al., 2022
Modifying the nature of the data exchanged can significantly reduce communication costs. This can be achieved by employing simpler architectures than the generic MPNN described in Section 3.4.3 achieved with techniques such as:
Pre-aggregaling before communicating, Embeddings can be pre-aggregated before being communi- cated, thus reducing the amount of communication. When the expanded subgraph G2 is transmitted for isolated inference, communicating the full L-hop neighborhood may still involve a prohibitive amount of node data. A practical solution is to leverage the properties of simple GNNs such as GCN (Kipf & Welling 2017) or SGC (Wu et al.] 2019), which use non-learnable fMe and sAgz:
56

---

Published in Transactions on Machine Learning Research (02/2026)
to pre-aggregate the required embeddings locally before transmission. For example, in GCN, the data of an I-hop neighborhood of a private boundary node v P. can be pre-aggregated as EneVeKnaNDe,, Tu instead of transmitting the set {x, uu E Vx, (u, v) E Ex (ao et al. 2023) This reduces the cost of communictting for GCN, and to that of the node itself for SGC. For SGC, the local inference cost can be further reduced by pre-computing the non-learnable operations (Lei et al. 2023
(-hop neighborhood to that of an (1 hh-p neighborhood
Squashing the embedding space, The dimensionality of node embeddings dæ directly impacts com- munication cost. In node-level prediction tasks, an agent can locally generate pseudo-labels Pu by applying a feature encoder fEne followed by a local predictor fv. Instead of transmitting full em- beddings, only the scalar pseudo-labels dn are sent, substantially reducing the communication load. Although these pseudo-labels do not initially incorporate graph topology, they can be refined via non-learnable diffusion algorithms (sce Section 2.4.1), which propagate labels across the graph in a topology-aware manner. Under the homophily assumption, this allows even unlabeled nodes not involved in the collaborative inference to receive meaningful predictions Vr. (Huang et al. 2021 This approach has been implemented in decentralized settings for node-label prediction, achieving minimal communication cost while maintaining competitive performance (Kaasaakii et al. 2022
Model aggregation. For model aggregation, centralized, decentralized, and hierarchical architect are proposed, very similarly to the Euclidean scenarios. The choice, however, is largely determined by the type of graph partition and by the commuicattin costs caused by the backpropagation of the gradients.
Centralized model aggregation, inherited from Algorithms and 3 is the dominant approach in cross-silo scenarios where net network constraints are moderate. Tt is commonly used for Multiple Graph Instance settings (He al. 2022), Vertical Subgraph settings (Chen et al. 202I), and, less frequently, for cross-silo subgraph cases (Yao el al.) 2023). As in Euclidean FL, periodic aggregation trades off convergence speed and communicatin cost. For Euclidean data, prior work typically assumed generic data distributions and derived convergence bounds from the local objectives ( (Stich 2019:
Yu et al., 2019). In the graph case, structural assumptions such as graph homophily allow a refined convergence characterization (Yao et al. 2023)
function of the input graphs {G* In particular, (Yao et al. 2023) uses stochastic random graph model (Keriven et al. 2020) with fixed homophily for node-level tasks to analyze convergence in the subgraph edge-cut configuration using a GCN F. Their analysis covers both i.i.d. and label-shift scenarios, showing that convergence depends on the structural shift between each G3 and the global graph G. Higher graph homophily, lower label shift, and smaller structural shift, achieved via higher-hop graph expansion, improve convergence. As the number of agents K increases, distributional shifts intensify, requiring proportionally larger hop expansions to sustain performance.
Decentrulized synchronous model aggregation is typically used in ego-graph scenarios (Olshevsky ct al.) 2025) and has been extensively analyzed for subgraph edge-cut cases (Scardapanc et al. 2020). The mixing matrix W = [was] (w)])(+,)))>>> corresponds to the adjacency matrix of the communication graph derived from the subgraph edge-cut. Convergence proofs rely on standard FL assumptions of
loss similarity and training stability (Stich 2019 Yu et al. 2019). Stability requires W to be doubly stochastic and connected. Dual-based distributed aggregation inspired by ADMM (Boyd et al. 2011) has also been proposed (Scardapane et al., 2020). Consistent with Euclidean FI., frequent aggregation accelerates convergence and improves resilience to link loss. Additional robustness to link failures is achieved in (Gao et al.) 2022) by centrally training GNN to perform distributed averaging that tolerates both link loss and partial participation. Other works propose to use differont mixing matrix at each round, subsampled from the original communication graph to limit communications. This subsampling optimizes the mixing matrix at every round using an actor- critic model, trained to minimize convergence time. During training, backpropagating the gradient 19 of Algorithm 3) requires communication between agents for collaborative inference and local gradient aggregation. This communication challenge can be mitigated (Olshvvskyi et al. (025) through batching and message piggybacking: gradients are backpropagated along the graph during
57

---

Published in Transactions on Machine Learning Research (02/2026)
each sample's backward pass with small messages, while expensive parameter consensus occurs only once per mini-batch, amortizing the communication cost. across multiple samples.
Decentrolized asynchronous model aggregation uses gossip-based protocols [Zeng al. 2023
Gia-
retta. & Girdzijauskas 2023), though the derivation of the formal convergence rates remains open problem. (Giarctta & Girdzijanskas 2023) propose asynchronously exchanging parameters at the layer level aggregating different layers at different times to enhance robustness to asynchronicity.
Hierurchical aggregation, discussed in (Guo el al. 2024h, improves the scalability of learning by com- bining local peer-to-peer aggregation with occasional centralized coordination, while collaborative inference is proposed to remain pecr-to-peer, to avoid central bottlenccks.
Beyond lopology choice, communication cost can be reduced by adjusting the liming of model aggregation:
Pre-training performs part of the training contrally before collaborative learning, avoiding commu-
nication during early training stages (Krasanakis et al. 2022 Gao et al. 2022). A hybrid approach
initializes parameters from a centrally pre-trained model (Chen et al. 2022 before fine-tuning col-
laboratively with Algorithm 3
Local SGD runs several local epochs between aggregation rounds. As in standard FL (Section 2.4.1 this trades reduced communication for slower model synchronization. This trade-off has also been studied in the subgraph setting Yao et al. 2023).
Open challenges. While aggregation techniques are well-studied, key questions remain about aligning computational and communccation topologies, establishing the viability of highly decentralized inference, and understanding the role of localized learning in GNNs leading:
Towards separating inpul, compulational, and communication graphs. Current work treats com- putational topology 9 and communication topology W independent. However, graph rewiring, that is, separating the input graph G, that is, the graph observed, from the computational graph, that is, the graph defining the node computational trees T (Figure 12 improves scalability in
centralized settings (Wang et al. 2019 Fatemi et al. 2021 Kazi et al. 2022 Topping et al. 2022: Gutteridge et al.) 2023). Further research could explore therefore collaborative solutions that jointly optimize input, computational, and physical lopologies to improve efficiency, for example by aligning message-passinng with the available communication links.
Towerds establishing viability of highly decentralized inference. Most work assumes centralized or modestly docentralized aggregation as scen in Table 2]. However, highly decentralizod scenarios (millions of agents representing one graph node each) with asynchronous, resource-constrrinee envi- ronments remain largely unexplored. Whether decentralized pressage-passingg can remain correct and stable under realistic conditions (e.g., asynchrony, heterogeneity, partial participation, or justify its complexity over simpler alternatives, remains uncertain (Scardapane al. 2020 Blumenkamp et al. 2022), and further research is needed to understand performance bounds and practical benefits.
Towards modular frameworks with topology post-processing. Current work trains end-to-end MPNNs coupling feature transformation with grssage-passing requiring substantial embedding communication. However, deep MPNNs may be unnccessarily complex for certain tasks, such as node classification on homophilous graphs (Krasanakis et al.] (022). Agents could train Euclidean models locally, then apply lightweight graph post-processing (Zhu & Ghahramani 2002). substan- tially reducing communication. Further rescarch needed to understand the limitations of this simplification (Bechler-Speicher al.) 2025), but also the possibilities brings, for example, pro- viding a way to utilize foundation models.
58

---

Published in Transactions on Machine Learning Research (02/2026)
Aggregation techniques for graph-structured data extend centralized, decentralized, and hicrarchical patterns from Euclidean collaborative learning (Section 2.4.1] to accommodate dual aggregation needs: model parameters during training and node embeddings during inference. Unlike Euclidean settings, where communication topology depends solely on physical networks, message passing couples computational topology 9 with communication topology W, either deriving W from odge cuts in G or maintaining independent topologics. This coupling enables graph-specific opti- mization: computational tree trimming removes marginally contributing branches, pre-aggregation using simple GNNs reduces transmission to lower-hop neighborhoods, and dimensionality reduction transmits pscudo-labels instcad of full embeddings. Future rescarch is needed, however, for the joint optimization of learning and network design, for the support of decettaiceet cases, and for resource-effiieent aggregation with modular learning frameworks.
Trilemma. Efficiency techniques (tree trimming, stale embeddings, pre-aggregation) reduce communcattio costs but degrade model effectiveness, with impact depending on structural proper- ties (homophilous level, critical branches of trees). Pre-communicating expanded subgraphs exposes neighborhood structure, embedding exchanges expose intermediale representations during inference, and decentralized aggregation distributes exposure across agents rather than centralizing it, requiring privacy preserving schemes.
3.5.2 Asynchronous Aggregation and Delay Control
The synchronous standard Algorithms and 3 assume synchronicity of agents for model aggregation and message aggregation. However, in practice communication and computing delays can differ from agent to agent and can also vary in time, VFnot only due to device heterogeneity, but also due to structural shifts in the graph topology. This leads to stale model parameters and embeddings, which challenge the efficiency of the learning. Therefore, solutions that address the communication delays have been explored. Conversely, asynchronicity in the form of introduced delay has also been exploited to improve the effectiveness of models.
Model aggregation under asynchronicity. In collaborative training of GNNs, as in Algorithms 1 and 3 the model aggregation is similar to traditional FL. Consequently, techniques and convergence results from Section 2.4.2 apply to GNN. With subgraph partitioning, the communication graph often matches the computation graph, making the system topology-dependent. This tics aggregation issues to asynchronous aggregation over graphs, as studied in [ian el al. 2018).
Embedding aggregation under asynchronicity. Standard GNN inference assumes synchronous message passing between nodes [18] (9) and (20), which introduces delays and requires synchronization steps. To avoid these, message passing under asynchronicity is considered, with two complementary approaches: to mitigate the effect of the delays, or conversely, to exploit delays to improve model performance.
Staleness is typically reflected in the index of the node embeddings (,, arising due to imperfect syn- chronization steps (lines 12 and 13 of Algorithm 3) during collaborative inference. Some works suggest to introduce topological delays or acceleration, that changes the hop count index I in the message aggrega-
tion 19 cither at the level of the aggregated messages me or in the neighborhood (1) GGutteridge et al.
(1)
2023
Chen et al. 2024).
The main solutions with asynchronous embedding aggregation focus on:
Modifying the collaborative inference protocol to maintain model performance despite embedding staleness. GNN training with stale embeddings is studied in (Fey et al., 2021 Peng et al. 2022) To reduce communication costs during training, these works propose to skip certain embedding transmissions and derive convergence bounds that account for the resulting staleness. Message passing under asynchronicity also addressod in (Yu et al., (024)), where the communication and processing burden of stale embeddings reduced by sequentially performing updates along pre- defined paths in the graph, referred to as update chains.
59

---

Published in Transactions on Machine Learning Research (02/2026)
Modifying the MPNN to mitigate staleness. Specifically, [8), and (20g modified to improve robustness to embedding staleness during collaborative inference. For example, Energy GNNs (Gu
et al. 2020), presented as a learnable version of diffusion algorithms on networks (see Section 2.4.1
allow for asymptotic inference that is provably robust to stale embeddings (Solodova et al. 2025
Alternatively, asynchronous MPNNs (Faber & Wattenholer, 2024 support scenarios in which a lea-
ture update an origin node triggers asynchronous updates throughout the graph. These models propose integrating the origin node's s state into the messages (18) to mitigate the effects of stale em-
beddings (Faber & Wattenhofer 2024 Another asynchronous MPNN class is proposed in (Mathys et al. 2024), where node updates are propagated outward from the origin up to distance I. (flood phase) and then back (echo phase), a process found to be more effective than standard MPNNs.
Modifying the MPNN to improve expressiveness. Variants of the MPNN have also been proposed to incorporate topological delays for enhanced expressiveness. These delays can be controlled by learnable node actions (e-g., listening, broadcasting, or remaining inactive) (Finkelshtein et al. 2024 Embedding updates may be delayed or accelerated based on topological considerations [Gutteridge ct al., 2023 Chen et al. 2024), typically by modifying the aggrogation step (19). Delays arise when
7-hop embeddings from 1-hop neighbors are used, whereas accelerations occur when embeddings
from nodes beyond 1-hop are included. These adjustments help address oversmoothing (Chen al.
2020) and oversquashing (Topping al. 2022 two expressive limitations of GNNs.
Open challenges. While asynchronous MPNN architectures are advancing, key questions remain about adapting to dynamic agent capabilities and thus dynamically changing staleness, and about handling extreme staleness regimes in prolonged-delay environments leading:
Towerds dynamic capability-aware MPNNs. Current work assumes homogeneous agents with static capabilitios. However, practical deployments involve dynamic fluctuations in computational power, memory, and bandwidth due to varying workloads and network conditions. While prior work bal- ances static capability constraints (Liu et al., 2021 Chen al. 2022: Zeng et al. 2022), dynamically adapting gressage-passingg depth, embedding dimensions, or aggregation respousibilities runtime capability changes remains unexplored. For graphs, this requires the challenging coordination of computational tree traversal across agents operating at varying speeds.
Towards aggregation strategies for eztreme stoleness regimes. Current asynchronous methods handle moderate staleness. However, prolonged delays (satellite networks, sensor networks with intermit- tent connectivity) create extreme staleness and cyclical connectivity patterns. Key challenges include helerogeneous embedding ages across nodes, trade-olfs between recent-but-distant versus stale-but- neighbor embeddings, and exploiting predictable patterns for optimization. Developing aggregation strategics that leverage temporal patterns when staleness varies by orders of magnitude across com- putational trees remains unexplored.
Asynchronous aggregation techniques for graph-structured data extend asynchronicity handling from Euclidean federated learning to address staleness in both model parameters and node embeddings. Unlike Euclidean settings, where asynchronicity affects only model synchronization, graphs require managing embedding staleness during message- message-passing. Two strategies emerge: mitigating delays through protocol modifications (skipping transmissions, sequential updates) or exploiting delays for effectiveness (topological axceleration/decelerallt addressing oversmoothing and/or oversquashing). Future research needed to handle dynamically changing staleness and emerging systems with extreme delays.
Trilemma. Skipping embeddings or using stale values reduces communication but degrades effectiveness. Specifically, deep computational trees and high-degree nodes are more sensitive to staleness. Asynchronous protocols expose temporal patterns (update timing, message sequences) and topology through sequential update paths, creating attack surfaces beyond synchronous aggregation.
60

---

Published in Transactions on Machine Learning Research (02/2026)
3.5.3 Communication Efficiency
For collaborative GNNs with topology partitioning, embedding vectors must be communicated among agents during the pusssag--passigg phase. These exchanges can occur in centralized, peer-to-peer, or mesh archi- tectures and are subject to heterogeneous, unreliable, and costly communication links. Similar to model aggregation in traditional FL, the efficiency and reliability of embedding aggregation depend on the state of the communication links and the cost of transmission.
However, communication for message passing differs from conventional data exchange: embeddings are intermediate representations, and the downstream learning task typically depends on aggregated functions of these embeddings rather than their exact values. This observation motivates the design of communication schemes tailored to collaborative GNNs, optimizing transmission efficiency, adapting embedding aggregation to wireless communication, and communication patterns to the network state.
Transmission efficiency. Transmission efficiency quantifies the utility of the transmitted embeddings rel- alive to the consumed communication resources, such as bandwidth and transmission power. In collaborative GNNs, this is particularly critical when each agent holds only a small portion of the graph, potentially a single node, as in fully decentralized settings. In (Lee et al., 2021), coding and retransmission scheme is proposed to improve the reliability of embedding aggregation, aiming maximize inference accuracy under constrained resources. The work in (Wang et al., 2025) proposes a node subsampling strategy for selecting communication edges that accommodates link bandwidth constraints, to minimize convergence time during training. Theoretical analysis for the same scenario is provided in (Gao et al. 2021). where disruptions in embedding communication are modeled by a probability of failed message passing due to wireless im- pairments or temporary disconnections. The results show that accurate inference remains possible if such stochastic failures are accounted for in the learning process.
Wireless communication for embedding aggregation. The principles of over-tho-air computation, originally proposed for FL model aggregation, have been extended to embedding aggregation in collaborative
GNNs (Gao & Gündüz, 2023 Lee et al. 2023: Gao & Gündüz 2025). In this setting, the superposition property of the wireless channel is exploited to aggregate embeddings directly during transmission, reducing communication costs and allowing simultaneous updates from multiple agents. These methods have been shown to maintain robustness against channel impairments and, in some cases, operate without channel state information. Extensions to more complex and dynamic topological structures are explored in (Foreelino et al. 2024). Practical demonstrations, such as (Blumenkamp el al.) 2022), further highlight that variable delays and unreliable communication links must be considered in system design, while resource allocation in wireless mesh networks (Wang et al. 2022 has been shown to significantly affect application performance.
Client scheduling and topology management. Client scheduling and topology management for col- laborative GNNs are mainly considered for subgraphs with topology partitioning in lcarning tasks over wide-area networks, where the message-passingg operation is costly. In (Liu et al., 2021), collaborative GNNs in wide-area. networks are considered. To reduce traffic. congestion, client selection is performed based on the importance of the communicated embeddings and the communication costs. Scheduling algorithms are designed to allow interleaved message passing and local updates. A multi-tier wide-area network with edge servers is considered in (Zeng et al. 2022 Nazzal et al., 2024). The topology of the GNN is given by the application, and these papers address the problem of GNN node assignment to odge servers and edgo-network topology design for communication-elicienn training, for both static (Zeng al. 2022) and dynamic (Nazza] et al. 2024 GNN topologies.
Most works on communcattin efficiency in collaborative GNN implementations assume that the communi- cation graph given by the possible connections between agents is identical to the computing graph given by the adjacency matrix A. This assumption is not necessary. An algorithm to find the optimal communication graph for subgraphs with topology partitioning is proposed in (Scardapanc et al. 2020). Conversely, (Wang et al. 2025) assumes fully connected communication graph (i.e., agents are free to communicate with any- one in a peer-to-peer network) but samples communication topology at each round. The authors design an
61

---

Published in Transactions on Machine Learning Research (02/2026)
adaptive algorithm that samples communication topologies adaptively throughout training to accommodate bandwidth constraints while maintaining fast. inference and training.
Open challenges. Existing work has started to address the challenges of communicating embeddings and model parameters over unreliable or costly communication channels or transmission paths. However, since both inference and learning involve communication, several interesting research questions remain open the co-design of communication and computing, in utilizing the broadcast and multicast capabilities of the wireless channel, and in ensuring safe learning and inference despite the communication impairments leading:
Towards co-designing communiicaton and computing. While recent methods consider the effects of unreliable communication channels (Lee et al. 2021 Gao el al. 202T), they treat communication and computation topologies independently. Co-designing these topologies while explicitly accounting for channel quality represents an avenue for further efficiency improvements. Additionally, existing embedding aggregation methods operate on static neighbor selection policies and could be extended to dynamically exploit time-varying wireless channel quality by scheduling selecting neighbors with temporarily favorable channel conditions.
Towards communicalion-nnali inference. The wireless medium itself, with its superposition and broadcast properties, natively supports manyyto-one and one-to-many communications These prop- erties could allow efficient embedding and model aggregation in a single operation among clusters of agents or agents connecting to the same base station or edge computing server. While initial results exist in (Fioreliioo et al. 2024), further research is still needed to fully exploit these network environment properties through complex topological structures beyond simple graphs.
Towards safe training and inference. Training and inference on graph data require communication in both phases. Consequently, communication impairments deteriorate inference quality even when the model itself is optimal, potentially leading unsafe decisions. This raises the question of whether safety requirements should be incorporated into the learning process or considered only during inference, design choice that requires further investigation.
Communication efficiency techniques for graph-structured data adapt transmission opti- mization and topology management from Euclidean collaborative learning, but the importance of communication efficiency is increased due to the need for both message passing for inference and for backpropagation during training. Most results address learning offectiveness and efficiency directly, or suggest solutions that utilize the broadcast. and superposition properties the wireless channel. Challenges remain to increase effectiveness by additional network-specific solutions, but also ensure safe decisions despite information loss both at inference and learning.
Trilemma. Coding and retransmission schemes improve reliability under channel impair- ments but increase transmission overhead, while topology subsampling and neighbor selection reduce communication costs but risk slower convergence or degraded inference effectiveness. Over-the-air computation and wireless transmission expose intermediate embeddings to any eavesdropper within radio range.
3.6
Privacy Preservation Techniques for Graph Data
A high variety of privacy-preserving techniques are proposed for distributed graph data, aiming to protect training and inference data from honest-but-curione agents and central servers. These techniques address the vulncrabilitics identified in Section 3.3.4 model parameter leakage from collaborative training, embedding, and lopology leakage from collaborative inference. Anonymization has to larget three main levels: (i) model parameters, (ti) exchanged node embeddings, and (iti) exchanged edge information. Many these techniques build on the blind computation methods, and L.DP introduced in Section [.5] As shown in Table [ privacy preservation is an important topic for collaborative GNNs, with most of the works focusing on cross-silo applications, with cases handling personal information, like health data user preferences.
62

---

Published in Transactions on Machine Learning Research (02/2026)
Anonymizing model parameters. To address the inherited vulnerabilities from collaborative learning (Section 2.2.4 model parameters or gradients exchanged under GNN training can be anonymized directly with the techniques proposed for FL in Section 2.5 (Liu et al. 2022 Wu et al. 2022 Yao et al. 2023 Yan ct al. 2024p. These techniques can be further refined to improve the stability peei-pprvvvcyy trade-off, such as adaptive noise scaling (Wu et al. 2022,, in which the injected noise magnitude is adjusted according to the gradient norm to maintain a consistent perturbation throughout training.
Anonymizing exchanged embeddings. As discussed in Scction 3.3.4 collaborative inference over graphs requires exchanging intermediate embeddings across subgraph boundaries, creating attack surfaces for feature extraction and membership inference. Early works like (Chen et al. 2021) incorrectly assumed that if local computations are performed entircly in isolation, including local model updates and isolated inference, then the raw node features X2 would be inherently protected from privacy leakage. However, it is now clear that even when only intermediate embeddings h are shared and aggregated, they must still be protected through some form of differential privacy (Chen et al. 2024).
Blind compulation methods securely aggregate embeddings (e.g., computing fage in 19 or average representations (t) without revealing their values. For example, secure multiparty computation can be applied to jointly perform specific GNN functions such as aggregating fAgg 19] or computing matrix multiplications (Wang et al., 2023). Due to their high computational cost, these methods are often applied only a subset of layers (Chen et al. 2021), with the remaining computations delegated to the server in split-learning setup. Homomorphie encryption offers another option, allowing the server to aggregate results while keeping them encrypted end-to-end (Ran et al. 2022
Peng et al. 2023), using techniques such as secure multiparty computation (Chen et al. 2021 or homomorphic encryption for server-side aggregation (Ni et al. 2021
LDP techniques can be applied directly to the communicated vectors 2n or he The simplest approach adds Gaussian noise the vectors ęZhang et al. 2023 Lee et al., 20033 Ni et al. (021))
More advanced methods adapt classical LDP protocols to the GNN sctting. These generally follow
pipeline of noise injection, vector clipping, dimension subsampling, and privacy budget adjustment.
Noise can follow Laplace distribution (Wu et al. 2022
2021
) a Gaussian distribution (Chen et al.
2021), or a Bernoulli distribution (Zheng et al. 2023 Pan et al. 2023). Clipping may be applied to
vectors or gradients (Wu et al. 2022 2021 Chen et al. 2021: Liu et al. 2022: Yan et al.
2024) and
can be combined with stochastic quantization (Lin et al.
2022 Zheng et al. 2023
Pan el al. 2023
Dimensionality reduction can be performed by random subsampling (Lin al. 2022
Pei et al.
2023; Zheng et al.] 2023 Pan et al.] 2023) or top-k sampling (Chen al.) 2022
In some cases,
these operations are unified through complex noise functions such piecewise noise injection (Pei et al. 2023 Wang et al. 2019). The privacy level can also be balanced against other objectives or system constraints. The noise added in LDP is known for degrading learning effectiveness (Lee) (et. al. 2023). mitigated while aggregating high degree nodes (Guo el al. 2024). Therefore, adaptive noise strategies propose to scale the noise by the gradient norm during training to stabilize learning (Wu ct al. (022). When communication occurs over wircless channels, the tradc-off between signal- Lo-noise ratio and DP can be jointly optimized (Lee al.; (023). LDP can also be applied to
node labels in transfer learning with adversarial losses to protect the training data of the source
agent Guan et al. 2021)
Anonymizing edge (or topology) information. Beyond node feature extraction, adversaries can ex- tract existing edges from shared embeddings (Section 3.3.4), threat known as link stealing (Pei et all, 2023). This represents a scrious privacy attack when the existence of the edge itself represents sensitive information, or when the edges encode most of the information, as is often the case in helerogeneous graphs 2024). Therefore, the edge or topology information needs to be protected in many scenarios. The concept of Edge Local Differential Privacy (Edge LDP) formalizod 2022 Han et al.] 2024 Lin et al. 2022) and is extended to heterogeneous graphs in (Yan)
[Luu et al., 2022 Wu et al. 2021 2022 Yan et al.
in (Wu et al.
et al. 2024), providing privacy guarantees for graphs differing by a single edge In some cases, protection
63

---

Published in Transactions on Machine Learning Research (02/2026)
also covers statisties the edge set, such degree distributions Pan al. 2023). Protection strategies build both on blind computations and on LDP.
Blind computations can be employed for praneccyppreeuiing subgraph expansion. Each agent obtains its L-hop expansion Gg of the local graph using blind computation techniques. The process begins with agents collectively running private set intersection (PSI) protocol to identify the vertex cut mathealVest between agents. Given the knowledge of DV,, each agent collects and communicates the missing expanded subgraphs, typically via central server (Zhang et al. 2022 Chen et al. 2021
2022 Agrawal et al. 2024 Qiu el al. 2022 Blind computation techniques can also be applied to
aggregate useful graph statistics, such as node degree (Pan et al. 2023 using methods such as secure group aggregation (Bonawitz et al. 2017 or zero-knowlodge protocols (Goldwasscr et al. 1985)
Such statistics may be required to compute auxiliary losses, for example, fairness losses ( (Agraaal
et al.
2024. or may be used by the local learning algorithm (Pan al. 2023).
Standard LDP
Dwork & Roth 2014) can be applied to edge information in set form E Yan et al, 2022 or in adjacency matrix form A& (Han et al. 2024 Qiu et al. 2022] For exam- 2021 projects A& into a lower-dimensionall Gaussian random subspace. However, introduces topological noise, which is known to degrade GNN performance even low noise
2024;
Lin et al.
ple,
[Zhang et al.
LDP
levels
Zügner al.. 2020). Excessive noise can densify the graph, leading to oversmoothing (Chen
ct al.
2020).
Mitigation strategies include regularization (Lin et al. 2022), degree presorvation (Yan allocating part of the privacy budget to maintain adjacency sparsity (Qiu et al.
et al.
2024
2022].
Protecting edge information is often formulated alongside protecting the raw feature XR, thus protecting the local graph data Gg Xx). The concept of nodo-level privacy is introduced informally for the ego-graph case (Guo et al., 2024), consisting of protecting the raw features of nodes alongside the 1- hop topology typical of ego ago-graph partitions. This concept is further extended to heterogeneous graphs, specifically focusing protecting nodes that correspond to users (aan et al.) 2024). Other works focus protecting collections of ego-graphs sampled from G₁ (Zhang et al., 2023) Wu et al. 2021 2022). Subgraph- level privacy extends this protection to entire subgraphs G5 obtained from aggregating the private ego-graphs of the agents (Guo et al. 2024). Techniques that anonymize Aş together with X₁ have been proposed in the following forms:
Blending ego-graphs logether. The embeddings of ego-graphs are averaged across all ego-graphs subsampled from Gg in the same batch, respectively between nodes at the same depth in the com- putational tree, to anonymize the ego-graphs and their embeddings (Zhang et al, 2023
Inferring pseudo-edges. This common technique is used for anonymizing As for heterogeneous graphs, usually in the vertex cut case (Liu et al.) 2022 Wu et al.) 2021 2022 Yan et al. 2024 It consists of constructing unobserved yet highly probable edges based on node representation simi- larity, and adding them to the observed edges in Gk.
In hierarchical setups (e.g., device and cenveal server) Guo et al. 2024 considers privacy protections between higher-rank and lower-rank entities as well as among peers of the same rank. The above anonymization techniques, focusing on model parameters, embeddings, and topology information, can then be selectively combined depending on the nature of the data exchanged at each hierarchy level.
techniques for collaborative GNNs operate at the intersection of two research domains: privacy mechanisms for GNNs and privacy-preserving collaborative training and inference. This convergence presents unique challenges and opportumities, as collaborative settings introduce multi-agent dynamics that both complicate and polentially enhance privacy guarantees, leading:
Open challenges.
Towards multi-agent privacy amplification. Most work applies LDP locally to individual training or inference data. However, LDP degrades learning effectiveness by compressing communicated
64

---

Published in Transactions on Machine Learning Research (02/2026)
information (Smith et al. 2017 Bonawitz et al. 2022). Privacy amplification (Section 2.5 offers
an alternative: agents sequentially amplify protection guarantees along collaborative computation chains rather than applying hard local anonymization. This approach fits naturally with sequential MPNN operations and could limit information loss while maintaining privacy guarantees, but is still unexplored for collaborative graph settings.
Towerds integrating privacy with defense mechanisms. Current anonymization techniques protect against honest-but-curion agents (Section 3.3.4) but are insufficient against byzantine malicious agents who actively manipulate graph structure or features. Privacy mechanisms are often treated as a first-line defense against attacks such as link stealing (Pei el al., 2023): embedding extraction (Chen jet al. 2024), and graph rewiring (Chen et al.) 2024). However, how privacy mechanisms can be combined with additional defenses (e.g.; embedding filtering (He et al., 2024 against sophisticated adversaries remains unclear, since privacy guarantees often hide the very signals needed for detecting these ad adversarial threats, calling for unified frameworks that jointly address honest-but-curious and malicious threat models. For instance, mitigating adversarial edge injection may require integrating verifiable private mechanisms (Daly et al. 2024 to ensure graph intogrity without sacrificing privacy.
Privacy-pie prraccyirrseriing techniques for graph-structured data adapt blind computation and local differential privacy from Euclidean federated learning (Section 2.5 to protect not only the model parameters, but also the intermediate embeddings, and the topology information. This three-level protection requirement is amplisied by repeated embedding exchanges across subgraph boundaries during multi-layer peesagg-passing, creating cumulative exposure particularly severe for boundary nodes with many cross-agent connections. Additionally, graph-specific techniques are proposed to randomize edge and subgraph data. Since privacy preservation affects both elficiency and effectiveness heavily, further research needed to fit the methods to the computation chains of GNNs and to effectively combine them with similarly costly security mechanisms.
Trilemma. Blind computation preserves data locality but is computationally heavy; LDP-based methods offer formal privacy guarantees for embeddings and edges but can degrade message- passing quality and inference accuracy. Edge LDP, in particular, introduces topological noise that may cause oversmoothing through graph densification, requiring mitigation strategies that can further constrain model effectiveness.
4 Conclusion
Collaborative learning has unlocked access to previously unavailable distributed data sources, leading to the success of federated learning and decentralized learning on Fuclidean data, and more recently, to emerging solutions for graph-structured data. Extending collaborative learning to graph-structured data opens up opportunities that exceed those of centralized graph ML through its ability to mine data across domains ranging from sensor networks to molecular biology.
With this survey, we systematically map the design choices from Suelidean-structuree to graph-structured collaborative learning. We first consolidate the foundational principles of collaborative learning on Euclidean data (Section 2), then extend this mapping to graph-structured data (Section 3) For both cases, we formulate the fundamental problem, discuss the challenges of data and system hetcrogeneity, and analyze how the trilemma of effectiveness, elficiency, and privacy preservation manifests in the design choices for addressing these challenges. Specifically for graph-data, identify emerging research challenges that define its future development.
Our analysis reveals that collaborative learning on graph-structured data has given rise to a nascent, yet rich, landscape of problems and solutions. This landscape can be broadly divided into two paradigms: isolated inference and collaborative inference. Notably, collaborative inference emerges as a more central paradigm for graph-structured data due the message passing mechanism of GNNs. Beyond challenges that persist in Euclidean collaborative learning, combining distributed relational data with collaborative training and/or
65

---

Published in Transactions on Machine Learning Research (02/2026)
inference introduces unique constraints that require novel solutions. This includes the joint consideration of topologies, communication, privacy, and heterogeneities.
By systematically bridging collaborative learning on Euclidean and graph-structured data, this survey es- tablishes a coherent vision to support researchers in developing the next generation of collaborative graph learning methods and enable their practical impact across scientific and industrial domains, leading societally important breakthroughs in drug discovery, healthcare, sustainable industries, and smart cities.
Acknowledgement
This work was supported in part by Vinnova, Sweden's 's Innovation Agency, under grant no. 2024-00648.
References
[1] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Dya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In ACM SIGSAC Conference on Computer and Commanications Security, 2016.
[2] Emmanuel Abbe. Community detection and stochastic block models: Recent developments. Journal of Machine Learning Research, 18(177):1-86, 2018
3 Naman Agarwal, Ananda Theertha Suresh, Felix Xinnan X Yu, Sanjiv Kumar, and Brendan McMa- han. epSGD: Communication-efiiiiee and differentially-private distributed SGD. Advances in Newral Information Processing Systems, 31, 2018.
[4] Nimesh Agrawal, Anuj Kunar Sirohi, and Sandeep Kumar. No prejudice! Fair federated graph neural networks for personalized recommendation. In AAAI Conference on Artificial Intelligence, 2024.
55 Galen Andrew, Om Thakkar, Brendan McMahan, and Swaroop Ramaswany. Differentially private learning with adaptive clipping. Advonces in Neural Information Processing Systems, 111117755-14455 2021.
66 Apple. Private cloud compute. Tochnical report, Apple, 2023.
7 Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary, Federated lcarning with personalization layers. 19Xiv:1912.00 00818, 2019.
88] Seyed Mohammad Azimi-Abarghone and Carlo Fischione. Multi-layer hierarchical federated learning with quantization. arXiv:2505.08.1 2025.
99] Seyed Mohammad Azimi- Abimi-Abarghouyi and Viktoria Fodor. Scalable hierarchical over-the-air federated learning. IEEE Transactions on Wireless Communécations, 2024.
[10] Stefan Baack. A critical analysis the largest source for generative ai training data: Common crawl. In ACM Conference on Fuirness, Accountabillty, and Transparency, 2024.
[11] Jinheon Back, Wonyong Jeong, Jiongdao Jin, Jachong Yoon, and Sung Ju Hwang. Personalized subgraph federated learning. In International Conference on Machine Learning, 2023.
[12] Borja Balle, Peter Kairouz, Brendan McMahan, Om Thakkar, and Abhradeep Guha Thakurta. Privacy amplification via random check-ins. Advances in Neural Information Processing Systems, 00:422344334 2020.
[13] Raef Bassily, Adam Smith, and Abhradeep Thakurta. Private empirical risk minimization: Efficient algorithms and tight error bounds. In IEEE Anauul Symposium Foundations of Computer Science, 2014.
[14] Mayank Bawa, Hector Marcia-Molina, Aristides Gionis, and Rajeev Motwani. Estimating aggregates on peer-to-peer network. Stanford InfoTab, 2003.
66

---

Published in Transactions on Machine Learning Research (02/2026)
[15] Gleb Bazhenov, Denis Kuznedelev, Andrey Malinin, Artem Babenko, and Liudmila Prokhorenkova. Evaluating robustness and uncertainty of graph models under structural distributional shifts. Aduances in Neural Information Processing Systems, 36:75567 75594, 2023.
[16] Maya Bechler-Speicher, Ben Finkelshtein, Fabrizio Frasca, Luis Müller, Jan Tönshoff, Antoine Sir- audin, Viktor Zaverkin, Michael M. Bronstein, Mathias Niepert, Bryan Perozzi, Mikhail Galkin, and Christopher Morris. Position: Graph learning will lose relevance due to poor benchmarks. In Interna- tional Conference on Machine Learning, 2025.
[17] Florence Bénézit, Vincent Blondel, Patrick Thiran, John Tsitsiklis, and Martin Vetterli. Weighted gossip: Distributed averaging using non-doubly stochastic matrices. In IEEE International Symposium on Information Theory, 2010.
[18] Florence Bénézit, Alexandros G Dimakis, Patrick Thiran, and Martin Vetterli. Order-optimal consen- sus through randomized path averaging, IEEE Transactions on Information Theory, 56(10):5150-51) 2010.
[19] Yoshua Bengio, Salem Lahlou, Tristan Deleu, Edward J Hu, Mo Tiwari, and Emmanuel Bengio. Gflownet foundations. Journal of Machine Learning Research, 24(210):1-55, 2023.
[20] Dimitri Bertsekas and John Tsitsiklis. Parallel and Distributed Computation: Numerical Methods.
Athena Scientific, 2015.
[21] Dimitri P Bertsekas. Incremental gradient, subgradient, and proximal methods for convex optimization:
A survey. Optimization for Machine Learning, 2010(1-38):3, 2011.
[22] Abhishek Bhowmick, John Duchi, Julien Freudiger, Gaurav Kapoor, and Ryan Rogers. Protection
against reconstruction and its applications in private federated learning. arXi::1812.0088 2018.
[23] Tlai Bistritz, Ariana Mann, and Nicholas Bambos. Distributed distillation for on-device learning. Aduances in Neural Information Processing Systems, 33:22593 22604, 2020.
[24] Sayan Biswas, Davide Frey, Romaric Gaudel, Mnri-Maric Kermarrec, Dimitri Lerévérend, Rafacl Pires, Rishi Sharma, and François Taïani. Low-cost privacy-preservinn decentralized learning. In Proceedings on Privacy Enhancing Technologies Symposium, 2025.
[25] Andrea Bittau, Úlfar Frlingsson, Petros Maniatis, Πlya Mironov, Ananth Raghunathan, David Lie, Mitch Rudominer, Ushasree Kode, Julien Tinnes, and Bernhard Seefeld. Prochlo: Strong privacy for analytics in the crowd. In Symposium on Operating Systems Principles, 2017.
[26] Vincent D Blondel, Julien M Hendrickx, Alex Olshevsky, and John N Tsitsiklis. Convergence in multiagent coordination, consensus, and flocking. In IEEE Conference Decision and Control, 2005.
[27] Michael Blot, David Picard, Matthieu Cord, and Nicolas Thome. Gossip training for deep learning. In Conference on Neural Information Processing Systems, 2016.
[28] Jan Blumenkamp, Steven Morad, Jennifer Gielis, Qingbiao Li, and Amanda Prorok. A framework for real-world multi-robot systems running decentralized GNN-based policies. In IEEE International Conference Robolics and Aulomation, 2022.
[29] Cristian Bodnar, Francesco Di Giovanni, Benjamin Chamberlain, Pietro Lio, and Michael Bronstein. Neural sheaf diffusion: topological perspective on heterophily and oversmoothing GNNs. Advances in Neural Information Processing Systems, 35:18527 18541, 2022.
[30] Aleksandar Bojchevski and Stephan Günnemann. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. In International Conference on Learning Representations, 2018.
[31] Kallista Bonawitz, Peter Kairouz, Brendan McMahan, and Daniel Ramage. Federated learning and privacy. Communications of the ACM, 19(5):87-114, 2022.
67

---

Published in Transactions on Machine Learning Research (02/2026)
[32] Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy-preservnng machine learning. In ACM SIGSAC Conference on Computer and Commumications Security, 2017.
[33] Marco Bornstein, Tahseen Rabbani, Evan Wang, Amrit S. Bedi, and Furong Huang. SWIFT: rapid decentralized federated learning via wait-free model communication. In International Conference Learning Represenlations, 2023.
[34] Léon Bottou. Largo-scale machine learning with stochastic gradient descent. In Proceedings of Com- pulational Slalistics, 2010.
[35] Léon Bottou and Olivier Bousquet. The tradeolis of large scale learning. Advances in Neural Infor- mation Processing Systems, 20, 2007.
[36] Léon Botiou, Frank E Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. STAM Review, (0(2):223-311 2018.
[37] Stephen Boyd, Arpita Ghosh, Balaji Prabhakar, and Devavrat Shah. Gossip algorithms: Design, analysis and applications. In IEEE Infocom Conference Computer Communications, 2005.
[38] Stephen Boyd, Arpita Ghosh, Balaji Prabhakar, and Devavrat Shah. Randomized gossip algorithms.
IEEE Transactions on Information Theory, 52(6))2508 2530, 2006.
[39] Stephen Boyd, Neal Parikh, Eric Chu, Borja Pelcato, and Jonathan Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends@ in Machine Learning, 111::1- 2011.
[40] Christopher Briggs, Zhong Fan, and Peter Andras. Federated learning with hierarchical clustering of local updates to improve training on non-iid data. In IEEE International Joint Conference on Neural Networks, 2020.
[41] Shaked Brody, Uri Alon, and Eran Yahav, How attentive are graph attention networks? In Interna- tional Conference on Learning Representations, 2022.
[42] Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric Deep Learning: Grids, Groups, Gruphs, Geodesics, and Gauges. arXiv:2104.13478, 2021.
[43] Joan Bruna, Wojeiech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks on graphs. 1rXix 6203, 2013.
[44] Due Bui, Kshitiz Malik, Jack Goetz, Honglei Liu, Seungwhan Moon, Anuj Kumar, and Kang G Shin. Federated user representation learning. ar xrx:: 12535, 2019.
[45] Ming Cao, Daniel A Spielman, and Edmund M Yeh. Accelerated gossip algorithms for distributed computation. In The Annual Allerton Conference on Communication, Control, and Computation, 2006.
[46] Sergio Casas, Cole Gulino, Renjie Liao, and Raquel Urtasun. Spagnn: Spatially-aware graph neural networks for relational behavior forecasting from sensor data. In IEEE International Conference on Roboties and Automation, 2020.
[47] Timothy Castiglia, Anirban Das, and Stacy Patterson. Multi-level local SGD for heterogeneous hier- archical networks. In International Conference on Learning Representations, 2021.
[48] Emir Ceyani, Han Xie, Baturalp Buyukates, Carl Yang, and Salman Avestimehr. Fedgrains: Personal- ized subgraph federated learning with adaptive neighbor sampling. In SIAM International Conference on Data Mining, 2025.
[49] Di Chai, Leye Wang, Kai Chen, and Qiang Yang. Secure federated matrix factorization. JEEE Intelligent Systems, 36(5): (115):11-20., 20, 2020.
68

---

Published in Transactions on Machine Learning Research (02/2026)
[50] Zheng Chai, Ahsan Ali, Syed Zawad, Stacey Truex, Ali Anwar, Nathalic Baracaldo, Yi Zhou, Heiko Ludwig, Feng Yan, and Yue Cheng. Tifl: A tier-based federated learning system. Tn International Symposium on High--Performance Parallel and Distributed Computing, 2020.
[51] Zheng Chai, Yujing Chen, Ali Anwar, Liang Zhao, Yue Cheng, and Huzefa Rangwala. Fedat: A high--perfommace and communictioon-effccee federated learning system with asynchronous tiers. In International Conference for High Performance Computing, Networking, Storage and Analysis, 2021.
[52] T-H Hubert Chan, Elaine Shi, and Dawn Song. Private and continual release of statistics. ACM Transactions on Information and System Security, 14(3):1-24, 2011.
[53] Chaochao Chen, Jun Zhou, Longlei Zheng, Huiwen Wu, Lingjuan Lyu, Jia Wu, Bingzhe Wu, Ziqi Liu, Li Wang, and Xiaolin Zheng. Vertically federated graph neural network for sivacc-preserving node classification. In International Joint Conference on Artificial Intelligence, 2021.
[54] Chuan Chen, Ziyue Xu, Weibo Hu, Zibin Zheng, and Jie Zhang. Fedgl: Federated graph learning framework with global self-supervision. Information Sciences, 657:119976,, 2024.
[55] Deli Chen, Yankai Lin, Wei Li, Peng Li, Jie Zhou, and Xu Sun. Measuring and relieving the over- smoothing problem for graph neural networks from the topological view. In AAAI Conference on Artificial Intelligence, 2020.
[56] Fahao Chen, Peng Li, Toshiaki Miyazaki, and Celimuge Wu. Fedgraph: Federated graph learning with intelligent sampling. IEEE Transactions on Parallel and Distribuled Systems, 33(8):1775-1777 1786, 2021.
[57] Fengwen Chen, Guodong Long, Zonghan Wu, Tianyi Zhou, and Jing Jiang. Personalized federated learning with graph. In International Joint Conference on Arlificial Intelligence, 2022.
[58] Jialong Chen, Tianchi Liao, Chuan Chen, and Zibin Zheng. Improving message- chronous aggregation. In ACM International Conference on Information and Knowledge Management, 2024.
message-passing passing GNNs by asyn-
[59] Jiayi Chen and Aidong Zhang. Fodmsplit: Correlation-adaptive foderated multi-task learning across multimodal split networks. In ACM SIGKDD Conference on Knowledge Discovery and Dala Mining, 2022.
[60] Jingdi Chen, Hanqing Yang, Zongiun Liu, and Carlee Joe- Wong. The five Ws of multi-agent commu- nication: Who talks to whom, when, what, and why-a survey from MARL to emergent language and LLMs. Authorea Preprints, 2025.
[61] Jinyin Chen, Guohan Huang, Haibin Zheng, Shanqing Yu, Wenrong Jiang, and Chen Cui. Graph- fraudster: Adversarial attacks on graph neural network- based vertical federated learning. IEEE Trans- actions on Computational Social Systems, 10(2):492-50) 506, 2022.
[62] Jinyin Chen, Ximin Zhang, and Haibin Zheng. Adversarial attacks on G.N--based vertical federated learning. In Attacks, Defenses and Testing for Deep Learning. Springer, 2024.
[63] Mingyang Chen, Wen Zhang, Zonggang Yuan, Yantao Jia, and Huajun Chen. Fede: Embedding knowledge graphs in federated setting. In International Joint Conference on Knowledge Graphs, 2021.
[64] Mingyang Chen, Wen Zhang, Zonggang Yuan, Yantao Jia, and Huajun Chen. Federated knowledge graph completion via enbedding-contrastive contrastive learning. Knowledge-Basse Based Systems, 252: 2022.
[65] Mingzhe Chen, H. Vincent Poor, Walid Saad, and Shuguang Cui. Convergence time optimization for federated learning over wireless networks. IEEE Transactions on Wireless Communications, 20: 2457-2471, 2020.
[66] Albert Cheu, Adam Smith, Jonathan Ulman, David Zeber, and Maxim Zhilyaev. Distributed differ- ential privacy via shuffling. In Annual International Conference on the Theory and Applications of Cryplographic Techniques, 2019.
69

---

Published in Transactions on Machine Learning Research (02/2026)
[67] Tsz-Him Cheung, Weihang Dai, and Shuhan Li. Fedsge: Federated simple graph convolution for node classification. In International Workshop on Federated and Transfer Learning for Data Sporsily and Confidentiality, 2021.
[68] Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. Adaptive universal generalized pagerank graph neural network. In International Conference on Learning Representations, 2021.
[69] Yac Jec Cho, Jianyu Wang, and Gauri Joshi. Client selection in federated lcarning: Convergence analysis and power-of-choice selection strategies. or Xii::2000.01243 2020.
[70] Yae Jee Cho, Pranay Sharma, Gauri Joshi, Zheng Xu, Satyen Kale, and Tong Zhang. On the conver- gence of federated averaging with cyclic client participation. In International Conference Machine Learning, 2023.
[71] Woo-Seok Choi, Matthew Tomei, Jose Rodrigo Sanchez Vicarte, Pavan Kumar Hanumolu, and Rakesh Kumar. Guaranteeing local differential privacy on ultra-low-power systems. In ACM/IEEE Anneual International Symposium on Computer Architecture, 2018
[72] Ching- Yao Chuang and Stefanie Jegelka. Tree mover's distance: Bridging graph metries and stability of graph neural networks. Advances in Neural Information Processing Systems, 35:2944 29577, 2022.
[73] Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. Exploiting shared represen- tations for personalized federated learning. In International Conference on Machine Learning, 2021.
[74] Jorge Cortés. Finitc-time convergent gradient flows with applications to network consensus. Automat- ica, 42(11):1993-200) 2006.
[75] Edwige Cyffers, Aurélien Bellet, and Jalaj Upadhyay. Differentially private decentralized learning with random walks. In International Conference on Machine Learning, 2024.
[76] Hanjun Dai, Hui Li, Tian Tian, Xin Huang, Lin Wang, Jun Zhu, and Le Song. Adversarial attack on graph structured data. In International Conference on Machine Learning, 2018.
[77] Rong Dai, Li Shen, Fengxiang He, Xinmei Tian, and Dacheng Tao. DisPFL: Towards communication- efficient personalized federated learning via recentralized sparse training. In International Conference on Machine Learning, 2022.
[78] Katharine Daly, Hubert Eichner, Peter Kairouz, H Brendan McMahan, Daniel Ramage, and Zheng Xu. Federated learning in practice: Reflections and projections. In IEEE International Conference on Trust, Privacy and Security in Intelligent Systems, and Applications, 2024.
[79] Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Marc'aurelis Ran- zato, Andrew Senior, Paul Tucker, Ke Yang, Quoc V. Le, and Andrew Y. Ng. Large scale distributed deep networks. Advances in Neural Information Processing Systems, 25, 2012.
[80] Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems, 2016.
[81] Morris H DeGroot. Reaching a consensus. Journal of the American Statistical Association, 69(345): 118 121, 1974.
[82] Patrick Denantes, Florence Bénézit, Patrick Thiran, and Martin Vetterli. Which distributed averag- ing algorithm should i choose for my sensor network? In IEEE Infocom Conference on Computer Communications, 2008.
[83] Yongheng Deng, Weining Chen, Ju Ren, Feng Lyu, Yang Liu, Yunxin Liu, and Yaoxue Zhang. TailorFL: federated learning under system and data heterogeneity. In ACM Conference on Frabedded Nebworked Sensor Systems, 2022.
70

---

Published in Transactions on Machine Learning Research (02/2026)
[84] Yuyang Deng, Mohammad Mahdi Kamani, and Mchrdad Mahdavi. Adaptive personalized federated learning. ar arxiv:2033. 13461, 2020.
[85] Francesco Di Giovanni, Lorenzo Giusti, Federico Barbero, Giulia Luise, Pietro Lio, and Michael M Bronstcin, On over-squashing in message passing noural networks: The impact of width, depth, and topology. In International Conference on Machine Learning, 2023.
[86] Enmao Diao, Jie Ding, and Vahid Tarokh. HeteroFL: Computation and communication efficient federated learning for heterogeneous clients. In International Conference on Learning Representations, 2021.
[87] Alexandros DG Dimakis, Anand D Sarwate, and Martin J Wainwright. Geographic gossip: Efficient averaging for sensor networks. IEEE Trunsaclions on Signal Processing, 56(3):1205-122 2008.
[88] Bolin Ding, Janardhan Kulkarni, and Sergey Yekhanin. Collecting telemetry data privately. Advances in Neural Information Processing Systems, 30, 2017.
[89] Guimin Dong, Mingyue Tang, Lihua Cai, Laura Barnes, and Mehdi Boukhechba. Semi-supervised graph instance transformer for mental health inference. In IEEE International Conference on Machine Learning and Applications, 2021.
[90] Guimin Dong, Mingyue Tang, Zhiyuan Wang, Jiechao Gao, Sikun Guo, Lihua Cai, Robert Gutierrez, Bradford Campbel, Laura E Barnes, and Mehdi Boukhechba. Graph neural networks in loT: A survey. ACM Transactions on Sensor Networks, 19(2):1-50, 2023.
[91] Bingqian Du and Chuan Wu. Federated graph learning with periodic neighbour sampling. In BEEE/CM International Symposium on Quality of Service, 2022.
[92] Moming Duan, Duo Liu, Xianzhang Chen, Renping Liu, Yujuan Tan, and Liang Liang. Self-balancing federated learning with global imbalanced data in mobile systems. IEEE Transactions Parallel and Distribuled Systems, 31(1):59 71, 2020.
[93] Moming Duan, Duo Liu, Xinyuan Ji, Renping Liu, Liang Liang, Xianzhang Chen, and Yujuan Tan. Fedgroup: Efficient federated learning via decomposed similarity-based clustering. In IEEE Interna- tioanal Conference on Parallel & Distributed Processing with Applications, Big Data Cloud Com- puting, Sustainable Computing & Communications, Social Computing & Networking, 2021.
[94] John Duchi and Ryan Rogers. Lower bounds for locally private estimation via communication com- plexity. In Conference Learning Theory, 2019.
[95] John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(7), 2011.
[96] John C Duchi, Michael I Jordan, and Martin J Wainwright. Local privacy and statistical minimax rates. In TEFE Annual Symposium on Foundations of Computer Science, 2013.
[97] John C Duchi, Michael Jordan, and Martin J Wainwright. Minimax optimal procedures for locally private estimation. Journal of the American Statistical Association, 113(521):1822200 2012 2018.
[98] Vasisht Duddu, Antoine Boutet, and Virat Shejwalkar. Quantifying privacy leakage in graph embed- ding. In EAT International Conference on Mobile and Ubiquilous Systems: Compuling, Nelworking and Services, 2020.
[99] Chi Thang Duong, Thanh Dat Hoang, Ha The Hien Dang, Quoe Viet Hung Nguyen, and Karl Aberer. On node features for graph neural networks. arXiw:1911.06 2019.
[100] Cynthia Dwork. Differential privacy. In International Colloquium on Automata, Languages, and Programming, 2006.
71

---

Published in Transactions on Machine Learning Research (02/2026)
[101] Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foundations and Trendo0 in Theovelical Compuler Science, 9(3-4) ((3-4):211- 11:2111-777 2014.
[102] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography, 2006.
[103] Cynthia Dwork, Moni Naor, Toniann Pitassi, and Guy N Rothblum. Diffcrential privacy under con- tinual observation. In ACM Symposium on Theory of Computing, 2010.
[104] Abdellah El Mrini, Edwige Cyffers, and Aurélien Bellet. Privacy attacks in decentralized learning. International Conference on Machine Learning, 2024.
[105] Ülfar Erlingsson, Vitaly Feldman, Tya Mironov, Ananth Raghunathan, Kunal Talwar, and Abhradoep Thakurta. Amplification by shuflling: From local to central differential privacy via anonymily. In Annual ACM-SIAM Symposium on Discrete Algorithms, 2019.
[066] European Union. General data protection regulation. Technical report, European Union, 2016.
[107] Mathicu Even, Anastasia Koloskova, and Laurent Massoulie. Asynchronous SGD on graphs: A unified framework for asynchronous decentralized and federated optimization. In International Conference on Artificial Intelligence and Statistics, 2024.
[108] Alexandre Evfimievski, Johannes Gehrke, and Ramakrishnan Srikant. Limiting privacy breaches in privacy preserving data mining. In ACM STGMOD-STGACT-SIGAAN Symposium on Principles of Dalabase Systems, 2003.
[109] Lukas Faber and Roger Wattenhofer. GwAC: GNNs with asynchronous communiattion In Learning on Graphs Conference, 2024.
[110] Lukas Faber, Yifan Lu, and Roger Wattenhofer. Should graph neural networks use features, edges, or both? 1rXiv:2103. 06857, 2021.
[111] Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalizod federated learning: A meta- learning approach. orXiv:2002.079/3 2020.
[122] Zhenan Fan, Huang Fang, and Michael P Friedlander. A dual approach for federated learning. ar wrXiv:2201.11188 2022.
[113] Hui Fang, Yang Gao, Peng Zhang, Jiangchao Yao, Hongyang Chen, and Haishuai Wang. Large lan- guage models enhanced personalized graph neural architecture search in federated learning. In AAAI Conference on Artificial Intelligence, 2025.
[114] Bahare Fatemi, Layla El Asri, and Seyed Mehran Kazemi. Slaps: Self-supervision improves structure learning for graph neural networks. Advonces Neural Information Processing Systems, 34:22667- 22681, 2021.
[115] J Alexander Fax and Richard M Murray. Information flow and cooperative control of vehicle formations. IEEE Transactions on Automatic Control, 49(9):1465 1476, 2004.
[116] Siwci Feng. Vertical federated learning bassing-based based feature selection with non-overlapping sample utilization. Expert Systems with Applications, 208:11 2022.
[117] Siwei Feng, Boyang Li, Han Yu, Yang Liu, and Qiang Yang. Semi-supervised federated heterogeneous transfer learning. Knowledge-Bool Based Systems, 252:109384, 2022.
[118] Matthias Fey, Jan E. Lenssen, Frank Weichert, and Jure Leskovec. GNNAutoscalc: Scalable and expressive graph neural networks via historical embeddings. In International Conference on Machine Learning, 2021.
72

---

Published in Transactions on Machine Learning Research (02/2026)
[119] Matthias Fey, Jimu Sunil, Akihiro Nitta, Rishi Puri, Manan Shah, Blaž Stojanovië, Ramona Bendias, Alexandria. Barghi, Vid Kocijan, Zecheng Zhang, Xinwei He, Jan Fric Lenssen, and Jure Teskovec. Pyg 2.0: Scalable learning on real world graphs. In KDD Worshop on Temporal Graph Learning, 2025.
[120] Miroslav Fiedlor. Algebraic connectivity of graphs. Czechoslovak Mathematical Journal, 23(2):298-23) 305, 1973.
[121] Ben Finkelshtein, Xingyue Huang. Michael Bronstein, and ismail ilkan Ceylan. Cooperative graph neural networks. In International Conference on Machine Learning, 2024.
[122] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostie meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, 2017.
[123]
Simone Fiorellino, Claudio Battiloro, and Paolo Di Lorenzo. Topological neural networks over the air. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2024.
[124] Fabrizio Frasca, Fabian Jogl, Moshe Eliasof, Matan Ostrovsky, Carola-Bibiane Schönlieb, Thomas Gärtner, and Haggai Maron. Towards foundation models on graphs: An analysis on cross-dataset transfer of pretrained GNNs. In NEURIPS Workshop on Symmelry and Geomelry in Neural Repre- sentations, 2024.
125] Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In ACM SIGSAC Conference on Computer and Communica- tions Security, 2015.
[166] Xingbo Fu, Song Wang, Yushun Dong, Binchi Zhang, Chen Chen, and Jundong Li. Federated graph learning with graphless clients. Transactions on Machine Learning Research, 2024.
[17] Dominik Fuchsgruber, Tom Wollschläger, Johannes Bordne, and Stephan Günnemann. Uncertainty estimation for heterophilic graphs through the lens of information theory. In International Conference on Machine Learning, 2025.
[28] Huiguo Gao, Mengyuan Lee, Guanding Yu, and Zhaolin Zhou. A graph neural network based decen- tralized learning scheme. Sensors, 22(3):1030, 2022.
[129] Rufei Gao, Zhaowei Liu, Chenxi Jiang, Yingjie Wang, Shenqiang Wang, and Pengda Wang. B1- fedGNN: Federated graph neural networks framework based on bayesian inference. Neural Networks, 169:143-18 153, 2024.
[130] Zhan Gao and Deniz Gündüz. Airguns: Graph neural networks over the air. In JEEE European Signal Processing Conference, 2023.
[131] Zhan Gao and Deniz Gündüz. Graph neural networks over the air for decentralized tasks in wireless networks. IEEE Transactions on Signal Processing, 2025.
[132] Zhan Gao, Elvin Isufi, and Alejandro Ribeiro. Stochastic graph neural networks. IEEE Transactions on Signal Processing, 69:4428 4443, 2021.
[133] Zhan Gao, Yulin Shao, Deniz Gündüz, and Amanda Prorok. Decentralized channel management in WLANs with graph neural networks. In IEEE International Conference Communications,, 2023.
[134] Johannes Gasteiger, Aleksandar Bojchevski, and Stephan Günnemann. Predict then propagate: Graph neural networks meet personalized pagerank. In International Conference on Learning Representations, 2019.
[135] Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. Inverting gradients how easy is it to break privacy in federated lcarning? In Advances Neural Information Processing Systems, 2020.
73

---

Published in Transactions on Machine Learning Research (02/2026)
[136] Robin C Geyer, Tassilo Klein, and Moin Nabi. Differentially private federated learning: A client level perspective. In NEURIPS Workshop on Machine Learning on the Phone and Other Consumer Devices, 2017.
[137] Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. An efficient framework for clustered federated learning. Advances in Neural Information Processing Systems, 33:19586-19597, 2020.
[138] Lodovico Giarctta and Sarmna Girdzijauskas. Fully-decentralizzdd training of GNNs using layer-wise self-supervision. diva2:1745180, 2023.
[139] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International Conference on Machine Learning, 2017.
[140] Jack Goetz, Kshitiz Malik, Duc Bui, Seungwhan Moon, Honglei Liu, and Amuj Kumar. Active federated learning. Technical report, Universily of Michigan, 2019.
[141] Shafi Goldwasser, Silvio Micali, and Chales Rackoff. The knowledge complexity of interactive proof- systems. In ACM Symposium on Theory of Computing, 1985.
[142] Yonghai Gong, Yichuan Li, and Nikolaos M Freris. FedADMM: A robust federated deep learning frame- work with adaptivity to system heterogeneity. In IEEE International Conference on Data Engineering. 2022.
[143] Tan Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016.
http:
//www. www.deeeaaninnbooooooo org
[144] Sven Gronauer and Klaus Diepold. Mueti-agent deep reinforcement learning: A survey, Artificiell Intelligence Revieu, 55(2):895-943) 2022.
[145] Fangda Gu, Heng Chang, Wenwu Zhu, Somayeh Sojoudi, and Laurent El Ghaoui. Implicit graph neural networks. Advances in Neural Information Processing Systems, 18:11984-19999 2020.
[146] Zeli Guan, Yawen Li, Zhe Xue, Yuxin Lüu, Hongrui Gao, and Yingxia Shao. Federated graph neural network for cross-graph node classification. In IEEE International Conference on Cloud Computing and Intelligent Systems, 2021.
[17] Shurui Gui, Xiner Li, Limei Wang, and Shuiwang Ji. Good: A graph out-of-distribution benchmark. Advances in Newral Information Processing Systems, 35:2059 2073, 2022.
[148] Jiayan Guo, Shangyang Li, and Yan Zhang. An information thcoretic perspective for heterogencous subgraph federated learning. In International Conference on Database Systems for Advanced Applica- lions, 2023.
[149] Pengxin Guo, Shuang Zeng, Yanran Wang, Huijic Fan, Feifci Wang, and Liangqiong Qu Selective aggregation for low-rank adaptation in federated learning. In International Conference on Learning Representations, 2025.
[150] Zhuoning Guo, Duanyi Yao, Qiang Yang, and Hao Liu. Hifgl: A hierarchical framework for cross-silo cross-device federated graph learning. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024.
[151] Otkrist Gupta and Ramesh Raskar. Distributed learning of deep neural network over multiple agents. Journal of Network and Computer Applications, 116:1-8, 2018.
[152] Suyog Gupta, Wei Zhang, and Fei Wang- Model accuracy and runtime tradcoff in distributed deep learning: A systematic study. In IEEE International Conference on Data Mining, 2016.
[153] Benjamin Gutteridge, Xiaowen Dong, Michael M Bronstein, and Francesco Di Giovanni. Drew: Dy- namically rewired message passing with delay. In International Conference on Machine Learning. 2023.
74

---

Published in Transactions on Machine Learning Research (02/2026)
[154] Mustafa Hajij, Ghada Zamzmi, Theodore Papamarkou, Nina Miolane, Aldo Guzmán-Sáenz, Karthikeyan Natesan Ramamurthy, Tolga Birdal, Tamal K. Dey, Soham Mukherjee, Shreyas N. Sam- aga, Neal Livesay, Robin Walters, Paul Rosen, and Michael T. Schaub. Topological deep learning: Going beyond graph data. Xix:22200 00600, 2022.
[155] Will Hamilton, Zhilao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in Neural Information Processing Systems, 30, 2017.
[156] William L Hamilton, Rex Ying, and Jure Leskovec. Representation learning on graphs: Methods and applications. IEEE Data Engineering Bulletin, 2017.
[157] David K Hammond, Pierre Vandergheynst, and Rémi Gribonval. Wavelets on graphs via spectral graph theory. Applied and Computational Harmonic Analysis, 30(2):129 -50, 2011.
[158] Zhaoxing Han, Chengyu Hu, Tongyaqi Li, Qingqiang Qi, Peng Tang, and Shanqing Guo. Subgraph-level federated graph neural network for privacy-preservig recommendation with meta-learning. Neural Networks, 179:106574, 2024.
[159] Andrew Hard, Kanishka Rao, Rajiv Mathews, Swaroop Ramaswamy, Françoise Beaufays, Sean Au- genstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage. Federated learning for mobile keyboard prediction. Technical report, Google, 2018.
1600] Chaoyang He, Murali Annavaram, and Salman Avestimehr. Group knowledge transfer: Federated learning of large cmns at the edge. Advonces in Neurol Information Processing Systems, 33:14068 14080, 2020.
[161] Chaoyang He, Keshav Balasubramanian, Emir Ceyani, Carl Yang, Han Xie, Lichao Sun, Lifang He, Liangwei Yang, Philip S. Yu, Yu Rong, Peilin Zhao, Junzhou Huang, Murali Annavaram, and Salman Avestimehr. Fedgraphnn: federated lcarning benchmark systom for graph ncural networks. Workshop on Distributed and Privale Machine Learning, 2021.
[162] Chaoyang He, Emir Ceyani, Keshav Balasubramanian, Murali Annavaram, and Salman Avestimehr, Spreadgnn: Decentralized multi-task federated learning for graph neural networks molecular data. In AAAI Conference on Artificial Intelligence, 2022.
[163] Guanghui He, Yanli Ren, Jingyuan Jiang, Guorui Feng, and Xinpeng Zhang. Privacy-enhanced fed- erated GNN inference against adversarial example allack. IEEE Transactions on Emerging Topics in Computational Intelligence, 2024.
[164] Mingguo He, Zhewei Wei, Hongteng Xu, al. BernNet: Learning arbitrary graph spectral filters via bernstein approximation. In Advances in Neural Information Processing Systems, 2021.
[165] Mingguo He, Zhewei Wei, and Ji-Rong Wen. Convolutional neural networks graphs with chebyshev approximation, revisited. In Advonces in Neural Information Processing Systems, 2022.
[666] Shiwen He, Shaowen Xiong, Yeyu Ou, Jian Zhang, Jiaheng Wang, Yongming Huang, and Yaoxue Zhang. An overview on the application graph neural networks in wireless networks. IEEE Open Journal of the Communications Society, 2:2547-2565, 2021.
[167] Xinlei He, Jinyuan Jia, Michael Backes, Neil Zhenqiang Gong, and Yang Zhang. Stealing links from graph neural networks. In USENIX Security Symposimm, 2021.
[168] Yuanqin He, Yan Kang, Xinyuan Zhao, Jiahuan Luo, Lixin Fan, and Yuxing Han. A hybrid self- supervised learning framework for vertical foderated learning. IEEE Transactions on Big Data, 2025.
169] István Hegodús, Gábor Danner, and Márk Jelasity. Gossip learning as decentralized alternative federated learning. In IFIP International Conference on Distributed Applications and Interoperable Systems, 2019.
75

---

Published in Transactions on Machine Learning Research (02/2026)
[170] Henrik Hellström, José Mairton B da Silva Jr, Mohammad Mohammadi Amiri, Mingzhe Chen, Viktoria Fodor, H Vincent Poor, and Carlo Fischione. Wireless for machine learning: A survey, Foundalions and Trendso in Signal Processing, 15(4):290 399, 2022.
[171] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. In NIPS Deep Learning and Representation Learning Workshop, 2014.
[172] Briland Hitaj, Giuseppe Ateniese, and Fernando Perez- Cruz. Deep models under the GAN: Infor- mation leakage from collaborative deep learning. Tn ACM SIGSAC Conference Computer and Communications Security, 2017.
[173] Paul W Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt. Stochastic blockmodels: First steps. Social Networks, 5(2):109 137, 1983.
[174] Sara Hooker. The hardware lottery. Communications of the ACM, 64(12):58 65, 2021.
[175] Charlie Hou, Kiran Koshy Thekumparampil, Giulia Fanti, and Sewoong Oh. Fedchain: Chained algorithms for near-optimal communication cost in federated learning. In International Conference on Learning Represenlations, 2022.
[176] Kevin Hsich, Amar Phanishayce, Onur Mutlu, and Phillip Gibbons. The non-iid data quagmire of decentralized machine learning. In International Conference. on Machine Learning, 2020.
[177] Chung-ju Huang, Leye Wang, and Xiao Han. Vertical federated knowledge transfer via representation distillation for healtheare collaboration networks. In ACM Web Conference, 2023.
[178] Huang, Andrew Shea, Huining Qian, Aditya Masurkar, Hao Deng, and Dianbo Liu. Patient clustering improves efficiency of federated machine learning to predict mortality and hospital stay time using distributed electronic medical records. Journal of Biomedical Informatics, 99:103291, 2019.
[179] Qian Huang, Horace He, Abhay Singh, Ser-Nam Lim, and Austin R Benson. Combining label propaga- tion and simple models out-performs graph neural networks. In International Conference on Learning Representations, 2021.
[180] Tiansheng Huang, Shiwei Liu, Li Shen, Fengxiang He, Weiwei Lin, and Dacheng Tao. Achieving personalized federated learning with sparse local models. orXiv:2201.11380 2022.
[181] Yujin Huang, Han Hu, and Chunyang Chen. Robustness of on-device models: Adversarial attack to deep learning models on android apps. In IEEE/ACM International Conference on Soflware Engi- neering, 2021.
[182] Yutao Huang, Lingyang Chu, Zirui Zhou, Lanjun Wang, Jiangchuan Liu, Jian Pei, and Yong Zhang, Personalized cross-silo federated learning on non-iid data. In AAAI Conference on Artificial Intelli- gence, 2021.
[183] Damitry Huba, John Nguyen, Kshitiz Malik, Ruiyu Zhu, Mike Rabbat, Ashkan Youselpour, Carole- Jean Wu, Hongyuan Zhan, Pavel Ustinov, Harish Srinivas, Kaikai Wang, Anthony Shoumikhin, Jesik Min, and Mani Malek. Papaya: Practical, private, and scalable federated learning. In Machine Learning and Systems, 2022.
[184] Ali Jadbabaie, Jie Lin, and A Stephen Morse. Coordination of groups of mobile autonomous agents using nearest neighbor rules. IEEE Transactions on Automatic Control, 48(6):988-1000 2003.
[185] Stefanic Jegelka. Theory of graph neural networks: Representation and learning. In The International Congress of Mathematicians, 2022.
[186] Márk Jelasity, Alberto Montresor, and Ozalp Babaoglu. Gossip- based aggregation in large dynamic networks. ACM Transactions Computer Systems, 23(3):219 252, 2005.
76

---

Published in Transactions on Machine Learning Research (02/2026)
[187] Eunjeong Jeong, Seungeun Oh, Hyesung Kim, Jihong Park, Mehdi Bennis, and Seong-Lyun Kim. Communication-eefiieeii on-device machine learning: Federated distillation and augmentation under non-iid private data. In NEURIPS Workshop on Machine Learning on the Phone and Other Consumer Devices, 2018.
[188] Shaoxiong Ji, Yue Tan, Teemu Saravirta, Zhiqin Yang, Yixin Liu, Lauri Vasankari, Shirui Pan, Guodong Long, and Anwar Walid. Emerging trends in federated learning: From model fusion to federated learning. International Journal of Machine Learning and Cybernetics, 15(9):3769-3777 2024.
[189] Fangeui Jiang and Long Wang. Finite-time information consensus for multi-agent systems with fixed and switching topologies. Physica D: Nonlinear Phenomena, 238(16):1550 1560, 2009.
190 Yihan Jiang, Jakub Konečný, Keith Rush, and Sreeram Kannan. Improving federated learning per- sonalization via model agnostic meta learning. Stt:1909.12488 2019.
[91]]
Bjorn Johansson, Tamás Keviczky, Mikael Johansson, and Karl Henrik Johansson. Subgradient meth- ods and consensus algorithus for solving convex optimization problems. In IEEE Conference on Decision and Control, 2008
[192] Peter Kairouz, Brendan McMahan, Shuang Song, Om Thakkar, Abhradeep Thakurla, and Zheng Xu. Practical and private (deep) learning without sampling or shuffling. In International Conference on Machine Learning, 2021.
[193] Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D'Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrelt, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konečný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Lin, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, and Sen Zhao. Advances and open problems in federated learning. Foundations and Trends@ in Machine Learning, 14(1-2):1-210, 2021.
[94] Honggu Kang, Seohyeon Cha, Jinwoo Shin, Jongmyeong Lee, and Joonhyuk Kang. NeFT: Nested model scaling for federated learning with system heterogeneous clients. IEEE Transactions on Mobile Compuling, 24(8):6734 6746, 2025.
[195] Yan Kang, Yuanqin He, Jiahuan Luo, Tao Fan, Yang Liu, and Qiang Yang. Privacy-preserving fed- erated adversarial domain adaptation over feature groups for interpretability. IEEE Transactions on Big Data, 10(6):879 890, 2022.
[196] Yan Kang, Yang Liu, and Xinle Liang. Fedevt: Semi-supervised vertical federated learning with cross-view training. ACM Transactions on Intelligent Systems and Technology, 13(4):1-16, 2022.
[197] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theortha Suresh. Scaffold: Stochastic controlled averaging for foderated learning. In Jn- ternational Conference on Machine Learning, 2020.
[198] Richard Karp, Christian Schindelhauer, Scott Shenker, and Berthold Vocking. Randomized rumor spreading. In IEEE Annual Symposium on Foundations of Compuler Science, 2000.
[199] Brian Karrer and Mark EJ Newman. Stochastic blockmodels and community structure in networks. Physicul Review E, 831):016617 2011.
[200] Shiva Prasad Kasiviswanathan, Homin K Lee, Kobbi Nissim, Sofya Raskhodníkova, and Adam Smith. What we learn privately? STAM Journal on Computing, 40(3):793 826, 2011.
77

---

Published in Transactions on Machine Learning Research (02/2026)
[201] Ances Kazi, Luca Cosmo, Seyed-Ahmad Ahmadi, Nassir Navab, and Michael M Bronstein. Differen- tiable graph module (dgm) for graph convolutional networks. IEEE Transactions on Pallern Analysis and Machine Intelligence, 45(2):1606 11111666--17,,,, 2022.
[202] Nicolas Keriven, Alberto Bietti, and Samuel Vaiter. Convergence and stability of graph convolutional networks on large random graphs. Advances in Neural Information Processing Systems, 33:21512- 21523, 2020.
[203] Ahmed Khaled, Konstantin Mishchenko, and Peter Richtárik. Tighter theory for local SGD on identical and heterogeneous data. In International Conference on Arlificial Intelligence and Statistics, 2020.
[204] Prashant Khanduri, Pranay Sharma, Haibo Yang, Mingyi Hong, Jia Liu, Ketan Rajawat, and Pramod Varshney. Stem: A stochastic two-sided momentum algorithm achieving near-optimal sample and com- munication complexities for federated learning. Advances in Newral Information Processing Systems. 34:6050 6061, 2021.
[205] Geeho Kim, Jinkyu Kim, and Bohyung Han. Communicttion-effiiieett federated learning with ac- celerated client gradient. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[206] Hansol Kim, Youngjun Kwak, Minyoung Jung, Jinho Shin, Youngsung Kim, and Changick Kim. ProtoFL: Unsupervised federated learning via protolypical distillation. In IEEE/CVF International Conference on Computer Vision, 2023.
[207] Sungwon Kim, Yoonho Lee, Yunhak Oh, Namkyeong Lee, Sukwon Yun, Junseok Lee, Sein Kim, Carl Yang, and Chanyoung Park. Subgraph federated learning for local generalization. In International Conference on Learning Representations, 2025.
[208] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
[209] Benjamin Klein, Lior Wolf, and Yehuda Afck. A dynamic convolutional layer for short range weather prediction. In IEEE Conference on Computer Vision and Pattern Recognition, 2015.
[210] Jakub Konečný, Н. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. NIPS Workshop on Private Multi- Machine Learning, 2016.
[211] Emmanouil Krasanakis, Symeon Papadopoulos, and Ioannis Kompatsiaris. p2pGNN: A decentralized graph neural network for node classification in peer-to-peer networks. IEFE Access, 10:34755-3415 347655 2022.
[212] Lorenz Kummer, Samir Moustafa, Anatol Ehrlich, Franka Bause, Nikolaus Suess, Wilffied N. Gansterer, and Nils Morten Kricge. Weisfeiler and leman go gambling: Why expressive lottery tickets win. In International Conference Machine Learning, 2025.
[213] Gihun Lee, Minchan Jeong, Yongjin Shin, Sangmin Bac, and Se- Young Yun. Presorvation of the global knowledge by not-true distillation in federated learning. In Advances in Neural Information Processing Systems, 2022.
[214] Mengyuan Lee, Guanding Yu, and Huaiyu Dai. Decentralized inference with graph neural networks in wireless communication systems. IEEE Transactions Mobile Computing, 22(5):2582-2558 2021.
(215) Mengyuan Lee, Guanding Yu, Huaiyu Dai, and GeofTrey Ye Li. Graph neural networks meet wireless communications: Motivation, applications, and future directions. IEEE Wireless Communications, 29 (5):12 19, 2022.
[216] Mengyuan Lee, Guanding Yu, and Huaiyu Dai. Privecy- decentralized inference with graph neural networks in wireless networks. IEEE Tronsactions on Wireless Communications, 23(1):5433555) 558, 2023.
78

---

Published in Transactions on Machine Learning Research (02/2026)
[217] Runze Lei, Pinghui Wang, Junzhou Zhao, Lin Lan, Jing Tao, Chao Deng, Junlan Feng, Xidian Wang, and Xiaohong Guan. Federated learning over coupled graphs. IEFF Transactions on Porallel and Distributed Systems, 34(4):1159 1172, 2023.
[218] Jichao Leng, Zihuai Lin, Ming Ding. Peng Wang, David Smith, and Branka Vucetic. Client scheduling in wireless federated learning based on channel and learning qualities. IEEE Wireless Communicaatins Letters, 11(4):732-78) 2022.
[219] Ang Li, Jingwei Sun, Xiao Zeng, Mi Zhang, Hai Li, and Yiran Chen. Fedmask: Joint computa- tion and communication-efficenn personalized federated learning via heterogeneous masking, In ACM Conference on Embedded Networked Sensor Systems, 2021.
[220] Daliang Li and Junpu Wang. Fedmd: Heterogenous federated learning via model distillation. In NEURIPS Workshop on Federated Learning for Data Privacy and Confidentiality, 2019.
[221] Li, Yuxi Fan, Mike Tse, and Kuo-Yi Lin. review of applications in federated learning. Compulers & Industrial Engineering, 149:106854, 2020.
[222] Li Li, Moming Duan, Duo Liu, Yu Zhang, Ao Ren, Xianzhang Chen, Yujuan Tan, and Chengliang Wang. Fedsae: A novel self-adaptive federated learning framework in heterogeneous systems. In IEEE International Joint Conference on Neural Networks, 2021.
[223] Qinbin Li, Bingsheng He, and Dawn Song. Model-contrastive federated learning. In IEEE/CVF Conference on Compuler Vision and Pullern Recognilion, 2021.
[224] Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data silos: An experimental study. In IEEE International Conference on Data Engineering, 2022.
[225] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in hetcrogencous networks. Proceedings of Machine Learning and Systems, 2: 429 450, 2020.
[226] Wenjie Li, Qiaolin Xia, Junfeng Deng, Hao Cheng, Jiangming Liu, Kouying Xue, Yong Cheng, and Shu-Tao Xia. VFed-SSD: Towards practical vertical federated advertising. In Trustworthy Federated Learning Workshop at the International Joint Conference on Artificial Intelligence, 2022.
[227] Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of FedAvg on non-iid data. In International Conference on Learning Representations, 2020.
[228] Xingyu Li, Zhe Qu, Bo Tang, and Zhuo Lu. Stragglers are not disasters: A hybrid federated learn- ing framework with delayed gradients. In IEEE International Conference on Machine Learwing and Applications, 2022.
[229] Xiangru Lian, Wei Zhang, Ce Zhang, and Ji Liu. Asynchronous decentralized parallel stochastic gradient descent. In International Conference on Machine Learning, 2018
[230] Gang Liang and Sudarshan S Chawathe. Privacy-preserving inter-database operations. In International Conference Intelligence and Security Informatics, 2004.
[231] Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B Allen, Randy P Aucrbach, David Brent, Ruslan Salakhutdinov, and Louis-Philippe Morency. Think locally, act globally: Federated learning with local and global representations. orXiv:2001.01522 2020.
[232] Wenfei Liang, Yanan Zhao, Rui She, Yiming Li, and Wee Peng Tay. Fedsheafhn: Personalized federated lcarning on graph-structured data. arXix:2405. 16056, 2024.
[233] Haiyang Lin, Mingyu Yan, Xiaochun Ye, Dongrui Fan, Shirui Pan, Wenguang Chen, and Yuan Xic. A comprehensive survey on distributed training of graph neural networks. Proceedings of the IEEE,
2023.
79

---

Published in Transactions on Machine Learning Research (02/2026)
[234] Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. Ensemble distillation for robust model fusion in federated learning. Advances in Neural Information Processing Systems, 33:2351 1233113333,, 2020.
[235] Wanyu Lin, Baochum Li, and Cong Wang. Towards private learning on decentralized graphs with local differential privacy. TEFE Transactions Information Forensics and Security,
2022.
[236] Dongzhu Liu, Guangxu Zhu, Jun Zhang, and Kaibin Huang. Wireless data acquisition for edge learning: Importance-aware retransmission. In IEFE International Workshop on Signal Processing Advonces in Wireless Communications,, 2019.
[237] Dongzhu Liu, Guangxu Zhu, Jun Zhang, and Kaibin Huang. Data-importance aware user schoduling for communication-ellicenu edge machine learning. IEEE Transactions on Cognilive Commmunications and Networking, 7(1):265 278, 2021.
[238] Fengxia Liu, Zhimming Zheng, Yexuan Shi, Yongxin Tong, and Yi Zhang. A survey on federated learning: A perspective from multi-party computation. Frontiers of Computer Science, 18(1):181336, 2024.
[239] Ji Liu, Brian DO Anderson, Ming Cao, and A Stephen Morse. Analysis of accelerated gossip a algorithms. Automatica, 49(4))873 883, 2013.
[240] Ji Liu, Jizhou Huang, Yang Zhou, Xuhong Li, Shilei Ji, Haoyi Xiong, and Dejing Dou. From distributed machine learning to federated learning: A survey, Knowledge and Information Systems, 64(4):885 917, 2022.
[241] Rui Liu, Pengwei Xing, Zichao Deng, Anran Li, Cuntai Guan, and Han Yu. Federated graph neural networks: Overview, techniques, and challenges. IEEE Transactions on Newral Networks and Learning Systems, 36(3):4279-422) 4295, 2025.
[242] Shikun Liu, Tianchun Li, Yongbin Feng, Nhan Tran, Han Zhao, Qiang Qiu, and Pan Li. Structural re- re-geighting improves graph domain adaptation. In International Conference on Machine Learning, 2023.
[243] Shuchang Liu, Yingqiang Ge, Shuyuan Xu, Yongfeng Zhang, and Amelic Marian. Fairness-aware federated matrix factorization. In ACM Conference on Recommender Systems, 2022.
[244] Tao Liu, Peng Li, and Yu Gu. Glint: Decentralized foderated graph lcarning with traffic throttling and flow scheduling. In IEEE/ACM International Symposium on Quality of Service 2021.
[245] Wanchun Liu, Xin Zang, Yonghui Li, and Branka Vucetic. Over-the-air computation systems: Opti- mization, analysis and scaling laws. IEEE Transactions on Wireless Communications, 19(8):4488-55 5502, 2020.
[246] Yang Liu, Yan Kang, Chaoping Xing, Tianjian Chen, and Qiang Yang. A secure federated transfer learning framework. IEEE Intelligent Systems, 35(4):70 82, 2020.
[247] Yang Liu, Yan Kang, Tianyuan Zou, Yanhong Pu, Yuanqin He, Xiaozhou Ye, Ye Ouyang, Ya-Qin Zhang, and Qiang Yang. Vertical federated learning: Concepts, advances, and challenges. IEEE Transactions on Knowledge and Data Engineering. 36(7):3615-3634, 2024.
[248] Zhaowei Liu, Yifan Jiang, Jisojiao Gu, Diantong Liu, Yongehao Song, Anzuo Jiang, and Peiyong Duan. Federated graph neural networks based multiscale residuals in industrial internet of things. IEEE Transactions on Computational Social Systems, 2025.
[249] Zhiwei Liu, Liangwei Yang, Ziwei Fan, Hao Peng, and Philip S Yu. Federated social recommendation with graph neural network. ACM Transactions on Intelligent Systems and Technology, 13(4): 24 2022.
[250] Ziheng Liu, Jiayi Zhang, Enyu Shi, Zhilong Liu, Dusit Niyato, Bo Ai, and Xuemin Shen. Graph neural network meets multi-agent reinforcement learning: Fundamentals, applications, and future directions. IEFE Wireless Communications, 31(33:39 47, 2024.
80

---

Published in Transactions on Machine Learning Research (02/2026)
[251] Ilan Lobel and Asuman Ozdaglar. Convergence analysis of distributed subgradient methods over random networks. In IEEE Allerton Conference on Communication, Control, and Compuling, 2008.
[252] Guodong Long, Ming Xie, Tao Shen, Tianyi Zhou, Xianzhi Wang, and Jing Jiang, Multi-center federated learning: Clients clustering for better personalization. World Wide Web, 26(1):481-500,
2023.
[253] Yucheng Low, Joseph Gonzalez, Aapo Kyrola, Danny Bickson, Carlos Guestrin, and Joseph Hellerstein. Graphlab: A new framework for parallel machine learning. In Conference Uncertainly in Artificial Intelligence, 2010.
[254] Linpeng Lu and Ning Ding. Multi-party private set intersection in vertical federated learning. In IEEE International Conference on Trusl, Security and Privacy in Compuling and Communiications, 2020.
[255] Ruichen Luo, Sebastian U Stich, Samuel Horváth, and Martin Takáč. Revisiting localSGD and SCAF- FOLD: Improved rates and missing analysis. In International Conference on Artificial Intelligence and Statisfies, 2025.
[256] Siqi Luo, Xu Chen, Qiong Wu, Zhi Zhou, and Shuai Yu. Hfel: Joint edge association and resource allocation for cost-efficient hierarchical federated edge learning. IEEE Transactions on Wireless Com- munications, ((110)6533--5488 2020.
[257] Lingjuan Lyu, Han Yu, and Qiang Yang. Threats to federated learning: A survey, arXiv:2003.02133, 2020.
[258]
Chengyuan Mai, Tianchi Liao, Chuan Chen, and Zibin Zheng, Fgul: Federated graph transfer learning for node classification. ACM Transactions on Knowledge Discovery from Data, 19(1):1-20 20, 2024.
[259] Yishay Mansour, Mehryar Mohri, Jae Ro, and Ananda Theertha Suresh. Three approaches for per- sonalization with applications to federated learning. arXiv:2002.1061 2020.
[260] Sohir Maskey, Raffacle Paolino, Fabian Jogl, Gitta Kutyniok, and Johannes F Lutzeyer. Graph repre- sentational learning: When does more expressivily hurt generalization? arXiv:2505. 11298, 2025.
[261] Joël Mathys, Florian Grötschla, Kalyan Varma Nadimpalli, and Roger Wattenhofer. Rethinking mes- sage passing for algorithmic alignment. In NEURIPS Workshop on Symmetry and Geometry in Neural Representations, 2024.
[262] Ryan Medonald, Mehryar Mohri, Nathan Silberman, Dan Walker, and Gideon Mann. Efficient large- scale distributed training of conditional maximum entropy models. Advances in Newral Information Processing Systems, 22, 2009.
[263] Ryan McDonald, Keith Hall, and Gideon Mann. Distributed training strategies for the structured perceptron. In Annual Conference of the North American Chapter of the Association for Computational Linguistics, 2010.
[264] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Commuiication-effiientt learning of deep networks from decentralized data. In Artificial Intelligence and Statistics, 2017.
[265] H Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. Learning iilferentially private recurrent language models. In International Conference on Learning Representations, 2018.
[266] Guangxu Mei, Ziyu Guo, Shijun Liu, and Li Pan. Sgnn: A graph neural network based federated learning approach by hiding structure. In IEEE International Conference on Big Data, 2019.
[267] Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. Exploiting unintended feature leakage in collaborative learning. In IEFF Symposium Security and Privacy, 2019.
81

---

Published in Transactions on Machine Learning Research (02/2026)
[268] Chuiyang Meng, Ming Tang, Mchdi Setayesh, and Vincent WS Wong. Tackling resource allocation for decentralized federated learning: A GNN-based approach. IEEE Transactions on Mobile Compuling, 2025.
[269] Chuizheng Meng, Sirisha Rambhatla, and Yan Liu. Cross-node federated graph neural network for spatio-temporal data modeling. In ACM SIGKDD Conference Knowledge Discovery & Data Mining, 2021.
[270] Fan Mo, Hamed Haddadi, Kleomenis Katevas, Eduard Marin, Diego Perino, and Nicolas Kourtel- lis. PPFL:
federated learning with trusted execution environments. In Annual International Conference on Mobile Systems, Applications, and Services, 2021.
[271] Bojan Mohar, Y Alavi, G Chartrand, and Ortrud Oellermann. The laplacian spectrum of graphs. Graph Theory, Combinatorics, and Applications, 2(871-898):12, 1991.
[272] Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. Geometric doep learning on graphs and manifolds using mixture model CNNs. In JEEE Conference on Computer Vision and Pullern Recognilion, 2017.
[273] Luc Morcau. Stability of multiagent systems with time-dopendent communication links. IEEE Trans- actions on Aulomalic Control, 50(2):169 182, 2005.
[274] Jose G Morcno-Torres, Troy Raeder, Rocío Alaiz-Rodriguee Nitesh V Chawla, and Francisco Herrera.
A unifying view on dataset shift in classification. Pallern Recognilion, (551 2012.
[275] Christopher Morris, Fabrizio Frasca, Nadav Dym, Haggai Maron, ismail Îlkan Ceylan, Ron Levie, Derek Lim, Michael Bronstein, Martin Grohe, and Stefanie Jegelka. Future directions in the theory of graph machine learning. Xiv::2400 02287, 2024.
[276] Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saced Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. A comprehensive overview of large language models. ACM Trunsactions Inlelligent Systems and Technology. 2023.
[277] Mahmoud Nazzal, Abdallah Khreishah, Joyoung Lee, Shaahin Angizi, Ala Al-Fuqaha, and Mohsen Guizani. Semi-decentralized inference in heterogeneous graph neural networks for traffic demand fore- casting: An edge-computing approach. IEEE Transactions Vehicular Technology, (3(12)119400- 19416, 2024.
[278] Angelia Nedic and Asuman Ozdaglar. Distributed subgradient methods for multi-agent optimization. IEEE Transoctions on Automatic Control, 54(1):48-61, 611 2009.
[279] Angelia Nedic, Alex Olshevsky, Asuman Ozdaglar, and John Tsitsiklis. On distributed averaging algorithms and quantization effects. IEEE Transactions on Automotic Control, 54(11):2506-2511) 2009.
[280] Angelia Nedic, Asuman Ozdaglar, and Pablo A Parrilo. Constrained consensus and optimization in multi-agent networks. IEEE Trensactions on Automatic Control, (5(4):922-99) 922, 2010.
[281] Yurii Nesterov. Primal-dual subgradient methods for convex problems. Mathematical Programming, 120(1):221-22) 2:212299, 2009.
[282] John Nguyen, Kshitiz Malik, Hongyuan Zhan, Ashkan Yousefpour, Mike Rabbat, Mani Malek, and Dzmitry Huba. Federated learning with buffered asynchronous aggregation. In International Confer- ence on Artificial Intelligence and Statistics, 2022.
[283] Xiang Ni, Xiaolong Xu, Lingjuan Lyu, Changhua Meng, and Weiqiang Wang. A vertical federated learning framework for graph convolutional network. arXiv:2106. 1 11593, 2021.
[284] Mingshuo Nie, Dongming Chen, and Dongqi Wang. Reinforcement learning on graphs: survey. IEEE Transactions on Emerying Topics in Compulational Intelligence, 7(4):1065-1082, 2023.
82

---

Published in Transactions on Machine Learning Research (02/2026)
[285] Takayuki Nishio and Ryo Yonetani. Client selection for federated learning with heterogencous resources in mobile edge. In IEEE International Conference. on Communicctions, 2019.
[286] Thomas Nisonger. The "80/20 rule" and core journals. The Serials Librarian, 55(12): 84, 2008,
[287] Yongjeong Oh, Yo-Seb Jeon, Mingzhe Chen, and Walid Saad. Fedvqes: Federated learning via vector quantized compressed sensing. IEEE Transactions on Wireless Communications, 23(3):1755-177) 2024.
[288] lyiola E Olatunji, Wolfgang Nejdl, and Megha Khosla. Membership inference attack on graph neural networks. In IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications, 2021.
[289] Reza Olfati-Saber and Richard M Murray. Consensus problems in networks of agents with switching topology and timo-dclays. IEEE Transactions on Automatic Control, 49(9):1520-15) 1533,, 2004.
[290] Reza Olfati-Saber, J Alex Fax, and Richard M Murray. Consensus and cooperation in networked multi-agent systems. IEEE Conference on Decision and Control, 95(1):215-233, 2007.
[291] Alex Olshevsky and John N Tsitsiklis. Convergence speed in distributed consensus and averaging, SIAM Journal on Control and Optimization, 48(1):33-50 33)5,,35,,,300 2009.
[292] Rostyslav Olshevskyi, Zhongyuan Zhao, Kevin Chan, Gunjan Verma, Ananthram Swami, and Santiago Segarra. Fully distributed online training of graph neural networks in networked systems. In IEEE International Conference on Machine Learning for Communication and Networking, 2025.
[293] Noa. Onoszko, Gustav Karlsson, Olof Mogren, and Edvin Listo Zec. Decentralized federated learning of deep neural networks on non-iid data. In International Workshop on Federated Learning for User Privacy and Data Confidentiality in Conjunction with ICML, 2021.
[294] Róbert Ormándi, István Hegedüs, and Márk Jelasity. Gossip learning with linear models on fully distributed data. Concurrency and Computation: Practice and Experience, 25(4):556 571, 2013.
[295] Qiying Pan and Yifei Zhu. Fedwalk: Communication efficient federated unsupervised node embedding with differential privacy. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022.
[296] Qiying Pan, Yifei Zhu, and Lingyang Chu. Lumos: Heterogeneily-aware federated graph learning over decentralized devices. In IEEE International Conference on Data Engineering, 2023.
[297] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEFE Transactions Knowledge and Data Engineering, 22(10):1345 1359, 2009.
[298]
Theodore Papamarkou, Tolga Birdal, Michael Bronstein, Gunnar Carlsson, Justin Curry, Yue Gao, Mustafa Hajij, Roland Kwitt, Pietro Liò, Paolo Di Lorenzo, Vasileios Maroulas, Nina Miolane, Farzana Nasrin, Karthikeyan Natesan Ramamurthy, Bastian Rieck, Simone Scardapane, Michael T. Schaub, Petar Veličković, Bei Wang, Yusu Wang, Guo-Wei Wei, and Ghada Zamzmi. Position: Topological deep learning is the new frontier for relational learning, In International Conference on Machine Learning, 2024.
[299] Pál András Papp and Roger Wattenhofer. An introduction to graph neural networks from a dis- tributed computing perspective. In International Conference on Distributed Computing and Internet Technology, 2022.
[300] Jong-Ik Ik Park and Carlee Joc-Wong Federated learning with flexible architectures. In Joint European Conference on Machine Learning and Knowledgge Discovery in Dalabases, 2024.
[301] Jong-Ik Ik Park and Carlce Joe- Mee-Wnng. Fedtlu: Federated learning with targeted layer updates. In JEEE International Conference on Acoustics, Speech and Signal Processing, 2025.
83

---

Published in Transactions on Machine Learning Research (02/2026)
[302] Matthias Paulik, Matt Seigel, Henry Mason, Dominic Telaar, Joris Kluivers, Rogier van Dalen, Chi Wai Lau, Luke Carlson, Filip Granqvist, Chris Vandevelde, Sudeep Agarwal, Julien Freudiger, Andrew Byde, Abhishek Bhowmick, Gaurav Kapoor, Si Beaumont, Aine Cahill, Dominic Hughes, Omid Javid- bakht, Fei Dong, Rehan Rishi, and Stanley Hung. Federated ovaluation and tuning for on-device System design & applications. Technical report, Apple, 2021.
personalization:
[303] Xinjun Pei, Xiaoheng Deng, Shengwei Tian, Jianqing Liu, and Kaiping Xue. Privacy-enhanced graph neural network for decentralized local graphs. IEEE Tronsactions on Information Forensics and Se- curily, 19:1614 1629, 2023.
[304] Hao Peng, Haoran Li, Yangqiu Song, Vincent Zheng, and Jianxin Li. Differentially private feder- ated knowledge graphs embedding. In ACM International Conference on Information & Knowledge Management, 2021.
[305] Hongwu Peng, Ran Ran, Yukui Luo, Jiahui Zhao, Shaoyi Huang, Kiran Thorat, Tong Geng, Chenghong Wang, Xiaolin Xu, Wujie Wen, et al. LinGCN: Structural linearized graph convolutional network for homomorphically encrypted inference. In Advances in Newral Information Processing Systems, 2023.
[306] Jingshu Peng, Zhao Chen, Yingxia Shao, Yanyan Shen, Lei Chen, and Jiannong Cao. Sancus: Staleness- aware coaauumiiaaiionvvviiiiii full-graph decentralized training in large-scale graph neural networks. Proceedings of the VLDB Endoument, 15(9):1937-1955) 2022.
[30]] Liang Peng, Nan Wang, Nicha Dvornek, Xiaofeng Zhu, and Xiaoxiao Li. Fedni: Federated graph learn- ing with network inpainting for population-based disease prediction. IEEE Transactions Medical Imaging, 42(7):2032-200) 2022.
[308] Le Trieu Phong, Yoshinori Aono, Takuya Hayashi, Lihua Wang, and Shiho Moriai. deep learning via additively homomorphic encryption. IEFE Transactions on Information Forensics and Security, 13(5):1333 1345, 2017.
[309] Vasyl Pihur, Aleksandra Korolova, Frederick Liu, Subhash Sankuratripati, Moti Yung, Dachuan Huang, and Ruogu Zeng. Differentialll-priies "draw and discard" machine learning: Training distributed model from enormous crowds. In International Symposium on Cyber Security, Cryptology, and Machine Learning, 2022.
(310] Benny Pinkas, Thomas Schneider, and Michael Zohner. Scalable private set intersection based on ot extension. ACM Transactions on Privacy and Security 21(2):1-35, 2018.
[311] Oleg Platonov, Denis Kuznedelev, Artem Babenko, and Liudmila Prokhorenkova. Characterizing graph datasets for node classification: Hoomopalyy-hetrrohhiy dichotomy and beyond. Advances in Neural Information Processing Systems, 36:523 548, 2023.
[312] Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova. A critical look at the evaluation of GNNs under heterophily: Are we really making progress? In International Conference on Learning Representations, 2023.
[313] Daniel Pérez Herrera, Zheng Chen, and Erik G. Larsson. Faster convergence with less communication: Broadcast-basel. subgraph sampling for decentralized learning over wireless networks. TEEE Open Journal of the Communications Society, 2025.
[314] Chendi Qian, Gaurav Rattan, Floris Geerts, Mathias Niepert, and Christopher Morris. Ordered sub- graph aggregation networks. Advances in Neural Information Processing Systems, 0--21030-21444 2022.
[315] Yeqing Qiu, Chenyu Huang, Jianzong Wang, Zhangchong Huang, and Jing Xiao. A preeacypg subgraph-level federated graph neural network via differential privacy. In International Conference on Knowledge Science, Engineering and Management, 2022.
84

---

Published in Transactions on Machine Learning Research (02/2026)
[316] Liang Qu, Ningzhi Tang, Ruiqi Zheng, Quoe Viet Hung Nguyen, Zi Huang, Yuhui Shi, and Hongzhi Yin. Semi-decentralized federated ego graph learning for recommendation. In ACM Web Conference, 2023.
[37] Joaquin Quiionero-Candeln, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. Dataset Shift in Machine Learning. MIT Press, 2022.
[318] S Sundhar Ram, Nedić, and Venugopal Voeravalli. Asynchronous gossip algorithms for stochastic optimization. In IEEE Conference on Decision and Control, 2009.
[319 S Sundhar Ram, A Nedić, and Venugopal V Vecravalli. Ineremental stochastic subgradient algorithms for convex optimization. SIAM Journal on Oplimization,, 20(2):691 717, 2009.
[320] Swaroop Ramaswany, Om Thakkar, Rajiv Mathews, Galen Andrew, H. Brendan McMahan, and Frangoise Beaulays. Training production language models without memorizing user data. Technical report, Google, 2020.
[321] Ran Ran, Wei Wang, Quan Gang, Jieming Yin, Nuo Xu, and Wujie Wen. CryptoGCN: Fast and scalable homomorphically encrypted graph convolutional network inference. In Advances in Newral Information Processing Systems, 2022.
[322] Saeed Razavikia, José Mairton Barros da Silva, and Carlo Fischione. ChannelComp: A general method for computation by communications. IEEE Transactions on Communications, 6:1497-1511, 2024.
[323] Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and H. Brendan McMahan. Adaptive federated optimization. In International Conference Learning Represenlations, 2021.
[324] Jinke Ren, Yinghui He, Dingzhu Wen, Guanding Yu, Kaibin Huang, and Dongning Guo. Scheduling for cellular federated edge learning with importance and channel awareness. IEEE Transactions Wireless Communiccatinns, 19(1)):7690 7703, 2020.
[325] Zhenghang Ren, Liu Yang, and Kai Chen, Improving availability of vertical federated learning: Re- laxing inference on non-overlapping data. ACM Transactions on Intelligent Systems and Techmology, 13(4):1-20, 2022.
[326] Mónica Ribero, Haris Vikalo, and Gustavo De Veciana. Federated learning under intermittent client availability and time-varying communication constraints. IEEE Journal of Selected Topics in Signal Processing, 17(1):98 111, 2022.
[327] Herbert Robbins and Sutton Monro. A stochastic approximation method. The Annals of Mathematical Statisties, pp. 400 407, 1951.
[328] Angelo Rodio, Francescomaria Faticanti, Othmane Marfoq, Giovanni Neglia, and Emilio Leonardi. Federated learning under heterogeneous and correlated client availability. In IEEE Infocom Conference on Computer Communications, 2023.
[329] Frank Rosenblatt. The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, (86,) 1958.
[330] Yichen Ruan, Xiaoxi Zhang, Shu-Cha Liang, and Carlee Joe- Wong. Towards flexible device partic- ipation in federated learning. In International Conference on Artificial Intelligence and Statistics, 2021.
[331] David E Rumelhart, Geoffrey Hinton, and Ronald J Williams. Learning representations by back- propagating errors. Natwre, 923(6089):5333-36 1986.
[332] Fahad Sabah, Yuwen Chen, Zhen Yang, Muhammad Azam, Nadcom Ahmad, and Rahcem Sarwar. Model optimization techniques in personalized federated learning: A survey. Expert Systems with Applications, 243:122874 2024.
85

---

Published in Transactions on Machine Learning Research (02/2026)
[333] R. Olfati Saber and Richard M Murray. Consensus protocols for networks of dynamic agents. In IEEE American Control Conference, 2003.
[334] Felix Sattler, Klaus- Roauu- Müller, and Wojciech Samek. Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints. IEEE Transactions on Newral Networks and Learning Systems, 02(8):3710-377 3722, 2020.
[335] Simone Scardapane, Indro Spinelli, and Paolo Di Lorenzo. Distributed training of graph convolutional networks. IEEE Transactions on Signal and Information Processing over Networks, 7:87-100, 2020.
[336] Aviv Shamsian, Aviv Navon, Ethan Fetaya, and Gal Chechik. Personalized federated learning using hypernetworks. In International Conference on Machine Learning. 2021.
[337] Yingxia Shao, Hongzheng Li, Xizhi Gu, Hongbo Yin, Yawen Li, Xupeng Miao, Wentao Zhang, Bin Cui, and Lei Chen. Distributed graph neural network training: A survey. ACM Computing Surveys, 56(8):1-39, 2024.
[338] Wei Shen, Mang Ye, Wei Yu, and Pong Yuen. Build yourself before collaboration: Vertical federated learning with limited aligned samples. IEEE Transactions on Mobile Computing, 24(7):6503 65168, 2025.
[339] Xiao Shen, Quanyu Dai, Fu-lai Chung, Wei Lu, and Kup-Sze Choi. Adversarial deep network embed- ding for cross-nelwork node classification. In AAAI Conference on Artificial Intelligence, 2020.
[340] Zitong Shi, Guancheng Wan, Wenke Huang, Guibin Zhang, He Li, Carl Yang, and Mang Ye. Eagles: Towards effective, efficient, and economical federated graph learning via unified sparsification. Tn International Conference Machine Learning, 2025.
[341] Nir Shlezinger, Mingzhe Chen, Yonina C. Eldar, Н. Vincent Poor, and Shuguang Cui. Uveqfed: Universal vector quantization for federated learning. IEEE Transactions on Signal Processing, 69: 500 514, 2021.
[342] Neta Shoham, Tomer Avidor, Aviv Keren, Nadav Israel, Daniel Benditkis, Liron for-Yosef, and Ttai Zeitak. Overcoming forgetting in federated learning non-iid data. In NEURIPS Workshop Federated Learning for Data Privacy and Confidentialitt, 2019.
[343] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. JEEE Symposium on Security and Privacy, 2017.
[344] Ilia Shumailov, Daniel Ramage, Sarah Meiklejohn, Peter Kairouz, Florian Hartmann, Borja Balle, and Eugene Bagdasarian. Trusted machine learning models unlock private inference for problems currently infcasible with cryptography. Technical report, Google, 2025.
[345] Adam Smith, Abhradeep Thakurta, and Jalaj Upadhyay. interaction necessary for distributed private learning? In IEEE Symposium on Security and Privacy, 2017.
[346] Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet Talwalkar. Federated multi-task learning. Advances in Neural Information Processing Systems, 30, 2017.
[347] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Ad- vances in Neural Information Processing Systems, 2017.
[348] John Snow. On the mode of communication of cholera. Edinburgh medical journal, 1855.
[349] Olga Solodova, Nick Richardson, Deniz Oktay, and Ryan P Adams. Graph neural networks gone hogwild. In The International Conference on Learning Representations, 2025.
[350] Shuang Song, Kamalika Chandhuri, and Anand D Sarwate. Stochastic gradient descent with differen- tially private updates. In IEFE Global Conference on Signal and Information Processing, 2013.
86

---

Published in Transactions on Machine Learning Research (02/2026)
[351] Joshua Southern, Yam Eitan, Guy Bar-Shalom, Michael M. Bronstein, Haggai Maron, and Fabrizio Frasca. Balancing efficiency and expressiveness: Subgraph GNNs with walk-based centrality. In Inter- national Conference on Machine Learning, 2025.
[352] Sebastian U. Stich. Local SGD converges fast and communicates little. In International Conference on Learning Representations, 2019.
[353] Lichao Sun, Yingtong Dou, Carl Yang, Kai Zhang, Ji Wang. Philip S Yu, Lifang He, and Bo Li. Adversarial attack and defense on graph data: A survey, IEEE Transactions on Knowledge and Data Engineering. 35(8):7698-7711, 2022.
[35] Youbang Sun, Zitao Li, Yaliang Li, and Bolin Ding. Improving LoRA in crivacy-preseviing federated learning. In International Conference on Learning Representalions, 2024.
[355] Zhenchao Sun, Hongzhi Yin, Hongxu Chen, Tong Chen, Lizhen Cui, and Fan Yang. Disease prediction via graph neural networks. IEEE Journal of Biomedical and Health Informaties, 25(3):818-822 2020.
[356] Zhenyu Sun, Ziyang Zhang, Zheng Xu, Gauri Joshi, Pranay Sharma, and Ermin Wei. Debiasing federated learning with correlated client participation. In International Conference on Learning Rep- resentations, 2025.
[357] Canh T Dinh, Nguyen Tran, and Josh Nguyen. Personalized federated learning with Moreau envelopes. Advances in Newral Information Processing Systems, 33:21394 21 404, 2020.
[358]
Kunal Talwar, Shan Wang, Audra McMillan, Vojta Jina, Vitaly Feldman, Pansy Bansal, Bailey Basile, Aine Cahill, Yi Sheng Chan, Mike Chatzidakis, Junye Chen, Oliver Chick, Mona Chitnis, Suman Ganta, Yusuf Goren, Filip Granqvist, Kristine Guo, Frederic Jacobs, Omid Javidbakht, Albert Liu, Richard Low, Dan Mascenik, Steve Myers, David Park, Wonhee Park, Gianni Parsa, Tommy Pauly, Christian Priebe, Rchan Rishi, Guy Rothblum, Michael Scaria, Linmao Song, Congzheng Song, Karl Tarbe, Sebastian Vogt, Luke Winstrom, and Shundong Zhou. Samplable anonymous aggregation for private federated data analysis. In ACM SIGSAC Conference on Computer and Communications Security, 2024.
[359] Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang. Towards personalized federated learning. IEEE Transactions on Neural Networks and Learning Systems, 2022.
[360] Yue Tan, Guodong Long, Lu Lit, Tianyi Zhou, Qinghua Lu, Jing Jiang, and Chengqi Zhang. Fed- Proto: Foderated prototype learning across heterogeneous clients. In AAAT Conference on Artificial Intelligence, 2022a.
[361] Yue Tan, Guodong Long, Jie Ma, Lu Liu, Tianyi Zhou, and Jing Jiang. Federated learning from pre-trained models: A contrastive learning approach. In Advances in Neural Information Processing Systems, 2022b.
[362] Zihan Tan, Guancheng Wan, Wenke Huang, and Mang Ye. FedSSP: Federated graph learning with spectral knowledge and personalized preference. In Advances in Neural Information Processing Sys- tems, 2024.
[363] Zihan Tan, Guancheng Wan, Wenke Huang, He Li, Guibin Zhang, Carl Yang, and Mang Ye. Fedspa: Generalizable foderated graph learning under homophily heterogeneity. In Computer Vision and Pat- tern Recognâtion Conference, 2025a.
3364 Zihan Tan, Suyuan Huang, Guancheng Wan, Wenke Huang, He Li, and Mang Ye. S2FGL: Spatial spectral fodcrated graph learning. In International Conference on Machine Learning, 2025b.
[365] Yijun Tian, Shichao Pei, Xiangliang Zhang, Chuxu Zhang, and Nitesh V Chawla. Knowledge distilla- tion graphs: A survey. ACM Computing Surveys, 57(8):1-16, 2025.
87

---

Published in Transactions on Machine Learning Research (02/2026)
[366] Jake Topping, Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong, and Michael M. Bronstein. Understanding over-squashing and bottlenecks on graphs via curvature. In International Conference on Learning Representations, 2022.
[367] John Tsitsiklis, Dimitri Bertsekas, and Michael Athans. Distributed asynchronous deterministic and stochastic gradient optimization algorithms. IEEE Trunsactions on Automatic Control, (1)9)8803 812, 1986.
3368 Kübra Uludag, Çigdem Eroglu Erdem, and Omer Korçak. Leveraging unlabeled data in federated learning: A review. IEEE Access, 13:181719-88174,, 2025.
[369] Saeed Vahidian, Mahdi Morafah, and Bill Lin. Personalized federated learning by structured and unstructured pruming under data heterogeneity. In IEEE International Conference Distributed Computing Systems Workshops, 2021.
[370] Leslie G Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134 1142, 1984.
[371] Paul Vanhacsebrouck, Aurélien Bellet, and Marc Tommasi. Decentralized collaborative learning of personalized models over networks. In Niernational Conference Arlificial Intelligence and Stalislics,
2017.
[322] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.
[373] Petar Veličković, William Fodus, William L. Hamilton, Pictro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. In International Conference on Learning Representations, 2019.
[374] Praneeth Vepakomma, Otkrist Gupta, Tristan Swedish, and Ramesh Raskar. Split learning for health: Distributed deep learning without sharing raw patient data. In ICLR Workshop on Al for Social Good, 2019.
[375] Tamás Viesek, András Czirók, Eshel Ben-Jacob, Inon Cohen, and Ofer Shochet. Novel type of phase transition in a system of self-driven particles. Physical Review Letters, 75(6):1226, 1995.
[376] Binghui Wang, Ang Li, Meng Pang, Hai Li, and Yiran Chen. Graphíl:: A federated learning framework for semi-supervised node classification graphs. In IEEE International Conference Data Mining, 2022.
[377] Hao Wang, Zakhary Kaplan, Di Niu, and Baochun Li. Optimizing federated learning on non-iid data with reinforcement learning. In JEEE Infocom Conference on Computer Communications, 2020.
[378] Jialei Wang, Mladen Kolar, and Nathan Srerbo. Distributed multi-task learning. In International Conference on Artificial Intelligence and Statistics, 2016.
[379] Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H. Vincent Poor. Tackling the objective inconsistency problem in heterogeneous federated optimization. In Advances in Neural Information Processing Systems, 2020.
[380] Kaibin Wang, Qiang He, Feifei Chen, Chunyang Chen, Faliang Huang, Hai Jin, and Yun Yang, Flexifed: Personalized foderated learning for edge clients with hetcrogencous model architectures. In ACM Web Conference, 2023.
[381] Ning Wang, Xiaokui Xiao, Yin Yang, Jun Zhao, Siu Cheung Hui, Hyejin Shin, Junbum Shin, and Ge Yu. Collecting and analyzing multidimensional data with local differential privacy. In IEFE International Conference on Data Engineering, 2019.
[382] Shilong Wang, Jianchun Liu, Hongli Xu, Chenxia Tang, Qianpiao Ma, and Liusheng Huang. Towards communicttion-effccene decentralized foderated graph learning over non-iid data. arXiv:2509.0m 08.09 2025.
88

---

Published in Transactions on Machine Learning Research (02/2026)
[383] Shiqiang Wang and Mingyue Ji. A unified analysis of federated learning with arbitrary client partici- pation. Advances in Newral Information Processing Systems, 1411112--1933, 2022.
[384] Shiqiang Wang and Mingyue Ji. A lightweight method for tackling unknown participation statistics in federated averaging. In International Conference on Learning Representations, 2024.
[385] Shuai Wang, Yik-Chung Wu, Minghua Xia, Rui Wang, and H. Vincent Poor. Machine intelligence at the edge with learning centric power allocation. IEEE Transactions on Wireless Communications, 21 (10):7926-79h)) 2020.
[386] Songlei Wang, Yifeng Zheng, and Xiaohua Jia. SecGNN: siivacy-preservigg graph neural network training and inference as a cloud service. IEEE Transactions on Services Computing, 16(4):2923-2938, 2023.
[387] Xinghan Wang, Anusha Lalitha, Tara Javidi, and Farinaz Koushanfar. Peer-to-per variational fed- erated learning over arbitrary graphs. IEEE Journal on Selected Areas Information Theory, 30(2): 129-150, 2022.
[388] Xiyuan Wang and Muhan Zhang. How powerful are spectral graph neural networks. In International Conference Machine Learning, 2022.
(389] Yiwei Wang, Wei Wang, Yuxuan Liang, Yujun Cai, and Bryan Hooi. Mixup for node and graph classification. In Web Conference, 2021.
[390] Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph emn for learning on point clouds. ACM Transactions on Graphies 38(5):1- 12, 2019.
[391] Yuzhuo Wang, Hongzhi Wang. Xianglong Liu, and Yu Yan. GFedKK: GNN-based federated embedding model for knowledge graph completion. Knowledge- Based Systems, 301:112290, 2024.
[392] Zehong Wang, Zheyuan Liu, Tianyi Ma, Jiazheng Li, Zheyuan Zhang, Xingbo Fu, Yiyang Li, Zhengqing Yuan, Wei Song, Yijun Ma, Qingkai Zeng, Xiusi Chen, Jianan Zhao, Jundong Li, Meng Jiang, Pictro Lio, Nitesh Chawla, Chuxu Zhang, and Yanfang Ye. Graph foundation models: A comprehensive survey. arXi:22505.1111 15116, 2025.
[393] Zheng Wang, Xiaoliang Fan, Jianzhong Qi, Haibing Jin, Peizhen Yang, Siqi Shen, and Cheng Wang. Fedgs: Federated graph-based sampling with arbitrary client availability. In AAAI Conference on Arlificial Intelligence, 2023.
[394] Zhiyang Wang, Mark Eisen, and Alejandro Ribeiro. Learning decentralized wireless resource allocations with graph neural networks. IEEE Transactions on Signal Processing, 70:1850-1863, 2022.
[395] Ermin Wei and Asuman Ozdaglar. Distributed alternating direction method of multipliers. In IEEE Conference on Decision and Control, 2012.
(396] Jinliang Wei, Wei Dai, Aurick Qiao, Qirong Ho, Henggang Cui, Gregory R Ganger, Phillip Gibbons, Garth A Gibson, and Eric P Xing. Managed communication and consistency for fast data-parallel iterative analytics. In ACM Symposium on Cloud Computing, 2015.
[397] Wanli Wen, Zihan Chen, Howard H Yang, Wenchao Xia, and Tony QS Quek. Joint scheduling and resource allocation for hierarchical federated edge learning. IEEE Trunsactions on Wireless Commu- nications, 21(8):5851 (185)55577577777 2022.
[398] Mariel Werner, Lie He, Michael Jordan, Martin Jaggi, and Sai Pranceth Karimireddy. Provably personalized and robust federated learning. Transactions on Machine Learning Research, 2024.
[399] Chuhan Wu, Fangzhao Wu, Yang Cao, Yongfeng Huang, and Xing Xie. Fedgnn: Federated graph neural network for preservin recommendation. Xrx:21102. 04925, 2021.
89

---

Published in Transactions on Machine Learning Research (02/2026)
[400] Chuhan Wu, Fangzhao Wu, Lingjuan Lyu, Tao Qi, Yongfeng Huang, and Xing Xic. A federated graph neural network framework for privacy-preservigg personalization. Nature Communicaaions, 13(1))3091, 2022.
[401] Fan Wu, Yunhui Long, Ce Zhang, and Bo Li. Linkteller: Recovering private odges from graph neural networks via influence analysis. In IEEE Symposium on Security and Privacy, 2022.
[402] Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Simplifying graph convolutional networks. In International Conference on Machine Learning, 2019.
[403] Qiong Wu, Xu Chen, Zhi Zhou, and Junshan Zhang. Fedhome: Cloud-odge based personalized feder- ated learning for in-home health monitoring. IEFE Transactions on Mobile Computing, 21 (8)) 2832, 2020.
[404] Qiong Wn, Xu Chen, Tao Ouyang, Zhi Zhou, Xiaoxi Zhang, Shusen Yang, and Junshan Zhang, Hiflash: Communccation-eficiiettt hierarchical federated learning with adaptive staleness control and heterogeneity-awii elient-edge association. IEEE Transactions on Parallel and Distributed Systems, 34(5):1560 1579, 2023.
[405] Ying-Xin Xin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat-Seng Chua. Discovering invariant rationales for graph neural networks. arXiv:2201. 12872, 2022.
[406] Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. Moleculenet: A benchmark for molecular machine learning. Chemical Science, 9(2):513 530, 2018.
[407] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A com- prehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1): :4-44:4444 2020.
[408] Ming Xiang, Stratis loannidis, Edmund Yeh, Carlee Joe- foe-Wong, and Lili Su. Elficient federated learn- ing against heterogeneous and non-stationary client unavailability. Advances in Newral Information Processing Systems, 37:104281-104322 2024.
[409] Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta. Asynchronous federated optimization. In Annual Workshop on Optimization for Machine Learning, 2020.
[410] Han Xie, Jing Ma, Li Xiong, and Carl Yang. Federated graph classification over non-iid graphs Advances in Newral Information Processing Systems, 34:18839-18852, 2021.
[411] Penglao Xie, Jin Kyu Kim, Yi Zhou, Qirong Ho, Abhimanu Kumar, Yaoliang Yu, and Eric Xing. Distributed machine learning via sufficient factor broadcasting. arXie:1511.08488 2015.
[412] Fric P Xing, Qirong Ho, Pengtao Xie, and Dai Wei. Strategies and principles of distributed machine learning on big data. Engineering, 2(2):179 195, 2016.
[413] Zheng Xu, Yanxiang Zhang, Galen Andrew, Christopher A Choquette-Choo Peter Kairouz, H Brendan McMahan, Jesse Rosenstock, and Yuanbo Zhang, Federated learning of gboard language models with differential privacy. Technical report, Google, 2023.
[414] Bo Yan, Yang Cao, Haoyu Wang, Wenchuan Yang, Junping Du, and Chuan Shi. Federated hei- erogeneous graph neural network for privacy-preserve recommendation. In ACM Web Conference, 2024.
[415] Howard H Yang, Zuozhu Liu, Tony QS Quck, and H Vincent Poor. Scheduling policies for federated learning in wireless net works IEEE Transactions on Communicationss, 68(1):3177333) -33,, 2019.
[416] Miao Yang, Ximin Wang, Hongbin Zhn, Haifeng Wang, and Hua Qian. Federated learning with class imbalance reduction. In IEF.E. Farropaan Signal Processing Conference, 2021.
90

---

Published in Transactions on Machine Learning Research (02/2026)
[417] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology, 10(2):1-19, 2019.
[418] Tao Yang, Xinlei Yi, Junfeng Wu, Ye Yuan, Di Wu, Ziyang Meng. Yiguang Hong, Hong Wang, Zongli Lin, and Karl H Johansson. A survey of distributed optimization. Annual Reviews Control, 47: 278 305, 2019.
[419] Timothy Yang, Galen Andrew, Hubert Eichner, Haicheng Sun, Wei Li, Nicholas Kong, Daniel Ramage, and Françoise Beaufays. Applied federated learning: Improving Google keyboard query suggestions. Technical report, Google, 2018.
[20 Yitao Yang, Xiucai Ye, and Tetsuya Sakurai. Multi-view federated learning with data collaboration. In International Conference on Machine Learning, 2022.
[421] Xin Yao and Lifeng Sun. Contimual local training for better initialization of federated models. In JEEE International Conference on Image Processing, 2020.
[422] Yuhang Yao, Weizhao Jin, Srivatsan Ravi, and Carlee Joe- Wong. Fedgen: Convergenc-oommiiatton tradeoffs in lederated training of graph convolutional networks. Advances in Neural Information Pro- cessing Systems, 36:79748 79760, 2023.
[423] Gilad Yehudai, Ethan Fetaya, Eli Meirom, Gal Chechik, and Haggai Maron. From local structures size generalization in graph neural networks. In International Conference on Machine Learning, 2021.
[424] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. Advances in Newral Information Processing Systems, 33: 5812-5823, 2020.
[425] Hao Yu, Sen Yang, and Shenghuo Zhu. Parallel restarted SGD with faster convergence and less communication: Demystifying why model averaging works for deep learning. In AAAI Conference Artificial Intelligence, 2019.
[426] Hui Yu, Yu Zhang, Ligang He, Donghao He, Qikun Li, Jin Zhao, Xiaofei Liao, Hai Jin, Lin Gu, and Haikun Liu. CDA-GNN: chain-driven accelerator for efficient asynchronous graph neural network. In ACM/IEEE Design Automation Conference, 2024.
[427] Edvin Listo Zec, Ebba. Ekblom, Martin Willbo, Olof Mogren, and Sarmnas Girdzijauskas. Decentralized adaptive clustering of deep nets is beneficial for client collaboration. In International Workshop on Trustworthy Federated Learning, 2022.
[428] Edvin Listo Zec, Johan Östman, Olof Mogren, and Daniel Gillblad. Efficient node selection in private personalized decentralized learning. In Northern Lights Deep Learning Conference, 2024.
[429] Liekang Zeng, Chongyu Yang, Peng Huang, Zhi Zhou, Shuai Yu, and Xu Chen. GNN at the edge: Cost-efficient graph neural network processing over distributed edge servers. IEEE Journal on Selected Arens in Communicationns, :1(3):720 739, 2022.
[430] Chen Zhang, Yu Xie, Hang Bai, Bin Yu, Weihong Li, and Yuan Gao. A survey on federated learning. Knowledge--aase Based Systems, 216:106775, 2021.
[431] Chenhan Zhang, Shuyu Zhang, JQ James, and Shui Yu. FASTGNN: A topological information pro- tected federated learning approach for traffic speed forecasting. IEEE Transactions on Industrial Informatics, 17(12):8464 8474, 2021.
[432] Hongyi Zhang, Moustapha Cisso, Yann N Dauphin, and David Lopez-Paz. Mixup: Beyond empirical risk minimization. In International Conference on Learning Represenlations, 2018.
[433] Huanding Zhang, Tao Shen, Fei Wu, Mingyang Yin, Hongxia Yang, and Chao Wu. Foderated graph learning-a position paper. Xin:21000.1 10111015.9199 11099, 2021.
91

---

Published in Transactions on Machine Learning Research (02/2026)
[434] Jianping Zhang, Lei Yang, Seyed Mahmoud Sajjadi Mohammadabadi, and Feng Yan. A survey self-supervised learning: Recent advances and open problems. Neurocompuling 655:131409, 2025.
[435] Kai Zhang, Yu Wang, Hongyi Wang, Lifu Huang, Carl Yang, and Lichao Sun. Efficient federated lcarning on knowledge graphs via privacy-preserving relation embedding aggregation. In ACL Workshop on Federaled Learning for Natural Language Processing, 2022.
[436] Ke Zhang, Carl Yang, Xiaoxiao Li, Lichao Sun, and Siu Ming Yiu. Subgraph federated learning with missing neighbor generation. Advances in Neural Information Processing Systems, 34:6671-6682, 2021.
[437] Kexin Zhang, Shuhan Liu, Song Wang, Weili Shi, Chen Chen, Pan Li, Sheng Li, Jundong Li, and Kaize Ding. A survey of deep graph learning under distribution shifts: From graph out-of-distribution generalization to adaptation. arXiv:24 10.1 00.1925, 2024.
[438] Taolin Zhang, Chengyuan Mai, Yaomin Chang, Chuan Chen, Lin Shu, and Zibin Zhong. Fedego: Privacy-preserving personalized federated graph learning with ego-graphs. ACM Transactions on Knowledge Discovery from Data, 18(2):1-22 27, 2023.
[439] Yuchen Zhang, Martin J Wainwright, and John C Duchi. Commuicattin--fficinn algorithms for statistical optimization. Advances in Newral Information Processing Systems, 25, 2012.
[440] Zaixi Zhang, Qi Liu, Zhenya Huang, Hao Wang, Chengqiang Lu, Chuanren Liu, and Enhong Chen. GraphMI: Extracting private graph data from graph neural networks. In International Joint Conference on Artificial Intelligence, 2021.
[441] Zezhong Zhang, Guangxu Zhu, Rui Wang, Vincent K. N. Lau, and Kaibin Huang, Turning channel noise into an accelerator for over-the-air principal component analysis. IEEE Transactions on Wireless Communications, 2022.
[442] Zhenxiao Zhang, Zhidong Gao, Yuanxiong Guo, and Yanmin Gong. Scalable and low-latency federated learning with cooperative mobile edge networking. IEEE Transactions on Mobile Computing, 23(1): 812 822, 2022.
[443] Bo Zhao, Konda Roddy Mopuri, and Hakan Bilen. iDLG: Improved deep leakage from gradients. arXiv:2001.02611 2020.
[444] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. ar 00582, 2018.
[445] Xiaolin Zheng, Zhongyu Wang, Chaochao Chen, Jiashu Qian, and Yao Yang. Decentralized graph neu- ral network for privacy-preseringg recommendation. In ACM International Conference Information and Knowledge Management, 2023.
[446] Dengyong Zhou, Olivier Bousquet, Thomas Lal, Jason Weston, and Bernhard Schölkopf. Learning with local and global consistency. Advances in Neural Information Processing Systems, 16, 2003.
[447] Lifeng Zhou, Vishmu D Sharma, Qingbiao Li, Amanda Prorok, Alejandro Ribeiro, Pratap Tokekar, and Vijay Kumar. Graph neural networks for decentralized multi-robot submodular action selection. arXiv:205508660 2021.
[448] Shenglong Zhou and Geoffrey Ye Li. Federated learning via inexact ADMM. JEEE Transactions on Pattern Analysis and Machine Intelligence, 45(8):9699-9777 2023.
[449] Xiaokang Zhou, Xiaozhou Ye, Kevin, Kai Wang, Wei Liang Nirmal Kumar C Nair, Shohei Shimizu, Zheng Yan, and Qun Jin. Hicrarchical foderated learning with social context clustering- basod partici- pant selection for internet of medical things applications. IEEE Transactions on Computational Social Systems, 10(4):1742 1751, 2023.
-
92

---

Published in Transactions on Machine Learning Research (02/2026)
[450] Zhou Zhou, Youliang Tian, and Changgen Peng. Privacy-preservin federated learning framework with general aggregation and multiparty entity matching. Wineless Communicaliins and Mobile Compuling, (026(1):692061, 2021.
[451] Hangyu Zhu, Jinjin Xu, Shiqing Liu, and Yaochu Jin. Federated learning on non-iid data: A survey. Newrocomputing, 465:371-390, 2021.
[452] Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. Aduances in Neural Information Processing Systems, 33:7793- 7804, 2020.
[453] Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. In Advances in Neural Infor- mation Processing Systems, 2019.
[454] Xiaojin Zhu and Zoubin Ghahramani. Learning from labeled and unlabeled data with label propaga- tion. Technical report, Carnegie Mellon University, 2002.
[455] Xiaojin Jerry Zhu. Semi-supervised learning literature survey, CS Technical Reports University of WiscoosinnMMadiiionn, 2005.
[456] Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou. Data-free knowledge distillation for heterogeneous federated learning. In International Conference on Machine Learning, 2021.
[457]
Martin Zinkevich, John Langford, and Alex Smola. Slow learners are fast. Advances in Neural Infor- malion Processing Systems, 22, 2009.
[488] Martin Zinkevich, Markus Weimer, Lihong Li, and Alex Smola. Parallelized stochastic gradient descent. Advances in Neural Information Processing Systems, 23, 2010.
[459) Daniel Zügner, Oliver Borchert, Amir Akbarnejad, and Stephan Günnemann. Adversarial attacks on graph neural networks: Perturbations and their patterns. ACM Transactions on Knowledge Discovery from Data, 14(5):1-31, 2020.
93

---

Published in Transactions on Machine Learning Research (02/2026)
A
Survey map, Notations and Glossary
Organization of the solutions
2.3 Improving Leaming Effectiveness
3,4 Improving Inference and Learning Effectiveness
2.3.1 Data-based Techniques
3.4.1 Data-based Techniques
Local regularization
Data augmentation
Data augmentation
Local regularization
Client selection
Subsampling strategies
Open challenges of data-based techniques
2.3.2 Model-based Techniques Parameter personalivation Architecture personalization
3.4.2 Model-based Techniques
Parameter personalization Architecture personalization
2.3.3 Training and Inference under Vertical Partitioning Training with collaborative inference Training for isolated inference
Open challenges of model-bastee techniques
3.4.3 Training and Inference under Aligned Partitions
Aggregating node representations Transfer learning
2.4 Improving Learning Efficiency
2.4.1 Model A Aggregation
Aggregating graph representations
Centralized aggregation Decentralized aggregation
Open challenges
Hierarchical aggregation
3.5 Improving Inference and Learning Efficiency
3.5.1 Aggregation network
2.4.2 Asynchronous Aggregation
Embedding aggregation
Asynchronous centralized aggregation
Model aggregation
Asynchronous aggregation for decentralized learning
Open challenges
Communication Efficiency
3.5.2 Asynchronous Aggregation and Delay Control
Transmission efficiency
Model aggregation under asynchromicity Embedding aggregation under asynchronicity
Wireless communication for model aggregation
Client scheduling
Open challenges
2.5 Privacy-presse Privacy-preservvnng Collaborative Learning
3.5.3 Communication Efficiency
Blind computations
Transmission efficiency
Differential privacy (DP)
Wireless communication embedding aggregation
Differential privacy for collaborative learning
Client scheduling and topology manugement
Open challenges
3.6 Privacy Preservation Techniques for Graph Data
Anonymizing model parameters Anonymizing exchanged embeddings Anonymizing edge (or topology) information Open challenges
Figure 21: The (axonomy of the solutions surveyed for effective, efficient, and privacy-preserving collaborative learning for Euclidean and graph data.
Glossary
In this work, we use several terms whose definitions are either subjective inconsistently used in the literature; clarify them here for consistency and clarity. These definitions limited to the present work.
Agent An autonomous entity with local data, computational capabilities, and the ability to communicate with other agents.
Server A central coordinating unit that supervises group of agents, possessing computational capabilities and the ability to communicate with agents.
Distributed property of a system in which some or all components (e.g., data, computation, or control) are spread across multiple agents.
Data parallelism A property of distributed system where data collected centrally is distributed across multiple agents.
94

---

Published in Transactions on Machine Learning Research (02/2026)
Model parallelism A property of distributed system where parts of the model maintained centrally are dist.ributed across multiple agents.
Collaborative (system) distributed system of agents (either federated or decentralized) where agents cooperate to achieve shared objectives, though their willingness to collaborate may be limited by local constraints or incentives such as, but not limited to, local objective, computational and com- munication capabilities, data availability, privacy requirements.
Federated (system) A distributed system of agents that are federated under the supervision of central coordinates.
Decentralized (system) A distributed system of agents where all agents operate without a central server.
Local Describes property object that pertains to single agent (e.g., local data, local model).
Global Describes property or object shared across or computed from or many agents. It does not imply that each agent has a full view of this property (e.g., global graph).
Topology
The discrete structure describing the communication or computational relationships between agents, typically modeled as a graph (see 154 for a formal equivalence between topology and graph).
Sample A single data point drawn from dataset.
Feature An observed attribute or measurable property of a sample.
Label The target property or variable of a sample that we aim to infer from its features.
Training The process where models learn patterns from data by optimizing parameters to minimize pre- diction errors the training dataset.
Inference The process by which the model predicts the label of a sample based on its features using learned parameters.
Learning The comprehensive framework encompasses various methodologies, algorithms, and theoretical foundations for training models to perform inference tasks effectively.
Euclidean Pertaining to data where each sample is represented as feature vector in R equipped with the standard Euclidean inner product.
Non-Euclidean Pertaining to data with inherent relational structure that cannot be faithfully represented in R with the Euclidean inner product. Graphs are the most commonly used example of discrete non-Fuclidean structures.
95

---

Published in Transactions on Machine Learning Research (02/2026)
Notations
Agents and Data
De.
Local and global data distributions
Ie. Z,
Local and global index sets of samples, and sample index Set agents, number agents, agent index, active agent, source target agent Number of samples in local and global datasets Similarity matrix representing a latent graph between agents
K, a,,
Q
II,
State of Markov chain and its i-th element
Sc. 9
Local and global datasets
W E gR*K
Communication topology matrix between agents
Features, Labels, Representatious
da. dys dz
Dimensioos of feature vector, labe) space, and embedding space Feature sample and feature domain
X
Feature of the i-th sample, global and local feature a local domain X₃
Fa. Ale
Local fraction Label and label domain
g,
Label of the i-th sample and its inferred value Reprseenation sample (æ, y) representation domain Local fraction of global representation local domain Zμ
2,
2
Graphs and Topology
A
Adjacency matrix of g Loral and glohal sets of odges Local cut edges
8
Eout.
G -= (A.X)
Graph data sample in matrix form Local subgraph and 6-th local graph sample
Gz. Gj
G", G
Global and local L-hop veighborhoods of node
6v, 9
Local and global graph structures (bomogencous)
Ge Su
Local subgraph data and structure (from Ga and Ge), respectively, expanded to I-hop ncighhorhoods
Hlp
Homogeneous projection
heterogeneous graph
Ni. N
Number of vertices in local and global vertex sets
Oz.
Local and global sets of object types
Re, R
Local and global sets of relation types Local and global relation type mappings Computational
for target node
Local and global node Local and global sets of vertices (nodes), and vertex index Sets foreign nodes, private nodes, and
mapplngs
Vu, V.
Vy Vo.ki Veut.
Node-Level Quantities
( A.
Global and local embeddings of node at layer Feature matrix of the graph Global and local Global and local node repreentation node
x
Re: Po.k
of
So.k
Models and Learning
E.
Number of local iterations and local ieeration index Loral model and its function space
Fe: Fz
F.F
Global model and its function space Model F parametrised by and by (0)
Fo. Fo.s
Message, aggregation, and update functions layer
Predictor parametrized
H
Hypernetwork
Pairwise
L.
Number of layers and layer index Local and glohal population losses and global empirical Local and global encoders Loral and global encoders parametrized by 0 Node, edge, graph encoder functions Predictor parameters and parameter space Number global iterations and global iteration index Local and global model parameters parameter domains
Can
Lz, C.
$1,0: die
Gnode: Pedges Pgroph
0. &
T,
Oz.
Privacy and Differential Privacy
Failure probability differential privary Privacy budget in differential privacy Raudomized mechanism (uraining slgorithm) for differential privacy Set of possible outcomes mechsnism M
Table 3: Notations used in this paper, organized by comprehensive categories and alphabetical order.
96


Reviews:
[{"review_id": "KWdig8sh7t", "reviewer": "Reviewer_Zpd3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper surveys collaborative learning (CL), with a particular focus on extending federated & decentralized learning from Euclidean data to graph settings. The paper's core contribution is a unification of CL, graph representation learning, and diffusion, along with a taxonomy of graph distribution & partition scenarios. The survey also identifies some open challenges related to efficiency, privacy, and inference in collaborative graph learning.\n\nDespite weaknesses in organization and presentation, I believe the technical content is thorough, timely, and well researched.", "claims_and_evidence_yn": "No", "claims_explanation": "The authors present their claims in a thorough, detailed survey with over 400 references, and they are well-supported with theory and pseudocode.\n\nWhile the survey is technically sound, I strongly believe the clarity with which this evidence is synthesized and presented should be improved. In particular, organizational and presentation issues sometimes obscure otherwise well-supported arguments. I should emphasize that these concerns are largely editorial rather than indicative of missing or incorrect evidence. \n\nThe modifications which can help are outlined in the \"Requested Changes\" section below.", "audience_yn": "Yes", "audience_explanation": "The survey is comprehensive, and can be useful to researchers in multiple domains including federated learning, Graph ML, and even in privacy and security.", "requested_changes": "As mentioned earlier, the survey is well researched. Below are some changes to enhance the overall readability and completeness. \n\nContent:\n\n- [C1] Wouldn't there be memory bottleneck issues with GNNs, especially when storing multi-hop neighbor embeddings? It would help to see some discussion in this regard\n- [C2] Section 3.6 privacy is very useful, but as with Euclidean FL, in a collaborative graph learning setting there can be a Byzantine agent who can inject adversarial edges that corrupt the embeddings of honest agents multi-hops away during message passing phase. To ensure a complete survey, it might be helpful to call out some topological defense mechanisms, ie how can an agent or server verify the structural integrity of a neighbor's sub-graph without violating privacy?\n- [C3] While the choice of picking only supervised learning is fair, it would help to share any references for unsupervised or self-supervised learning\n- [C4] (good to have) I believe the survey would benefit from a brief discussion on how graph density affects the communication complexity of Algorithm 3. Are there graph topologies where a single agent can become a communication bottleneck?\n- [C5] (good to have) CL assumes the agents are willing to participate, and one agent's privacy can be compromised simply because their neighbor reveals a connection. There are also agents with critical/influential nodes (high centrality) while others can hold isolated nodes. This brings a challenge of \"data valuation\" in graphs (and with it the whole field of game theory, but that's definitely out of scope!). How can we reward agents who contribute critical topological bridges vs those who only provide redundant local clusters. This might help in linking \"Effectiveness\" with \"System heterogeneity\" of agent contributions (section 3.3.3).\n- [C6] (good to have) I would clarify some language around privacy guarantees from data locality. These can be tightened to clearly distinguish architectural choices from formal privacy guarantees. The author's call for tighter integration with defense mechanisms, while responsible and necessary, should be emphasized even more strongly in the conclusion.\n\nEditorial:\n\n- [E1] Optimize reference management\n    - [E1.1] With over 400 citations, more selective synthesis focusing on representative or contrasting works rather than exhaustive enumeration would significantly improve clarity and reduce cognitive load for readers. Table 1 is very useful in picking the top papers in a particlular field. However, I'd recommend using distinct markers to identify the pillar papers that define a field or area of study when cited in-line (like McMahan et al 2017, or Kipf & Welling 2017).  There are also many places where the references are just listed one after another for a given area (for example, in page 36 or 38), it would help to also highlight if these references also differ in some sense.\n    - [E1.2] (good to have) As the authors plan to release resources, I suggest organizing the bibliography into a searchable format split into categories & use cases. There are many ways to do this, one way I can think of is having a timeline, but am open to thoughts from the authors.\n\n- [E2] Presentation:\n    - [E2.1] Heavy reliance on long, nested bullet points for complex taxonomies can make it difficult to follow at times. This substitutes for narrative structure and obscures relationships between ideas. Bullets should be reserved for summaries or compact enumerations following a clearly established framework; core concepts would benefit from being presented in prose, supported by figures or tables. It also would help to add a brief \"Summary of insights\" or \"Key takeaways\" box at the end of major sections to distill which techniques are best suited for specific cases\n    - [E2.2] Given the length of the survey, while Figure 3 helps in giving an overview, a \"decision tree\" style branching with all the subsections listed can help readers navigate the paper better. I think this also ties back to the bullet lists, doing some reorg can help improve the overall readability. Even a detailed table of contents can help!\n    - [E2.3] Table 2 is very useful and anchors the survey well, but is again tough to follow since all the references are bunched into the space and a lot more space is unused. One thought is to split it into multiple tables (probably one for each section?) and use highlights to let the readers know which papers are seminal in the area. I'm also open to hearing thoughts from the authors.\n\n---\nWhile all the points above would strengthen the submission, I think addressing points C1-3, E1.1 and E2 would make the paper a stronger candidate for recommendation.", "broader_impact_concerns": "The main concern is potential over-interpretation of privacy benefits in collaborative learning settings. Clearer language around threat models and limitations would mitigate this risk. Additionally, since the paper discusses attacks in detail, it can be interpreted to inadvertently provide a detailed roadmap for adversarial behavior.\n\nBeyond this, the work primarily serves a clarifying and organizational role and does not raise significant broader-impact concerns.", "additional_comments": "The submission contains a large amount of high-quality & well-researched material, and reflects significant effort by the authors for which they should be commended. However, as outlined earlier, the organization & presentation limit the accessibility and impact of the survey. The current manuscript reads more like a set of comprehensive curated notes than a guided survey. Importantly, these are structural and editorial issues rather than conceptual: the underlying content is largely strong! The suggested modifications would likely resolve these issues. \n\nFor these reasons, I view the paper in its current form as lacking in clarity of presentation, but well-positioned to become a strong and valuable survey after targeted revisions.", "extra_scores": {}, "extra_text": {}}, {"review_id": "1gOI9o2a0C", "reviewer": "Reviewer_D1d5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper is a survey of Collaborative Learning on graph data. It presents Collaborative Learning (CL) as a Machine Learning problem in which independent agents want to use learned representations of other agents to improve theirs, without sharing data.\n\nAfter presenting the canonical CL algorithm, Federated Averaging, they introduce the main challenge of CL, that is, the different data distribution scenarios across the learning agents. The authors guide us through the literature by explaining how different methods try to improve the learning effectiveness, efficiency and privacy of CL in the different agent data distribution scenarios.\n\n**Strengths**  \n- Introducing CL using the FedAvg algorithm and then exposing its flaws is a very nice ease-in introduction.  \n- The categorization of the different data partitions allows to go beyond the classical Centralized, Federated Learning, Decentralized Learning categorization of Collaborative Learning.  \n- For each goal (effectiveness, efficiency and privacy), the authors suggest clear categorizations of the main directions in the literature.  \n- By presenting the euclidean CL literature with the same structure as the graph CL literature (data scenarios then classification of methods based on their goals), the authors highlight similarities between graph and euclidean CL.\n\n**Weaknesses**  \n- The main motivation of CL according to the authors, i.e. using other agents knowledge without sharing data, is under-studied in the survey. Agent’s privacy is presented as an enhancement target, not as a fundamental tradeoff of knowledge exchange between the agents. The authors do mention that collaborative learning is not private, but, to a CL newcomer, the survey overall does not make it clear that the absence of data transfer does not imply good privacy. This is exacerbated by the relatively small size of the subsections on privacy enhancements compared to the effectiveness and efficiency subsections.  \n- While the categorization of CL methods and their goals is clear, there is little discussion on the tradeoffs (or lack thereof) they imply.  \n- The separation of the survey into a section on euclidean data and graph data hurts the unification claim (see the claims section) and the presentation. Since section 2 acts as an introduction to the survey structure (data scenarios, then effectiveness/efficiency/privacy goals), section 3 appears as a litany of methods, stripped of motivations and comparisons (because they were already made in section 2), which makes the reading lengthy.", "claims_and_evidence_yn": "No", "claims_explanation": "The paper is presented as a survey and makes the following claims\n\n**Claim 1: As a survey, the paper is a pedagogical starting point to the field of collaborative learning on graphs.** Yes\n- See strengths section above\n\n\n**Claim 2: As a survey, it depicts a comprehensive picture of the field of collaborative learning on graphs.**  Yes\n- Each of the presented research direction points to several references to learn more about it  \n- The table 2 is very clear, useful to follow the graph CL part of the survey and a good overview of the graph CL survey.\n\n\n**Claim 3: The survey collaborative learning on euclidean data and graph data.** No\n- The claim is, in my opinion, not supported enough.  \n- The common structure between section 2 (euclidean CL) and section 3 (graph CL) highlights the similarities between the challenges of euclidean and graph CL (effectiveness, efficiency and privacy).  \n- Section 3.3 highlight the greater complexity of graph data scenarios compared to euclidean data (more ways to partition the data among agents).  \n- But, in sections 3.4, 3.5 and 3.6, there is almost no mention of similarity, differences or unification of graph and euclidean CL methods.  \n- While the introduction of CL using FedAvg and its shortcomings is very pedagogical, the rest of section 2 is not re-used to understand the differences between graph and euclidean CL in section 3.\n\n\n**Claim 4: Clearly presents the main open problems and promising directions in graph CL** No\n- The open challenges in 3.4.1 are not motivated enough. E.g. what makes graph-specific regularization different from euclidean data?  \n- The open challenges on 3.4.2 are: “we need to adapt what has been done in euclidean data (personalization, foundation models, …) to graph data”. But again does not say what is different/similar between graph and euclidean CL.  \n- The open challenges in 3.4.3 are “towards more advanced methods”,  there is no insights on why this is necessary nor how this would be different from euclidean data", "audience_yn": "Yes", "audience_explanation": "Yes, it is a thorough review of collaborative learning on graph data. The decomposition of the different data partitioning brings clarity in the challenges of collaborative learning.", "requested_changes": "Typos\n- “l(Fθk(t− 1 2 )(x), y”  p. 8 missing parenthesis\n- “horisontal” p. 24 typo\n- “decentrized”  p. 45 decentralized ?", "broader_impact_concerns": "NA", "additional_comments": "- Categorizing frozen layers (model decoupling item p.16) into the term architecture personalization (section 2.3.2) is strange, especially since the previous paragraph is exactly about parameter personalization.  \n-  “Cluster-based” section 2.3.2. It is unclear what is clustered. I guess this is parameter clustering, but then what is the difference with similarity-based aggregation ?\n- “in practice, agents might obtain a more effective model by collaborating under the vertical partitioning described in Section 2.2.1.”  p. 16. is the partitioning a choice of the agents ?\n- Definition of the local datasets, p18: Isn’t x_k^i in X_k instead of X?\n- The justification of training under vertical partition is strange: we do not want to share the data, have no label but still want to do something. Wouldn’t it be easier to sign a data sharing (w/ protection guarantees) agreement instead of implementing all of this? Asked differently, In what situation is data strictly forbidden to be shared but sharing features OK?\n    \n- Description of the two GNN paradigms p.28: the distinction between spectral and spacial filters is unclear as spectral and spacial filter as not defined.\n- Strange artifact on figure 11: half a node is visible on the right\n- A bit of history on the communities that worked on collaborative learning on graph would have help to better understand the context.", "extra_scores": {}, "extra_text": {}}, {"review_id": "GQppRE7wBo", "reviewer": "Reviewer_CjWc", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This survey paper tries to addres the growing need to move beyond centralized machine learning pipelines, which suffer from scalability and privacy limitations, in the context of graph representation learning. While collaborative learning paradigms such as federated and decentralized learning have gained traction, existing research largely focuses on Euclidean data types (e.g., text, images) and overlooks graph-structured data, which are prevalent in many real-world applications. The authors argue that message-passing mechanisms in graph learning naturally align with collaborative settings but remain significantly underexplored. However, they left the Spectral GNN literature unexplored.\n\nThe survey first synthesizes foundational principles of collaborative learning on Euclidean data, organizing prior work across three fundamental dimensions: learning effectiveness, computational efficiency, and privacy preservation. It then extends this lens to graph-structured data, proposing a taxonomy for distributed graph scenarios, analyzing the resulting statistical heterogeneities, and providing unified problem definitions and algorithmic frameworks. The paper concludes by outlining open challenges and future research opportunities.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The authors clearly justify the need for extending collaborative learning from Euclidean to graph-structured data by highlighting the limitations of centralized graph ML and the natural compatibility of GNN message passing with collaborative environments. The paper systematically maps foundational principles from Euclidean collaborative learning to the graph domain and provides a coherent taxonomy, problem formulations, and analyses that reinforce the central claims.\n\nMoreover, the survey presents a clear breakdown of design dimensions—effectiveness, efficiency, and privacy—and discusses how they manifest under both Euclidean and graph-structured settings. This structure supports the authors’ assertion that the field is emerging yet rich, and that graph-based collaborative learning introduces new constraints not present in traditional federated learning.", "audience_yn": "Yes", "audience_explanation": "Yes. The paper would be of clear interest to TMLR’s audience—especially researchers working on federated learning, decentralized optimization, graph machine learning, and distributed systems. However, the survey notably omits coverage of spectral GNN architectures, an active area with several recent and influential contributions, and this gap limits the completeness of its perspective.", "requested_changes": "Major Revision: The paper overlooks a substantial portion of the message-passing literature, particularly in the area of spectral GNN architectures. Although the authors cite one reference, this significantly understates the depth of existing work. Notable models such as APPNP, GPRGNN, BernNet, ChebNetII, JacobiNet, PolyGCL, ArnoldiGCL, LORA, and related studies—several of which also incorporate contrastive learning—are missing. A thorough review of these spectral approaches is essential for a complete and balanced survey.\n\nIn addition, the paper omits important work in Graph Federated Learning, including the ICLR 2025 paper “Subgraph Federated Learning for Local Generalization.” This line of research directly tackles critical challenges such as:\n\nUnseen Nodes – introducing new nodes with known classes that alter local graph structure,\n\nMissing Classes – nodes whose labels have not appeared previously in a client’s graph, and\n\nNew Clients – entirely new graphs with distinct structural and label distributions.\n\nThese challenges, along with the corresponding references, should be incorporated into the survey to strengthen its completeness and relevance.", "broader_impact_concerns": "No problem in terms of  the ethical implications of the work.", "additional_comments": "The paper is well-written however it is missing an important line of GNN research.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)