Incorporating New Knowledge into Federated Learning: Advances, Insights, and Future Directions
Authors: Lixu Wang
Sun Yinggang
Yang Zhao
Jiaqi Wu
Jiahua Dong
Ating Yin
Qinbin Li
Qingqing Ye
Dusit Niyato
Tianwei Zhang
Kwok-Yan Lam
Yu Haining
Haibo Hu
Wei Dong
Venue: Accepted by TMLR ()

Abstract: Federated Learning (FL) is a distributed learning approach that allows participants to collaboratively train machine learning models without sharing the raw data. It is rapidly developing in an era where privacy protection is increasingly valued. It is this rapid development trend, along with the continuous emergence of new demands for FL in the real world, that prompts us to focus on a very important problem: How to Incorporate New Knowledge into Federated Learning? The primary challenge here is to effectively and timely incorporate various new knowledge into existing FL systems and evolve these systems to reduce costs, upgrade functionalities, and facilitate sustainable development. In the meantime, established FL systems should preserve existing functionalities during the incorporation of new knowledge. In this paper, we systematically define the main sources of new knowledge in FL, including new features, tasks, models, and algorithms. For each source, we thoroughly analyze and discuss the technical approaches for incorporating new knowledge into existing FL systems and examine the impact of the form and timing of new knowledge arrival on the incorporation process. Unlike prior surveys that primarily catalogue FL techniques under a fixed system specification, we adopt a lifecycle evolution perspective and synthesize methods that enable time-varying integration of new features, tasks, models, and aggregation algorithms while preserving existing functionality. Furthermore, we comprehensively discuss the potential future directions for FL, incorporating new knowledge and considering a variety of factors, including scenario setups, security and privacy threats, and incentives.

Published in Transactions on Machine Learning Research (04/2026)
Incorporating New Knowledge into Federated Learning: Advances, Insights, and Future Directions
Lixu Wang Yinggang Sun?; Yang Zhao"; Jiaqi Wur, Jiahua Dong', Ating Yin Qinbin Li Qingqing Ye², Dusit Niyato", Tianwei Zhang', Kwok- Yan Lam', Haining Yu², Haibo Hu?, Wei Dong!!
kra. wanggnnten edu.sg, 23B903085 hww.edid.c... edu..cn en, zhao04660c.nu edu.sg, unrjaqiimail tsinghua eda. en, dongji- alua 1995 @gmaal..com, com, alingyýn Chnu. edu.en, ginbin@hust. edu. en, ypolyy..po....oommmmo edu.kk, dniyalo@ntu..c edu.s,, tian- zhinzhang Gratu. adu.sg, maookyon.lam@ntee edu.sg, yuhaining@hät edu.cn, haibo. www.hhuuoooouuuuccuuouue edu.kk, wei dong@ntu. edu.sg Nanyang Technological University, 2 Harbin Institute of Technology, TTsinghua University, &Mohamed bin Zayed University of Artificial Intelligence, $ Hunan University, "Huazhong University of Science and Technology, "The Hong Kong Polytechnic University
Reviewed on OpenReview: https: openreview. net/ formmo id= Bd-BWWBK3B37
Abstract
Federated Learning (FL) is distributed learning approach that allows participants to col- laboratively train machine learning models without sharing the raw data. It is rapidly developing in an era where privacy protection is incrcasingly valued. It is this rapid devel- opment trend, along with the continuous emergence of new demands for FL in the real world, that prompts us to focus on a very important problem: How to Incorporate New Knowledge into Federated Learning? The primary challenge here is effectively and timely incorporate various new knowledge into existing FL systems and evolve these systems to reduce costs, upgrade functionalities, and facilitate sustainable development. In the meantime, estab- lished FL systems should preserve existing functionalities during the incorporation of new knowledge. In this paper, we systematically define the main sources of new knowledge in FL, including new features, tasks, models, and algorithms. For each source, we thoroughly analyze and discuss the technical approaches for incorporating new knowledge into existing FL systems and examine the impact of the form and timing of new knowledge arrival on the incorporation process. Unlike prior surveys that primarily catalogue FL techniques under a fixed system specification, that enable time-vare varyivg integration of new features, tasks, models, and aggregation algo- rithms while preserving existing functionality. Furthermore, potential future directions for FL, incorporating new knowledge and considering a variety of factors, including scenario setups, security and privacy threats, and incentives.
adopt lifecycle evolution perspective and synthesize methods
comprehensively discuss the
1 Introduction
Federated Learning (FL), as an emerging paradigm in distributed learning, has gained significant traction and undergone substantial development over the past decade. This interest is primarily driven by its ability to enable multiple partics to collaboratively train machine learning (ML) models without sharing the raw data. Consequently, this characteristic has led to its adoption a growing number of critical domains, such as healthcare, transportation, finance, and e-commerce, resulting in numerous real-world applications (Yang
ct al. 2019 Kairouz et al. 2021 Wang al.] 2021).
These practical applications demonstrate significant demand for FL, driving its rapid development and evolution. In the field of ML, technological advancements often lead to the replacement of old models with
"Equal contributions. 1 CCorrsponning author.

---

Published in Transactions on Machine Learning Research (04/2026)
new ones. However, given the large number of participating clients, in terms of data volume, computation overhead, and communication bandwidth, training each FL. model incurs substantial costs (Kairouz et al. 2021). Furthermore, FL applications are often situated in rapidly changing environments, such as the emergence of new discases, antonomous vehicles encountering unfamiliar roads, or the appearance of new investment trends (Yoon el al. 2021 Dong el al., 2022 Peng el al. 2020: Li et al. 2025c Yu et al. 2023). In addition to these new concepts, there may be sensor degradation, hard ware damage, product evolution, and so on, in FL deployment scenarios, such as edge computing (Wu et al., 2025d). However, current FL systems typically assume fixed and predetermined distribution of data and tasks (Yang et al.] 2019 Kairouz et al,, 2021), making it challenging to deal with dynamic changes in data and tasks in real-world scenarios. These practical considerations reflect a dilermma in the future development of FL: on the one hand, there is a desire to establish new FL systems in response to emerging demands and technologies, but on the other hand, the significant investment in established FL systems makes it impractical to discard them readily. Therefore, how to achieve sustainable evolution FL bocomes a critical rescarch direction.
Table 1: A comparative analysis of survey studies on federated learning.
Year
Title
Difference
We viewed FDG as an important tool to enable easier incorporation of new features into FL within broader evolutionary framework.
Federated dumain generalization: A survey Li al. 2023c
Foundational models and federated learning: survey, taxonony.
We focused on FL evolution through new knowledge integration rather than focusing on foundational models integration with FL.
challenges and practical insights (Hatfaludi & Serban 2025
A Survey on Cluster-based Federated Learning E]- et al. 2025
We regarded personalized FL as an effective way to prepare task- generalizable basis for incorporating new tasks.
A survey on federated fine-tuning of large language models
(Wu]
We talked about incorporating LLMs to enhance established FL sys- tems rather than finetuning LLMs using FL.
20255
2025
Federated learning design and functional models: survey (Ay- We emphasize FL evolution via new knowledge integration mechanisms,
celyan et al. 2024
rather than conventional system design analyais.
Pederaled Contioull Learning: Contepla, Challengess, and Solu-
We reviewed si sludess with a much wider and more recent scupe and analyzed the correlations between FCL and other new knowledge.
tions (Hamed) 2025
Advauces robust lederated learning: survey with betero-
We locused on FL system evolution through dynamic new knowledge integration rather than addressing static heterogeneity challenges.
geneity considerations Chen et al. 2025a
Kuowledge distilaatonn federated learning: a compreheusive
We emphasize FL system evolution through dynamic new knowledge integration rather than surveying knowledge distillation in FL.
survey (Salman el al. 2025
Recent Advancements in Federated Learuing: State of the
We explored a more evolutionary and systematic research perspertive, and defined a dymamic framewurk for new knowledge integration.
Art. Pundamentals, Principles, IoT Applications and Future
Trends Paaadoopulos et al. 2024
2021
Federated learning: Overview, strategies, applications, tools and future directions (Yurdem et al 2024)
We explored FL dynamii evolution through new knowledge integration rather than static classification of existing strategies and applications.
2024
Recent advances federated learning: A systematic survey
Liu
We investigated FL system evolution through new knowledge integra- tion rather than static classification of existing FI. mots
ct al. 2024a]
Federated continual learning via knowledge fusion: A sur-
We reviewed studies with a much wider and more recent acope and analyzed the correlations between FCL and other new knowledge.
vey (Yang et al. 2024a)
A Compreeeensive Survey of Federated Transfer Learning: Chal..
We reviewed techniques of dynamic FL evolution through systematic new knowledge integration.
lenges, Methods and Applications [uuo et al. 2024b]
A systematic review of federated learning: Challenges, aggrega- 2023 tion methods, and development tools (Guendouzl et al. 2023
We emphasize FL system evolution through dynamic new knowledge integration rather than reviewing existing aggregation methods.
A survey on lederated learning: challenges and applica-
We discussed how to incorporate new demands and functionalities into the build FL systems rather than considering static FL scenarios.
tions (Wen et al 2023
Enabling the effective evolution of FL systems requires addressing several fundamental challenges. First, these systems may continually acquire new knowledge in diverse forms. For instance, a model may need to support an increasing number of novel lasks, or even for existing tasks to generalize across broader domains. Such newly encountered knowledge is typically non-identical and independently distributed (non-IID) across clients and may arrive at different time points. Specifically, in FI., each client optimizes the model using its own data. When client data are non-III, the updates computed on different devices are optimized for different local distributions, so they can point in inconsistent directions: an update that helps one client
2

---

Published in Transactions on Machine Learning Research (04/2026)
may hurt others. As result, simple averaging is no longer equivalent to taking a clean descent step single shared objective, which can slow convergence and reduce stability. In this setting, the FL objective at each timestep can be highly complex and potentially shift dramatically over time. Since this new knowledge often reflects transient user demands that appear sporadically, the system must be able to adapt rapidly. Second, learning new knowledge inevitably leads to the forgelting of previously acquired knowledge. This forgetting is primarily due to insufficient storage for the data of prior tasks. Furthermore, the diversity of new knowledge means that different types of old knowledge may be forgotten different rates, complicating mitigation efforts. Finally, the dynamic evolution of FL systems also presents additional challenges, including increased computational and storage overheads, the emergence of new vulnerabilities and attack surfaces, and the breakdown of existing incentive mechanisms.
Fortunately, an increasing mumber of studies are starting to explore relevant problems. For instance, [Sun et al. (023)) and Peng et al. (2020) achieved domain-genene -ennain-eneraalzablle learning to improve FL. models' generalization D ability of existing functionalitics, and Dong et al. (2022) and Qi et al. (2022) incorporated continual learning techniques to learn new tasks. While few existing surveys have attempled to review portions of the relevant literature, they are either outdated or lack comprehensive coverage. More critically, none of them examines these efforts through the lens of the continuous evolution of FL systems, which can inspire new thinking about FL's environmental sustainability and continual functionality iteration. A detailed comparison between these survey studies and ours is provided in Table 1 Prior surveys largely provide a snapshot taxonomy of FL methods under a fixed system specification, where the teaturettask space, model architecture, and aggregation protocol are treated as predetermined, and heterogeneity is addressed within this static pipeline. Consequently, they seldom discuss the system's 's lifecycle questions that arise in deployments, including when new knowledge should trigger upgrade, how the upgrade should be realized (c.g., detection, adaptation, migration), and how to bound regressions previously supported functionality. In this survey, we adopt a lifecycle evolution perspective and synthesize upgrade mechanisms along four evolving variables, including features, tasks, models, and aggregation algorithms, while highlighting how the arrival form and timing of new knowledge affect the incorporation process.
Contributions: In this paper, we present the first systematic and comprehensive survey with analysis and discussion on how FL can achieve continuous updates and development in the face of new knowledge. We define the FL system across four variables: features, tasks, models, and algorithms. Specifically, we review approachos for incorporating new features through an end-to-end workflow, including three key stages: (i) improving model generalization before new features emerge, (ii) enabling accurate detection when new features appear; (iii) facilitating model adaptation to the new features. For new tasks, we focus on enhancing the cross-task generalization ability of FL models. Given that new features and new tasks often co-occur in practice, we further explore how federated continual learning can support the continual integration of both, while maintaining strong performance on previously learned knowledge. Finally, we discuss the role of new models and algorithms as effective mechanisms for incorporating novel knowledge while preserving or even improving retention of prior capabilitics. In addition, we also discuss the potential new threats and vulnerabilities and the need of new incentive mechanisms when incorporating new knowledge into FL. In summary, our major contributions include:
We define a framework that categorizes new knowledge in FL into four types: features, tasks, models, and algorithms. For each category of new knowledge, we provide a detailed analysis and discussion on what it looks like and how incorporate it into the current FL system in a timely and efficient manner.
We analyze the impact of the arrival form and timing of the new knowledge on its incorporation into the FL system.
We comprehensively discuss future research priorities for the continuous development of FL, especially with the integration of new knowledge.
2 Preliminaries
Federated Learning (FL) YYang et al., 2019). as a distributed learning algorithm, enables multiple clients (Cx) k= to collaboratively train machine learning model m without sharing their training data Dj-
3

---

Published in Transactions on Machine Learning Research (04/2026)
( Vk,i) (Rg, 1.)) The model m: foow is usually composed of a feature extractor fo and task module Ow parameterized by 0 and w, respectively. In FI., clients possess their training data in a non-ITD manner, ie., the feature and task marginal distributions of different clients are not identical, A4 # Rw, Y4 * De where + K. In this work, we consider the horizontal FL setting, where each participant holds a distinct subset of samples with the same leature space. This differs from verlical FL,, where each participant holds subset of features for the same set of samples. A typical federated training round begins with central server S distributing the latest global model m to each client. Additionally, the server randomly selects subset of clients {C*) and requests them to conduct model training locally using their own data. Once local training is completed, these selected clients upload their locally trained models my to the server S. The server then aggregates these models {mi))=>s to update the global model, and the most popular aggregation algorithm is FedAvg (McMahan et al. 2017), i.e., m' = k my This process is repeated over multiple rounds until the global model converges and exhibits satisfactory performance for given task. From the description above, we can define different. FL systems F(X, M,A, using the variables of Features = = Am Y Tasks y = U Dk; Models M = fa. ... Iw So o (gw) -a.... :);;;; Aggregation
a algorithrs A= Avg,, FedProx (Li et al. 2020b), Moon eL al. 2021a), SecAgg (Bonawitz et al, 2017)
CaPC Choqueetee Choo et al. 2020
Clientaie
Central Server
Features:
Time
Classiiicatoon
Classification
Clessification
Transforees
New Features
New Tasks
Time
Clientaix
New Models
Features:
Tasks:
Classification
Classification
Object Detection
Time
Scaffold
CaPC
New Features
New Tasks
Clientow
Time
New Algorithms
Aggregate
Features:
Time
Tasks:
Classification
Anomaly Delection
Figure 1: Overview of an FL system with new knowledge from different sources. Different lypes of clients encounter new features and tasks over time, which reflect new demands for FL systems. From global perspective, new advanced models with better architecture (TTransfrrmess) and larger sizes (GPT-44 are also needed to incorporate for enhancing existing functionalities or incorporating new demands. Furthermore, new algorithms with improved performance (Scaffold) and security guarantees (CaPC) should be contimuously incorporated into established FL as well.
2.1 Federated Learning with New Knowledge
Starting from the moment of the first training round, we denote the existing period of an FL system F(X, M, until the present time as as a sequence of time stamps [1, .......,,......... ....,, then the new knowledge defined as a particular variable at a particular timestamp that is unseen previously. we take the feature variable as an example, we have 2 & xf To facilitate the coming of more new knowledge, we do not restrict clients' participation, which means clients are free join or leave the FL system any Kald.natd time. Under this framework, we categorize the FL clients into three groups: Cold (C+ k==koal corresponds Know, to clients that possess only old knowledge; Cnew (C* [[C*)==kiow corresponds to clients that only possess new knowledge; Cmix = {Cx)
kmix. kt k=nnii.11 corresponds to clients that possess both old and new knowledge. Note that the category belonging of a particular client C₁ changing over time; thus, these three client groups Cold. Cnew: and Cmix are also changing dynamically. Overall, regardless of which variable the new knowledge belongs to,
4

---

Published in Transactions on Machine Learning Research (04/2026)
when it arrives, who introduces it, need to incorporate the new knowledge promptly. Incorporating such new knowledge is crucial because the arrival of new knowledge essentially signifies clients' new demands for enhancing the performance and functionality of the current FL system. We visualize an FL system with various kinds of new knowledge in Figure Within such setups, we primarily focus on cross-device FL in the Tollowing sections, as it offers greater flexibility for client participation and withdrawal compared cross- silo FL. This sctting typically involves larger number of parties, components, and concepts, all of which contribute to higher system dynamics. In addition, cross-device FL systems are often resource-constrained. Overall, incorporating new knowledge into cross-device FL systems is more needed than cross-silo FL.
2.2 Security Setups of FL with New Knowledge
A trusted setting is assumed when incorporating new knowledge into an FL system, i.e., (i) all participants including both the central server S and clients {C*] are trusted not to eavesdrop private information launch active attacks at any time; (ii) all carriers of new knowledge including features X, tasks V, models M, and algorithms A do not intend to disclose private information or contain active allacks.
The reason why we consider such trusted settings that we aim to discuss the technical advances of FL with new knowledge in the main context, and we found that all cxisting related works assume such trustod scenarios. However, this does not mean that the security and safety research is unimportant; instead, we regard it highly important and discuss threats in both semi-honest and malicious settings as well the potential solutions in Section 8.2
3 Incorporate New Features
When FL models are deployed on new devices with unseen features, performance often degrades due to the discrepancy between learned features (source domain) and new features (larget domain) (Shenaj et al. 2023). Therefore, it is important to improve the generalization ability of FL systems to unseen new features. Specifically, we consider that Federated Domain Generalization (FDG) can equip the FL system with a certain degree of generalization ability on unseen features. Federated Out-of- Distrribtttiitii Detection (FODD) can detect the arrival of new features and then determine whether to incorporate them. After that, Federated Domain Adaptation (FDA) conducts adaptive learning unlabeled data with new features, achieving the incorporalion of new features.
3.1 Federated Domain Generalization
FDG aims to train a dommain-eeneralizabee global model that can generalize well across different distribu- tions. To provide a structured overview of the technical landscape, we summarize and compare representative FDG approaches in Table in terms of their timing support, overhead, and utility. Such domain samain-gennealizablle learning is usually achieved by extracting domain-shared semantics across local clients with old knowledge Cold, Cmix (source domaīns). In this way, model m can extract similar semantics from the unseen new features - Xr in terms of the same task y.
Xnew. Here, we assume that Xnew is relevant but distinct from UK
3.1.1 Methods
Distribution Alignment. Several FDG methods (Li et al. 2025ch employ feature aligument techniques to minimize domain discrepancy. Generally, they achieve this by optimizing with distribution alignment regularizers (uu al. 2021), such as contrastive loss, adversarial loss, and maximum mean discrepancy (MMD). For example, Xu et al. (023) proposed negative-free contrastive loss at the logit level to mini- mize the distribution gap between the original sample and its hallucinated counterparts. Building this work, Gupta et al. (2025) onhanced model generalization through domain invariant feature perturbation alignment. Regarding the adversarial loss, Wang el al.) (2022ch used adversarial learning for multi-client feature alignment. Similarly, Zhang et al. (2023) introduced another generalization adjustment model via dynamically calibrating aggregation weights with an adversarial objective. As for the MMD aspoct, [Tian] et al. (2023) used MMD for gradient alignment, encouraging aggregated gradients unify information from multiple domains. Additional gradient-based distributional alignment approaches have emerged. Wei & Han
5

---

Published in Transactions on Machine Learning Research (04/2026)
Table 2: Comparison of approaches in Federated Domain Generalization. While distribution alignment strategies typically incur higher costs and possess limited scenario support, data augmentation and op- timization enhancement approaches effectively address these limitations by offering improved flexibility and superior efficiency, respectively.
Timing Support
Period Support
Synchronisu
Overbaedd
Utility
Strategy
Methods
Abrupt Graduate Short
Long
Sync
Computation
Communicaation
Storage
Performance
FADH Xu
2023
Medium
Low
Low
Near SOTA
FedAligondd
Gupla
20225
Medium
Low
Low
SOTA
FADGN
Wang
21220
High
Medium
Low
Behind SOTA
Distribution
FedADG
Ziang
2123
x
V
High
Mediuum
Low
Near SOTA
Aligament
PCDG
Jan
2025
V
Mediuum
High
Low
Near SOTA
FedDM
Sun of ml.
A123
V
High
High
Medium
Near SOTT
MCGDM
Wei &r Han 21/24
x
V
High
High
Low
Near SOTA
FedU
Pourpanab ct
20725
V
Medium
Low
Low
SOTA
FedGM Nguyen et ml 21225
x
Mediuum
High
Low
SOTA
FedDX
Lia et al.
2021
V
<
Low
Low
Low
Behind SOTA
FedCST
Chen ct m..
2123b
Medium
Medium
Medium
Near SOTA
Data
FISC Ng guyen et sl
202
4
Mediuum
Medium
Medium
SOTA
Augmentation
FdGGC
Liu ot al.
2024h
High
Low
Tow
Near SOTA
CAL-2SA A
Yong et ml. 2023e
/
Low
Low
Low
Near SOTA
FedCCRL,
Wang ot m. 2024c
/
Medium
Medium
Low
SOTA
Frdl.
Soltany et al
21724
1
Low
Low
Low
Near SOTA
Optimisation
Eabancement
BiPPdl
Lai 06 al
2024
1
Low
Low
Low
Near SOTA
FedSemi DC
Deng et
2025
Low
Low
Low
SOTA
2024) proposed reducing distribution differences through intra-domain and inter-domain gradient match- ing. Nguyen et al. (2025) regarded local gradients as the representation of specific domains and maximized the inner product gradients find invariant gradient directions across all domains. Similarly, Ye el al.] (2025) proposed the GAFedDG framework, which combines intra-domain gradient alignment between raw and augmented signals with inter-domain gradient alignment across domain classifiers to achieve effective distribution alignment.
Data Augmentation. To improve the model's generalization to unseen target domains, some FDG meth- ods use data augmentation techniques to diversify training data distributions (Liu et al. 2023). Common augmentation techniques include domain randomization, domain generation, and domain mixup. Liu et al. (2021) used domain randomization techniques like random rotation, scaling, and flipping to address the FDG problem. However, as research progressed, the limitations of simple domain randomization in tack- ling complex domain shift issues became clear. Chen al. (2023b) introduced cross-client style transfer method, where local clients define their styles and share them with central server. Building this con- cept, Nguyen et al. (2024a) proposed the FISC method, which refines cross-domain learning by extracting an interpolative style from local client styles and using contrastive learning. Simultaneously, efforts were made to enhance data diversity through domain generation. Liu et al. (2024b) increased data diversity by gencrating samples with diverse domain styles using style-complement module. For domain mixup, YYagg et al. (202cc) proposed mixing local and global feature statistics by randomly interpolating instance and global statistics. Wang al. (2024c) followed this by boosting local domain diversity through cross-client domain transfer and Nomain-invariant feature perturbation.
Optimization Enhancement. Many FDG methods introduce regularization terms to prevent the model from overfitting domain-specific features. For instance, (ootany et al. (2024) proposed client-side label smoothing to reduce the model's reliance on domain-specific features, enhancing generalization across diverse domains. To further ensure stable and robust global model updates against data heterogeneity, [Lai el al.) (2024) proposed cistance-basee weighted aggregation combined with parameter moving averaging. Building these advancements, Deng al. (2025) developed gencralization-awar aggregation method, dynamically adjusting aggregation weights based each client model's cross-domain performance.
Wee add the subscript "DG" " to FedAlign (Cupta al. 2025 to differentiate from the studics Ravi & Shomorony 2024
Ma al. (2024b)) with the name. Ravi Shomorony (2024
introduced Fed select and unify non-priority chents align with the priority clients. Ma 2024b] proposed Fed FedAlign achieve alignment different. modalities in FL.

---

Published in Transactions on Machine Learning Research (04/2026)
Table 3: Commonly used experimental datasets in incorporating new knowledge into FL systems.
Dateset
Sample
Task Description
PACS
[Li el al. 2017
9,991
Iunges of daily objects with artistic styke, covering classes.
DomainiNet Peng
2019
600,000
Images of daily objrcts collerted from 6 different domains with 345 classes.
Digits-DG (Zhon et al.
2020
2,400
Images of handwritten digits with 4 domains, covering 40 classes.
VLCS (Shalitatell
10,729
Images of common objects from domains, covering classes.
Terralaoogita [Beery el Bl. 2018
24,000
Wildlife camueaa images from 20 locatious, covering 10 classes.
Camelyon17
[Ba]
410,359
Lymph node slice image data, covering domains and classes,
FEMNIST aldlas et a.. 2018
800,000
Iuages of handwritten digits and characters acruss 3 domains and 62 classes.
CIFAR,10 Krizhevsky al.
60,000
popular small scale natural image classfication dataset with classes. A dataset similar to CIFAR-10 but with 100 classes.
CIFAR-100
Krizzeevsy et
2009
60,000
CIFAR-10-C
Heudd vCks Dietterch
2019
10,000
CIFAR-10 with damage processing from domains, covering classes.
VisDA2017
[Peng al. 20017
280,000
Images from syuthetic rendering real shooting, covering 12 classes.
Imag:CLFF
IZhank cl al.
2019
1,800
Images from 3 collection scenarios, cuvering 12 classes.
Offine-31 Saenno
2010
4,110
Office supply images from 3 photography scenarios, covering classes.
Office-Homee (ennattewarre et al
2017
15,588
Office supply images from 4 domains, covering 65 classes.
Offie-Caltechi1 Zinani Davisol
2020
2,500
Office supply images from 4 domains, covering classes.
Digil-Five (Saito et al. 2015
700,000
Handwritten digit images from 5 domains, covering 10 classess
Tiny ImageNel, TT Yang 2015)
100,000
Images daily objects and subeet of ImageNet, covering classes.
Omniglot (Song ot al 2024
1,623
Images of handwritten characters from 50 different alphabets, with 50 classes.
Shakespeare (Caiuss ot 81.7
2018
4,226,158
Text data
works by Shakespeare.
CelebA [Liu et al (2113
202.599
Face images with detailed attribute aunotations.
S4nt Cairtas ef al
2018
1,600,000
Texts collected from Twitter sentiment analysis, with 2 classes.
SVIIN [Notact
600,000
Images of real-world house number digits, with 10 classes.
COVID.TL Yan et Bl.
2022
20,000
Medical images for COVID-19 FL research, rovering 4 classes.
Kinetics 400
Rvay
306,245
Videos of real-world human actions, covering 400 classes.
UCF101 (Soomro et al.
2012
13,320
Video clips of human actions in different scenarios, covering classes.
New Features
PACS
DomainNet Digits-DG VLCS
Terralncognita Camelyon17 CIFAR-IO-C VisDA2017
ImageCLEF-DA Office-31 Office-Home Office-Caltech Digit-Five
FEMNIST CIFAR-10 CIFAR-100
Shakespeare Sent140 SVHN COVID-FL COVID-FL Kinetics 400 UCF101
Tiny ImageNet Omniglot CelebA
New Tasks with New Features
New Tasks
Figure 2: The mapping relationship between the commonly used datasets and different new knowledge types.
3.1.2 Evaluation
The commonly used experimental datascts for FDG, as shown in Table 3 include PACS, Do- mainNet, Digits-DG, VLCS, Ierrann,, Camelyon17, Office-31, OlfieeHHome, and Ollice-Cateec10..
Datascts.
setup, where each client owns data from one source domain. To better mimic real-world scenarios where client may have data from multiple sources, some studies explore intra-client domain heterogeneily selups. For example, Stable- FedADG (Zhao et al. 2025a) allowed clients to hold unbalanced data mixtures from multiple source domains or include dominant and secondary domains in a "mixed' sctup. Additionally, this sctup (Bai et al.) 2023) introduced a parameter to control inter-client domain heterogeneity, highlighting the impact of different data distributions and heterogeneity levels.
Setup. The most common FDG configuration is the single-client-singl--doe

---

Published in Transactions on Machine Learning Research (04/2026)
Table 4: Comparison of approaches in Federated Out-of- of--isitribbtion Detection. Notably, FedDD distin- guishes itself with minimal overhead across all dimensions, while FedOE achieves state-of-tho-art perfor- mance with moderate resource consumption.
Timing Support
Period Support
Syuchronissm
Overbead
Utility
Methods
Abrupt Graduate Short
Long
Syne
Asyue
Computationn
Commumication
Storage
Performance
FeAnD
Nardi et al.
2022
Mediom
Medium
Medium
Behind SOTA
ME
(Molammeddi
2023
Medium
Medium
Medium
Near SOTA
FxtOoD
Yu al.
2023
High
Medium
Tow
Near SOTA
OT-FODD
He
20256
High
Medium
Low
SOTA
Jeong Clasi
2025
4
Medium
Meditmm
SOTA
FedDD
Rahimli et
20724
/
Low
Low
Near SOTA
FOOGD
Lino ut al.
2124
X
Medium
Medium
Low
SOTA
FrIFISS
Dong ct al.
2023
/
Law
Modium
Low
Near SOTA
FedCNCL
Wang et al
21125a
V
X
High
Medium
Medium
Near SOTA
Metric. Four metrics measure how well an FL model generalizes unseen target domains: accuracy on unseen larget domains, average accuracy across all domains, worst domain accuracy, and accuracy difference
across domains.
3.1.3 Real-world Applications
FedBM (Zhu al. 2025b used FDG principles to address medical data heterogeneity, achieving accurate and robust medical image analysis and diagnosis by eliminating local learning biases. DGSST, (Liu et al. 2025a) proposed semi-supervised domain generalization learning, combining autoregressive discriminators and reconstruction tasks to achieve robust and accurate recognition by addressing cross-individua. domain shifts in human activity recognition. DG-FaultDiag (Zhao el al. 2024a) addressed multi-source data privacy and distribution differences in industrial scenarios by integrating domain generalization techniques with FL, enabling robust fault diagnosis for different devices and conditions.
3.2 Federated Out-of- Distribution Detection
FODD was designed to detect new features across local clients (typically Cuix and Cnew) using a binary classifier 9b placed after the global feature extractor fo- To provide a structured overview, we summarize and compare representative FODD approaches in Table Tubblerrege regarding their detection strategies, overhead, and utility. Once new feature data is detected, it should be incorporated into the current FL system. Therefore, Pb should be trained lo determine if sample comes from a previously unseen feature distribution æ U The following section reviews relevant studies.
3.2.1 Methods
Several approaches have been used to achieve FODD. Reconstruction prediction is approach. Nardi al. (2022) proposed a decentralized anomaly detection framework. Clients are clustered into communities based on the similarity of their representations, followed by collaborative training of a federated autoencoder. Simi- larly, Mohammadi et al. 2023) used a long short-term memory mechanism within a modified autoencoder to detect outliers in FL systems. Another effective approach is synthesizing potential out-of of--if-iiibtttionnnnn (OOD)
data and differentiating from in-distribution (ID) data. For instance, Yu et al. (2023) proposed training capegory-specifie generators to create OOD samples for detector training, He et al. (2025a) proposed generat- ing psendo-OOD samples via. the Wasserstein distance, creating virtual outlier samples from known samples to address the scarcity of OOD data in federated environments. Instead of generating OOD data, Jeong & Choi (2025) assumedd the server stores some outlier samples whilc clients keep minimal normal data. This approach ensures efficient outlier detection through internal feature separation and background collabora- tion. Feature-based detection represents a third approach, which relies on feature extractors to characterize ID data and employs weigh detection mechanisms. Rahimli al. (2024) used Fisher's Exact Test identify concept and data drift. Liao et al. (2024) developed the FOOGD framework, modeling client data distributions with OOD scoring mechanism and achieving end-to-end FODD by integrating semantic
8

---

Published in Transactions on Machine Learning Research (04/2026)
shift detection with covariate shift generalization module. Dong et al. (023) introduced task transition monitor using classification entropy of client data representations to identify OOD lasks, enabling real-time
adaptation to evolving data patterns. Wang et al. (2025a) developed hierarchical clustering mechanism on data reprosentations to discover and learn previously unobserved features.
3.2.2 Evaluation
Datasets. The commonly used datasets for FODD, as shown in Table 3 include FEMNIST, CIFAR- 10/CIFAR-100, CIFAR-1O-C, PACS, Digit-Five, and Office-Home.
Setup. The evaluation setup of FODD primarily focuses on two scenarios: unseen class detection and feature shift detection. First, unlike conventional label shift, which only causes an imbalance in the number of categories, FODD simulates scenarios where clients detect previously unseen categories. This is an extreme form of label heterogeneity (Liao et al. 2024 typically constructed using a Dirichlet distribution with a small paramcter a. It roquires the model to determine whether an unknown sample comes from known calegories of other clients or is entirely new for all clients. Second, FODD is evaluated in scenarios with feature shifts, where clients share labels but the data features come from different distributions. This is demonstrated in various benchmark tests, such as the domain migration experiments of FOSTER (Yu et al. 2023 on DomainNet and the HDR-FL benchmark used by FedPD (Yang et al.) 2023a
Metric. Five metrics are used to evaluate FODD performance: the area under the receiver operating characteristic curve, the area under the precision-recall curve, the false positive rate at 95% true positive rate, the true negative rate 95% true positive rate, and the F1 Score.
3.2.3 Real-world Applications
Li et al. (2025a) proposed secure and backpropagation-free FODD approach, useful for local OOD identi- fication and data quality control in FL, as well as detecting OOD threats in video analysis. Zamzmi et al. (2024) suggested using statistical process control to monitor changes in the statistical characteristics of med- ical image data streams, enabling early identification of potential data drift and enhancing FODD response capabilities radiology. ZZhan el al. (2024f) introduced a deep ensemble method guided by maximum consistency and minimum similarity, significantly improving the model's uncertainty estimation and OOD sample discrimination.
3.3 Federated Domain Adaptation
FDA aims to collaboratively adapt the global model to data with new features detected by FODD. We assume there is a significant domain discrepancy between the old and new features, d(Xnew, Xx) >> max d(K,, where dl-) denotes a function measuring distribution discrepancy. As most data with new features appears without labels, FDA should address the domain discrepancy between the labeled source data with old features and the unlabeled target data with new features. To facilitate clear comparison, we summarize representative FDA approaches in Table 5 evaluating their support for different learning scenarios (timing, periodicity, and synchronism), associated overheads, and overall utility.
3.3.1 Methods
Adversarial Training. The difference in feature distributions between domains major obstacle effective knowledge transfer. Adversarial training aligns inter-domain features by training discriminators and feature extractors adversarially, [Peng al. (2020) were the first extend adversarial adaptive training to FL, aiming to align learning representations of different nodes with the target node's data distribution. They enhaneed knowledge transfer by combining dynamic attention mechanisms and feature disentanglement. More studies have enhanced this adversarial framework in various ways. First, Wu et al. (2024b) proposed decoupling features using mutual information theory before adversarial training. This approach ensures that minimizing distribution differences is not affected by domain-specific features. Additionally, Chi et al.) (2024) designed multiple tightly coupled joint classifiers for adversarial training to promote alignment of the joint distribution of features and classes. Zhang & Li 2022 combined federated deep adversarial networks

---

Published in Transactions on Machine Learning Research (04/2026)
Table 5: Comparison of approaches in Federated Domain Adaptation. While adversarial training strate- gies achieve statc-of-the-art performance, they generally suffer from high computational costs and limited scenario support. In contrast, pseudo-label learning and distribution alignment approaches offer su- perior flexibility in timing and synchronism with significantly reduced overheads.
Timing Support
Petidd Support
Synchrouism
Overlead
Utility
Strategy
Methods
Abrupt Graduate
Short
Long
Syne Asyne
Compatation Communication
Storage
Performance
FADA (Peng
2020
x
High
Mediam
Belind SOTA
FedFDDA
Wu et sl. 202
x
x
High
Medium
Low
SOTA
Adversarial
FMDADA
Chi et al.
2024
x
v
x
High
Medium
Low
SOTA
Training
FTLAN
Zhang Li 2022
X
V
x
High
Medium
Low
Near SOTA
FedDASK
Gong et m. 2024a
x
V
x
High
Medium
Low
SOTA
FMTDA
Yao et
2022
X
x
High
Medium
Low
Noar SOTA
CM-FDA
Zong c$ ml 20/2
x
/
High
Medium
Low
Noar SOTA
FedUDA
Zhuang al
2022
4
V
V
Medium
Medium
Low
Near SOTA
FMSDAA
Zhao ot ml. 2024
4
V
4
Medium
Medium
Low
Near SOTA
Pacado-label
COPA
Wh Ar Gong 2021
4
Medium
Medium
Low
Near SOTA
Leariing
RrIFTI.
al
2024h
Medium
Medium
Low
SOTA
FoIWCA
Mori
2014
Medium
Medium
Low
SOTA
BFTL
Wang al.
2024b
/
Medium
Medium
Low
SOTA
Distribatton
HPSDA
Rizzoli
2024
Medium
Medium
Low
SOTA
Alignment
FrIRP
2024
Medium
Low
Low
SOTA
at both the image and pixel levels to comprehensively reduce the feature gap between source and target domains. Gong et al. (2024a) introduced cross-domain semantic similarity measurements to conventional domain-level adversarial Training, accelerating convergence and improving the quality of aligned features. Morcover, to address the single-source multi-target scenario, Yao et al. (2022) proposed a federated dual adaptation strategy, achieving cross-domain feature alignment through local classifier adaptation on the client side and Gaussian mixture model weighted mixup on the server side. Similarly, Zeng al. (2022) reduced domain shift in multi-site brain imaging data through adversarial domain adaptive pre-training.
Pseudo-label Learning. The lack of annotations for target domain data limits model adaptation, natural approach is to generate leeudo-labels to include them in the learning process. [Zhuang et al. (2022) proposed the FedFR method, introducing distance constraints into the clustering algorithm to improve pscudo-label accuracy. In multi-source domain adaptation sconarios, Zhao et al. (2023) enhanced pscudo- label accuracy through a joint voting scheme. Similarly, Wu & Gong (2021) used a prediction consistency mechanism to generate more accurate pseudo-labels and compute aggregation weights, facilitating effective multi-source domain adaptation. Considering not only domain shift but also the more complex issue of class shift, Li et al. (2024d) proposed generating pseudo-labels through global clustering and local semantic consistency clustering. For the challenging federated source-free domain adaptation task, where source data is inaccessible, Mori et al. (2024) proposed a weighted clustering aggregation method. This method first clusters clients to obtain domain-specific global models and then performs local adaptive fine-tuning on clients using the generated pseudo-labels.
Distribution Alignment. Reducing the distribution difference between the source and target domains is straightforward approach to domain adaptation. A common method is to use MMD to measure and minimize the domain gap. Wang et al. (2024d) proposed aligning these domains by minimizing the MMD distance using random features and employing low-rank approximation techniques to compress exchanged information, enhancing communication efficiency. Unlike aligning on the client side locally, [Wang el al. (024b) used MMD as a dynamic weight to balance the contributions of different local models. Another major category of methods aligns feature distributions through prototype learning. For instance, Rizzoli et al. (2024) proposed expand the geometric space of prototype learning from the traditional Euclidean space to the hyperbolic space, which is more suitable for hierarchical data.
10

---

Published in Transactions on Machine Learning Research (04/2026)
3.3.2 Evaluation
Datasets. The commonly used experimental datasets for FDA, as shown in Table 3 include VisDA2017, ImageCLEFF-A, Office-31, Office-Home, Office-Caltech10, Digit-Five, PACS, DomainNet, and VT.CS.
Setup. Most FDA studies employ a setup with multiple source-domain clients and a single target-domain client. Source-domain clients have labeled data from one or more source domains, while the target-domain client has data from a different target domain, which may be minimally labeled completely unlabeled.
Metric. Two metrics are commonly used: average accuracy and F1 Score on the target domain test data.
3.3.3 Real-world Applications
In the medical field, WeiWtight (Zhu et al. 2025ap applied distribution quantization method based on density estimation to accurately capture patient data characteristics at each medical institution, effectively addressing covariate shift in electronic health record processing. In autonomous driving, FFreeDA [Mori et al. 2025 improved semantic segmentation performance under adverse weather conditions. In edge com- puting, Li et al. 2024b) adapted WiFi-based human activity recognition models to new users or environments by aggregating diverse hypotheses from different domains, significantly reducing reliance on labeled target domain data. Wu et al. (2025c) developed an efficient anomaly object detection tool that can generalize different scenarios.
3.4 Future Work
We cnvision that future FL will evolve into a highly integrated and automated end-to-end framework more efficiently incorporate new features. Ideally, when an FL system encounters features outside its training distribution, it should first exhibit robust OOD generalization. Following this, the framework must accurately detect the arrival of these new features. Once detected, the system should automatically trigger data collection process. When the new data reaches a sufficient scale or meets predefined criteria, the system will seamlessly initiate an adaptation process to integrate the new knowledge into the existing model. Within this overarching vision, future work on each component can focus on the following aspects:
FDG. Future research could pursue more fundamental theoretical and paradigm innovations based on ex- isting work. Theoreticallly, while deriving stricter generalization bounds remains important (Wang et al. 2025e3, greater challenge is to establish framework that unifics the tradc-offs among generalization, pri- vacy, and elficiency, guiding algorithm design under extreme heterogeneity. In terms of model paradigms, although studies have begun adapting large models to federated scenarios using techniques like prompt learn- ing (Gong ct al. 2024b), future rescarch could explore constructing a shared adapter library with diverse domain knowledge. Clients can flexibly use these modules based on dynamic needs at minimal cost. Addi- tionally, regarding learning paradigms, beyond exploring unsupervised FDG reduce reliance on labels (Yan) & Guo 2024h, designing proxy tasks that require models distinguish data distributions across clients may enable learning of essential Nomain-invarian features.
FODD. Current methods mainly focus on enhancing OOD generalization through model intervention or knowledge distillation (Qi et al.) 2025b), which is essentially passive. Future research should shift to proac- tive and predictable OOD response mechanisms. By exploring new perspectives like data generation and feature space reconstruction, we can systematically enhance the OOD generalization ability of FI. systems when facing data distribution shifts. New FL architectures can enable autonomous learning and dynamic adjustments for OOD problems, e.g,, some (virtual) clients can be dedicated to monitoring and detecting new data distributions by comparing aggregated global models over time.
FDA. Current research is largely empirically driven and lacks rigorous theoretical guidance. While some work provides preliminary theoretical analysis (Feng et al. 2024b), future breakthroughs lie in understanding the correlation between the generalization of adapted models and heterogeneity in FL. In terms efficiency, although onc-shot methods reduce communication overhead (Abedi et al.) 2025 Zhang al.] 2024c), they often lack flexibility. Future work could explore novel mechanisms minimize the extra communication overhead incurred when transmitting domain-specific information. Additionally, customizing recent. test-
11

---

Published in Transactions on Machine Learning Research (04/2026)
time adaptation approaches to FL setups is worth studying. Such customization should consider adjusting test-time adaptation according to data heterogeneity while preserving client privacy.
4 Incorporate New Tasks
In the dynamic field of FL, traditional systems face significant challenges when addressing the diverse and continuously emerging tasks from clients. The introduction of new functionalities, similar to tasks with comparable features, is common. To effectively respond to these evolving requirements, it is crucial to enhance the cross-task generalization capabilities of FL models. This need directs our focus towards advanced methodologies in FL, specifically task-personalized FL and self-supervised FL.
4.1 Task-Personalizer Federated Learning
Task-Personalie FL (TPFL) is essential to satisfy the diverse and specific requirements of clients within FL systems. This approach deals with the inherent diversity in feature and task distributions among different clients. Each client C2 adapts the global model m: fo Iw to better align with its local data characteristies and specific needs. Initially, the central server S distributes the global model to each client. The clients then use their unique datasets Dx to locally train models m optimizing a personalized objective function Cx (X+)..))) These models are aggregated by the server using FL algorithms from set A, resulting in an updated global model m' with better cross- generalization ability due to this multi-task training process. When new tasks emerge in the future, the adaptation be easily achieved by little transfer learning from the latest global model. To facilitate comprehensive understanding, summarize representative TPFL approaches in Table 0 comparing their support for learning seenarios, computational and communication overheads, and overall utility.
Table 6: Comparison of approaches in Task- Pesss-eessnnlieed Federated Learning. While client grouping methods typically lack support for asynchronous settings and impose strict scenario constraints, regularized optimization strategics demonstrate superior flexibility, offering broader applicability across diverse timing and synchronization requirements.
Timing Support
Period Support Syachronism
Overhoad
Utility
Strategy
Methods
Abrupt Graduate Short
Long
Syne
Asyne
Computation
Comounication
Storage
Performance
PPedMe
Dinh et
2020
v
v
High
Medium
Low
Behiid SOTA
Regularbedd
FedProto
Tau
3022
r
x
Low
Low
Low
Behind SOTA
FI-ERS
et al
M1244
r
v
Low
Low
Low
SOTA
Optimization
FelAS
Yang
A724
v
x
High
Medium
Low
SOTA
FedRB
Yau
2726
v
x
Low
Low
Low
SOTA
FrdCS
Han al.
2023
Medium
Mediom
Low
Near SOTA
Client
MinPPL
Pan
2124
/
Medium
Mediom
Low
SOTA
Grouping
CA-PFL
2024
High
Medium
Low
SOTA
APFL-OS
Low
Low
Low
SOTA
4.1.1 Methods
Regularized Optimization. Regularized optimization in TPFL effectively addresses data and model het- erogeneity by customizing task-specific modules, while a shared global feature extractor facilitates knowledge aggregation across tasks. A common approach involves decoupling updates of global and local models by introducing regularization terms into local optimization objectives. PFedMe T Dinh et al.) 2020) used the Moreau envelope [Moreau 1965) client-side regularization term, separating the optimization of personal- ized models from global model Iearning. thcoretically achieving faster convergence rates. Morcover, to tackle parameler inconsistency in personalized FL, some studies adopt a consistency aligument optimization strat- egy. Li et al. (2024c) incorporated local batch normalization layers into client models to adapt to local data distributions without altering the global model structure, thus enhancing model personalization performance. The FedAS (Yang et al., (0224b) framework used the trace of the Fisher information matrix to measure client training progress and performs weighted aggregation accordingly, effectively reducing the negative impact
12

---

Published in Transactions on Machine Learning Research (04/2026)
of clients with slower training progress on the global model. More lightweight methods (Tan al. 2022 Yao et al. 2026) rely on communicating global representation prototypes across clients to aggregate shared knowledge rather than using feature extractors.
Client Grouping. Client grouping is an effective strategy for addressing client heterogencity in TPFL. Farly studies, such as FedCS (Han et al. 2023), introduced a confidence-based, similarit--aware strategy that enables flexible client grouping and similarity weight adjustment. Subsequent research has explored more advanced clustering techniques. MiniPFL (Fan et al., 2024) used the BIRCH clustering algorithm to effectively distinguish similar clients and conduct collaborative training within divided "mini federations", enhancing the efficiency of server-side federated updates. Other studies leverage endogenous structures within systems or networks for explicit clustering. CA-PFL (Zhao et al. 2024b) constructed a graph-structured federated social network and employs community detection to group clients. This ensures that clients within the same community have similar label representations, allowing for the aggregation of shared layers within each community while retaining personalized layers for cach client to address concept drift. Morcover, client screening can be seen as special binary clustering method, aiming to create a homogeneous group of clients. Ge et al. (2024) proposed one-shot screening scheme that converts local losses of clients on pre-trained model into p-values and uses adaptive thresholds to filter out client groups with homogencous data distributions to the task initiator, thereby tailoring FL for specific tasks.
4.1.2 Evaluation
Datasets.
The commonly used experimental datasets for TPFL, as shown in Table include Tiny ImageNet, Omniglot, Shakespeare, CelebA, and Sent140.
Setup. Under heterogeneous task scenarios, clients are assumed to divided in different proportions to perform various types of tasks. Most existing studies tested five types of learning tasks: object detection, semantic segmentation, image classification, object tracking, and natural language processing related tasks. Clients are randomly assigned to these tasks according to preset proportions. Meanwhile, to simulate the real- world data heterogeneity, even clients performing the same type of task can be assigned different datasets. In addition, each dataset is distributed to clients in a non-lID manner: for label types, skewed sampling is performed using the Dirichlet distribution; regarding data volume, there is a significant imbalance among clients, and datasets of different sizes are split and allocated lo each client. Each client may adopt a distinct model architecture tailored to its task type and dataset characteristics, while the aggregability among models is ensured through unified feature representation layer.
Metric. Two metrics are usually used: average accuracy and average lask loss of all clients" local test datasets. Besides, specific metrics are used for specific tasks.
4.1.3 Real-world Applications
Several representative applications highlight the practical value of TPFL across various domains. In health- care, ProtoHAR (Cheng et al.) 2023) developed a prototype-guided mechanism for privacy-preseringg and individualized human activity recognition in wearable healthcare systems, while PPFL (Kim et al., 2024 proposed personalized progressive FI. approach to leverage institution-speecfic features for collaborative medical diagnosis across different healthcare institutions. An adaptive FL approach to break data silos for Parkinson's disease diagnosis via privacy-preserving facial expression analysis proposed in (Pang
et al. 2025b). A tri-factor adaptive FL framework for Parkinson's disease diagnosis via multi-source facial expression analysis was introduced in (Pang et al.) 2025a). In edge computing scenarios, Ge et al. (2024) proposed one-shot screening mechanism to identify dala-homogeneous edge users for personalized FL in edge intelligent networks. In industrial applications, FedScrap (Zhang et al., 2024eb adopted a layer-wise personalization strategy for customized model training in industrial waste detection scenarios.
4.2 Supe-Super Federated Learning
Surervieev FL (SSFL) stands out as another crucial approach in improving the cross-task generaliza- tion ability of FL models, especially in environments where labeled data is scarce unavailable. SSFL is
13

---

Published in Transactions on Machine Learning Research (04/2026)
not dedicated to certain explicit tasks; instead, it aims to learn general global model that extract task-generalizable representations from certain input features. Specifically, each client Cs conducts the same self-supervised learning (SSL), such as contrastive learning and rotation prediction, on its local data Dj to optimize the local model m4 with the loss Sett (X,)) Overall, SSFL enables the extraction and inte- gration of diverse, self-generated features and knowledge across clients, which is beneficial for incorporating new tasks into FL systems. To facilitate a clear comparison, we summarize representative SSFL, approaches in Table evaluating their support for different learning scenarios (timing, periodicity, and synchronism), associated overheads, and overall utility.
Table 7: Comparison of approaches in Sclf-Supervieed Federated Learning. While data reconstruction methods typically incur high computational and communication costs, contrastive learning approaches effectively address these resource constraints by offering superior system efficiency and reduced overheads.
Timing Support Period Support Synchronism
Overhead
Utllity
Category
Methods
Abrupt Graduace Short
Long
Syne Async Computasion
Communication
Storage
Performance
MoooSFI
[6 al
2022b
d
Modium
High
Figh
Behind SOTA
Contrastive
FedSC
Jing ml.
2024
x
Modium
Medium
Low
Near SOTA
Acroltee
Xin es
2024a
Low
Low
Low
SOTA
Learning
FedCAD
Kong ct
2172
x
6
ligh
Medium
Low
SOTA
FLSimnCo
Gu et al
2023
v
<
x
Low
Medium
Low
SOTA
PedIT Wi e5 all
20222
x
High
High
Medium
Behind SOTA
FedMAT Yang ct al 2023b
/
x
/
ligh
llgh
Medium
Near SOTA
Data
FodNI
Prng ct al. 20223
x
ligh
Medium
Medium
Near SOTA
Reconsttruction
Graph1 L
Wang et al. 2022a
v
X
V
V
x
Mediuan
Medium
Low
Behind SOTA
MS-DINO
Park et al
2014
V
High
High
High
SOTA
FedReeuu
Liu el s).
2025b
x
x
High
High
High
SOTA
4.2.1 Methods
Contrastive Learning. Contrastive learning aims to learn discriminative feature representations by max- imizing the consistency between similar samples while minimizing the similarity among dissimilar ones. Early studies primarily focused on effective sample comparisons. MocoSFL (Li al., 2022b), based on the momentum contrast mechanism, maintains a shared feature memory and employs frequent synchronization strategies, laying the groundwork for contrastive methods in SSFL. In parallel, by sharing correlation matri- ces of data representations across clients, FedSC (Jing et al. 2024) enabled cross-client sample comparisons. To further improve efficiency, AeroRec (Xia et al. 2024a) combined contrastive learning with knowledge dis- tillation techniques, proposing a contrastive distillation approach for efficient knowledge transfer. Moreover, significant progress has also been made in contrastive lcarning tailored to specific data types. FedCAD (Kngg et al. 2024) extended contrastive SST to graph data by aggregating embeddings of neighboring nodes around anomalous nodes, enhancing the discriminability between positive and negative sample pairs, and providing a novel solution for federated graph learning. FLSimCo (Gu et al. 2024) considered the impact of data quality contrastive learning, proposing dynamic aggregation strategy based on image blur levels, demonstrating the adaptability of contrastive learning with low-quality data.
Data Reconstruction. Apart from contrastive learning, other approaches typically use data reconstruction for SSL. Wu et al. (2022) introduced the masked autoencoder reconstruction task into FL, achieving effective feature learning through the auxiliary task of reconstructing masked image regions. (aag al. (023bb proposed more general FedMAE framework, extending this pixel reconstruction strategy from specific medical scenarios to general visual tasks. However, pixel reconstruction mainly focuses on low-level texture information and has limited ability to model high-level relationships in structured data. FedN1 (Peng et al. 2 extended the idca of reconstruction to graph data, capturing more complex topological structural relationships by predicting missing node and edge information through network repair tasks. GraphFL (Wang et al. 2022a) provided an FL framework for semi-supervised node classification on graph-structured data. With the widespread application of transformer architecture, Park et al. (2024) proposed MS-DINO that performs distributed SSL by predicting masked image patches. In multi-modal scenarios, Liu al. (022b) proposed the RedRecon framework, which performs SSL by reconstructing missing modal information.
14

---

Published in Transactions on Machine Learning Research (04/2026)
4.2.2 Evaluation
Datasets. The commonly used datasets, as shown in Table 3 include Tiny ImageNet, Shakespeare, CelebA, 14, SVHN, COVID-FT, Kinetics 400, and UCF101.
Setup. First, a federated environment is simulated by dividing the dataset among a group of clients. This partitioning typically uses the Dirichlet distribution to manage data heterogeneity among clients. The experiment is divided into two stages: the first is the collaborative sels-supervised pre-training stage. During this stage, all clients use only their local unlabeled data to jointly optimize a shared feature extractor. In each communication round, clients perform SSL tasks locally, then send the updated model paramcters to the server for aggregation. After pre-training, the downstream task evaluation stage begins. At this point, the shared feature extractor frozen, and the quality of the features is evaluated using a reserved, small-scale labeled dataset (e.g., 1% or 10% of the total training data).
Metric. Two metries are used to evaluate the performance of SSFL: linear probe accuracy is the accuracy of using a small number of labeled data to fine-tune linear classifier based on the SSI-pretrained feature extractor; K-nearest neighbor classification accuracy is the accuracy of assuming the classifier is a KNN.
4.2.3 weal-world Applications
In transportation scenarios, Dai et al. 2024) proposed using prototype clustering contrastive learning for traffic scene recognition and vehicle behavior analysis, while while[Soares Soares et al. (0225) proposed an SSFL approach with Bayesian optimization for monocular depth estimation in autonomous vehicle applications. In medical diagnosis, Sun et al. (2025) proposed a hierarchical remi-supervised FL method for dermatosis diagnosis. In industrial monitoring, Youn et al. (2025) proposed a self-supervised asynchronous FL approach for diagnosing partial discharge in gas-insulated swilchgear for power system moniloring.
4.3 Future Work
Future research on integrating new lasks into FL can focus on constructing a more intelligent, efficient, and robust learning system. An ideal FL framework should possess the full-chain autonomous adjustment capa- bility from task characteristic perception, cross-tast knowledge transfer to model adaptive optimization when facing dynamically emerging new tasks. It should not only maintain generalization performance in scenarios where heterogeneous data and complex tasks are interwoven, but also adapt to the resource constraints of edge devices through lightweight design, while taking into account privacy security and system scalability under multi-task collaboration. The specific discussions are carried out from the two major directions of TPFL and SSFL as follows:
TPFL. Future TPFL research should aim to overcome current limitations in handling dual heterogeneity of lasks and data among clients, and develop new theoretical models and algorithm frameworks. First, within knowledge sharing and personalization paradigms, although existing works balance sharing and personaliza- tion using adaptive masks (Lv et al., 2024h or hypernetworks (Scott et al.] 2024), a revolutionary direction is to develop a federated multi-task learning mechanism based on causal inference. This achieve pre- cise knowledge transfer by identifying causal relationships between tasks, surpassing traditional correlation analysis. Second, regarding system architecture and collaboration mechanisms, decentralization is crucial for enhancing scalability (Feng et al.) 2025a). By evolving TPFL system architecture, self-organizing and self-healing lederated network topologies can be designed, allowing clients to form optimal collaboration patterns via smart contracts without central coordination. Finally, resolve inter-task interference, current methods focus on suppressing interference through aggregation strategies (Wei et al., 2025). A viable solu- tion could be constructing model representations by encoding knowledge of different tasks into orthogonal neural subspaces, ensuring minimal impact on old tasks and enabling forgetfulness-free lifelong task learning.
One potential future direction is to develop a multimodal SSFL framework to address differences in modal combinations among clients. Initial progress includes CroSSL (Deldari et al. 2024), which introduced cross-modal SSL via latent masking for time series, handling motion sensors and biological signals without negative pair sampling. Zhang al. (2024g) used complementary data and pseudo-labeling algorithms improve cross-modal learning. Secondly, the high computational and communication costs incurred by
SSFL.
15

---

Published in Transactions on Machine Learning Research (04/2026)
SSFL algorithms issues that also need to be addressed. For instance, LW-FedSSL [Tu et al. 2024 adopted a layer-wise incremental training strategy, allowing edge devices to train one model layer at a time. Combined with server-side calibration and representation alignment, it significantly reduces SSFL algorithms' computational and communication overhead. Additionally, the problem of catastrophic forgetting is more complex in the SSFL, environment; so future systems need strong continuous learning capabilities.
5 Incorporate New Tasks with New Features
In practical applications of FL, the arrival of new tasks is often accompanied by new features. Typically, learning new lasks based on the models that have already undergone some training is referred to as continual learning (CL). As a result, it is necessary to extend CL to the FL context, achieving Federated Continual Learning (FCL) (Yang et al. 2023d). While regular CL is not limited to classification, existing research in FCL nearly all assumes that new tasks are presented as new data classes. Therefore, we use classification as a representative example elucidate FCT. here. We make a systematic review of current technical approaches in FCL and compare them the supported arrival timing and existing period of new knowledge, overhead, and performance, shown in Table
Table 8: Comparison of approaches in Federated Continual Learning. All these methods cannot support the effective learning when the new knowledge data exists for short period. Considering the overhead and performance, Architecture Decomposition presents promising solution.
Tining Support
Period Support
Synchtoniem
Overhewl
Utiliyy
Category
Methods
Abrupt Graduate
Short
Long
Asyne
Computation
Communication
Storage
Performance
FedCurv
(lobaa
High
Iligh
CPPFL
Huxig
Medium
Medium
Low
Belind SOTA
PsICI.
Sun
2020
High
High
High
New SOTA
FsdSyR
M72e
V
High
Medium
Medium
Near SOTA
Optimization
Regularisstion
FLwF
kinanoow el
2021
V
High
High
High
Bebind SOTA
GLFC
Dong ct. al.
A122
High
High
Hiigh
Near SOTA
FOCL
Hunnng et sl 21722
x
High
High
Medium
Near SOTA
RwIC3D8-51
P00.S//Pongggg Pong
2025
x
High
High
Medium
SOTA
CGoFed
Peng et
2026
x
High
High
Low
SOTA
Re-Fot
2025h
High
High
High
Behind SOTA
DP.PCL. Zizzo os 2022
High
High
Medium
Behind SOTA
FCIL
Sun x1 2X24
High
High
Low
Neac SOTA
FeGGR Q es al 20122
High
High
Medium
Near SOTA
Po1MAECL
Wang
High
High
Medium
FeDDDR
Liong
High
High
Medium
SOTA
Koswledge
FeIDGR
20721
High
Mediom
SOTA
Replay
FaCTTG
20245
High
High
Low
SOTA
AF-HFCL Woerkaixd
2115
High
High
Low
SOTA
NGFL Li dl
20222a
High
High
Medium
Bebind SOTA
PF-PCIL
Yoo 32 Paric
2024
High
High
Medium
Near SOTA
PoTMR
Wi nt ml.
A022h
High
High
Hlgh
SOTA
AFL-
Zhang et al. 2025a
High
High
High
SOTA
FedSeIT
Chaudlarry et al
2022
x
High
Mediam
High
Bebind SOTA
Loci
Luopan et [ 2125
High
Mediom
Medium
SOTA
Architecture
Rwl1A
Va ot al 2124
High
Mediam
Medium
New SOTA
Decomposition
POT
Dakman et nl.
2024
High
Low
Medium
Near SOTA
FedPDT
[Pixo es
2124
High
Mediam
Medium
SOTA
5.1
Definition of Federated Continual Learning
5.1.1
Synchronous FCL
Similar to standard CL, Synchronous FCL assumes that there are a scries of datascts {D')T with N data samples D = {(a). 3 continuously arriving in the FL, systems globally. The majority of each dataset is assumed to belong previously unseen data classes, i.c. y" nu 6. Morcover, actually, the dataset at each time & is also non-IID owned by part of clients at that time, i.e., each client C.
16

---

Published in Transactions on Machine Learning Research (04/2026)
possesses Dj = ((o 11 and the corresponding marginal distributions are distinct among clients, X o x D + Dhr (where k + W, but y S )*). Consistent with existing Synchronous FCL studies, we also consider practical constraints from limited storage space and privacy regulations (Wang et al.) 2022b; Guo et al. 2023 and assume that old class data is unavailable or can be only accessed partially by the clients that own new class data (Cuew and Cmix). Then the objective to minimize the classification errors for new classes while preserving the good performance of learned classes
min E Bawu ae [llaw (for Iw (fo- (x)))]² + ((a.,)) [C(g v)].
(1)
6' p
5.1.2 Asynchronous FCL
Different from Synchronous FCL, Asynchronous FCL assumes that FL clients learn a series of classes
their own distinct orders. Specifically, suppose there is a set of class spaces {>*) with the order O I,....,.... T] from the global perspective, and these class spaces are disjoint from each other ) n n 0 (where t # t¹). However, from the local perspective, each client Cg continually receives the dataset la.,.. Ta.- The orders of clients are distinct from cach other
corresponding to a particular order O₂ = as well as from the global one, i.e., On + Ow, On + O (where k + k"). Asynchronous FCL also conforms to the constraints of limited storage space and privacy regulations. As for the difference between different client categories, we assume that Cota's 's data is fixed until the category changes into others, while the data of both Cuew and Cmix is changing over time as long as they do not change into Cold. Then, the objective of Asynchronous FCL is to minimize the classification errors for new classes without or with partial access to previously learned classes on the local side, which can be viewed as local personalization of Eq. (
5.2 Methods
Optimization Regularization. During CL, optimizing model parameters for new tasks can lead to for- getting previously learned tasks. Adding regularization to limit large parameter changes helps alleviate this forgetting. Fisher information and synaptic intelligence are commonly used to assess the importance of model parameters. FedCury (Shoham et al. 2019b) and CPPFI. (Huang et al. 2022b) were pioneers in applying these methods to local FL training. However, they do not consider the impact of data heterogeneity in FL, which is the main focus of Yao & Sun (2020). Li al. (2024c) also addressed this issuc by modifying synaptic intelligence to reflect model parameter importance based on local datasets and their correlation to the global data distribution. Model parameter change is also reflected as intermediate layer output change, thus, another forgetting alleviation is to conduct knowledge distillation for preserving learned task knowl- edge. FLwF (Usmanova et al.) 2021) first built teacher-student framework among a stored old task model and the current model, and regularizes similar logits over the same input, which corresponds to the first term of Eq. ( GLFC (Dong et al. 2022) designed a weighting mechanism for logit regularization to alle- viate the impact of data heterogeneity. In contrast, FCCL (Huang et al. 2022a)) assumed a public dataset could be used to distill knowledge from global old models, a method also adopted by [Png et al. (2025) Model parameter updates align with model gradients during training. Thus, CGoFed (Feng et al., 2025b) proposed constraining the gradient update direction to a space that minimizes interference with historical tasks, helping reduce forgetting of old data and speeding up adaptation to new data.
Knowledge Replay. The forgetting of learned tasks occurs because the model is optimized only for new tasks. This raises the question: 'Can we incorporate objectives for learned tasks into the optimization process?' Accordingly, replay-basod solutions are proposed, which replay old tasks during FCL. Intuitively, random storing can be inefficient; thus, GLFC (Dong et al. 2022) tried to store the samples that are closest to each class center. However, Re-Fed+ (Li et al. 2025b) argued that data heterogeneity should be considered when selecting the data to store, and proposed to use a personalized informative model to ensure the selected data contributes to both local and global knowledge. In addition to such local storage of old class data, Zizzo et al. (2022) built a global memory bank hosted on the central server to better deal with the global forget ting caused by data heterogeneity. However, managing this global memory requires clients to upload their private data, which violates the privacy preservation of FL., Zizzo et al. (2022) introduced Laplace noise to protect uploaded data. Instead of uploading the raw data, Sunn et al. (2024) leveraged dataset distillation to condense the local old task data by ensuring gradient consistency between the original and condensed data. Other
17

---

Published in Transactions on Machine Learning Research (04/2026)
2022 masked autoencoders (He & Wang, 2024), and diffusion models (Liang et al.; 2024 Mei et al. 2024) locally to generate old class data for replay. However, communicating these locally trained generative models also results in privacy leakage; thus, Nguyen ct al. 2024b) used data reconstruction attacks for good to train a global old data generator on the server side at the end of each lask training, which generates data to be replayed in future task training. Besides, different from data sample replay, Wuerkaixi et al. (2025) used a normalization flow model to generate old task features for replay while learning new tasks. Similar ideas can also be found in Li et al. (2022a); Yoo & Park (2024), which replayed old class prototypes across clients and the server. Besides, not only can the data be replayed, Wu et al. (2025b) proposed replaying historical client models and developed a weighted distillation mechanism to alleviate forgetting of old tasks. This model-based replay strategy was also adopted by Zhang et al. 2025a))
works enabled training generative models like GAN (Qi et al.
Architecture Decomposition. To preserve the old task knowledge, one intuitive approach is to separate different tasks and assign them to distinct model parameters. FodSeIT (Chaudhary et al.) 2022), a natural language processing study, divided the model parameters into separate parts. Except. for similar task-general and specific parts, there is another part to measure the correlation between tasks in FedSelT, thus it can spare the parameter bank. Similar idcas are also adopted by Lu et al. (2024) and Wu et al. (2025a). In fact, the model decomposition introduces additional communication cost. To reduce the cost, Loci (Luopan et al. 2025) proposed to compress the task-shared models into compact versions while leaving task-specific models untransmitted. From the perspective of feature sensitivity, FedTA (Yu et al. 2024) was built on the observation that the last few model layers are associated with task variations, and achieved FCL by adjusting the locations of task-shared and task-specific representations in the latent space. FOT (Bakman et al., 2024) modified the new task training to make its model updates orthogonal to the previous task activation princi- pal subspace, which can prevent interference between tasks. Although this strict model separation prevents interference, it also discards the inter-task performance enbancements due to their positive knowledge cor- relations. Piao et al. (2024) proposed to separate the positive and negative knowledge correlations between tasks in model parameters and then amplify the positive transfer while mitigating the negative transfer.
5.3 Evaluation
Datasets.
The commonly used experimental datasets for FCL, as shown in Table 3 include FEMNIST, CIEAR-10/CIFAR-100 Tiny ImageNet, Omniglot, and CelebA.
Setup. In a typical FCL, setup, a dataset is first partitioned into a predefined number of sequential tasks based on data categories. A common strategy is uniform sequential allocation, where each task is assigned an equal number of non-overlapping classes, following the dataset's original class order. More flexible selups relax these constraints, for instance, by randomly determining the number of classes per task and which specific classes are assigned. For any given task, the data is then distributed across clients in a non-IID manner. This is often achieved by inducing label skew, where each client only observes data for a subset of the task's classes. A widely adopted method to model this is to use a Dirichlet distribution to allocate class proportions to each client. The experimental setup also defines the task arrival sequence and duration. In synchronous FCL, all clients encounter the same sequence of tasks. In contrast, asynchronous FCL allows each client to learn from lasks in different, often randomized, order. As for task duration, most. studies assume fixed period for each task (e.g., set number of epochs), thus obviating the need for detection. However, a growing body of work is focused on designing sensitive, real-time detectors to enable FCL to operate under more realistic, arbitrary task boundaries.
Metrics. Two metrics are used to measure the performance of FCL: average accuracy (ACC) and forgetting (FGT) (Bakman et al. 2024),
[-1 FGT 1- a
ACC 3 a
at
(2)
where alt is the model accuracy of task Yi right after learning task yr, As for Asynchronous FCL, ACC, and FGT of local clients are also used to measure the performance.
18

---

Published in Transactions on Machine Learning Research (04/2026)
5.4 Real-world Applications
Some preliminary studies may provide valuable insights. FCLLM-DT (Xia et al, 2024b) built digital twin physical model to synthesize virtual anomaly data, which is integrated with FCL to realize spatial and temporal anomaly detection in the industrial Internet of Things. Similar tries can be seen in the intrusion detection (Zhang el al. 2024h; Quyen et al. 2024 Mao el. al. 2024) for industrial manufacturing systems.
In multi-satellite networks, USNL-MSSN [Niu et al. 2024 achieved FCL in satellite scene recognition.
5.5 Future Works
First of all, potential future studies involve extending FCL to multi-label and multi- aulti-grained classification. Besides, some preliminary works have explored achieving federated incremental segmentation on medical data (Peng et al. 2025 Zhang et al.] 2024a Dong et al. 2026). Therefore, extension to more complicated tasks like object tracking and visual question answering is waiting to be explored. Additionally, considering FCL in practical scenarios is underexplored, such as learning new tasks with insufficien, weakly labeled, or unlabeled data. Although there are a few studies related to few-shot FCT. (Liang et al. 2025al and federated novel class discovery, their effectiveness usually relics on unrealistic assumptions. In addition, doveloping FCL techniques for more data modalities is needed. Fortunately, some examples in graph classification (Miao el al., 2025 Zhu al. (024)) and text entity recognition (Zhang et al. 2025c) have been witnessed. Employing FCL to tackle some fundamental challenges of FL is another potential direction. Wang al. (2024a) constructed digital twin model for each FL client to generate data continuously, addressing non-IID issues in FL. Future research could explore leveraging FCT, to reduce computation costs and accelerate model convergence. Besides, more applications of FCL in healthcare, finance, and even scientific discovery are worthy of research.
6 Incorporate New Models
Integrating a broader range advanced models, including Foundation Models (FMs) and Large Language Models (LLMs), into existing FL systems significantly enhances traditional FL frameworks (Zhuang et al., 2023). Traditional FL systems often struggle with complex data distributions and the demands of sophis- ticated tasks. Incorporating models with superior architecture and larger scales is crucial. This approach not only addresses the limitations of older FL systems but also promotes evolution by increasing data avail- ability, boosting collaborative development, and enhancing both the utility and privacy of FL models in various applications. In the following context, we consider two cases for incorporating new models Mnew into FL: (i) From New to Old: use new models fa (e.g., FMs and LLMs) to enhance existing models fo: (ii) From Old to New: transfer architecture and knowledge (e.g., from convolution neural networks to transformers) from the existing models fo to new models fa. both cases, the new model fa is assumed be pre-trained certain datasets outside the target FI. systems. In the first scenario, clients can use their local data to distill knowledge from fa. which is typically hosted the central server S (as fa is usually quite large). In the second scenario, the focus is on efficiently transferring old knowledge from So to provide a better starting point for optimizing fa- Beyond these two transfer directions, new model architectures can also act as a flexible interface to integrate continuously arriving knowledge (c.g., new features or new tasks) into the evolving FL system. Accordingly, we organize the technical approaches in this section into two complementary objectives: (i) enhancing current functionality via model transfer (Section (6.1.1); ; and (ii) facilitating new knowledge integration via architectural modularity and adaptation mechanisms (Section The potential technical approaches are reviewed below, shown in Figure 3
6.1.2.
6.1
Methods
6.1.1
For Enhancing Current Functionality
From New to Old. When transferring models from new to old, two major issues need consideration: 1) local client data alone insufficient to fully utilize the knowledge new models; 2) new models are typically too large to be loaded on clients or communicated efficiently between clients and the server. For the first issue, CROSSLM (Deng al. 2023 demonstrated using small language models to refine LLMS
19

---

Published in Transactions on Machine Learning Research (04/2026)
CROSSLM Deng et
2023
RodPL.
Wang et
2023
RedSKS Shan
2024
From New to Old
FedAT (Dara al
2024
FSSC
Yan al. 2014
Lau
Enhmacing Functionaltty
Fedbiot [Wi ot al.
20246
Photon Sani ot Al. 20144
Openfor
Yc al
2024
From Old New
FedRolex Alam et al.
2022
HFL-SRC
Yo ot al.
2023
FCCI. Tuang et al. 2022
New Models
FedlProto-FT (Zhang ei al. 202
FedDAG Che el al 2025
EPPL
(He el al. 2021
Facilitating
FedFSCILL sum 2024
2025
HePCo (Halbe al
2022
CAP (Linng 12125
Integration
CF-Merge (Salami ct nl.
FedWolT (Yonn ot m..
2021
MDR Zhang ct al.
27246
SacFT. (Zhoing at m. A720
DAH-PCL
Q 24 23200)
Figure 3: A taxonomy of technical approaches for incorporating new models into FI. can be organized into two broad categories: (i) methods that enhance existing functionalities, and (ii) methods that facilitate the integration of additional new knowledge. For the former, we consider architectural transfer cither from the new model to the old model from the old model the new model.
and generate synthetic data, enabling data-free knowledge transfer. Additionally, approaches such as using public data and LLMs to enhance federated distillation by accurately selecting knowledge from local and ensemble predictions, as investigated by [Wang et. al.] (2023) and Shao et al. (2024), show the potential for knowledge transfer from new models to established FL models. Beyond distilling knowledge from the model's output layer, another approach focuses on aligning intermediate feature representatious. [Dazzi et al. (2024) introduced a feature alignment loss ensure that the intermediate representations of local models align with global vision transformer (VIT), addressing unfair enhancements for clients with limited data. For the second issuc, one solution is adopting more communication-eficel
model architectures. For instance, Yan el al. 2024) integrated small-scale Swin Transformers into FL instead of regular VIT. A more common approach involves parameter-efficient fine-tuning techniques, which avoid updating and communicating the entire model. Liu et al.) (202cc) introduced low-rank adaptation into FL, allowing clients to learn and upload only a small adaplation matrix, significantly reducing communication and computation overhead. Similarly, Wu et al. (2024a) designed a mechanism for on-demand parameter transmission, where only necessary updates are exchanged, achieving performance close to centralized fine-tuning with minimal communication cost. In addition to selective parameter updates, other works have integrated dedicated optimization techniques to address communication bottlenecks. For example, the Photon framework (Sani et al. 2024) combined gradient quantization, dynamic communication frequency adjustment, and efficient ring- base parameter synchronization to reduce bandwidth needs siguificantly. Similarly, Ye et al. (2024) developed an open-source FL framework for LLMs that optimizes communication compression and parallel scheduling, further addressing the challenges of training large models over low-bandwidth networks.
From Old to New. When transferring from old models new ones, existing FL studies on heterogeneous model architectures can be beneficial. For example, FedRolex (Alam et al. 2022) implemented a rolling window mechanism that allows clients to selectively extract sub-models from the global model on the contral server, tailoring them to train different. local models. Similar concepts were explored in the work of Ye et al. (2023). Another approach (Huang et al. 2022a) used a public dataset and SSL to distill knowledge from old FL models to new ones with heterogeneous architectures. However, this requires the public dataset to closely match the learned data distribution, which is often impractical. In future rescarch, data availability will remain significant challenge. Not all learned data can be stored for transferring to new architectures, necessitating selective data storage. Additionally, if the new architectures are large, specific designs for split learning tuning are needed, such as using adapters for different attention layers when training transformers.
6.1.2 For Facilitating New Knowledge Integration
Beyond enhancing existing capabilities, new model architectures increasingly serve flexible interface to facilitate the intogration of other types of new knowledge, effectively transforming the global model into a universal foundation. To facilitate the integration of new features, particularly unseen distributions, architectures are evolving to include auxiliary alignment components. For instance, Zhang et al. (2024c) employed global adapters trained on multi-domain prototypes to structurally bridge heterogencous feature spaces, while Che et al. (2025) incorporated somainosimulation modules to actively synthesize features for alignment. Conversely, when incorporating new tasks, structural modularity becomes essential to manage
20

---

Published in Transactions on Machine Learning Research (04/2026)
sequential knowledge without forgetting. Prompt-based architectures offer a lightweight solution: He et al. (2024); Ma'sum et al. (2025), and Wu et al. (2025c) utilized task-specific prompts to manage sequential new tasks, while Halbe et al. (2023) and Liang et al. (2025b) leveraged prompt pools to enable efficient task adaptation. Similarly, Salami et al. (2024) trained low-rank adapters to extend model capacity for specific tasks. Decomposing the architecture into decoupled components is another robust pattern. FedWeIT (Yoon) et al. 2021) and pFodC (Zhang et al., 2024bb demonstrated that by separating paramcters into "task- general" and "task-specifie" categories or using conflict-aware masks, the system can selectively integrate new knowledge. Furthermore, SacFL (Zhong et al. 2025) adopted an encoder-decoder split to facilitate continuous task arrival. Finally, Qi et al. [2025a] utilized dynamic hypernetworks to learn the correspondence between task identities and model weights, automating the architectural adaptation to continuously arriving lasks.
6.2 Real-world Applications
On the one hand, transitioning from new to old models, Galal et al. (024) proposed using pre-trained BERT to enhance existing FL models in distributed text classification. In drug discovery, Wang et al.] (2025b) intro- duced a federated knowledge distillation approach to improve the accuracy of compound-proteii interaction predictions through collaborative learning across pharmaceutical institutions. For time series analysis, Chen ct al. (2025b) suggested leveraging foundation models to enhance FL. approaches for heterogeneous time series data across distributed clients. On the other hand, transitioning from old to new models, in edge computing scenarios, Zuo et al. (2024) proposed a federated continual learning approach to transfer knowledge from traditional CNN models to VIT for improved visual recognition edge devices. In medical imaging, Tölle ct al. (2025) proposed transferring knowledge from cxisting CNN-based models to advanced transformer
architectures for enhanced cardiac CT image analysis in FL settings.
6.3 Future Works
Current federated knowledge distillation methods face significant communication and computational chal- lenges, and they lack robustness in diverse environments. These methods primarily depend on transmitting model outputs (logits) for distillation. Future research should investigate more efficient knowledge repre- sentation forms. For instance, studies could explore how to distill and transmit compressed or quantized knowledge (He et al. 2025b), or design protocols that only transmit the most essential knowledge (Gad et al. 2024). Additionally, when enhancing cxisting models with the general knowledge of foundation mod- els, there is a risk of overwriting valuable expertise acquired in specifie tasks. Future work should incorporate continuous learning into knowledge distillation. For example, using techniques like elastic weight consolida-
tion (Shoham et al. (2019a) or knowledge replay (Pennisi et al.] (024),, new knowledge can be introduced while effectively preserving and balancing existing expertise. Model evolution expands the capacity and interface of an FL. system, but system-level evolution also depends on the coordination logic that turns heterogeneous client updates into global progress. As client populations, data heterogeneity, and privacy constraints change over time, an establishod FL system may need to upgrade its aggregation rule to pre- serve existing functionality while absorbing newly arriving knowledge. We therefore next review aggregation algorithms as fourth carrier of new knowledge in evolving FL systems.
7 Incorporate New Aggregation Algorithms
Aggregation algorithms define the coordination logic of FL systems: they determine how client updates are weighted, synchronized, protected, and ultimately translated into global model improvements. As FL deployments evolve, naive averaging becomes increasingly insufficient under practical constraints such severe non-IID data, system hetcrogencity, privacy and security requirements, and continuously changing client populations. Consequently, incorporating new aggregation algorithms is not merely a performance tweak but a key mechanism for enabling system-level evolution. In this section, we review aggregation mothods from two complementary perspectives: (i) For Enhancing Current Functionality, where new algorithms improve robustness, convergence, and privacy guarantees under existing learning objectives; and (ii) For Facilitating New Knowledge Integration, where aggregation evolves from static averaging to
21

---

Published in Transactions on Machine Learning Research (04/2026)
Strengthening
LDP-Fod
TTrex
2020
Shaffle-PP GGrggi al. 2021
Privacy
SecAgr
Bonawila
2017
CaPC (Choqued lo- Cloo el
20220
Enhancing Functiounlity
Mitigating
FedProx
[La ct al
2020b
Scaffold
Karimireddy et al
2020b Moon Li ct al.
2021a
New Algorithms
Heterogmettty
FodProto
TTn al.
Facilitating
FedDaDiL
Montesuma
al.
2024 Risk Extrap [Ma
2024a
Li et
2021b
Integration
PFL-MC
MCGDM
Wet & Han.
7500
2024
FeaPOC
FeaCM
Nauyen et mi. 204b
Da
2124
MmPFL.
Zhang et nl. 2024d
Figure A taxonomy of technical approaches for incorporating new algorithms into FL systems can be organized into two broad categories: (i) methods that enhance existing functionalitics, and (ii) mothods that lacilitate the integration of additional new knowledge. For the former, we consider algorithm switching to strengthen privacy protection and mitigate data heterogeneity.
dynamic, structure-aware mechanisms that help absorb new features, new tasks, and heterogeneous models without destabilizing the federated process. A detailed taxonomy of these technical approaches is illustrated in Figure
7.1 Methods
7.1.1 For Enhancing Current Functionality
Mitigating Data Heterogeneity. A long-standing challenge in FL is that client data are typically non-IID and unevenly distributed, which induces client drift and makes naive averaging unstable or slow to converge. To address this, serics of aggregation and local optimization designs explicitly incorporate hetcrogeneity into the training dynamics. FedProx (Li et al., 2020b) modifies the local objective with proximal regular- ization term, constraining local updates from deviating excessively from the current global model, thereby stabilizing training when local objectives differ substantially. Complementarily, Scaffold (Karimireddy et al, 2020b) targets the bias introduced by heterogeneous local updates by using control variates to correct local gradients, encouraging local optimization to better align with the global descent direction. Beyond con- straining parameters or gradients, Moon (Li et al., 2021a) emphasizes that heterogeneity can manifest as representation misalignment across clients; adopts a model-contrastive learning objective to encourage local representations remain consistent with the global model, reducing over-specialization
local dis- tributions. FedProto (Tan al. 2022) further explores more semantic and communicttion-fieedlly form of alignment by exchanging class prototypes rather than raw gradients/paraametrr parameters, enabling clients to align class-wise feature representations under distribution skew. Collectively, these methods demonstrate that enhancing functionality under non-llD data often requires moving from purely averaging-base aggregation to eeterogencity-cwuar aware optimization and knowledge alignment mochanisms.
Strengthening Privacy Protection. In practical FL deployments, maintaining functionality also requires protecting client information throughout training and inference-related exchanges. Two mainstream direc- tions are differential privacy (DP) (Dwork) 2006) and secure Multi-Party Computation (MPC) (Goldreich) 1998 DP-based methods typically perturb client updates (e.g., gradients or parameters) to ensure that each client's s contribution is protected within a formal privacy budget. For example, FL with local DP [Truex et al. 2020 performs on-device perturbation before transmission, providing strong protection even under a weak trust assumption about the server, though it may introduce additional noise into optimization. FL with shuffle DP (Girgis et al. 2021) improves the privacy-utility trade-off by anonymizing the association between clients and their updates through shuffling, yielding privacy amplification while often retaining better model utility than purely local perturbation. Recent work further aims to harmonize heterogeneous differential privacy mechanisms and privacy budgets across dieens/rouudds to improve both accuracy and convergence in DP-FL (Feng et al., 2024a). In parallel, MPC-based approaches provide cryptographic protection by transforming client-side information, including model gradients, intermediate neural espresentations and in- ference predictions, into secrets for secure sharing and computation. SecAgg (Bonawitz et al. 2017) ensures that the server can only recover the aggregated sum (or average) of client updates without inspecting any in- dividual update, making it a foundational primitive for secure aggregation at scale. CaPC (Choquette Choo et al. 2020) similarly enables confidential collaborative computation by secret-sharing client data for secure
22

---

Published in Transactions on Machine Learning Research (04/2026)
learning when richer exchanged signals are needed beyond raw pa- rameter updates. Overall, these privacy-preserving techniques enhance current. FI, functionality by reducing information leakage risks while allowing aggregation to proceed in a principled (DP) or cryptographically protected (MPC) manner.
processing, supporting
7.1.2 For Facilitating New Knowledge Integration
New aggregation algorithms play a pivotal role in facilitating the integration of new knowledge by evolving from static averaging to dynamic mechanisms that act as the system's control logic. To explicitly address the heterogeneity of new features, aggregation stralegies are shifting towards metric-driven and fine-grained alignment. For instance, FedDaDiL (Montesuma et al., 2024) modeled distribution shifts using Wasserstein barycenters to align feature spaces based on optimal transport geometry, while Ma et al. (Ma et al.) 2024a) designed a risk extrapolation strategy to optimize aggregation coefficients for causal invariance. At finer level, Wei & Han (024) and Nguuyen et al. (2025) maximized gradient consistency to reduce distribution dis- crepancies by matching intra/inter /inter-domain gradients. When new tasks introduce severe functional confiicts, algorithms facilitate integration by dynamically modifying the aggregation topology. PFL-MCL (Zhou et al., 2024) and MuPFL (Zhang el al., 2024dp proposed iterative or adaptive clustering mechanisms that route diverse updates to different "knowledge communities", effectively preventing negative transfer between con- flicting tasks. Finally, to accommodate new models or heterogeneous architectures, algorithms are adopting decoupled or parastieer-ggnoticc aggregation schemes. Ditto (Li et al. 2021b) introduced bi-level mech- anism that regularizes local models towards the global average without forcing synchronization, creating an elastic buffer for diverse model parameters. Similarly, FedPCC (Dai et al. 2024 integrated prototype clustering into aggregation to unify feature spaces, allowing the integration of knowledge regardless of the underlying model structural differences.
7.2 Real-world Applications
In the healthcare domain, Khan et al. (2025) proposed EAH-FL, an encrypted in-network aggregation frame- work deployed at the network edge to effectively reduce both communication and computation overhead. It enables efficient federated training under bow-bandwidth conditions and substantially improves communica- tion latency and energy consumption without noliceably sacrificing model accuracy, demonstrating strong practicality for real-time medical monitoring. In industrial IoT, Mughal et al. (2024) proposed MEC-AT (HetFL), which adopts a multi-tier edge aggregation architecture with asynchronous updates to reduce com- munication costs. By orchestrating collaborative training across edge clusters and dynamically selecting clients and allocating resources based on network conditions, it avoids long-haul cloud transmissions and mitigates the straggler offect that can slow down global convergence. In smart agriculture, Li ct al. (2024a) proposed pruning-based aggregation approach that jointly reduces communication and compulation costs. The method prunes client models in each round and aggregates the pruned updates, significantly decreasing the transmitted parameters and the local training burden on resource-constrained devices.
7.3 Future Works
In this direction, future research should address the seamless integration of novel aggregation algorithms into existing frameworks. First, it remains to be determined whether directly switching aggregation algorithms could slow down the convergence speed of the FL model or impair its final convergence performance, as differ- ent algorithms possess distinct optimization trajectories and implicit regularization effects (Lim et al. 2025) that may lead to transient instability or divergence when the aggregation rule is abruptly altered. Second, if such a switch is viable, identifying the optimal timing for this transition is critical, requiring rescarch to determine if specific training phases exist where switching algorithms minimizes disruption while maximizing convergence gains and how to determine this timing systematically (Karimireddy et al. (020a)). Third, signif- icant technical difficulties arise when the new algorith requires the communication of additional historical information that was not stored during the training process of the old algorithm, necessitating methods to approximate missing state variables, such as momentum buffers or control variates to enable a smooth transition between statcless and stateful algorithms (Kiani et al., (2025). Finally, the aggregation mechanism must be capable of handling scenarios where the model utility and privacy requirements of different clients
23

---

Published in Transactions on Machine Learning Research (04/2026)
are distinct at both spatial and temporal levels (Su et al. 2023 developing adaptive strategies that can dynamically reconcile these heterogeneous and evolving constraints remains an open problem for the next generation of crivacy-preserving federated systems.
8
Future Works of FL with New Knowledge
8.1
Future Tendency and Open Problems
8.1.1
Future Tendency
FL is increasingly moving from one-off training to lifelong system evolution, where an established system must continuously incorporale new knowledge to satisly new demands while preserving existing functionality. In our setting, new knowledge can arise from four sources: new features, new tasks, new models, and new algorithms, and may be introduced by dynamically changing client groups (clients with only old knowledge, only new knowledge, or mixed knowledge). Such continual arrivals create time-varying objectives, higher system dynamics, and tighter resource constraints, especially in cross-device FL.
Future FL rescarch will increasingly treat sustainable upgrades as explicit system-level objectives, rather than side effects of improving optimization in fixed setting. This implies a shift from "improving a fixed task/moded to "maintaining living system" that supports recurrent upgrades with bounded cost and bounded regressions. Practically, this calls for versioned paloels/aggorithmms, upgrade policies, and rollback- ready deployment pipelines, making evolutionary FL closer to an MLOps--ikke continual delivery process. In parallel, as model technology advances, new architectures (e-g,, transformers and foundation models) are likely to serve as universal interfaces for absorbing diverse new knowledge, which encourages modular designs (adapters, prompt modules, task heads, split components) to enable parameter-efficien upgrades, partial activation, and backward compatibllity under strict device and bandwidth constraints.
Moreover, aggregation will evolve from static averaging to dynamic, structure-aware coordination that ex- plicitly handles helerogeneity in knowledge states, model structures, and client availability. In particular, algorithm evolution (new optimizers, new privacy mechanisms, new robustness designs) will be coupled with forming a unified coordination layer that decides who trains
the integration of new paatures/tasks/moonee what, when, and how. As contimual integration becomes the norm, governance primitives will also become indispensable: detecting whether incoming changes are truly new knowledge (vs. noise or transient drift), validating benefit under partial observability, attributing gains/losses to knowledge sources, and maintain- ing reprodueible upgrade history. Consequently, benchmarks will shift from singlo-stage performance to long-horizon outcomes that measure adaplation speed, retention of prior capabilities, cumulative utility over time, resource overhead, and robustness under non-stationarity, with protocols that simulate diverse arrival patterns (form and timing), dynamic client participation, and realistic system constraints.
8.1.2 Open Problems
Despite rapid progress, several fundamental problems remain open and will likely define the next phase of FL, with new knowledge. First, we still lack principled formulations and guarantees for non-stationary evolution: it remains unclear how to mathematically define "successful evolution" " when objectives shift over time, and how to unify fast adaptation to new knowledge, bounded forgetting of old knowledge, and constraints on communication/enann ammery/llttenyy /latency into a single optimization target with stability and convergence guarantoes under dynamic participation. Closely related is the challenge of delecting and characterizing new knowledge in realistic heterogeneous enviromments: when new features tasks appear, the system must decide whether to adapt, ignore, or defer, yet reliable novelty detection without labels is difficult, and one must distinguish genuine new demands from transient fluctuations, device faults, or low-quality data while controlling false alarms and missed detections.
Second, cvolutionary FL must support regression-free integration across hetcrogencous client knowledge states (only old, only new, or mixed). Updates that benefit new-knoweedge clients can harm old-knowledge clients, raising open questions on how to enforce "no significant harm" constraints during upgrades, how
24

---

Published in Transactions on Machine Learning Research (04/2026)
to design personalization that remains compatible with future global evolution, and how to quantify and control long-term fairness and accessibility as the system evolves.
Finally, we need dependency-aware co-evolution and safe switching mechanisms at the system level. New knowledge types often co-occur and depend on each other (e.g., a new task may require a representational upgrade), but we lack robust methods to build dependency graphs and plan integration sequences under re- source budgets. Meanwhile, switching aggregation algorithms or model architectures may require additional states (e.g., control variates, momentum, masks, prompts) that are missing or inconsistent across clients, making state migration, compatibility layers, and rollback/ /aanary mechanisms under federated constraints key unresolved issues. In addition, the community still lacks benchmarking standards for simulating arrival form/ himinignn dynamic client sets, and long-horizon evaluation; beyond final accuracy, metrics should cap- ture adaptation speed, forgetting, cumulative utility, stability under upgrades, and system costs to enable reproducible progress and meaningful comparisons.
Table 9: A synthesis of the threats, typical defenses, and the trade-offs that should be considered when incorporating each type of new knowledge into FL.
Koowledge Type
Theet
Defrase
Tradeoof
New
Resturn/gnaddinn holaaee Liger injuction
Saniticnsoo
apppapation clecks
Loser uilltys computta/connu
Dark tnderence; regitiee trandder backcoors
Daok solation (sagtt/ robuet ago: DP/SA
shactig: largee mocel; coordiaation
New models
Teijaned weights; lnvession Beakage suppl--caaio risk
Preveuses/ istestution barkdoos souss DP/SA, seadhoo Privacy socounting DP, Byzuttinc tobust agu: SA/MPC
Sceeening catl: slowss training; accumaes loss (DP)
Now algertiaass
Wealer privacy gununtos: Wedionisg/yyaaatte
Rebuszacs accuracy, extra rounds. vining complaily
8.2 Threats
Incorporating new knowledge into an FL system often requires communicating additional information be- yond standard model parameters or gradients. Therefore, it is urgent to propose dedicated methods to protect these additional contents in the semi-honest scenarios, in particular, against a variety of inference attacks /Wang et al. 2019: Luo et al. 2021). From a representation perspective, Soteria provided a prov- representation components to reduce reconstruction leakage 2021 Complementarily, task-agnostic privacy-preserving representa-
able defense by perturbing privacy-sensilie
while preserving utility (Sun et al.)
tion learning has been proposed to mitigate attribute inference attacks in FL without assuming a specific downstream task (Arevalo et al. 2024). Some preliminary tries include adding random (Dong et al.) 2022)
DP noise (Zizzo et al.] 2022) the domain task-specific models or class prototypes of FCL. However, such noise-based protection often comes at the cost of model performance. Given FL's multi-party nature, leveraging cryplographic methods like MPC is a promising alternative, as it can provide strong security guarantees without sacrificing utility. However, such privacy protection hurts the performance of FL sys- tems. Aligning with the intrinsic nature of multiple parties in FL, it may be possible to leverage MPC to communicate the additional contents. MPC is built on cryptography, and there are strong security guaran- tees. Besides, because the carriers of new knowledge are features, tasks, models, and algorithms, we need to consider the worst case that these carriers are polluted or poisoned in malicious scenarios. In this context, FedTil studied robust FL under persistent outliers while incorporating multi-level fairness constraints across clients and groups (Zhang et al. 2025b). Provenance primitives such as watermarking can further support ownership verification and model provenance tracking; for example, FedGMark provided a certifiably robust watermarking approach for federated graph learning (Yang et al. 2024c). For instance, features with poison attacks may be disguised as new features, and if there is no detection tool, they will be naturally incorpo- rated into FL. An optimization- based attack framework that broke statc-of-the-art poisoning defenses FL was presented in (Yang et al. 2024e).. Furthermore, malicious clients may conduct unintended local model training for society-hann -hamffl tasks; in this case, there is need to distinguish them from new benign tasks. New models may also contain backdoors (Xie et al. 2020), blindly incorporating them will cause serious out- comes. A secret sharing-inspired robust distributed backdoor attack against FL was studied in (Yang et al. 2025). Distributed backdoor attacks on federated graph learning and certified defenses were investigated in (Yang et al. 2024d). Finally, new algorithms themselves can introduce new attack surfaces; for example, some gradient inversion attacks (Huang et al.) (021) are only effective against specific FL algorithms. In summary, we provide a synthesis of the threats, typical defenses, and the trade-offs that should be considered for each type of new knowledge, shown in Table g
25

---

Published in Transactions on Machine Learning Research (04/2026)
8.3 Incentive
To build a continuously evolving FL ecosystem, the core is designing a dynamic incentive mechanism that effectively manages the knowledge lifecycle. This mechanism must address three key issues: how to fairly evaluate the contributions of different types of knowledge, how to incentivize the addition of high-quality knowledge, and how to handle the secure exit of knowledge.
Fair quantification of contributions is the cornerstone of all incentive mechanisms. Within the new knowl- edge framework defined in this paper, we must go beyond traditional evaluation methods that focus solely on data volume (Li et al. 2020a) and establish a multi-dimensional model capable of measuring the value of heterogeneous Enowledge. Table I0 summarizes a compact instantiation of this model across the four carriers. For contributions of new features, the Shapley value method can be employod. Guo et al. (2024a) state that when a participant introduces new feature, their contribution can be measured by the marginal improvement of this feature on the global model's performance. For contributions of new tasks, the net value evaluation method (Kirkpatrick et al.) 2017) is utilized. When a new task is introduced, it brings in new knowledge but may also interfere with existing tasks, causing catastrophic forgetting; thus, its contri- bution must. be a net effect. For contributions of new models or algorithms, a multi-dimensional weighted gain method can be adopted, comprehensively considering communication efficiency (Chen et al. 2023a computational efficiency (Pan et al. 2024), and model performance (Nilsson & Smith 2018
After being able to measure contributions, it's crucial to incentivize participants with high-quality knowl- edge to contribute honestly. Since participants have private information about the true quality of their contributions, contract theory (Liu et al. 2022) offers a suitable solution. Instead of offering a fixed price, a "contract menu" " can be designed for participants to choose from. Each menu item pairs a commitment level with a corresponding reward. Importantly, the reward for each contract must exceed the cost incurred by participants in contributing their knowledge; otherwise, participation will be discouraged. Meanwhile, the reward gap should be carefully designed so that participants with high-quality knowledge maximize benefits only by selecting high-commitment, high-return contracts. This encourages them to "tell the truth," " thus attracting high-quality knowledge into the system.
A healthy ecosystem must allow participants to withdraw their contributions safely when needed. Federated unlearning (Zhao et al.] 2025b) is central to achieving this. When a participant requests to withdraw their knowledge (whether features, tasks, or models), the system must perform an unlcarning operation, which incurs unlearning costs. Such unlearning requests may be proposed by different. participants at different times, thus preserving normal model functionality while achieving effective continuous unlearning (Gao et al.) is roquired. This cost depends on the thoroughness of unlcarning, categorized into approximate unlearning (Xiong et al. 2024) and exact unlearning (Qiu et al. 2023). Approximate unlearning efficiently reduces the influence of specific knowledge with lower computational costs and faster speed. Exact unlearning usually requires partial or complete retraining from a checkpoint, excluding the target knowledge, leading to higher computational costs. The unlearning cost forms the basis for designing the exit clause in the incentive mechanism. Within this framework, the exit clause can explicitly link the unlearning cost to the participant's responsibilitics. Participants can choose between approximate or exact unlcarning basod on their needs. Choosing the former may involve a small fee or a deduction of reputation points, while the latter entails bearing higher proportion of computational costs. Additionally, for long-tern contributors with high reputation scores, the system can subsidize part of the unlearning cost when they exit, recognizing their past contributions.
Table 10: A compact multi-dimensionn model for valuing contributions of new knowledge in evolving FL.
Carrier
What changes
Contribution
New features
Imput distribution/ / mudality
Marginal utility (e.g. shapley-style altribution) and rubustness to feature shiff Net utility: gain on new tasks minus loes (forgetting) on prior tasks Weighted gain over accurary, retention, compute, and communication cost Weighted gain over convergence, robustness, privacy; plus transition cost
New basks
Task set / functionality
New models
Architerture capacity
New algorithmu
Aggregation protocol
26

---

Published in Transactions on Machine Learning Research (04/2026)
9 Conclusion
In this paper, we focus comprehensive review of studies relevant to incorporating new knowledge into FL to achieve its sustainable development. We consider new knowledge from four sources: features, tasks, models, and algorithms. For incorporating new features, we discuss a set of processes to enhance the out-of- distribution detection and generalization ability of FL systems. Rogarding new tasks, we discuss how to use pask-personalized and self-supervised FL to strengthen the cross-task generalization capability of FL, and we also introduce how learn new tasks through federated contimual learning. Additionally, we review existing rescarch that may facilitate the fusion of different FL models and the transition between different algorithms. Finally, comprehensively discuss future research directions, considering factors such as scenario setups, security and privacy threats, and incentives.
Acknowledgments
We thank the meta and anonymous reviewers for their valuable comments. This research is supported by the National Research Foundation, Singapore, and the Cyber Security Ageney of Singapore under the National Cybersecurity R&DD Programme and the CyberSG R&D Programme Office (Award CRPO-GC3- NTU-001). Any opinions, findings, conclusions, or recommendations expressed in these materials are those of the author(s) and do not reflect the views of the National Research Foundation, Singapore, the Cyber Security Agency of Singapore, or the CyberSG R&D Programme Office.
References
Ali Abedi, QM Wu, Ning Zhang, and Farhad Pourpanah. One-shot federated unsupervised domain adaptation with scaled entropy attention and multi-source smoothed pscudo labeling. arXiv preprint (xXiu:2503.1002) 2025.
Samiul Alam, Luyang Liu, Ming Yan, and et al. Fedrolex: Model-heterogencons federated learning with rolling sub-model extraction. NeurIPS, 2022.
Caridad Arroyo Arevalo, Sayedeh Leila Noorbakhsh, Yun Dong, Yuan Hong, and Binghui Wang. Task- agnostic privacy- pencccy- representation learning for federated learning against attribute inference at- tacks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 10909- 10917, 2024.
John Ayeelyan, Sapdo Utomo, Adarsh Rouniyar, Hsiu-Chun Chun Hsu, and Pao-Ann Hsiung. Federated learning design and functional models: Survey. Artificial Intelligence Review, 58(1):21, 2024.
Ruqi Bai, Saurabh Bagchi, and David I Inouye. Benchmarking algorithms for federated domain generaliza- tion. arXiv preprint arXiv:2307.04900 2023.
Yavuz Faruk Bakman, Duygu Nur Yaldiz, et al. Federated orthogonal training: Mitigating global catastrophic forgetting in continual federated learning. ICLR, 2024.
Sara Beery, Grant Van Horn, and Pietro Perona. Recognition terra incognita. In Proceedings of the European conference on computer vision (ECCV), pp. 456-473, 2018.
Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, et al. Practical secure aggregation for privacy-preserving machine learning. In ACM CCS, 2017.
Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečný, H Brendan McMa- han, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings. arXio preprint arXiv:1812.01097, 2018.
Yatin Chaudhary, Pranav Rai, Matthias Schubert, and et al. Federated continual learning for text classifi- cation via selective inter-client transfer. In Findings of EMNLP, 2022.
27

---

Published in Transactions on Machine Learning Research (04/2026)
Haoxuan Che, Yifei Wu, Haibo Jin, Yong Xia, and Hao Chen. Feddag: Federated domain adversarial generation towards generalizable medical image analysis. Xiv preprint or NN::2001113967, 2025.
Chuan Chen, Tiancii Liao, Xiaojun Deng, Zihou Wu, Sheng Huang, and Zibin Zheng. Advances in robust federated learning: A survey with heterogeneity considerations. IEEE Transactions on Big Data, 2025a.
Dengsheng Chen, Jie Hu, Vince Junkai Tan, Xiaoming Wei, and Enhua Wu. Elastic aggregation for federated optimization. In Proceedings of the IEEE/CVF Conference Computer Vision and Patlern Recognition, pp. 12187-12197, 2023a.
Junming Chen, Meirui Jiang, Qi Dou, et al. Federated domain generalization for image recognition via cross-client style transfer. In WACV, 2023b.
Shengchao Chen, Guodong Long, Jing Jiang, and Chengqi Zhang, Federated foundation models het- crogencous time scries. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 15839-15847, 2025b.
Dongzhou Cheng, Lei Zhang, Can Bu, Xing Wang, Hao Wu, and Aiguo Song. Protohar: Protolype guided personalized federated learning for human activity recognition. IEEE Journal of Biomedical and Health Informatics, 27(8):3900-300 2023.
Hao Chi, Hui Xia, Shuo Xu, Yusheng Hc, and Chunqiang Hu. Fmdada: Federated multi-diserrminative adversarial domain adaptation. Applied Intelligence, 54(17):7849 7863, 2024.
Christopher A Choquette-Choo Choo, Natalie Dullerud, et al. Cape learning: Confidential and private collabora- tive learning. In ICLR, 2020.
Cheng Dai, Shuai Wei, Shengxin Dai, Sahil Garg, Georges Kaddoum, and M Shamim Hossain. Federated self- supervised learning based on prototypes clustering contrastive learning for internet-of-velicles applications. IEEE Internet of Things Journal, 2024.
Erfan Darzi, Yiqing Shen, Yangming Ou, Nanna M Sijtsema, and Peter MA van Ooijen. Tackling hetero- geneity in medical federated learning via aligning vision transformers. Artificial Intelligence in Medicine, 155:102936, 2024.
Shohreh Deldari, Dimitris Spathis, Mohammad Malekzadeh, Fahim Kawsar, Flora D Salim, and Akhil
Mathur. Crossl: Cross-modal self-supervised lcarning for time-scries through latent masking. In Proceed- ings of the 17th ACM International Conference on Web Search and Data Mining, pp. 152 2024.
Yongheng Deng, Ziqing Qiao, Ju Ren, Yang Liu, and Yaoxue Zhang. Mutual enhancement of large and small language models with cross-silo knowledge transfer. vrXiu:2312.058// 2023.
Zhipeng Deng, Zhe Xu, Tsuyoshi Tsshiki, and Yefeng Zheng. Fedsemidg: Domain generalized federated semi-supervised medical image segmentation. arXio preprint orXiv:2501.07376 2025.
Jiahua Dong, Lixu Wang, et al. Federated inss-incrementll lcarning. In CVPR, 2022.
Jiahua Dong, Duzhen Zhang, Yang Cong, Wei Cong, Henghui Ding, and Dengxin Dai. Federated incremental semantic segmentation. In CVPR, 2023.
Jiahua Dong, Wenqi Liang, Yang Cong, Gan Sun, Lixu Wang, Henghui Ding, Yulun Zhang, and Lue Van Gool. Learning from each other: Generalized federated incremental semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026.
Cynthia Dwork. Differential privacy. In International colloquium on automata, languages, and programming, pp. 1-12. Springer, 2006.
Omar El-Rifai, Michael Ben Ali, Imen Megdiche, André Peninou, and Olivier Teste. A survey on cluster- based federated learning. Xriio preprint xxXi:2501.17511 2025.
28

---

Published in Transactions on Machine Learning Research (04/2026)
Yuwei Fan, Wei Xi, Hengyi Zhu, and Jizhong Zhao. Minipfl: Mini federations for hicrarchical personalized federated learning. Fulure Generation Computer Systems, 157:41-50, 2024.
Chao Feng, Nicolas Fazli Kohler, Alberto Huertas Celdran, Gerome Bovet, and Burkhard Stiller. Col-
net: Collaborative optimization in decentralizod federated multi-task learning systems. orXiv preprint arXiv:2501.10447, 2025a.
Jiyuan Feng, Xu Yang, Liwen Liang, Weihong Han, Binxing Fang, and Qing Liao. Cgofed: Constrained gradient optimization strategy for federated class incremental learning. IEFE Transactions Knowledge and Data Engineering, 2025b.
Shuya Feng, Mcisam Mohammady, Hanbin Hong, Shenao Yan, Ashish Kundu, Binghui Wang, and Yuan Hong. Harmonizing differential privacy mechanisms for federated learning: Boosting accuracy and con- vergence. In Proceedings of the Fifteenth ACM Conference on Data and Application Security and Privacy, pp. 60 71, 2024a.
Zhanbo Feng, Yuanjic Wang, Jie Li, Fan Yang, Jiong Lou, Ticbin Mi, Robort C Qiu, and Zhenyu Liao. Robust and communication-efiiiin federated domain adaptation via random features. IEEE Transaclions on Knowledge and Data Engineering, 2024b.
Gad Gad, Zubair Md Fadlullah, Mostafa M Fouda, Mohamed I Ibrahem, and Nei Kato. Federated learning with selective knowledge distillation over bandwiill--onnstrained wireless networks. In ICC 2024-IEEE International Conference on Communications, pp. 3476-3481. IEEE, 2024.
Omar Galal, Ahmed H Abdel-Gawad, and Mona Farouk, Federated freeze bert for text classification. Journal of Big Data, 11(1):28, 2024.
Chongyang Gao, Lixu Wang, Kaize Ding, Chenkai Weng, Xiao Wang, and Qi Zhu. On large language model continual unlearning. In The Thirteenth International Conference Learning Representations.
Yang Ge, Yang Zhou, and Li Jia. Adaptive personalized federated learning with one-shot screening. IEEE Internet of Things Journal, 11(9):15375-158) 2024.
Antonious Girgis, Deepesh Data, Suhas Diggavi, Peter Kairouz, and Ananda Theertha Suresh. Shuffled model of differential privacy federated learning. International Conference on Artificial Intelligence and Statistics, pp. 2521-2529. PMLR, 2021.
Oded Goldreich. Secure multi-party computation. Manuscript. Preliminary version, 1998.
Kaijie Gong, Yi Gao, and Wei Dong. Privacy-preservee and cross-domain human sensing by federated domain adaptation with semantic knowledge correction. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 8(1):1 26, 2024a.
Shuai Gong, Chaoran Cui, Chunyun Zhang, Wenna Wang, Xiushan Nic, and Lei Zhu. Federated domain generalization via prompt learning and aggregation. arXiv preprint 1rX 2024b.
Xueying Gu, Qiong Wu, Qiang Fan, and Pingyi Fan. Mobility-aware federated self-supervised learning in vehieular network. Urban Lifeline, 2(1):10, 2024.
Badra Souhila Guendouzi, Samir Ouchani, Hiba EL Assaad, and Madeleine EL Zaher. A systematic review of federated learning: Challenges, aggregation methods, and development tools. Journal of Network and Computer Applications, 220:103714, 2023.
Junfeng Guo, Yiming Li, Lixu Wang, Shu-Tao Xia, Heng Huang, Cong Liu, and Bo Li. Domain watermark: Effective and harmless dataset copyright protection is closed at hand. In NewrIPS, 2023.
Peng Guo, Yanqing Yang, Wei Guo, and Yanping Shen. A fair contribution measurement method for federated learning. Sensors, 24(15):4967, 2024a.
29

---

Published in Transactions on Machine Learning Research (04/2026)
Wei Guo, Fuzhen Zhuang, Xiao Zhang, Yiqi Tong, and Jin Dong. A comprehensive survey of federated transfer learning: challenges, methods and applications. Frontiers of Computer Science, 18(6):186356, 2024b.
Sunny Gupta, Vinay Sutar, Varunav Singh, and Amit Sethi. Fedalign: Federated domain generalization with cross-client feature alignment. arXiv preprint arXiv:2501.154 2025.
Shaunak Halbe, James Seale Smith, Junjiao Tian, and Zsolt Kira. Hepco: Data-free heterogeneous prompt consolidation for continual federated learning. arXi::2366.09) 2023.
Parisa Hamedi, Roozbeh Razavi-Far, and Ehsan Hallaji. Federated continual learning: Concepts, challenges, and solutions. Xrwi preprint urXiv:2502.0705, 2025.
Xuming Han, Qiaohong Zhang, Zaobo He, and Zhipeng Cai. Confidence- based similarity-aware personalized federated learning for autonomous iot. IEEE Internet of Things Journal, 11(7):13070 13081, 2023.
Cosmin-Andrei Hatfaludi and Alex Serban. Foundational models and federated learning: survey, taxonomy, challenges and practical insights. PeerJ Computer Science, 11:c2993, 2025.
Yuan Hc, Yingchun Cui, Zhengda Wu, Heran Xi, and Jinghua Zhu. Optimal transport-driven federated oul-of-distribution detection in heterogeneous data. In Proceedings of the 2025 International Conference on Multimedia Retrienal, pp. 424-432, 2025a.
Yuchen He and Xiangleng Wang. Masked autoencoders are paramette--lffcient federated continual learners. In 2024 IEEE International Conference on Big Data (BigData), pp. 3682 3691. IEEE, 2024.
Yuchen He, Chuyun Shen, Xiangfeng Wang, and Bo Jin. Fppl: An efficient and non-iid robust federated continual learning framework. In 2024 IEEE International Conference on Big Data (BigData), pp. 3692- 3701. IEEE, 2024.
Zixiao He, Gengming Zhu, Shaobo Zhang, Entao Luo, and Yijiang Zhao. Feddt: A commuicctionn-ffciintt federated learning via knowledge distillation and ternary compression. Electronics, 14(1)):2183, 2025b.
Dan Hendryeks and Thomas Dietterich. Benchmarking neural network robustness common corruptions and perturbations. arXiv preprint arXiv:1903. 12261, 2019.
Wenke Huang, Mang Ye, and Bo Du Learn from others and be yourself in heterogeneous federated learning. In CVPR, 2022a.
Yangsibo Huang, Samyak Gupta, Zhao Song, Kai Li, et al. Evaluating gradient inversion attacks and defenses in federated learning. NeurIPS, 2021.
Yixing Huang, Christoph Bert, Stefan Fischer, Manuel Schmidt, Arnd Dörfler, et al. Continual learning for peer-to-peer federated learning: A study on automated brain metastasis identification. arXiv:2204.1388 135911 2022b.
Gu-Bon Jeong and Dong- Wan Choi. nut-of-distribution detection via outlier exposure in federated learning. Newral Networks, pp. 107141, 2025.
Shusen Jing, Anlan Yu, Shuai Zhang, and Songyang Zhang. Fedsc: provable federated self-supervisel learning with spectral contrastive objective over non-iid data. arXiv preprint (tl::2405.09443 2024.
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Bennis, et al. Advances and open problems in federated learning. Foundations and Trends in Machine Learning, 2021.
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International conference machine learning, pp. 5132-510 5143. PMLR, 2020a.
Sai Pranecth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, al. Scaffold: Stochastic controlled averaging for federated learning. Tn ICML, 2020b.
30

---

Published in Transactions on Machine Learning Research (04/2026)
Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv preprint Nxfe:1775.009550 2017.
Huzaif Khan, Rahul Kavati, Sriven Srilakshmi Pulkaram, and Ali Jalooli. End-to-end privacy-aware federated lcarning for wearable health devices via encrypted aggregation in programmablc networks Sensors, 25 (22):7023, 2025.
Shahrzad Kiani, Nupur Kulkarni, Adam Dziedzic, Stark Draper, and Franziska Boenisch. Differentially private federated learning with time-adaptive privacy spending, arXiv preprinl arx::2502. 18706, 2025.
Tae Hyun Kim, Jae Yong Yu, Won Seok Jang, Sun Cheol Heo, MinDong Sung, JaeSeong Hong, KyungSoo Chung, and Yu Rang Park. Ppfi: A personalized progressive federated learning method for leveraging different healthcare institution-specifi features. Iscience, 27())), 2024.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaune Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwwnskke et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521 3526, 2017.
Xiangjie Kong, Wenyi Zhang, Hui Wang, Mingliang Hou, Xin Chen, Xiaoran Yan, and Sajal K Das. Federated graph anomaly detection via contrastive self-supervised learning. IEEE Transactions on Neural Networks and Learning Systems, 2024.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images.(2009) 2009.
Huilin Lai, Ye Luo, Bo Li, Jianwei Lu, and Junsong Yuan. Bilateral proxy federated domain generalization for privacy-preserving medical image diagnosis. IEEE Journal of Biomedlccal and Health Informatics, 2024.
Yann Le and Xuan Yang. Tiny imagenct visual recognition challenge. CS 231N, 7(7):3, 2015.
Andy Li, Milan Markovic, Peter Edwards, and Georgios Leontidis. Model pruning enables localized and efficient federated learning for yield forecasting and data sharing, Expert Systems with Applications, 242: 122847, 2024a.
Bing Li, Wei Cui, Le Zhang, Qi Yang, Min Wu, and Joey Tianyi Zhou. Democratizing federated wifi-based human activity recognition using hypothesis transfer. IEEE Transactions Mobile Computing, 2024b.
Boyuan Li, Shengbo Chen, and Zihao Peng. New generation federated learning. Sensors, 2022a.
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain gener- alization. In Proceedings of the IEEE international conference on compuler vision, pp. 5542 5550, 2017.
Hanzhe Li, Shiji Zhou, Bo Yuan, and Mingxuan Zhang. Optimizing intelligent edge computing resource scheduling based on federated learning. Journal of Knowledge Learning and Science Technology ISSN: 6959-6386 (online), 3(3):235 260, 2024c.
Jingtao Li, Lingjuan Lyu, Daisuke Iso, Chaitali Chakrabarti, and Michael Spranger. Mocosfl: enabling cross-client collaborative self-supervised learning. In The Eleventh International Conference on Learning Representations, 2022b.
Keqiuyin Li, Jie Lu, Hua Zuo, and Guangquan Zhang. Federated fuzzy transfer learning with domain and category shifts. IEEE Trunsactions on Fuzzy Systems, 2024d.
Qinbin Li, Bingsheng He, and Dawn Song, Model-conntaative federated learning. In CVPR, 2021a.
Shawn Li, Peilin Cai, Yuxiao Zhou, Zhiyu Ni, Renjie Liang, You Qin, Yi Nian, Zhengzhong Tu, Xiyang Hu, and Yue Zhao. Secure on-device vidco ood detection without backpropagation. arXiv preprint arXiv:2503.0611 2025a.
Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Challenges, methods, and future directions. IEFE signal processing magazine, 37(3):50 60, 2020a.
31

---

Published in Transactions on Machine Learning Research (04/2026)
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, et al. Federated optimization in heterogeneous networks. MT,Sys, 2020b.
Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust federated learning through personalization. In ICML, 2021b.
Yichen Li, Yuying Wang, Tianzhe Xiao, Haozhao Wang, Yining Qi, and Ruixuan Li. Rehearsal-free continual federated learning with synergistic regularization. arXiv preprint (r1t:22422.19779, 2024e.
Yichen Li, Haozhao Wang, Yining Qi, Wei Liu, and Ruixuan Li. Re-fed+: A better replay strategy for federated incremental learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025b.
Ying Li, Xingwei Wang, Rongfei Zeng, Praveen Kumar Donta, Ilir Murturi, Min Huang, and Schahram Dustdar. Federated domain generalization: A survey. Proceedings of the IEEE, 2025c.
Fang-Y Yi Liang, Yu-Wei Zhan, Jiale Liu, Chong-Yo Yu Zhang, Zhen-Duo Chen, Xin Luo, and Xin-Shun Xu. Class-aware prompting for federated few-shot class-incremennall learning. IEEE Transactions on Civcuils and Systems for Video Technology, 2025a.
Fang- Yi Liang Yu-Wer Zhan, Jiale Liu, Chong Yu Zhang, Zhe Chen, Xin Luo, and Xin-Shun Xu. Class-aware prompting for federated few-shot class-incremental learning. IEEE Transactions on Circuits and Systems for Video Technology, 2025b.
Jinglin Liang, Jin Zhong, Hanlin Gu, Zhongqi Lu, Xingxing Tang, Gang Dai, Shuangping Huang, Lixin Fan, and Qiang Yang. Diffusion-driven data replay: A novel approach to combat forgetting in federated class continual learning. In European Conference on Computer Vision, pp- 303-319. Springer, 2024.
Xinting Liao, Weiming Liu, Pengyang Zhou, Fengyuan Yu, Jiahe Xu, Jun Wang, Wenjie Wang, Chaochao Chen, and Xiaolin Zheng. Foogd: Federated collaboration for both out-of-distribution generalization and detection. Advances in Neural Information Processing Systems, 37:132908-132945 2024.
Jinwoo Lim, Suhyun Kim, and Soo-Mook Moon. Convergence analysis of federated learning methods using backward error analysis. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pp. 2025.
Bingyan Liu, Nuoyan Lv, Yuanchun Guo, and Yawen Li. Recent advances on lederated learning: A systematic survey. Neurocomputing, 597: 597:128019, 2024a.
Chenxi Liu, Lixu Wang, Lingjuan Lyu, Chen Sun, Xiao Wang, and Qi Zhu. Deja vu: Continual model gencralization for unseen domains. In ICLR, 2023.
Jing Liu, Wei Zhu, Di Li, Xing Hu, and Liang Song, Domain generalization with semi-supervised learning for people-centric activity recognition. Science China Information Sciences, 68(1):112103, 2025a.
Junming Liu, Guosun Zeng, Ding Wang, Yanting Gao, and Yufci Jin. Fodrecon: Missing modality recon-
struction in distributed heterogeneous environments. arXiv preprint arXiv:2504 2025b.
Qianli Liu, Zhaorui Zhang, Xin Yao, and Benben Liu. Hlora: Efficient foderated lcarning system for Ilm
heterogeneous fine-tuiigg arXiv preprint arXie:2503. 000001500,0 2025c.
Quande Liu, Cheng Chen, Jing Qin, Qi Dou, el al. Feddg; Federated domain generalization on medical image segmentation via episodic learning in continuous frequency space. In CVPR, 2021.
Yuan Liu, Mengmeng Tian, Yuxin Chen, Zehui Xiong, Cyril Leung, and Chunyan Miao. A contract theory based incentive mechanism for federated learning. In Federated and Transfer Learning, pp. 117-137. Springer, 2022.
Yuan Liu, Shu Wang, Zhe Qu, Xingyu Li, Shichao Kan, and Jianxin Wang. Fodgea: Global consistent augmentation based singlo-source federated domain generalization. In 2024 IEEE International Conference on Multimedia and Expo (ICMF), pp. 1 1-6. TEFE, 2024b.
32

---

Published in Transactions on Machine Learning Research (04/2026)
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Yanyan Lu, Lei Yang, Hao-Rui Chen, Jiannong Cao, Wanyu Lin, and Saiqin Long. Federated class- incremental learning with dynamic feature extractor fusion. IEEE Transactions on Mobile Computing. 2024.
Xinjian Luo, Yuncheng Wu, Xiaokui Xiao, and Beng Chin Ooi. Feature inference attack on model predictions in vertical federated learning. In ICDE, 2021.
Yaxin Luopan, Rui Han, Qinglong Zhang, Xiaojiang Zuo, Chi Harold Liu, Guoren Wang, and Lydia Y Chen. Loci: Federated continual learning of heterogeneous tasks at edge. IEEE Transactions on Parallel and Distributed Systems, 2025.
Kexin Lv, Rui Ye, Xiaolin Huang, Jie Yang, and Siheng Chen. Learn what you need in personalized federated learning. arXiv preprint 0rXiv:2401.08322 2024.
Shuran Ma, Weiying Xie, Daixun Li, Haowei Li, and Yunsong Li. Reducing spurious correlation for federated domain generalization. arXiv preprint orXic:247..19174 2024a.
Yuting Ma, Shengeng Tang, Xiaohua Xu, and Lechao Cheng. Modality alignment. meets federated broad- casting, arXiv preprint arXiv:2411.1588 2024b.
Jingxin Mao, Zhiwei Wei, Bing Li, Rongqing Zhang, and Lingyang Song. Towards ever-evolution network threats: A hierarchical federated class-incrmmentll learning approach for network intrusion detection in iiot. IEEE Internet of Things Journal, 2024.
Muhammad Anwar Ma'sum, Mahardhika Pratama, Lin Liu, Habibullah Habibullah, and Ryszard Kowalczyk. Federated few-shot class-incremental learning. In The Thirteenth International Conference on Learning Representations, 2025.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communictiion-cffcient learning of deep networks from decentralized data. In AISTATS, 2017.
Yongsheng Mei, Liangqi Yuan, Dong-Jun Han, Kevin S Chan, Christopher G Brinton, and Tian Lan. Using diffusion models as generative replay in continual federated cearningwhaat will happen? arXie preprint wrXiv:2411066618 2024.
Hao Miao, Yan Zhao, Chenjuan Guo, Bin Yang, Kai Zheng, and Christian S Jensen. Spatio-temporal prediction on streaming data: A unified federated continuous learning framework. JEEE Transactions on Knowledge and Data Engineering, 2025.
Mohammadreza Mohammadi, Rakesh Shrestha, Sima Sinaei, et al. Anomaly detection using lstm- autoencoder in smart grid: A federated learning approach. In SSRN, 2023.
Eduardo Fernandes Montesuma, Fabiola Espinoza Castellon, Fred Ngolè Mboula, Aurélien Mayoue, Antoine Souloumiac, and Cédric Gouy-Pailler. Dataset dictionary learning in wasserstein space for federated domain adaptation. arXiv preprint wcXto:2407.11647 2024.
Jean-Jacques Moreau. Proximité et dualité dans un espace hilbertien. Bulletin de lo Société mathématique de France, 93:273-20 299, 1965.
Junki Mori, Kosuke Kihara, Taiki Miyagawa, Akinori F Ebihara, Isamu Teranishi, and Hisashi Kashima. Federated source-free domain adaptation for classificatio:: Weighted cluster aggregation for unlabeled data. ar.iio preprint arXiv:2412.1375 13757, 2024.
Junki Mori, Kosuke Kihara, Taiki Miyagawa, Akinori F Ebihara, Isamu Teranishi, and Hisashi Kashima. Foderated sourcc-free domain adaptation for classificatin:: Weighted cluster aggregation for unlabeled data. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 6879- 6889. IEFF, 2025.
33

---

Published in Transactions on Machine Learning Research (04/2026)
Fahad Razaque Mughal, Jingsha He, Bhagwan Das, Fayaz Ali Dharejo, Nafei Zhu, Surbhi Bhatia Khan, and Saeed Alzahrani. Adaptive federated learning for resourc--onsttriined iot devices through edge intelligence and multi-edge clustering, Scientific Reports, 14(1):28746, 2024.
Mirko Nardi, Lorenzo Valerio, and Andrea. Passarella. Anomaly detection through unsupervised federated learning, 2022.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsuperuised feature learning, volume 2011, pp. 4. Granada, 2011.
Dung Thuy Nguyen, Taylor T Johnson, and Kevin Leach. Fisc: Federated domain generalization via inter- polative style transfer and contrastive learning. arXiv preprint arXiv:2410.22622 2024a.
Thinh Nguyen, Khoa D Doan, Binh T Nguyen, Danh Le-Phuoc, and Kok-Seng Wong. Overcoming catas- trophic forgetting in federated mass-incremental learning via federated global twin generator. arXiv preprint ar trXiv:2407.1177 11078, 2024b.
Trong-Bine Binh Nguyen, Minh-Duong Nguyen, Jinsun Park, Quoc- Pham, and Won Joo Hwang. Federated domain generalization with data-frec on-server gradient matching. arXiv preprint arXi:22501.14655 2025.
Adrian Nilsson and Simon Smith. Evaluating the performance of federated learning case study of distributed machine learning with erlang. 2018.
Ziqing Niu, Peirui Cheng, Zhirui Wang, Liangjin Zhao, Zheng Sun, Xian Sun, and Zhi Guo. Feil-msn: A federated inaseinnrementll learning method for multi-satellite networks. IEEE Transactions Geoscience and Remole Sensing, 2024.
Kai Pan, Yapeng Tian, Yinhe Han, and Yiming Gan. Benchmarking and optimizing federated learning with hardware-related metries. 2024.
Meng Pang, Houwei Xu, Zheng Huang, Yintao Zhou, Shengbo Chen, Binghui Wang, and Wei Huang. A tri-factor adaptive federated learning framework for parkinson's 's discase diagnosis via multi-source facial expression analysis. IEEE Journal of Biomedical and Health Informatics, 2025a.
Meng Pang, Houwei Xu, Zheng Huang, Yintao Zhou, Wei Huang, and Binghui Wang. Breaking data silos in parkinson's 's disease diagnosis: An adaptive lederated learning approach for privacy-preserving facial expression analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 14352 14360, 2025b.
Christos Papadopoulos, Konstantinos-Fllippos Kollias, and George F Fragulis. Recent advancements in federated learning: state the art, fundamentals, principles, iot applications and future trends. Future Internet, 16(11):415, 2024.
Sangjoon Park, Ik Jae Lee, Jun Won Kim, and Jong Chul Ye. Ms-dino: Masked self-supervised distributed learning using vision transformer. IEEE Journal of Biomedical and Health Informatics, 2024.
Can Peng, Qianhui Men, Pramit Saha, Qianye Yang, Cheng Ouyang, and J Alison Noble. Federated continual 3d segmentation with single-round communication arXiv preprint arXiv:2508.15414 2025.
Liang Peng, Nan Wang, Nicha Dvornek, Xiaofeng Zhu, and Xiaoxiao Li. Fedni: Federated graph learning with network inpainting for population-base disease prediction. IEEE Transactions Medical Imaging, 42(7):2032-2043, 2022.
Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: The visual domain adaptation challenge. arXiv preprint ar Xrxi::1710.0080 06924, 2017.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Sacnko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1406-1455 2019.
34

---

Published in Transactions on Machine Learning Research (04/2026)
Xingchao Peng, Zijun Huang, Yizhe Zhu, and Kate Saenko. Federated adversarial domain adaptation. In RCLR, 2020.
Matteo Pennisi, Federica Proietto Salanitri, Giovanni Bellitto, Bruno Casella, Marco Aldinucci, Simone
Palazzo, and Concetto Spampinato. Feder: Federated learning through experience replay and privacy- preserving data synthesis. Computer Vision and Image Understanding, 238:103882, 2024.
Hongming Piao, Yichen Wu, Dapeng Wu, and Ying Wei. Federated continual learning via prompt-basod dual knowledge transfer. In Forty-first International Conference on Machine Learning, 2024.
Farhad Pourpanah, Mahdiyar Molahasani, Milad Soltany, Michael Greenspan, and Ali Etemad. Federated unsupervised domain generalization using global and local aligument of gradients. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 11948-19955 2025.
Daiqing Qi, Handong Zhao, and Sheng Li. Better generalive replay for continual federated learning. In ICLR, 2022.
Xiaoming Qi, Jingyang Zhang, Huazhu Fu, Guanyu Yang, Shuo Li, and Yueming Jin. Dynamic allocation hypernetwork with adaptive model recalibration for fel. arXiv preprint arXie:2503.1806 2025a.
Zhuang Qi, Runhui Zhang, Lei Meng, Wei Wu, Yachong Zhang, and Xiangxu Meng. Global intervention and distillation for federated out-of-distribution generalization. arXiv preprint orXiv:2504.008: 2025b.
Hongyu Qiu, Yongwei Wang, Yonghui Xu, Lizhen Cui, and Zhiqi Shen. Fedcio: Efficient exact federated unlearning with clustering, isolation, and one-shot aggregation. In 2023 IEEE International Conference on Big Data (BigData), pp. 5568. IEEE, 2023.
Nguyen Huu Quyen, Nguyen Viet Hoang, Phan The Duy, and Van-Hau Pham. Fi-ids: federated incre- mental learning approach for intrusion detection system. In 2024 International Conference on Advanced Technologies for Communications (ATC), pp. 432-437. IEFE, 2024.
Leyla Rahimli, Feras M Awaysheh, Sawsan Al Zubi, and Sadi Alawadi. Federated learning drift detection:
An empirical study on the impact of concept and data drift. In 2024 2nd International Conference on Federated Learning Technologies and Applications (FLTA), pp. 241 250. IEEE, 2024.
Aditya Narayan Ravi and Ilan Shomorony. Prioritized federated learning: Leveraging non-priority clients for targeted model improvement. Transactions on Machine Learning Research, 2024.
Giulia Rizzoli, Mattco Caligiuri, Donald Shenaj, Francesco Barbato, and Pictro Zanuttigh. When cars meet drones: Hyperbolic federated learning for source-free domain adaptation in adverse weather. arXiv preprint arx::22403. 13762, 2024.
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In Computer vision ECCV 2010: 11th European conference on computer vision, Heraklion, Crete, Greece, Seplember 5-11, 2010, proceedings, part iV 11, pp. 213 226. Springer, 2010.
Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximun classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
2018.
Riccardo Salami, Pietro Buzzega, Matteo Mosconi, Jacopo Bonato, Luigi Sabetta, and Simone Calder- ara. Closed-form merging of parameter-efficient modules for federated continual learning, arXiv preprint arXiv:2410 17961, 2024.
Hassan Salman, Chamseddine Zaki, Nour Charara, Sonia Guehis, Jean-François Pradat-Peyre, and Abbass Nasser. Knowledge distillation in federated learning: comprehensive survey. Discover Computing, 28(1): 145, 2025.
Lorenzo Sani, Alex Iacob, Zeyu Cao, Royson Lee, Bill Marino, Yan Gao, Dongqi Cai, Zexi Li, Wanru Zhao, Xinchi Qiu, et al. Photon: Federated I1m pre-training. arXiv preprint or 1i::2411.0200 2024.
35

---

Published in Transactions on Machine Learning Research (04/2026)
Jonathan A Scott, Hossein Zakerinia, and Christoph Lampert. Pefil: Personalized federated learning by learning to learn. In 12th International Conference on Learning Representations, 2024.
Soroosh Shahtalebi, Jean- Sha-Chhistoppee Gagnon-Audet, Touraj Laleh, Mojtaba Faramarzi, Kartik Ahuja, and Trina Rish. Sand-mask: An enhanced gradient masking strategy for the discovery of invariances in domain generalization. arXiv preprint or Xix:2:1066 02266, 2021.
Jiawei Shao, Fangzhao Wu, and Jun Zhang. Selective knowledge sharing for distillation without a good teacher. Nature Communications, 2024.
federated
Donald Shenaj, Eros Fani, Marco Toldo, Debora Caldarola, Antonio Tavera, Umberto Michicli, Marco Ciecone, Pietro Zinuttigh, and Barbara Capulo. Learning across domains and devices: Style-driven source-free domain adaptation in clustered federated learning. In WACV, 2023.
Neta Shoham, Tomer Avidor, Aviv Keren, Nadav Tsrael, Daniel Benditkis, Liron Mor- Mos-Yosef, and Itai Zeitak. Overcoming forgetting in federated learning non-iid data. arXiv preprint 3Xxx:910000770 2019a.
Neta Shoham, Tomer Avidor, Aviv Keren, et al. Overcoming forgetting in federated learning on non-iid data, arXi::1910.07796 2019b.
Elton F de S Soares, Emilio Vital Brazil, and Carlos Alberto V Campos. Enhancing federated averaging of self-supervised monocular depth estimators for autonomous vehicles with bayesian optimizatiinn Future Generation Computer Systems, 167:107752, 2025.
Milad Soltany, Farhad Pourpanah, Mahdiyar Molahasani, Michael Greenspan, and Ali Etemad. Feder- ated domain generalization with label smoothing and balanced decentralized training. arXio preprint 1.Xiv2412.114000 2024.
Congzheng Song, Filip Granqvist, and Kunal Talwar. Flair: Federated learning annotated image repository. Advances in Newral Information Processing Systems, 35:37792-37805, 2022.
Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012.
Lili Su, Jiaming Xu, and Pengkun Yang. A non-parametric view of fedavg and fedprox: Beyond stationary points. Journal of Machine Learning Reseavch, 24(203):1-44 -48, 2023.
Jingwei Sun, Ang Li, Binghui Wang, Huanrui Yang. Hai Li, and Yiran Chen. Soteria: Provable defense against privacy leakage in federated learning from representation perspective. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9311-9319, 2021.
Rui Sun, Yumin Zhang, Varun Ojha, Tejal Shah, Haoran Duan, Bo Wei, and Rajiv Ranjan. Exemplar- lcarning. arXiv preprint arxi 18926, 2024.
condensed federated
Ruizhe Sun, Bixiao Zeng, Yiqiang Chen, Xiaodong Yang, and Hanchao Yu. Hicrarchical sumi-super fed- erated learning method for dermatosis diagnosis. In 2025 IEEE Wireless Communications and Networking Conference (WCNC), pp. 1-6. IEEE, 2025.
Yuwei Sun, Ng Chong, and Hideya Ochiai. Feature distribution matching for federated domain generalization. In ACML, 2023.
Canh T Dinh, Nguyen Tran, and Josh Nguyen. Personalized federated learning with moreau envelopes. NewrIPS, 2020.
Yue Tan, Guodong Long, Lu Liu, Tianyi Zhou, Qinghua Lu, Jing Jiang, and Chengqi Zhang. Fedproto: Federated prototype learning across heterogeneous clients. In Proceedings of the AAAI conference artificial intelligence, volume 36, pp. 1832-8440, 2022.
Chris Xing Tian, Haoliang Li, Yufei Wang, et al. Privacy-preserving constrained domain generalization via gradient alignment. TKDE, 2023.
36

---

Published in Transactions on Machine Learning Research (04/2026)
Malte Tölle, Philipp Garthe, Clemens Scherer, Jan Moritz Seliger, Andreas Leha, Nina Krüger, Stefan Simm, Simon Martin, Sebastian Ebll, Halvar Kelm, et al. Real world federated learning with a knowledge distilled transformer for cardiac ct imaging, npj Digital Medicine, 8(1):88, 2025.
Stacey Truex, Ling Liu, Ka-Ho Chow, Mchmet Emre Gursoy, and Wenqi Wei. Ldp-fed: Federated learning with local differential privacy. In Proceedings of the third ACM international workshop on edge systems, analytics and networking, pp. 61 61-66, 2020.
Ye Lin Tun, Chu Myaet Thwal, Le Quang Huy, Minh NH Nguyen, and Choong Seon Hong- Lw-fedssl: Resource-efficient layer-wise federated self-supervised learning, arXiv preprint arXiv:2/01.11647, 2024.
Anastasiia Usmanova, François Portet, et al. A distillation-bseed approach integrating continual learning and federated learning for pervasive services. In Workshop in IJCAI, 2021.
Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pottern recognition, pp. 5018-5027, 2017.
Binghui Wang, Ang Li, Meng Pang, Hai Li, and Yiran Chen. Graphfl: A federated learning framework for semi-supervised node classification on graphs. In 2022 IEEE International Conference on Data Mining (ICDM), pp. 997. IEEE, 2022a.
Boxin Wang, Yibo Jacky Zhang, Yuan Cao, Bo Li, et al. Can public large language models help private cross-device federated learning? arXiv: 12132, 2023.
Lixu Wang, Shichao Xu, Xiao Wang, and Qi Zhu. Eavesdrop the composition proportion of training labels in federated learning. arXiv:1910. 1910.06044, 2019.
Lixu Wang, Shichao Xu, Xiao Wang, and Qi Zhu. Addressing class imbalance in federated learning. Tn AAAI, 2021.
Lixu Wang, Shichao Xu, Ruiqi Xu, Xiao Wang, and Qi Zhu. Non-transferable learning: A new approach for model ownership verification and applicability authorization. In ICLR, 2022b.
Lixu Wang, Chenxi Liu, Junfeng Guo, Qingqing Ye, Heng Huang, Haibo Hu, and Wei Dong. Federated continuous category discovery and learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2429-2439, 2025a.
Qian Wang, Siguang Chen, Meng Wu, and Xue Li. Digital twin-empowered federated incremental learning for non-iid privacy data. IEEE Transactions Mobile Computing, 2024a.
Rui Wang, Weiguo Huang, Mingkuan Shi, et al. Federated adversarial domain generalization network: A novel machinery fault diagnosis method with data privacy. Knowledge-Basee Systems, 2022c.
Tianjing Wang, Zhao Yang Dong, and Lingzhi Su. Blockchain-enabled federated transfer learning for anomaly detection of power lines. In 2024 IEEE Power & Energy Sociely General Meeling (PESGM), pp. 1-5. IEEE, 2024b.
Xinpeng Wang, Yongxin Guo, and Xiaoying Tang. Fedccrl: Federated domain generalization with cross-client representation learning, Xiee preprint 2rXiv:2410.11266 2024c.
Xuetao Wang, Qichang Zhao, and Jianxin Wang. Fedkd-cpi: Combining the federated knowledge distilla- tion technique to accomplish synergistic compound-protein interaction prediction. Methods, 234:275-28 2025b.
Yuanjic Wang, Zhanbo Feng, and Zhenyu Liao. Fodrf-adapt: Robust and commmication-cffiiinett federated domain adaptation via random features. In 2024 IEFE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), pp. 615 155-619. IEFE, 2024d.
37

---

Published in Transactions on Machine Learning Research (04/2026)
Ziqiao Wang, Cheng Long, and Yongyi Mao. Generalization in federated learning: A conditional mutual information framework. arXiv preprint arXie:2503.4000 2025c.
Yikang Wei and Yahong Han. Multi-source collaborative gradient discrepancy minimization for federated domain generalization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 15805-15813, 2024.
Yipan Wei, Yuchen Zou, Yapeng Li, and Bo Du Towards unified modeling in federated multi-task learning via subspace decoupling. arXiv preprint orXiv:2505. 24185, 2025.
Jie Wen, Zhixia Zhang, Yang Lan, Zhihua Cui, Jianghui Cai, and Wensheng Zhang. A survey on federated learning: challenges and applications. International Journal of Machine Learning and Cybernelics, 14(2): 513 535, 2023.
Feijie Wu, Zitao Li, Yaliang Li, Bolin Ding, and Jing Gao. Fedbiot: Lim local fine-tuning in federated learning without full model. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3355, 2024a.
Feng Wu, Siwei Feng, Yuanlu Chen, and Libang Zhao. Personalized federated dlass-ncrrmmentl learning through critical parameter transfer. In ICASSP 2025-2025 IEFE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1 5. IEEE, 2025a.
Feng Wu, Alysa Ziying Tan, Siwei Feng, Han Yu, Tao Deng, Libang Zhao, and Yuanlu Chen. Federated class-incremental learning via weighted aggregation and distillation. IEEE Internet of Things Journal, 2025b.
Guile Wu and Shaogang Gong. Collaborative optimization and aggregation for decentralized domain gener- alization and adaptation. In ICCV, 2021.
Jiaqi Wu, Simin Chen, Jing Tang, Yuzhe Yang, Yiming Chen, Lixu Wang, Song Lin, Zehua Wang, Wei Chen, and Zijian Tian. Fdpt: Foderated discrete prompt tuning for black-box visual-language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2461 2470, 2025c.
Jiaqi Wu, Jing Liu, Yang Liu, Lixu Wang, Zehua Wang, Wei Chen, Zijian Tian, Richard Yu, and Vie- tor Leung. survey on cloud-edge-terminn tege-terminal collaborative intelligence in aiot networks. arXiv preprint arXiv:2508.1880 2025d.
Jiaqi Wu, Shihao Zhang, Simin Chen, Lixu Wang, Zehua Wang, Wei Chen, Fangyuan He, Zijian Tian, F Richard Yu, and Victor CM Leung. Efficient detection framework adaptation for edge computing: A alug-and-play neural network toolbox enabling edge deployment. IEEE Transactions on Mobile Comput- ing, 2025e.
Jie Wu, Xuezhong Qian, and Wei Song. Federated learning based feature decoupling and domain adapta- tion. In 2024 5th International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT), pp. 424-427. IEEE, 2024b.
N Wu, L Yu, X Yang, KT Cheng, and Yan. Federated learning with imbalancod and agglomerated data distribution for medical image classification. arXio:2206. 13803, 2022.
Yebo Wu, Chunlin Tian, Jingguang Li, He Sun, Kahou Tam, Li Li, and Chengzhong Xu. A survey on federated fine-tuning of large language models. Xix preprint or arxiv:2503. 12016, 2025f.
Abudukelimu Wuerkaixi, Sen Cui, Jingfeng Zhang, Kunda Yan, Bo Han, Gang Niu, Lei Fang, Changshui Zhang, and Masashi Sugiyama. Accurate forgetting for heterogeneous federated continual learning. anXiv preprint ar Xix::2202 14205, 2025.
Tengxi Xia, Ju Ren, Wei Rao, Qin Zu, Wenjic Wang, Shuai Chen, and Yaoxue Zhang. Acrorec: an efficient on-device recommendation framework using federated self-supervised knowledge distillation. In IEEE INFOCOM 2024-IRRF Conference on Computer Communications, pp. 121-130 IEFF, 2024a.
38

---

Published in Transactions on Machine Learning Research (04/2026)
Yingjie Xia, Yuhan Chen, Yunxiao Zhao, Li Kuang, Xuejiao Liu, Ji Hu, and Zhiquan Liu. Fellm-dt: Enpow- ering federated continual learning with large language models for digital twin-based industrial iot. IEEE Internet of Things Journal, 2024b.
Chulin Xie, Keli Huang, Pin Yu Chen, and Bo Li. Dba Distributed backdoor attacks against. federated learning. In ICLR, 2020.
Zuobin Xiong, Wei Li, and Zhipeng Cai. Appro-fum: Approximate machine unlearning in federated setting, In 2024 33rd International Conference on Computer Communications and Networks (ICCCN), pp. 1-9. IEEE, 2024.
Qinwei Xu, Ruipeng Zhang, Ya Zhang, Yi-Yan Wu, and Yanfeng Wang. Federated adversarial domain hallucination for
domain generalization. IEEE TMM, 2023.
Shichao Xu, Lixu Wang, Yixuan Wang, and Qi Zhu. Weak adaptation learning: Addressing cross-domain data insulficiency with weak annotator. In ICCV, 2021.
Hao Yan and Yuhong Guo. Local and global flatness for federated domain generalization. In European Conference Computer Vision, pp. 71-87. Springer, 2024.
Rui Yan, Liangqiong Qu, Qingyue Wei, Shih-Cheng Huang, Liyue Shen, Daniel L Rubin, Lei Xing, and Yuyin Zhou. Label-efficient self-supervised federated learning for tackling data heterogencity in medical 2023.
imaging. IEEE Transactions on Medical Imaging, 42(7):1932-1)
Yuna Yan, Xin Zhang, Lixin Li, Wensheng Lin, Rui Li, Wenchi Cheng, and Zhu Han. Fssc: Federated learning of transformer neural networks for semantic image communication. arXiv preprint 7xxv:2407.21507, 2024.
Chen Yang, Meilu Zhu, Yifan Liu, and Yixuan Yuan. Fedpd: Federated open set recognition with parameter disentanglement. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4882 4891, 2023a.
Nan Yang, Xuanyu Chen, Charles Z Liu, Dong Yuan, Wei Bao, and Lizhen Cui. Fedmae: Federated self- supervised learning with one-block masked auto-encoder. arXiv preprint arXiv:2303.1 11339, 2023b.
Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and
applications. TIST, 2019.
Seunghan Yang, Sookcon Choi, et al. Client-agnostic learning and zero-shot adaptation for federated domain generalization, 2023c.
Xin Yang, Hao Yu, Xin Gao, Hao Wang, Junbo Zhang, and Tianrui Li. Foderated continual learning via knowledge fusion: survey. arXiv:2312.16/ 2023d.
Xin Yang, Hao Yu, Xin Gao, Hao Wang, Junbo Zhang, and Tianrui Li. Federated continual learning via knowledge fusion: A survey. IEEE Transactions on Knowledge and Data Engincering, 36(8):3832-388) -8500, 2024a.
Xiyuan Yang, Wenke Huang, and Mang Ye. Fedas: Bridging inconsistency in personalized federated learning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pottern Recognition, pp. 11986- 11995, 2024b.
Yuxin Yang, Qiang Lú, Yuan Hong, and Binghui Wang- Fedgmark: Certifiably robust watermarking for federated graph learning. Advances in Neural Information Processing Systems, 37:48971 1494997 2024c.
Yuxin Yang, Qiang Li, Jinyuan Jia, Yuan Hong, and Binghui Wang. Distributed backdoor attacks on foderated graph learning and certified defenses. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Securily, pp. 2829 2843, 2024d.
Yuxin Yang, Qiang Li, Chenfei Nic, Yuan Hong, and Binghui Wang. Breaking statc-of-thc-art poisoning defenses to federated learning: An optimization based attack framework. In Proceedings of the 33nd ACM International Conference on Information and Knowledge Management, pp. 2930 1930-2939,, 2939, 2024e.
39

---

Published in Transactions on Machine Learning Research (04/2026)
Yuxin Yang. Qiang Li, Yuede Ji, and Binghui Wang. A secret sharing- robust distributed backdoor attack to federated learning. ACM Transactions on Privacy and Securily, 28(2):1-19, 2025.
Chun-Han Yao, Boqing Gong, Hang Qi, Yin Cui, Yukun Zhu, and Ming- Hsuan Yang. Federated multi-targer domain adaptation. In WACV, 2022.
Xin Yao and Lifeng Sun. Continual local training for better initialization of federated models. In ICIP, 2020.
Yuan Yao, Lixu Wang, Jiaqi Wu, Jin Song, Simin Chen, Zehua Wang, Zijian Tian, Wei Chen, Huixia Li, and Xiaoxiao Li. Fedre: representation entangloment framework for modd-hheterogencous federated learning. CVPR, 2026.
Mang Ye, Xiuwen Fang, Bo Du, et al. Heterogeneous federated learning: State-of-the-art and research challenges. ACM Computing Surveys, 2023.
Rui Ye, Wenhao Wang, Jingyi Chai, Dihan Li, Zexi Li, Yinda, Xu, Yaxin Du, Yanfeng Wang, and Siheng Chen. Openfedllm: Training large language models on decentralized private data via federated Icarning. Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and dala mining, pp. 6137 6147,
2024.
Zongzhen Ye, Jun Wu, Xuesong He, and Weixiong Jiang. A gradient alignment. federated domain general- ization framework for rotating machinery fault diagnosis. IEEE Internet of Things Journal, 2025.
Min Kyoon Yoo and Yu Rang Park. Federated class incremental learning: A pseudo feature based approach without exemplars. In Proceedings of the Asian Conference on Computer Vision, pp. 488-498, 2024.
Jachong Yoon, Wonyong Jeong, Giwoong Lee, et al. Federated continual learning with weighted inter-client transfer. In TCML, 2021.
Young- Woo Youn, Hyeon-Soo Choi, Hong Nhung-Nguyen, Kong-Hwa Kim, et al. Self-supervised asyn- chronous federated learning for diagnosing partial discharge in gas-insulated switchgear. Eneryies, 18(12): 1-17, 2025.
Hao Yu, Xin Yang, Le Zhang, Hanlin Gu, Tianrui Li, Lixin Fan, and Qiang Yang, Addressing spprallltr data heterogeneity in federated continual learning via tail anchor. arXiv preprint arXiv:2412. 18355, 2024.
Shuyang Yu, Junyuan Hong, Haotao Wang, Zhangyang Wang, and Jiayu Zhou. Turning the curse of het-
erogeneity in federated learning into a blessing for out-of-distributiio detection. In ICLR, 2023.
Betul Yurdem, Murat Kuzlu, Mehmet Kemal Gullu, Ferhat Ozgur Catak, and Maliha Tabassum. Federated lcarning: Overview, strategics, applications, tools and future directions. Heliyon, 10(19), 2024.
Ghada Zamzmi, Kesavan Venkatesh, Brandon Nelson, Smriti Prathapan, Paul Yi, Berkman Sahiner, and Jana G Delfino. Out-of-distribution detection and radiological data monitoring using statistical process control. Journal of Imaging Informatics in Medicine, pp. 1 1-19, 2024.
Ling-Li Zeng, Zhipeng Fan, Jianpo Su, Min Gan, Limin Peng, Hui Shen, and Dewen Hu. Gradient matching federated domain adaptation for brain image classification. TNNLS, 2022.
Benteng Zhang, Yingchi Mao, Haowen Xu, Yihan Chen, Tasiu Muazu, Xiaoming He, and Jie Wu. Overcoming forgetting using adaptive federated learning for iiot devices with non-iid data. IEFE Internet of Things Journal, 2025a.
Binghui Zhang, Luis Mares De La Cruz, and Binghui Wang. Fedtilt: Towards multi-level ciirness-preserviing and robust federated learning. In 2025 IEEE Security and Privacy Workshops (SPW), pp. 160 166. IEEE, 2025b.
Duzhen Zhang, Yahan Yu, Chenxing Li, Jiahua Dong, and Dong Yu. Federated incremental named entity recognition. IEFE Transactions on Audio, Speech and Language Processing, 2025c.
40

---

Published in Transactions on Machine Learning Research (04/2026)
Fan Zhang, Huiying Liu, Qing Cai, Chun-Mei Feng, Binglu Wang, Shanshan Wang, Junyu Dong, and David Zhang. Federated cross-incrementa self-supervised learning for medical image segmentation. IEEE Transactions on Neural Networks and Learning Systems, 2024a.
Jie Zhang, Song Guo, Xiaosong Ma, Wenchao Xu, Qihua Zhou, Jingcai Guo, Zicong Hong, and Jun Shan. Model decomposition and reassembly for purified knowledge transfer personalized federated learning. IEEE Transoctions on Mobile Computing, 2024b.
Jingyuan Zhang, Yiyang Duan, Shuaicheng Niu, Yang Cao, and Wei Yang Bryan Lim. Enhancing fed- crated domain adaptation with multi-domain mototyye-based federated fine-tuning. arXiv preprint arXiv:2110077730 10.07788, 2024c.
Liling Zhang, Xinyu Lei, Yichun Shi, et al. Federated learning for iot devices with domain generalization. IEFE To Journal, 2023.
Rongyu Zhang, Yun Chen, Chenrui Wu, and Fangxin Wang. Multi-level personalized federated learning on heterogencous and long-tailed data. IEEE Transactions Mobile Computing, 2024d.
Wei Zhang and Xiang Li. Federated transfer learning for intelligent fault diagnostics using deop adversarial networks with data privacy. IEEE/ASME Transactions on Mechalronics, 2022.
Weidong Zhang, Dongshang Deng, and Lidong Wang. Fedscrap: Layer-wins wise personalized federated learning for scrap detection. Electromics, 13(3):527, 2024c.
Xiaochen Zhang, Chen Wang, Wei Zhou, Jiajia Xu, and Te Han. Trustworthy diagnostics with out-of- distribution detection: A novel max-consistency and min-similarity guided deep ensembles for uncertainty estimation. IEEE Internet of Things Journal, 2024f.
Xifan Zhang, Zhenyu Yan, and Guoliang Xing. Fedmod: Towards cross-modal training for heterogeneous federated learning systems. In Proceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems, pp. 828 829, 2024g.
Yabin Zhang, Hui Tang, Kui Jia, and Mingkui Tan. Domein-symmetric networks for adversarial domain adaptation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5031-5040, 2019.
Youshan Zhang and Brian D Davison. Impact of imagenet model selection on domain adaptation. In Proceedings of the IEEE/CVE winter conference on applications of computer vision workshops, pp. 173- 182, 2020.
Zhao Zhang, Yong Zhang, Hao Li, Shenbo Liu, Wei Chen, Zhigang Zhang, and Lijun Tang. Federated continual representation learning for evolutionary distributed intrusion detection in industrial internet of things. Engineering Applications of Arlificial Intellignnee, 135:108826, 2024h.
Chao Zhao, Enrico Zio, and Weiming Shen. Domain generalization for cross-domain fault diagnosis: An application-oriented perspective and a benchmark study. Reliability Engineering 8 System Safety, 245: 109964, 2024a.
Hongwei Zhao, Qiyuan Liu, Haoyun Sun, Liang Xu, Weishan Zhang, Yikang Zhao and Fei- Yue Wang. Community awareness personalized foderated learning for defect detection. IEEE Transactions Com- putational Social Systems, 2024b.
Jujia Zhao, Wenjie Wang. Chen Xu, See Kiong Ng, and Tat-Seng Chua. A federated framework for Ilm- based recommendation. In Findings of the Association for Computational Linguistios: NAACL 2025, pp. 2852 2865, 2025a.
Ke Zhao, Junchen Hu, Haidong Shao, et al. Federated multi-source domain adversarial adaptation framework for machinery fault. diagnosis with data privacy. Reliability Engineering and System Safely, 2023.
41

---

Published in Transactions on Machine Learning Research (04/2026)
Yang Zhao, Jiaxi Yang, Yiling Tao, Lixu Wang. Xiaoxiao Li, Dusit Niyato, and H Vincent Poor. Exploring federated unlearning: Review, comparison, and insights. IEEE Network, 2025b.
Zhengyi Zhong, Weidong Bao, Ji Wang, Jianguo Chen, Lingjuan Lyu, and Wei Yang Bryan Lim. Sacfi: Self-adaptive federated continual learning for resource-constrained end devices. IEFE Transactions Newral Networks and Learning Systems, 2025.
Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. Deep domain-adversari image genera- tion for domain generalisation. In Proceedings of the AAAI conference on artificial intelligence, volume 34, 2020.
pp.
Xiaokang Zhou, Qiuyue Yang, Xuzhe Zheng, Wei Liang, Kevin I-Kai Wang, Jianhua Ma, Yi Pan, and Qun Jin. Personalized federated learning with contr-contrastive learning for multi-mode user modeling in human-centric metaversc. IEEE Journal on Selected Areas in Communications, 42(4))817-831, 2024.
He Zhu, Jun Bai, Na Li, Xiaoxiao Li, Dianbo Liu, David L Buckeridge, and Yue Li. Fedweight: mitigating covariate shift of federated learning on electronic health records data through patients re-weighting, npj Digital Medicine, 8(1):1-19, 2025a.
Meilu Zhu, Qiushi Yang, Zhifan Gao, Yixuan Yuan, and Jun Liu. Fedbm: trained language models for heterogeneous federated learning. Medical Image Analysis, 102:103524, 2025b.
Stealing knowledge from pre-
Yinlin Zhu, Xunkai Li, Miao Hu, and Di Wu. Federated continual graph learning. arXiv:2411.18010, 2024.
arXiv preprint
Weiming Zhuang, Xin Gan, Xuesen Zhang, Yonggang Wen, et al. Federated unsupervised domain adaptation for face recognition. In JCME, 2022.
Weiming Zhuang, Chen Chen, and Lingjuan Lyu. When foundation model meets federated learning: Moti- vations, challenges, and future directions. arXiv:2306.1554 2023.
Giulio Zizzo, Ambrish Rawat, Naoise Holohan, al. Federated continual learning with differentially private data sharing. In Workshop NewrIPS, 2022.
Xiaojiang Zuo, Yaxin Luopan, Rui Han, Qinglong Zhang, Chi Harold Liu, Guoren Wang, and Lydia Chen. Fedvit: Federated continual learning vision transformer edge. Future Generation Computer Systems, 154:1-15, 2024.
42


Reviews:
[{"review_id": "gaMP9IRy3y", "reviewer": "Reviewer_xne5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "**Strengths**\n- The focus on FL system evolution and lifespan is timely and addresses a critical gap between rapid ML advancements and the high cost of training FL models from scratch.\n- The 4-category framework (Features, Tasks, Models, Algorithms) is logical and provides a clear structure for reviewing a large body of work.\n- The paper reviews recent literature (up to 2025) across multiple sub-fields within FL.\n- Future work on threats and incentives are valuable for guiding future research.\n\n\n\n**Improvement / Clarifications**\n\n1. Positioning & Contribution Clarity: The abstract and introduction clearly state the problem. However, the novelty claim as the \"first\" survey to address FL evolution could be strengthened by more sharply contrasting the proposed evolutionary framework with the \"static\" assumptions of prior surveys mentioned in Table 1.\n2. Structural Flow: Section 3 (New Features) is very detailed and well-organized. The transition to Section 7 (New Algorithms) feels somewhat abrupt and less developed compared to previous sections. It raises excellent questions but lacks a review of existing work that attempts such transitions. Expanding this section would improve balance.\n3. Technical Depth vs. Readability: The draft is highly technical and assumes significant reader expertise in FL. While appropriate for a survey, adding more intuitive explanations of core challenges (e.g., why non-IID data makes FCL so hard) in early sections would improve accessibility.\n4. Figures and Visualizations: The referenced figures (Fig 1, 2, 3, 4, 5) are crucial for understanding the taxonomies and workflows. Ensure these are clear, well-labeled, and directly referenced in the text. Their captions should be self-explanatory.\n5. Future Directions (Section 8): The discussion on Incentives is insightful, linking contribution evaluation to the knowledge taxonomy. Consider adding a brief diagram or table to summarize the proposed \"multi-dimensional model\" for quantifying contributions of features, tasks, and models. The Threats subsection rightly identifies new attack surfaces. It could briefly mention the tension between proactive detection of malicious new knowledge (e.g., poisoned features) and the privacy of client data.", "claims_and_evidence_yn": "No", "claims_explanation": "## Missing Important Relevant References\n\n**FL for Healthcare**\n\nPang et al. A Tri-Factor Adaptive Federated Learning Framework for Parkinson's Disease Diagnosis via Multi-Source Facial Expression Analysis. IEEE Journal of Biomedical and Health Informatics, 2025\n\nPang et al. Breaking Data Silos in Parkinson’s Disease Diagnosis: An Adaptive Federated Learning Approach for Privacy-Preserving Facial Expression Analysis. AAAI Conference on Artificial Intelligence, 2025 \n\n**FL Algorithm for Graph Data**\n\nWang et al. GraphFL: A Federated Learning Framework for Semi-Supervised Node Classification on Graphs. IEEE International Conference on Data Mining, 2022\n\nYang et al. FedGMark: Certifiably Robust Watermarking for Federated Graph Learning. Neural Information Processing Systems, 2024\n\n**Security of FL**\n\nYang et al. A Secret Sharing-Inspired Robust Distributed Backdoor Attack to Federated Learning. ACM Transactions on Privacy and Security, 2025\n\nZhang et al. FedTilt: Towards Multi-Level Fairness-Preserving and Robust Federated Learning. Deep Learning Security and Privacy Workshop, 2025\n\nYang et al. Distributed Backdoor Attacks on Federated Graph Learning and Certified Defenses. ACM Conference on Computer and Communications Security, 2024\n\nYang et al. Breaking State-of-the-Art Poisoning Defenses to Federated Learning: An Optimization-Based Attack Framework. ACM International Conference on Information and Knowledge Management, 2024\n\n**Privacy of FL**\n\nFeng et al. Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence. ACM Conference on Data and Application Security and Privacy, 2025\n\nArevalo et al. Task-Agnostic Privacy-Preserving Representation Learning for Federated Learning Against Attribute Inference Attacks. AAAI Conference on Artificial Intelligence, 2024\n\nSun et al. Soteria: Provable Defense against Privacy Leakage in Federated Learning from Representation Perspective. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021", "audience_yn": "Yes", "audience_explanation": "This survey paper systematically reviews the challenge of integrating new knowledge—such as new features, tasks, models, and algorithms—into existing Federated Learning (FL) systems to enable sustainable evolution and extended lifespan without discarding prior investments. \n\nThe paper positions itself as the first survey to examine FL literature through the lens of system evolution and lifespan extension due to new knowledge integration, differentiating it from prior surveys focused on static settings or isolated techniques.", "requested_changes": "See Improvement / Clarification", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "LWT09yG9wx", "reviewer": "Reviewer_4jP1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "### Summary\n\nThis paper surveys how deployed federated learning systems can **incorporate “new knowledge”** to extend their **lifespan**, organizing the literature into four sources of new knowledge: **new features, new tasks, new models, and new aggregation algorithms**, with sections on feature shift pipelines (FDG/FODD/FDA), task generalization/personalization, federated continual learning, model transitions (e.g., foundation models/LLMs), and security/incentives. \n\n### Strengths\n\n* **Timely and relevant** framing around evolving FL systems rather than static FL. \n* **Broad coverage** with structured taxonomies/figures that help navigation. \n* Includes discussion of **threats and incentives/unlearning**, which is often missing in purely algorithmic surveys. \n\n### Weaknesses\n\n* Currently reads more like a **well-organized catalog** than a survey that **creates new connections / highlights trends / sharpens open problems** (the bar stated in the TMLR survey guidance).\n* The claimed focus on **arrival timing/form** and **lifespan** is not yet made **operational** (few scenario axes or “what to use when” guidance). \n* Security is discussed, but not well integrated into the earlier taxonomy (missing “new-knowledge type × threat × defense tradeoff” synthesis).", "claims_and_evidence_yn": "Yes", "claims_explanation": "**Partially.** The submission provides a broad literature-backed narrative and many cited examples that plausibly support the high-level taxonomy (“new features / tasks / models / algorithms”) and the motivation that FL deployments must evolve over time. \n\nHowever, several *central* claims are **not supported with sufficiently clear or convincing evidence** in their current form:\n\n* **“Lifespan” as a core objective**: the paper introduces a lifespan notion (e.g., based on client count dropping to a threshold), but does not substantiate why this is the right operational definition or how it correlates with real system utility/service quality; nor does it provide evidence that surveyed methods extend this lifespan under consistent metrics. \n* **“Arrival timing/form of new knowledge” analysis**: the paper states this as a key contribution, but the evidence is mostly descriptive; it lacks explicit scenario axes, empirical comparisons, or a decision framework showing that timing/form materially changes which methods work. \n* **Connections and trends**: many sections list methods, but the paper rarely backs “trend” statements with aggregated evidence (e.g., comparative tables, consensus findings, or clearly stated open problems grounded in recurring limitations). \n\n**Overall rating for this question:** *Moderate support*. The survey is credible as a reference map, but the strongest “survey-style” claims (new synthesis, trends, and operational guidance) need clearer evidence and more explicit consolidation to be fully convincing.", "audience_yn": "Yes", "audience_explanation": "**Yes.** A meaningful portion of TMLR’s audience working on federated learning, distributed/edge ML, continual learning, personalization, and deployment robustness would likely find this survey useful—especially because it frames FL as an **evolving system** and tries to unify several subareas (feature shift/adaptation, new tasks, model transitions, and aggregation changes) under a single “new knowledge” lens. \n\nThat said, the *level* of interest will depend on revisions: the paper will be substantially more compelling to TMLR readers if it strengthens its **synthesis** (new cross-area connections, trend summaries, and clearly stated open problems) beyond method cataloging.", "requested_changes": "### Requested changes (high impact)\n\n1. Add 2–4 explicit **new cross-area connections** (design patterns) that tie the four categories together, each with takeaways.\n2. Make **timing/form** concrete via scenario axes (abrupt/gradual, sync/async, churn, labeled/unlabeled) + a decision table mapping scenarios → method families → failure modes/costs.\n3. Strengthen **benchmark/metrics synthesis** (which datasets map to which “new knowledge” settings; what common metrics miss).\n4. Integrate security with a compact matrix: **knowledge type × threat × typical defenses × tradeoffs**.", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "uklbHBmJhr", "reviewer": "Reviewer_cghH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a comprehensive survey on incorporating knowledge into federated learning frameworks, a natural and increasingly important area given that clients typically receive streaming data. The authors systematically categorize existing approaches from multiple perspectives, including tasks, features, models, and algorithms. In addition, the survey highlights open challenges and discusses potential directions for future research.\n\n**Strengths**\n- The paper is well-structured and easy to follow.\n- The paper covers a wide range of literature on incorporating new knowledge — from adaptation and generalization to personalization and continual learning — going beyond existing surveys that usually focus on only one aspect.\n- Given the growing interest in continual learning, the survey is timely.\n\n**Weaknesses**\n- The paper provides limited comparison and analysis across different methods, making it difficult to understand the current progress of the field and the stage it has reached.\n- The discussion of real-world feasibility is limited. While several real-world applications are mentioned, most cited works are still evaluated in relatively controlled research settings. Important real-world complexities, such as synchronization issues and system/data heterogeneity, are not sufficiently discussed.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper provides a well-organized categorization and, to the best of my knowledge, covers all major topics related to incorporating new knowledge. Each category is supported by a substantial body of relevant literature, which supports the claims made in the paper.", "audience_yn": "Yes", "audience_explanation": "Given the growing interest in continual learning across the ML community, as well as the streaming-data nature of federated learning, this paper is likely to be of interest to a broad audience.\n\nAlso, by covering a wide range of related areas, the paper can attract readers from different subfields and help them understand the current status of research in this space.", "requested_changes": "Building on the weaknesses mentioned above, I suggest the following improvements:\n\n- The authors could provide benchmark comparisons across representative methods in each category, or at least point to existing benchmark studies, to help readers better understand the relative strengths and limitations of different approaches.\n- A deeper discussion on the gap between research and practice would be valuable, particularly in the context of **continual learning–oriented federated learning**, rather than standard static FL settings. Discussing challenges *unique* to continual and streaming scenarios would further strengthen the paper. Related perspectives from prior work (e.g., Daly et al.; Kuo et al.) could serve as useful references.\n\n[1] Daly et al. Federated Learning in Practice: Reflections and Projections\n\n[2] Kuo et al. Research in Collaborative Learning Does Not Serve Cross-Silo Federated Learning in Practice", "broader_impact_concerns": "N/A", "additional_comments": "N/A", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)