Buffer-based Gradient Projection for Continual Federated Learning
Authors: Shenghong Dai
Jy-yong Sohn
Yicong Chen
S M Iftekharul Alam
Ravikumar Balakrishnan
Suman Banerjee
Nageen Himayat
Kangwook Lee
Venue: Accepted by TMLR ()

Abstract: Continual Federated Learning (CFL) is essential for enabling real-world applications where multiple decentralized clients adaptively learn from continuous data streams. A significant challenge in CFL is mitigating catastrophic forgetting, where models lose previously acquired knowledge when learning new information. Existing approaches often face difficulties due to the constraints of device storage capacities and the heterogeneous nature of data distributions among clients. While some CFL algorithms have addressed these challenges, they frequently rely on unrealistic assumptions about the availability of task boundaries (i.e., knowing when new tasks begin). To address these limitations, we introduce Fed-A-GEM, a federated adaptation of the A-GEM method, which employs a buffer-based gradient projection approach. Fed-A-GEM alleviates catastrophic forgetting by leveraging local buffer samples and aggregated buffer gradients, thus preserving knowledge across multiple clients. Our method is combined with existing CFL techniques, enhancing their performance in the CFL context. Our experiments on standard benchmarks show consistent performance improvements across diverse scenarios. For example, in a task-incremental learning scenario using the CIFAR-100 dataset, our method can increase the accuracy by up to 27%. Our code is available at https://github.com/shenghongdai/Fed-A-GEM.

Published in Transactions on Machine Learning Research (01/2025)
Buffer-based Gradient Projection for Continual Federated Learning
Shenghong Dai
sdai87Qwisc.edd
Deportment of Electrical and Computer Engineering University of Wisconsin-Madisn Madison
Jy-yong Sohn
1ysohu11008@yymsei..cco
Department of Applied Statistics
Yonsei University
Yicong Chen
ychen2229@wiser edu
Department of Electricol and Computer Engineering University of Misconsin-Madisonn
S M Iftekharul Alam Intel Labs
s. s.m. s.m. w.m.ifekharul..alo alam Dintel..miio..ooo
Ravikumar Balakrishnan Intel Labs
wwnkkummr..uulakiiiinnnnn..
Suman Banerjee
suman &ic wisc.....iic.uuuuu
Deportment of Computer Sciences University of Wisconsin-Madison
Nageen Himayat
mageen.himmayt&@intee Gintel. com
Intel Labs
Kangwook Lee
kangwrook.eee@wisee Swwisc..eddd edu
Department of Electrical and Computer Engineering University of Wisconsin-Matiion
Reviewed on OpenReview: https: // openreview. net/ forum? id= Xe5IcOii2
Abstract
Continual Federated Learning (CFL) essential for enabling real-world applications where multiple decentralized clients adaptively Iearn from continuous data streams. A significant challenge in CFL is mitigating catastrophic forgetting, where models lose previously acquired knowledge when learning new information. Existing approaches often face difficulties due to the constraints of device storage capacities and the heterogeneous nature of data distributions among clients. While some CFL algorithms have addressed these challenges, they frequently rely on unrealistic assumptions about the availability of task boundaries (i.e., knowing when new tasks begin). To address these limitations, we introduce Fed-A-GEM, foderated adaptation of the A-GEM method (Chaudhry al., 2019), which employs a buffer-based gradient projection approach. Fed-A-GEM alleviates catastrophic forgetting by leveraging local buffer samples and aggregated buffer gradients, thus preserving knowledge across multiple clients. Our method is combined with existing CFL techniques, enhancing their performance in the CFT, context. Our experiments on standard benchmarks show consistent. performance improvements across diverse scenarios. For example, in task-incremental lcarning scenario using the CIFAR-100 datasot, our method can increase the accuracy by up to 27%. Our code is available https: / //tpth/b.ttttt//////// com/ cttp://hennghoggg///eee------e-

---

Published in Transactions on Machine Learning Research (01/2025)
1 Introduction
Federated Learning (FL) is machine learning technique that facilitates collaborative model training among large number of users while keeping data decentralized for privacy and efficient commuiccation. In real-world applications, models trained via FL, need the flexibility to continuously adapt to new data streams without forgetting past knowledge. This is critical in a variety of sconarios, such as autonomous vehicles, which must adapt to changes in the surroundings like new buildings or vehicle types without losing proficiency in previously encountered contexts. These real-world considerations make it essential to integrate FL with continual learning (CL) (Shmelkov et al., 2017; Chaudhry et al., 2018; Thrun, 1995; Aljundi et al., 2017; Chen & Liu, 2018; Aljundi et al., 2018), thereby giving rise to the concept of Continual Federated Learning (CFL).
The biggest challenge in CFL, as in CL, is catostrophic forgetting, where the model gradually shifts its focus from old data to new data and unintentionally discards proviously acquired knowledge. Initial attempts to mitigate catastrophic forgetting in CFL incorporated existing CL solutions at each client of FL, such as storing previous task data a buffer (a storage area) and reusing them or penalizing the updates of weights that are crucial for preserving the knowledge from earlier tasks. However, recent works (Bakman al., 2024; Ma et al., 2022; Yoon et al., 2021) have observed that this naive approach cannot fully mitigate the problem due to two reasons: (i) small-scale devices participating in FL cannot store much of the previous tasks' data due to the limited buffer size, and (ii) while clients update the model to prevent forgetting based on their local data, the non-lID data distributions across clients cause aggregated updates to fail in preventing global catastrophic forgetting, as observed in previous works (Bakman et al., 2024).
To address these challenges, rescarchers have developed various CFL algorithms. For example, Foderated Weighted Inter-client Transfer (FedWeIT) (Yoon et al., 2021) minimizes the interferences by selectively transferring knowledge across clients, while Continual Federated Learning with Distillation (CFeD) (Ma et al., 2022) limits the buffer size and uses knowledge distillaion to retain old knowledge. Global-Local Forgetting Compensation (GLFC) (Dong et al., 2022) addresses the local and global forgetting through the gradient compensation and the relation distillation, and Federated Orthogonal Training (FOT) (Bakman et al., 2024) ensures updates for new tasks are orthogonal to old tasks to reduce the interference. Additionally, a data-free free approach (Babakniya et al., 2024) uses generative models to synthesize past data distributions.
While these algorithms tackle unique challenges of CFL, they still share crucial constraint: the need for explicit task boundaries. This means that the learners need to know when tasks change to effectively manage and update the model. Mitigating catastrophic forgetting in practical scenarios where task boundaries are absent throughout the training process, known as general continual learning (Buzzega et al., 2020), remains an important open question.
To address these challenges in CFL, we consider leveraging general continual learning methods, specifically A- A-GEM (Chaudhry et al., 2019), while also considering the existing constraints in FL. To this end, we propose method called Fed-A-GEM (illustrated in Figure 1), which involves two key components:
1. Global Buffer Gradients: Each client computes the local buffer gradient of the global model with respect to its own local butler data. These local buller gradients, serving as reference gradients for model updates, are then securely averaged by the server to obtain the aggregated global buffer gradient.
2. Local Gradient Projection: Each client updates its local model such that the direction for the model update does not conflict with aggregated global buffer gradient from the previous round of continual learning, ensuring that the information learned by all clients in previous rounds is maintained during every client's 's model updating process.
Our contributions: In this paper, we introducc a simple method for CFL, called Fed-A-GEM. This method utilizes the information learned from previous tasks across clients to elfectively mitigate the catastrophic forgetting without having access to task boundaries. Fed-A-GEM can be integrated with existing CFL techniques to enhance their performance. We conduct comprehensive experiments to demonstrate the effectiveness of Fed-A-GEM across various classification tasks in the image and text domains. Fed-A-GEM consistently improves the accuracy and reduces the forgetting across diverse benchmark datasets. We find
2

---

Published in Transactions on Machine Learning Research (01/2025)
Continuous data stream
Server
Aggregation
Client A
Server
Local model Aggregatzn modill
Aggragated butfer gradiant
Local butfer gradient
Client B
Client
Client
Client
O Client C
Model Buffer
Gredient projection
Streaming Data
Streaming
Streaming
(a)
(b)
Overview of Continual Federated Learning. (a) Challenge of Catastrophic Forgetting: Different driving scenarios encountered by clients in an autonomous vehicle network. Each client faces diverse, dynamic environments, causing vision detection models to forget previous knowledge when learning new tasks. This issue is aggravated by the lack of task boundarics, limited buffer sizes, and non-IID data distribution across clients. (b) System architecture of Fed-A-GEM, showing client-server interactions, local model buffers, and gradient projection mechanisms.
Figure 1:
that Fed-A-GEM achieves superior performance without, increasing the communication overhead between the server and the clients. Further, we evaluate the robustness of our method considering various buffer sizes, asynchronous environments, and different numbers of tasks and users. Our evaluation also includes an ablation study to examine the role of each key component in the proposed Fed-A-GEM method.
2
Related Work
2.1 Continual Learning (CL)
CL addressos the problem of learning multiple tasks consccutively using single model. Catastrophic forgetting (McCloskey & Cohen, 1989; Ratcliff, 1990; French, 1999), where classifier trained for a current task performs poorly on previous tasks, is a major challenge of CL. Existing approaches in CL can be categorized into regularization-based architecture-based,, and replay-based methods.
Regullriization-basdd methods Some CL methods add regularization term the loss used for the model update; they penalize the updates weights that are important for retaining knowledge from previous tasks. Elastic Weight Consolidation (EWC) (Kirkpatrick et al., 2017), Synaptic Intelligence (Zenke al., 2017), Riemannian Walk (Chaudhry et al., 2018) are methods within this category. EWC uses Fisher information matrix to evaluate the importance of parameters for previous tasks. Besides, Learning without Forgetting (Ti & Hoiem, 2017) leverages knowledge distillation to preserve outputs on previous tasks while learning the current task.
crsttectuee-baseee methods A class of CL methods assigns a subset of model parameters to each task, that different tasks are learned by different parameters. This class of methods is also known parameter isolation methods. Some methods including Progressive Neural Networks (Rusu al., 2016) and Dynamically Expandable Network (Yoon et al., 2018) use dynamic architectures where the architecture changes dynamically as the number of tasks increases. These methods have issues where the nunber of required parameters grows linearly with the number of tasks. To tackle this issue, fixed network are used in the recent methods including PackNet (Mallya Lazebnik, 2018), Hard Attention to the Task (Serra al., 2018) and PathNet (Fernando al., 2017). These methods maintain constant architecture regardless of the number of tasks. More recent ad advanceennts include Supermasks in Superposition (Wortsman et al., 2020),
3

---

Published in Transactions on Machine Learning Research (01/2025)
which uses supermask to determine which parameters are used for each task, and DualNet (Pham et al., 2021), which consists of a shared network and task-specific network.
Replay-based methods To avoid catastrophic forgetting, a class of CL methods employs a replay buffer to save a small portion of the data seen in previous tasks and reuses it in the training of subsequent tasks. One of the carly works this area is Experience Replay (Ratcliff, 1990; Robins, 1995), which uses a memory buffer to store and randomly sample past experiences during training to reinforce previously learned knowlodge. In more contemporary studies, Incremental Classifier and Representation Learning (iCaRL) (Rebulfi et al., 2017) stores exemplars of data from previous tasks and adds distillation loss for old exemplars to mitigate the forgetting issuc. Deep Generative Replay (Shin et al., 2017) retains the memories of the previous tasks by loading the synthetic data generated by GANs without replaying the actual data for the previous tasks. Gradient based Sample Selection (Aljundi et al., 2019) optimally selects data for replay buffer by maximizing the diversity of samples in terms of the gradient in the parameter space. Gradient Episodic Memory (GEM) (Lopez-Paz & Ranzato, 2017) and its variant Averaged- GEM (A-GEM) (Chaudhry et al., 2019) leverages episodic memory that stores part of seen samples for each task and use them to compute gradient constraints to prevent forgetting old knowlodge. Similarly, Orthogonal Gradient Descent (Farajtabar et al., 2020) stores gradients as opposed to actual data, providing a reference in projection. More recent work include Greedy Sampler and Dumb Learner (Prabhu et al., 2020), Bias Correction (Wu et al., 2019), and Contrastive Continual Learning (Cha et al., 2021). Despite its simplicity, replay-based tochniques have shown great performances on multiple beuchmarks (Mai et al., 2022; Parisi et al., 2019). Fed-A-GEM leverages this replay-based method, particularly A-GEM, to alleviate forgetting by reusing a portion of previous task data.
Theoretical efforts on CL Recent theoretical advances have deepened our understanding of CL. Bennani et al. (Bennani et al., 2020) establish generalization bounds for both Stochastic Gradient Descent (SGD) and Orthogonal Gradient Descent (OGD) by leveraging the Neural Tangent Kernel (NTK) regime. Their analysis demonstrates that OGD achieves tighter generalization bounds compared to SGD. Kim et al. (Kim et al., 2022) presents a theoretical framework for understanding and improving Class Incremental Learning (CIL). Their work decomposes CIL into two sub-problems: Within-task Prediction and Task-ID Prediction, and shows that Task-ID Prediction is related to Out-of-Distribution detection. Pentina and Lampert (Pentina & Lampert, 2014) bridge the gap between theory and practice in cont learning by proposing a PAC-Bayesian framework. Their work provides theoretical learning bounds the expected error in future tasks based on the average loss observed in previous tasks. Lee et al. (Lee et al., 2021) provide a rigorous analysis of the relationship between task similarity and two key phenomena in continual learning: forgetting and transfer.
2.2 General Continual Learning (GCL)
Existing CL. met often rely on explicit task boundaries to trigger crilical actions. For example, some regularization-baet methods store neural network responses (e.g., activations) at task boundaries; architecture- based methods update the model architecture after one task is finished; and some replay-based methods update the replay buffer at task boundaries to include samples from the completed task. However, in practical settings, task boundaries are not always clearly defined. This scenario, where sequential tasks are learned continuously without explicit knowledge of task boundaries, is referred as general contirual learning (Buzzega et al., 2020; Aljundi et al., 2019; Chaudhry et al., 2019). Dark Experience Replay (DER) (Buzzega et al., 2020) is a notable method in GCL, which retains the network's 's logits of previous tasks to mitigate forgetting without relying on explicit task boundaries. Gradient Sample Selection (GSS) (Aljundi et al., 2019) deals with the general continual learning and optimizes the diversity of samples in the replay bulfer based on gradient feature. A-GEM (Chaudhry et al., 2019) can operate without task boundaries by utilizing a reservoir sampling strategy for its memory buffer. However, most existing works have not thoroughly investigated general continual learning in the context of federated learning. Fed-A-GEM addresses this gap by specifically focusing on the integration and application of general continual learning principles in federated learning.
4

---

Published in Transactions on Machine Learning Research (01/2025)
2.3 Federated Learning (FL)
FI. enables collaborative training of a model with improved data privacy (McMahan et al., 2017; Kairouz et al., 2021; Lim et al., 2020). FedAvg (McMahan et al., 2017) is a widely used FL algorithm, which averages locally trained models to create global model. Subsequent works have focused on various challenges in FL, such as improving communication efficiency (Konecný et al., 2016), handling data heterogeneity (Zhao el al., 2018; Karimireddy et al., 2020; Li et al., 2020), and implementing privacy-preserving techniques, including differential privacy (Geyer et al., 2017) and secure aggregation (Bonawitz et al., 2017). Differential privacy aims to ensure that individual data cannot be inferred from the model outputs, while secure aggregation ensures that the individual data remains private when clients share gradients or model updates. FL has been successfully applied in various domains, including autonomous vehicle (Elbir et al., 2022; Dai et al., 2023), healthcare (Chen et al., 2020) and others (Shaheen et al., 2022). However, most existing methods (Li el. al., 2020; Shoham et al., 2019; Karimireddy et al., 2020; Li et al., 2019; Mohri et al., 2019) assume static data distribution over time, ignoring temporal dynamics. Our paper addresses this gap by considering temporal dynamics and focusing on a continual federated learning setup.
2.4 Continual Federated Learning (CFL)
CFL lackles the problem of learning multiple consecutive lasks in the FL setup. FedProx (Li et al., 2020) and Federated Curvature (FedCurv) (Shoham et al., 2019) aim to preserve previously learned tasks, while FelWeIT (Yoon et al., 2021) and NetTailor (Venkatesha et al., 2022) prevent interference between irrelevant tasks. CFeD (Ma et al., 2022) use surogate datasets, which are auxiliary datasets approximating past data from previous tasks, to perform knowledge distillation and thus mitigate forgetting. Other methods, including FedCL (Yao & Sun, 2020) and GLFC (Dong et al., 2022), utilize importance weights or class-aware techniques to distill the knowledge from previous lasks. Mimicking Federated Continual Learning (Babakniya el al., 2024) employs generative models to create synthetic data for past distributions. FOT (Bakman et al., 2024) is the state-of-tho-art CFL method that aggregates the activation representations of local models and projects the global gradient accordingly on the server side. However, existing CFL met face several limitations. Some approaches lack scalability the number of tasks increases (Yoon et al., 2021; Venkatesha al., 2022). Some methods require surrogate datasets (Ma et al,, 2022), which can be difficult and resource-intensive to generate or collect. Furthermore, certain methods incur substantial communication overhead (Yao & Sun, 2020), which can be impractical in federated seltings with limited bandwidth. Moreover, many of these methods depend on explicit task boundaries, making them less applicable to general continual learning settings where tasks are not clearly defined (Ma et al., 2022; Dong et al., 2022; Babakniya et al., 2024; Bakman et al., 2024). Our Fed-A-GEM does not rely on explicit task boundaries while maintaining marginal communication overhead.
3 Preliminaries
CL. focuses on finding a single classifier f (parameterized by w) that performs well T tasks. At time slot. E [T', the classifier will only have access to the data for task t. The notation [N] == (1,...., .N } is used for positive integer N. The feature-label pair (Iz, Vr) of the samples for task are drawn from an unknown distribution Dr. The goal of CL is to solve the following optimization problem:
min B(r.n)-D [e (ye, (xe: w))l
(1)
where é is the loss function, and f(x; w) is the output of classifier f with parameter w, for inputs xy- In practical scenarios, there may be insufficient storage to save all the data seen for the previous tasks. To address this, replay-based methods employ a memory buffer M that selectively stores a subset of data, which acts as a proxy to summarize past samples and refine the model updates.
Some methods such as DER (Buzzega et al., 2020) use regularization techniques to find the model parameter w that minimizes the loss with respect to the local replay buffer M well as current samples. For given
5

---

Published in Transactions on Machine Learning Research (01/2025)
regularization coefficient y, this optimization problem for CL with replay buffers at time [T] is:
min B- le (yr, fZrr w))] + y B (y6, f w))l,
(2)
where B is a uniform distribution over the samples in buffer M, and (æs, Uh) are sampled from B. Other methods, such as A-GEM (Chaudhry et al., 2019), introduce constraint to ensure that the average loss for the data in buffer M does not increase. Given the model wh-1 trained on previous tasks, the constrained optimization problem at time T E [2]] for A-GEM is represented as:
min B(
[f (Nr, (]);
(3)
s.L. B( [e (16, (r); S - (e (yh, (166
An approximate solution to Eq. 3 can be found as follows. Specifically, A-GEM promotes the aligmment of the gradient with respect to the current batch of data (Iz, U-) and that for the buffer data (xb. yb) sampled from the distribution B. Thus, A-GEM, a proxy of the problem Eq. 3 is written as:
min
1911:
2
(4)
s.t. g gref 2 0
where g = V E Bis, 2h ..) In, [e(vr; f(x;; represents the gradient of the loss with respect to the current batch, Jref = V Ee 10s [e,,, (1s; wy)1))] is the gradient of the loss with respect to the buffer data, and 9 is the projected gradient we aim to find. The gradient g obtained from solving Eq. 4 is then used to update the model.
For the continual federated learning setup where the data is owned by K clients, we use the superscript. k E [K] to denote each client, i.e., client k samples the data from Dj at time and employs a local replay buffer M In the case of using FedAvg (MeMahan et al., 2017), each round of the CFL is operated as follows. First, each client & E [K] performs local updates with D with the assistance of replay buffer M Second, once the local training is completed, each client sends the model updates to the central server. Finally, the central server aggregates the model updates and transmits them back to clients.
4 Fed-A-GEM
We propose Fed-A-GEM, federated adaptation of the A-GEM method (Chaudhry et al., 2019). Note that Fed-A-GEM is designed to be compatible with various CFL techniques, significantly enhancing their performance in the CFL context. Our approach draws inspiration from A-GEM, which projects the gradient with respect to its own historical data. Building upon this idea, we utilize the global buffer gradient, which is the average buffer gradient across all clients, as a reference to project the local gradient. This allows us to take advantage of the collective experience of multiple clients and mitigate the risk of forgetting the previously Icarned knowledge in FL scenarios. While reference gradient techniques have been used in Byzantine FL (Cao et al., 2020; Prakash et al., 2020) to defend against malicious updates, we leverage reference gradients to preserve knowledge of old tasks in federated learning. As replay-based method, Fed-A-GEM maintains local butfer each client, which is memory buffer storing subset of data sampled from old tasks. The local buffer at client k is denoted by M As the continuons data is loaded to the client, it keeps updating the buffer so that M becomes a good representative of old tasks.
Algorithm provides the overview of our method in the CFL setup, including the process of sharing information (model and buffer gradient) between the server and each client. For each new round E [R], where R is the total number of communication rounds per task, the server first aggregates the local models w from client k [K), getting a global model Afterwards, the server aggregates the local buffer gradient g which is the gradient compuled on the global model with respect to the local buffer M³, from client k E [K] to obtain a global buffer gradient gref. It is worth noting that the term "aggregation" in this context refers to the averaging of locally computed values across all clients. Such aggregation can be securely performed by the central server using secure aggregation, which is denoted as "SecAgg" in Algorithm 1.
6

---

Published in Transactions on Machine Learning Research (01/2025)
Note that here have two functions used at the client side, ClientUpdate and ComputeBufferGrad, which are given in Algo- rithms and 3, respectively. ClientUpdate shows how client k updates its local model for task The client first loads the global model w and the global buffer gradient fref which are received from the server in the previous round. It also loads the local buffer M storing a subset of samples for previous tasks, and the data DI for the current task. For each batch ((z4,,)) in Dj, the client computes the batch gradient for the model w. The client then compares the direction of with the direction of the global buffer gradient gref received from the server. When the angle between and Jref is greater than 90, implics that using the direction of g as a reference for gradient descent may improve performance task, but at the cost of degrading performance on previous tasks. To retain the knowledge on the previous tasks, we do the following: whenever g and Jref have a negative inner product, we project the gradient g based on the global buffer gradient (which can be considered as a reference) gref and remove this component from g- The adjusted gradient ÿ is defined as:
Algorithm FedAvg ServerUpdate with Fed-A-GEM
Initialize random w", MM = { for all k, Iref None for each task = 1 to T do for each communication r= to R do
w +- ClientUpdate( w grot), Vk: w + SecAgg ([")
k-=1)
Peet ComputeBuff (w, Ak
Jxef SecAgg (f9/et) (k
end for
end for
Return the final global model w
Algorithm ClientUpdate(t, w, gret) at client k
Input: Task index i, model w, global buffer gradient gref, batch size 3
Load the dataset Dt, local buffer M Initialize 0 at the first lask for each batch {<x;, v:)] in D do
the current
g = V. w 1/2 Ei= (yr, f(x; g g proje 9 10 9 S 0)
w + w - ag for some learning rate a {(x,, un) m))
DeservoirSamplinggMM
n - n + end for Return W
g = proig 19 S 0),
(5)
Algorithm ComputeBufferGraauu. MM
where proj tion of onto 9ref, and 19 0) is an indicator function that ensures the adjustment is only ap- plied when g and gref are in conflicting directions. This projection provides the solution to the con- strained optimization problem in Eq. 4, following the idea suggested in A-GEM (Chaudhry et al., 2019). As illustrated in Fig. 2, this projection helps prevent the model updates along the direction that is harming the performance on previous tasks.
Input: global model w, local buffer Mk (I1, 812 ..) ... (Imms Vmn) f- random samples from M 9 Ei V. [e(ye, (x:)
g- -i 9ref gref represents the projec- Iusttet
Return
After gradient projection, the client updates its local model w by applying the gradient descent step with the updated gradient ÿ. Finally, the client updates the contents of the buffer M by using the reservoir sam- pling (Vitter, 1985) written in Algorithm 4. Reservoir sampling selects a random sample of |M** elements from a local input stream, while ensuring that each element has equal probability of being included in the sample. One of the advantages of this method is that it does not require any prior knowledge of the data stream. Once the updated local models {w*) ) are securely aggregated on the server using secure ag- gregation, the server transmits the updated global model back each client. Then, cach client i computes the local buffer gradient (i.e., the gradient of the model with respect to the samples in the local buffer M*) as shown in Algorithm 3.
g
grof
grojg
Figure 2: Illustration of the gradient projection in Eq. 5. If the angle between the gradient update g and global buffer gradient (considered as reference) Pret is larger than 90º, we project g to minimize the interference and merely update along the directions of ğ that is orthogonal to Gref-
7

---

Published in Transactions on Machine Learning Research (01/2025)
After each client computes the local buffer gradient g the server allows the use of secure aggregation to combine these local bulfer gradients and updale the global buffer gradient Gret- This compatibility with secure aggregation enhances the privacy safeguards of our proposed approach, effectively minimizing the risk of data leakage from individual clients. The aforementioned process takes place between each commuicattio and is repeated R times within each task. After traversing T lasks, the final global model w is obtained, as Note that the pscudocode describes the FedAArg-eed-AGGe A-MM process. Fed-A-GEM can be combined with various CFL methods. Specifically, the integration involves using the aggregated bulfer additional constraint during the local updates in these CFL methods.
shown in Algorithm
gradient computed in Fed-A-GEM The detailed examples are given in Appendix E.
5 Experiments
In this section, we assess the efficacy of our method, Fed-A-GEM, in combination with various CFL baselines, under non-lID data distribution across clients. To evaluate these methods, we conduct experiments on image classification tasks for benchmark datasots including rotated-MNISST (Lopez-Paz & Ranzato, 2017), permuteddMMNIST (Goodfellow el. al., 2013), sequentiall-CIFARRO,, and sequential-CIFARROO (Lopez-Paz & Ranzato, 2017) datasets, as well as text classification task (Mehta et al., 2023) on sequential- sequential-YabooQQa YahooQA dataset (Zhang et al., 2015). We also explore Fed-A-GEM on an object detection task on streaming CARLA dataset (Dai et al., 2023: Dosovitskly et al., 2017) in Appendix D. We have further explored an ablation study, which examines each element of our approach, highlighting their roles in performance enhancement. All experiments were conducted on a Linux workstation equipped with NVIDIA GeForce RTX 2080Ti GPUs and averagod across five runs, each using different sced. For further details and additional rcsults, please refer to Appendix C.
5.1 Image Classification
5.1.1
Settings
Evaluation Datasets. We evaluate our approach on three CL scenarios: domain incremental learning (domain-IL), class incremental learning (class-IL), and task incremental learning (task-IL). We explain these three types of incremental learning (IL) settings with examples in Appendix A. For domain-IL, the data distribution of each class changes across different tasks. We use the rotated-MNIST (R-MNIST) and permuted-MNIST (P-MNIST) datasets for domain-IL, where cach task rotates the training digits by random angle or applies random permutation. We create T = 10 tasks for domain-IL experiments. For class- IL and task-IL, we use the sequential- CIFARRO (S-CIFAR10) and sequential- requential-IFFARIOO0 (S-CIFAR100) datasets, which partition the set of classes into disjoint subsets and treat each subset as a separate task. For instance, in our image classification experiments for class-IL and lask-IL, we divide the CIFAR-100 dataset (with C 100 classes) into T 10 subsets, each of which contains the samples for C/T = 10 classes. Fach task E [T'] is defined the classification of images from each subset E [Z']. The difference between class-IL and task-IL is that in the task-IL setup, we assume the task identity t is given at inference time. That is, the model ʃ predicts among the C/T = 10 classes corresponding to task 1. Sequential-CTFARIIO is defined by splitting the CIFAR-10 dataset into T = 5 tasks, with each task having two unique classes.
In the FL setup, we assume that the data distribution is non-IID across different clients. Once we define the data for each task, we assign the data to K clients in a non-ITD manner. For the rotAted-MNIST or permuted-MNISST datasct, cach client receives samples for two MNIST digits. To create a sequential- CIFARIO or requential-CITARRII distribution (Hsu et al., 2019). Specifically.we draw q q Dir(ap), where p represents prior class distribution over N classes, and a is concentration parameter that controls the degree of heterogeneity among clients. For our experiments, we use a = 0.3, which provides a moderate level of heterogeneity. Communications of models and buffer gradients occur whenever all clients complete the local training for E epochs.
dataset, we partition the dataset among multiple clients using a Dirichlet
Architectures and Hyperparameters. For the votated-MNIs, and permuted-MNISST dataset, we use a simple CNN architecture (McMahan et al., 2017). and split the dataset into K 10 clients. Each client performs local training for F epoch between communications, and we set the number of communication
8

---

Published in Transactions on Machine Learning Research (01/2025)
rounds as R 20 for each task. For the sequential-CIFARR10 and sequential-CIAARIO datasets, use a 18 architecture, and divide the dataset into K = 10 clients. Each client trains for E = 5 epochs between communications, and uses R- = 20 rounds of communication for each task. During local training, Stochastic Gradient Descent (SGD) is employed with a learning rate of 0.01 for MNIST and 0.1 for CIFAR datasets. Unless otherwise noted, the buffer size is set to = 200, a negligible storage for edge devices.
Baselines. We compare the performance of Fed-A-GEM with three types of baselines: 1) FL, the foundational FodAvg which trains only on the current task without considering performance on previous tasks; 2) FL+ CL, which is FedAvg with continual learning solutions applied to clients; and 3) CFL, which represents the existing Continual Federated Learning methods.
CL methods we lested include A-GEM (Chaudhry et al., 2019), which aligns model gradients for buffer approximated
and incoming data; GPM (Saha et al., 2020), using the network pearesetaation/activaaat by top singular vectors as the old tasks' reference vector; DER (Buzzega et al., 2020), utilizing network output logits for past experience distillation; iCaRL (Rebulli et al., 2017), which stores small nunber of representative examples for each class and counters representation deterioration with a delf-distillation loss term; and L2P (Wang et al., 2022), shate-of-the-art approach that instructs pre-trained models to sequentially learn tasks using prompt pool memory spaces.
CFL methods we tested are FedCurv (Shoham et al., 2019), which avoids updating past task-critical weights; FedProx (Li et al., 2020), introducing a proximal weight for global model aligument; CFeD (Ma et al., 2022), using surrogate dataset-based knowledge distillation; and GLFC (Dong et al., 2022), a tripartite method to counteract the forgetting issue: 1) clients adjust the gradients for both old and new classes to ensure balanced updates, 2) clients save the prior model, compute the KL divergence loss between the new and old model outputs (from the last layer), and 3) proxy server is used to collect perturbed gradient samples from the clients, which are used to select the best previous model. Similar to Fed-A-GEM, FOT (Bakman et al., 2024) projects the gradients on the subspace specified by previous tasks. FedWeIT (Yoon et al., 2021) may not serve as a suitable benchmark given its focus on personalized FL without a global model accuracy to contrast with.
Note that CFeD, GLFC, GPM, iCaRL, and FOT require task boundaries during training. These methods exploit task changes to snapshot the network, where iCaRL further relies on these task boundaries for memory buffer updates. Details of hyperparameters used for the baseline methods are given in Appendix C.7.
Performance Metrics. We assess the performance of the global model on the union of the test data for all previous tasks. The average accuracy (measured after training on task t) is denoted as Acc, = E 0r where a,,s is accuracy of the global model evaluated on task i after training up to task t. Additionally, we measure a performance metric called forgetting, which defined as the difference between the best accuracy obtained throughout the training and the current accuracy (Chaudhry al., 2018). This metric measures the model's ability to retain knowledge of previous tasks while learning new ones. The average forgetting after seeing t tasks is defined as: Fgt,- E --,
(a..1 | a.,). We also compute the Backward transfer (BWT) and Forward transfer (FWT) metrics (Lopez-Paz & Ranzato, 2017), details of which is given in Appendix C.5.
max
5.1.2 Overall Results
Table presents the average accuracy Accr of various methods on image classification benchmark datasets measured upon completion of the final task T. For each setting, we compare the performance of an existing method with/witd without Fed-A-GEM. We observe that the proposed methods (represented by "w/ Fed-A- GEM") improves the base methods ("w/o Fed-A-GEM") in most cases, seen from the upward arrows indicating performance improvements in Table 1. Similarly, Table 14 in Appendix C.1 compares the forgetting performance Fgtr, which shows that combining existing methods with Fed-A-GEM reduces the amount of forgetting. Moreover, the analysis in Appendix C.2 demonstrates that Fed-A-GEM consistently outperforms other baselines over time and across tasks. In this Table 1, FL + A-GEM projects the gradient along the local buffer gradient. FL + Fod-A-GEM, on the other hand, projects the gradient along the aggregated buffer gradient. Finally, FL + A-GEM + A-GEM combines both strategies by first projecting the gradient along the local buffer gradient and then along the aggregated buffer gradient.
9

---

Published in Transactions on Machine Learning Research (01/2025)
Table I: Average accuracy ACer (%) on standard benchmark datasets where 4_" indicates experiments we were unable to run, because of compatibility issues (e.g. GLFC and iCaRL in Domain-IL) or the absence of a surrogate dataset (e.g. CFeD on MNIST). The results, averaged over 5 random seeds, demonstrate the benefits of our proposed method in combination with baselines. A buffer of 200 is utilized whenever mothods require it. Note that FL-L2P needs an additional pretrained VIT.
rotated rotated-MNIST (Domain-1L)
sequeutial- CIFARIO (Class
sequentialCIFFAI0 CIFARIO (Tesk
Method
w/o Fed-A-GEM
w/ Fod-A-GEM
w/o Fed-A-GEM
w/ Fod-A.GEM
w/o Fed-A-GEM
w/ Fed-A-GEM
FL (McMalon et al., 2017)
68.02*8.1
79.46=44
(111.44)
17.44415
18.0240. (10.58)
70,56+40
80,83 (*10.25)
FLMA-GEM (Chandhay et aL. 2019)
68.3
74.74=23 (16.40)
17,8240,9
19.4 (11.62)
77.14
63.16+1 (*6.02)
FL+ GPM (Scha et al., 2020)
74.42-644
81.12-25
(16.70)
17.5940..
20.9541 (13.36)
74.50436
81,93103 ('7.43)
FL+DER (Buzeega el sl., 2020)
57.73496
81.33=9 (128.60)
18, 18.41337
30.9 (112.50)
69. 31
77.99 77.9928 (78.65)
FL++CaRL (Rebafii et al., 2017)
26.51 18.5ttis
33.99430 (15.98)
80,85
80,09 (J0.76)
F12 (Wang al., 2022)
80.00
86.05-977 (14.15)
28.61
81.8gana (153.25)
08.49
58.63483 (70.14)
RelCurr (Shoham at al., 2019)
69.21 286
80.53-4-3 (112.32)
17.364007
17.86488 (10.50)
67.77434
81,28 (713.51)
FedProx (Li al., 20200
67.7y482
78.74=4.1 (110.955
16.6742.7
17,97300.8 (1130)
09677556
81,23+18 (*11.66)
CFcD (Ma al., 2022)
16.30
24.07 (17.77)
77.35
79.30 (*1.95)
GLPC (Dong et al., 2022)
41.4211.
41.6 41.81 (10.19)
81.84
62.87 (*1.03)
FOT (Bakman et al., 2024)
70.02472
84.14
(114.12)
73.73+19
76.94477 (*3.21)
permuted MIIIT (Domain-IL)
requential-CFFARLOO (Closs-12)
sequential-CFFARR000 (Task-IL)
Method FL FLEA-GEM
w/o Fod-A-GEM
w/ Fod-A-GEM
w/o Fed-A-GEM
w/ Fed-A-GEM
w/o Fed-A-GEM
w/ Fed-A-GEM
25.92#21
34.23=27
(18.31)
8,76
17.05+ (18.32)
47.74
74.71
(*26.97)
30.43-14
39.09-33 (15.66)
8.90
19.5041 (110.63)
63.84408
74.84185 (*11.00)
FL+GPM
31,92494
22.38=35 (110.46)
8,1820
13.3241 (15.14)
51.18+14
65.51463 (*11.08
FL-DER
19.79
38. 38.81=20
(119.02)
13.324416
22 22.9640.0 (19.61)
57.71
65.57 (*7.86)
FL+HCARL
21.7641.
27.41411 (15.68)
69.91 297
72.83465 (*2.92)
FL.+12P
68:98 144
60:15-033
(12.17)
23.124
46.16.004 (123.04)
94.46
94.91 10.2 (70.45)
FedCurv
26.00244
35.21-53 (19.21)
16477109 (17.76)
49.14
74.64 (*25.49)
FedPrux
25.92424
35.60=4.7
(19.68)
8,75202
16 16.92 (f8.17)
47.06*52
73.95+08 (-26.89)
CF&D
13.76
26.66 (112.9)
$1.4
72.20+0. (*20.79)
CLFC
18.1840.4
13.4730.7 (10.29)
40.78498
49.20=i1 (J0.58)
FOT
26.08+24
29.3 (13.28)
68.51+16
71.66 (*5.52)
Remarkably, even a simple integration of the basic baseline, FL, with Fed-A-GEM surpassed the performance of most FL+CL and CFL baselines. For instance, in the -TTuential-CFFARR000 experiment, FT. with Fed-A-GEM (17.08% class-IL, 74.71% task-IL) outperformed majority of the basclines. Specifically, it exceeds the performance of the two advanced CFL baselines: GLFC (13.18% class-IL, 49.78% task-IL) and CFeD (13.76% class-IL, 51.41% task-IL). This underscores the substantial capability of our method in the CFL setting. Importantly, Fed-A-GEM can achieve competitive performance even without utilizing information about task boundaries, unlike CF&D, GLFC, GPM, iCaRL, and FOT.
We also note that the FL+L2P method consistently exhibited the highest accuracy, largely due to the utilization of a pretrained Vision Transformer (VIT) (Dosovitskiy et al., 2020; Zhang ct al., 2022), which helps mitigate the catastrophic forgetting. This is why we wrote the numbers in gray with caveat in the caption. Yet, our approach still managed to achieve significant performance augmentation on top of it. Moreover, we observe that Fed-A-GEM++2P is most effective in the challenging class-IL cases and least effective in task-IL.
We also compare Fe--Fed-A-GEM with the state-of-the-ar method, FOT (Bakman et al., 2024). In FOT, at the end of each task, the server aggregates the activations of each local model (computed for local data points) and computes the subspace spanned by the aggregated activations. This subspace is used during the global model update process; the gradient is updated in the direction that is orthogonal to the subspace. While both FOT and Fed-A-GEM project the gradients main differences. First, the subspace is defined in a different manner. FOT relics on the representations of local model activations. Fed-A-GEM, on the other hand, relies on the gradient of model computed on its local buffer data. Second, FOT projects the gradient computed at the server side, while Fed-A-GEM projects the gradient computed at each client. We observe that our method, FL.+Fed-A FredAA-GE A-GEM, consistently outporforms FL or FOT across datasets. Additionally, combining Ped-A-GEM with FOT yields superior performance compared to FOT alone, demonstrating the effectiveness of integrating our approach with existing techniques.
the subspace specified by previous tasks, they have two
10

---

Published in Transactions on Machine Learning Research (01/2025)
We further explored weighted variant of Fed-A-GEM inspired by FLTrust (Cao et al., 2020), where local updates from clients are weighted according to their similarity with the reference gradient. In this approach, when aggregating client models, we compute a cosine similarity score between each client's local update and the global reference gradient, using these scores as weights during aggregation. Our experiments showed that this weighted aggregation approach did not yield significant. improvements over our original method across different datasots and scttings. For example, on the soquential-CTFARII dataset under the class-IL setting, the weighted variant achieved & marginal improvement of 0.22%. While different weighting schemes and similarity metrics could potentially be explored further, these initial results suggest that our original aggregation approach already captures the essential benefits of gradient-based knowledge preservation.
Table 2: Effect of weighted aggregation on accuracy (%)
R-MNIST P-MNIST
S-CIFAR10
S-CIFAR100
Method
IL Domain-ll,
Class-IL
Task-IL ILL
Class-IL
Task-IL
Fed-A-GEM
79.16+4.1
31.23+27
18.02
80.83+20
17.08=18
71.71=003
Fed-A-GEM w/ weighted Agg
77.63
34.28:44
18.24
80.42
13.46=20.6
74.74
5.1.3 Effect of Buffer Size.
Table 3 reports the performances baseline CL methods (FL+A-GEM and FL+DER) with/without Fed-A- GEM for different buffer sizes, ranging from 200 to 5120. For most of datasets and IL settings, increasing the buffer size further improves the advantage of applying Fed-A-GEM, by providing more data for replay and mitigating forgetting. However, finite buffer cannot maintain the entire history of data. In Fig. 3 we reported the effect of buffer size on the accuracy of old tasks. At the end of each task, measured the accuracy of the trained model with respect to the test data for task 1. We tested on dataset, and considered task incremental learning (task-IL) setup. One can observe that when the buffer size B is small, the accuracy drops as the model is trainod on new tasks. On the other hand, when B 2 100, the task-IL accuracy for lask is maintained throughout the process. Note that training with our default setting B 200 does not hurt the accuracy for task 1 throughout the continual learning process.
Table 3: Impact of the buffer size on Accr (%)
MNNIS (Domain IT.)
CIFARI0D (Class-II.)
sequential-CTFARRIO (Task-II.)
Butfer Size
Mothod
w/o Fed-A-GEM
w/ Fed-A-GEM
w/o Fed-A-GEM
w/ Fed-A-GEM
w// Fed-A-GEM
w/ Fed-A-GEM
200
68.34-56
74.74423
(16.40)
8.90
19.53413
(110.33)
63.84-08
74.84405
($11.00)
FL+A-GEM
70.18=8.7
78.74#82
(18.56)
8.87+01
25.89=0.0
(f17.02)
64.38=14
79.35=0.6
(114.97)
5120
69.97=8.2
79.17=4.
(19.20)
8.85
33.30=2.5
(124.45)
64.99=16
8132200
(119.53)
57.73=9.
87.13*11
(129.40)
13.32=14
22.96=a.8
(19.64)
57.71=12
65.57*19
(17.86)
500
FL+DER
60.00=7.2
88.83416
(128.83)
15.44=1.
34.872177
(119.43)
60.79-1.2
73.53
(f12.74)
5120
58.63 -3.6
89.4641.2 (130.83)
18.89=1.0
45.7043
(126.87)
62.77-1-5
83.11 (120.64)
permuted-MNIST (Domakn-IL)
sequential-CIFAR10 (Clase-1L)
Dequential-CIFAR10 (Thai-1L)
Butfer Size
Method
w/o Fed-A-GEM
w/ Fed-A-GEM
w/o Fed-A-GEM
w/ Fed-A-GEM
w/o Fed-A-GEM
w/ Fed-A-GEM
200
33.43-144
39.09
(15.66)
17.82-0.
19.44
(f1.2)
77.14-31
83.1641. (f6.02)
FL+A-GEM
33.35=Lo
42.45=6.9
(19.10)
18.39=0.2
20.3
(f1.95)
78.43=0.
85.95#0.6 (17.52)
5120
12
u007=2.5 (17.35)
16.41=2.6
2066 (14.23)
73.89=4a
Si.8 (f12.93)
19.79=17
43.43=0.9
(123.64)
18.44=0.7
30.94=8.
(t12.50)
60.34=42
77.9g*0.8 (18.65)
500
FL+DER
19.17-16
43.38424
(124.21)
20.81
29.78
(18.97)
71.12115
74.98 (13.81)
5120
18.57-1.4
44.68 13.4 (126.11)
34.75-22
42.38
(17.63)
78.22-23
21.944177 (13.72)
We assume that every client has the same buffer size. If the buffer sizes vary during model training, clients with larger buffers may contribute more diverse data, potentially biasing the model. A possible solution involves using reweighting algorithm, which we plan to explore in future.
11

---

Published in Transactions on Machine Learning Research (01/2025)
Accuracy
40
Ours (Burter sizb 101
30
Oues thurfer size 50)
Curo (Buffer s22p 100)
(Butter
2000, debott
20
Dur (5u/ar 500)
Teak
Teak
Tark3
Teak4
Talk
Taak-
Task
Teat
Tel 9
Task
Figure 3: Change in accuracy (%) for task 1 upon completion of subsequent tasks for different buffer sizes, under S-CIFAR100 (Task-IL) setup. Fed-A-GEM with a larger buffer size (B) more effectively mitigates lorgetting of lask 1.
5.1.4 Effect of Communication Frequency.
Compared with baseline methods, Fed-A-GEM has extra communication overhead for transmitting the buffer gradients from each client the server. This means that the required amount of communication is doubled for Fed-A-GEM. We consider variant of Fed-A-GEM which updates the model and buffer gradient less frequently (i.e., reduces the communication rounds for each task), which has reduced communication than the vanilla Fed-A-GEM. Table 4 reports the performance for different datasets, when the communication overhead is set to 2x, 1x, 0.5x and 0.2x. First, in most cases, Fed-A-GEM with equalized (1x) commmication overhead outperforming FL. In addition, for most the tested datasets including R-MNIST, P-MNIST and S-CIFAR100, Fed-A-GEM outperforms FL with at most 0.5x communication overhead. This means that Fed-A-GEM enjoys higher performance with less communication, in the standard CFL benchmark datasets.
Table 4: Effect of commuication on accuracy (%), with values in brackets indicating differences from the FL.
R-MNIST
P-MNIST
S-CIFAR10
S- S-CIFAR100
Method
Domain-IL
Domain-IL
Cless-IL
Tesk-IL
Class-IL
Task-IL
FL FL w/ Ours (2x comm)
68.02a81
27.49F23
17,4
70.58=40
8,76
47.74#12
79.46244 (111-4))
35.91 (1842)
18.02 (10.58)
80.83220 (110.25)
17.08414 (18.32)
74.71 (126.97)
FL w/ Our (equelized comm)
75,63+a9 (17.6L)
31.96*32 (17.47)
16.65=1e (J0.79)
78.79=8 (18.21)
13.62404
(11.66)
73.96*04 (126.22)
FL w/ Ours (0.5x commn
76,05+40
(18.03)
29.70*48 (12.28)
11,30=1a (13.14)
66,90=a0 (L3.68)
13.09402
(11.33)
69.96 (122.22)
FL w/ Our (0.2x comm)
70.39447 (12.57)
15.51 (111.988
18.37420
(14.07)
39.73=64 (110.83)
18.59409
(14.83)
59.31 (111.57)
5.1.5 Effect of Computation Overhead.
Computation overhead is also an important aspect consider and we have conducted experiments the actual wall-clock time measurements. Taking CIFAR100 experiment as an example, the running time for 200 epochs for FedAvg on our device is 4068.97s. When Fed-A-GEM, which is built on top of FedAvg, was used, it ran for an additional 293.26s. This indicates that it ran 7.2% longer over the same 200 epochs. Thus, adding Fed-A-GEM has negligible increment in the required computational overhead. The secure aggregation implementation follows the approach described in this work (Bonawitz et al., 2017). Our results indicate that secure aggregation introduces an additional overhead of 172.01 seconds, representing 4.23% increase in training time. This overhead includes the generation of cryptographic key pairs for all participating clients, and the application of masks to client weights during both model secure aggregation and buffer gradient. secure aggregation.
The time consumed by Fed-A-GEM can divided into two parts: (i) computing the global reference gradient after each FodAvg, and (ii) projecting the gradient. For part (i), the computational complexity of computing the global reference gradient for each client involves sampling from the buffer, computing the gradient for each sample, and averaging these gradients. This process has complexity of O(mP), where m is the
12

---

Published in Transactions on Machine Learning Research (01/2025)
number of samples in the buffer and P is the total number of parameters. In our experiment, the reference gradient, computation was performed 200 times, taking a total of 49.07s. For part (ii), the gradient projection was performed on 109,471 batches, which is 68.38% of the total batches, taking a total of 244.19s. The computational complexity of the gradient projection step, which involves operations such as dot products, scalar multiplication, vector scaling, and subtraction, is O(P), where P the total number of parameters.
5.1.6 Asynchronous Task Boundaries.
In our previous experiments, we assumed synchronous task boundaries where clients finish tasks at the same time. However, in many real-world scenarios, different clients finish each task asynchronously Motivated by this practical setting, we conducted experiments in an asynchronous task boundary setting on sequential- CIFARIO0. The CIFAR-100 dataset was divided into 10 tasks, each containing 10 unique classes, totaling 100 classes Data was distributed across 10 clients in a non-IID manner using a Dirichlet distribution with 0.3. Figure illustrates the task distribution across users. In the asynchronous setup, each client processes exactly 500 samples at time, regardless of task boundaries. The vertical dashed lines indicate 500-sample intervals. As shown by the colored bars in Figure 4, clients have varying amounts of data per task, thus clients may work on different classes and be at different stages of a task compared to others at any given time. This setup more closely aligns with our general continual learning settings, when the task boundary is unknown. Table 5 shows the accuracy of each method averaged over tasks after finishing all training, under the asynchronous setting. Similar to the synchronous case, Fed-A-GEM improves the accuracy of baseline methods including F-+A-GEM and FE.+ Notably, we have better performance in the asynchronous setting (see Table 5) compared with the synchronous sctting (see Table 1). For example, under the sequential-CIFAR task, the Fedd A-M method achieves 72.02% accuracy for the asynchronous case while achieving 65.57% for the synchronous case. This might be because, in the asynchronous setting, some clients receive new tasks carlier than others, which allows the model to be exposed to more diverse data for each round, thus reducing the longetting.
Table 5: Acor (%) for asynchronous task boundaries on the sequentinl-CTFARR00 dataset.
Method
Class-IL
Task-IL
FL
16.22+12
50.04+1.7
F-EA-GEM
16.92410
69.4141.
FL+A-GEM+ Ours
30.74415
77.70 +0.4
FL+DER
31.95
68.28
DER Ours
36.29
72.02+0.7
5.1.7 Effect of the Number of Tasks.
As shown in Table 6, we have conducted experiments with different numbers of tasks for each dataset. For CIFAR100, we experimented with task numbers and 10, while for CIFAR10 we tested with task numbers 2 and 5. Our results in Table 6 consistently demonstrate that the Fed-A-GEM algorithm provides a significant. improvement in performance across all these different task mumbers. An interesting observation is that the number of tasks increases, Fed-A-GEM have better performance improvement to baseline. For example, under the sequential-CIAARIOO task with 10 tasks, FL-FeddA-GEM achieves a performance improvement of 26.97% compared to the bascline, while with 5 tasks, it achieves an improvement of 18.81%. This is because a higher number of lasks increases the likelihood of data distribution shifts and therefore the problem of catastrophic forgetting becomes more prominent. As such, Fed-A-GEM, designed to handle this issue, has more opportunities to improve the learning process in such scenarios. This might also partly explain why, in Table 1, Fed-A-GEM shows generally higher improvement over the baselines the AequentialCIFARRAI dataset compared to the sequential-CTFAR10
5.1.8 Effect of the Number of Users.
We also conducted experiments to assess the scalability of Fed-A-GEM by increasing the client count K 20. Table 7 shows the results for K 20 clients. These results demonstrate that Fed-A-GEM consistently
13

---

Published in Transactions on Machine Learning Research (01/2025)
1vd b0c4
1xd
ther
Meer
Over
3
Number Samples
Figure 4: Task distribution across users for the asynchronous setting with CIFAR-100. The dataset is divided into 10 tasks (different colors). Dala is distributed non-III) across 10 clients. Clients process 500 samples at time (vertical dashed lines), progressing through tasks asynchronously.
Table 6: Average accuracy Acct (%) across various lask numbers.
sequential-CIFAR10 (Class II)
Dequential-CIFARIO (Task-IL)
(# Task, # of Classes per Task)
FL
FL w/ Fed-A-GEM
FL
FL w/ Fed-A-GEM
43.53=0.8
44.05 (10.52)
75.54
77.52 (11.98)
(5,
17.44=13
18.0240.6 (10.6)
70.58=4.0
80.83+20 (110.25)
aequential-CIFARII (Class-II) II)
sequential-IFFARRO00 (Tash: IT))
FI.
FT. w/ Fed-A-GEM
FI.
FI. w/ Fed-A-GEM
20) (10, 10)
16.49#0.3
22.71 (16.22) 17.08#1. (18.32)
50.60
69.41 (118.81)
8.76=0.1
47.74
74.71 10.9 (126.97)
improves the performance baselines, across different numbers of clients. When the number of users increases from 10 20, methods without Fed-A-GEM generally perform worse because each user has less data. However, Fed-A-GEM performs better with more users in several cases because it uses the increased diversity of data across users to guide learning and preserve knowledge more effectively. This shows Fed-A-GEM ability to turn the challenge of more users into advantage. Additionally, we evaluated real-world scenario where only a random subset of clients participates in training during each round. Moreover, inspired by the client incremental setup described in the GLFC paper, we simulated a dynamic environment where new clients are gradually introduced. Detailed information and results are available Appendix C.3.
5.1.9 Effect of Larger Datasets.
We have conducted experiments the Tiny-ImageNee dataset, which offers more naturally diverse and challenging sctting for continual Icarning. For the Tiny-ImageNet dataset, we divided it into 10 tasks, with each lask containing 20 unique classes. Given the increased complexity and scale of the dataset, we increased the number local epochs to 20. We used a pre-trained ResNet-18 model for our experiments. The results are shown in the Table 8. Our method demonstrates significant improvements both Class-II. and Task-IL accuracy compared to the basclines this more large-scale dataset. For example, the Task- IL setting, FL.+A-GMM FedG-M+Fed--GGEEE Fed-A-GEM achieves 50.27% accuracy, while the baseline FL achieves 32.41%
14

---

Published in Transactions on Machine Learning Research (01/2025)
Table 7: The ACcr (%) performance measured when we have K 20 users.
rolAted-MNIST (Domain-IL))
Nequential-CIFARII (Cloxs-L.)
sequential-CTFARIO (Tosk-TE)
Method
w/oo Fed-A-GEM
w/ Fed-A-GEM
w/o Fed-A-GEM
w/ Fed-A-GEM
w/o Fed-A-GEM
w/ Fed-A-GEM
FL
02.45+8.5
76,01
(113.56)
16.44$14
15.82#17
(J0.62)
68.18
73.43+43 (15.27)
FedCury
02.57
76,46*4.1
(113.89)
17.31
14.64#3.1 (J2.67)
67.3345.3
70.31 (12.98)
FedProx
02. 12.14
75.84
(113.70)
16.37
16.15#13
(J0.22)
06,24
74.79*39 (18.55)
FL+A-GEM
67 66666.
78.10455
(110.44)
16,15419
17.36 (11.21)
72.3945.4
80.61 (18.22)
FL+DER
57.8 43333
87.84
(130.51)
17.13423
19.18 (12.05)
70.82419
77.04 (16.22)
permuted-MINIST (Domein-IL)
sequential-CFFARI00 (Closs-IL)
sequential-CFFARL00 ( (Task-IL)
FL
20.2641.
20.672477
(10.41)
8.61
17.4741-1
(18.86)
50.00
76.29 (126.29)
FedCury
20.25
23.30
(13.05)
8.93400
19.4241.
49.8341.4
79.58
(129.75)
FedProx
20.19414
23.7845.2
(13.59)
8.882011
18.86410
(19.98)
50.86412
78.1949.
(127.38)
FL+A-GEM
24.4342.1
23.29
(J1.14)
8.6240.1
19.58+1.
(110.96)
63. 63.0249.6
76.23#9.6
(113.21)
DER
17.89
46.1743.0
(128.28)
11.53*0.
26.64
(115.11)
57.00 57.00++1.4
69.42+10
(112.42)
Table 8: Average accuracy Acer (%) on ImTinylmageNet
S- S-TinyImageNe
Methods
Class-Il
Task-II
FL
6.48
32.41
FL+A-GEM
6.58 (10.10)
39.66 (t7.25)
FL+DER
6.36 (10.12)
32.53 (10.12)
Fl.+Ours
7.95 (f1.47)
45.97 (113.56)
10.20 (13.72)
50.27 (117.86)
5.1.10 Effect of Out-of-Distribution (OOD) Datasets
To further validate Fed-A-GEM effectiveesss in handling distribution shifts, we conducted experiments on two challenging OOD benchmarks: PACS (Li et al., 2017) and Office- Caltech-10 (Wang et al., 2018). These datasets are particularly relevant for evaluating continual learning methods as they introduce natural domain shifts that models must adapt to while preserving knowledge. The PACS dataset consists of images across four distinct domains (Photo, Art, Cartoon, Sketch), presenting significant visual style variations. OfficeCCallec--10 contains images of office objects from different sources (Amazon, DSLR, Webcam, Caltech), introducing real-world domain shifts. We have maintained consistent experimental conditions with our main experiments: 1esNett18 (pretrained on ImageNet) as the backbone, 10 clients with non-IID data distribution, and a memory buffer size of 200 per client. The results demonstrate that Fed-A-GEM significantly improves performance on OOD data, with particularly strong gains on the more challenging PACS dataset (+30.42%) These results complement our main findings and demonstrate Fed-A-GEM effectiveness in challenging real-world scenarios where distribution shifts common.
Table 9: Average accuracy Acer (%) on Cut-oo-Distributioon datasets
Oustr--Distibbttoon Datasets
Methods
Office-Caltech-10
PACS
FL
90.4741122
40.30:12.1
FL Fed-A-GM
93.10-10.8 (*2.63)
70.7211.8 (130.42)
5.2 Text Classification
In addition to image classification, we also extended the evaluation of our method on text classification task (Mehta el al., 2023). For this purpose, we utilized the YahooQA (Zhang et al., 2015) dataset which comprises texts (questions and answers), and usee-generated labels representing 10 different topics. Similar to the approach taken with the CIFAR1O datasot, we partitioned the YahooQA dataset into 5 tasks, where each task consisted of two distinct classes. Within each task, we used LDA partition data across 10 clients in a non-IID manner. To conduct the experiment, employed a pretrained Disti (Sanh et al., 2019) with
15

---

Published in Transactions on Machine Learning Research (01/2025)
linear classification layer. We freeze the DistilBERT model and only fine-tune the additional linear layer. The results of this experiment can be found in Table 10. We can observe that Fed-A-GEM consistently enhances the accuracy (Acer) over baselines, particularly in class-IL scenarios. For example, FL-AA-CEM+FedA-GEEN achieves 47.02% accuracy, while the baseline FL achieves 17.86%.
Table 10: Average classification accuracy Acer (%) on split-YahooQA dataset.
requential-YahooQe (Class-IL)
sequential-YahooQ. (Task-IL)
Method
w/o Fed-A-GEM
w/ Fed-A-GEM
w/o Fed-A-GEM
w/ Fed-A-GEM
FL
17.86 L0.6
60.6724.1111...
80.87
(8..471.
FL+A-GEM
20.86 40.3
47.02 (12.616)122616)) (12..16)
87.
90.20 (12.911
FL+DER
43.64 12.1
122811.0110.666
89.57
5.3
Ablations on Algorithm Design
We have performed ablation studies on our algorithm, which consists of two main components: the gradient refinement algorithm and the buller updating algorithm. These experiments help in understanding the individual contributions of each component the overall performance of our system.
5.3.1 Gradient Refinement
First, considered different ways of refining the gradient 9, given the reference gradient gref. We define the refined gradient I as follows:
Avorage: The gradient ğ is the arithmetic mean of g and grets expressed as g 2
Rotate: The gradient g is rotated towards gret, maintaining its original magnitude. This can be represented as:
9 + 9ref
|g|||
[[ + 9ret
where
||s|| denoles the magnitude of g.
Project: projected onto a space orthogonal to Jref-
Project & Scale: This method extends the Project method by scaling the resultant vector to match the original magnitude of
Our Fed-A-GEM applics "Project" method only when the angle between and Jref is larger than 90 degree, i.e, when the reference gradient fret (measured for the previous lasks) and the gradient g (measured for the
the current task) conflicts to each other. Our intuition for such choice is, it is better to manipulate direction favorable for current task is conflicting with the direction favorable for previous tasks. To support that this choice is meaningful, we compared two ways of deciding when we manipulate the gradients:
Conditional Refinement (> 90%) Gradient g is updated only when the angle between g and gret is greater than 90 degrees.
Unconditional Refinement (Always): Gradient g is always updated irrospective of the angle.
The performance of these stteeiee is demonstrated in Table 11, for sequential-CIFARII dataset. The results reveal that our Fed-A-GEM (denoted by Project (> 90) in the table) far outperforms all other combinations, showing that our design (doing projection for conflicting case only) the right choice. Investigating cach component (Project and (> 90)) independently, we can observe that choosing "Project" outperforms "Average", "Rotate" and "Project & Scale" in most cases, and choosing (> 90) outperforms " "Always" in all cases.
16

---

Published in Transactions on Machine Learning Research (01/2025)
Table 11: Effect of gradient refinement methods on the accuracy (%) of FedGP on sequential-CIFAARO0
Method
Class-IL
Task-IL
FL
8.76-0.1
47.74#1.2
Average (Always)
7.26-11.5
35.96-33.23
Average (> 90)
7.79*0066
36.57#1.55
Rotate (Always)
7.59=
36.15+2.83
Rotate (> 90)
8.41 70.78
38.97=1.88
Project & Scale (Always)
1.009
32.96-1.10
Project & Scale (> 90)
12.30=0.65
73.61
Project (Always)
8.90
34.00
Project (> 90), ours
17.08+1.8
74.71
We also tested whether doing the projection is helpful in all cases when angle(g, fret) 90. We considered applying the projection for p% of the cases having angle(g, 9ret) > 90, for p 10, 50, 80 and 100. Note that 100% case reduces to our Fed-A-GEM. Table 12 shows the effect of projection rate p% on the accuracy, tested on sequential-CTFARR000 dataset. both class-IT. and task-II, settings, increasing always improves the accuracy of the Fed-A-GEM method. This supports that the projection used in our mothod suitable for the continual federated learning selup.
Table 12: Effect of projection rate p% the accuracy (%) of Fed-A-GEM, tested sequential-CIFARRJOO
Method
Class- IL
Task-IL
FL, 0%
8.76
47.74
Fed-A-GEM, p = 10%
8.82
54.90=21.61
Fed-A-GEM, p 50%
#0.07
67.89+0.07
Fed-A-GEM, p 80%
10.36+0042
72.73
Fed-A-GEM, p 100% (ours)
17.08=1.
74.71 #0.9
5.3.2 Buffer Updating
In Table 13, we compared three different buffer updating algorithms:
Sliding Window Sampling: This method replaces the earliest data point in the buffer when new data arrives
Random Sampling: It randomly replaces data point in the buffer with incoming new data
Reservoir Sampling (Vitter, 1985) (Used in Ours): We employ it for buffer of size |M and total number of observed samples up to now, which operates as follows:
When n < LM we simply add the current sample to the buffer.
When n > MM with probability [M we replace sample in buffer with the current sample.
This method ensures that each of the n samples has an equal probability of being included in the buffer, crucial for maintaining uniform sample representation from each task throughout the contimual learning process. The effectiveness of using Reservoir Sampling in our method is validated in Table 13, where it outperforms other methods.
17

---

Published in Transactions on Machine Learning Research (01/2025)
Table 13: Effect of buffer updating algorithms on the accuracy (%) of Fed-A-GEM, tested on S-CIFAR100
Method
Class-IL
Task-IL
FL
8.76
47.74=12
Sliding Window Sampling
8,8240.15
46.16-12.38
Random Sampling
9.72=#0.10
54.82+1..8
Reservoir Sampling (used in ours)
17.08#1.8
74.71 #0.9
We also present the pscudocode for the ReservoirSamplingg algorithm in Algorithm Reservoir sampling ensures each sample has an equal probability of being inchuded in the buffer. The probability of sample being contained in the buffer is |M This can be shown by induction, assuming the statement is true for π 1 samples and showing it holds when one additional sample is observed. The probability of a sample
n
|MM contained in the buffer can be computed as n-1 (1 |M4 x [ |M where
a
n
|M is the probability that a sample is initially in the buffer.
|M**| x [ is the probability that a sample is not displaced from the buffer.
|M
is the probability that sample is displaced from the buffer.
In conclusion, the reservoir sampling method used in Fed-A-GEM allows us to have balanced sample distribution across different tasks, thus allowing us to mitigate catastrophic forgetting and to improve the accuracy in the continual federated learning setting, Additionally, we recognize the potential of biased reservoir sampling (Aggarwal, 2006), a variant of reservoir sampling that employs exponential bias functions to prioritize recent data. This technique is particularly advantageous for data streams where recent observations are more significant. We believe that integrating such approaches could align with and further complement our method.
6 Conclusion
In this paper, we present Fed-A-GEM, simple yet highly effective method of using buffer data for mitigating the copastrophic forgetting issues in CFL. Our approach leverages a buffer-based gradient projection strategy that integrates scamlessly with existing CFL techniques to enhance their performance across various tasks and settings. Through extensive experiments on benchmark datasets such rolated-MNIST, permuted-MNIST. sequential-CIFARROO sequential-CIFARRIOO0 and sequential- AequentialYYahooAAA. we demonstrate that Fed-A-GEN consistently improves accuracy and reduces forgetting compared to baseline methods. Notably, our method achieves these improvements without increasing the communication overhead between the server and clients.
Despite these promising results, our work has certain limitations. First, while Fed-A-GEM effectively mitigates forgetting, it requires maintaining a buffer of past samples, which might not be feasible for all devices. Second, the assumption of secure aggregation, while essential for preserving privacy, adds computational overhead that may alfect scalability in extremely large federated networks. There are several directions for future research. One potential avenue is to explore more efficient buffer management strategies that further reduce storage requirements while maintaining performance. Another interesting dircction to integrate advancod techniques, such as differential privacy or homomorphic encryption, with our approach to enhance the security and privacy guarantees sensitive applications.
Acknowledgments
Authors are supported in part by the following: US National Science Foundation awards 2NS-2112562, CNS-2107060, CNS-2213688, CNS-2312716, and the US Department of Commerce award 7ONANB21H043. Jy-yong Sohn is supported by the National Research Foundation of Korea (NRF) grant funded by the Korea
18

---

Published in Transactions on Machine Learning Research (01/2025)
government (Ministry of Science and ICT; MSIT) (RS-2024-00345351, RS-2022--00408003) the IITPttnteetee of Information & Coummunications Technology Planning & Exalnation)-ICAN(IITT Challenge and Advanced grant funded by the Korea government (MSIT), and the Yonsei University Research Fund (2024-22-0124) Kangwook Lee is supported by NSF CAREER. Award CCF2339978, an Amazon Research Award, and grant from FuriosaAI.
Network of (RLD))RR-002-000259900
References
Charu C Aggarwal. On biased reservoir sampling in the presence of stream evolution. In Proceedings of the 32nd international conference on Very large data bases, pp. 607- -18, 2006.
Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. Expert Gate: Lifelong learning with a network of experts. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, pp. 3366 3375, 2017.
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Timne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Conference on Computer Vision, pp. 139 139-154, 154,
2018.
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. Advances in Neural Informalion Processing Systems, 32:11816 1186661-11,,-1 2019.
Sara Babakniya, Zalan Fabian, Chaoyang He, Mahdi Soltanolkotabi, and Salman Avestimehr. data- free approach to mitigate catastrophic forgetting in federated class incremental learning for vision tasks. Advances in Neural Information Processing Systems, 36, 2024.
Yavuz Faruk Bakman, Duygu Nur Yaldiz, Yahya H Ezzeldin, and Salman Avestimehr. Federated orthogonal
training: Mitigating global catastrophic forgetting in continual federated learning. In International Conference on Learning Representations, 2024.
Mehdi Abbana Bennani, Thang Doan, and Masashi Sugiyama. Generalisation guarantees for continual learning with orthogonal gradient descent. arXiv preprint arXiv:2006. 11942, 2020.
Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcodone, H Brendan McMahan, Sarvar Patel, machine
Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for learning. In proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 1175-1191, 2017.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: strong, simple baseline. Advances in Neurol Information Processing Systems, 33:15920-15930, 2020.
Xiaoyu Cao, Minghong Fang, Jia Liu, and Neil Zhenqiang Gong. Fltrust: Byzantinc-robust federated learning via trust bootstrapping. Xiv preprint or Xrx::2201 13995, 2020.
Hyuntak Cha, Jacho Lee, and Jinwoo Shin. Co Contrastive continual learning. In Proceedings of the IEEE international conference computer vision, pp. 9516 9525, 2021.
Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the Europeon Conference on Compuler Vision, pp. 532-54424 547, 2018.
Arslan Chaudhry, Marc' Aurelio Ranzato, Mareus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong Marc'Aurelio
learning with A-GEM. In International Conference Learning Representations, 2019.
Yiqiang Chen, Xin Qin, Jindong Wang, Chaohui Yu, and Wen Gao. Fedhealth: A federated transfer learning framework for wearable healthcare. IEEE Intelligent Systems, 35(4):83 93, 2020.
Zhiyuan Chen and Bing Liu. Lifelong machine learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 12(3):1-200 2018.
19

---

Published in Transactions on Machine Learning Research (01/2025)
Shenghong Dai, SM Iftekharul Alam, Ravikumar Balakrishnan, Kangwook Lee, Suman Banerjee, and Nageen Himayat. Online federated learning based object detection across autonomous vehicles in virtual world. In IEEE Consumer Commumications & Networking Conference, pp. 919 920, 2023.
Jiahua Dong, Lixu Wang, Zhen Fang, Gan Sun, Shichao Xu, Xiao Wang, and Qi Zhu. Federated class- incremental learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10164-10173, 2022.
Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. CARLA: An open urban driving simulator. In Conference robot learning, volume 78, pp. 1-16, 2017.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image worth 16x16 words: Transformers for image recognition at scale. In International Conference Learning Representations, 2020.
Ahmet M Elbir, Burak Soner, Sinem Çôleri, Deniz Gündüz, and Mchdi Bennis. Fodcrated learning in vehieular networks. In 2022 IEEE International Medilerranean Conference on Communitcations and Nelworking (MeditCom)), pp- 72-77. TEFE, 2022.
Mehrdad Farajtabar, Navid Azizan, Alex Molt, and Aug Li. Orthogonal gradient descent for continual learning. In International Conference on Artificial Intelligence and Statistics, pp.
2020.
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A Rusu, Alexander Pritzel, and Daan Wierstra. PathNet: Evolution channels gradient descent in super neural networks. arXie preprint 1ii::1711.0770 2017.
Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4): 128 135, 1999.
Robin C Geyer, Tassilo Klein, and Moin Nabi. Differentially private federated learning: A client level perspective. arXiv preprint Xrxi 07557, 2017.
lan J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient-baset neural networks. arXiv preprint arXie:1312.6211 2013.
Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXic:1909. 00335 2019.
Peler Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. Foundations and Trendso in Machine Learning, 14:1 210, 2021.
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. SCAFFOLD: Stochastic controlled averaging for federated learning. In Interna- tional Conference on Machine Learning, volume 119, pp. 5132-5143, 2020.
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, Zixuan Ke, and Bing Liu. A theoretical study on solving continual learning. Advances in neural information processing systems, 14.5065-5079 2022.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, et al. Overcoming catastrophic
Kieran Milan, John Quan, Tiago Ramalho, Agnieszka forgetting in neural networks. Proceedings of the national academy of sciences, 114:3521-3526, 2017.
Jakub Koneený, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategics for improving communication efficiency. arXiv preprint arXiv:1610.05.4/ 05440, 8, 2016.
Sebastian Lee, Sebastian Goldt, and Andrew Saxe. Continual learning in the teacher-studens setup: Impact of task similarity. In International Conference on Machine Learning, pp. 6109 6119. PMI.R, 2021.
20

---

Published in Transactions on Machine Learning Research (01/2025)
Da Li, Yongxin Yang. Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on compuler vision, pp. 5542-55550000 -5550, 2017.
Tian Li, Maziar Sanjabi, Ahmad Beirami, and Virginia Smith. Fair resource allocation in federated learning.
In International Conference on Learning Representations, 2019.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in hetcrogencous networks. Proceedings of Machine learning and systems, 2:429-450, 2020.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEFE transactions on pottern analysis and machine intelligence, 2017.
Wei Yang Bryan Lim, Nguyen Cong Luong, Dinh Thai Hoang, Yutao Jiao, King-Chang Liang, Qiang Yang, Dusit Niyato, and Chunyan Miao. Federated learning mobile edge networks: A comprehensive survey. IEEE Communications Surveys & Tutorials, 2020.
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in Neural Information Processing Systems, 30:6467 6476,, 2017.
Yuhang Ma, Zhongle Xie, Jue Wang, Ke Chen, and Lidan Shou. Continual federated learning based on knowledge distillation. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, volume 3, pp. 2182 2188, 2022.
Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. Online continual learning in image classification: An empirical survey. Neurocomputing, 019 2022.
Arun Mallya and Svetlana Lazebnik. PackNet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the IEEE Conference on Computer Vision and pattern Recognition, pp. 7765-7773, 2018.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learniin and molivalion, volume 24, pp. 109 165. Elsevier, 1989.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficient learning of deep networks from decentralized data. In Artificial intelligence and statístics, volume 54, pp. 1273 1000,1222222022 2017.
Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar, and Emma Strubell. An empirical investigation of the role of pre-training in lifelong learning. J. Mach. Learn. Res., 24:214:1-2144 2:::00, 2023.
Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh. Agnostic federated learning. International Conference on Machine Learning, volume 97, pp. 4615 4625, 2019.
German T Parisi, Ronald Kemker, Jose I, Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: review. Neural networks, 113:54 71, 2019.
Anastasia Pentina and Christoph Lampert. A mac-bayesian bound for lifelong learning. In International Conference on Machine Learning, pp. 991-9999 PMLR, 2014.
Quang Pham, Chenghao Liu, and Steven Hoi. DualNet: Continual learning, fast and slow. Advances in Neural Information Processing Systems, 34:16131-16144, 2021.
Ameya Prabhu, Philip HS Torr, and Puncet K Dokania. GDumb: A simple approach that questions our progress in continual learning, In Proceedings of the Europeun Conference Computer Vision, pp. 524 540, 2020.
Saurav Prakash, Hanich Hashemi, Yongqin Wang, Murali Annavaram, and Salman Avestimehr. Secure and fault tolerant decentralized learning. arXiv preprint arXiv:2010.07544, 2020.
Roger Ratcliff. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological review, 97(2):285, 1990.
21

---

Published in Transactions on Machine Learning Research (01/2025)
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. iCaRL: Incremental classifier and representation learning. In Proceedings of the IEEE Conference on Compuler Vision and Pottern Recognition, pp. 2001 2010, 2017.
Joseph Redmon and Ali Farhadi. YOLO9000: Better, faster, stronger. In Proceedings of the IEEE Conference on Compuler Vision and Pallern Recognition, pp. 7263 7271, 2017.
G Anthony Reina, Alexey Gruzdev, Patrick Foley, Olga Perepelkina, Mansi Sharma, Igor Davidyuk, Ilya Trushkin, Maksim Radionov, Aleksandr Mokrov, Dmitry Agapov, et al. Open FT. An open-source framework for federated learning. arXiv preprint arXiv:2105. 10X,::2005.0643,,, 2021.
Anthony Robins. Catastrophic forgetting, rehearsal and pscudorchearsal. Connection Science, 7(2):123-146, 1995.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606. 04671, 2016.
Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. In International Conference on Learning Representations, 2020.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXit:1910.01100 2019.
Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard allention to the lask. International Conference on Machine Learning, volume 80, pp. 4548 4557, 2018.
Momina Shaheen, Muhammad Shoaib Farooq, Tariq Umer, and Byung-Seo Kim. Applications of federated learning; taxonomy, challenges, and research trends. Electronics, 11(4):670, 2022.
Hamul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. Advances in Neural Information Processing Systems, 11-2990-2990 2017.
Konstantin Shmelkov, Cordelia Schmid, and Karteek Alahari. Incremental learning of object detectors without catastrophic forgetting. In Proceedings of the IEEE international conference on computer vision, pp. 3400-3409, 2017.
Neta Shoham, Tomer Avidor, Aviv Keren, Nadav Israel, Daniel Benditkis, Liron Mor-Yosef, and Itai Zeitak. Overcoming forgetting in federated learning non-llI data. arXiv preprint arXic:1910.0771 07796, 2019.
Sebastian Thrun. Is learning the n-th thing any easier than learning the first? Aduances in Newral Information Processing Systems, 8:640 646, 1995.
Yeshwanth Venkatesha, Youggeun Kim, Hyoungscob Park, Yuhang Li, and Priyadarshini Panda. Addressing client drift lederated continual learning with adaptive optimization. arXiv preprint orXiv:2203. 1102200..3333 13321, 2022.
Jeffrey S Vitter. Random sampling with reservoir. ACM Transactions Mathematical Software, 11:37-57, 1985.
Jindong Wang, Wenjie Feng, Yiqiang Chen, Han Yu, Meiyu Huang, and Philip S Yu. Visual domain adaptation with manifold embedded distribution alignment. In Proceedings of the 26th ACM international conference on Multimedia, pp. 402-410, 2018.
Zifeng Wang, Zizhao Zhang, Chen-Yu Yu Lee, Han Zhang, Ruoxi Sun, Xiaqqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE Conference Compuler Vision and Pottern Recognition, pp. 139-149, 2022.
22

---

Published in Transactions on Machine Learning Research (01/2025)
Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosinski, and Ali Farhadi. Supermasks in superposition. Aduances in Neural Inf Information Processing Systems, 33:15173 15184, 2020.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 374 382, 2019.
Xin Yao and Lifeng Sun. Continual local training for better initialization of federated models. In IBEE International Conference on Image Processing, pp. 1736-1740, 2020.
Jachong Yoon, Eunho Yang. Jeongtac Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. In International Conference on Learning Represenlations, 2018.
Jachong Yoon, Wonyong Joong, Giwoong Lee, Eunho Yang, and Sung Ju Hwang. Federated continual learning with weighted inter-client transfer. In International Conference on Machine Learning, volume 139, pp. 12073-12086, 2021.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Cont learning through synaplic intelligence. In International Conference on Machine Learning, volume 70, pp. 3987-3995, 2017.
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in Neural Information Processing Systems,
657, 2015.
Zizhao Zhang, Han Zhang, Long Zhao, Ting Chen, Sercan Ö Arik, and Tomas Pfister. Nested hierarchical transformer: Towards accurate, data-efficient and interpretable visual understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 3417-3425, 2022.
Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-IID data, arXiv preprint ar Xrxi::186..00 005882 2018.
A Incremental Learning Settings in Continual Learning
We describe three types of incremental learning (IL) settings in Continual Learning (CL) with examples.
Task-IL (Task-Incremest
Learning): In this setting, each task is explicitly labeled, and the model knows which task it is performing during testing. For example, a model learns to classify: Task 1: Dogs vs. Cats, and Task 2: Cars vs. Buses. At test time, the model is informed whether "this is dog/cat classification lask" or "this is a vehicle classification task." This scenario is considered simpler because the model can utilize task-specific parameters or heads, given that it knows the task it is performing.
Domain-IL (Domain-Incremente Learning): The classes remain the same across tasks, but the input distribution changes. For instance, consider the task of classifying digits (0-9) across different domains: Task 1: Handwritten digits (MNIST) and Task 2: Street View House Numbers (SVHN). While the classes (0-9) remain constant, the visual characteristics (domain) vary. The model must learn to recognize the same classes under differing conditions and styles.
Class-IL (Class- Taass-Incremetall Learning): In this setting, new classes are introdueed with each task, and the model must learn to distinguish among all classes encountered so far. For example, consider an animal classifier learning: Task Dog and Cat, Task 2: Bird and Fish, and Task 3: Horse and Cow. At test time, after lcarning Task 3, the model is required to correctly classify any of the six animals without being informed which lask the input belongs to. This is often regarded as the most challenging setting.
B Reservoir Sampling
In the following, we provide the buffer updating algorithm for the Reservoir Sampling strategy.
23

---

Published in Transactions on Machine Learning Research (01/2025)
Algorithm ReservoirSsmplinggMM {(z,, yi B:))==,,))
Input: local buller M incoming data ((2s, mi)) and the number of previously observed samples for i = to 3 do
n + n ++
n S |M| then
Add data (z,, M) into local buffer M
else
j Uniform (1,2, .n}
if j |M then
MM + (1)
end
end end for Return M the updated local buffer
C Supplementary Results
In this section, furnish additional experimental outcomes that serve to further bolster the findings of our primary investigation.
C.1 Forgetting analysis across datasets
We present the complementary information to Table in Table 14, illustrating the extent of Egtz observed across multiple benchmark datasets. Our method exhibits exceptional effectiveness in mitigating forgetting. Remarkably, demonstrates consistent performance across all datasets and baselines, making versatile solution.
Table 14: Average forgetting Fgt, (%) (lower is better) on benchmark datasets at the final task T.
rotated-MNIST (Domain-IL)
sequential-CIAARI0 (Class-IL)
Sequential-CIFANN CIFAR10 (Task-IL)
Method
w/o Fed-A-GEM
w/ Fed-A-GEM
w/o Fed-A-GEM
w/ Fed-A-GEM
w/o Fed-A-GEM
w/ Fed-A-GEMM
FL
25,98#a.2
11.66-27
(J14.32)
80.09=3.6
78.62443
(42.07)
15.37=4.8
4.49+19
(L10.88)
FodCurv
25.80+24
11.18=27 (114.62)
80.90=6.6
79.85+a9
(11.05)
19.37#48
4.77416 (L14.60)
FedProx
25.74
11.76=2.
(113.98)
84 91.35+24
80.21
(14.11)
18.24
4.1*110
(114.07)
FL+A-GEM
26.30
15.18=2.4
(111.12)
18-6.66
80,38+25
(11.80)
10.00=3.0
4.15 (15.85)
FL+DER
21.22110
5.51=12
(J15.91)
10.98=144
47.88472
(113.10)
6.34
2.73+1.
(13.61)
permuted-MNISST (Domain-IL)
Coguential-CIAARRRI (Class-IL)
sequemtial-CIIAARRO
(Task-IL)
FL
43.17#563
21.40=4.
(J22.07)
77.69=0.5
67.02
(110.67)
34.38#16
5
(128.99)
FedCurv
42.88$5.0
22.85=35
(120.03)
78.40=0a
67.75
(110.65)
33.71
6.80
([27.85)
FedProx
42.5915.6
20.77=5.
(J21.82)
77.35=04
62.
(110.54)
34.79236
5.0g
([29.10)
FL+A-GEM
35.6115.3
24.05=24
(J11.56)
77.9720.
63.99
(113.98)
16.92#14
5.16
([11.76)
FL DER
45.33
34.71=5.0
(J10.62)
69.37*17
53.81+6.7
(115.53)
22.43=0.7
14.16*1.7
(18.27)
line with the presentation of forgetting in Table 14, we present the forgetting analysis when the number of clients is 20 in Table 15. Notably, our method exhibits consistent and impressive performance across varying numbers of users. It consistently proves its effectiveness regardless of the specific user count, showcasing its robustness and reliability.
24

---

Published in Transactions on Machine Learning Research (01/2025)
Table 15: The Fgt, (%) (lower better) performance measured when we have K users.
rolated-MNISS (Domaiu-IL)
sequential-CFFAR100 (Class-IL)
sequential-CFFAR10 (Task-TT.)
Method
w/o Fed-A-GEM
w/ Fed-A-GEM
w/o Fed-A-GEM
w/ Fed-A-GEM
w/o Fed-A-GEM
w/ Fed-A-GEM
FL
31,00
13.45=36
(U17.55)
82.62 82.62#5.1
73.3944. (19.23)
17,93=27
6,14 (111.79)
FedCurv
30.7349.3
12.97=3a
(U17.76)
79.5543.
75.3845. (J4.17)
18.19430
9.14 (19.05)
FedProx
31.0449.
13.31=3.4
(J17.73)
82.94=11
78.67442 (J4.27)
20.60#26
8,52*3. (L12.08)
FL A-GEM
25. 22.
11.02=30
(U14.20)
82 82.30
80,25+4.1 (J2.14)
12,29
4.00 (18.29)
FL ILIDER
28.9346.6
5,18
(J23.75)
55. 15.10
60.90
(15.80)
3,20
2.71 (10.49)
permuted-MNNIST (Domoin-[L)
CIQAARO0 (Clase-12)
sequential-CIFARRO00 (2ask-1L)
FL 24.27#5.2
8.67=7. (J15.60)
73.05=0.
62.71
(U10.34)
27.07=17
2.48*077 (124.59)
FodCurv
24.02#5.4
8,10=54 (J15.92)
80.0 30.07#0.5
68.58#11
((11.49)
34.63=17
3.48*0.
(131.115
FedProx
23.01#5.7
5,93=5. (J17.08)
79.46*05
68.40
(111.06)
32.82=14
4.13
(128.69)
FL+A-GEM
22.12
9.45#5.
(J12.67)
72.97#1.
60.27
(L12.70)
12,54=13
2.60+0.2 (19.88)
FL+DER
32.2641.1
27.30-4.2 (14.96)
67.07408
47.74
(119.33)
19.78-17
8.67414 (111.11)
C.2 Progressive performance of Fed-A-GEM across tasks
Fig. depicts the average accuracy Acc, measured at task t 1,2, 10 and the average forgetting Fgt, measured at task 2, ......... ,00 The accuracy of FedAvg rapidly drops different tasks are given to the model, as expected. FedCurv and FedProx perform similarly to FedAvg, while A-GEM and DER partially alleviate forgetting, resulting in higher accuracies and reduced forgetting compared to FedAvg. Combining these baselines with Fed-A-GEM lead to significant performance improvements, which allows the solid lines in the accuracy plot consistently remain at the lop. For example, for the experiment on lask-IL for sequentia-CIIAARII 71.12% for FodProx Fed-A-GEM. These results demonstrate that Fed-A-GEM effectively mitigates forgetting and enhances existing methods in CFL.
the accuracy measured task 5 (denoted by Accs) is 55.37% for FedProx, while
rotated-MNIST Domain-IL
sequential-CIFAR100 Class-IL
sequential-CIFAR100 Task-IL
80
60
70
Accuracy
70
40
60
60
20
1
2
8
9
10
2
3
4
5
6
7
8
9
10
1
2
3
5
10
80
40
40
Forgetting (%))
30
60
20
40
10
0
2
3
5
6
7
8
9
10
2
3
4
6
8
9
10
2
3
4
5
9
10
Tasks
Tasks
Tasks
A-GEM A-GEM+Ours
DER DER+Ours
FedCurv
FedProx FedProx+Ours
FedAvg
FedCurv+Ours
Figure 5: Evaluating accuracy (t) and lorgetting (L) in multiple datasets with and without Fed-A-GEM- using buffer size of 200. The solid lines indicate the results obtained with our method, while the dotted lines represent the results obtainod without our method. The results show significant improvement in accuracy well reduced forgetting for all settings.
25

---

Published in Transactions on Machine Learning Research (01/2025)
C.3 Random sampling
We implement a more realistic federated learning enviromment by applying uniform sampling techniques to randomly select the participating clients in each round. We conduct experiments on CIFAR100. A total of 50 clients is set up, and during each communication, only a random 50% of the clients participate in training. As can be seen, even in such a scenario, where our algorithm cannot update the reference gradient using the local buffer from all clients, there is still an improvement performance using our algorithm.
Table 16: Average accuracy ACct (%) with 50 clients and 50% client sampling rate, for sequential-CIFARIIO CIFARR00
Method
Class-IL
Task-IL
FL
7.46 + 0.08
43.83 1.33
FL--eedA-GEM
9.31 188). 65.76 (1.
Moreover, inspired by the client incremental setup described in the GLFC paper, simulated dynamic euvironment where new clients and new classes gradually introduced, assuming non-llD data distribution among clients. Starting with 30 local clients in the CIFAR-100 setting, introduced 10 additional new local clients with each incremental task. At each global round, 10 clients were randomly selected to achieve partial client participation. The results are as follows.
Table 17: Average accuracy Acer (%) under client incremental scenario
Methods
Class-IL
Task-IL
FL
9.31 =0.0
49.81 #1.9
FL+A-GGM
9.38
56.81 1114
FL+Ours
10.22
70.63#1.0
FL+A-GEM+Ours
10.82
72.88
As can be seen, method compared to the bascline FL, even in this more complex scenario involving the introduction of new clients. The parameter details of this experimental setup are shown in Table 18.
demonstrates significant improvements in -L accuracy
Table 18: The parameter detail of introducing new client setting that differs from our main setting.
Parameter
Value
learning rate
2.0
buffer size
2000
number of local training epoch
20
number of tasks local batch size communication task number of clients weight decay client participation rate
10
128
200
120
0.00001
0.512
C.4 Performance on the current task
Balancing the retention of old tasks and the learning of new ones is common challenge in continual learning. It can be difficult. to determine the best approach, especially when two tasks are significantly different. This is a challenge faced by many methods continual learning.
We provided additional experimental results on the performance measured for the current task. The below Fig. 6 shows the Class-IT. accuracy of Fed-A-GEM (with buffer size 200) and FL for sequential-CTPAARRO0,
26

---

Published in Transactions on Machine Learning Research (01/2025)
where the total number of tasks is set to 10. During the continual learning process, measured the accuracy of each model for the current task. One can confirm that using Fed-A-GEM does not hurt the current, task accuracy, compared with FL. Note that this shows that Fed-A-GEM does not impair the performance of the current task, while also alloviating the forgetting in upcoming rounds.
FedGP
FL
Accuracy
20
Task 1
Task
Task 3
Task
Task 5
Täsk 6
Task 7
Task 8
Task
Task 10
Figure 6: Class-IL Accuracy (%) of current task for FL and FedGP on the scquentialCIFFAR100
C.5 Backward and forward transfer metrics
Backward Transfer (BWT):
Backward Transfer measures the influence that learning new task has on the performance of previously learned tasks. After the model finishes learning all T tasks, BWT is defined as:
BWT T
or. - -4,4)
where:
Ør,i is the accuracy of the global model on task after training up to task T. 0gg is the accuracy of the global model on task after training up to task i.
Forward Transfer (FWT):
Forward Transfer measures the influence that learning a task has on the performance of future tasks. After the model finishes learning all T tasks, FWT is defined as:
FWT T- Σ (1-11 -b,)
where:
0;-1, 1, is the accuracy of the global model on task i before learning task
by the baseline accuracy of task i at random initialization.
Our method outperforms FedAvg (FL) both Backward and Forward Transfer metrics across the sequential- CIFAR10 and sequential-CIFARRR00 datasets, shown in the Table 19.
27

---

Published in Transactions on Machine Learning Research (01/2025)
Table 19: Backward and Forward Transfer (t) Results for sequential-CIFARIOO and sequential-CIFAROO
Metric
Dataset
Methods
Class-II
Toak-IL
Backward
CIFAR100
FL
-78.11
-36.52
Backward
CIFAR100
FL Fed-A-GEM
-72.24 5.87)
-3.68 (f 32.84)
Backward
CIFAR10
FL
-78.78
-14.48
Backward
CIFAR10
Fed+ A GEM
-78.55 (1 0.23)
(0.60 13.88)
Forward
CIFAR100
FL
16.98
16.98
Forward
CIFAR100
FLe--ed-GGEM
17.16 (1 0.18)
17.48 (1 0.50)
Forward
CIFAR10
FL
12.75
12.71
Forward
CIFARIO
Fl.+Fed-A Fed-A-GGMM
12.98 (1 0.23)
12.99 (1 0.25)
C.6 Effect of different curriculum.
We evaluate how the performance of Fed-A-GEM changes when shuffle the order of tasks in the continual learning. We randomly shuffle the sequential-CTFARIO task order and label them as curriculum 1 to 4, shown in the Table 20, Regardless of the different curriculum, FL-Fed-A-GEN outperforms FedAvg.
Table 20: Average accuracy Accr (%) across randomized curriculum in scquential-IFFAR100,
Curriculum
Methods
Class-IL
Task-IL
8.15
46.25
A.
12.10 3.95)
72.69 26.44)
8446
47.56
A
14.37
(1 5.91)
73.19 25.63)
8.82
45.04
Fed GEM
(1 3.76)
74.71 29.67)
7.87
43.87
FM (f 6.98)
73.74 29.87)
C.7 Additional hyperparameters for specific methods
In addition to the hyperparameters discussed in the main paper, additional method-speeific hyperparameters are outlined in Table 21.
Table 21: Additional hyperparameters for specific methods.
Method
Parameter
Values
FL+DER
Regularization Coefficient
sequential CIFAR10 (0.3), Others (1)
FL++2PP
Communication Round R
MoISS (5), bermated-MNIST (1), sequential-IFAAROO (20), sequential-CIFARI (20)
CFeD
Surrogate Dataset
sequential-CTAAN CTFARIIO (CTFAR100), sequeniall (CTFAR10)
Note: No server distillation included.
Object Detection
Here we test Fed-A-GEM on realistic strcaming data (Dai al., 2023) which leverage two open source tools, urban driving simulator (CARLA (Dosovitskiy et al., 2017)) and FL framework (OpenFL (Reina el. al., 2021)). As shown in Fig. 7a, CARLA provides OpenFL with real-time collection of continuous streaming vehicle camera output data and automatic annotation about object detection. This streaming data capture the spatio-temporal dynamics of data generated from real-world applications. After loading data of vehicles from CARLA, OpenFL performs collaborative training over multiple clients.
We evaluate the solutions to the forgetting problem by spawning two vehicles in a virtual town. During the training of the tOny (Redmon & Farhadi, 2017) object detection model, we use a custom loss that combines classification, detection and confidence losses. In order to quantify the quality of the incremental
28

---

Published in Transactions on Machine Learning Research (01/2025)
CARLA
OpenFL
1.0
DER
0.8
DER+FedGP
Continuous Data Loader
FedAvg
A
Server
mAp
0.6
Ideal
0.4
Client B
0.2
0.0
Client C
0
10
20
30
40
50
60
70
Communication Rounds
(a) Pramework for automotive data evaluation.
(b) Object detection performance comparison.
Figure 7: (a) The data loader continuously supplies data from CARLA camera outputs to individual FL clients. Fach client trains on its local data and updates buffer to retain old knowledge. (b) The result. shows the object detection performance comparison between Ideal, FedAvg, DER, and DER-Fed-AA-GM on realistic CARLA dataset.
Algorithm DER ClientUpdate client &
Algorithm 6 A-GEM ClientUpdate at client k
Task index t, model w, buffer gradient grefs batch size regularization coefficient X, and learning rate
Input:
Input: Task index t, model w, buffer gradient grefs batch size &
Load the dataset D local buffer M² Initialize = 0 at the first task for each batch ((z,,))){ in D do
Load the dataset D and local buffer M Initialize n = at the first task for each batch {(r,, y:) in D do
9e = V [1 Ei E(vs. [x;;w)
Let X = {2x) and Y = (u + h(x;w)
Sample {(x1, 26) from M
gb V.. [ Ei_ C(V'. M(d;; w))
where f(X; w) := (h(X; w))
Sample (X', Zz, ) from M
g 9. projg. 9c 1 Je 0)
frog + A 2||Z' h(X'; w|]l
ğ - g proj 1 1 S 0)
g = V, [(XY, w) Loog
w w ag for some learning rate a ((1,, (1))
9 - g proj 9 (g S 0)
deservoirSampligge
w og
+- n B end for Return w
DeservoirSamlingg/M
Z,
n n 3 3 end for Return w
model trained by various baselines, we report a common metric, namely, mean average precision (mAP). This metric assesses the correspondence between the detected bounding boxes and the ground truth, with higher scores indicating better performance. To calculate mAP, we analyze the prediction results obtained from pre-collected driving snippets of vehicular clients. These driving snippets are gathered by navigating the town over duration of 3000 simulation seconds.
For those experiments on realistic CARLA streaming data, we compare the performances of Ideal, FedAvg, DER and DER+Fed-A-GEM. In the Ideal scenario, the client possesses sufficient memory to retain all data from prior tasks, enabling joint training on all stored data. The last two methods are equipped with buffer size of 200. We train for 70 commuiccttion rounds and each round continues for about 200 simulation seconds. The results are presented in Fig. 7b. Note that commuicatiin round 60, one client gets on the highway, which incurs domain shift. One confirm that the performance of FedAvg degrades in such domain shift.
29

---

Published in Transactions on Machine Learning Research (01/2025)
scenario, whereas DER and D maintain the accuracy. Moreover, Fed-A-GEM nearly achieves the performance of the ideal scenario with infinite buffer size, demonstrating the elfectiveness of our method.
E Continual learning methods with Fed-A-GEM
We provided the pseudocode for Algorithm 2 modifications when implementing FL+DER-Fed and FE---GM---ee---GE- respectively presented in Algorithm 5 and Algorithm 6. Other FL+CL and CFL methods are also combined with Fed-A-GEM in a similar manner.
A Algorithm incorporates Dark Experience Replay (DER) into the local update process on client E [K]. When the server sends the global model w client the client calculates the output logits or pre-softmax response In addition, the client samples past data (a' y') and the corresponding logits z' from the buffer M To address forgetting, the regularization term considers the Euclidean distance between the sampled output logits and the current model's output logits buffer data. The gradient is then refined using this regularization term to minimize the discrepancy between the current and past output logits, thereby mitigating forgetting. The following steps are the same as in the main text.
Algorithm combines with A-GEM, applying gradient projection twice. First, the client computes the gradient Je with respect to the new data from D. After replaying previous samples (x'.8') stored in the local buffer M the client computes the gradient Ib with respect to this buffered data. If these gradients differ significantly in terms of their direction, the client projects 9e onto gb to remove interference.
30


Reviews:
[{"review_id": "xV8Ve3JCXn", "reviewer": "Reviewer_oUGN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "***Strengths***\n- The paper contains extensive experiments, on multiple datasets, studying the impact of multiple factors on the eventual performance. I also appreciate that the authors included limitations of their work in the conclusion.\n\n***Weaknesses***\n- This is an empirical work. The authors have extended the A-GEM idea to the federated setting. However, there is not much theoretical justification for many things, beyond some basic intuition. This does not necessarily limit the contribution of the paper. However, I would really appreciate it if the authors could refer to some relevant theory papers (if they exist) on CL.\n\n***Questions***\n\nGiven my limited background in CL, I also have a few clarification questions:\n- There seem to be two kinds of information - present in model update and past in buffer gradient. Are there CL variants that maintain a more fine-grained distinction in past information, weighting the more recent information more highly?\n- The two methods DER (2) and A-GEM (3) solve penalized and constrained versions of the same problem. Under convexity, these are equivalent under mild conditions. But, since convexity is not assumed, how do they compare?\n- Section 5.1.1: perhaps domain-IL, class-IL, and task-IL are standard terminology in CL. But, can you explain them in a bit more detail, with an example if possible?\n- At the beginning of Section 4, it is said that Fed-A-GEM is ``a federated adaptation of the A-GEM method (Chaudhry et al., 2019).'' However, in Table 1, the 2nd row is FL+A-GEM. This also appears later in the paper. How is this different from Fed-A-GEM?\n- Table 1: the difference between w/o Fed-A-GEM and w/ Fed-A-GEM in the rows on FL-L2P is noticeable. Why is the difference so stark for Class-IL, much less for domain-IL, and barely for task-IL? This is also related to the earlier comment about explaining the difference between these three notions.\n- Section 5.1.3: in Table 2, the performance of FL-DER on CIFAR10 is a bit surprising, going from $B=200$ to $B=500$. In both cases, accuracy improves for w/o Fed-A-GEM with increasing $B$, but is worse for w/ Fed-A-GEM. Can this be explained somehow?\n- Section 5.1.5: I'm not sure of the utility of the 2nd paragraph in this section. Also, the $O(mP)$ complexity does not include the averaging of gradients, while the wall-clock time in seconds does, right?\n- Section 5.1.8: in almost all cases in Table 6, w/o Fed-A-GEM accuracy is smaller than in Table 1. But, in multiple cases, w/ Fed-A-GEM accuracy is larger in Table 6 ($K=20$) than in Table 1 ($K=10$). How do we explain this?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a buffer-based approach for federated general continual learning (when task boundaries are not known). The authors have extended an existing Averaged-Gradient Episodic Memory (A-GEM) algorithm to the federated setting. \n\nThe clients maintain in their memory, a buffer of relevant past samples, which are used to construct local buffer gradients. The server aggregates them and uses this global buffer gradient to constrain the direction of the update based on the current samples. \n\nThe paper is impressive because of the extensive experiments the authors have done.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Additional explanation for a few things requested above", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "tWQ4xhHAEI", "reviewer": "Reviewer_PahG", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n* The proposed problem is very interesting.\n* The authors explain prior works in detail.\n* The paper includes a comprehensive experiment setting.\n\nCons:\n* Novelty of the paper is limited. Particularly, the work is apparently a simple extension of the A-GEM method in a federated setting. The authors do consider the privacy implications, and the proposed Fed-A-GEM is compatible with the standard secure aggregation protocols. However, the paper seems to be a simple combination of relevant prior algorithms.   \n* Do all clients participate in each round? Assuming that all the clients are always available does not seem very realistic. In most FL settings, only some clients participate in each round. \n* This is perhaps fundamental to the general continual learning setting. In a class incremental setting, is the total number of classes known in advance? Is that practical? What if the total classes are not known in advance, how would that change the choice of model architectures?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper targets the federated setting where the clients' datasets can change during the course of training. The training dataset is composed of tasks. While previous works focus on the setting where the clients are aware of the task boundaries, the authors in this paper consider a more realistic setting where task boundaries are not explicitly known. Their proposed framework, Fed-A-GEM, is essentially a federated version of a prior A-GEM method. Particularly, in the A-GEM method, gradient over a buffer of samples is used as a reference to project the current gradient, which avoids increasing the model error over the buffer samples. This in turn allows improves the model performance in continual learning. In Fed-A-GEM, the key contribution is that the server obtains the global reference gradient by aggregating the local reference gradients computed by the clients over their local buffers. Each client then utilizes the global reference gradient to remove the undesired component of the local gradients during local model updates. As a result, the global buffer gradient guides the clients towards local model updates that do not hurt the model performance of the buffer samples.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* While the reviewer acknowledges that Fed-A-GEM provides good performance, it is important for the authors to identify and discuss the algorithmic novelty in more detail.\n* Provide experimental results for FL settings where only a random subset of clients participate in each communication round.\n* Provide in-depth information for the implementation in Section 5.1.6. For example, it is not clear Class-IL setting, how are the classes incrementally locally and globally?\n* The idea of using reference gradients for improving model performance has been previously developed in Byzantine FL works such as [1] and [2]. The authors should provide references to such works. \n* In [1], the idea of using weighted aggregation is developed. Specifically, the local update of a client is weighted according to the similarity of the local update with the reference gradient. The authors should provide a weighted variant of Fed-A-GEM to see if that improves performance.\n\n[1] Cao X, Fang M, Liu J, Gong NZ. Fltrust: Byzantine-robust federated learning via trust bootstrapping. arXiv preprint arXiv:2012.13995. 2020 Dec 27.\n[2] Prakash S, Hashemi H, Wang Y, Annavaram M, Avestimehr S. Secure and fault tolerant decentralized learning. arXiv preprint arXiv:2010.07541. 2020 Oct 15.", "broader_impact_concerns": "The reviewer doesn't have any concern that would require a Broader Impact Statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "3ewxkcwB55", "reviewer": "Reviewer_fioA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. Fed-A-GEM leverages historical data (via shared buffers) from previous tasks, which helps in preserving knowledge without needing to identify task boundaries explicitly.\n\n2. The method is designed to integrate seamlessly with other CFL approaches, enhancing their performance by providing a robust framework against forgetting and improving overall learning stability.\n\n3. Comprehensive experiments have been conducted to verify the effectiveness of Fed-A-GEM across a wide range of classification tasks, including both image and text domains, ensuring the method's versatility.\n\nWeaknesses:\n\n1. Lack of a Comprehensive Architecture Diagram: - The paper does not include an overall architecture figure, which could limit the reader's understanding of the method's workflow and integration points. Such visual aids are crucial for summarizing complex systems and facilitating quicker and clearer comprehension.\n\n2. Inefficiency in Secure Aggregation: - The use of secure aggregation, while beneficial for privacy-preserving purposes, is mentioned to waste time and computational resources. The article fails to provide a detailed analysis of the time complexity associated with this process. An explicit breakdown of the computational costs and the impact on overall system efficiency would be beneficial for evaluating the practical viability of deploying Fed-A-GEM in resource-constrained environments.\n\n3. Utilization of Out of Distribution (OOD) Datasets: - The article employs datasets such as PACS and Office-Caltech-10, which are considered Out of Distribution for many common tasks in federated continual learning. The use of OOD datasets can challenge the generalizability of the findings, as the models might not perform as well when applied to data distributions that are more representative of typical real-world scenarios. Further validation on more standard and diverse datasets could enhance the credibility and applicability of the results.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This submission introduces Fed-A-GEM, a novel adaptation of the A-GEM method tailored for Continual Federated Learning (CFL). By employing a buffer-based gradient projection strategy, Fed-A-GEM effectively mitigates catastrophic forgetting, a prevalent issue where models lose previously acquired knowledge upon learning new information. Unlike existing CFL algorithms, Fed-A-GEM does not rely on predefined task boundaries and leverages local buffer samples along with aggregated buffer gradients to preserve knowledge across diverse clients. This methodology facilitates seamless integration with existing CFL techniques, enhancing their performance substantially. The paper's empirical validations demonstrate considerable performance improvements, with scenarios like task-incremental learning on the CIFAR-100 dataset showing up to a 27% increase in accuracy. The introduction of Fed-A-GEM represents a significant advancement in managing the typical challenges of decentralized, continuous data streams in federated settings, contributing both to academic research and practical applications in federated learning environments.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Inclusion of a Comprehensive Architecture Diagram: Please include a detailed architecture diagram that illustrates the entire workflow and integration points of Fed-A-GEM within the context of Continual Federated Learning (CFL).\n\n2. Analysis of Time Complexity for Secure Aggregation: Amend the section on secure aggregation to include a detailed analysis of its time complexity, along with computational resource requirements. If possible, compare these metrics with other non-secure and secure methods to contextualize the efficiency of Fed-A-GEM.\n\n3. Validation Using More Representative Datasets: Conduct additional experiments using OOD datasets like PACS and Office-Caltech-10.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Mostly see above.

Novelty of the algorithmic innovation, while noted, played no part in the individual reviewers' assessments, nor mine.  Even on that front, straightforward combinations of existing methods can still represent novelty (when a new combination or used in a new problem), especially if it gives impressive results on impactful problems.
Decision: Accept (poster)