Federated Continual Learning with Weighted Inter-client Transfer
Authors: Jaehong Yoon
Wonyong Jeong
Giwoong Lee
Eunho Yang
Sung Ju Hwang
Venue:  (2021)
Keywords: Continual Learning
Federated Learning
Deep Learning

Abstract: There has been a surge of interest in continual learning and federated learning, both of which are important in deep neural networks in real-world scenarios. Yet little research has been done regarding the scenario where each client learns on a sequence of tasks from a private local data stream.  This problem of federated continual learning poses new challenges to continual learning, such as utilizing knowledge from other clients, while preventing interference from irrelevant knowledge.  To resolve these issues, we propose a novel federated continual learning framework, Federated Weighted Inter-client Transfer (FedWeIT), which decomposes the network weights into global federated parameters and sparse task-specific parameters, and each client receives selective knowledge from other clients by taking a weighted combination of their task-specific parameters.FedWeITminimizes interference between incompatible tasks, and also allows positive knowledge transfer across clients during learning. We validate ourFedWeITagainst existing federated learning and continual learning methods under varying degrees of task similarity across clients, and our model significantly outperforms them with a large reduction in the communication cost.

Under review as a conference paper at ICLR 2021
FEDERATED CONTINUAL LEARNING WITH WEIGHTED INTER-CLIENT TRANSFER
Anonymous authors Paper under double-blind review
ABSTRACT
There has been a surge of interest in continual learning and federated learning, both of which are important in deep neural networks in real-world scenarios. Yet little research has been done regarding the scenario where each client learns on a sequence of tasks from private local data stream. This problem of federated continual learning poses new challenges to continual learning, such as utilizing knowledge from other clients, while preventing interference from irrelevant knowl- edge. To resolve these issues, we propose a novel federated continual learning framework, Federated Weighted Inter-client Transfer (FedWelT), which decom- poses the network weights into global federated parameters and sparse task-specific parameters, and each client receives selective knowledge from other clients by taking a weighted combination of their task-specific parameters. FedWelT mini- mizes interference between incompatible tasks, and also allows positive knowledge transfer across clients during learning. We validate our FedWelT against existing federated learning and continual learning methods under varying degrees of task similarity across clients, and our model significantly outperforms them with a large reduction in the communication cost.
INTRODUCTION
Continual learning (Thrun, 1995; Kumar & Daume III, 2012; Ruvolo & Eaton, 2013; Kirkpatrick et al., 2017; Schwarz et al., 2018) describes a learning scenario where a model continuously trains on sequence of tasks; is inspired by the human learning process, as person learns to perform numerous tasks with large diversity over his/her lifespan, making use of the past knowledge to learn about new tasks without forgetting previously learned ones. Continual learning a long-studied topic since having such an ability leads to the potential of building general artificial intelligence. However, there are crucial challenges in implementing with conventional models such as deep neural networks (DNNs), such as catastrophic forgetting, which describes the problem where parameters or semantic representations learned for the past tasks drift to the direction of new tasks during training. The problem has been tackled by various prior work (Kirkpatrick et al., 2017; Lee et al., 2017: Shin et al., 2017; Riemer et al., 2019). More recent works tackle other issues, such as scalability or order-robustness (Schwarz al., 2018; Hung al., 2019; Yoon et al., 2020).
However, all of these models are fundamentally limited in that the models can only learn from its direct experience they only learn from the sequence of the tasks they have trained on. Contrarily, humans can learn from indirect experience from others, through different means (e.g. verbal communications, books, or various media). Then wouldn't 't it be beneficial to implement such ability to a continual learning framework, such that multiple models learning on different machines can learn from the knowledge of the tasks that have been already experienced by other clients? One problem that arises here, is that due to data privacy on individual clients and exorbitant communication cost, it may not be possible to communicate data directly between the clients or between the server and clients. Federated learning (MeMahan et al., 2016; Li et al., 2018; Yurochkin et al., 2019) is a learning paradigm that tackles this issue by communicating the parameters instead of the raw data itself. We may have a server that receives the parameters locally trained on multiple clients, aggregates it into a single model parameter, and sends it back to the clients. Motivated by our intuition on learning from indirect experience, we tackle the problem of Federated Continual Learning (FCL) where we perform continual learning with multiple clients trained on private task sequences, which communicate their task-specific parameters via global server. Figure (a) depicts example

---

Under review as a conference paper at ICLR 2021
Target Client on SVHN
Public Newth Spstem (Centrot Server)
Hospital A
Hospital B
Hosp
SNNV
Assistont Client
Positive Knowledge Transfer
Accuracy (%6
Torget Client
PR
80
78
Knowlodge Interference
Fast Thenseennc
3 5 8
Coartheety Trenelerng Indruct Experterous
CONNST 120)
CHMP
Disruptive Client
Epach
(a) Federated Continual Learning
(b) Challenges of Federated Continual Learning
Figure I: (a): Concept. A continual learner at hospital which leams on sequence disease prediction tasks may want to utilize relevant task parameters from other hospitals. FCL allows such inter-client knowledge transfer via the communication of task-dccomposed parameters. (b): Challenge of FCT., Interference from other elients, resulting from sharing irrelevant knowledge, may hinder optimal training of target clients (Red) while relevant knowledge from other clients will be beneficial for their learning (Green).
scenario of FCL. Suppose that we are building network of hospitals, each of which has disease diagnosis model which continuously learns to perform diagnosis given CT scans, for new types of diseases. Then, under our framework, any diagnosis model which has learned about new type of disease (e.g. COVID-19) will transmit the task-specific parameters to the global server, which will redistribute them to other hospitals for the local models utilize. This allows all participants to benefit from the new task knowledge without compromising the data privacy.
Yet, the problem of federated continual learning also brings new challenges. First, there is not only the catastrophic forgetting from continual learning, but also the threat of potential interference from other clients. Figure (b) describes this challenge with the results of simple experiment. Here, we train a model for MNIST digit recogmition while communicating the parameters from another client trained on a different dataset. When the knowledge transferred from the other client relevant the target task (SVHN), the model starts with high accuracy, converge faster and reach higher accuracy (green line), whereas the model underperforms the base model if the transferred knowledge is from a task highly different from the target task (CIFAR-10. red line). Thus, we need to selective utilize knowledge from other clients to minimize the inter-client interference and maximize inter-client knowledge transfer. Another problem with the federated learning is efficient communication, as communication cost could become excessively large when utilizing the knowledge of the other clients, since the communication cost could be the main bottleneck in practical scenarios when working with edge devices. Thus we want the knowledge to be represented as compactly as possible.
To tackle these challenges, we propose novel framework for federated continual learning, Federated Weighted Inter-client Transfer (FedWelT). which decomposes the local model parameters into a dense base parameter and sparse task-adaptive parameters. FedWeIT reduces the interference between different tasks since the base parameters will encode task-generic knowledge, while the task-specific knowledge will be encoded into the ads-tide parameters. When we utilize the generic knowledge, we also want the client to selectively utilize task- knowledge obtained at other clients. To this end, we allow each model to take a weighted combination of the task-adaptive parameters broadcast from the server, such that it can select task-specific knowledge helpful for the task at hand. Fed communication-efficen
since the task adaptivee parameters are highly spurse and only need to be communicated once when created. Moreover, when communication efficiency is not critical issue as in cross-silo federated learning (Kairouz et al., 2019), we can use our framework to incentivize each client based on the attention weights on its task-adaptive parameters. We validate our method on multiple different scenarios with varying degree of task similarity across clients against various federated learning and local continual learning models. The results show that our model obtains significantly superior performance over all baselines, adapts faster to new tasks, with largely reduced communication cost. The main contributions of this paper are as follows:
We introduce a new problem of Federated Continual Learning (FCL), where multiple mod- cls continuously learn on distributed clients, which poses new challenges such as prevention of inter-client interference and inter-client knowledge transfer.
We propose novel and communication--ffice ing, which allows each elient to adaptively update the federated parameter and selectively utilize the past knowledge from other clients, by communicating sparse parameters.
framework for federated continual learn-
2

---

Under review as a conference paper at ICLR 2021
2 RELATED WORK
While continual learning (Kumar & Daume III, 2012; Ruvolo & Eaton, 2013) is a long-studied topic with a vast literature, we only discuss recent relevant works. Regularization- based: EWC (Kirkpatrick et al., 2017) leverages Fisher Information Matrix to restrict the change of the model parameters such that the model finds solution that is good for both previous and the current task, and IMM (Lee al., 2017) proposes to learn the posterior distribution for multiple tasks mixture of Gaussians. Architecture-based:: DEN (Yoon et al., 2018) 8) tackles this issue by expanding the networks size that are necessary via iterative neuron/filter pruning and splitting, and RCL (Xu & Zhu, 201 8) tackles the same problem using reinforcement learning. APD (Yoon et al., 2020) additively decomposes the parameters into shared and task task-specific parameters to minimize the increase in the network complexity. Coreset- : GEM variants (Lopez-Paz & Ranzato, 2017; Chaudhry et al., 2019) minimize the loss on both of actual dataset and stored episodic memory. FRCL. (Titsias et al., 2020) memorizes approximated posteriors of previous tasks with sophisticatedly constructed inducing points. To the best of our knowledge, none of the existing approaches considered the communicability for continual learning of deep neural networks, which we tackle. CoLLA (Rostami et al., 2018) aims at solving multi-agent lifelong learning with sparse dictionary learning, but is not applicable to federated learning or continual deep learning.
Continual learning
distributed learning framework under differential privacy, which aims to learn a global model on server while aggregating the parameters learned at the clients on their private data. FedAvg (McMahan et al., 2016) aggregates the model trained across multiple clients by computing weighted average of them based on the number of data points trained. FedProx (Li et al., 2018) 8) trains the local models with a proximal term which restricts their updates to be close to the global model. FedCurv (Shoham et al., 2019) aims to minimize the model disparity across clients during federated learning by adopting a modified version of EWC. Recent works Yurochkin al. (2019); Wang al. (2020) introduce well-designed aggregation policies by leveraging Bayesian non- non-parametric methods. A crucial challenge of federated learning is the reduction of communication cost. TWAFL (Chen al., 2019) tackles this problem by performing layer-wise parameter aggregation, where shallow layers are aggregated at every step, but deep layers are aggregated in the last few steps of loop. Karimireddy et al. (2020) suggests an algorithm for rapid convergence, which minimizes the interference among discrepant tasks at clients by sacrificing the local optimality. This is an opposite direction from personalized federated learning methods (Fallah et al., 2020; Lange et al., 2020; Deng et al., 2020) which put more emphasis on the performance of local models. FCL is a parallel research direction to both, and to the best of our knowledge, ours is the first work that considers isssiincremental learning of clients under federated learning framework.
Federated learning Federated learning
3 FEDERATED CONTINUAL LEARNING WITH FEDWEIT
Motivated by the human learning process from indirect experiences, we introduce a novel continual learning under federated learning setting. which we refer to as Federated Continual Learning (FCL). FCL assumes that multiple clients are trained on sequence of tasks from private data stream, while communicating the learned parameters with global server. We first formally define the problem in Section 3.1, and then propose naive solutions that straightforwardly combine the existing federated learning and continual learning methods in Section 3.2. Then, following Section 3.3 and 3.4, we discuss about two novel challenges that are introduced by federated continual learning, and propose novel framework, Federated Weighted Inter-client Transfer ( (FedWelT) which can effectively handle the two problems while also reducing the client-to-sever communication cost.
3.1 PROBLEM DEFINITION
In the standard continual learning (on a single machine), the model iteratively learns from a sequence
of tasks (1( T ..... I where (1)) is a labeled dataset of thh task, (1)) = =xx(), (f) )
which consists of N, pairs of instances x." and their corresponding labels (1) Assuming the most realistic situation, we consider the case where the task sequence is a task stream with an unknown arriving order, such that the model can access T()) only at the training period of task which becomes inaccessible afterwards. Given (1) and the model learned so far, the learning objective at task is as follows: minimizegu (( ((-11) T(0), where (()) is set of the model parameters at task t.
3

---

Under review as a conference paper at ICLR 2021
We now extend the conventional continual learning to the federated learning setting with multiple clients and a global server. Let us assume that we have C clients, where at each client Ce E (c) trains model on a privately accessible sequence of tasks (Te") (1) Te! ..... 7 CT. Please note that there is no relation among the tasks T received at step i, across clients. Now the goal is to effectively train C' continual learning models their own private task streams, via communicating the model parameters with the global server, which aggregates the parameters sent from each client, and redistributes them clients.
3.2 COMMUNICABLE CONTINUAL LEARNING
In conventional federated learning settings, the learning is done with multiple rounds of local learning and parameter aggregation. At each round of communication r, each client ce and the server perform the following two procedures: local parameter transmission and parameter aggregation & broadcasting. In the local parameter transmission step, for randomly selected subset of clients round r, Cir) S {er, C2, ... cc) J. each client c. sends updated parameters ((r) the server. The server-clients transmission is not done at every client because some of the clients may be temporarily disconnected. Then the server aggregates the parameters ( sent from the clients into a single parameter. The most popular frameworks for this aggregation are FedAvg (McMahan et al., 2016) and FedProx (Li et al., 2018). However, naive federated continual learning with these two algorithms on local sequences of tasks may result in catastrophic forgetting. One simple solution is to use a regularization-basee such as Elastic Weight Consolidation (EWC) (Kirkpatrick et al., 2017), which allows the model to obtain a solution that is optimal for both the previous and the current tasks. There exist other advanced solutions (Rusu al., 201 6; Nguyen et al., 2018; Chaudhry et al., 2019) that successfully prevents catastrophic forgetting. However, the prevention of catastrophic forgetting at the client level is an orthogonal problem from federated learning.
Thus we focus on challenges that newly arise in this federated continual learning setting. In the federated continual learning framework, the aggregation of the parameters into a global parameter
I allows inter-client knowledge transfer across clients, since a task T,(9) learned at client e; at round q may be similar or related to ( learned at client Cj round r. Yet, using single aggregated parameter Ig may be suboptimal in achieving this goal since knowledge from irrelevant tasks may not to be useful or even hinder the training at each client by altering its parameters into incorrect directions, which we describe inter-client interference. Another problem that is also practically important, is the communication-effieii server, and server client will incur large communication cost, which will be problematic for the continual learning setting, since the clients may train on possibly unlimited streams of tasks.
Both the parameter transmission from the client to the
3.3 FEDERATED WEIGHTED
TRANSFER
How can we then maximize the knowledge transfer between clients while minimizing the inter-client interference, and communication cost? We now describe our model, Federated Weighted Inter-client Transfer (FedWelT), which can resolve the these two problems that arise with a naive combination of continual learning approaches with federated learning framework.
The main cause of the problems, as briefly alluded to earlier, is that the knowledge of all tasks learned at multiple clients is stored into single set of parameters 0g.. However, for the knowledge transfer to be effective, each client should selectively utilize only the knowledge of the relevant tasks that trained at other clients. This selective transfer is also the key to minimize the inter-client interference as well as it will disregard the knowledge of irrelevant tasks that may interfere with learning.
We tackle this problem by decomposing the parameters, into three different types of the parameters with different roles: global parameters (0c) that capture the global and generic knowledge across all clients, local base parameters (B) which capture generic knowledge for each client, and task-adaptive parameters (A) for each specific task per client, motivated by Yoon et al. (2020). A set of the model parameters (()) for task at continual learning client e, is then defined as follows:
( =
B
+ A
ΣΣ
!
(2)
VEC jKM
where BBC) 1 is the set of base parameters for 2h client shared across all tasks in the
4

---

Under review as a conference paper at ICLR 2021
Central Server
Central Server
0c
Heigleed Intendean Townkr
Clieet
Cloen
Client ei
C
(a) Communication of General Knowledge
Communication of Task-adaptive Knowledge
Figure 2: Updates of FedWelT. (a) A client sends sparsified federated parameter B. O m! After that, the server redistributes aggregated parameters to the clients. (b) The knowledge base stores previous tasks- aaasttad parameters of clients, and each client selectively utilizes them with an attention mask.
client, m E (IRO0) Jim is the set of sparse vector masks which allows to adaptively transform B(O)) for the task t, A E (R/x00) Ji=1 is the of sparse adsk-adepiive parameters at client c.. Here, T is the number of the layer in the neural network, and It, O₁ are input and output dimension of the weights at layer respectively.
The first term allows selective utilization global knowledge. We want the base parameter B, at each client to capture generic knowledge across all tasks across all clients. In Figure (a), we initialize at each round with the global parameter from the previous iteration, 0 which aggregates the parameters sent from the client. This allows
(t) to also benefit from the global knowledge about all the tasks. However, since 0 also contains knowledge irrelevant to the current task, instead of using it as is, we learn the sparse mask m (t) to select only the relevant parameters for the given task. This sparse parameter selection helps minimize inter-client interference, and also allows for efficient communication. The second term is the task-adaptive parameters (!)), Since we additively decompose the parameters, this will learn to capture knowledge about the task that is not captured by
the first term, and thus will capture specific knowledge about the task (t The final term describes weighted inter-client knowledge transfer. We have a set of parameters that are transmitted from the server, which contain all task- aaa-tadativee parameters from all the clients. To selectively utilizes these
indirect experiences from other clients, we further allocate attention a! these parameters, to take a weighted combination of them. By learning this attention, each client can select only the relevant task ada-tivee parameters that help learn the given task. Although we design A to be highly sparse, using about 2 3% of memory of full parameter in practice, sending all task knowledge is not desirable. Thus we only transmit the task-adaptive parameter of the previous task (t 1), which we empirically find to achieve good results in practice.
Training.
We learn the decomposable parameter o( by optimizing for the following objective:
minimize B m!.) A
C ((0!) Am (
JBB m I
(4)
(2)
where L is a loss function and 22-)) is sparsity-inducing regularization term for all task-ada parameters and the masking variable (we use 4 -norm regularization), to make them sparse. The second regularization term is used for retroactive update of the past task-adaptive parameters, which helps the task-adaptin adaptivee parameters to maintain the original solutions for the target tasks, by reflecting the change of the base parameter. Here, (B(!) ((*) B(t-1) is the difference between the base (5)
parameter at the current and previous timestep, and A
is the difference between the task-adaptive parameter for task at the current and previous timestep. This regularization is essential for preventing catastrophic forgetting. A and Ag are hyperparameters controlling the effect of the two regularizers.
3.4 EFFICIENT COMMUNICATION VIA SPARSE PARAMETERS
FedWeIT learns via server-to-ciee communication. As discussed earlier, a crucial challenge here is to reduce the communication cost. We descrihe what happens the client and the server each step.
5

---

Under review as a conference paper at ICLR 2021
Algorithm Federated Weighted Inter-client Transfer
Lobels
input Dataset {D(2:()
output {Be, m (1:t) Je=1 and Global Parameter Og:
Srampier
ac (1:t)
A (
D
Da
1: Initialize B. to c for all c CEC= (I, C}
D2
D2
2: for task t = 1, do
3: for round 1,
do
4: Transmit ((.r)
and A! (1-18)
of client de to server (tr)
Figure 3: Configuration of task sequences: We first split datasct D into multiple sub-tasks in non-Ill manner ((a) and (b)). Then, we distribute them to multiple clients (C(). Mixed tasks from multiple datasets (colored circles) are distributed across all clients ((c)).
5: Compute 0g [ Lege Be
Distribute 0 and (A (A(-1.")))
Jeee to client e
7: Minimize Eq. (2) for solving each local CL problems
8: end for
9: end for
At each round r. each client ce partially updates its base parameter with the nonzero components of the global parameter sent from the server; that is, Br(n) (n) where is nonzero element the global parameter. After training the model using Eq. (2), it obtains a sparsified (4)
Client:
base parameter B
m and task-adaptive parameter (t) for the new task, both which sent to the server, at smaller cost compared to naive FCL baselines. While naive FCL baselines
_
require [C] x R [0] for client-to-server communication, FedWelT requires |C] x (R
(A])
where is the number of communication round per task and is the number of parameters.
Server: The server first aggregates the base parameters sent from all the clients by taking an weighted average of them: 0G Ee B." Then, it broadcasts @ to all the clients. Task adaptive parameters of 1, (A (t-1); Cie. Vi=1 are broadcast at once per client during training task t. While naive FCL baselines requires [C] R x |@) for server-to-client communication cost, FedWeIT requires [C] x (R x 00c + (CC 1) x |A) in which Oc, A are highly sparse. We describe the FedWeIT algorithm in Algorithm 1. For a detailed version the algorithm, please see Section D in appendix.
4 EXPERIMENTS
We validate Wel under different configurations of task sequences against baselines which namely Overlapped-CIFAR-IO0 and NonIID-50. 1) Overlapped-CIFARR R-100: We group 100 classes of CIFAR-101 dataset into 20 non-iid superclasses tasks. Then, we randomly sample 10 tasks out of 20 tasks and split instances to create a task sequence for each of the clients with overlapping tasks. 2) NonIID-50: We use the following eight benchmark datasets: MNIST (LeCun al., 1998), CIFAR- 10/-100 (Krizhevsky & Hinton, 2009), SVHN (Netzer et al., 2011), I), Fashion-MNIST (Xiao et al., 2017), Not-MNIST (Bulatov, 2011), FaceScrub (Ng Winkler, 2014), and TraffieSigns (Stallkamp et al., 2011) We split the classes in the 8 datasets into 50 non-IlI tasks, each of which is composed of classes that are disjoint from the classes used for the other tasks. This a large-scale experiment, containing 280, 000 images of 293 classes from 8 heterogeneous datasets. After generating and processing tasks, we randomly distribute them to multiple clients as illustrated in Figure 3.
Experimental setup We use modified version of LeNet (LeCun et al., 1998) for the experiments with both Overlapped-CIFARR with NonIID-50 dataset. We followed other experimental setups from Serrà al. (2018) and Yoon et al. (2020). For detailed descriptions of the task configuration and hyperparameters used, please see Section B in appendix. Also, for more ablation studies, please see Section C in appendix.
and NonIII>-00 dataset. Further, we use ResNet-18 He al. (2016)
Baselines and our model 1) STL: Single Task Learning at each arriving task. 2) Local-EWC: In- dividual continual learning with EWC (Kirkpatrick et al., 2017) per client. 3) Local-APD: Individual continual learning with APD (Yoon et al., 2020) per client. 4) FedProx: FCL using FedProx (Li et al., 2018) algorithm. 5) Scaffold: FCL using Scaffold (Karimireddy al., 2020) algorithm. 6) FedCurv: FCL using FedCury (Shoham et al., 2019) algorithm. 7) FedProx- FCL, that is trained using FedProx algorithm with [model]. 8) Wel:: Our FedWelT algorithm.

---

Under review as a conference paper at ICLR 2021
Table I: Averaged Per-task performance on both dataset during FCL with 5 clients (fraction=1.0), We mcasured task accuracy and model size after completing all learning phases over 3 individual trials. We also measured C2S/S2C communication cost for training each task.
No Dataset (F=1.0, R=20)
Overlapped CIFAR- 100
[F.00 R=20)
Methods
Accuracy
Model Size
C2S/S2C Cost
Accuracy
Model Size
C2S/S2C Cost
STL
85,78 0.17
0.610
NIA
57.15 0:00
0.610
N/A
Local-EWC
74.30 1 0.5
7..06] GB
'NA
44.226 1 6.4
0.061 Ga
'NAA"
Local-APD
81.42 + 0,72
0.090 QB
N/A
50.82 + 0.33
0.073 GB
N/A
FedlProx
63.69 = L.75
0.006 GB
1122/7..22 aB
33.83 1 0.48
0.061 Gn
7.22/
Scaffold
30.84 = 1.11
0.061
2.44 3.44/2.44.0 GR
22.80 + 0.47
0.061
2.44/ GB
FedCurv
72.39 0.32
0.061
1.22 / 1.22 GR
40.36 0.44
0.061
1.22 / 1.22 GB
FedProx-EWC
68.18 = 0.58
0.061 GB
1.22 / 1.22 GB
41.91
+ 0.47
0,061 GB
1.22 / 1.22 GB
FedProx-APD
81.20
= 1.24
0.079 GB
1.22 / 1.22 GR
52.20
+ 0.41
0.075 GB
1.22 / 1.22 GB
FedWeIT
84.11
0.078
0.37
1.07 GB
55.16
0.19
0.075
0.37 / 1.07 GB
gog SClents
dar
Test acc (50
100 clients (F=0.05, R-20, 1,000 tasks in total)
APD
PedWWeT
APD
FedWeiT
Methods
Accuracy
Model Size
C2S/S2C Cost
5
25
20
STL
32.96 + 0:033
12.20 GB
N/A
Local-APD
37.50
4.01 cB
N/A
Test AcC. (9
100 Clbb
(N)) The 1in
FeaProx
24.1
7.22
7.227 T.22 GB
FedCury
29.11 0.00
1.22 cn
1.22 / 1.22 cB
APD
PedWWel
S
APD
TedkietT
FedCury-EWC
29.72 0.20
1,22 on
1.22 / 1.22 GB
FedWeIT
39.58 + 0.27
4.03 GB
0.38 / 1.10 GB
15
20
13
Figure 4: Left: Averaged task adaptation during training last two (9th and 10th) tasks with 5 and 100 clients. Right: Average Per-task Performance on Overlapped-CTFAR--00 during FCL with 100 clients.
4.1 EXPERIMENTAL RESULTS
We first validate our model on both Overlapped-CIFARR CIFAR-100 and NonIID-50 task sequences against single task learning (STL), continual learning (EWC, APD), federated learning (FedProx, Scaffold, FedCurv), and naive federated continual learning (FedProx-based baselines. Table shows the final average per-task performance after the completion of (federated) continual learning on both datasets. We observe that FedProx-bet based federated continual learning (FCL) approaches degenerate the performance of continual learning (CL.) methods over the same methods without federated learning. This is because the aggregation of all client parameters that are learned on irrelevant tasks results in severe interference in the learning for each task, which leads to catastrophic forgetting and suboptimal task adaptation. Scaffold achieves poor performance on FCL, as its regularization on the local gradients is harmful for FCL, where all clients learn from a different task sequences. While FedCurv reduces inter-task disparity in parameters, it cannot minimize inter-task interference, which CL methods. On the other hand, FedWelT significantly
results it to underperform sinle-machinn outperforms both single-machine CL. baselines and naive FCL. baselines on both datasets. Even with larger number of clients (C = 100), FedWelT' consistently outperforms all baselines (Figure 4). This improvement largely owes to FedWeIT's ability to selectively utilize the knowledge from other clients to rapidly adapt to the target task, and obtain better final performance (Figure 4 Left).
The fast adaptation new task is another clear advantage of inter-client knowledge transfer. To further demonstrate the practicality of our method with larger networks, we experi- ment on Non-IID dtaset with ResNet-18 (Table 2), on which FedWelT still significantly outperforms the strongest base- line (FedProx-ABl)
Table 2: FCL. results on NonIID-50 dataset with 18.Net-18.
ResNet-18
Methods
Acc.
M Size
while using fewer parameters. Also, our model is not sensitive to the hyperparameters ê1 and A2. if they are within reasonable scales (Figure 6 Left).
Local-APD
92.44 %
1.86
FedProx-APD
92,89 %
2.05 aB
FedWeIT
94,86 %
1.84 GB
Efficiency of FedWeIT We also report the accuracy as a function of network capacity in Table 1, 2, which we measure by the number of parameters used. We observe that FedWeIT obtains much higher accuracy while utilizing less number of parameters compared to FedProx-APD, This efficiency mainly comes from the reuse of adsk-idae parameters from other clients, which is not possible with single-man machine CL methods naive FCL methods. We also examine the communication cost (the size of non-zero parameters transmitted) of each method. Table I reports both the client-to-server (C2S) server-to-client (S2C) communication cost at training each task. FedWelT, uses only 30%
7

---

Under review as a conference paper at ICLR 2021
NonilD-S0 50
Accuracy (N0
2122
Saar
-3-
Fec APD
-2
FidMETiOan! Puckog-rr
20 60
60
100
80 150
Communication Cost (%)
Communication Cast (8)
(a) Accuracy over client-to-server cost
(b) Inter-client Knowledge Transfer of FedWelT
Figure 5: (a) Accuracy over C2S cost. We report the relative communication cost to the original network. All results are averaged over the clients. (b) Inter-client transfer for NonlII--50. We compare the scale of the
attentions at first FC layer which gives the weights on transferred task adaptive parameters from other clients.
Non
5.
Ap: 40:4
Ag:
Models
BWT
Accuraoy
A,000
Accuracy 186)
80
Accuracy (96)
Lecal-APD
Local-EWC
-0.004639
0.045395
A210.3
FedCurs
0.191666
60
FedCurv-EWC
-0.186957
3e-3
As Ot
A: 04g
31
1009
60
75
FedPrrx-APD
FedProx-EWC
-0,080083
0.002015
sio
6
8 Task
10
10
FedWarf
00.000655
Copacity(%s
Task
Figure 6: Left: Performance of FedWelT with different scale of hyperparameters on Non-iid 50. Middle:
Performance comparison about current task adaptation 6th and 8 tasks during federated continual learning on NonlID-50. Right: Forgetting measure using Backward Transfer (BWT).
and 3% of parameters for B and A of the dense models respectively. We observe that FedWeIT is than FCL baselines although it broadcasts task-adaptive
significantly more communication-effiiiee parameters, due to high sparsity of the parameters. Figure 5 (a) shows the accuracy as a function of C2S cost according transmission of top-x% informative parameters. Since FedWeIT selectively utilizes task-specific parameters learned from other clients, it results in superior performance over APD-baselines especially with sparse communication of model parameters.
Catastrophic forgetting Further, we examine how the performance of the past tasks change during continual learning, to see the severity of catastrophic forgetting with each method. Figure 6 Left shows the performance of FedWelT and FCL baselines on the 6th and 8th tasks, at the end of training for later tasks. We observe that naive FCL baselines suffer from more severe catastrophic forgetting than local continual learning with EWC because of the inter-client interference, where the knowledge of irrelevant tasks from other clients overwrites the knowledge of the past tasks. Contrarily, our model shows sign of catastrophic forgetting. This is mainly due to the selective utilization of the prior knowledge learned from other clients through the global/task-adappiin parameters, which allows it to effectively alleviate inter-client interference. FedProx-APe also does not suffer from catastrophic forgetting, but they yield inferior performance due to ineffective knowledge transfer. We also report Backward Transfer (BWT), which a measure on catastrophic forgetting for all models (more positive the better). We provide the details of BWT in the Section B in appendix.
Weighted inter-client knowledge transfer By analyzing the attention a in Eq. (1), we examine which task parameters from other clients each client selected. Figure 5 (b), shows example of the attention weights that are learned for the 0th split of MNIST and 10th split of CIFAR-I00. We observe that large attentions are allocated to the task parameters from the same dataset (CIFAR- 1000 utilizes parameters from CIFAR- 100 tasks with disjoint classes), or from a similar dataset (MNIST utilizes parameters from Traffic Sign and SVHN). This shows that FedWeIT effectively selects beneficial parameters to maximize inter-client knowledge transfer. This is an impressive result since it does not know which datasets the parameters are trained on.
5 CONCLUSION
We tackled a novel problem of federated continual learning, whose goal is to continuously learn local models at each client while allowing it to utilize indirect experience (task knowledge) from other clients. This poses new challenges such as inter-client knowledge transfer and prevention of inter- client interference between irrelevant tasks. To tackle these challenges, the model parameters at each client into the global parameters that are shared across all clients, and sparse local task-adaptive parameters that are specific to each task. Further, allowed each
additively decomposed
8

---

Under review as a conference paper at ICLR 2021
model to selectively update the global task-shared parameters and selectively utilize the task-adaptive parameters from other clients. The experimental validation of our model under various task similarity across clients, against existing federated learning and continual learning baselines shows that our model obtains significantly outperforms baselines with reduced communication cost. We believe that federated continual learning is practically important topic of large interests to both research communities of continual learning and federated learning, that will lead to new research directions.
REFERENCES
Yaroslav Bulatov. Not-mnist dataset. 2011.
Arslan Chaudhry, Mare' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. Proceedings of the International Conference on Learning Repre- sentations (ICIR), 2019.
Yang Chen, Xiaoyan Sun, and Yaochu Jin. Communication-efficinnn federated deep learn- ing with asynchronous model update and temporally weighted aggregation. arXiv preprint arXiv:1903.07424, 2019.
Yuyang Deng, Mohammad Mahdi Kamani, and Mehrdad Mahdavi. Adaptive personalized federated learning. arXiv preprint arXiv:2003.13461 2020.
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized federated learning: A meta- learning approach. arXiv preprint arXiv:2002.07948, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Ching-Yr Yi Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu-Song Chen. Compacting, picking and growing for unforgetting continual learning. In Advances in Neural Information Processing Systems (NIPS), 2019.
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. arXiv preprint (rXiiv:192.040: 2019.
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J Reddi, Sebastian U Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for on-device federated learning. In Proceedings of the International Conference on Machine Learning (ICML), 2020.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwing et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, pp. 201611835, 2017.
Alex Krizhevsky and Geoffrey E. Hinton. Learning multiple layers of features from tiny images. Technical report, Computer Science Department, University of Toronto, 2009.
Abhishek Kumar and Hal Daume III. Learning task grouping and overlap in multi-task learning. In Proceedings of the International Conference on Machine Learning (ICML), 2012.
Matthias De Lange. Xu Jia, Sarah Parisot, Ales Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. Unsupervised model personalization while preserving privacy and scalability: An open problem. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998.
Sang-Woo Woo Lee, Jin-Hwa Kim, Jachyun Jun, Jung-Wor Woo Ha, and Byoung- Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. In Advances in Neural Information Processing Systems (NIPS), 2017.

---

Under review as a conference paper at ICLR 2021
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith.
Federated optimization in heterogeneous networks. arXiv preprint arXiv: 181.00611.. 2018.
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems (NIPS), 2017.
H Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, et al. Commumication--ffiiieii learning of deep networks from decentralized data. arXiv preprint arXiv:1602.05629 2016.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Hong-We Wei Ng and Stefan Winkler. A data-driven approach to cleaning large face datasets. In 2014
IEEE international conference on image processing (ICIP), pp. 343-347. IEEE, 2014.
Cuong V. Nguyen, Yingzhen Li, Thang D. Bui, and Richard E. Turner. Variational continual learning. In Proceedings of the International Conference Learning Representations (ICLR), 2018.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In Proceedings of the International Conference on Learning Representations (ICLR), 2019.
Mohammad Rostami, Soheil Kolouri, Kyungnam Kim, and Eric Eaton. Multi-agent distributed lifelong learning for collective knowledge acquisition. In Proceedings of the 17th International Conference Autonomous Agents and MultiAgent Systems. International Foundation for Au- tonomous Agents and Multiagent Systems, 2018
Andrei Rusu, Neil Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. In Advances in Neural Information Processing Systems (NIPS), 2016.
Paul Ruvolo and Eric Eaton. Ella: An efficient lifelong learning algorithm. In Proceedings of the International Conference Machine Learning (ICML), 2013.
Jonathan Schwarz, Jelena Luketina, Wojeiech M Czarnecki. Agnieszka Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. Progress & compress: A scalable frame- work for continual learning. arXiv preprint arXiv: 1805.06300, 2018.
Joan Serrà, Dídac Surís. Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In Proceedings of the International Conference on Machine Learning (ICML), 2018.
Hanul Shin, Jung Kwon Lee, Jaehon Kim, and Jiwon Kim. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems (NIPS), 2017.
Neta Shoham, Tomer Avidor, Aviv Keren, Nadav Israel, Daniel Benditkis, Liron Mor- Yosef, and Itai Zeitak. Overcoming forgetting in federated learning on non-iid data. arXiv preprint 19X1v:1910.07796. 2019.
Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel. The german traffic sign recognition benchmark: multi-class classification competition. In The 201 international joint conference on neural networks, 2011.
Sebastian Thrun. A Lifelong Learning Perspective for Mobile Robot Control, Elsevier, 1995.
Michalis Titsias, Jonathan Schwarz, Alexander G de G Matthews, Razvan Pascanu, and Yee Whye Teh. Functional regularisation for continual learning with gaussian processes. In Proceedings of the International Conference on Learning Representations (ICLR), 2020.
Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni. Fed- erated learning with matched averaging. In International Conference on Learning Representations, 2020. URL https: : openreview net. forum? d=PkkuqlSFDDS
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv: 2017.
10

---

Under review as a conference paper at ICLR 2021
Ju Xu and Zhanxing Zhu. Reinforced continual learning. In Advances in Neural Information Processing Systems (NIPS), 2018.
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. In Proceedings of the International Conference Learning Representations (ICLR), 2018.
Jaehong Yoon, Saehoon Kim, Eunho Yang, and Sung Ju Hwang. Scalable and order-robust continual learning with additive parameter decomposition. In Proceedings of the International Conference on Learning Representations (ICLR), 2020.
Mikhail Yurochkin, Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Trong Nghia Hoang, and Yasaman Khazaeni. Bayesian nonparametric federated learning of neural networks. Proceedings of the International Conference Machine Learning (ICML), 2019.

---

Under review as a conference paper at ICLR 2021
A APPENDIX
Organization The appendix is organized as follows: In Section B, we further describe the ex- perimental details, including the network architecture, hyper-parameter configurations, forgetting measures, and datasets. Also, we report additional experimental results in Section C about the effect of the communication frequency (Section C.I) and an additional ablation studies for model components Overlapped- CAaecd-CIRR----e 100 dataset (Section C.2). We include detailed algorithm for our FedWelT in Section D.
B EXPERIMENTAL DETAILS
We further provide the experimental settings in detail, including the descriptions of the network architectures, hyperparameters, and dataset configuration.
Network architecture We utilize a modified version of LeNet and conventional ResNet-18 18 as the backbone network architectures for validation. In the LeNet, the first two layers are convolutional neural layers of 20 and 50 filters with the 5 x 5 convolutional kernels, which are followed by the two fully- cunlecco layers of 800 and 500 units each. Rectified linear units activations and local response normalization are subsequently applied to each layers. We use 2 x 2 max maxxlooling after each convolutional layer. All layers are initialized based on the variance scaling method. Detailed description of the architecture for LeNet is given in Table 3.
Configurations We use an Adam opti- mizer with adaptive learning rate decay, which decays the learning rate by factor 3 3 for every 5 epochs with no consee- utive decrease in the validation loss. We stop training in advance and start learn- ing the next task (if available) when the learning rate reaches p. The experiment for LeNet with 5 clients, we initialize by 1e-3 3 x 2/ at the beginning of each new task and
Table 3: Base Network Architecture (LeNet) and Total Number of Parameters of both FedWeIT and All Base- line Models. T describes the number of arrived tasks in continual learning.
Layer
Filter Shape
Stride N/A
Output
Input Cony
N/A
32 x 32
5 x 5 x 20
1
32 x 32 x 20
Max Pooling
3 x 3
2
16 x 16 x 20
Conv 2
5 x 5 50
I
16 x 16 x 50
Max Pooling
3
x 8 50
1e-7. Mini-batch size 100, the rounds per task is 20, an the
Flatten
3200
N/A
1 x 3200
FC 1
800
N/A
1 1 x 800
FC
500
N/A
1 1 500
epoch per round is 1. The setting for
Softmax
Classifier
N/A
1 1x1x5x7
R8 is identical, excluding the ini- tial learning rate, 1e In the case of
Total Number of Parameters
3,012,920
experiments with 20 and 100 clients, we set the same settings except reducing minibatch size from
100 to 10 with an initial learning rate 1e_ We use client fraction 0.25 and 0.05, respectively, at each communication round. we set A1 = [11e_1_ =[1e-14e-_) and A2 = 100 for all experiments. Further,
μ 5e-3 for FedProx, A [1e- 1.0 for EWC and FedCurv. We initialize the attention (t) parameter ae as sum to one, o 2. 1/
Backward-transter (BWT) Backward transfer (BWT) is measure for catastrophic forgetting. BWT compares the performance disparity of previous tasks after learning current task as follows:
BWT T
p
p(o)
KT
where p!T) is the performance of task after task T is learned (i T). Thus, large negative backward transfer value indicates that the performance has been substantially reduced, in which case catastrophic forgetting has happened.
For NonIID-50 dataset, we utilize 8 heterogenous datasets and create 50 non-iid tasks total as shown in Table 4. Then arbitrarily select 10 tasks without duplication and distribtue them to clients. The average performance of single task learning on the dataset is 85.78 + 0.17(%), measured by our base LeNet architecture.
12

---

Under review as a conference paper at ICLR 2021
Table 4: Detailed configuration of NonIID--00 Dataset.
NonlID-50
Dataset
# Classes
# Tasks
# Classes (Task)
# Train Set
# Valid Set
# #Test Set
CIFAR-100
100
15
5
36,750
10,500
5,250
Face Scrub
100
16
13,859
3,959
1,979
Traffic Signs
43
9
5 (3)
32,170
9,191
4,595
SVHN
10
2
5
61,810
17.660
8,830
MNIST
10
2
5
42,700
12,200
6,100
CIFAR-10
10
2
5
36,750
10,500
5,250
Not MNIST
11,339
3,239
1,619
Fashion MNIST
10
2
5
42,700
12,200
6,100
Total
293
50
248
278,078
39,723
79,449
Datasets We create both Overlapped-CIFAR 100 and NonIID-50 datasets. For Overlapped-CIEXX 100, we generate 20 non-iid tasks based on 20 superclasses, which hold 5 subclasses. We split instances of 20 tasks according to the number of clients (5, 20, and 100) and then distribute the tasks across all clients.
FedWeiT
Overlapped-CIEAR-1100
Accuracy (96)
Epochs
54.0
Methods
Accuracy
Model Size
C2S/S2C Cost
Round
WelT
55,16 0.19
0.075 GB
0.37 1.07 on
1
53.5
Fed
35.18
0.077 GB
0.19 /0,53
2
FedWeIT
53.73 0.44
0.083 GB
0.08 0.22
5
5
10
Fed
53.22 0.14
0.088 GB
0.02 / 0.07 GB
20
Epoch per Round
Figure 7: Average Per-task Performance with bars the number of training epochs per munication rounds on Overlapped CIFAR- for FedWeIT with clients. models transmit full of local base parameters and highly sparse task adaptive parameters. All results are the mean accuracy over 5 clients and we run 3 individual trials. Red arrows at each point describes the standard deviation of the performance.
C ADDITIONAL EXPERIMENTAL RESULTS
We further include a quantitative analysis about the communication round frequency and additional experimental results across the number of clients.
C.1 EFFECT OF THE COMMUNICATION FREQUENCY
We provide an analysis on the effect of the communication frequency by compar- ing the performance of the model, mea- sured by the number of training epochs per communication round. We run the 4 different FedWelT with 1, 2, 5, and 20 training epochs per round. Figure 7 shows the performance of FedWeIT variants. As clients frequently update the model pa- rameters through the communication with the central server, the model gets higher performance while maintaining smaller network capacity since the model with frequent communication efficiently updates the model parameters as transferring the inter-client knowledge. However, it requires much heavier communication costs than the model with sparser communication. For example, the model trained for 1 epochs at each round may need to about 16.9 times larger entire communication cost than the model trained for 20 epochs at each round. Hence, there is trade-off between model performance of federated continual learning and communication efficiency, whereas FedWelT variants consistently outperform (federated) continual learning baselines.
Table 5: Experimental results on the Overlapped-CIFARR-100 dataset with 20 tasks. All results are the mean accuracies over 5 clients, averaged over 3 individual trials.
Everlapped-CIFATE --00 with 20 tasks
Methods FedProx
Accuracy
M Size
C2S/S2C Cost
29.76 0.39
0.061 GB
1.22 / 1,22 GB
FedProx-EWS
27.80 0.58
0.061
1.22 / 1.22 GB
FedProx-APD
43.80
0.093
1.22 1.22 GB
FedWerT
46.78 0.14
0.092
0.37 1.07 GB
13

---

Under review as a conference paper at ICLR 2021
Ras
pesseesoo
FeeWeil
FeeNeeT
TedWeeT
FedWeiT
FedWeil
cen
Commu kox on Round
Communiction Round
Communcation Roun)
Communcation Hound
Commuricatin Hourd
(a) Performance Comparison between FedWelT and APD with 20 clients
R
Fedieelf
FediVelT
Fie Te-tWelT
FedWeil
FedWeil
APD
APD
Commuriizaoo lon Sound
Carmun Casion Rouns
Commun Cation Sound
Commurkctton Sound
Commun Casion Round
(b) Performance Comparison between FedWelT and APD with 100 clients
Figure 8: (a) Comparison of adaption for tasks between WedWeIT and APD with 20 clients in federated continual learning scenario (b) Comparison of adaptation for tasks between FedWelT and APD with 100 clients in federated continual learning scenario. visualize the last tasks out 10 tasks per client. Overlapped-CIFAR- 100 dataset are used after splitting instances according to the number of clients (20 and 100).
-2- LOCaLARD
FesProssEW
30
Forgelting
Forgell
Accunccy (M
Accurery
15
60
10
6
10
Forgal ling TASK
Fonged mg TASK
Forgelling of TASK
Accinscy 176
(96)
Task
Tisk
Task
Forgetting of N7
Fongesting of TASK #B
Forgetting TATA #9
ACC accurny 180
Aecurery D80
-0
3
80
70+
Task
Task
Figure 9: Forgetting analysis. Performance change over the increasing number of tasks for all tasks except the last task (1 to gdh during federated continual learning on NonIID-50. We observe that our method does not suffer from task forgetting on any tasks.
C.2 ABLATION STUDY FOR MODEL COMPONENTS
We perform an ablation study to analyze the role of each component of our FedWeIT. We compare the performance of four different variations of our model. w/o B communica- tion describes the model that does not transfer the base parameter B and only communi- cates task-adaptive ones. w/o A communication is the model that does not communicate task-adaptive parameters. w/o A is the model which trains the model only with sparse trans- mission of local base parameter, and w/o m is the model without the sparse vector mask. As shown in Table 6, without communi- cating B or A. the model yields signifi- cantly lower performance compared to the full model since they do not benefit from inter-client knowledge transfer. The model w/o obtains very low performance due to catastrophic forgetting, and the model w/o sparse mask m achieves lower accu- racy with larger capacity and cost. which demonstrates the importance of performing selective transmission.
Table 6: Ablation studies to analyze the effectiveness of pa- rameter decomposition on WelT. All experiments performed on NonlID-50 50 dataset.
Non
Methods Fel
Acc.
M Size
C2S/S2C Cost
84.11% %
0.078 cB
0.37 /.07 GB
wo comm.
77.88%
0.070 GB
0.011/ 0.01 cn
w/o A comm.
79.21%
0.079 cB
0.37 / 1.04 om
w/o A
65.66%
0.061 cB
0.37 / 1.04
w/o
78.71%
0.087 GB
1.23 1.25
14

---

Under review as a conference paper at ICLR 2021
D DETAILED ALGORITHM FOR FEDWEIT
Algorithm 2 Algorithm for FedWeIT
input Datasct {De (
Je=1, and Global Parameter Og
output {B., m (2:1) O (1:t)
A
(1:))CC
J
I: Initialize B. 2g for all c C = (1,..,
2: for task 1, 2......
for round r 1, ..., do
4: Select communicable clients chr) C.C.
5: if then
6:
A (1-1,R)
<ecC()) and B Beect transmitted from ((r) to the central server
7:
Sct a new knowledge base (b-(11) {AA
(t-11, Jec())
8:
else
9:
B (tr))
eec(r) are transmitted from ( to the central server
10:
end if
Update ((m)
[CCP] Eeeco (fr)
11:
12:
Distribute 0 and (t-) to client c E (()) Ce meets (b first, otherwise distribute only (-))
13:
Minimize Eq. (2) for solving each local CI. problems
14:
end for
end
E FEDWEIT FOR ASYNCHRONOUS FEDERATED CONTINUAL LEARNING
Algorithm 3 Algorithm for Asynchronous FedWelT
input Datasct (D(
Je=1, and Global Parameter
output (Be, (2:t) (1:t)
(1:t)
(1:0))
]c=
I: Initialize B, Ag all e (1,....)
2: the knowledge hase kb + 0
3: for round r 1,2, do
4: if all clients finished the training then break
else
7: Select communicable clients c(r) C for (ir) do
8:
if new task f' is arrived the client ce then
10:
Update the knowledge base kb + lsb U ( } at the central server end
12:
B_ transmitted from the client c. the central server
14:
Update 000" (C()] ECEco=) B if client Cege(e) still learn then
15:
16:
Distribute g and kbb C kb the client ee if new task arrived, otherwise distribute ((r) Minimize Eq. (2) for solving local CL problems at the client ce
17:
18:
end if
19:
end if
20:
end
Table 7: Averaged performance of FedWelT with synchronous and asynchronous federated continual learning scenario.
We now consider FedWelT under the asynchronous federated continual learning scenario, where there is no synchronization across clients for each task. This is a more realistic scenario since each task may require different training rounds to converge during federated continual learning. Here, asynchronous implies that each task requires different training costs (i.e., time, epochs, or rounds) for training. Under the asynchronous federated learning
N5n
Fed
Accuracy (%)
Synchronous Asynchronous
84.11 1 0.27
84.40 1 0.41
15

---

Under review as a conference paper at ICLR 2021
Synchronous Federated Continual Learning
Asynchronous Federated Continual Learning
80
80
70
70
Accurasy (%)
60
Acruracy
50
client-0
50
client-0
client-1
client-1
40
client-2
40
client-2
client-3
30
client-3
client-4
client-4
20
20
40
60
80
100
120
140
160
180
200
20
40
60 80 100 120 140 160 180 200 220 Communication Round
Communication Round
Figure 10: FedWeIT with asynchronous federated continual learning on Non-iid 50 dataset. We measure the test accuracy all tasks per client.
scenario, FedWeIT transfers any available task-adaptive parameters from the knowledge base (kbb) to each client. We provide the detailed algorithm in Algorithm 3. In Table 7 and Figure 10, we plot the average test accuracy over all tasks during synchronous asynchronous federated continual learning. As shown in Figure 10, different tasks across clients at the same timestep require the same number of training rounds, receiving new tasks and task-adaptive parameters from the knowledge base simultaneously with the synchronous FedWeIT. On the other hand, with asynchronous FedWelT, each task requires different training rounds and receives new tasks and adsptivee parameters an asynchronous manner. The results in Table shows that the performance of asynchronous FedWeIT almost similar to that of the synchronous FedWeIT.
16


Reviews:
[{"review_id": "zz3iNwq-QFk", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a federated continual learning setting where each node has a non-iid stream and a different dataset. The authors address this by extending the parameter decomposition method of Yoon et al\n\nStrengths:\n\n- The authors introduced a relevant new task that introduces concepts from continual learning to federated learning. The task setting seems practical overall as different nodes will often be following a different distribution\n-The algorithm proposed is interesting. I would like further discussion on the differences to Yoon et al\n-The results are good and evaluate a lot of  relevant factors\n\nWeakness:\n\n- Although the setting proposed is interesting, certain aspects of it seem artificial: although it seems very relevant that each node is a different dataset, strong non-iid behavior per node seems not realistic for many settings \n-What happens when some nodes start learning much earlier than others\n- A discussion of challenges in these settings and other potential methods\n- Results are shown for very simple LeNet architecture only\n- The algorithm proposed is interesting, but more motivation and other possible alternatives in this setting would improve the paper\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "nZ5JjMSgkHj", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper investigates a new problem – federated continual learning by Federated Weighted Inter-client Transfer. The key idea is to decompose the network weights into global federated parameters and sparse task-specific parameters such that each client can selectively receive knowledge from other clients by taking a weighted combination of their task-specific parameters. The experiment results in two contrived datasets demonstrate the effectiveness of the proposed method.\n\nStrength:\n+ This paper is well presented and organized.\n+ The proposed federated continual learning framework is innovative and technically sound.\n+  The experiment results are solid.\n\nWeakness:\n- The optimization procedure for Eq. (2) is not provided.\n- Some details are missing (as shown below).\n\nThe following are some questions that I concern.\n\n1.\tThe authors address the importance of federated continual learning from the aspect of continual learning, but is there any difference between federated continual learning and federated learning considering that most existing federated learning methods (fedavg and fedprox) are agnostic of client id?\n2.\tHow to train alpha in Eq. (1)? Is this a learnable parameter with sigmoid activation? If yes, how to set the parameters for testing on different tasks? Moreover, the whole testing process is confusing to me.\n3.\tThe detailed optimization procedure for Eq. (1) is not provided.\n4.\tFor the training objective Eq. 2, intuitively, authors propose decomposable parameters and want Base parameters B to be sparse with the help of task adaptive parameters A.  However, it is interesting to see that there is no constraint to encourage B and A to focus on different aspects. I thus doubt that the communication efficiency brought by the sparse parameters mainly benefits from the sparsity constraints, i.e., 2nd term in Eq (2), but not the decomposable parameters. The authors are encouraged to show that the base parameters (m*B) are more sparse than the model trained with existing federated methods with similar sparsity constraints.  Otherwise, the communication efficiency of the proposed method would not stand. \n5.\t For the third term in Eq. (2), whether it is necessary to impose the constraint on  A_c^i (i<|t|), as in Eq. (1)?  \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "TWjlKeYv0k_", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors present a federated continual learning framework. By decomposing the local client parameter, the method could alleviate the effect of negative transfer and improve efficiency. Empirical results partly show the effectiveness of the proposed algorithm. \n\n**Strength**\nTo my best knowledge, the problem setting is quite novel. Decomposing the parameter parts are also interesting. \n\n**Weakness**\nWriting is not good and many details are not clear. Some parts of the methodology lack explanation and the experiment results cannot support all claims of the authors. Below is my detailed opinion.\n\nEquation 1 describes the decomposition.  However, some notations are not clearly defined. For example, what is L, I, O£¿ I have to guess that L is the depth of the network, I represent the input dimension, O represents the output dimension after I reading it many times.  I am not sure whether the above guess is right. Also, A_c^{(t)} is sparse task-adaptive parameters, this is not clear. I am not sure the sparse means in task-level or for every matrix A_c^{t} is sparse. I have to guess from equation 2, the author tries to concatenate all A, and perform sparse constrain.  I am not sure whether the above guess is right since the author did not explain it well.\n\nIn this paper, it seems that the author assumes that when a new task comes is knew ahead. In practice, it is hard to get such prior knowledge. Also, in the experiment part, it seems that different clients receive different tasks simultaneously, which is also quite not realistic. In practice, the new tasks could come asynchronously.\n\nOn the 2nd line of page 6, the author claims the efficiency of the algorithm is boosted since it requires |C| * (R * |B| + A). Isn't B is the same shape as \\theta? I understand that the author assumes B is sparse, but we do not know how sparse it could be or can we recover the ground-truth sparse matrix or not. I can see that in the worst-case, without assumptions, this cannot improve efficiency. Empirical results only support that using the l1 constraint can output sparse results, while when this will work and correct is not clear in the methodology part. \n\nAlso, for this sparsity, lambda 1 is fixed across different tasks. This indicates different tasks have the same penalty. I do not see why this works since some tasks could be very similar to other tasks while some tasks are very distinct.  The same reason can be applied to lambda2 since the relationship between the current task and the previous task could be different. In figure 6, it seems that hyperparameters are very sensitive.\nThe plot of learned attention (Fig 5b).  I am not sure whether this is top-5 attention since there is no description (maybe I missed it). I assume that this is top-5 attention results. However, this cannot support the author that the proposed method can handle the negative transfer. First, empirically, why Traffic Sign has more weight than SVHN in the MNIST task? Moreover, This figure shows that empirically, using attention can focus on more important tasks/features and this should not be in the main contribution, and this is a well-known phenomenon. Has the ability to include the right task does not indicate excluding negative tasks.\n\nIn the middle plot of figure 6, I wish to see the whole task performance rather than selected tasks (this should be included in the appendix at least).", "extra_scores": {}, "extra_text": {}}, {"review_id": "93zyu5Swt6G", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Combination of federated learning and continual learning is a timely problem. Given the trending popularity of either of them it's the time to tackle this problem. The problem that is tackled is nice but the proposed formulation looks incremental.\n\nThe paper is generally well written but has a few typos along the paper like:\nPage 2: \"... need to selective utilize ..\"\nPage 2: ... once when ...\"\nPage 7: \"dtaset\"\n\nIn section 3.1 the relation between tasks of the clients is not clear. Are the tasks at step i of all tasks related to each other? \n\nIn equation 1: In the third term --> Isn't there a case for transfer between the task of the same client to future tasks?\n\nSection 3.3 for training: Imagine we are at task t. Then, the minimization problem in equation 3 involves solving for A^i. Why do one need to find A^i for a previous task and find a new parameter for that? Isn't that task already gone?Although, does this mean that the parameters to be estimated at each step is growing with the number of tasks?\n\nThe NonIID data set is a cool combination of different smaller datasets.\n\n The experiments look overall good. However, the part that shows alleveriating catastrophic forgetting seems rather less elaborated. Why only 6th and 8th tasks? Why not demonstrate forgetting of task 1 over the time. Why not all tasks? And, Fig. 6 only shows up to 5 tasks. How many consecutive tasks does the proposed method handle?\n\nThis works look like an increment on the top of (Yoon et al, 2020). Like making their approach adapted to a federated learning case. Thus, the contribution of the works is rather limited. \n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper tackles an interesting problem (that of federated continual deep learning) and proposes an effective approach for it with good results.  This is a good contribution. However, there are presentation issues in several aspects of the paper that require improvement before publication. The authors' claims of the novelty of the federated continual learning problem is overstated (there was an AAMAS'18 paper they cited which IS applicable to the federated setting, although their method is certainly a substantial improvement, more flexible, and supports deep models), and there are aspects of the analysis and experiments that could be improved (the authors are somewhat dismissive of one reviewer's concerns about the insensitivity to the regularization parameters. While I agree with the authors that this aspect of the review is focusing on this one detail in lieu of the bigger picture, the author's insensitivity study in Figure 6 and subsequent analysis are lacking and could use improvement).  The authors' revisions did help clarify/address a number of issues raised in the initial reviews,  but it was felt that more improvement was needed.
In particular, there are still mistakes with characterizing this work with respect to existing work, especially in overstating the novelty of the federated continual learning problem.  I do believe this paper is the first to call it "federated continual learning" (a name I like), but the paper should be less dismissive of existing works on "multi-agent lifelong learning" that could also apply to this setting, albeit with shallow models.  Consequently, while this reduces the novelty of the federated continual learning problem, the authors still have a substantive contribution -- just one that needs improvement in presentation before publication.
Decision: Reject