Federated Continual Learning with Weighted Inter-client Transfer

1 INTRODUCTION . Continual learning ( Thrun , 1995 ; Kumar & Daume III , 2012 ; Ruvolo & Eaton , 2013 ; Kirkpatrick et al. , 2017 ; Schwarz et al. , 2018 ) describes a learning scenario where a model continuously trains on a sequence of tasks ; it is inspired by the human learning process , as a person learns to perform numerous tasks with large diversity over his/her lifespan , making use of the past knowledge to learn about new tasks without forgetting previously learned ones . Continual learning is a long-studied topic since having such an ability leads to the potential of building a general artificial intelligence . However , there are crucial challenges in implementing it with conventional models such as deep neural networks ( DNNs ) , such as catastrophic forgetting , which describes the problem where parameters or semantic representations learned for the past tasks drift to the direction of new tasks during training . The problem has been tackled by various prior work ( Kirkpatrick et al. , 2017 ; Lee et al. , 2017 ; Shin et al. , 2017 ; Riemer et al. , 2019 ) . More recent works tackle other issues , such as scalability or order-robustness ( Schwarz et al. , 2018 ; Hung et al. , 2019 ; Yoon et al. , 2020 ) . However , all of these models are fundamentally limited in that the models can only learn from its direct experience - they only learn from the sequence of the tasks they have trained on . Contrarily , humans can learn from indirect experience from others , through different means ( e.g . verbal communications , books , or various media ) . Then wouldn ’ t it be beneficial to implement such an ability to a continual learning framework , such that multiple models learning on different machines can learn from the knowledge of the tasks that have been already experienced by other clients ? One problem that arises here , is that due to data privacy on individual clients and exorbitant communication cost , it may not be possible to communicate data directly between the clients or between the server and clients . Federated learning ( McMahan et al. , 2016 ; Li et al. , 2018 ; Yurochkin et al. , 2019 ) is a learning paradigm that tackles this issue by communicating the parameters instead of the raw data itself . We may have a server that receives the parameters locally trained on multiple clients , aggregates it into a single model parameter , and sends it back to the clients . Motivated by our intuition on learning from indirect experience , we tackle the problem of Federated Continual Learning ( FCL ) where we perform continual learning with multiple clients trained on private task sequences , which communicate their task-specific parameters via a global server . Figure 1 ( a ) depicts an example scenario of FCL . Suppose that we are building a network of hospitals , each of which has a disease diagnosis model which continuously learns to perform diagnosis given CT scans , for new types of diseases . Then , under our framework , any diagnosis model which has learned about a new type of disease ( e.g . COVID-19 ) will transmit the task-specific parameters to the global server , which will redistribute them to other hospitals for the local models to utilize . This allows all participants to benefit from the new task knowledge without compromising the data privacy . Yet , the problem of federated continual learning also brings new challenges . First , there is not only the catastrophic forgetting from continual learning , but also the threat of potential interference from other clients . Figure 1 ( b ) describes this challenge with the results of a simple experiment . Here , we train a model for MNIST digit recognition while communicating the parameters from another client trained on a different dataset . When the knowledge transferred from the other client is relevant to the target task ( SVHN ) , the model starts with high accuracy , converge faster and reach higher accuracy ( green line ) , whereas the model underperforms the base model if the transferred knowledge is from a task highly different from the target task ( CIFAR-10 , red line ) . Thus , we need to selective utilize knowledge from other clients to minimize the inter-client interference and maximize inter-client knowledge transfer . Another problem with the federated learning is efficient communication , as communication cost could become excessively large when utilizing the knowledge of the other clients , since the communication cost could be the main bottleneck in practical scenarios when working with edge devices . Thus we want the knowledge to be represented as compactly as possible . To tackle these challenges , we propose a novel framework for federated continual learning , Federated Weighted Inter-client Transfer ( FedWeIT ) , which decomposes the local model parameters into a dense base parameter and sparse task-adaptive parameters . FedWeIT reduces the interference between different tasks since the base parameters will encode task-generic knowledge , while the task-specific knowledge will be encoded into the task-adaptive parameters . When we utilize the generic knowledge , we also want the client to selectively utilize task-specific knowledge obtained at other clients . To this end , we allow each model to take a weighted combination of the task-adaptive parameters broadcast from the server , such that it can select task-specific knowledge helpful for the task at hand . FedWeIT is communication-efficient , since the task-adaptive parameters are highly sparse and only need to be communicated once when created . Moreover , when communication efficiency is not a critical issue as in cross-silo federated learning ( Kairouz et al. , 2019 ) , we can use our framework to incentivize each client based on the attention weights on its task-adaptive parameters . We validate our method on multiple different scenarios with varying degree of task similarity across clients against various federated learning and local continual learning models . The results show that our model obtains significantly superior performance over all baselines , adapts faster to new tasks , with largely reduced communication cost . The main contributions of this paper are as follows : • We introduce a new problem of Federated Continual Learning ( FCL ) , where multiple models continuously learn on distributed clients , which poses new challenges such as prevention of inter-client interference and inter-client knowledge transfer . • We propose a novel and communication-efficient framework for federated continual learn- ing , which allows each client to adaptively update the federated parameter and selectively utilize the past knowledge from other clients , by communicating sparse parameters . 2 RELATED WORK . Continual learning While continual learning ( Kumar & Daume III , 2012 ; Ruvolo & Eaton , 2013 ) is a long-studied topic with a vast literature , we only discuss recent relevant works . Regularizationbased : EWC ( Kirkpatrick et al. , 2017 ) leverages Fisher Information Matrix to restrict the change of the model parameters such that the model finds solution that is good for both previous and the current task , and IMM ( Lee et al. , 2017 ) proposes to learn the posterior distribution for multiple tasks as a mixture of Gaussians . Architecture-based : DEN ( Yoon et al. , 2018 ) tackles this issue by expanding the networks size that are necessary via iterative neuron/filter pruning and splitting , and RCL ( Xu & Zhu , 2018 ) tackles the same problem using reinforcement learning . APD ( Yoon et al. , 2020 ) additively decomposes the parameters into shared and task-specific parameters to minimize the increase in the network complexity . Coreset-based : GEM variants ( Lopez-Paz & Ranzato , 2017 ; Chaudhry et al. , 2019 ) minimize the loss on both of actual dataset and stored episodic memory . FRCL ( Titsias et al. , 2020 ) memorizes approximated posteriors of previous tasks with sophisticatedly constructed inducing points . To the best of our knowledge , none of the existing approaches considered the communicability for continual learning of deep neural networks , which we tackle . CoLLA ( Rostami et al. , 2018 ) aims at solving multi-agent lifelong learning with sparse dictionary learning , but it is not applicable to federated learning or continual deep learning . Federated learning Federated learning is a distributed learning framework under differential privacy , which aims to learn a global model on a server while aggregating the parameters learned at the clients on their private data . FedAvg ( McMahan et al. , 2016 ) aggregates the model trained across multiple clients by computing a weighted average of them based on the number of data points trained . FedProx ( Li et al. , 2018 ) trains the local models with a proximal term which restricts their updates to be close to the global model . FedCurv ( Shoham et al. , 2019 ) aims to minimize the model disparity across clients during federated learning by adopting a modified version of EWC . Recent works Yurochkin et al . ( 2019 ) ; Wang et al . ( 2020 ) introduce well-designed aggregation policies by leveraging Bayesian non-parametric methods . A crucial challenge of federated learning is the reduction of communication cost . TWAFL ( Chen et al. , 2019 ) tackles this problem by performing layer-wise parameter aggregation , where shallow layers are aggregated at every step , but deep layers are aggregated in the last few steps of a loop . Karimireddy et al . ( 2020 ) suggests an algorithm for rapid convergence , which minimizes the interference among discrepant tasks at clients by sacrificing the local optimality . This is an opposite direction from personalized federated learning methods ( Fallah et al. , 2020 ; Lange et al. , 2020 ; Deng et al. , 2020 ) which put more emphasis on the performance of local models . FCL is a parallel research direction to both , and to the best of our knowledge , ours is the first work that considers task-incremental learning of clients under federated learning framework . 3 FEDERATED CONTINUAL LEARNING WITH FEDWEIT . Motivated by the human learning process from indirect experiences , we introduce a novel continual learning under federated learning setting , which we refer to as Federated Continual Learning ( FCL ) . FCL assumes that multiple clients are trained on a sequence of tasks from private data stream , while communicating the learned parameters with a global server . We first formally define the problem in Section 3.1 , and then propose naive solutions that straightforwardly combine the existing federated learning and continual learning methods in Section 3.2 . Then , following Section 3.3 and 3.4 , we discuss about two novel challenges that are introduced by federated continual learning , and propose a novel framework , Federated Weighted Inter-client Transfer ( FedWeIT ) which can effectively handle the two problems while also reducing the client-to-sever communication cost . 3.1 PROBLEM DEFINITION . In the standard continual learning ( on a single machine ) , the model iteratively learns from a sequence of tasks { T ( 1 ) , T ( 2 ) , ... , T ( T ) } where T ( t ) is a labeled dataset of tth task , T ( t ) = { x ( t ) i , y ( t ) i } Nt i=1 , which consists of Nt pairs of instances x ( t ) i and their corresponding labels y ( t ) i . Assuming the most realistic situation , we consider the case where the task sequence is a task stream with an unknown arriving order , such that the model can access T ( t ) only at the training period of task t which becomes inaccessible afterwards . Given T ( t ) and the model learned so far , the learning objective at task t is as follows : minimizeθ ( t ) L ( θ ( t ) ; θ ( t−1 ) , T ( t ) ) , where θ ( t ) is a set of the model parameters at task t. We now extend the conventional continual learning to the federated learning setting with multiple clients and a global server . Let us assume that we have C clients , where at each client cc ∈ { c1 , . . . , cC } trains a model on a privately accessible sequence of tasks { T ( 1 ) c , T ( 2 ) c , ... , T ( t ) c } ⊆ T . Please note that there is no relation among the tasks T ( t ) 1 : c received at step t , across clients . Now the goal is to effectively train C continual learning models on their own private task streams , via communicating the model parameters with the global server , which aggregates the parameters sent from each client , and redistributes them to clients .

Summary: This paper investigates a new problem – federated continual learning by Federated Weighted Inter-client Transfer. The key idea is to decompose the network weights into global federated parameters and sparse task-specific parameters such that each client can selectively receive knowledge from other clients by taking a weighted combination of their task-specific parameters. The experiment results in two contrived datasets demonstrate the effectiveness of the proposed method.