Federated Continual Learning for Text Classification via Selective Inter-client Transfer (2022)

Abstract: In this work, we combine the two paradigms: Federated Learning (FL) and Continual Learning (CL) for text classification task in cloud-edge continuum. The objective of Federated Continual Learning (FCL) is to improve deep learning models over life time at each client by (relevant and efficient) knowledge transfer without sharing data. Here, we address challenges in minimizing inter-client interference while knowledge sharing due to heterogeneous tasks across clients in FCL setup. In doing so, we propose a novel framework, Federated Selective Inter-client Transfer (FedSeIT) which selectively combines model parameters of foreign clients. To further maximize knowledge transfer, we assess domain overlap and select informative tasks from the sequence of historical tasks at each foreign client while preserving privacy. Evaluating against the baselines, we show improved performance, a gain of (average) 12.4% in text classification over a sequence of tasks using five datasets from diverse domains. To the best of our knowledge, this is the first work that applies FCL to NLP.

## introduction federated learning @xcite @xcite @xcite in edge computing 1 @xcite has gain attraction in recent years due to (a) data privacy and sovereignty-especially imposed by government regulations @xcite , and (b) the need for sharing knowledge across edge (client) devices such as mobile phones, automobiles, wearable gadgets, etc. while maintaining data localization. federated learning (fl) is a privacy-preserving machine learning (ml) technique that enables collaborative training of ml models by sharing model parameters across distributed clients through a central server -without 1 extends cloud computing services closer to data sources sharing their data. in doing so, a central server aggregates model parameters from each participating client and then distribute the aggregated parameters, where ml models at each client are optimized using them -achieving inter-client transfer learning. in this direction, the recent works such as fedavg @xcite , fedprox @xcite , fedcurv @xcite have introduced parameter aggregation techniques and shown improved learning at local clients -augmented by the parameters of foreign clients. on the other hand, the edge devices generate a continuous stream of data where the data distribution can drift over time; therefore, the need for continual learning like humans do. continual learning (cl) @xcite @xcite @xcite empowers deep learning models to continually accumulate knowledge from a sequence of tasks -reusing historical knowledge while minimizing catastrophic forgetting (drift in learning of the historical tasks) over life time. federated continual learning (fcl): this work investigates the combination of the two paradigms of ml: federated learning and continual learning with an objective to model a sequence of tasks over time at each client via interclient transfer learning while preserving privacy and addressing heterogeneity of tasks across clients. there are two key challenges of fcl: (1) catastrophic forgetting, and (2) inter-client interference due to heterogeneity of tasks (domains) at clients. at central server, fedavg @xcite aggregates-averages model parameters from each client without considering inter-client interference. to address this, fedweit @xcite approach performs fcl by sharing task-generic (via dense base parameters) and task-specific (via taskadaptive parameters) knowledge across clients. in doing so, at the server, they aggregate the dense base parameters however, no aggregation of the task-adaptive parameters, and then broadcast both the types of parameters. see further details in figure 2 and section 2.2. fedweit, the first approach in fcl, investigates computer vision tasks (e.g., image classification), however the technique has limitations in aligning domains of foreign clients while augmented learning at each local client using task-adaptive parameters -that are often misaligned with local model parameters in parameter space @xcite due to heterogeneity in tasks. therefore, a simple weighted additive composition technique does not address inter-client interference and determine domain relevance in foreign clients while performing transfer learning. contributions: to the best of our knowledge, this is the first work that applies fcl to nlp task (text classification). at each local client, to maximize the inter-client transfer learning and minimize inter-client interference, we propose a novel approach, federated selective inter-client transfer (fedseit) that aligns domains of the foreign task-adaptive parameters via projection in the augmented transfer learning. to exploit the effectiveness of domain-relevance in handling a number of foreign clients, we further extend fedseit by a novel task selection strategy, selective inter-client transfer (sit) that efficiently selects the relevant task-adaptive parameters from the historical tasks of (many) foreign clients -assessing domain overlap at the global server using encoded data representations while preserving privacy. we evaluate our proposed approaches: fedseit and sit for text classification task in fcl setup using five nlp datasets from diverse domains and show that they outperforms existing methods. our main contributions are as follows: inter-client knowledge transfer: server transmits global aggregated parameter θ g to all participating clients for inter-client knowledge transfer in the next training round r + 1. challenges: however, there are two main sources of inter-client interference within fcl setup: (1) using a single global parameter θ g during parameter aggregation at server to capture the cross-client knowledge @xcite due to model parameters trained on irrelevant foreign client tasks, and (2) non-alignment of the foreign client model parameters given the heterogeneous task domains across clients. this leads to the hindrance of the local model training at each client by updating its parameters in erroneous directions, thus resulting in: (a) catastrophic forgetting of the client's historical tasks, and (b) sub-optimal learning of client's current task. for brevity, we will omit notation of round r from further equations and mathematical formulation except algorithms. c parameters are initialized for each task t. at the global server, we have global parameter θ g which accumulates task-generic knowledge across all clients i.e., global knowledge, by aggregating local base parameters sent from all clients. finally, for each client c c and task t, the model parameters θ t c can be described as: ## experiments and analysis to demonstrate the effectiveness of our proposed fedseit framework, we perform evaluation on text classification task using five datasets from diverse domains and present our qualitative and quantitative analysis in fcl setup. datasets: we present experimental evaluation results on reuters8 (r8), tag my news (tmn), trec6, trec50 and subjectivity (subj) datasets. here, r8 and tmn datasets belong to the news domain, trec6 and trec50 belong to question classification domain and subj is a movie reviews dataset with binary labels. please see appendix a for more details regarding datasets. experimental setup: we follow @xcite for our fcl experimental setup where we use cnn based text classification model @xcite as the local client model. for all experiments, we use three clients i.e., @xmath0, five tasks per client i.e., @xmath1, 10 rounds per task i.e., @xmath2, with 50 epochs in each round, λ 2 ∈ {0.1, 1.0} and @xmath3, 5} for selective inter-client transfer (sit). please see appendix c for detailed hyperparameter settings for all of our experiments. to run the experiments in fcl setup, we need to generate task datasets t t c for each task t. consider a dataset d d with a unique label set l d ≡ {l 1 , ..., l n d }, where n d is the count of unique labels. now, for each task dataset t t c , we randomly pick a fixed number of unique labels l t c ⊆ l d , where the count of unique task labels |l t c | = 4 is fixed for all tasks across all clients except subjectivity, which has 2 unique labels shared by all tasks. if label l 1 gets selected for 3 tasks, then we follow the non-iid splitting strategy which simply divides the documents labeled with l 1 i.e., d d (l 1 ), into three mutually exclusive and equal parts, thus ensuring heterogeneity. we use this strategy to split the training and validation sets. however, to create the test set for each task dataset t t c , we select all of the documents labeled with l t c in the complete test dataset i.e., {d test d (l i )|l @xmath4. as this work focuses on the new challenges which arise due to combination of fl and cl paradigms in fcl setup, we compare our work with related fcl methods and not with standalone fl, cl methods. baseline: as the only existing work in fcl domain, fedweit has shown significantly superior performance compared to näive fcl methods that is why we adopt fedweit as the baseline method. evaluation metric: in each experiment, once the training is finished for all c clients, we freeze the model parameters for all tasks and compute micro-averaged accuracy (maa) score for each of the t past tasks of all c clients. finally, we average c × t maa scores to compute the final task-averaged test accuracy (tta) score to compare our proposed model with baseline. for each experiment, we report an average tta score of 3 runs using 3 different seed values for ordering of tasks at each client. ## conclusion and future work we have applied federated continual learning to text classification for heterogeneous tasks and addressed the challenges of inter-client interference and domain-alignment in model parameters of local vs foreign clients while minimizing catastrophic forgetting over a sequence of tasks. we have presented two novel techniques: fedseit and sit that improves local client augmented-learning by assessing domain overlap and selecting informative tasks from the sequence of historical tasks of each foreign client while preserving privacy. furthermore, the novel selection strategy using sit determines relevant foreign tasks from the complete historical tasks of all foreign clients by assessing domain overlap while preserving privacy. we have evaluated the proposed approaches using five text classification data sets and shown a gain (average) of 12.4% over the baseline. although we have applied fedseit framework to the document-level text classification task, we can further apply the proposed framework to additional nlp tasks. inspired by continual topic modeling @xcite at document-level and continual named entity recognition @xcite at token-level classification, we can further extend these existing works with the proposed fedseit framework in federated settings.