Better Generative Replay for Continual Federated Learning
Authors: Daiqing Qi
Handong Zhao
Sheng Li
Venue: ICLR 2023 poster (2023)
Keywords: federated learning
continual learning

Abstract: Federated Learning (FL) aims to develop a centralized server that learns from distributed clients via communications without accessing the clients’ local data. However, existing works mainly focus on federated learning in a single task sce- nario with static data. In this paper, we introduce the continual federated learning (CFL) problem, where clients incrementally learn new tasks and history data can- not be stored due to certain reasons, such as limited storage and data retention policy 1. Generative replay (GR) based methods are effective for continual learning without storing history data. However, we fail when trying to intuitively adapt GR models for this setting. By analyzing the behaviors of clients during training, we find the unstable training process caused by distributed training on non-IID data leads to a notable performance degradation. To address this problem, we propose our FedCIL model with two simple but effective solutions: 1. model consolidation and 2. consistency enforcement. Experimental results on multiple benchmark datasets demonstrate that our method significantly outperforms baselines. Code is available at: https://github.com/daiqing98/FedCIL.

Published as a conference paper at ICLR 2023
BETTER GENERATIVE REPLAY FOR CONTINUAL FEDERATED LEARNING
Daiqing Qi', Handong Sheng Li' University of Virginia, Adobb Research
{daiqing.qi. shengli} @ virginia.edu, hazhao @ adm
ABSTRACT
Federated Learning (FL) aims to develop a centralized server that learns from distributed clients via communications without accessing the clients' local data. However, existing works mainly focus on federated learning in a single task sce- nario. with static data. In this paper, we introduce the continual federated learning (CFL) problem, where clients incrementally learn new tasks and history data can- not be stored due certain reasons, such as limited storage and data retention pol- icy Generative replay (GR) based methods are effective for continual learning without storing history data. However, we fail when trying to intuitively adapt GR models for this setting. By analyzing the behaviors of clients during training, we find the unstable training process caused by distributed training on non-III) data leads to notable performance degradation. To address this problem, we propose our FedCIL model with two simple but effective solutions: I. model consolida- tion and 2. consistency enforcement. Experimental results multiple benchmark datasets demonstrate that our method significantly outperforms baselines. Code is available at: https: /ttgttttthuuutt///g////ggg. com. http:///www..ooiiii//ggg////////
INTRODUCTION
Federated learning (McMahan al. 2017) an emerging topic in machine learning, where pow- erful global model is maintained via communications with distributed clients without access to their local data. A typical challenge federated learning the non-IlI data distribution (Zhao et al.] 2018 Zhu et al. 2021a), where the data distributions learnt by different clients are different (known as heterogeneous federated learning). Recent methods (Li et al.) 2020; Chen & Chao 2020 Zhu ct al. 202 lb) gain improvements in the typical federated learning setting, where the global model is learning a single task and each client is trained locally on fixed data. However, real-wort wolld appli- cations, it is more practical that each client is continuously learning new tasks. Traditional federated learning models fail to solve this problem.
In practice, history data sometimes inaccessible considering privacy constraints (e.g., data pro- tection under GDPR) or limited storage space (e.g.. mobile devices with very limited space), and the unavailability of previous data often leads to catastrophic forgetting (McCloskey & Cohen 1989
in many machine learning models. Continual learning (Thrun) 1995 Kumar & Daume III 2012
Ruvolo & Eaton 2013 Chu & Li (023) aims to develop an inte intelligent system that can continuously
leamn from new tasks without forgetting learnt knowledge in the absence of previous data. Common continual learning scenarios can be roughly divided into two scenarios (Van de Ven & Tolias 2019):
task incremental learning (TH.) and class incremental learning (CII.) (Rebuffi et al. (017). In both scenarios, the intelligent system required to solve all tasks so far. In TIL., IDs of different task accessible, while in CIL, they are unavailable, which requires the system to infer task-IDs. The unavailability of task-IDs makes the problem significantly harder.
In this paper, propose challenging and realistic problem, continual federated learning. More specifically, we aim to deal with the class-incremental federated learning (CI-FL) problem. this setting, each client is continuously learning new classes from sequence of tasks, and the centralized server learns from the clients via communications. is more difficult than the single FL or CIL,

---

Published as a conference paper at ICLR 2023
because both the non-llD and catastrophic forgetting issues need to be addressed. Compared with traditional continual learning settings that only involve one model, our problem is more complex because there multiple models including one server and many clients.
Owning to the success of generative replay ideas in continual learning, we propose to adopt Auxil- iary Classifier GAN (ACGAN) (Odena al., 2017). generative adversarial network (GAN) with an auxiliary classifier, as the base model for the server and clients. Then the generator of the model can be used for generative replay avoid forgetting. Interestingly, experiments show that a sim- ple combination of ACGAN and FL algorithms fails to produce promising results. It is already known that the unstable training process (Roth et al., 2017, Kang et al., 2021, Wu al. (2200) and imbalanced data can worsen the performance of generative models. We find that this phenomenon becomes more severe the context of federated learning, which leads to the failure of the intuitively combined model.
To overcome these challenges, we propose federated glass-incrementa learning (FedCIL) frame- work. In FedCIL, the generator of ACGAN helps alleviate catastrophic forgetting by generating syn- thetic data of previous distributions for replay, meanwhile, it benefits federated learning by transfer- ring better global and local data distributions during the communications. Our model is also subject to the privacy constraints federated learning. During the communications, only model parameters transmitted. With the proposed global model consolidation and local consistency enforcement, our model significantly outperforms baselines on benchmark datasets.
The main contributions of this paper are follows:
We introduce a challenging and practical problem of continual federated learning, i.e., class-incremental federated learning (CI-FL), where global model continuously learns from multiple clients that are incrementally learning new classes without memory buffers. Subsequently, we propose generative replay (GR) based methods for this challenge.
We empirically find the unstable learning process caused by distributed training on highly non-III data with popular federated learning algorithms can lead to notable performance degradation of GR based models. Motivated by this observation, we further propose to solve the problem with model consolidation and consistency enforcement.
We design new experimental settings and conduct comprehensive evacuations on bench- mark datasets. The results demonstrate the effectiveness of our model.
2 RELATED WORK
Glas--Incrementa Learning. Class-incremental learning (CIL) (Rebuffi et al., 2017) is a hard continual learning problem due the unavailability of the task-IDs. Existing approaches to solve the CIL problem can be divided into three categories (Ebrahimi et al., 2020). including the replay- based methods (Rolnick et al., 2019 Chaudhry et al.] 2219). srueture-based methods (Yoon et al.) 2017). and regularization-basee the generative replay based methods (Shin al. 2017: Wu et al. 2018) belong to the replay- methods, but they do not need replay buffer for storing previous data. Instead, an additional generator trained to capture the data distribution of data learnt so far, and it generates synthetic data for replay when the real data becomes unavailable. The generative replay based methods have obtained promising results on various CIL benchmarks, where storing data is not allowed (data-free).
methods KKrkpatrick et al. 2017 Aljundi et al. 2018). In addition.
Federated Learning. Federated learning has been extensively studied in recent years. We mainly discuss the ones that involve generative models in this section. For instance, some recent federated learning methods train a GAN across distributed resources (Zhang et al. 202T Rasouli et al.) 2020) in federated learning paradigm. They are subject to privacy constraints in that the parameters the GAN are shared instead of the real data. In our work, we also strictly follow the privacy constraints by only transmitting GAN parameters.
Continual Learning and Federated Learning. So far, only few works lie in the intersection of federated learning and continual learning. Casado et al. (2020) discussed federated learning with changing data distributions, but it only involves single-task scenario of federated learning. Yoon jet al. (202T) introduced a federated continual learning setting from the perspective of continual Tearning. Our work is significantly different from it. First, in (Yoon al. 2021), the federated
2

---

Published as a conference paper at ICLR 2023
continual learning focuses the performance of each single client, and thus it does not maintain global model on the server side. Our objective is opposite to in that we aim to develop a global model that incrementally learns from the clients. Second, the method in (Yoon et al. 202I) focuses on the task-incremental learning, where task-IDs are required for inference, while our work aims to deal with the more challenging task of class-incremental learning. where fask-IDs
available. In addition, the methods proposed in (Usmanova et al.) 2021 Guo et al. 2021] Hendryx et al.) 2021] are also relevant to our work, all of which have a global model that incrementally learns new classes. In particular, federated reconnaissance (Hendryx et al. 2021) uses prototype networks for different from our work that, in federated recomnaissance, new classes
its proposed task. maybe overlapped with old classes, which is not allowed in the standard class incremental scenario. Also, focuses on few-shot learning and uses a buffer to store prototypes of learnt classes, which is not needed in our framework. (Guo et al., 202T) is very relevant to (Hendryx et al. (02T), which allows overlapped classes among a client's private tasks. Thus it is inconsistent with the standard incremental learning (IL) setting. (Usmanova al. 202T is different from us in the communication settings. Dong et al. (2022) used a memory buffer to store old data, which decreases the difficulty to large extend. We focus generative replay based continual learning methods which works with no memory buffers. We further clarify the differences between our work and the most related methods in Appendix D.
PRELIMINARY
In this section, we first present the problem definitions. Then we clarify the differences between our setting and existing work.
3.1 PROBLEM FORMULATION
Federated Learning (FL). A typical Federated Learning problem can be formalized by assuming of n different clients, where each client Cg owns private data DD with its corresponding task Th. and global model parameterized by Og is deployed on centralized server, which communicates with the clients during their training on their own local data. The aim of FL is to learn a global model that minimizes its risk on each of the client tasks. where Cs is the objective of Tu. On the other hand, we present the definition of ctass-incrementall learning, which is challenging continual learning problem task-IDs not available.
a set [C1, C2,,.., C2
Class-Incremental Learning (CIL). In maassinnrrementl learning, a model sequentially learns from sequence of data distributions denoted as D (D1 D ..., D J. where each D is the data distribution of the corresponding task ' and label space is )". During T all data distri- butions (Dili t} will be unavailable, and the goal of CIL is effectively learn from Dr, while maintaining its performance on learnt tasks. After all m tasks, the model is expected to map samples from D = [Di, D2 .... Dmn y U y² U without Task-IDs.
Innseenncemlttll Federated Learning (CI-FL). In CI-FL, each client C, sequentially learns from a series of m tasks locally (denoted as T; (T,' T,,,,... .... T,"h ) in a class-incremental way, during which time, the clients communicate with global model on centralized server. The goal of CI-FL is
to maintain global model that predicts all classes seen by all clients so far.
3.2 DESIDERATA
We have some desiderata for our proposed setting, which makes our clas-incrrmeenal federated learning more general, realistic and challenging, compared with existing works. Discussions includ- ing differences with existing works are available in Appendix D.
METHODOLOGY
We propose a new framework, FedCIL, for In this section, first review ACGAN, which is adopted the base model for server and clients. Then we discuss an interesting observation, i.e., simply combining ACGAN and federated learning algorithms fails work well. Motivated by the findings, we finally present our FedCIL. framework.
federated learning, as shown in Fig.
3

---

Published as a conference paper at ICLR 2023
Centralized Server
Client
lel.
Gar plokel
PN
G
Lx2
Lowcowr
Datasci
To
J
Baw
Bams
Boow
1- PN
GoP
logits
Liow
Clent
Cient
Cient
Figure 1: Framework of our FedCIL. The left figure illustrates the model consolidation on the server side. During each communication, the server model first initialized from merged clients, then consolidated with synthetic samples generated by client generators. The right figure illustrates consistency enforcement on client side (client for example), which achieved by consistency loss functions applied to the output logits of the classification module of the client.
4.1 ACGAN
Generative replay based methods are popular in a strict CII. setting, where storing data is not allowed (data-free). DGR (Shin al. 2017), representative generative replay based method, trains GAN for replay in addition to the classification model. ACGAN (Odena et al. 2017 Wu et al. 2018)) is better than DGR in that, reduces the complexity of the model by integrating the the classifier and the generative model for learning an unbiased classifier. Thus we choose ACGAN the base model for each client.
single model. Besides, it enables conditional generation, which is crucial
In ACGAN, each sample is generated by providing the noise and a class label c to the generator. The objective function of the generator can be written as:
Lyen min (C (),X) 2 (0, 8G
gan (0, X) = - E Exnps log((ppp (Gyc (z, e)))},
(2)
( X) -Egpp. [ye log(Cgo (Gee (z, c)))
(3)
where C, D and G are the classification module, discriminator module and the generator, respec- tively. Note that C and D share the feature extractor layers and only differ in the last out put layer. GG (0, X) and LG (0, X) respectively the standard cross-entropy loss for classification and the generator loss in a typical GAN. X the training dataset. denotes overall parameters, 0C, 0 and D are the parameters for classification module, discriminator module and the generator, respectively. Pe is the label distribution, and ps is the noise distribution. Ye is the ground truth label. The corresponding discriminator loss function is:
Lais
min (CDan (0,X) (O,))) 00.00
CDam (0, X) = -E [log( Deo (x)) - Eerper [log(1 (1 - DAD (Goo (z, c))) CD (0, X) - - [yo log(Coc (x))] E. EZmp, We (og(Coc (Goo e)))]
(5)
Overall, write the loss function of ACGAN as:
Locgan (0, X) Egen + Lais
When trained properly, The ACGAN can generate synthetic images while it can also give predictions of real images.
4.2 WHY DOES INTUITIVE COMBINATION FAIL TO WORK WELL?
However, simply combining the ACGAN and federated learning algorithm to adapt ACGAN to heterogeneous federated learning yields unpromising results. The instability of ACGAN during training, which is aggravated in the context of non-III federated learning leads to the degradation
4

---

Published as a conference paper at ICLR 2023
AC-GAN (Dlline)
Norm of orgrrddeents
Loss Value
0.3
AC GoN--oi
FedCII (Ours)
0.2
6-
(Offize)
0.1
AC-IANN+EedPiox
FodCIL (Ours)
0.0
500
1000
1.000
2000
500
1000
1500
2000
Iterations
Iterations
(a) Classification loss
(b) Gradient Norm
Figure 2: The classification losses and gradient norms (i.e., Eq. 6)) of (1) ACGAN FedProx, (2) ACGAN (offline) and (3) FedCIL (Ours) of randomly selected client during a period of training on EMNIST.
of the model performance. Fig. illustrates this phenomenon. In the simply combined model, the classification loss experiences undesired peaks in each communication round, which impedes the training of the generator ACGAN.
In single model scenario, the gradient exploding in the classifier of ACGAN have a negative impact on the training of the generator, which appears more often when the classifier is not well-trained yet, e.g., on the early training stage. When it happens, the classification loss from Eq. (6) comes into dominance and breaks the balance between the classification loss from Eq. (6) and the discrimination loss from Eq. (5) The generation ability will be limited due to such over-tilt since then. Finully the generation ability impaired, which impacts the quality of generative replay, leading to more forgetting. Therefore, we need to prevent the classification loss from growing too high at the early stages each communication round.
The standard cross-entropy loss function is written as:
Lce = -Eg-xx(log exp (Pp(F(x) exp(F(x)Tw.)) wy)
(8)
E exp
F (x)) wi)
where is the instance from the training data distribution X and is its label, F is the feature extractor and W = [W1,.... We] is the parameter set of the classifications layer. c is the number of total classes.
Then the derivative of Eq. w.r. (1.........e)) is:
öCCE Ex-xExxPFE (F())(((())1 log( exp(F y)) "ws)
))
(9)
Owk
Ej=1 exp
F (I:) wi
where 1y=k is the indicator function whose value equals 1 if k else 0.
Eq. Op indicates that the gradient norm of Loe is associated with the feature norm (i.e., the norm of F(x) and the classification possibilities. Thus the generation ability is more likely degrade when the classifier gives inaccurate predictions.
The way clients are trained federated learning makes such degradation more often. In each com- munication round, each local client first synchronizes parameters with the global model, which worsens the performance of the classifier
its local dataset. According Eq. ( the gradient norm tends to grow, which can lead to the gradient exploding. And the large values of _ (6, X) from Eq. (6) break the balance between cD (0, X) and C X). The over-tilt C (6, X) limits the generaTion ability and finally decreases the model performance, which relies on the quality of generative replay. Moreover, the training process federated learning consists of many commu- nications rounds, which makes it worse because such degradation happens in early stages of every communication round, throughout the whole training process.
Figures and 4 illlssrrtee this process. At the beginning of each communication round, the classifi- cation loss and FID score ACGAN experiences an obviously peak if simply combining ACGAN with FL algorithm (FedProx is used in experiments). It is observed every 400 iterations local intention T = 400. a single model scenario without federated learning, i.c., ACGAN (of- fline), the loss is stable. Our model works in federated learning while the loss curve is comparable
set
5

---

Published as a conference paper at ICLR 2023
to ACGAN (offline). Note that ACGAN (offline) indicates the ideal case. Because in FL, clients synchronized with the server at every communication round and offline training is unavailable.
4.3 FEDCIL
To address the problems, we propose a new framework (Fig. for class-incremental federated learn- ing. We denote all parameters of single client model C, as 0 and that of the global model g as l ggoobet- Clients sequentially learn a series of non-overlapped classes. Meanwhile, the server com- municates with the clients multiple rounds to learn all classes seen by all clients so far. During each round, clients first learn the current or new classes from their own dataset for certain iterations. Then the model parameters of selected clients are sent to the server, where the global model aggregates the parameters and conduct a model consolidation to enrich itself with the knowledge from different clients. Finally, the parameters of the global model are sent back the all clients.
Model consolidation. On the server side, the centralized server first receives the parameters (de- noted by 01. ...) sent by selected clients Cselected CC1,C2.... Then the parameters are merged to initialize the global model. Note that in CI-FI. setting, clients continuously learn new classes, thus new output nodes added to the glohal model if any of the clients learns new classes. In a practical CI-FL setting, the local data of clients are usually non-lll) and more importantly, new classes are continuously collected and learnt by clients, thus simply merging the parameters often fails in such case. We proposed to solve this problem by model consolidation. The global model is first initialized with the merged parameters and an ensemble of classification heads from different clients. Then we generate balanced synthetic data with collected generator parameters from selected clients. Finally the global model is consolidated by the ACGAN loss function Loegan with generated data:
Laerver Luegan (oagpan(Oggoee ggohbat, X²);
(10)
where X are synthetic data generated by collected generators this communication round and Ogtobal is the parameter of the generator on the server.
Consistency enforcement. On the client side, each client owns three modules during the local training: (1) a ACGAN (2) a copy of the generator of itself Gb5² (yellow generator in Fig. I at the end of last task (3) the global generator Gge. globat (red generator in Fig. [ which it received in the last communication. In local training, the client first initializes itself with received parameter ggtobal from the global model. During the training. the client is expected to effectively learn from new classes while maintain its knowledge on learnt classes. For this purpose, Gge² generates samples x conditioned on labels from previous tasks and they are mixed up with training data X to form the input for ACGAN loss:
(11) Although the classifier of a client trained to give same predictions of the real images and the synthetic images generated by generators if they have the same label, their features can be different because their sources are different.
Liocat = Laegan (0, XLIUX) UX),
Denote generated samples by the generator of current model, the generator from last task, and the generator from received global model as X and X and denote the real sample as X. Given X²_ Xg and X, to learn a better global model, their features are expected be close to each other when their labels sticking in a local optimality or diverges. We propose to achieve it by consistency enforcement, i.e., aligning the their features produced by intermediate layers of the classifier. To make it simple, instead of aligning features output by intermediate layers, we try to align their output logits by the last layer, i.e., the classification layer, which is common in knowledge distillation. In this way, given different images with the same label but from different generators or the real dataset, the classifier tends to output similar distributions. Thus the classifier is more robust against images from different distributions. We use consistency loss functions to ensure this:
the same. In this case, it is easier to reach global optimality instead of
Let min(- 0c
- Beg
x
DKI(f ,00,),)))111111))))))11)1 11 f(00, (x))))
(12)
L.z min( -E E (K0 .x) || f(oC ,z()i),,
(13)
0c
La min(
Eyg w
( DRL xx?) [(((g))))
(14)
Dc

---

Published as a conference paper at ICLR 2023
c== Lei Lez + Les-
(15)
Finally, the overall loss function for client is:
Letient = Ltocat + Les
(16)
where DR || Q) is the Kullback-Leibeer divergence, measure of how one probability distri- bution Q is different from the second probability distribution P. 6c is the parameter related to the classification module and f(0C, x) are the softmaxed output logits of the classification head.
Eq. can also be interpreted as using some soft labels for the classification task of real samples. With proper soft label, the classification module can trained more stably and has better general- ization thus can have better performance
& Lampert, (019). It is especially fitful for our CI-FL setting because better generalization benefits both class incremental learning (Mirzadeh et al.) 2020 (Buzzega et al. 2020) and federated learning, where non-IID data requires better generaliza- tion of the model. Intuitively, Eq. (14) can transfer knowledge from the global model to the client by the generated data X global to enrich the client with global knowledge.
Privacy. To protect privacy in federated learning, only sharing parameters is simple yet effective solution. Related works that train a GAN across distributed resources (Zhang et al., 2021 Rasouli al. 2020) adopts this strategy. We also adopt this solution and mainly Tocus on addressing the catastrophic forgetting problem in CI-FL. Some advanced methods like Private FL-GAN (Xin et al. 2020), which specifically address the privacy issue, be integrated into our framework if higher levels of privacy are required.
EXPERIMENTS
5.1 DATASETS
Choice of Datasets. We adopt commonly used datasets for federated learning (FL) Zhu et al. 2021b), suggested by the LEAF benchmark (Caldas et al., 2018). Proper modifications are made according to the definition of our CI-FI. setting. For vision tasks, MNIST, EMINST and CelebA are suggested. We not use CelebA because is built as a binary classification task in FL and not suitable for continual learning setting. Besides, add use CIFAR-1O. 10, which is a simple but still challenging dataset in data-free class-incremental learning (CIL). They are simple datasets for classification, but difficult and far from being solved in non-llI FL and CIL (Prabhu et al. 2020). For all datasets: MNIST, EMINST-Letters. EMNIST-Balanced and CIFAR-10, we build tasks for each client with 2 classes every task. Details of datasets and data processing can be found in Appendix A.
5.2 BASELINES
We compare our FedCIL approach with the following baselines, including two representative FL models and three models for continual federated learning. Details of baselines can be found in Appendix B. FedAvg (McMahan al. 2017). FedProx (Li et al. 2020). A representative FL model which is better at tackling heterogeneity in federated networks than FedAvg. FedLwF. 2T (Usmanova et al.] 2021). It exploits the idea of Learning without Forgetting(LwF) (Li & Hoiem 2017) in federated learning. FedDPro/Fed/Avg-ACC..
Replay (Also referred as Fed- Prox/FedAg--AGGAl in the paper). It is a combination of FedProx/FedAvg and ACGAN. Fed- Prox/FedAvg-DGR (Shin et al. 2017). It is a combination of FedProx/FedAvg and Deep Genera- tive Replay (DGR) (Shin et al. 2017).
5.3 EXPERIMENTAL. SETUP
Setting. We use five clients in all experiments and all of them participate in each communication with the server, following related works (Usmanova et al. 2021 Hendryx et al., (021). where five or less than five clients are used. In (Yoon et al.) 2021), the same setting is also used non-iid-50 dataset. Taking CIL into account when dealing with FL is very challenging, thus existing works and work adopt this setting to simplify the case for analysis.
Configuration. Unless otherwise specified, we set local training iteration 400 and global communication round R 200 for all models. As build five tasks for each client, there 40

---

Published as a conference paper at ICLR 2023
Table 1: Results on MNIST, EMNIST--ecters, EMNIST-Balanced and CIFAR-IO. We report the accuracy of the global model when all clients finish their training on all tasks, which is referred as global accuracy.
Model
MNIST
EMNIST-L
EMNIST-B
CIFAR-10
FedAvg (McMahan et al. 2017
72.28 + 0.82
19.36 = 0.95
0.225
27.21 = 2.39
FedProx (Li ct al. 2020)
72.84 0.73
19.69 = 0.75
0.55
27.43 = 2.46
Fed2. (Usmanova et al. 2021
75.61 + 0.93
23.91 + 0.78
17.22 + 0.90
27.02 + 2.38
FedAvg+DGR
97.46 + 0.51
71.92
0.74
63.55
0.46
37.93 + 2.27
FedProx+DGR
97.55 1 0.48
71.83 0.65
63.55 0.27
37.87 1 2.47
FedAvg+ACGAN Replay
97.13 + 0.35
73.85 + 0.17
66.87 + 0.79
38.31 + 2.64
FedProx#ACGAN Replay
97.38 + 0.63
73.91 = 0.29
66.19 = 0.92
38.34 = 2.55
FedCIL (Ours)
99.13 + 0.34
78.15 0.30
73.12 0.47
45.27 = 2.42
(a) FedCIL. (Ours)
(b) FedProx+DGR
(c) FeCANO
Figure 3: The confusion matrices for the global model (classifier) on the server in FedCIL, Fed- Prox+DGR and FedProx+ACGAN.
rounds per task. For each local iteration, we adopt mini-batch size EMNIST-B and B 100 for CIFAR-10. The number of generated samples in iteration is the same as the mini-batch size. The backbone of the feature extractor is a 3-layer CNN and that of generator is 4-layer CNN with batch normalization layers. The Adam optimizer is used with the learning rate lc-4. We run each experiment three times and report the mean and standard deviation.
for MNIST, EMNIST-L,
5.4 RESULTS AND ANALYSIS
Table shows the results on MNIST, EMNIST-Letters, EMNIST-BBlanced and CIFAR-10. From the results
catastrophic forgetting becomes more notable the similarity between tasks of different clients reduces. FedLwF-2T (Usmanova et al. 202T) outperforms FedAvg and FedProx on MNIST and E.MNIST-Letters. But on EMNIST-Letters and E.MNISTBalanced. with overlapped classes becoming less or almost none, its performance drops significantly. FedProx/FedAvg+DGGR and TedPPox/FedAAv--ACGAN Replay outperforms other baselines by large margin, which is more obvious when the setting becomes more difficult. Our model FedCIL further gains more improvement compared the two models. We further summarize how our proposed two methods benefit continual federated learning and illustrate it with examples in Appendix E. More discussions and results on FedCIL are available in Appendix F.
FedCIL learns better generators. To prevent forgetting, generative replay based methods exploit generator to generate data for replay. Thus the quality of generated images has an important influence on the model performance. The generators of clients in our model generate better data than others. As stated the weakness of simple combination, directly training ACGAN in heterogeneous federated learning setting with FL algorithm leads to the performance degradation of generators.
This is because the sever-client synchronization in every communication round damages each client's classification performance its local dataset at the early training stages of each round. Such classification accuracy decrease then lead to much larger classification loss according to Eq. (9). Fig. 2 shows this phenomenon. The unexpected huge classification loss can break the balance be- tween the losses from Eq. (5) and Eq. (6). thus model will overly tilt to the classification part. Eventually it weakens the performance of generator.
From Fig. 2 is observed that our model doesn't 't suffer from this problem. Our propose model con- solidation and consistency enforcement effectively prevent the gradient exploding in early stages of
8

---

Published as a conference paper at ICLR 2023
FID flower is botee)
-00
AUGIAN (Cffine)
FodProx ACGAN
.00
FedCIL (Duns)
FaN-Fallut
200
H
100
FedCaL (Chng)
4000
2000
2000
Nerations
Figure 4: FID scores of ACGAN + FedProx, ACGAN (offline) and ours. random client is selected during training EMNIST.
Visualization of generated images from clients in FedProx+AC-GAN (first row) and FedCIL (second row).
Figure 5:
every communication round, which guarantees the stability of the generator. Fig. 5 shows the gen- erated images by FedCIL and FedProx+ACGAN. visualizes that generators in FedCIL generates images of higher quality than FedProx*AAGGAN. FID score can be a measurement of the quality of generated images (lower better). Fig. 4 shows that generators our model is more stable and have better FID score than FedProx+ACGAN during the training.
FedCIL learns less biased classifiers and generators. Fig. shows the confusion matrices for the global models in FedCII., FedProx+DGiR and FedProx+ACGAT on EMNIST-Letters. It is observed that the global classifier our model less biased compared with two baselines. setting, each client is learning classes with instances for one class during a task. If some clients are learning same classes, the proportion of this class will be larger than other classes. Thus when the server and clients are synchronized, the server model is prone to be biased towards this class. Our model consolidation on the server side can effectively alleviate this phenomenon by consolidating the merged model with balanced generated data. In this way the knowledge on the less frequent classes can be strengthened. On the client side, with Eq. (15), the global knowledge can be better transferred the local clients, preventing the local clients from overfitting to its local dataset.
our CI-FL
FedCIL allows more effective server-elient communication. Different from iaaseinncreeetall learning (CIL), where we only need address the problem of catastrophic forgetting in a single model, both server and clients models need to be considered in CI-FL. From the perspective of CIL, each model can of course leam from the history state of itself. However, with other models available in the CI-FL, they are also expected to learn from each other. Compared with FedProx+DGR, where each client keeps a private GAN for generative replay and only synchronize classifiers with the server model, the generators in FedCII can effectively learn form others for better generation ability. Thus the generative replay can be more effective and achieve better performance.
Table 2: Ablation study
Method
Accuracy
Full method
78.15 0.30
Ablate model consolidation (Eq. 10
75.02 0.41
Ablate consistency enforcement (Eq
5
73.91 = 0.29
Ablate generative replay
19.36 + 0.95
Ablation Study. We perform ablation studies on EMNIST-Letter to evaluate the contribution of each component. Table _ proves the effectiveness of our proposed methods. Each module can improve the model performance effectively. The results are consistent with our analysis above.
Influence of local iterations. We study how the local iterations T affects our model in Appendix C.
6 CONCLUSION
In this paper, we introduce practical and challenging federated continual learning scenario, where a centralized server continuously learns from clients that are incrementally learning new tasks from streaming of local data. Furthermore, find simply adapting generative replay ideas into the setting fails to work as expected. Then we analyze its weaknesses and propose a new model FedCIL to address the problems. Experimental results benchmark datasets manifest its effectiveness.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENT
This research is supported by the the U.S. Army Research Office Award under Grant Number W911NF-21-1-0109 and Adobe Data Science Research Award.
REFERENCES
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pp. 139-154, 2018.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark expe- rience for general continual learning: strong, simple baseline. Advances in Neural Information Processing Systems, 33:15920-15999 2020.
Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečný, H Brendan McMa- han, Virginia Smith, and Ameet Talwalkar. Leaf: benchmark for federated settings. arXiv preprint arXiv:1812.01099 2018.
Fernando E Casado, Dylan Lema, Roberto Iglesias, Carlos V Regueiro, Senén Barro. Fed- erated and continual learning for classification tasks in society of devices. arXiv preprint arXiv:2006.07122, 2020.
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Mare' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv: 1902. 10486, 2019.
Hong-You You Chen and Wei-Lun Chao. Fedbe: Making bayesian model ensemble applicable feder- ated learning. arXiv preprint arXiv:2009.01974, 2020.
Zhixuan Chu and Sheng Li. Continual treatment effect estimation: Challenges and opportunities. arXiv preprint arXiv:2301.0000 2023.
Jiahua Dong, Lixu Wang. Zhen Fang, Gan Sun, Shichao Xu, Xiao Wang, and Qi Zhu. Federated class incremental learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pp. 10164-10173, 2022.
Sayna Ebrahimi, Franziska Meier, Roberto Calandra, Trevor Darrell, and Marcus Rohrbach. Adver- sarial continual learning. In European Conference Computer Vision, pp. 386-402. Springer, 2020.
Yongxin Guo, Tao Lin, and Xiaoying Tang. A new analysis framework for federated learning time-evolving heterogeneous data. FL-ICML, 2021.
Sean M Hendryx, Dharma Raj KC, Bradley Walls, and Clayton T Morrison. Federated reconnais- sance: Efficient, distributed, class-incremental learning. NeurIPS Workshop on New Frontiers in Federated Learning, 2021.
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.025311 2(7), 2015.
Minguk Kang. Woohyeon Shim, Minsu Cho, and Jaesik Park. Rebooting acgan: Auxiliary classifier gans with stable training. Advances in Neural Information Processing Systems, 34:23505-23518, 2021.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska Barrkkkkka al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3566 2017.
Abhishek Kumar and Hal Daume III. Learning task grouping and overlap in multi-task learning. arXiv preprint arXiv: 1206.6417, 2012.
10

---

Published as a conference paper at ICLR 2023
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Sys- tems, 2:429-455 2020.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40 (0(12):2935-2947 2017
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and mofivation, volume 24, pp. 109-165. Elsevier, 1989.
H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. Communication-effiiiee learning of deep networks from decentralized data, 2017.
Seyed Iman Mirzadeh, Mehrdad Farajtabar, Razvan Pascanu, and Hassan Ghasemzadeh. Under- standing the role of training regimes in continual learning. Advances in Neural Information Pro- cessing Systems, 33:7308-7322 7320, 2020.
Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxil- iary classifier gans. In International Conference on Machine Learning, pp. 1651-2651. PMLR, 2017.
Mary Phuong and Christoph Lampert. Towards understanding knowledge distillation. Interna- tional Conference on Machine Learning, pp. 5142-5151. PMLR, 2019.
Ameya Prabhu, Philip HS Torr, and Puneet K Dokania. Gdumb: A simple approach that questions our progress in continual learning. In European Conference on Computer Vision, pp. 524-540. Springer, 2020.
Mohammad Rasouli, Tao Sun, and Ram Rajagopal. Fedgan: Federated generative adversarial net- works for distributed data. arXiv preprint arXiv:2006.07228 2020.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. Advances in Neural Information Processing Systems, 32, 2019.
Kevin Roth, Aurelien Lucchi, Sebastian Nowozin, and Thomas Hofmann. Stabilizing training of generative adversarial networks through regularization. Advances in Neural Information Process- ing Systems, 30, 2017.
Paul Ruvolo and Eric Eaton. Ella: An efficient lifelong learning algorithm. International Confer- ence on Machine Learning, pp. 507-515. PMLR, 2013.
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. Advances in Neural Information Processing Systems, 30, 2017.
Sebastian Thrun. Is learning the n-th thing any easier than learning the first? Advances in Neural Information Processing Systems, 8, 1995.
Anastasiia Usmanova, François Portet, Philippe Lalanda, and German Vega. A distillation-based approach integrating continual learning and federated learning for pervasive services. IJCAI 3rd Workshop on Continual and Multimodal Learning for Internet of Things, 2021.
Gido M Van de Ven and Andreas S Tolias. Three scenarios for continual learning. arXiv preprint 190.00190 2019.
Chenshen Wu. Luis Herranz, Xialei Liu, Joost van de Weijer, Bogdan Raducanu, et al. Memory Learning to generate new categories without forgetting. Advances in Neural Infor- mation Processing Systems, 31, 2018.
replay gans:
11

---

Published as a conference paper at ICLR 2023
Yue Wu, Pan Zhou, Andrew G Wilson, Eric Xing, and Zhiting Hu. Improving gan training with probability ratio clipping and sample reweighting. Advances in Neural Information Processing Systems, 33:5729-57445 2020.
Bangzhou Xin, Wei Yang. Yangyang Geng, Sheng Chen, Shaowei Wang. and Liusheng Huang. Private fl-gan: Differential privacy synthetie data generation based on federated learning. In ICASSP 2020-2020 IEEE International Conference on Acousties, Speech and Signal Processing (ICASSP), pp. 2927-2931. IEEE, 2020.
Kunran Xu, Lai Rui, Yishi Li, and Lin Gu. Feature normalized knowledge distillation for image classification. European Conference on Computer Vision, pp. 664-680. Springer, 2020.
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint ar.iv:1708.01.5550 2017.
Jachong Yoon, Wonyong Jeong, Giwoong Lee, Eunho Yang, and Sung Hwang. Federated contin- ual learning with weighted inter-client transfer. In International Conference on Machine Learn- ing. pp. 12073-12086. PMLR, 2021.
Yikai Zhang, Hui Qu, Qi Chang, Huidong Liu, Dimitris Metaxas, and Chao Chen. Training federated gans with theoretical guarantees: A universal aggregation approach. arXiv preprint 0455055002. 2021.
Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. arXiv preprint arXiv:/806.0055: 2018.
Hangyu Zhu, Jinjin Xu, Shiqing Liu, and Yaochu Jin. Federated learning on non-iid data: survey. Neurocomputinng 465:371-390, 2021a.
Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou. Data-free knowledge distillation for heterogeneous federated learning. In International Conference on Machine Learning, pp. 12878-12889, PMLR, 2021b.
12

---

Published as a conference paper at ICLR 2023
A DATASET
MNIST: It is a digit image classification dataset of 10 classes with training set 60,000 instances and test set of 10,000 instances. We split instances of each class into non- overlapped parts. Each client randomly picks up two classes without replacement to form a task and it is repeated for five times for form a sequence of five classes. Note that for any client, a class already exists in any of its tasks, the client will not pick up this class again. This ensures that for any client, a class will not appear more than once in its private sequence, which is subject to the definition of inssemeactmttlll learning. In the setting, classes from different clients lapped from the overall view, however, during each task, clients are still learning different tasks, the data is non-IID and is still challenging federated learning problem.
highly overlapped. Note that although it is highly over-
It is a character classification dataset of 26 classes containing 145,600 instances. We process this dataset similar to MNIST and each client also randomly picks up two classes to form task and repeats five times in total. It is harder because classes from different clients are less overlapped and the sever eventually learns more classes.
EMNIST-Letters:
EMMIST-Balanced: It is a digit and character classification dataset of 47 classes contain- ing 131,600 instances. We follow the same steps to process this dataset. It is the hardest because there are almost no overlapped classes from different clients and the server learns more classes.
CIFAR-10: It is an image classification dataset of 10 classes with 60,000 instances. The process on this dateset is the same as MNIST.
Figure illustrates the how we build tasks for clients.
Dataset
Instances
Task
Task
Cliant
labela
Cliant
2
:A batch of Instances af clsss
Figure 6: Illustration of how to build tasks for clients. Different color represents different classes. We split instances of each class into non-overlapped parts. Each client randomly picks up two classes without replacement to form a task and it is repeated for five times for form a sequence of five classes. Note that for client, if class already exists in any of its tasks, the client will not pick up this class again. This ensures that for any client, class will not appear more than once in its private sequence, which is subject to the definition of class-incremental learning.
B BASELINES
FedAvg (McMahan et al. 2017): It is a representative federated learning model, which aggregates client parameters in each communication. It is a simply yet effective model for federated learning.
FedProx (Li et al. 2020 It is also a representative federated learning model, which is better at tackling heterogeneity in federated networks than FedAvg.
FedLwF-2T (Usmanova et al. | 2021). >) uses the idea of Learning without Forgetting(Lwd
&Li Hoiem 2017) in federated learning. LwF keeps copy of itself at the end of each task, which acts as a teacher to remind the current model of previous knowledge via knowledge distillation (Hinton et al. (2015). FedLwF-2T similar LwF and uses the global model as the second teacher during the local training of the clients.
FedPros/FedAggAAGGAN Replay (Also referred as FedProwFFdAAAgAAGGAY in the paper). It is combination of FedProx/FedAvg and ACGAN. Each client is ACGAN
13

---

Published as a conference paper at ICLR 2023
Centralteed Surver
Centralized Servee
Centralized Server
Server
Medel Consoldattin
Clizat side: Cunsitency Enfarcement
To
Ta
I
Clieat
Clisnt
Client
Client
Clicett
Client
Cllent
Cilent
FedCIL (Ours)
FedProx-ACGAN
FedProx-DGR
Figure 7: Illustration of our model and baselines.
based model and uses generative replay (i.e., exploiting the samples generated by the gen- erator of the ACGAN for replay to prevent forgetting) locally. The global model and the clients are synchronized by FedProx or FedAvg.
FedProo/FedArg=DDGH (Shin et al., 2017). It is combination of FedProx//edAAg and Deep Generative Replay (DGR) (Shin et al., 2017). Each client consists of a GAN and classifier. The GAN is trained additionally for generative replay. The train process of client is the same as the process in (Shin et al. 2017). The global model and the clients (the classifiers) synchronized by FedProx FedAvg.
Figure illustrates our model and baselines.
C INFLUENCE OF LOCAL ITERATIONS
To study how the number local iterations affect our model, we experiment on EMNIST-Letters with different local iterations from 50 to 1000. The accuracy curve tends to be more gentle as the number local iteration increases, which helpful when the local training time is limited. We can have compromise between performance and speed according to different application scenarios.
Accuracy
FoICI.
200
400
600
800
1000
Numbr of local Iterations
Figure 8: Influence of different local iterations
D DESIDERATA
We have some desiderata for the proposed setting, which makes the class-incremental federated learning more general, realistic and challenging, compared with existing works.
1. For a client, it is continuously learning new classes, and the learnt classes will be unavailable in future tasks. We strictly follow typical continual learning setting, in which there are no overlapped classes among tasks, and thus more challenging. This is significant difference from (Hendryx) (et al. 202I) and (Guo et al. 2021). where any instance only appears once but different instances of the same class may appear different tasks in the task sequence of client.
For different clients, there may be overlapped classes in their respective private tasks. This is practical because each client is learning their tasks without communicating with each other. Thus, it is reasonable that we do not put extra assumptions on the overall data distribution of all clients.
14

---

Published as a conference paper at ICLR 2023
0.75
0.75
Accuracy
Accuracy
03
04
0.85
0.66
0.80
0.60
0.55
0.88
0.50
0.90
Communication rounds (R)
Communication rounds (R)
(a) FedCIL (Ours)
(b) FedProx+ Replay
Figure 9: Case study on EMNIST. We record the performance of each client (1-5) on its own local dataset immediately after receives the parameters from the global model. We plot this perfor- mance of each client for several communication rounds during training.
3. The clients et al. 2021 and (Yoon et al. 2021).
allowed store previous data in buffer, which is consistent with (Ummanoaa
4. The clients not allowed to communicate with each other at any time. To follow the privacy constraints in typical federated learning, neither the raw data sharing between the server and the clients nor among the clients is allowed.
At any given time-step i, the global model should be able to classify all classes that all clients collected and learned so far. means that the global model is dynamically updated all the time, which makes it a powerful lifelong federated learning model. On the contrast, (Yoon et al., [021] learns client models and does not maintain global model that discriminates all classes without task-IDs.
The number of communication rounds should be flexible according to different settings. This is different from (Usmanova et al.) (2021), where a round of communication happens when client finishes its learning on local dataset, which will be no longer available in the next round. Our assumption is more general and follows a typical continual learning paradigm.
E
CASE STUDY
In Section 4.2, we demonstrate when the generation ability of ACGAN degrades and how this phe- nomenon is aggravated in the context of federated learning. To summarize, the performance of ACGAN degrades when the gradient norm (Eq. (9) grows too large and breaks the balance be- tween CD (0, X) and CD (0,X) X). The over-tilt to L (0, X) limits the generation ability and finally decreases the model performance.
The gradient exploding happens in early stages of each local training (Fig Zlnd Fig. 4 According to Eq. (9) the gradient norm associated with 1. the predicted class probability distribution and 2. feature norms. Our model consolidation and consistency enforcement designed from the two perspectives.
E. PREDICTED CLASS PROBABILITY DISTRIBUTION
The gradient norms of baseline models are very large at early stage of local training because the predictions highly inaccurate the beginning. Received global model are directly merged from clients from last round via FedAvg or FedProx. Empirically we find such merging hurts the local performance of clients to large extend and yields highly inaccurate predictions on local data, leading to large gradient norms (Eq. 9h)
E.1.1 CASES: WHEN MERGED GLOBAL MODEL IS INACCURATE?
We conduct case study, where record the performance of each client its local dataset after it receives the parameters from the global model. Higher performance at this moment indicates lower gradient norms according Eq. (9.
15

---

Published as a conference paper at ICLR 2023
In Fig. 9 one client (C4) consistently behaves notably worse than others in the baseline model. In fact it is common in baseline models and sometimes more than one clients behaves this way. And the global model is biased towards some classes (Fig 3 We observe this phenomenon happens to some of the clients in the following cases:
Imbalanced classes For instance, during the first task in EMNIST-L. Client 1 learns classes {18, 23), Client 2 learns classes {10, 11), Client 3 learns classes {15, 23). In this case, after model averaging the server, performance on Client 2's local data is impaired and is notably worse than others. Because Client 1 and Client both learn from class 23, the merged model tends to be biased towards class 23.
Different difficulty levels (a) For instance, during the second task in EMNIST-L, Client 4 learns classes [14, 21}, none of which are learnt by other clients before. Client learns classes {2. 20), where class 2 is learnt by Chient in Client 3's previous task. In this case, the merged model has worse performance on the Client 4's private task than on Client I's. This is because the merged model tends to be biased towards easier classes.
Different difficultyy levels (b) On EMNIST-B, We also observe that, even when classes are balanced and all incoming classes are not learnt by any of the clients, i.e., it is not one of the two mentioned cases above, some clients still learn worse than others. We attribute inherit difficulty of different tasks. Clients that are learning more difficult tasks tend to have worse performance on their local data immediately after receiving the merged global model from the server.
the differences of the
E.1.2 SOLUTIONS: How FEDCIL WORKS IN THE CASES?
In model consolidation, synthetic samples, which are used to consolidate the merged model, are gen- erated by uploaded client models respectively before they merged to global model. Thus the generated image quality is not affected by three cases above. Therefore, consolidating the merged model with these high-quality synthetic images largely eliminates the model performance degrada- tion caused by above cases. Besides, we generate balanced samples conditioned on classes, which additionally helps solve the class imbalance problem.
After our model consolidation, the server model is less biased towards different clients' tasks or classes. Fig. and Fig. illustrate our effectiveness towards this aspect. Our consistency enforce- ment does the same thing, but from the perspective of client side: we distill the knowledge from the global model to local clients by generating class-balance samples with the global model.
To summarize, in FedCIL, each client, which is initialized from the our better consolidated global model, makes more accurate and consistent predictions at the early stage of the local training. largely prevents the ACGAN from the early gradient exploding problem.
E.2 FEATURE NORMS
According to Eq. (9). lower feature norms also help reduce the gradient norms. Previous study (Xu cet al.) 2020) reveals that knowledge distillation can help to reduce the overly high feature norms caused by noisy samples.
From this perspective, the consistency enforcement, which is also a kind of knowledge distillation in terms of form, help reduce the feature when training with noisy generated samples, in addition to its benefits on learning less biased models mentioned in Section 4.3 and appendix E.
F DISCUSSIONS
F.I DIFFERENCES WITH REACGAN
From a high-level view, Both ReACGAN Kang et al. 2021) and our method can stabilize the train- ing process of ACGANs, but they are designed to solve entirely different problems with different motivations. Thus not surprising that ReACGAN hardly helps to improve the model perfor- mance in Tab.
16

---

Published as a conference paper at ICLR 2023
Model
Accuracy
FedAvg + DGR
63.55
FedAvg + ACGAN Replay
66.19
FedAvg + ReACGAN Replay
65.87
FedCII. (Ours)
73.12
Table 3: Model Performance with Difference GR Strategies on EMNIST-B.
ReACGAN stabilizes the offline training of one ACGAN by exploiting feature normalization and relational information in the class-labeled dataset. It doesn't consider challenges from federated learning and continual learning. In other words, it is neither a federated learning nor continual learning algorithm. Instead, FedCII, stabilizes the distributed training of multiple ACGANs by improving the server-side aggregation of client models and client-side feature learning (with the help of the global model). Our model consolidation and consistency enforcement are designed for the non-iid distribution and the catastrophic forgetting challenges our continual federated learning setting. ReACGGN is not designed for these purposes.
F.2 LOCAL PERFORMANCE
A typical federated learning scenario aim to learn a global model with distributed data, and thus local performance is usually not considered in evaluation. We added experiments see how the private performance vary across different models. Tab shows the results.
Model
Accuracy
FedProx DGR
85.97
FedProx + ACGAN Replay
86.40
FedCIL (Ours)
88.73
Table 4: Local Model Performance EMNIST-L.
It is interesting that our model has higher average private accuracy on the client side. We agree that federated learning. better global accuracy often contradicts local accuracy. the extreme case, if each client only does local training without communications, it is more likely to have higher performance on own dataset with low global accuracy. Here we believe the continual learning part plays an important role. Because our model better remembers previous knowledge, the extra benefits from continual learning leads to its higher accuracy.
17


Reviews:
[{"review_id": "Ob2n2VfcCpB", "reviewer": "Reviewer_sMxb", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes the continual federated learning problem which has properties of both federated learning and continual learning, and proposes modifications such as model consolidation and enforced consistency to stabilize ACGAN to solve the proposed problem. The proposed approach achieves the best performance in the proposed setting, and achieves stable training similar to single-model learning of ACGAN.\n", "summary_of_the_review": "Due to concerns around the relevance of the problem, the lack of practical motivation, and the overclaimed contribution, I recommend reject for the manuscript.", "strength_and_weaknesses": "Strength\n1. The work is reasonably well written and easy to follow.\n2. The empirical results of the proposed approach (Table 1) is strong compared to baselines.\n\nWeakness\n1. Overclaiming. The authors frame “proposing the continual federated learning problem” as a major contribution, but much of the related work already focuses on various forms of continual federated learning despite slight difference from the proposed setting (e.g., class overlap). \n2. Relevance. It felt to me that the authors curated a very narrow setting for which their method can outperform existing baselines (e.g., multi-task, global model, no class overlap, etc). If this curated setting is indeed highly relevant, it would be nice to motivate the proposed setting with practical examples.\n3. It is unclear from the experimental results how crucial model consolidation and enforced consistency are in stabilizing ACGAN. Ablations or inclusions of other approaches to stabilization can help with this.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "E6STBt0GvS", "reviewer": "Reviewer_qLZV", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In the continual federated learning, the stream input data usually is provided non__iid which affects adversely on the learning performance.  The authors used generative replay idea for continual learning, in which there is no need to transfer data among the clients and servers (in federated learning framework), rather the learned parameters can be shared. Namely, Auxiliary classifier GAN (ACGAN), a study from Odena et al. (2017) has been used as the generative model to avoid forgetting of the previously learned tasks. As the mentioned non_iid nature of the incoming data makes the federated learning difficult and using ACGAN as the generative replay fails, the authors proposed two technical tricks which could help this combination succeed.  The first trick, called as model consolidation, in which the clients parameters are aggregated and used to initialize the server parameter. Then, from the clients generators, balanced synthetic data is generated to use for enhancing training in the server side.\nAs the second trick, “consistency enforcement”, is performed in the client side. Each client’s loss includes two terms. In the first term, the new coming training data  are mixed with the synthetic data generated by the global generator conditioned on the labels from previous tasks, hence the client not only learns the new classes, but also retains the previously learned classes. \nFor the second term, consistency loss function, the idea is to include kullback divergence to enhance generating the same distribution for the real data and the data generated from the global model generator and the generated data of the past or current task generators, when the label is the same. ", "summary_of_the_review": "At this point, this paper is a border-line study in my opinion. \n ", "strength_and_weaknesses": "The paper is well written and the topic is well motivated in the introduction. The contribution is clearly mentioned and explained.\nThe experimental results seem to outperform the state of the art approaches.\n\nIt mainly addresses the instability in this specific learning framework by enhancing the generalization of both clients and servers. Consequently, some locality should be sacrificed. Have you observed this in your experiments?\n\n\nThe proposed tricks seem to be effective and interesting however I doubt the significance of the contributions make it succeed to be published as a full paper at ICLR.  ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "8TT9nWpKdG", "reviewer": "Reviewer_S7Qy", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a continual federated learning problem where each client deals with a class-continual learning problem with a sequence of tasks independently, and the server learns the tasks of all clients via simple model averaging. This paper observes that the aggregate server model suffers high loss and FID score on individual client data, and proposes two methods (model consolidation and consistency enforcement) to handle this issue. Extensive experiments on multiple classification datasets show that their method has significant advantages over baseline methods which naively adopt data-free class-incremental learning approaches on a federated learning framework.", "summary_of_the_review": "While the writing needs to be improved and a more comprehensive evaluation is required, I believe that this paper has two major contributions: 1) it proposes a realistic continual federated learning problem, 2) it tackles the problem of training a GAN-based model in federated learning in the presence of data heterogeneity over clients.\n", "strength_and_weaknesses": "### Strong points\n\n- This paper introduces continual federated learning, which is a challenging but more realistic problem than existing federated learning. Continual learning without a memory buffer in the federated setting is an interesting and important problem\n- The idea of finetuning the server model using individual clients’ generators (expressed as “model consolidation” in the paper) seems original to me. Since the server can receive the generators trained with individual local data distributions, fine-tuning the averaged server model with the synthetic data is a straightforward approach for transferring local knowledge to the server model.\n\n### Weak points\n\n- The experiments are difficult to understand, and missing details.\n    - How many communications are performed for each task?? It seems that the degenerated server aggregation issue pointed out in the paper can be alleviated via enough communication per task.\n    - How the classification nodes are added and aggregated when training the next task?\n    - How many synthetic images are generated for each generator per iteration?\n- Evaluation on the more challenging datasets is required such as CIFAR-100 or ImageNet\n- Each client should transfer the generator to the server for every communication round, which can raise privacy concerns since we can easily synthesize the raw data using the generator.\n- The writing is hard to follow, a few statements are vague, and notations are imprecise\n    - Why do the proposed two methods mitigate the increased loss issue at server aggregation?\n    - Subscript at $\\theta$ denotes a client index in the left figure in Figure 1, but the subscript denotes task index in other parts of the paper, e.g., eq. 11\n- Questions\n    - $\\theta_{\\text{global}}$ in Eq. 10 only denotes the parameter of the generator? or denotes parameters of the generator, discriminator, and classifier?\n- FedCIL requires more communication cost than FedAvg or FedProx + DGR since each client communicates all parameters of the model including generator, discriminator, and classifiers.\n\n### Minor comments\n\n1. Typos\n    1. Page 5: federated leading → federated learning\n2. The paper should include a discussion about the recent federated learning approach [1].\n\n\n### Reference\n\n[1] J. Dong et al., Federated Class-Incremental Learning, CVPR, 2022", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster