Federated Continual Learning Goes Online: Uncertainty-Aware Memory Management for Vision Tasks and Beyond

Authors: Giuseppe Serra, Florian Buettner
Primary area: transfer learning, meta learning, and lifelong learning
Keywords: Federated Continual Learning, Catastrophic Forgetting, Uncertainty Estimation

Abstract:
Given the ability to model more realistic and dynamic problems, Federated Continual Learning (FCL) has been increasingly investigated recently. A well-known problem encountered in this setting is the so-called catastrophic forgetting, for which the learning model is inclined to focus on more recent tasks while forgetting the previously learned knowledge. The majority of the current approaches in FCL propose generative-based solutions to solve said problem. However, this setting requires multiple training epochs over the data, implying an offline setting where datasets are stored locally and remain unchanged over time. Furthermore, the proposed solutions are tailored for vision tasks solely. To overcome these limitations, we propose a new approach to deal with different modalities in the online scenario where new data arrive in streams of mini-batches that can only be processed once. To solve catastrophic forgetting, we propose an uncertainty-aware memory-based approach.  Specifically, we suggest using an estimator based on the Bregman Information (BI) to compute the model's variance at the sample level. Through measures of predictive uncertainty, we retrieve samples with specific characteristics, and – by retraining the model on such samples – we demonstrate the potential of this approach to reduce the forgetting effect in realistic settings while maintaining data confidentiality and competitive communication efficiency compared to state-of-the-art approaches.

Decision: accept (poster)
Reviewer scores: 8, 6, 6, 5

Reviews and discussion:
<thread>
creation date:
1730500875567

thread_type:
official_review

<thread_object>
creator: reviewer_o8mc

<content>
summary:
The paper addresses Federated Continual Learning under the challenges of catastrophic forgetting and online learning, where new data arrives in sequential mini-batches and can only be processed once. The authors introduce an uncertainty-aware memory management strategy that leverages Bregman Information to measure predictive uncertainty. This approach selectively retains samples in memory based on their uncertainty, aiming to improve learning retention while handling various data modalities beyond vision tasks. The paper also showcases the method’s performance on vision and text data, demonstrating its capability to mitigate forgetting across diverse, real-world settings.

strengths:
1. The paper presents a novel framework for Federated Continual Learning in an online setting, effectively addressing the practical limitations faced in scenarios where clients continuously receive new data without the ability to revisit previous data.

2. The innovative use of Bregman Information for uncertainty estimation allows for selective sample retention based on epistemic uncertainty.

3. The authors conduct extensive experiments with detailed analyses, demonstrating the effectiveness and depth of the proposed approach across various data types and scenarios.

weaknesses:
see questions.

questions:
1. The performance improvements on the CRC-Tissue and KC-Cell datasets are marginal. Given that these datasets contain more samples than others, could you provide a justification for this outcome?

2. How does the Bregman Information approach to uncertainty estimation compare with recent probabilistic methods in Federated Continual Learning, particularly regarding computational efficiency and predictive accuracy?

3. Most results are primarily compared to other uncertainty measurements, while comparisons with previous continual learning or federated learning methods appear limited.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730653971653

thread_type:
official_review

<thread_object>
creator: reviewer_bins

<content>
summary:
The paper proposes a federated learning with end points doing online continual learning (CL), called online-FCL. To address the new task, the paper proposes a memory based CL method that manage samples based on its uncertainty score using Bregman Information (BI). The proposed method is evaluated on CIFAR10/100. On vision, medical and textual datasets, the proposed method does not show much of gain but in multi-modal data, the proposed method improves the performance over the state of the arts by significant gains.

strengths:
* Proposing a new federated learning with **online** continual learning.
* Proposed method improves the accuracy significantly over the state of the arts in multi-modal (vision-and-textual task) setup

weaknesses:
* The proposed method is a straightforward application of Gruber & Buettner (2023), thus the technical novelty is limited.
* Empirical gain seems marginal (also considering the standard deviation) compared to prior arts (Table 1-4). But in Table 5, the empirical gain in vision and textual task seems significant. Any reasoning for this?
* Empirical validation is limited due to the size of the dataset. Although CIFAR-10/100 are popularly used datasets in CL literature, they are quite small sized. ImageNet-1K would be the minimum large scale dataset to validate the value of the proposed method.
* Some results are not clear 
  * In Fig. 3, why the BI and MFCL improve the last accuracy significantly only at the last accuracy?
* In L291-292, it is claimed that "the BI-based estimation of the epistemic uncertainty is meaningful also under distribu- tion shift and able to identify robust and representative samples". But is this fact used in your method? If so, can you analyze that this is true in the Fed CL context?
* Presentation can be improved.
  * In several lines, line break has been removed -- L056, L271, L290 to name a few
  * Some abbreviation is not used without definition -- L072 CF -> catastrophic forgetting (this should be defined in L048)
  * Figure 1 could be larger and clearer. Currently it is drawn with too small fonts, making visibilty bad.
* Misc.
  * Similar previous work that uses CE loss for uncertainty of a sample is missing: Koh et al., **Online Boundary-Free Continual Learning by Scheduled Data Prior**, *ICLR 2023* (the method called CLIB)

questions:
Please refer to my comments in the weakness section.

======== Justification for my final rating after the discussion with authors ================
The authors argue the technical novelty of the method of using BI for online continual learning and I appreciate/agree to it. While the technical novelty is acknowledged, the empirical validation with a small scale dataset still limits the value of the work. But with the novelty issue has been resolved, I raised my rating to borderline accept as my final rating.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731059039554

thread_type:
official_review

<thread_object>
creator: reviewer_2eag

<content>
summary:
The paper proposes an uncertainty aware-memory-based approach for federated continual learning in an online setting where an estimator based on Bregman Information is employed to compute model’s variance at sample level. The proposed method includes predictive uncertainty-aware updated coupled with random replays.

strengths:
1.	The problem statement is meaningful and not widely talked about.

2.	The proposed method works considering memory management in an uncertainty sampling setting, and handles class imbalance and data scarcity well. 

3.	The performance looks promising and overall the paper is easy to read.

weaknesses:
1.	The fundamental idea of memory management is based on predictive uncertainty which is highly model-dependent and widely used.

2.	While federated continual learning in online settings is not very common, the paper proposed uncertainty estimator and random sampling for replay sets are not novel.

3.	A scalability issue may arise for large datasets such as ImageNet.

4.	It’d be interesting to see how the proposed method would work if task numbers were increased (>20).

questions:
In table 5, for BI method, why do you think the last forgetting(F) score is lower than FedCIL for CIFAR100? Is it related to the increased number of classes?

Please refer to weaknesses section.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731076644689

thread_type:
official_review

<thread_object>
creator: reviewer_hu4e

<content>
summary:
This paper presents a new federated continual learning setting to deal with different modalities in the online scenario. The proposed uncertainty-aware memory-based approach uses an estimate to measure of predictive uncertainty of samples, The extensive experiments demonstrate the effectiveness of reducing the forgetting effect in realistic settings while maintaining data confidentiality and competitive communication efficiency.

strengths:
- A real-world online federated continual learning setting is proposed to adapt the practical scenario where new data arrive in streams of mini-batches that can only be processed once.
- Using a bias-variance decomposition of the cross-entropy loss for classification tasks has some merits.
- Experiments are comprehensive and well-designed.

weaknesses:
- Some advanced memory replay methods are lacking.
- Why do not consider some uncertainty-aware continual learning methods, such as [1] for comparison?
- Why the number of $M$ is different from different datasets?
- The improvement on the KC-Cell dataset is incremental compared to the other datasets. why?
- On the left of Fig. 3, it is better to discuss the notable performance gap between the proposed method and the others on the last task.

[1] NPCL: Neural Processes for Uncertainty-Aware Continual Learning.

questions:
Please see the weakness above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732203570213

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General response to reviewers

comment:
We thank the reviewers for their detailed and thoughtful feedback. We are pleased that the reviewers appreciate the _new_ (reviewer BinS) **online**-FCL problem which represents a practical scenario (reviewer hU4E) to address the limitations faced in scenarios where clients continuously receive new data without the ability to revisit previous data (reviewer o8MC). The problem statement is deemed as "meaningful and not widely talked about" (reviewer 2eAg) and the experiments are "comprehensive, well-designed" (reviewer hU4E), and "extensive with detailed analyses" (reviewer o8MC).

We are thankful for the encouraging and constructive comments aimed at improving the quality and soundness of our manuscript. Below we summarize the key points addressed during the rebuttal period:

- **Novelty**: We further discuss novel aspects of our submission in response to reviewers 2eAg and BinS. We agree that the *technical* novelty is limited, as we indeed build on the theoretical contribution from [1] who introduce a general bias-variance decomposition for strictly proper scores, and identify the Bregman Information as the variance term. We leverage this insight and, interpreting BI as a measure of epistemic uncertainty that is statistically well grounded in a bias-variance decomposition, are the first to suggest to use the Bregman Information (that we estimate via TTA) for memory management. We would like to emphasize that it is widely accepted that novelty in science does not only stem from technical aspects, but could also come from proposing new scenarios, shedding light on overlooked aspects, drawing new connections between research areas and applications, and so on [2]. In this spirit, we argue that the novelty of our submission does not only stem from the introduction of TTA-based BI as an uncertainty measure in online FCIL. More specifically, novel aspects of our paper include that we (i) formalise online federated CIL as a practically relevant generalisation of FCIL; (ii) demonstrate that state-of-the-art generative approaches such as MFCL and FedCIL adapted from offline federated CIL to online federated CIL do not work in the online setting and discuss the reasons for the observed performance gap; (iii) show that uncertainty-based approaches consistently outperform generative approaches for this task; (iv) point out that most of the uncertainty metrics mostly capture the irreducible aleatoric uncertainty and hypothesize that it may be more beneficial to focus on the epistemic uncertainty, and (v) introduce the use of the Bregman Information as an alternative method for predictive uncertainty estimation in the context of memory-based methods – which is largely unexplored in both federated and non-federated settings.

- **Scalability**: Given the online nature of the proposed framework, the total number of samples in a dataset does not affect the scalability of the algorithm. Since we do not use the whole dataset at once but rather process mini-batches sequentially, the only important factor is the time for processing a mini-batch. To address the scalability concerns of reviewers 2eAg and BinS, we now include an additional experiment to evaluate the time for processing a mini-batch for different image sizes for our method and MFCL.

- **Clarification about Figure 3**: In response to Reviewer hU4E and Reviewer BinS, we clarify that the notable performance gap on the last task graphically describes what is the effect of Catastrophic Forgetting (CF) on the learning model. In general, the learning model tends to mainly focus on the last task. This effect is pronounced for approaches without replay mechanisms (such as FedAvg) while it can be mitigated with generative-based replay (such as MFCL) and even more with our approach. 

- **Additional experiments and ablation studies**: We follow the suggestions of reviewers 2eAg and BinS and now include an analysis of the effect of the number of tasks (5, 10, 20, 25) on the performance, and an additional evaluation on the TinyImageNet dataset to further expand the number of datasets considered in our experiments.

[1] Sebastian Gruber, Florian Buettner. "Uncertainty Estimates of Predictions via a General Bias-Variance Decomposition." Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:11331-11354, 2023. 

[2] Michael J. Black, “Novelty in Science”, https://perceiving-systems.blog/en/news/novelty-in-science, 2022
</content>
</thread_object>

</thread>
<thread>
creation date:
1732635791595

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Revised version of the paper

comment:
Dear all, 

Following the comments and suggestions in the reviews, we uploaded a new version of the paper. The revision includes the following major changes:

1. **Additional experiments and ablation studies**: following the suggestions of reviewers 2eAg and BinS, we now include an analysis of the effect of the number of tasks (5, 10, 20, 25) on the performance (in Appendix A.7), and an additional evaluation on the TinyImageNet dataset to expand further the number of datasets considered in our experiments (Appendix A.4).

2. **Clarification about Figure 3**: in response to reviewers hU4E and BinS, we add a clarification about the notable performance gap on the last task graphically described in Figure 3. Due to space constraints, the clarification is in Appendix A.6.

3. **Scalability**: to address the scalability concerns of reviewers 2eAg and BinS, we now include an additional experiment in the ablation study (Appendix A.7) to evaluate the time for processing a mini-batch for different image sizes.

4. **Presentation and minor changes**: following suggestions from reviewer BinS, we improve the presentation of the paper by fixing the abbreviation definition of catastrophic forgetting, removing the mentioned line breaks, and increasing the image size and font size of Figure 1 to increase its visibility.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes a new framework for Federated Continual Learning in an online setting. The authors introduce a mechanism to selectively retain representative samples in memory, emphasizing epistemic uncertainty. The method demonstrates competitive performance across multiple modalities, including vision and text, and effectively balances learning retention and communication efficiency. 

After the author-reviewer discussion period, this paper received mixed final ratings. Two reviewers are positive towards this submission, while the other two reviewers still remain negative. Specifically, Reviewer o8MC didn't actively participate in the discussions, while Reviewer BinS confirmed they still have remaining concerns. 

After checking the paper, reviews, and authors' responses, I think the overall quality of this paper is impressive. Extensive results and detailed responses are provided. Therefore, I recommend acceptance of the paper. The authors should include the additional discussions and results in the final revision. They should also revise their paper to address Reviewer BinS's remaining concerns.