Paper: A Bayesian Framework for Clustered Federated Learning (ICLR-2024)

Reviewer comment (Writing):
The paper's overall clarity is compromised due to unnecessarily complex and at times undefined notation, making it challenging to follow.

Author response:
The notation complexity is due to the indexing of the multiple association hypotheses, being random finite sets themselves, which is at the core of the conceptual solution of the association problem brought by clustering. It would be helpful if the reviewer could point out notation issues so we could address them. Notice that there is a summary table with notation in the appendix and that we are revising it to make sure relevant variables are defined.

Reviewer comment (Theory):
The update mechanism for the parameters of the Bayesian model lacks explicit explanation, leaving a crucial aspect of the methodology unclear.

Author response:
At every communication round we use the Laplace approximation approach as discussed in "Baseline models and system settings" in Section 5.

Reviewer comment (Novelty):
The comparison and discussion of related work are superficial, lacking a fundamental exploration of how the proposed method differs from other clustering approaches.

Author response:
We agree with the reviewer that the discussion presented in the related work section could be improved. We are working on improving the discussion for the revised manuscript.

Reviewer comment (Novelty):
The claim that the paper provides a "unified framework with a rigorous formulation" and offers "new theoretical insights and algorithms" is disputed, as the paper fails to present any novel theoretical contributions.

Author response:
We strongly disagree with the reviewer on this point. We formulated the problem of clustered federated learning as a Bayesian data association problem between clients and model parameter distribution. The theoretical approach presented in Section 3 is completely novel in the FL literature: section 3.1 presents the conceptual formulation and solution to compute the joint posterior of model parameters and data association; section 3.2 extends 3.1 to recursive Bayesian updates and associations, which are implemented as FL iterations and yield to an optimal yet intractable solution as pointed out in section 3.3. Furthermore, the derivations are far from trivial, they are rigorous in the sense that they rely on consolidated Random Finite Set and Bayesian theories with few assumptions, no approximations or heuristics. The proposed approach is a unified framework since the theory proposed in Section 3 is general and can lead to multiple solutions as those in Section 4. The proposed conceptual solution, however, leads to a high number of data associations and, consequently, to high computational complexity if applied naively. Nevertheless, the insights provided by the theory laid out in Section 3 were used to propose the approximations discussed in Section 4 which successfully aim at reducing the complexity through novel algorithmic solutions. We modified the title of Section 3 as "Optimal Bayesian solution for clustered federated learning" to make it clear that the material there is already part of the novel contribution of this paper, which we agree was not clear earlier when titled "Problem formulation". Overall, we believe that the proposed approach touches all the points we claimed. We hope that the reviewer can reconsider such a strong claim that the paper fails to present any novel theoretical contribution.

Reviewer comment (Theory):
Section 3 contains derivations that are deemed obvious, contributing minimally to the understanding of the proposed approach.

Author response:
1) We call the reviewer's attention to the fact that the conceptual ideas motivating those derivations are not trivial, and different aspects of the formulation were performed to either, result in a more tractable problem or to connect with the distributed processing paradigm employed in FL. 
 For instance, the fact that we are not computing the posterior as in Equation (1) but as in Equation (2), allows us to: $(i)$ develop the distributed Bayesian framework which is discussed in Equations (5)-(9); and $(ii)$ allows for a simpler recursive strategy that is presented in Section 3.2. 
 We highlight that all those steps were carefully thought through and conceptualized to cope with the Bayesian data association and the CFL paradigms, bringing challenges regarding its  motivation, design, and conceptualization. 
 For instance, in Equations (2)-(9) intricate application of Bayes Theorem, marginalization, product rules, and integral approximation were leveraged leading to a posterior formulation that connects with the FL paradigm.
 For these reasons, we believe that the derivations presented in Section 3.1, despite their algebraic simplicity, are not trivial and show the chain of thought applied to the problem.  
Of course, a Bayesian framework will rely on the usage of the Bayesian Theorem in its derivations.

Reviewer comment (Presentation):
The content in Sections 3 and 4.1 communicates a message that, in my interpretation, appears rather straightforward. The outlined two-step iteration involves computing the "optimal" association $\theta^{*}_{t}$ given cluster weights $\Omega$ and updating cluster weights based on the current association rule. Furthermore, the solution presented in equation (16) suggests associating a client $j$ with a cluster $i$ that maximizes $\log(D^j|w^i)$. While these concepts are essential, the simplicity of the presented message does not seem commensurate with the extensive coverage given to these sections (four pages). The detailed explanation does not justify the space allocated, and as such, Sections 3 and 4.1 may be perceived as overextended for the relatively straightforward content they provide.

Author response:
The reviewer is mistaken. Section 3 and 4.1 present different pieces of the proposed BCFL: while section 3 presents the conceptual solution where all association hypotheses are kept, section 4.1 presents the basics to weight different hypotheses and design approximate solutions to the computational challenge. The solution of equation (16) leads to the set of data associations that maximizes the sum, over all $K$ clusters and all clients in each cluster, of the $\log$ unnormalized marginal posterior of data association weights $\tilde{\pi}^{\theta^{ij}_t|h}$, where $h\triangleq \theta\_{1:t-1}$ indicates the history of past associations as discussed below equation (11). 
In the case where Equation (9) is approximated using the expected value of the model parameters then $\tilde{\pi}^{\theta^{ij}\_t|h} = p(\mathcal{D}^j|\omega^i\_{t-1})$. Nevertheless, only in the greedy approach discussed in Section 4.2 the selection is made solely on the $\log p(D^j|\omega^i\_{t-1})$.
We hope that the reviewer can revisit this point and change the overall evaluation of the manuscript.

Reviewer comment (Novelty):
In light of the preceding remark, it might be beneficial for the paper to discuss soft clustering approaches based on the EM algorithm.

Author response:
We will add these references to the related work section discussing it connects to BCFL. Note that a FedSoft is already cited in our work.  In our paper, we focused on the clustered FL problem as stated in (Ghosh et al., 2020; Mansour et al., 2020), where the objective is to learn multiple models under the assumption that data from each client comes from a specific distribution.  
Our approach focuses in clustering the different clients using a data association Bayesian framework that provides a joint posterior distribution of client associations and model parameters. Opposed to existing methods, we consider multiple data association hypotheses that enable the uncertainty quantification of client-to-cluster associations, leading to noticeable performance improvement.
In EM-based FL approaches the assumption is that the data at each client belongs to a mixture of distributions. Then, FL approaches are used to independently train multiple models under this assumption, which does not explore different association hypotheses. Furthermore, FedEM still relies on frequentist estimation strategies which contrasts with our Bayesian framework.

Reviewer comment (Theory):
The proposed methods lack theoretical guarantees, relying solely on heuristics without a solid theoretical foundation.

Author response:
We disagree with this comment. Our proposed framework is optimal in the Bayesian sense, where the objective is the computation of the joint posterior distribution of the model parameters and the clustering hypotheses. The solutions in Section 4 are suboptimal, although they ultimately aim at approximating the joint posterior while maintaining tractability. The theoretical guarantees come from our choice of the Bayesian paradigm, which states that all information is contained in the full posterior distribution that we are estimating. Consequently, the claim that our results rely solely on heuristics without a solid foundation is false.

Reviewer comment (Evaluation):
The success of the proposed method heavily depends on the higher computational cost of BCFL-MH, raising concerns about the practicality and efficiency of the cheaper variants (BCFL-G and BCFL-C). In fact, BCFL-G and BCFL-C do not show a significant improvement over FedAvg and WeCFL; the improvement never exceeds $1$ p.p., and is often lower than $0.1$ p.p.

Author response:
The performance can vary depending on the dataset and algorithms used, but according to the results in the main body and those in the appendix all proposed methods show improvement.
The cheaper variants pointed out are comparable to WeCFL, which was expected due to the assumptions made by the methods and the way the clients were clustered in the experiment. However, they outperform WeCFL in more challenging clustered scenarios, such as that presented in Table 7 in the appendix for Fashion-MNIST and Amazon-Review datasets. Noticeably, all variants of the method outperform FedAvg results in general, sometimes with substantial improvements (Fashion-MNIST and Cifar10) and sometimes with marginal improvements (Digit-5 and Amazon-Review).

Reviewer comment (Writing):
Several minor issues, including inconsistent citation style, grammatical errors, and unclear notation choices, need attention for a more polished presentation:  
- The paper seems to use the wrong citation style. It refers to the authors when it should refer to the paper. For example, the sentence "FL allows collaborative model training without data sharing across clients, thus preserving their privacy McMahan et al. (2017)" should be "FL allows collaborative model training without data sharing across clients, thus preserving their privacy (McMahan et al., 2017)."  
- In Section 1, "the is a lack" -> "there is a lack".  
- What is the reason behind using $\mathbb{P}$ instead of $P$ in page 3?  
- In Page 4, "to denote the a particular" -> "to denote a particular".  
- In Page 7, "both feature- and label- skew" -> "both feature---and label---skew".

Author response:
We thank the reviewer for pointing those typos out. We fixed them in the revised manuscript.

Reviewer comment (Experiments):
A major concern I have is regarding the efficacy and practicality of the proposed framework. Although the authors have proposed the three variants of the proposed BCFL framework, they still require the downloading of the models and weights under past associations, and then again uploading the associations of the weights to get the association decisions again from the server. Then finally the clients upload the local models based on the conditional associations. This requires at least 2 times the communication rounds compared to the conventional FL framework as well as more computation imposed to the clients. I became more skeptical after looking at the experimental results which only include 10 clients in total or 8 clients in total for more cross-device like datasets such as digits-dive or amazon review.

Author response:
There are three comments within the first item. Namely: 
1) It is worth clarifying that downloading the model for all past hypotheses is only the case for the conceptual solution discussed in Section 3 of the paper. However, the approximate solutions proposed in Section 4 are aimed to reduce those communication needs and thus to improve the practicality of BCFL. Other things to consider are that i) the weight associations variable sent along the model weights are just a few scalar values, thus requiring minimal extra communication cost; and ii) this paper tackles the additional communication cost by way of the proposed approaches that reduce the need to send an otherwise growing number of hypotheses. 
2) We also conducted tests involving a larger number of clients and have presented additional examples with 40 clients in the appendix, we will emphasize this in the updated paper. In the main body, we included only a limited number of clients for the Digit5 and Amazon Review datasets to ensure the clarity of our cluster analysis results, as done in other related works. As depicted in Figure 4, the behavior of the clusters during training is distinctly observable.

Reviewer comment (Experiments):
Another concern regarding this approach of BCFL is the sensitivity of the number of clusters $K$ to the performance. It will be difficult to know this value in advance in practice. How does the authors address this problem as well?

Author response:
There are three comments within the first item. Namely: 
3) Regarding the knowledge of the number of clusters $K$, we preliminary followed common practice in CFL works where this is often predetermined. We agree that estimating $K$ is of interest and it is actually on our future research plan, which our Bayesian framework can account for although not straightforwardly enough to be included in this paper so we instead decided to keep it fixed (although not necessarily correctly specified). To address this comment we updated the appendix with an experiment where the sensitivity to under- and over-estimating $K$ is discussed.

Reviewer comment (Theory):
Another concern I had regarding the practical variant approximate BCFL, the authors assume that there is no overlap in the distribution across different client partitions. This is quite a strong assumption which does not hold in most of the cases in FL. Can the authors comment on this assumption and how realistic it is?

Author response:
We would like to clarify that the main assumptions in the paper are that given an hypothesis the model parameters per cluster are independent and that the local datasets are conditionally independent. The intuition behind the validity of those assumptions comes from the conditioning on the clustering hypothesis, whereby if we were to explore all of the hypotheses we would find that different clusters obey different models. We agree that this might not be always true, that is the reason behind explicitly stating it as an assumption. Doing so, we enable tractability of the problem, which otherwise would be extremely challenging. Finally, it is worth noting that other non-Bayesian CFL works implicitly make this assumption, although since those works are often not probabilistic this might not be as apparent as in our framework.

Reviewer comment (Writing):
The writing of the paper can be improved. For instance, in pg1 when the authors address problem 1 and problem 2 in bold, there seems to be a typo/error. Ex: Problems2: The is a lack of a united theory. Moreover, optimality is used throughout the paper from the beginning without a proper explanation on what the authors exactly mean by this. It can mean differently for different readers. In addition, the presentation of Figure 3 can be improved; it is quite hard to see the differences across the curves.

Author response:
Thanks for pointing out the typos and figure suggestions, they are fixed. As for the use of `optimality', we refer to the fact that the conceptual solution in Section 3 provides the full posterior distribution of the model, where all association hypotheses are explored. That is, BCFL is optimal in the Bayesian sense, where the objective is the computation of the joint posterior distribution of the model parameters and the clustering hypotheses. Optimality of the algorithms is thus referring to a Bayesian inference notion of optimality, which is now mentioned in the first paragraph of Section 3. The approximate solutions developed in Section 4 are therefore suboptimal, although they ultimately aim at approximating that same joint posterior while maintaining tractability and practicality of BCFL.

Reviewer comment (Presentation):
There is a clarity problem before section 4. Specifically, the full procedure of the learning is poorly illustrated. Though Figure 1 is supposed to illustrate the essential steps and communications, the confusing coloring and the lack of clarity on the calculation of key values pose unnecessary challenges for the audience. The pseudo-code for the general learning, clustering procedure, and each variant of association selection, along with a detailed congregate discussion of the communication, would greatly improve the clarity of the paper. Also, it might help to focus on the conceptual discussion in Figure 1 rather than placing notations, which are introduced later in sections 3 and 4.

Author response:
We improved the notation readability for those sections, as well as added pseudo-code algorithmic description for both the conceptual solution (section 3) and the practical schemes (section 4). These are in Section B of the Appendix.
We acknowledge the possibility of enhancing these sections for clarity. However, given the constraints of time, it is challenging to undertake substantial revisions in less than a day. Nevertheless, we are committed to addressing the reviewer's concerns to the best of our ability within the time available and in future iterations.

Reviewer comment (Novelty):
Section 2 mainly focuses on discussing federated learning works. Yet, the paper needs to pay attention to the existing product partition models and needs more discussion over the progression of existing Bayesian frameworks, which could better position the paper's theoretical contribution.

Author response:
Thanks for the pointers regarding how to improve the related work section. We improved it by adding more references and better motivating the contribution of this work within those works. Particularly, we discussed Bayesian frameworks within FL, as the reviewer suggested, which has indeed improved the positioning of this paper.

Reviewer comment (Theory):
The paper proposes to leverage the cost function to transform the association selection into the M-best assignments problem. Yet, as admitted in the discussion of section 4, the greedy or top $M$ trajectory could be myopic. This problem seems intrinsic in the definition of $L$ and is reflected in the empirical results where the BCFL-MF-6 does not constantly improve upon BCFL-MF-3.

Author response:
Thanks, as the reviewer points out, the 3 methods proposed in the work are approximated because we cannot account for all the associations (discussed in section 3.3). More precisely: BCFL-G only keeps the best association every time; BCFL-C chooses the $M$ best associations at a given time instant and merges them into a single posterior solution, thus only propagating one hypothesis to the next communication round; and BCFL-MH keeps the $M$ best associations onto the next recursively update. There might be other approaches to approximating the full posterior, which itself can lead to different BCFL algorithms. Indeed, the results for BCFL-MH-6 do not always improve those from BCFL-MH-3. There could be multiple reasons to that such as related to the dataset and its setting, the chosen associations, and the randomness of each trial. Ultimately, when taking a close look at the result, MH-3 or MH-6 are noticeably close to each other, which indicates that for those experiments going from $M=3$ to $M=6$ does not provide substantial performance improvement.

Reviewer comment (Experiments):
The datasets tested in section 5 seem to be manually synthesized, raising questions about the motivation of CFL and undermining the significance of the experimental results.

Author response:
All these datasets are real datasets. The usage of those datasets in the context of non-IID challenges is according to similar CFL works, which we compare to. We do not understand which questions regarding the motivation of CFL the reviewer is referring to. More details about the datasets are provided in the Appendix.

Reviewer comment (Evaluation):
The results only show the downstream performance while lacking evidence about the efficiency, especially for different approximations and corresponding key hyperparameters, including the $M$ discussed in sections 4.3 and 4.4 and $K$ that could potentially impact the communication efficiency and learning convergency.

Author response:
Per the reviewer's suggestion, we included a discussion on the communication efficiency for each practical algorithm in Appendix B.

Reviewer comment (Novelty):
The paper claims the existing CFL methods do not effectively exploit similarities between different clusters, yet needs more detailed discussion over the statement.

Author response:
We provide a bit more discussion of the claim in the `Problem 1' description in the introduction section. The statement regarding other CFL methods is that the knowledge of each participating client is exploited by only one cluster during each round, which potentially results in an inefficient utilization of the local information that could contribute to the training of multiple clusters instead;

Reviewer comment (Writing):
The adopted merge operator in section 4.3 needs to be clarified.

Author response:
We improved it and more details can be found in a newly added Appendix C, as well as the references therein;

Reviewer comment (Experiments):
The model warm-up relies on a different partitioning method based on the local model initialization. And it is unclear why it contributes to the performance.

Author response:
We implemented the warm-up method to see if it could help achieve faster convergence. And indeed, it did help. With the same number of communication rounds, it usually achieves better results. The reason behind this is that we provide a good initial estimate at the beginning, as shown in Figure 3. Warm-up methods typically have good initial accuracy and are used in other works.

Reviewer comment (Reproducibility):
The paper has an intense layout but lacks details of the algorithm design, e.g., a concrete pipeline of the algorithm and congregated discussion of the details of the communication between clients and servers corresponding to the proposed three variants.

Author response:
We improved the notation readability for those sections, as well as added pseudo-code algorithmic description for both the conceptual solution (section 3) and the practical schemes (section 4).
These are in Section B of the Appendix. These pipeline descriptions should help improve the presentation of the work, as well as the information flow in the different methods (including the conceptual solution.

Reviewer comment (Evaluation):
Based on the discussion and the experimental results, it seems that BCFL-MH is the best option among all the three approximations. If so, why not propose BCFL-MH as the main algorithm, allowing a better illustration of the algorithm's key concepts? Is there any additional trade-off on the efficiency and performance when choosing the approximation?

Author response:
Regarding the option of simply presenting BCFL-MH, our main objective is to present the BCFL framework from which one can derive the three proposed solution and many others that can be explored in future research from us or other groups. 
The objective of this work is to present a new framework for the problem of clustered FL, rather than just presenting a detailed method. The three approaches are practical implementations, derived from the framework. There could be many other variations by pruning, merging, and combining them, which would expand the possibilities to make progress in the CFL field. The BCFL-MH approach yields better results because it considers more associations compared to the other approaches. However, it is not as efficient as others, as discussed in the communication section of the appendix. That is the reason why we presented the three alternatives, balancing communication needs and performance. Each application might have different requirements.

Reviewer comment (Writing):
Overall, there is a significant concern over the clarity of writing throughout Sections 3 and 4. For example, even though the authors tabulate their notations in Appendix A, it was pretty hard to follow the arguments in Section 3 because of notation overload. Possibly, some of them can be removed, or some of the derivations can be written down by dropping unnecessary indexing wherever possible.

Author response:
We improved the notation readability for those sections, as well as added pseudo-code algorithmic description for both the conceptual solution (section 3) and the practical schemes (section 4).
We acknowledge the possibility of enhancing these sections for clarity. However, given the constraints of time, it is challenging to undertake substantial revisions in less than a day. Nevertheless, we are committed to addressing the reviewer's concerns to the best of our ability within the time available and in future iterations.

Reviewer comment (Experiments):
Since the different possible associations of clients can be combinatorially very large to evaluate, the authors propose approximations in Section 4. But, it is unclear if BCFL-G/C is helpful in practice, compared to WeCFL. In fact, BCFL-G which does greedy association at each round is quite similar to WeCFL.

Author response:
Firstly, the methods we propose are not only for experimental comparison only, but our primary objective is to introduce a framework that can be adapted and extended to accommodate a variety of methods for future research. Secondly, WeCFL represents a non-trivial algorithm, and we do not claim that our methods will universally outperform it. However, as indicated by the results, BCFL-G/C achieve similar or improved performance compared to WeCFL across a diverse range of datasets and experimental configurations, with some additional results presented in the Appendix.

Reviewer comment (Evaluation):
From point 2, the main contribution I see is BCFL-MH, but this method is computationally heavier given that it requires computing densities in Sec 3.1 and 3.2, and I could not see a computational cost / performance benefit tradeoff to justify BCFL-MH.

Author response:
The main contribution is the Bayesian framework for CFL, the specific algorithms like BCFL-MH are options within it.  While BCFL-MH seems to be a good solution performance-wise, the other BCFL-G/Ca approaches provide remarkable performance at lower communication/complexity burdens. We added a discussion on this in the revision, as well as a discussion of the communication and computation cost in Appendix, Section B.

Reviewer comment (Theory):
The authors claim "new theoretical insights" in their introduction, but the paper does not have any formal guarantees for the proposed approximations.

Author response:
The theoretical insights come from the formulation of the clustering FL problem as a probabilistic model which we address from a Bayesian perspective, dealing with the multiple data associations. This allowed us to propose different practical methods, solidly based on the Bayesian paradigm. We acknowledge the reviewer's concern regarding the absence of a formal guarantee in this part, or at least guarantees in the sense of those provided in frequentist-based methods. As a matter of fact, our proposed approximation BCFL-G has strong connections to the IFCA [ghosh2020efficient], when formulated in a Bayesian context, which provides those theoretical guarantees. Our paper concentrates on the Bayesian framework rather than on particular deterministic algorithms, which is why we did not initially include a convergence proof. However, we recognize the importance of such an analysis, and we are certainly open to investigating convergence guarantees in future research.

Reviewer comment (Experiments):
The authors do not provide any ablations on the choice of number of clusters K, which can be quite critical for some of the proposed approximations like BCFL-MH.

Author response:
We added sensitivity analysis. Please refer to our appendix for results on the selection of $K$ and its justification.

Reviewer comment (Theory):
The derivations in equations 2-7 seem to be simple application of Bayes rule under conditional independence of clients data given model parameters. On the other hand, the approximation in equation 9, which seems to be quite critical is unclear. The authors mention some numerical integration approximation, but it would be useful if they could clarify this further.

Author response:
This is not a mere application of Bayes rule. Of course, as any Bayesian solution, that result is used, but in the derivations pointed out there is much more than this, as testified by the formal notation we provided.
We call the reviewer's attention to the fact that the conceptual ideas motivating those derivations are not trivial, and different aspects of the formulation were performed to either, result in a more tractable problem or to connect with the distributed processing paradigm employed in FL. 
 For instance, the fact that we are not computing the posterior as in Equation (1) but as in Equation (2), allows us to: $(i)$ develop the distributed Bayesian framework which is discussed in Equations (5)-(9); and $(ii)$ allows for a simpler recursive strategy that is presented in Section 3.2. 
 We highlight that all those steps were carefully thought through and conceptualized to cope with the Bayesian data association and the CFL paradigms, bringing challenges regarding its motivation, design, and conceptualization. 
 For instance, in Equations (2)-(9) intricate application of Bayes Theorem, marginalization, product rules, and integral approximation were leveraged leading to a posterior formulation that connects with the FL paradigm.
 For these reasons, we believe that the derivations presented in Section 3.1, despite their algebraic simplicity, are not trivial and show the chain of thought applied to the problem.  
Of course, a Bayesian framework will rely on the usage of the Bayesian Theorem in its derivations. 
 Regarding equation (9), it aims to explain a numerical integration approximation, allowing for effective factorization across local clients. Typically, numerical integration involves sampling; in our implementation approach, we can simplify the process by taking a single sample—specifically, the expected value of the parameter. While Bayesian theory itself is straightforward, the critical aspect lies in its connection to data association theory and its subsequent application to Cluster Federated Learning.

Reviewer comment (Evaluation):
In Table 1, what are the standard deviations of the reported metrics? Since the difference between WeCFL and BCFL-G/C is quite small, this would help clarify statistical significance of the reported results.

Author response:
We have now included the standard deviation in Appendix, Table 10. Initially, due to the limited space and the small magnitude of the standard deviation, we have omitted these results.

Reviewer comment (Reproducibility):
The motivation for warmup is unclear. Also, how are the local weights clustered? Is it in the euclidean metric?

Author response:
Yes, the metric employed in our experiment is the Euclidean metric. We have updated this in the `Baseline Models and System Settings' section of the Appendix. The motivation is just to check if warm-up can speed up the training and improve the results, which indeed did. With the same number of communication rounds, it usually achieves better results. The reason behind this is that we provide a good initial estimate at the beginning, as shown in Figure 3. Warm-up methods typically have good initial accuracy and are used in other works.

Reviewer comment (Experiments):
Have the authors tried using the learnt associations for fine-tuning, as opposed to global training? Maybe there are some general image features that can be learnt from all users?

Author response:
That's good idea that we would like to explore in future works. There is little time to incorporate this in the present review.

Reviewer comment (Experiments):
Have the authors run IFCA on their datasets for comparison?

Author response:
Indeed, we implemented the IFCA algorithm and tested in the experiments. Actually, IFCA can be interpreted as the deterministic variant of our greedy BCFL-G method. The results between both were very similar. However, as our study does not focus on contrasting deterministic approaches with Bayesian methods, the comparisons have been excluded from this analysis.