Title: Personalized Federated Learning with Mixture of Models for Adaptive Prediction and Model Fine-Tuning
Venue: NeurIPS 2024
Accepted: yes
Keywords: Federated Learning

Personalized Models

Real-time Predictions
Abstract: Federated learning is renowned for its efficacy in distributed model training, ensuring that users, called clients, retain data privacy by not disclosing their data to the central server that orchestrates collaborations. Most previous work on federated learning assumes that clients possess static batches of training data. However, clients may also need to make real-time predictions on streaming data in non-stationary environments. In such dynamic environments, employing pre-trained models may be inefficient, as they struggle to adapt to the constantly evolving data streams. To address this challenge, clients can fine-tune models online, leveraging their observed data to enhance performance. Despite the potential benefits of client participation in federated online model fine-tuning, existing analyses have not conclusively demonstrated its superiority over local model fine-tuning. To bridge this gap, the present paper develops a novel personalized federated learning algorithm, wherein each client constructs a personalized model by combining a locally fine-tuned model with multiple federated models learned by the server over time. Theoretical analysis and experiments on real datasets corroborate the effectiveness of this approach for real-time predictions and federated model fine-tuning.

Personalized Federated Learning with Mixture of Models for Adaptive Prediction and Model Fine-Tuning

Abstract

Federated learning is renowned for its efficacy in distributed model training, ensuring that users, called clients, retain data privacy by not disclosing their data to the central server that orchestrates collaborations. Most previous work on federated learning assumes that clients possess static batches of training data. However, clients may also need to make real-time predictions on streaming data in nonstationary environments. In such dynamic environments, employing pre-trained models may be inefficient, as they struggle to adapt to the constantly evolving data streams. To address this challenge, clients can fine-tune models online, leveraging their observed data to enhance performance. Despite the potential benefits of client participation in federated online model fine-tuning, existing analyses have not conclusively demonstrated its superiority over local model fine-tuning. To bridge this gap, the present paper develops a novel personalized federated learning algorithm, wherein each client constructs a personalized model by combining a locally fine-tuned model with multiple federated models learned by the server over time. Theoretical analysis and experiments on real datasets corroborate the effectiveness of this approach for real-time predictions and federated model fine-tuning.

1 Introduction

Federated learning enables a group of learners, known as clients , to collaborate and collectively train a model under the coordination of a central server , without revealing their data. In this framework, clients perform local model updates and share these updates with the server. By aggregating these local updates, the server globally updates the model. Many prior works in the literature have assumed that each client stores a batch of training data and updates models locally based on this stored data (see e.g., [38, 29, 47, 5, 34, 53]). However, in some cases, clients may need to make real-time predictions, and streams of data arrive sequentially, making it challenging to store and process data in batch. Furthermore, if clients operate in a non-stationary and dynamic environment, employing pre-trained models may fall short in prediction accuracy, requiring clients to fine-tune their models to adapt to their data.

A federated learning framework, commonly referred to as online federated learning [37, 26, 19], specifically addresses situations where clients engage in real-time predictions using a shared global model. After making predictions, clients collaborate with the server to update the global model for subsequent predictions in the future. Specifically, after making predictions, clients incur losses and based on the incurred losses, clients update the model locally and send their local updates to the server. The server updates the global model upon aggregating local updates and then distributes the updated global model to clients for use in the ongoing online prediction task. In this context, the

performance of clients can be assessed using the notion of regret [4, 3]. The regret of a client at a given time step is defined as the difference between its prediction loss and that of the best model in hindsight. The best model in hindsight is determined as the model that achieves the minimum total prediction loss over time across all clients' data. The primary objective is to minimize the cumulative regret of all clients over time.

In the literature, federated learning algorithms based on online gradient descent have been proposed that achieve sub-linear regret upper bounds [37, 26, 19]. This suggests that over the long run, these algorithms perform as well as the best model in hindsight. However, in Section 3, this paper demonstrates that these federated learning algorithms do not obtain tighter regret bounds compared to the scenario where each client learns its own model locally without participating in federated learning. This indicates that participation in federated learning may not provide any benefit for clients performing online prediction. Specifically, if data is distributed non-i.i.d. among clients and data distributions are time-variant and not known a priori, it is likely that local model training by clients will achieve better prediction accuracy than participation in federated learning. Although the benefit of federated learning in online prediction is not evident in existing theoretical analyses, models learned through federated learning enjoy higher generalizability as they are trained on all data samples distributed among clients. This motivates the idea that combining the model learned through federated learning with the locally learned one may prove effective in scenarios where clients need to perform online prediction.

This paper proposes the Fed-POE (Federated Learning with Personalized Online Ensemble) algorithm, through which each client constructs a personalized model for online prediction by adaptively ensembling the locally learned model and the model learned through federated learning. With FedPOE, clients can adapt their local models to their data while benefiting from the higher generalizability of the federated model. Theoretical analysis for convex cases demonstrates that Fed-POE achieves sublinear regret bounds with respect to both the best federated and local models in hindsight. This indicates that Fed-POE effectively leverages the advantages of both federated and local models. Providing such theoretical guarantees may not be feasible for non-convex models such as neural networks. These models may suffer from the forgetting process [51, 45], where fine-tuning on streaming data 'on the fly' can lead to forgetting previously observed data samples. To overcome this challenge, the present paper proposes a novel framework in which the server periodically stores federated model parameters over time. Each client adaptively selects a personalized subset of stored models on the server based on the performance of these models in the client's online prediction task. Clients then use the selected models, along with the federated and local models, to construct an ensemble model for prediction. Clients select a subset of models to both control the memory and computational complexity of prediction and to prune models with relatively lower accuracy, thereby improving prediction performance. Theoretical analysis proves that Fed-POE achieves sublinear regret with respect to the best model in hindsight among the local model, federated model, and all models stored by the server. The contributions of the present paper are summarized as follows:

Fed-POE enables clients to utilize the advantages of both local and federated models for online prediction tasks.

To address the issue of forgetting in online prediction, Fed-POE introduces a novel federated framework for collaboration between clients and the server.

Theoretical analyses for both convex and non-convex cases prove that Fed-POE achieves sublinear regret with respect to the best model in hindsight.

Extensive experiments on regression and image classification datasets show that Fed-POE effectively leverages the benefits of local and federated models, achieving higher online prediction accuracy compared to state-of-the-art federated learning algorithms.

2 Related Works

Personalized Federated Learning. Personalized federated learning involves developing individualized models for each client, derived from a global model learned through federated learning. Several personalized federated learning algorithms have been proposed in the literature [50, 6, 54, 56]. In [24, 11, 30, 33], clients construct their personalized models by adding a regularization term to the local objective and using the global federated model. Algorithms in [12, 1] allow clients to fine-tune the global federated model using model-agnostic meta-learning [14] to learn their personalized

models. With Fed-Rep [8], each client generates a representation using the global model and learns its own local head for prediction. Algorithms in [10, 32, 57] enable clients to achieve personalized models by combining local and global models. However, none of these works have addressed the problem of online prediction while clients collaborate on training their personalized models.

Federated Learning with Streaming Data. Several studies have explored the problem of federated learning when clients receive a stream of data in real time. While [7, 36, 35] investigate federated learning scenarios where clients receive new training data in each learning round, they do not address the aspect of online prediction by clients. Consequently, these works cannot provide regret guarantees for online prediction. Additionally, [9] studies the issue of staleness in online federated learning to both train and make prediction with the model; however, it lacks rigorous theoretical analysis. In [37], an online federated learning algorithm is introduced, utilizing online mirror descent, with a proven sublinear regret bound. Similarly, [42, 16] propose online federated learning algorithms with guaranteed sublinear regret. Furthermore, [43] analyzes the benefits of collaboration in online federated learning, particularly in scenarios where only loss values at queried points are available to clients, without access to loss gradients. Regarding online federated kernel learning, algorithms are introduced in [17, 23], albeit without accompanying regret analysis, leading to an absence of guaranteed regret bounds. In [26, 19], multiple kernel-based models and random feature-based online federated kernel learning algorithms are proposed, with demonstrated sublinear regret.

3 Preliminaries

This section explains the problem of federated learning for real-time prediction and model training. The present section studies the cases where clients either collaborate in federated learning or employ online gradient descent methods to locally train the model in real-time. This study highlights the motivation behind the proposed ensemble approach to federated learning.

3.1 Online Prediction and Federated Learning

Let there are N clients interact with a server to train a model f ( · ; · ) . Also, let [ N ] := { 1 , . . . , N } . At each time step t , client i , ∀ i ∈ [ N ] receives a data sample x i,t ∈ R d and makes the prediction f ( x i,t ; θ t ) where θ t denotes the parameter of the model at time step t . Note that generalization to the scenario where at each time step, each client receives a dataset instead of a single data sample is straightforward. After making prediction, client i , ∀ i ∈ [ N ] observes label y i,t . Then client i , ∀ i ∈ [ N ] computes the loss of its prediction L ( f ( x i,t ; θ t ) , y i,t ) where L ( · , · ) denotes the loss function. After computing the loss, clients send their updates to the server (e.g., by sending the gradient loss ∇L ( f ( x i,t ; θ t ) , y i,t ) ) and the server aggregates information from clients to update θ t to θ t +1 to be used by clients to make predictions at time step t +1 . For ease of presentation, it is assumed that all clients can send their updates to the server every time step. Generalizing the results for the cases where only a fraction of clients can send their updates is straightforward. Furthermore, it is assumed that the label y i,t for any i and t is determined by the environment through a process unknown to the clients. This implies that the data distribution observed by client i can be nonstationary, and client i , ∀ i ∈ [ N ] does not know the distribution. Additionally, the data distribution can differ across clients. The goal is to enable clients to collaborate with the server to minimize the cumulative regret of clients over time. The regret of client i at time step t is defined as the difference between the prediction loss L ( f ( x i,t ; θ t ) , y i,t ) and the loss L ( f ( x i,t ; θ ∗ ) , y i,t ) corresponding to the model with the optimal parameter θ ∗ . Therefore, the average cumulative regret of clients up to time horizon T is defined as

where θ ∗ denotes the optimal model parameter in hindsight and can be expressed as

One common way to solve the problem is that clients employ online gradient descent to update models locally and the server exploits federated averaging [31, 37] to update the global model.

3.2 Federated Learning with Online Gradient Descent

At each time step t , client i obtains the locally updated model parameter ψ i,t +1 as follows:

where η is the learning rate. Aggregating locally updated model parameters, the server obtains the updated global model parameter for time step t +1 as

This continues up until the time horizon T . This paper examines regret under some or all of the following assumptions:

A1. The loss function L ( f ( x ; θ ) , y ) is convex with respect to θ .

A2. The gradient of the loss is bounded as ∥∇L ( f ( x ; θ ) , y ) ∥ ≤ G .

A3. For any x , θ , the loss is bounded as 0 ≤ L ( f ( x ; θ ) , y ) ≤ 1 .

The following theorem specifies the regret bound for federated learning employing online gradient descent under A1 and A2.

Theorem 1. Employing online gradient descent, the following cumulative regret upper bound is guaranteed for federated learning under assumptions A1 and A2:

Proof of Theorem 1 can be found in Appendix A. According to Theorem 1, choosing η = O ( √ 1 /T ) , the cumulative regret in (5) is bounded from above as O ( √ T ) . If the time horizon T is unknown, the doubling trick technique (see e.g., [2, 22]) can be effectively used to set the learning rate to maintain theoretical guarantees. Consider the case where each client learns the model locally using online gradient descent without collaborating with other clients and the server. Let ϕ i,t denote the local model parameter learned by client i at time step t , employing online gradient descent locally. The regret of client i in this case is equivalent to federated learning regret where there is only one client. Therefore, substituting N = 1 and θ 0 = 0 in (27) of Appendix A, for the cumulative regret of client i with respect to any θ , we get

Averaging (6) over all clients and substituting θ with θ ∗ in (6), we obtain

Comparing (5) with (7), it can be inferred that under assumptions A1 and A2, federated learning does not achieve tighter regret bound than the case where each client independently learns its local model. Hence, from a theoretical standpoint, it remains uncertain whether collaboration in federated learning yields any improvement over local online model training. Intuitively, collaboration in federated learning may prove advantageous when there exists similarity among data samples observed by clients over time. However, in cases where data distribution is heterogeneous and such similarities are lacking, employing online local training may yield superior results for a client. Given the lack of prior information on data distribution in online scenarios, each client can independently assess over time whether utilizing the model learned through federated learning for predictions is beneficial.

Furthermore, according to assumption A1, the theoretical guarantees obtained in (5) and (7) hold if the loss is convex with respect to the model parameter. However, in the case of non-convex models, such as neural networks, these theoretical guarantees are not applicable. Non-convex models, like neural networks, may encounter the forgetting process [51, 45], where the model tends to overfit to recently observed data samples. Consequently, it may not be feasible to derive a single model parameter using online gradient descent that achieves sublinear regret with respect to the best model in hindsight. The subsequent section introduces a novel algorithm aimed at assisting clients in addressing such scenarios.

4 Personalized Federated Learning Methods

The current section introduces personalized federated learning algorithms where each client dynamically learns the prediction performance of both the models trained through federated learning over time and the locally learned model.

4.1 Ensemble Learning

At each time step, each client constructs an ensemble model comprising the federated model and its locally learned model to make a prediction. Let ϕ i,t be the model parameter locally learned by client i such that at each time step t , client i updates ϕ i,t via gradient descent as

Furthermore, let clients and the server collaborate to update the federated model parameter θ t as outlined in (3) and (4). At time step t , client i makes the prediction for x i,t using its personalized ensemble model f i,t ( · ) expressed as

where α i,t and β i,t represent the weights assigned by client i to the federated and local models, respectively, indicating the credibility of predictions from each model. After making predictions and observing the label y i,t , client i computes the loss of predictions from the federated and local models, updating the weights α i,t and β i,t using the multiplicative update rule as follows:

where η c is a learning rate. Client i initializes α i, 1 = 1 and β i, 1 = 1 . The proposed algorithm is personalized since according to (9), each client constructs its own ensemble model to perform prediction. The personalized regret of client i is defined as C i,T = ∑ T t =1 L ( f i,t ( x i,t ) , y i,t ) -∑ T t =1 L ( f ( x i,t ; ϕ ∗ i ) , y i,t ) where ϕ ∗ i denotes the best hindsight model parameter for client i which can be expressed as ϕ ∗ i = arg min ϕ ∑ T t =1 L ( f ( x i,t ; ϕ i ) , y i,t ) . The following theorem establishes the personalized regret upper bound for client i as well as global regret of all clients with respect to the best model parameter in hindsight.

Theorem 2. Under assumptions A1-A3, employing the ensemble model in (9) for online prediction, the global regret of all clients is bounded from above as

while client i achieves the following personalized regret upper bound:

Proof of Theorem 2 is given in Appendix B. According to (11) and (12) in Theorem 2, setting η = O (1 / √ T ) , η c = O (1 / √ T ) both the global regret for all clients and the personalized regret of each client i achieve a regret bound of O ( √ T ) . This demonstrates that while the ensemble model in (9) ensures personalized regret guarantees for each client with respect to its best model in hindsight, it also enables clients to leverage federated learning, thus enjoying sublinear regret in comparison to the best global model in hindsight.

Comparison with Federated and Local Models. The main advantage of using the ensemble model instead of the federated model lies in Theorem 2, where it is shown that the ensemble model can attain the global regret guarantee provided by the federated model while employing the federated model, achieving the personalized regret guarantee in (12) is not feasible. However, according to (6) and (7), using the online local training, each client achieves sublinear regret with respect to its best model while all clients achieve sublinear regret with respect to the best global model in hindsight.

This efficacy of local training stems from clients adapting the model to their individual data. In contrast, in federated learning, the model is trained on all data samples across clients, potentially leading to higher generalizability compared to its local counterpart. If there are similarities in the distribution of data samples among clients over time, the federated model is anticipated to achieve greater accuracy in online prediction. However, due to the lack of available information regarding the relationships between data samples observed by clients before online prediction, these advantages may not be reflected in theoretical bounds. The proposed method constructs an ensemble to harness the advantages of both federated and local models for online prediction, as evidenced by Theorem 2. Experimental results in Subsection in 5.1 confirm that the ensemble model achieves superior performance compared to both local and federated models.

4.2 Model Selection

Regret guarantees for the ensemble method, as outlined in Theorem 2, are contingent upon the model's convexity. However, if the model is non-convex, achieving such guarantees may not be feasible. Particularly, non-convex models such as neural networks are susceptible to the forgetting process [51, 45], wherein applying online gradient descent may lead to overfitting to recently observed data samples. This section introduces a novel algorithm that allows clients to make online predictions using non-convex models while simultaneously collaborating to fine-tune the model. The scenario assumes the existence of a pre-trained model, and the objective for clients is two-fold: to make real-time predictions and to refine the model for alignment with their preferences. This situation may arise, for instance, in fine-tuning large foundation models to tailor them to client preferences.

Let the server and clients collaborate to fine-tune the non-convex model f ( · ; · ) . At each time step t , client i updates the model on the batch of recently observed samples with size b as

Then the server aggregates locally updated parameters and updates the federated model parameter as in (4). Furthermore, each clients learns its own local model by fine-tuning the pre-trained model locally via online gradient descent as in (8) on the batch of b recently observed samples. While online gradient descent methods are well-known for their efficiency in handling dynamic environments, employing the update rule of (13) for non-convex models may lead to overfitting to recently observed batches. To mitigate potential forgetting, the server saves the federated model parameters every n time step, where n is an integer hyperparameter.

Let D t represent the set of model parameters stored by the server at time step t . At time step τ = ( j -1) n +1 , the server adds θ τ to D τ meaning that D τ +1 = D τ ∪ { θ τ } . Let ρ j denotes the j -th model parameter in D t . It can be concluded that ρ j = θ ( j -1) n +1 . The server continues saving model parameters every n time steps until time step U ≤ T . Client i assigns a weight w ij,t to the j -th model in D t , which assesses the credibility of the predictions given by the model parameter ρ j . Client i initializes w ij, 1 = 1 . At each time step t , client i selects M models with replacement from D t to construct its model set M i,t . Algorithm 1 illustrates the model selection process conducted by client i at time step t . During each round of model selection, client i chooses a model according to a probability mass function (PMF) proportional to weights { w ij,t | 1 ≤ j ≤ | D t |} where | · | denotes the cardinality of a set (see step 3 in Algorithm 1). Client i adds the chosen model to M i,t if it is not already present. Therefore, it can be concluded that | M i,t | ≤ M , ∀ i, t . Then at each time step t ,

client i downloads all models in M i,t from the server. Upon receiving the models, client i constructs an ensemble model ˜ f i,t ( · ) as

Client i makes the prediction for x i,t as

where f i,t ( x i,t ) is the ensemble of local and federated models as defined in (9). Furthermore, γ i,t and δ i,t are weights assigned by client i to ensemble models f i,t ( · ) and ˜ f i,t ( · ) , respectively. Upon observing the label y i,t after prediction, client i updates weights γ i,t and δ i,t as

Furthermore, α i,t and β i,t which are used to construct f i,t ( x i,t ) in (9) are updated as in (10). The weight w ij,t is updated using the importance sampling loss as

where 1 j ∈ M i,t denotes the indicator function and is 1 if j ∈ M i,t . Moreover, q ij,t is the probability that client i selects the j -th model in D t to be in M i,t and can be expressed as q ij,t = 1 -(1 -p ij,t ) M where p ij,t is defined in step 3 of Algorithm 1. The proposed algorithm, named Federated Learning with Personalized Online Ensemble (Fed-POE) is summarized in Algorithm 2. It is useful to note that the proposed method in Subsection 4.1 is a special case of Fed-POE by setting M = 0 and b = 1 .

Efficiency of Fed-POE. To perform model selection using Fed-POE, clients do not need to store all model parameters in D t for all t ∈ [ T ] . Instead, the server, which has higher storage capacity than the clients, stores all model parameters, and clients download a subset of at most M model parameters. The hyperparameter M can be chosen such that clients can handle the memory and computational requirements of making predictions with the selected subset of models. Let C F denote the number of computations required to fine-tune the model f , and let C I represent the number of computations required to make an inference with model f . Assume that the complexity of model selection in Algorithm 1 is negligible compared to fine-tuning and making inferences with model f . According to Algorithm 2, each client performs 2 C F + ( M + 2) C I computations per time step. Therefore, the computational complexity of Fed-POE for each client is O ( C F + MC I ) . Beyond memory and computational considerations, selecting a subset of models from D t helps clients improve their

prediction accuracy. Specifically, using the model weights { w ij,t } at time step t , client i selects models that perform better on its data while pruning those with lower performance.

Let h j ( x i,t ) = f ( x i,t ; ρ j ) denote the model associated with the j -th model parameter in D t . Furthermore, h loc ( x i,t ) = f ( x i,t ; ϕ i,t ) and h fed ( x i,t ) = f ( x i,t ; θ t ) represent the local and federated models, respectively, . Let the set of models H be defined as H := { h j | ∀ j : 1 ≤ j ≤ | D T |} ∪ { h loc , h fed } . This set H includes all models that can be employed by each client using Fed-POE. The best model in hindsight h ∗ and the best model in hindsight for client i , h ∗ i are defined as

The following theorem provides the regret upper bound of Fed-POE.

Theorem 3. Under assumption A3, employing Fed-POE in Algorithm 2, the expected global regret of all clients is bounded from above as

while client i achieves the following expected personalized regret upper bound:

The expectation is taken with respect to randomization in model selection.

The proof of Theorem 3 can be found in Appendix C. According to (19) and (20), setting η c = O (1 / √ T ) , n = O ( √ T ) and U = O ( √ T ) , both the personalized and global regrets of clients achieve sublinear regret of O ( √ T ) . Since clients construct their ensemble models using a timevarying subset of models, employing existing model selection and ensemble learning approaches [13, 15, 39, 46, 20, 41, 21] may not ensure the sublinear regrets stated in Theorem 3. However, by using the proposed Algorithm 1, Fed-POE guarantees sublinear regret bounds while allowing clients the flexibility to select time-varying and personalized subsets of models for their ensembles.

5 Experiments

The present section studies the performance of Fed-POE in Algorithm 2 compared to other baselines. Experiments are conducted on both image classification and regression tasks. The performance of federated learning is examined in both convex and non-convex cases. Codes are available at https://github.com/pouyamghari/Fed-POE .

5.1 Regression

The performance of the proposed Fed-POE is evaluated on online regression tasks. For these tasks, clients and the server collaborate to train a random feature kernel-based model, which is known to be convex [25, 48, 18]. Details about the random feature kernel-based model used in the experiments can be found in Appendix D. The performance of Fed-POE is compared with a baseline called Local, where clients train their models locally without participating in federated learning. Additionally, Fed-POE is compared to personalized federated learning baselines Ditto [30] and Fed-Rep [8], the online federated learning baseline Fed-OMD [37], and online federated

kernel learning baselines eM-KOFL [26] and POF-MKL [19]. Mean square error (MSE) is used as the metric to evaluate the performance of algorithms on regression task. MSE for client i can be expressed as MSE i = 1 T ∑ T t =1 (ˆ y ij,t -y i,t ) 2 where ˆ y ij,t denotes the prediction of client i at time step t . The performance of algorithms are tested on two regression datasets Air [58] and WEC [40]. Air dataset is a time-series dataset that each data sample contains air quality features and the goal is to predict the concentration of contamination in the air. Each sample in WEC dataset contains features of different wave energy converters and the goal is to predict power output. Data samples are distributed non-i.i.d among 400 clients. Time horizon T for both datasets is 250 . More information about datasets and distributed data among clients is presented in Appendix D.

Table 1 presents the MSE of algorithms and their standard deviation across clients. For all algorithms, the learning rates are set to η = η c = 1 / √ T . Table 1 shows that when data is distributed non-i.i.d. among clients, local model training can achieve higher accuracy compared to federated learning. For the Air dataset, Local achieves lower MSE than other federated learning baselines except for Fed-POE. For the WEC dataset, only POF-MKL achieves lower MSE than Local. This indicates that the performance of federated learning approaches compared to Local depends on the dataset. By utilizing both

federated and local models, Fed-POE achieves the lowest MSE. Table 1 shows that the performance of Fed-POE relative to other baselines is more consistent across different datasets.

5.2 Image Classification

The performance of the proposed Fed-POE on an image classification task is compared with Local, Ditto [30], Fed-Rep [8], Fed-OMD [37], Fed-ALA [57], and Fed-DS [35]. Fed-ALA [57] is a personalized federated learning model suitable for deep neural networks, while Fed-DS [35] is a federated learning algorithm designed to handle data streams. Experiments are conducted on CIFAR10 [28] and Fashion MNIST (FMNIST) [55] datasets. CIFAR-10 and FMNIST contain 60 , 000 and 70 , 000 images. For both CIFAR-10 and FMNIST, a CNN with a VGG architecture [49], consisting of two blocks, is pre-trained on a subset of training samples from each dataset. The training datasets are biased towards class 0 . More details about training the CNNs can be found in Appendix D. For both the CIFAR-10 and FMNIST datasets, 10 , 000 test samples are sequentially received by clients. There are 20 clients in total, and the data samples are distributed non-i.i.d. among them. For CIFAR-10, each client is biased towards one specific class, with 55% of the samples belonging to that class and 5% of the samples belonging to each of the other 9 classes. For FMNIST, each client is biased towards two classes, and the distribution of samples is time-variant. More details about the data distribution among clients and experimental setup can be found in Appendix D. At each time step, all algorithms employ batch of size 10 for model update. The learning rates for all algorithms are set to η = 0 . 01 / √ T and η c = 1 / √ T . The server stores models every n = 20 time steps. The metric to evaluate the performance of algorithms is the accuracy. The accuracy for client i is defined as Accuracy i = 1 T ∑ T t =1 1 ˆ y i,t = y i,t where ˆ y i,t denotes the label predicted by client i at time step t .

Average accuracy and its standard deviation across clients for CIFAR-10 and FMNIST are reported in Table 2. At each time step t , clients can store 10 model parameters. Therefore, M is set to M = 8 for Fed-POE. The results indicate that the performance of Local relative to federated learning baselines depends on the dataset. While Local outperforms all federated baselines except for Fed-Rep and Fed-POE on FMNIST, it obtains the worst accuracy on CIFAR-10. Conversely, Fed-POE achieves the highest accuracy for both datasets, indicating

that Fed-POE efficiently leverages the advantages of both federated and local models. To analyze the effect of the number of models M and batch size b on the Fed-POE performance, experiments are conducted on the CIFAR-10 dataset, varying the batch size b and the number of models M selected by each client to construct the ensemble model. As observed in Table 3, the batch size b = 1 results in the worst accuracy, mainly due to the forgetting process where models overfit to the most recently observed data. However, increasing the batch size from b = 10 or b = 20 to b = 30 does not significantly improve the accuracy. Larger batch sizes may lead the model to perform better on older data, as the model is trained on older data over more iterations. Therefore, this study concludes that a moderate batch size is optimal, considering that increasing the batch size also increases computational complexity. Table 4 in Appendix D presents the accuracy of Fed-POE on both the CIFAR-10 and FMNIST datasets with varying values of M . The table shows that employing the saved models by the server in D t improves performance, as setting M = 0 results in the worst accuracy. Moreover, it can be observed that increasing M does not necessarily lead to further accuracy improvement. This aligns with the intuition behind selecting a subset of models rather than using all models.

6 Conclusions

This paper proposed Fed-POE, a personalized federated learning algorithm designed for online prediction and model fine-tuning. Fed-POE constructs an ensemble using local models and federated models stored by the server periodically over time. Theoretical analysis demonstrated that Fed-POE achieves sublinear regret. Experimental results revealed that the relative performance of local models compared to federated models depends on the dataset, making the decision between local model training and federated learning challenging. However, experimental results also show that Fed-POE consistently outperforms both local and federated models across all datasets. This indicates that Fed-POE effectively leverages the advantages of both local and federated models.

References

Durmus Alp Emre Acar, Yue Zhao, Ruizhao Zhu, Ramon Matas, Matthew Mattina, Paul Whatmough, and Venkatesh Saligrama. Debiasing model updates for improving personalized federated training. In Proceedings of International Conference on Machine Learning , volume 139, pages 21-31, Jul 2021.

Noga Alon, Nicolò Cesa-Bianchi, Claudio Gentile, Shie Mannor, Yishay Mansour, and Ohad Shamir. Nonstochastic multi-armed bandits with graph-structured feedback. SIAM Journal on Computing , 46(6):1785-1826, 2017.

Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. The nonstochastic multiarmed bandit problem. SIAM Journal on Computing , 32(1):48-77, Jan 2003.

Nicolo Cesa-Bianchi and Gabor Lugosi. Prediction, Learning, and Games . Cambridge University Press, USA, 2006.

Zachary Charles, Zachary Garrett, Zhouyuan Huo, Sergei Shmulyian, and Virginia Smith. On large-cohort training for federated learning. In Advances in Neural Information Processing Systems , 2021.

Huili Chen, Jie Ding, Eric W Tramel, Shuang Wu, Anit Kumar Sahu, Salman Avestimehr, and Tao Zhang. Self-aware personalized federated learning. Advances in Neural Information Processing Systems , 35:20675-20688, 2022.

Yujing Chen, Yue Ning, Martin Slawski, and Huzefa Rangwala. Asynchronous online federated learning for edge devices with non-iid data. In IEEE International Conference on Big Data (Big Data) , pages 15-24, Dec 2020.

Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. Exploiting shared representations for personalized federated learning. In Proceedings of the International Conference on Machine Learning , volume 139, pages 2089-2099, Jul 2021.

Georgios Damaskinos, Rachid Guerraoui, Anne-Marie Kermarrec, Vlad Nitu, Rhicheek Patra, and Francois Taiani. Fleet: Online federated learning via staleness awareness and performance prediction. ACM Transactions on Intelligent Systems and Technology , 13(5), Sep 2022.

Yuyang Deng, Mohammad Mahdi Kamani, and Mehrdad Mahdavi. Adaptive personalized federated learning. arXiv preprint arXiv:2003.13461 , 2020.

Canh T. Dinh, Nguyen H. Tran, and Tuan Dung Nguyen. Personalized federated learning with moreau envelopes. In Proceedings of International Conference on Neural Information Processing Systems , page 21394-21405, Dec 2020.

Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. In Advances in Neural Information Processing Systems , volume 33, pages 3557-3568, Dec 2020.

Alan Fern and Robert Givan. Online ensemble learning: An empirical study. Machine Learning , 53:71-109, 2003.

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of International Conference on Machine Learning , volume 70, pages 1126-1135, Aug 2017.

Dylan Foster, Satyen Kale, Mehryar Mohri, and Karthik Sridharan. Parameter-free online learning via model selection. In Proceedings of International Conference on Neural Information Processing Systems , 2017.

Bhargav Ganguly and Vaneet Aggarwal. Online federated learning via non-stationary detection and adaptation amidst concept drift. arXiv preprint arXiv:2211.12578 , 2022.

Francois Gauthier, Vinay Chakravarthi Gogineni, Stefan Werner, Yih-Fang Huang, and Anthony Kuh. Resource-aware asynchronous online federated learning for nonlinear regression. In IEEE International Conference on Communications , pages 2828-2833, 2022.

Pouya M Ghari and Yanning Shen. Online multi-kernel learning with graph-structured feedback. In Proceedings of the International Conference on Machine Learning , volume 119, pages 3474-3483, Jul 2020.

Pouya M. Ghari and Yanning Shen. Personalized online federated learning with multiple kernels. In Advances in Neural Information Processing Systems , 2022.

Pouya M. Ghari and Yanning Shen. Graph-assisted communication-efficient ensemble federated learning. In European Signal Processing Conference (EUSIPCO) , pages 737-741, 2022.

Pouya M. Ghari and Yanning Shen. Budgeted online model selection and fine-tuning via federated learning. Transactions on Machine Learning Research , 2024.

Pouya M. Ghari and Yanning Shen. Online learning with uncertain feedback graphs. IEEE Transactions on Neural Networks and Learning Systems , 35(7):9636-9650, 2024.

Vinay Chakravarthi Gogineni, Stefan Werner, Yih-Fang Huang, and Anthony Kuh. Communication-efficient online federated learning strategies for kernel regression. IEEE Internet of Things Journal , 10(5):4531-4544, 2023.

Filip Hanzely, Slavomír Hanzely, Samuel Horváth, and Peter Richtárik. Lower bounds and optimal algorithms for personalized federated learning. In Proceedings of International Conference on Neural Information Processing Systems , page 2304-2315, Dec 2020.

Steven C. H. Hoi, Rong Jin, Peilin Zhao, and Tianbao Yang. Online multiple kernel classification. Machine Learning , 90:289-316, Feb 2013.

Songnam Hong and Jeongmin Chae. Communication-efficient randomized algorithm for multi-kernel online federated learning. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(12):9872-9886, 2022.

Markelle Kelly, Rachel Longjohn, and Kolby Nottingham. UCI machine learning repository, 2023.

Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.

Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine , 37(3):50-60, 2020.

Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust federated learning through personalization. In Proceedings of the International Conference on Machine Learning , volume 139, pages 6357-6368, Jul 2021.

Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of fedavg on non-iid data. In International Conference on Learning Representations , 2020.

Xin-Chun Li, De-Chuan Zhan, Yunfeng Shao, Bingshuai Li, and Shaoming Song. Fedphp: Federated personalization with inherited private models. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases , pages 587-602, 2021.

Ken Liu, Shengyuan Hu, Steven Wu, and Virginia Smith. On privacy and personalization in cross-silo federated learning. In Advances in Neural Information Processing Systems , 2022.

Othmane Marfoq, Giovanni Neglia, Aurélien Bellet, Laetitia Kameni, and Richard Vidal. Federated multi-task learning under a mixture of distributions. In Proceedings of International Conference on Neural Information Processing Systems , volume 34, pages 15434-15447, Dec 2021.

Othmane Marfoq, Giovanni Neglia, Laetitia Kameni, and Richard Vidal. Federated learning for data streams. In Proceedings of The International Conference on Artificial Intelligence and Statistics , volume 206, pages 8889-8924, Apr 2023.

Cobbinah B. Mawuli, Jay Kumar, Ebenezer Nanor, Shangxuan Fu, Liangxu Pan, Qinli Yang, Wei Zhang, and Junming Shao. Semi-supervised federated learning on evolving data streams. Information Sciences , 643:119235, 2023.

Aritra Mitra, Hamed Hassani, and George J. Pappas. Online federated learning. In IEEE Conference on Decision and Control (CDC) , pages 4083-4090, Dec 2021.

Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh. Agnostic federated learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the International Conference on Machine Learning , volume 97, pages 4615-4625, Jun 2019.

Vidya Muthukumar, Mitas Ray, Anant Sahai, and Peter Bartlett. Best of many worlds: Robust model selection for online supervised learning. In Proceedings of the International Conference on Artificial Intelligence and Statistics , volume 89, pages 3177-3186, Apr 2019.

Mehdi Neshat, Bradley Alexander, Markus Wagner, and Yuanzhong Xia. A detailed comparison of meta-heuristic methods for optimising wave energy converter placements. In Proceedings of the Genetic and Evolutionary Computation Conference , page 1318-1325, Jul 2018.

Aldo Pacchiano, Christoph Dann, and Claudio Gentile. Best of both worlds model selection. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems , 2022.

Jonghwan Park, Dohyeok Kwon, and Songnam Hong. Ofedqit: Communication-efficient online federated learning via quantization and intermittent transmission. arXiv preprint arXiv:2205.06491 , 2022.

Kumar Kshitij Patel, Lingxiao Wang, Aadirupa Saha, and Nathan Srebro. Federated online and bandit convex optimization. In Proceedings of the International Conference on Machine Learning , volume 202, pages 27439-27460, Jul 2023.

Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In Proceedings of International Conference on Neural Information Processing Systems , pages 1177-1184, Dec 2007.

Vinay Venkatesh Ramasesh, Aitor Lewkowycz, and Ethan Dyer. Effect of scale on catastrophic forgetting in neural networks. In International Conference on Learning Representations , 2022.

Mohammad Reza Karimi, Nezihe Merve Gürel, Bojan Karlaš, Johannes Rausch, Ce Zhang, and Andreas Krause. Online active model selection for pre-trained classifiers. In Proceedings of The International Conference on Artificial Intelligence and Statistics , volume 130, pages 307-315, Apr 2021.

Daniel Rothchild, Ashwinee Panda, Enayat Ullah, Nikita Ivkin, Ion Stoica, Vladimir Braverman, Joseph Gonzalez, and Raman Arora. FetchSGD: Communication-efficient federated learning with sketching. In Proceedings of the International Conference on Machine Learning , volume 119, pages 8253-8265, Jul 2020.

Doyen Sahoo, Steven C.H. Hoi, and Bin Li. Online multiple kernel regression. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , page 293-302, 2014.

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations , May 2015. URL http://arxiv.org/abs/1409.1556 .

Yue Tan, Guodong Long, Jie Ma, Lu Liu, Tianyi Zhou, and Jing Jiang. Federated learning from pre-trained models: A contrastive learning approach. Advances in neural information processing systems , 35:19332-19344, 2022.

Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J Gordon. An empirical study of example forgetting during deep neural network learning. In International Conference on Learning Representations , 2019.

Grace Wahba. Spline Models for Observational Data . Society for Industrial and Applied Mathematics, 1990.

Yujia Wang, Lu Lin, and Jinghui Chen. Communication-efficient adaptive federated learning. In Proceedings of the International Conference on Machine Learning , volume 162, pages 22802-22838, Jul 2022.

Yue Wu, Shuaicheng Zhang, Wenchao Yu, Yanchi Liu, Quanquan Gu, Dawei Zhou, Haifeng Chen, and Wei Cheng. Personalized federated learning under mixture of distributions. In International Conference on Machine Learning , pages 37860-37879, 2023.

Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 , 2017.

Rui Ye, Zhenyang Ni, Fangzhao Wu, Siheng Chen, and Yanfeng Wang. Personalized federated learning with inferred collaboration graphs. In International Conference on Machine Learning , pages 39801-39817, 2023.

Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Fedala: Adaptive local aggregation for personalized federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 11237-11244, 2023.

Shuyi Zhang, Bin Guo, Anlan Dong, Jing He, Ziping Xu, and Song Xi Chen. Cautionary tales on air-quality improvement in beijing. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences , 473(2205):20170457, 2017.

A Proof of Theorem 1

This section provides the proof of Theorem 1. The proof follows similar steps to those in [37], and it is included here for the sake of completeness and to make the paper self-contained.

According to (3) and (4), for any θ it can be written that

Due to convexity of L ( · , · ) it can be concluded that

Combining (21) with (22), we get

Using assumption A2 and Arithmetic Mean Geometric Mean (AM-GM) inequality it can be written that

Combining (23) with (24), we arrive at

Dividing both sides by 2 η N yields

Summing (26) over time, we obtain

Plugging in θ = θ ∗ and θ 0 = 0 into (27) and considering the fact that ∥ θ T +1 -θ ∥ 2 ≥ 0 , we find

which proves the Theorem.

B Proof of Theorem 2

According to (10), we can write

Using the inequality e -x ≤ 1 -x + 1 2 x 2 , ∀ x ≥ 0 , from (29) we arrive at

Taking the logarithm from both sides of (30) and using the inequality 1 + x ≤ e x , we get

Considering the assumption that 0 ≤ L ( f ( x ; θ ) , y ) ≤ 1 , ∀ x , θ , (31) can be relaxed to

Summing (32) over time, we obtain

According to Hölder's inequality, for any positive real numbers p and q satisfying 1 p + 1 q = 1 , the following inequality holds:

To meet the condition 1 p + 1 q = 1 , it is necessary that p ≥ 1 and q ≥ 1 . Consequently, (34) can be modified to:

Considering the fact that α i, 1 = β i, 1 = 1 , based on (35) we can write

According to the update rule in (29), (36) is equivalent to

Combining (33) with (37) we arrive at

Due to the convexity of L ( · , · ) , we can write

Combining (38) with (39) we get

Substituting p = ∞ and q = 1 in (40), we obtain

Since Fed-POE updates ϕ i,t locally using online gradient descent as outlined in (8), according to (6), for any ϕ we can write

Substituting ϕ ∗ i in (42) along with combining (41) with (42), we can conclude that

which proves the personalized regret upper bound of Fed-POE in (12). Furthermore, plunging in p = 1 and q = ∞ into (40), we get

Summing (44) over all clients, we obtain

Combining (45) with (28), we arrive at

which proves the Theorem.

C Proof of Theorem 3

According to assumption A3 that 0 ≤ L ( f ( x ; θ ) , y ) ≤ 1 , it can be written that

Let ℓ ij,t denote the importance sampling loss estimate, which is expressed as

Let the total number of model parameters stored by the server after time step U is D and W i,t = ∑ D j =1 w ij,t . For any t > U , considering (17), we can write

Employing the inequality e -x ≤ 1 -x + 1 2 x 2 , ∀ x ≥ 0 , from (49) we obtain

Taking the logarithm from both sides of (50) and using the inequality 1 + x ≤ e x , we arrive at

Summing (51) over time, we get

Considering the fact that the weights { w ij,t } D j =1 are initialized as w ij, 1 = 1 , ∀ j ∈ [ D ] , it can concluded that W i,U ≤ D . Theefore, for any j ∈ [ D ] , the left hand side of (52) is bounded from below as

Combining (53) with (52) yields

The expected values of ℓ ij,t and ℓ 2 ij,t given observed losses up until time step t can be obtained as

Taking the expectation from (54), we arrive at

Since q ij,t = 1 -(1 -p ij,t ) M = p ij,t (1 + (1 -p ij,t ) + (1 -p ij,t ) 2 + . . . +(1 -p ij,t ) M -1 ) , it can be concluded that q ij,t ≥ p ij,t . Therefore, (56) can be relaxed to

According to model selection procedure adopted by Fed-POE presented in Algorithm 1, client i chooses a subset of models by sampling them in M rounds with replacement. Let a ij,t ≥ 0 denote the number of times that the model j in D t is chosen by client i at time step t . The number of different situations for selected subset of models M i,t is equal to the number of solutions for the linear equation

Let A denote the set of all possible solutions for (58) such that if a ∈ A where a = [ a 1 , . . . , a D ] , a 1 , . . . , a D satisfies (58). Therefore, for the expected loss of the ensemble ˜ f i,t ( x i,t ) in (14), we can write

where ˜ f ( k ) i,t ( x i,t ) denote the k -th possible ensemble model generated by client i using Fed-POE. Using the Jensen inequality and convexity of the loss function, we can relax (59) to

where M ( k ) i,t and W ( k ) i,t are the k -th possible model subset and weight summations, respectively. Rearranging the right hand side of (60), we can write

where B j is the set of all possible solutions for the linear equation in (58) condition on a ij,t ≥ 1 . Since for any k , we have w ij,t ≤ W ( k ) i,t , (61) can be relaxed to

Since ∑ | B j | k =1 ( ∏ D m =1 p b m,k im,t ) includes all possibilities in B j , we can conclude that ∑ | B j | k =1 ( ∏ D m =1 p b m,k im,t ) = 1 . Combining this with (62), we obtain

Combining (63) with (57), we arrive at

Since 0 ≤ L ( f ( x ; θ ) , y ) ≤ 1 , it can be written that

Combining (65) with (64), we get

Since ¯ f i,t ( · ) similar to f i,t ( · ) is the ensemble of two models, following the same derivation steps from (29) to (41) by substituting ¯ f i,t ( x i,t ) , f i,t ( x i,t ) and ˜ f i,t ( x i,t ) with f i,t ( x i,t ) , f ( x i,t ; θ t ) and f ( x i,t ; ϕ i,t ) , respectively, we can conclude that

Taking the expectation from both sides of (67b) with respect to randomization in model selection along with combining (67b) with (66) yields

Furthermore, combining (67a) with (41) and (44) and taking the expectation with respect to model selection randomization, we obtain

Recall that h j ( · ) associated with the j -th model parameter in D t be defined as h j ( x i,t ) = f ( x i,t ; ρ j ) while h loc ( · ) and h fed ( · ) correspond to the local and federated models, respectively, defined as h loc ( x i,t ) = f ( x i,t ; ϕ i,t ) and h fed ( x i,t ) = f ( x i,t ; θ t ) . Also recall that H := { h j | ∀ j : 1 ≤ j ≤ | D T |} ∪ { h loc , h fed } . Comparing the right hand sides of (69a) and (69b) with that of (68) and considering the fact that D ≥ 2 , for any h ∈ H we can write

By substituting h ( · ) with h ∗ i ( · ) as defined in (18b) and considering the fact that D ≤ U/n , we obtain the personalized regret upper bound for client i as shown in (20). Moreover, taking the average of (70) across clients and substituting h ( · ) with h ∗ ( · ) , we arrive at

which proves the theorem.

D Supplementary Experimental Details

This section presents supplementary experimental results and details about experimental setup. All experiments were carried out using Intel(R) Core(TM) i7-10510U CPU @ 1.80 GHz 2.30 GHz processor with a 64-bit Windows operating system.

D.1 Regression Data Distribution

As it is pointed out in section 5, the present paper tests the performance of algorithms for online regression task on Air and WEC datasets. These datasets are downloaded from UCI Machine Learning Repository [27]. Each data sample in Air dataset includes air quality information such as concentration of some chemicals in the air. Data samples in Air dataset collected from four different geographical locations. Moreover, data samples in WEC, collected from wave energy converters in four different geographical locations. In order to distribute data, clients are partitioned into 4 groups. For each group, 70% of data samples observed by each client in the group belongs to a specific geographical location while 10% of observed data samples belong to each of the rest 3 locations.

D.2 Random Feature Kernel-based Models

As it is pointed in section 5, the proposed Fed-POE and all baselines utilize a random feature kernelbased model to perform online regression task. In what follows we explain random feature-based kernel models. Let κ ( · , · ) be a positive-definite function called kernel such that κ ( x , x ′ ) measures the similarity between x and x ′ . In online kernel learning context, at time step t +1 , the following prediction is made for x (see e.g. [52, 25, 48]):

where α t = [ α 1 , 1 , . . . , α N, 1 , . . . , α 1 ,t , . . . , α N,t ] denotes the learnable parameters. Therefore, the number of parameters that should be learned grows with time. In order to alleviate the computational complexity of online kernel learning, random feature approximation [44] can be employed. In fact, using random feature approximation, the number of parameters that needs be learned is timeinvariant and is selected by the algorithm. Assume that κ ( · ) is a shift-invariant kernel function such that κ ( x , x ′ ) = κ ( x -x ′ ) . Also, suppose that κ ( · ) is scaled such that κ ( 0 ) = 1 . Let ξ ( · ) denotes the Fourier transform of κ ( · ) . According to definition of inverse Fourier transform κ ( 0 ) = ∫ ∞ -∞ ξ ( ω ) d ω = 1 . Therefore, it can be concluded that ξ ( · ) is a probability density function (PDF). Let ω 1 , . . . , ω D be drawn randomly from ξ ( · ) and called random features. Using the random features ω 1 , . . . , ω D , the representation z ( x ) is defined as

Given random features ω 1 , . . . , ω D and using the proposed Fed-POE, at time step t , client i makes prediction f ( x i,t ; θ t ) = θ ⊤ t z ( x i,t ) . Clients and the server employ the proposed Fed-POE to learn the parameter θ t . As it can be inferred since the model f ( · ; · ) is linear with respect to model parameter θ t , using convex loss functions, the loss L ( f ( x i,t ; θ t ) , y i,t ) is convex as well.

Furthermore, in section 5, the proposed Fed-POE utilizes three Gaussian kernels for online regression on Air and WEC datasets. In order to implement multi-kernel learning for Fed-POE, the prediction of kernels are linearly combined and the weights for linear combination is learned locally by each client. Let f 0 . 1 ( x i,t ) , f 1 ( x i,t ) and f 10 ( x i,t ) represent predictions of Gaussian kernels with variances of 0 . 1 , 1 and 10 , respectively. Then at time step t , client i makes prediction w 0 . 1 ,it f 0 . 1 ( x i,t ) + w 1 ,it f 1 ( x i,t ) + w 10 ,it f 10 ( x i,t ) . In order to update weights w 0 . 1 ,it , w 1 ,it and w 10 ,it , client i employs multiplicative update rule. As an example after observing the loss L ( f 1 ( x i,t ) , y i,t ) , client i updates w 1 ,it as w 1 ,i ( t +1) = w 1 ,it exp( -γ i L ( f 1 ( x i,t ) , y i,t )) where γ i is a learning rate specified by client i .

D.3 Image Classification Experimental Setup

The pre-trained CNN used by Fed-POE and other baselines is biased toward class label 0 . For CIFAR-10, the pre-trained CNN is trained on a subset of the CIFAR-10 training data, consisting of 5 , 000 samples with label 0 and 500 samples from each of the other 9 class labels. For FMNIST, the model is trained on a subset of the FMNIST training data, consisting of 6 , 000 samples with label 0 and 500 samples from each of the other 9 class labels. The CNN models are trained using Tensorflow 2.16.1. We used the SGD optimizer with a learning rate of 10 -3 and momentum of 0 . 9 . The models for CIFAR-10 and FMNIST were trained for 100 epochs and 10 epochs, respectively.

Clients receive test data samples sequentially and make prediction for the newly received sample. To distribute test data samples of CIFAR-10 among clients, we split clients into 10 groups. For

(a) CIFAR-10

(b) WEC

gret

500

Local

gret

17.5

Local

Ditto

6

15.0

Ditto

ative

400

Fed-Rep

Fed-OMD

ative

R

12.5

Fed-Rep

Fed-OMD

300

Fed-DS

Fed-ALA

E

10.0

POF-MKL

eM-KOFL

200

Fed-POE

7.5

Fed-POE

100

6

5.0

Av

2.5

0

0.0

0

100

200

300

400

500

0

50

100

150

200

250

Time

Time

CIFAR-10, 55% of samples observed by a client belongs to a specific class label while only 5% of received samples belong to each of other 9 class labels. For FMNIST, client data distribution is time-variant. Since the number of test samples is 10 , 000 and the number of clients is 20 , it can be concluded that time horizon T is 500 . In the first half of time steps (i.e. t ≤ 250 ), each client observes 200 samples from the first 5 class labels and 50 samples from other 5 class labels. In the second half, this is reversed: clients observe 200 samples from the last 5 class labels and 50 samples from the other class labels. In each half, each client is biased toward one of the five majority classes. For example, if a client is biased toward class 0 in the first half, it observes 100 samples from class label 0 , 25 samples from each of class labels 1 to 4 , and 10 samples from each of class labels 5 to 9 . Therefore, in each half, each client observes 100 samples from one class, 25 samples from each of four other classes, and 10 samples from each of the remaining five classes.

To implement Fed-OMD for both regression and image classification, we used the ℓ 2 -norm as a regularizer function for mirror descent. For implementing Ditto for both regression and image classification, we set the regularization factor λ to 1. In the case of image classification using Fed-Rep, clients locally fine-tune the last two layers of the CNN model, while the rest of the network is used as the global backbone to generate representations. Furthermore, Fed-POE is compatible with any federated learning method and can utilize any federated algorithm. For CIFAR-10, Fed-POE uses Fed-OMD, while for FMNIST, Fed-POE uses Fed-Rep. To fine-tune the CNN model, Fed-POE and all baselines use the SGD optimizer and the cross-entropy loss function.

D.4 Supplementary Results

Table 4 presents the accuracy of clients for image classification using Fed-POE with varying values of M . As can be seen for both CIFAR-10 and FMNIST, when M > 0 , the accuracy is higher than in the case where M = 0 . The case where M = 0 corresponds to clients not using models stored by the server in their ensemble. Therefore, these results show that constructing the ensemble using previous federated model parameters stored by the server improves the accuracy of Fed-POE. This indicates the effectiveness of the model selection procedure of Fed-POE presented in Algorithm 1. Additionally, these results show that increasing M does not necessarily lead to further accuracy improvement. This implies the effectiveness of Fed-POE's model selection in pruning model parameters from the ensemble that have relatively lower accuracy. Figure 1 illustrates the average cumulative global regret of clients over time using Fed-POE and all other baselines. As depicted, Fed-POE achieves sublinear regret for both CIFAR-10 and WEC datasets. This corroborates the theoretical results in Theorems 2 and 3.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction reflect that the contribution of this paper is the proposal of a novel personalized federated learning algorithm for online prediction and model fine-tuning.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: In Section 4, the paper makes it clear that Fed-POE utilizes multiple models instead of a single model for prediction. This requires more computations than the case where clients use only one model. Computational efficiency of Fed-POE is discussed in Section 4.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The paper provides the full set of assumptions in Section 3 and the complete proofs of all theorems can be found in Appendices A, B and C.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: In Section 5 and Appendix D, we provide detailed information needed to reproduce the results.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We provide open access to the data and code.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: In Section 5 and Appendix D, we provide all the training and test details necessary to understand the results.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: In Tables 1, 2 and 4, We reported the standard deviation of the results.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: In Appendix D, we clarify that all experiments were carried out using Intel(R) Core(TM) i7-10510U CPU @ 1.80 GHz 2.30 GHz processor with a 64-bit Windows operating system.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We believe the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This paper studies the problem of personalized federated learning. While personalized federated learning, in general, has societal impact, we do not foresee any specific societal impact resulting from our work.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We believe that the paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: In Section 5 and Appendix D, we properly cited the original owners of assets.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper introduces a personalized federated learning algorithm to address the challenges of real-time predictions in non-stationary environments. Clients fine-tune models online, combining their locally fine-tuned models with multiple federated models learned over time. This approach ensures efficient adaptation to evolving data streams, with theoretical analysis and experiments on real datasets demonstrating its effectiveness.

STRENGTHS:
- The proposed algorithm effectively addresses the challenge of making real-time predictions in non-stationary environments by allowing clients to fine-tune models online, ensuring continuous adaptation to evolving data streams.
- By combining locally fine-tuned models with multiple federated models, the approach enhances personalization and leverages the strengths of both local and federated learning, resulting in improved performance.
- The paper provides a solid theoretical analysis alongside experimental validation on real datasets, demonstrating the practical effectiveness and robustness of the proposed algorithm in real-world scenarios.

WEAKNESSES:
1. Contributions are suggested to list by items for clear summaries.
2. The baselines in Table 1 are all before the 2022 year, more latest related methods published in 2023 should be compared.
3. Fed-POE has limited improvements on Air and FMNIST datasets.
4. The process of combining locally fine-tuned models with multiple federated models may introduce significant computational overhead for clients, especially those with limited resources.
5. As the number of clients increases, managing and integrating multiple personalized models can become complex, posing scalability challenges for the proposed algorithm.

QUESTIONS:
No

LIMITATIONS:
Yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel personalized federated learning algorithm, Fed-POE, which is designed for adaptive prediction and model fine-tuning in dynamic environments. It addresses the challenge of real-time predictions on streaming data by constructing a personalized model that combines a locally fine-tuned model with multiple federated models. Theoretical analysis and experiments on real datasets demonstrate its effectiveness in achieving sublinear regret bounds and improved online prediction accuracy.\n\nSTRENGTHS:\n1. The paper proposes a unique ensemble method that dynamically combines local and federated models, which is a novel approach in the field of federated learning.\n\n2. It provides a solid theoretical analysis, demonstrating sublinear regret bounds for convex models.\n\n3. The paper is well organized.\n\nWEAKNESSES:\n1. Although the presented method is novel, it is simply a combination of the previous personalized federated learning approaches as well as ensemble learning and provides comparably little conceptual originality.  The contribution's main novelty seems to be that integrating results from prior models would be beneficial in mitigating catastrophic forgetting in online federated learning.\n\n2. Experimental results show that the improvement in the accuracy of Fed-POE compared to other methods is not significant, but ensemble learning inevitably increases the computational overhead increase. The paper needs to analyze whether this trade-off is reasonable.\n\n3. The paper needs more experiments to prove the effectiveness of the method, for example, for real-time predictions, the size of the old data replay is crucial, and the authors should design experiments to analyze the effect of batch size b on the experimental results. This paper also needs experiment results on the accuracy over the time step.\n\nQUESTIONS:\n1. The method in this paper does not significantly improve accuracy and even has a larger standard deviation. Can you give me more reasons to support your methods？\n\n2. The method is designed with two parts to mitigate catastrophic forgetting (old data replay and integration of multiple old models), The complex model updating process is unreasonable for real-time prediction. Can you design more ablation experiments to analyze these two parts？\n\nLIMITATIONS:\nsee the weaknesses and questions", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces an interesting perspective about the role of ensembles of models in federated learning. The provocative claim is the fact that federated learning is not always better than locally-trained models. This is contextualized in the field of not IID data and time-varying data generating processes. To address this issue the paper introduces from the theoretical point of view how to quantify the regret in federated and locally trained models. In addition it includes an analysis of non-convex models by managing an history of models. The overall impression about the paper is positive even if some points could have been better explored (in particular the part related to not IID data that is somehow the core of the paper).\n\nSTRENGTHS:\n- The paper introduces a theoretical evaluation of the gain produced by federated models w.r.t. locally trained models. This results show that federated learning is relevant only when models can be considered iid (hence averaging providing better results). This is somehow a know results but I appreciated the theoretical analysis\n- The proposed solution is to combine with a convex mean a locally-trained models with the federated models\n- This is further extended in case of non-convex models by considering an \"history\" of models to be used when needed (i.e., according to the loss)\n\nWEAKNESSES:\n- The federated models somehow includes the locally-trained model. I would have appreciated a further analysis about the fact that the two \"sides\" of the average model are related each other. \n- The setting in which eta and eta_c scales with T prevents adaptation in the long run (which is somehow the core of the paper). How to deal with that?\n- Federated learning typically takes also into account the complexity of the learning phase (i.e., the amount of info to be transmitted, e.g., the models). This is not quantified here. And this could be also a weak point in the fed-poe algorithm.\n\nQUESTIONS:\nSee Weaknesses box\n\nLIMITATIONS:\nSee Weaknesses box", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-14"}

comments: SUMMARY:
This paper introduces Fed-POE, a novel personalized federated learning algorithm tailored for online prediction and model fine-tuning. Fed-POE creates an ensemble by integrating local models with those periodically contributed by the server over time. Theoretical analysis confirms that Fed-POE attains sublinear regret. Empirical results demonstrate that Fed-POE consistently surpasses the performance of both local and federated models across all evaluated datasets, which indicates that Fed-POE effectively leverages the advantages of both local and federated models.

STRENGTHS:
- The technical content of the paper appears to be accurate, although I did not check all the details carefully.
- This paper is generally well-written and structured clearly.
- The experiments substantiate the main theoretical analysis, and the proposed algorithm demonstrates superior performance over the baseline methods

WEAKNESSES:
My primary concern is that the assertion the proposed algorithm can effectively harness the combined advantages of federated and local models is not clearly demonstrated within the theoretical bounds. The paper presents two principal theoretical results: Theorem 2 provides the regret upper bound for the proposed algorithm in convex scenarios, while Theorem 3 addresses non-convex cases. Both theorems establish sublinear regret bounds that are consistent with those for federated learning using a straightforward online gradient descent approach.  I recommend enhancing the clarity of the proposed method's advantages in the theorems by incorporating assumptions about the data distributions.

QUESTIONS:
See weaknesses.

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-15