Personalized Federated Learning with Mixture of Models for Adaptive Prediction and Model Fine-Tuning
Authors: Pouya M. Ghari
Yanning Shen
Venue: NeurIPS 2024 poster (2024)
Keywords: Federated Learning
Personalized Models
Real-time Predictions
Primary area: other

Abstract: Federated learning is renowned for its efficacy in distributed model training, ensuring that users, called clients, retain data privacy by not disclosing their data to the central server that orchestrates collaborations. Most previous work on federated learning assumes that clients possess static batches of training data. However, clients may also need to make real-time predictions on streaming data in non-stationary environments. In such dynamic environments, employing pre-trained models may be inefficient, as they struggle to adapt to the constantly evolving data streams. To address this challenge, clients can fine-tune models online, leveraging their observed data to enhance performance. Despite the potential benefits of client participation in federated online model fine-tuning, existing analyses have not conclusively demonstrated its superiority over local model fine-tuning. To bridge this gap, the present paper develops a novel personalized federated learning algorithm, wherein each client constructs a personalized model by combining a locally fine-tuned model with multiple federated models learned by the server over time. Theoretical analysis and experiments on real datasets corroborate the effectiveness of this approach for real-time predictions and federated model fine-tuning.

Personalized Federated Learning with Mixture of Models for Adaptive Prediction and Model Fine- Tuning
Pouya M. Ghari University of California Irvine pmollaeb@uci. edu
Yanning Shen University of California Irvine yannings@uci.een edu
Abstract
Federated learning renowned for its efficacy in distributed model training, ensur- ing that users, called clients, retain data privacy by not disclosing their data the central server that orchestrates collaborations. Most previous work on federated learning assumes that clients possess static batches of training data. However, clients may also need to make real-time predictions on streaming data in non- stationary environments. In such dynamic environments, employing pre-trained models may be inefficient, as they struggle to adapt to the constantly evolving data streams. To address this challenge, clients can fine-tune models online, leveraging their observed data to enhance performance. Despite the potential benefits of client participation in federated online model fine-tuning, existing analyses have not conclusively demonstrated its superiority over local model fine-tuning. To bridge this gap. the present paper develops a novel personalized federated learning algo- rithm, wherein each client constructs personalized model by combining a locally fine-tuned model with multiple federated models learned by the server over time. Theoretical analysis and experiments real datasets corroborate the effectiveness this approach for real-time predictions and federated model fine-tuning.
Introduction
Federated learning enables a group of learners, known as clients, to collaborate and collectively train model under the coordination of central server, without revealing their data. In this framework. clients perform local model updates and share these updates with the server. By aggregating these local updates, the server globally updates the model. Many prior works in the literature have assumed that each client stores a batch of training data and updates models locally based on this stored data B4 (33). However, in some cases, clients may need to make real-time predictions, and streams of data arrive sequentially, making it challenging to store and process data in batch. Furthermore, if clients operate in non-stationary and dynamic environment, employing pre-trained models may fall short prediction accuracy, requiring clients to fine-tune their models to adapt their data.
(see e.g.. [38] 29 47
federated learning framework, commonly referred to as online federated learning 37
26
[
specifically addresses situations where clients engage in real-time predictions using shared global model. After making predictions, clients collaborate with the server to update the global model for subsequent predictions in the future. Specifically, after making predictions, clients incur losses and based on the incurred losses, clients update the model locally and send their local updates to the server. The server updates the global model upon aggregating local updates and then distributes the updated global model to clients for use in the ongoing online prediction task. In this context, the
"Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

performance of clients can be assessed using the notion of regret [4 The regret of a client at given time step is defined as the difference between its prediction loss and that of the best model in hindsight. The best model in hindsight is determined as the model that achieves the minimum total prediction loss over time across all clients' data. The primary objective is to minimize the cumulative regret of all clients over time.
In the literature, federated learning algorithms based on online gradient descent have been proposed
This suggests that over the long run, these algorithms perform as well the best model in hindsight. However, in Section 3 this paper demonstrates that these federated learning algorithms do not obtain tighter regret bounds compared to the scenario where each client learns its own model locally without participating in federated learning. This indicates that participation in federated learning may not provide any benefit for clients performing online prediction. Specifically, if data distributed non-i.i.d. among clients and data distributions are time-variant and not known a priori, it is likely that local model training by clients will achieve better prediction accuracy than participation in federated learning. Although the benefit of federated learning in online prediction is not evident in existing theoretical analyses, models learned through federated learning enjoy higher generalizability as they are trained on all data samples distributed among clients. This motivates the idea that combining the model learned through federated learning with the locally learned one may prove effective in scenarios where clients need to perform online prediction.
that achieve sub-linear regret upper bounds [37 26 [9]
This paper proposes the Fed-POE (Federated Learning with Personalized Online Ensemble) algorithm. through which each client constructs personalized model for online prediction by adaptively ensembling the locally learned model and the model learned through federated learning. With Fed- POE, clients can adapt their local models to their data while benefiting from the higher generalizability of the federated model. Theoretical analysis for convex cases demonstrates that Fed-POE achieves sublinear regret bounds with respect to both the best federated and local models in hindsight. This indicates that Fed-POE effectively leverages the advantages of both federated and local models. Providing such theoretical guarantees may not be feasible for non-convex models such as neural networks. These models may suffer from the forgetting process [51 45), where fine-tuning on streaming data "on the fly' can lead to forgetting previously observed data samples. To overcome this challenge, the present paper proposes a novel framework in which the server periodically stores federated model parameters over time. Each client adaptively selects a personalized subset of stored models on the server based on the performance of these models in the client's 's online prediction task. Clients then use the selected models, along with the federated and local models, to construct an ensemble model for prediction. Clients select a subset of models to both control the memory and computational complexity of prediction and to prune models with relatively lower accuracy, thereby improving prediction performance. Theoretical analysis proves that Fed-POB achieves sublinear regret with respect to the best model in hindsight among the local model, federated model, and all models stored by the server. The contributions of the present paper are summarized as follows:
Fed-POE enables clients utilize the advantages of both local and federated models for online prediction tasks.
To address the issue of forgetting in online prediction, Fed-POE introduces novel federated framework for collaboration between clients and the server
Theoretical analyses for both convex and non-conver cases prove that Fed-POE achieves sublinear regret with respect to the best model in hindsight.
Extensive experiments on regression and image classification datasets show that Fed-POE effectively leverages the benefits of local and federated models, achieving higher online prediction accuracy compared to state-of-the-art federated learning algorithms.
2 Related Works
Personalized Federated Learning. Personalized federated learning involves developing individual-
ized models for each client, derived from global model learned through federated learning. Several
personalized federated learning algorithms have been proposed the literature (50 6 54 [6). In
24
30 33
clients construct their personalized models by adding regularization term to the
local objective and using the global federated model. Algorithms 12 allow clients to fine-tune
the global federated model using model-agnostic meta-lcarning [4] to learn their personalized

---

models. With Fed-Rep 8 each client generates representation using the global model and learns its own local head for prediction. Algorithms [T0 32 57 enable clients to achieve personalized models by combining local and global models. However, none of these works have addressed the problem of online prediction while clients collaborate on training their personalized models.
Federated Learning with Streaming Data. Several studies have explored the problem of federated learning when clients receive stream of data in real time. While 7 36 35
investigate federated learning scenarios where clients receive new training data in each learning round, they do not address the aspect of online prediction by clients. Consequently, these works cannot provide regret guarantees for online prediction. Additionally, [9] studies the issue of staleness in online federated learning to both train and make prediction with the model; however, lacks rigorous theoretical analysis. In [37], an online federated learning algorithm is introduced, utilizing online mirror descent, with a proven sublinear regret bound. Similarly, 42
16 propose online federated learning algorithms with guaranteed sublinear regret. Furthermore, 4 analyzes the benefits of collaboration in online federated learning, particularly in scenarios where only loss values at queried points are available to clients, without access to loss gradients. Regarding online federated kernel learning, algorithms are introduced [17 23], albeit without accompanying regret analysis, leading to an absence of guaranteed regret bounds. In 126 19], multiple kernel-based models and random feature-based online federated kernel learning algorithms are proposed, with demonstrated sublinear regret.
3
Preliminaries
This section explains the problem of federated learning for real-time prediction and model training. The present section studies the cases where clients either collaborate in federated learning or employ online gradient descent methods to locally train the model in real-time. This study highlights the motivation behind the proposed ensemble approach to federated learning.
3.1 Online Prediction and Federated Learning
Let there are N clients interact with server to train model f(::)) )- Also, let [N] := (1..... .l.....NY N). At each time step t. client i, Vi E [N] receives data sample Ii, Rd and makes the prediction f(x,,;; 0e) where 0g denotes the parameter of the model at time step t. Note that generalization to the scenario where at each time step, each client receives dataset instead of single data sample is straightforward. After making prediction, client i, Vi E [N] observes label yi,-- Then client Vi E [N] computes the loss of its prediction C(f 0,), Vs,e) where L(,,-) denotes the loss function. After computing the loss, clients send their updates to the server (e.g.. g. by sending the gradient loss V 0,), Vi.r)) and the server aggregates information from clients to update O, to 0.+1 to be used by clients to make predictions at time step / + 1. For ease of presentation, it is assumed that all clients can send their updates the server every time step. Generalizing the results for the cases where only a fraction of clients can send their updates is straightforward. Furthermore, is assumed that the label Hi.t for any and is determined by the environment through a process unknown to the clients. This implies that the data distribution observed by client can be non- stationary, and client i, Vi E [JV] does not know the distribution. Additionally, the data distribution can differ across clients. The goal is to enable clients to collaborate with the server to minimize the cumulative regret of clients over time. The regret of client at time step t defined as the difference between the prediction loss ((f 0z), Vs.e) and the loss [(f 6*), Vi,e) corresponding to the model with the optimal parameter 0". Therefore, the average cumulative regret of clients up to time horizon T is defined as
T
N
RT
(U(
Σ
M
0"),
where 0* denotes the optimal model parameter in hindsight and can be expressed as
T N
0" arg min Σ
(l
(2)
One common way to solve the problem that clients employ online gradient descent to update models locally and the server exploits federated averaging [3] 37 to update the global model.
3

---

3.2 Federated Learning with Online Gradient Descent
At each time step client i obtains the locally updated model parameter 44,,+1 as follows:
1i.+1 - ( (0),,ui1
where 7) is the learning rate. Aggregating locally updated model parameters, the server obtains the updated global model parameter for time step + 1 as
1
0++1
Σ
1.0+1
(4)
N
This continues up until the time horizon T. This paper examines regret under some or all of the following assumptions:
A1. The loss function [x; v) is convex with respect 0.
A2. The gradient of the loss is bounded as [IV 0), y)1l < G.
A3. For any æ, 0, the loss bounded as S (xf V) 1
The following theorem specifies the regret bound for federated learning employing online gradient descent under Al and A2.
Theorem 1. Employing online gradient descent, the following cumulative regret upper bound is guaranteed for federated learning under assumptions Al and A2:
10 27
((((), vo.e)
((((ar,;;))) Mh.))
+ GTT.
(5)
Proof of Theorem can be found in Appendix A According to Theorem choosing 1/ O OVVI/T)
the cumulative regret in (5) is bounded from above as O(vT) If the time horizon T unknown, the doubling trick technique (see e.g., [2] (22) can be effectively used to set the learning rate to maintain theoretical guarantees. Consider the case where each client learns the model locally using online gradient descent without collaborating with other clients and the server. Let Φi., denote the local model parameter learned by client at time step f, employing online gradient descent locally. The regret of client i in this case is equivalent to federated learning regret where there is only one client. Therefore, substituting = and 0g = 0 in (7) of Appendix A i with respect to any 0, we get
for the cumulative regret of client
|01 "
Σ (Lf(z,,........
Σ ( s
GºT.
(6)
2n
Averaging over all clients and substituting 0 with 0r (6). we obtain
T
T Σ
00'12 G 2
C(f(ii (1.))
C(f(e, 0'), W..)
+
(7)
21
Comparing (5) with (7). it can be inferred that under assumptions Al and A2, federated learning does not achieve tighter regret bound than the case where each client independently learns its local model. Hence, from theoretical standpoint, it remains uncertain whether collaboration in federated learning yields any improvement over local online model training. Intuitively, collaboration in federated learning may prove advantageous when there exists similarity among data samples observed by clients over time. However, cases where data distribution is heterogeneous and such similarities are lacking, employing online local training may yield superior results for a client. Given the lack of prior information on data distribution in online scenarios, each client can independently assess over time whether utilizing the model learned through federated learning for predictions is beneficial.
Furthermore, according to assumption AI, the thcoretical guarantees obtained in (5) and 7 hold if the loss is convex with respect to the model parameter. However, in the case of non-convex models, such as neural networks, these theoretical guarantees are not applicable. Non-convex models, like neural networks, may encounter the forgetting process [1] 45 where the model tends to overfit to recently observed data samples. Consequently, it may not be feasible to derive a single model parameter using online gradient descent that achieves sublinear regret with respect to the best model hindsight. The subsequent section introduces a novel algorithm aimed assisting clients in addressing such scenarios.

---

Personalized Federated Learning Methods
The current section introduces personalized federated learning algorithms where each client dynami- cally learns the prediction performance of both the models trained through federated learning over time and the locally learned model.
4.1 Ensemble Learning
At each time step, each client constructs an ensemble model comprising the federated model and its locally learned model to make prediction. Let Φi,t be the model parameter locally learned by client i such that at each time step i, client updates Φi,t via gradient descent as
Pi.++ Φ 7/ ( (s,)).
(8)
Furthermore, let clients and the server collaborate to update the federated model parameter O, as outlined (3) and (4). At time step i, client makes the prediction for at,, using its personalized ensemble model (- expressed as
aigt
( + Beg f o..)
f(,()),))) = a4 + Bi,,
(9)
at,t + Bi,t"
where ar, and B.,, represent the weights assigned by client the federated and local models, respectively, indicating the credibility of predictions from each model. After making predictions and observing the label yi,r, client computes the loss of predictions from the federated and local models, updating the weights Oi,t and Bi, using the multiplicative update rule as follows:
0,,+1 = a4, exp ( [ 0.), vr,)))
(10a)
B1, = Br,, exp (. (mi,s 2i,)), vi,e))
(10b)
where Te is a learning rate. Client i initializes a,, = and Pr, = The proposed algorithm
personalized since according to ( each client constructs its own ensemble model to perform prediction. The personalized regret of client is defined as C2,r
Σ (3.,)) E. ((f( (i), vt.)) where Φf denotes the best hindsight model parameter for client i which be expressed as o arg min Σ (ar (b)), V..)). The following theorem establishes the personalized regret upper bound for client i as well as global regret of all clients with respect to the best model parameter in hindsight.
Theorem 2. Under assumptions A Al-A3, employing the ensemble model in 99 for online prediction, the global regret of all clients bounded from above as
1 N
(() 1.
N +> ((f(a,,; v.)))))
110112
In(2)
G nTT (11)
"1
+
+2
27 ne
2
while client i achieves the following personalized regret upper bound:
T Σ ((Ua((aa)).aaa)
Σ (as ( 5
|| 2n
In(2)
GT
1 +
noT
(12)
πe
2
2
Proof of Theorem given in Appendix B According to 11) and 12 in Theorem setting n = (L/VTT), ne (1/VT) both the global regret for all clients and the personalized regret of each client achieve a regret bound of O(VT). This demonstrates that while the ensemble model in (9) ensures personalized regret guarantees for each client with respect to its best model in hindsight, it also enables clients to leverage federated learning, thus enjoying sublinear regret in comparison to the best global model in hindsight.
Comparison with Federated and Local Models. The main advantage of using the ensemble model instead of the federated model lies in Theorem [2] where it is shown that the ensemble model can
attain the global regret guarantee provided by the federated model while employing the federated
model, achieving the personalized regret guarantee in (12) is not feasible. However, according to (6)
and (7). using the online local training, each client achieves sublinear regret with respect to its best model while all clients achieve sublincar regret with respect to the best global model in hindsight.
5

---

Algorithm 1 Model selection by client i at time step
1: Input: Di M, wights {Wig.e)}=1 [Drl and M2, 2.
2: for m ...... M do
3: Sample model index l according to the PMF Pij.t
Wisie
Vj E [Dr]].
Σ)= weste
4: if l: $ M.,, then
5: Append model index & to M2,
6: end if
7: end for
8: Output: M.
This efficacy of local training stems from clients adapting the model to their individual data. In contrast, in federated learning, the model is trained on all data samples across clients, potentially leading to higher generalizability compared to its local counterpart. If there are similarities in the distribution of data samples among clients over time, the federated model is anticipated to achieve greater accuracy in online prediction. However, due to the lack of available information regarding the relationships between data samples observed by clients before online prediction, these advantages may not be reflected in theoretical bounds. The proposed method constructs ensemble to harness the advantages of both federated and local models for online prediction, as evidenced by Theorem 2 Experimental results in Subsection in [5.1 confirm that the ensemble model achieves superior performance compared to both local and federated models.
4.2 Model Selection
Regret guarantees for the ensemble method, as outlined in Theorem 2 are contingent upon the model's convexity. However, if the model is non-convex, achieving such guarantees may not be feasible. Particularly, non-convex models such as neural networks are susceptible to the forgetting process 51 45), wherein applying online gradient descent may lead to overfitting to recently observed data samples. This section introduces novel algorithm that allows clients to make online predictions using non-convex models while simultancously collaborating to fine-tune the model. The scenario assumes the existence of pre-trained model, and the objective for clients is two-fold: to make real-time predictions and to refine the model for alignment with their preferences. This situation may arise, for instance, in fine-ture fine-tuning large foundation models to tailor them client preferences.
Let the server and clients collaborate to fine-tune the non-convex model [(; -). At each time step I, client i updates the model on the batch of recently observed samples with size b as
11,+1 = 0, - Σ
VC(f(E,,,: T
(13)
Then the server aggregates locally updated parameters and updates the federated model parameter as in (4). Furthermore, each clients learns its own local model by fine-tuning the pre-trained model locally via online gradient descent as in (8) on the batch of b recently observed samples. While online gradient descent methods are well- for their efficiency in handling dynamic environments, employing the update rule of (13) for non-convex models may lead to overfitting to recently observed batches. To mitigate potential forgetting, the server saves the federated model parameters every time step, where n an integer hyperparameter.
Let ID. represent the set of model parameters stored by the server at time step L. At time step = [j 1)n + the server adds 0, D. meaning that D2 D, U ( Let Pj denotes the j-th model parameter in Dt. It can be concluded that P, = 0 f The server continues saving model parameters every n time steps until time step U < T. Client assigns weight Wiy,t to the j-th model in Dr which assesses the credibility of the predictions given by the model parameter Pj- At each time step f, client selects M models with replacement from D, to construct its model set M,,- Algorithm| Iillustraaes the model selection process conducted by client i at time step I. During each round of model selection, client i chooses a model according to probability mass function (PMF) proportional to weights (wij, 1 < j S [D:]]) where |. denotes the cardinality of a set (see step 3 in Algorithm Tp. Client i adds the chosen model to M,, if not already present. Therefore, it can be concluded that eM., M, Vi, t. Then at each time step t,
Client initializes Wij.1

---

Algorithm 2 Fed-POE: Federated Learning with Personalized Online Ensemble
Input: Model f(::-) batch size b, n, U, Di,1 Ø.
2: for = -....., do
3: The server transmits 0 to all clients.
4: for all i [N] client i do
5: Performs model selection given D, according to Algorithm I to obtain M1,
6: Makes prediction fz,)(Z in (15) using chosen model set MI,..
7: Upon receiving Yi,t. updates di,t, Bi, 7/,,, òi,t and (wszee ) as in 10 16 and 17
8: Updates the local model as Øs,t+1 = Φi, Ei=e- VL(f Di.r).
Updates the federated model 21,,+11 0₁ ET VL(f(2) 0o), Vi,,) 10: Sends 25,,4+1 to the server.
9:
end for
12:
if S U and l mod n = 0 then
13:
The server updates D. as De+1 Dr {0.).
14:
end if'
The server updates O, as 0.+1
#i= 11.411
16: end for
client i downloads all models in M,,, from the server. Upon receiving the models, client i constructs an ensemble model (-) as
fx)(
Σ
White f(x; P.).
(14)
mEM. Wimat
JEML,
Client i makes the prediction for xi,
F(,())))) = Thet
fin (x..e)
64, (x;)))
(15)
Tist + 6i,
Vift + Si,
where (.,)( is the ensemble of local and federated models as defined in (9) Furthermore, 7i and 64, are weights assigned by client i to ensemble models f+,(()) and f-, respectieely.. Upon observing the label ys after prediction, client updates weights Ye, and d,,
R.1+1 = Ni. exp (- (2.).).
(16a)
, 5,, (xpp(mnaa.....(ee Uk:.))
(16b)
Furthermore, as and Sr., which are used to construct fi,, (xL.,) in (9) are updated as in (10.. The weight wij, is updated using the importance sampling loss
Wij.t1 Wij.t exp (-ne
(( P)). Vi.e) 1yexu,
(17)
9:5.t
where 1 LJM,, denotes the indicator function and is if E M,,,- Moreover, Cij, the probability
that client selects the j-th model in Dy to be in Ml., and can be expressed as 9ij.t 1 where Pij.a defined in 3f Algorithm T The proposed algorithm, named Federated Learning with Personalized Online Ensemble (Fed-POE) is summarized in Algorithm 2 It is useful to note that proposed method Subsection 4.1 is a special case of Fed-POE by setting M 0 and 1.
Pij.t)
Efficiency of Fed-POF. To perform model selection using Fed-POF, clients do not need to store all model parameters in D, for all [T']. Instead, the server, which has higher storage capacity than the clients, stores all model parameters, and clients download subset of at most M model parameters. The hyperparameter M can be chosen such that clients can handle the memory and computational requirements of making predictions with the selected subset of models. Let Cpp denote the number of computations required to fine-tune the model f. and let C₁ represent the number computations required to make an inference with model f. Assume that the complexity of model selection in Algorithm is negligible compared fine-tuning and making inferences with model f. According to Algorithm 2, each client performs 2Cp + (M + 2)C, computations per time step. Therefore, the computational complexity of Fed-POR for each client is OCCp + MC,). Beyond memory and computational considerations, selecting subset of models from ID; helps clients improve their
7

---

prediction accuracy. Specifically, using the model weights faor, J at time step t, client i selects models that perform better on its data while pruning those with lower performance.
Let hy (ma,,) = (( P5) denote the model associated with the j-th model parameter in Dr- Further- more, (2:it)) ((a,t (2.2) and heaa (ea(()).)))
[a:;, 0x) represent the local and federated models, respectively. Let the set of models H. be defined as Hi := {hy Vj S S. [DT] U { hea This set includes all models that can be employed by each client using Fed-POE. The best model
in hindsight h² and the best model in hindsight for client i, hg are defined as
h² min KEB
(1 (18a)
14
min KEB
(18b)
The following theorem provides the regret upper bound of Fed-POE.
Theorem 3. Under assumption A3, employing Fed-POE in Algorithm the expected global regret of all clients is bounded from above as
E. E.
Σ
N
t=
In 20
1n
U +
+IT
Tee
UYU
(19)
TLe
2m.
while client achieves the following expected personalized regret upper bound:
T
Σ E. E.((((e,r(
21.)) - Σ L(h) (aii.),
1m1
In
2U
In 2n
Ne
U +JT
ne U)U.
(20)
T]e
2n
The expectation is taken with respect randomization in model selection.
The proof of Theorem 3 can be found in Appendix C According to (99 and 20 setting Te = O(L/VT),
(vT)) and U = O(VT), both the personalized and global regrets of clients achieve sublinear regret of O(VT). Since clients construct their ensemble models using time- varying subset of models, employing existing model selection and ensemble learning approaches
41 21 may not ensure the sublinear regrets stated in Theorem 3 However, by Fed-POE guarantees sublinear regret bounds while allowing clients the flexibility to select time-varying and personalized subsets of models for their ensembles.
13 15
39
46
20
using the proposed Algorithm |
5
Experiments
The present section studies the performance of Fed-POE in Algorithm comritheeommmpeedd to other baselines. Experiments are conducted both image classification and regression tasks. The performance of federated learning is examined in both convex and non-convex cases. Codes are available at https: /tte::///ittae github. hatpuu/..ccoooyyggggaar///a.a-a
5.1 Regression
The performance of the proposed Fed-POE is evaluated on online regression tasks. For these tasks, clients and the server collaborate train a random feature kernel-based model, which is known to be convex [25 48 18]. Details about the random feature kernel- based model used in experiments can found in Appendix D The performance of Fed-POE is compared with
baseline called Local, where clients train their models locally without participating in federated learning. Additionally, Fed-POE compared to personalized federated learning baselines Ditto 30 and Fed-Rep (8), the online federated learning baseline Fed-OMD [37], and online federated

---

kernel learning baselines eM- KOFL [66 and POF-MKL [9 Mean square error (MSE) is used as the metric to evaluate the performance of algorithms on regression task. MSE for client can be expressed as MSE; = E (Uis, vr,a)² where Uiy, denotes the prediction of client i at time step f. The performance of algorithms tested on two regression datasets Air [58] and WEC [40]. Air dataset a tme-series dataset that each data sample contains air quality features and the goal is to predict the concentration of contamination in the air. Each sample in WEC dataset contains features of different wave energy converters and the goal is to predict power output. Data samples are distributed non-i.i.d among 400 clients. Time horizon T for both datasets is 250. More information about datasets and distributed data among clients is presented in Appendix D
Table[present the MSE of algorithms and their standard deviation across clients. For all algo- rithms, the learning rates are set l = Tlc 1/VT. Table I shows that when data is dis- tributed non-i.idd among clients, local model training can achieve higher accuracy compared to federated learning. For the Air dataset, Local achieves lower MSE than other federated learn- ing baselines except for Fed-POE. For the WEC dataset, only POF-MKL achieves lower MSE than Local. This indicates that the performance of federated learning approaches compared to Local depends on the dataset. By utilizing both federated and local models, Fed-POE achieves the lowest MSE. Table Table[[show: shows that the performance of Fed-POE relative to other baselines is more consistent across different datasets.
Table 1: Average MSE (x10-3) and its standard deviation (x (x10-3) across clients for online regres- sion on Air and WEC datasets.
Methods
Air 9.12 = 3.59
WEC
Local
17.64 + 0.44
Ditto
10.65 + 5.69
33.88 + 16.08
Fed-Rep
10.48 + 5.23
35.13 + 10.38
Fed-OMD
11.48 6.84
32.61 27.38
eM-KOFL
11.516677
72.29 + 62.48
POF-MKL
10.66
16.94 + 15.92
Fed-POE
9.06 = 3.73
11.83 = 4.60
5.2 Image Classification
The performance of the proposed POR on an image classification task is compared with Local, Ditto [30], Fed-Rep (8) Fed-OMD 37 Fed-ALA [57], and Fed-DS 35 Fed-ALA
(77 is
personalized federated learning model suitable for deep neural networks, while Fed-DS 35 is
federated learning algorithm designed to handle data streams. Experiments are conducted on CIFAR-
10 (28] and Fashion MNIST (FMNIST) (55) datasets. CIFAR-10 10 and FMNIST contain 60, 000 and 70, 000 images. For both CIFAR- 10 and FMNIST, a CNN with a VGG architecture [49), consisting of two blocks, is pre- trainre
subset of training samples from each dataset. The training datasets are biased towards class O. More details about training the CNNs can be found in Appendix D For both the CIFAR-10 10 and FMNIST datasets, 10, 000 test samples clients. There are 20 clients in total, and the data samples are distributed non-i.i.d. among them. For CIFAR-10, 10, each client is biased towards
sequentially received by
specific class, with 55% of the samples belonging to that class and 5% of the samples belonging to each of the other 9 classes. For FMNIST, each client is biased towards two classes, and the distribution of samples is time-variant, More details about the data distribution among clients and experimental setup can be found in Appendix D At each time step, all algorithms employ batch of size 10 for model update. The learning rates for all algorithms set to 7 0.01 / /VT and T/e 1/VT. The server stores models every 20 time steps. The metric to evaluate the performance of algorithms is the accuracy. The accuracy for client i is defined as Accuracy, + Σ 18,.,... where Di,e denotes the label predicted by client at time step I.
Average accuracy and its standard deviation across clients for CIFAR-10 and FMNIST are reported in Table At each time step t, clients can store 10 model parameters. Therefore, M for Fed-POE, The results
Table 2: Average accuracy and its standard deviation across clients for image classification.
Methods
CIFAR-10
FMNIST
is set to M indicate that the performance of Local relative to federated learning baselines depends on the dataset. While Local outperforms all federated baselines except for Fed-Rep and Fed-POE on FMNIST, it obtains the worst accuracy on C1. Conversely, Fed-POF achieves the highest accuracy for both datasets, indicating
Local
50.35% + 10.11%
78.81% 2.12%
Ditto
56.87% = 9.06%
78.73% + 1.89%
Fed-Rep
63.86% = 7.97%
79.04% 1.77%
Fed-OMD
65.09% = 7.39%
74.60% + 66.52%
Fed-ALA Fed-DS
61.48% = 8.88%
75.13% + 6.39%
64.26% = 7.03%
75.62% + 6.58%
Fed-POE
66.54% 1 8.07%
79.23%
+ 1.88%
9

---

Table 3: Average accuracy and standard deviation across clients employing Fed-POE for image classification on CIFAR-10 with varying values of M and batch size b.
M = 0
M 4
=
M = 16
b = 1
53.80% + 6.71%
62.73% + 8 8.29%
62.73% + 8.29%
62.73% + 8.26%
b 10
65.55% = 8.77%
66.50% = 8.00%
66.54% - 8.08%
66.46% = 7.98%
b = 20
65.72%
= 8.662%
66.13% = 8.20%
66.64% = 7.94%
66.53% = 8.00%
b = 30
66.83% = 8.54%
66.32% = 7.92%
66.24% = 8.05%
66.39% = 8.02%
that Fed-POE efficiently leverages the advantages of both federated and local models. To analyze the effect of the number of models M and batch size b on the Fed-POE performance, experiments are conducted on the CIFAR-I0 dataset, varying the batch size b and the number of models M selected by each client to construct the ensemble model. observed Table 3 the batch size b = results in the worst accuracy. mainly due to the forgetting process where models overfit to the most recently observed data. However, increasing the batch size from b or b 20 to b 30 does not significantly improve the accuracy. Larger batch sizes may lead the model to perform better older data, as the model is trained on older data over more iterations. Therefore, this study concludes that moderate batch size is optimal, considering that increasing the batch size also increases computational complexity. Table 4 in Appendix [ presents the accuracy of POE on both the CIFAR- 10 and FMNIST datasets with varying values of M. The table shows that employing the saved models by the server in WD improves performance, as setting M = 0 results in the worst accuracy. Moreover, can be observed that increasing M does not necessarily lead to further accuracy improvement. This aligns with the intuition behind selecting subset of models rather than using all models.
6 Conclusions
This paper proposed Fed-POE, personalized federated learning algorithm designed for online prediction and model fine-tuning. Fed-POE constructs an ensemble using local models and federated models stored by the server periodically over time. Theoretical analysis demonstrated that Fed-POR achieves sublinear regret. Experimental results revealed that the relative performance of local models compared federated models depends on the dataset, making the decision between local model training and federated learning challenging. However, experimental results also show that Fed-POE consistently outperforms both local and federated models across all datasets. This indicates that Fed-POE effectively leverages the advantages of both local and federated models.
Acknowledgement
Work in this paper is supported by NSF ECCS 2207457 and NSF ECCS 2412484.
References
[1] Durmus Alp Emre Acar, Yue Zhao, Ruizhao Zhu, Ramon Matas, Matthew Mattina, Paul Whatmough, and Venkatesh Saligrama. Debiasing model updates for improving personalized federated training. In Proceedings of International Conference on Machine Learning, volume 139, pages 21-31, Jul 2021.
[2] Noga Alon, Nicolò Cesa- BisscBianchi, Claudio Gentile, Shie Mannor, Yishay Mansour, and Ohad Shamir. Nonstochastic multi-armed bandits with graph-structured feedback. SIAM Journal on Computing, 46(6)) 2017.
[3] Peter Auer, Nicolò Cesa-Bianchi Yoav Freund, and Robert E. Schapire. The nonstochastic multiarmed bandit problem. SIAM Journal on Computing, 32(1):48-77) Jan 2003.
[4] Nicolo Cesa-Bianchi and Gabor Lugosi. Prediction, Learning, and Games. Cambridge University Press, USA, 2006.
10

---

[5] Zachary Charles, Zachary Garrett, Zhouyuan Huo, Sergei Shmulyian, and Virginia Smith. On large-cohort training for federated learning, In Advances in Neural Information Processing Systems, 2021.
[6] Huili Chen, Jie Ding, Eric W Tramel, Shuang Wu, Anit Kumar Sahu, Salman Avestimehr, and Tao Zhang. Self-aware personalized federated learning. Advances in Neural Information Processing Systems, 35:20675-20666 2022.
[7] Yujing Chen, Yue Ning, Martin Slawski, and Huzefa Rangwala. Asynchronous online federated learning for edge devices with non-iid data. In IEEE International Conference on Big Data (Big Data), pages 15-24, Dec 2020.
[8] Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. Exploiting shared rep- resentations for personalized federated learning. In Proceedings of the International Conference on Machine Learning, volume 139, pages 2089-2099, Jul 2021.
[9] Georgios Damaskinos, Rachid Guerraoui, Anne-Marie Kermarrec, Vlad Nitu, Rhicheek Patra, and Francois Taiani. Fleet: Online federated learning via staleness awareness and performance prediction. ACM Transactions on Intelligent Systems and Technology, 13(5), Sep 2022.
[10] Yuyang Deng, Mohammad Mahdi Kamani, and Mehrdad Mahdavi. Adaptive personalized federated learning. arXiv preprint / 13461, 2020.
[11] Canh T. Dinh, Nguyen H. Tran, and Tuan Dung Nguyen. Personalized federated learning with moreau envelopes. In Proceedings of International Conference on Neural Information Processing Systems, page 11394-21405, Dee 2020.
[12] Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. In Advances in Neural Information Processing Systems, volume 33, pages 3557-3568, Dec 2020.
[13] Alan Fern and Robert Givan. Online ensemble learning: An empirical study. Machine Learning. 53:71-109, 2003.
[14] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adap- tation of deep networks. In Proceedings of International Conference on Machine Learning, volume 70, pages 1126-1135 Aug 2017.
[15] Dylan Foster, Satyen Kale, Mehryar Mohri, and Karthik Sridharan. Parameter-free online learning via model selection. In Proceedings of International Conference Neural Information Processing Systems, 2017.
[16] Bhargav Ganguly and Vancet Aggarwal. Online federated learning via non-stationary detection and adaptation amidst concept drift. arXiv preprint arXiv:2211.12555 2022.
[17] Francois Gauthier, Vinay Chakravarthi Gogineni, Stefan Werner, Yih-Fang Huang, and Anthony Kuh. Resource-award aware asynchronous online federated learning for nonlincar regression. In IEEE International Conference on Communications, pages 2828-2833, 2022.
[18] Pouya M Ghari and Yanning Shen. Online multi-kernel learning with graph-structured feedback. In Proceedings of the International Conference on Machine Learning, volume 119, pages 3483,,,83, Jul 2020.
[19] Pouya M. Ghari and Yanning Shen. Personalized online federated learning with multiple kernels. In Advances in Neural Information Processing Systems, 2022.
[20] Pouya M. Ghari and Yanning Shen. Graph-assisted communication-effiiieee ensemble federated learning. In European Signal Processing Conference (EUSIPCO), pages 737-741, 2022.
[21] Pouya M. Ghari and Yanning Shen. Budgeted online model selection and fine-tuning via federated learning. Transactions on Machine Learning Research, 2024.
[22] Pouya M. Ghari and Yanning Shen. Online learning with uncertain feedback graphs. IEEE Transactions on Neural Networks and Learning Systems, 35(7):963699650 2024.

---

[23] Vinay Chakravarthi Gogineni, Stefan Werner, Yih-Fang Huang, and Anthony Kuh. Communication-effictne online federated learning strategies for kernel regression. IEEE Internet of Things Journal, 10(5):4531-455 2023.
[24] Filip Hanzely, Slavomír Hanzely, Samuel Horváth, and Peter Richtárik. Lower bounds and opti- mal algorithms for personalized federated learning. In Proceedings of International Conference on Neural Information Processing Systems, page 2304-2315, Dec 2020.
[25] Steven C. H. Hoi, Rong Jin, Peilin Zhao, and Tianbao Yang. Online multiple kernel classification. Machine Learning, 90:289-316, Feb 2013.
[26] Songnam Hong and Jeongmin Chae. Communication-efficcenn randomized algorithm for multi-kernel online federated learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):9872-9888 2022.
[27] Markelle Kelly, Rachel Longjohn, and Kolby Nottingham. UCI machine learning repository, 2023.
[28] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
[29] Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Chal- lenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3):50-60, 2020.
[30] Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust federated learning through personalization. In Proceedings of the International Conference on Machine Learning, volume 139, pages 6368, Jul 2021.
[31] Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang. and Zhihua Zhang. On the convergence of fedavg on non-iid data. In International Conference Learning Representations, 2020.
[32] Xin-Chun Li, De-Chuan Zhan, Yunfeng Shao, Bingshuai Li, and Shaoming Song. Fedphp: Federated personalization with inherited private models. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 587-602, 2021.
[33] Ken Liu, Shengyuan Hu, Steven Wu, and Virginia Smith. On privacy and personalization in cross-silo federated learning. In Advances in Neural Information Processing Systems, 2022.
[34] Othmane Marfoq, Giovanni Neglia, Aurélien Bellet, Laetitia Kameni, and Richard Vidal. Federated multi-task learning under a mixture of distributions. Proceedings of International Conference Neural Information Processing Systems, volume 34, pages 15434-15447, Dec 2021.
[35] Othmane Marfoq, Giovanni Neglia, Laetitia Kameni, and Richard Vidal. Federated learning for data streams. In Proceedings of The International Conference on Artificial Intelligence and Statistics, volume 206, pages 8889-8924, Apr 2023.
[36] Cobbinah B. Mawuli, Jay Kumar, Ebenezer Nanor, Shangxuan Fu, Liangxu Pan, Qinli Yang, Wei Zhang, and Junming Shao. Semi-supervised federated learning on evolving data streams. Information Sciences, 643:1 143:11 2023.
[37] Aritra Mitra, Hamed Hassani, and George J. Pappas. Online federated learning. In IEEE Conference on Decision and Control (CDC), pages 4083-4090, Dec 2021.
[38] Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh. Agnostic federated learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the International Conference on Machine Learning, volume 97, pages
Jun 2019.
[39] Vidya Muthukumar, Mitas Ray, Anant Sahai, and Peter Bartlett. Best of many worlds: Robust model selection for online supervised learning. In Proceedings of the International Conference on Artificial Intelligence and Statistics, volume 89, pages 3177-3186, Apr 2019.
[40] Mehdi Neshat, Bradley Alexander, Markus Wagner, and Yuanzhong Xia. A detailed comparison of heta-heuristic methods for optimising wave energy converter placements. In Proceedings of the Genetic and Evolutionary Computation Conference, page 1318-1325, Jul 2018
12

---

[41] Aldo Pacchiano, Christoph Dann, and Claudio Gentile. Best of both worlds model selection. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
[42] Jonghwan Park, Dohyeok Kwon, and Songnam Hong. Ofedqit: Communication online federated learning via quantization and intermittent transmission. arXiv preprint arXiv:2205. 06491, 2022.
[43] Kumar Kshitij Patel, Lingxiao Wang, Aadirupa Saha, and Nathan Srebro. Federated online and bandit convex optimization. In Proceedings of the International Conference on Machine Learning, volume 202, pages
Jul 2023.
[44] Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In Proceed- ings of International Conference on Neural Information Processing Systems, pages 1177-1184,
Dec 2007.
[45] Vinay Venkatesh Ramasesh, Aitor Lewkowyzz, and Ethan Dyer. Effect of scale on catastrophic forgetting in neural networks. In International Conference on Learning Representations, 2022.
[46] Mohammad Reza Karimi, Nezihe Merve Gürel, Bojan Karlaš, Johannes Rausch, Ce Zhang, and Andreas Krause. Online active model selection for pre-trained classifiers. In Proceedings of The International Conference on Artificial Intelligence and Statistics, volume 130, pages 307-315, Apr 2021.
[47] Daniel Rothchild, Ashwinee Panda, Enayat Ullah, Nikita Ivkin, Ion Stoica, Vladimir Braverman, Joseph Gonzalez, and Raman Arora. FetchSGD: Communication-effiiise federated learning with sketching. In Proceedings of the International Conference on Machine Learning, volume 119, pages 8253-8265, Jul 2020.
[48] Doyen Sahoo, Steven C.H. Hoi, and Bin Li. Online multiple kernel regression. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, page 293-302, 2014.
[49] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference Learning Representations, May 2015. URL
http:://aaxvv.orr//ee 1409. 1556
[50] Yue Tan, Guodong Long. Jie Ma, Lu Liu, Tianyi Zhou, and Jing Jiang. Federated learning from pre-trained models: A contrastive learning approach. Advances in neural information processing systems, 15:322211 2022.
[51] Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J Gordon. An empirical study of example forgetting during deep neural network learning. In International Conference on Learning Representations, 2019.
[52] Grace Wahba. Spline Models for Observational Data. Society for Industrial and Applied Mathematics, 1990.
[53] Yujia Wang, Lu Lin, and Jinghui Chen. Communication-effiiin adaptive federated learning. In Proceedings of the International Conference on Machine Learning, volume 162, pages Jul 2022.
[54] Yue Wu, Shuaicheng Zhang, Wenchao Yu, Yanchi Liu, Quanquan Gu, Dawei Zhou, Haifeng Chen, and Wei Cheng. Personalized federated learning under mixture of distributions. In International Conference on Machine Learning, pages 37860-37879, 2023.
[55] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for
benchmarking machine learning algorithms. arXiv preprint arXiv: I 7708.07747 2017.
[56] Rui Ye, Zhenyang Ni, Fangzhao Wu, Siheng Chen, and Yanfeng Wang. Personalized federated learning with inferred collaboration graphs. International Conference on Machine Learning, pages 39801-39817, 2023.
13

---

[57] Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Fedala: Adaptive local aggregation for personalized federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11237-11244, 2023.
[58] Shuyi Zhang, Bin Guo, Anlan Dong, Jing He, Ziping Xu, and Song Xi Chen. Cautionary tales on qir-quality improvement in beijing. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 473(2205):2017045,, 2017.
14

---

A Proof of Theorem 1
This section provides the proof of Theorem 1 The proof follows similar steps those in (37. and it included here for the sake of completeness and to make the paper self-contained.
According to and for any 0 it can be written that
0 011
111+1 0
0 N
Vf(f(x v.,)) -0
va.1)
N
=110:
011 +
VL(f(a,, 01), vs.e)
2n
Σ
0, ( : (.).)
(21)
Due to convexity of C(.,.) :) it can be concluded that
2m
21
L(f(ar, 0.). (i. -
(0.
()) (22)
N
Combining
21
with
we get
N
N
2n
2n
((f(x,,,;))) (1)))
N
N
1m1
< 0 ||0
01
+
n
Vf(f( yr,e)
(23)
N
Using assumption A2 and Arithmetic Mean Geometric Mean (AM-GM) inequality can be written that
N
N (V( s N²G².
VL(f(xr,
S
(24)
Combining
(33
with
24
we arrive at
27
2n N
[(f(xr (f) v().
Σ
((f(ax;;)) S ||0, -01 -1|--10047 - 0 G²
(25)
Dividing both sides by 2x yields
N
N(110: 0||2 -1 1 n
((f 0))
G².
2n
2
(26)
Summing 226
over time, we obtain
T N
T N ();,;;;))
C(((E 0.).
Σ
Am 2m
1m1
1m1
N O
2
|
0
NN GºT.
(27)
2m
Plugging in 0 0" and 0 into
(7)
and considering the fact that ||0
0 2 we find
T N
N1100112 n
ΣΣ ((f v..)
C(f ()) 0)), =
G2T
(28)
2m
2
which proves the Theorem.
15

---

B Proof of Theorem 2
According to 10P. we can write
0,141 + Bset1
04,11
exp (-1oo( (1s, 2i)))
05g + Pige
Oiit + St,
exp (f o1,e), v(a))
(29)
Oi,t + Big
Using the inequality e e= - x +
Vx 2 0, from
29 we arrive at
0,,++1 Bret1
ai,t
(1 [1-ro[([(a::) 0.), v1.)) +
E L² (f(z,, 0.),,, 02).
0iit + Bi,t
Os,t + Big
2
Big (1 (1-1af(((m (1,), v1.) + - c² ( (..))) (30)
0,i,t + Bi,t
2
Taking the logarithm from both sides of (30) and using the inequality 1 + x S ex, we get
In ai1+1 + 3,,411
S Osit
(-oofffa 0.). + ne _ (f 01) whe))
04,1 + Sa,t ais
+ Bi,
Big
(f (...). _ c (f oi,) mae)) (31)
+
011
Considering the assumption that 0 ((f(x; 0), S Vz, 0, 31 can be relaxed to
In( 04,111 + B1,e41
Citt (- 0,), wt.))
Og Br,
Oi Be,
Bie
(-noo(f(a (..), M..)) + Ne
+
(32)
05it + Ba,t
2
Summing (32 over time, obtain
In( 01,711 + B,.T11
ai,t
%o Σ (()))))))
01,1 + Bi,1
aitt + Big
Bigt
T2 + 2
Σ
(33)
Obi,t + Big
According Hölder's inequality, for any positive real numbers p and q satisfying following inequality holds:
1, the
0.T+1
BLT+I
20.
(34)
P q
To meet the condition 1 + 1, it necessary that p and q 1. Consequently, can be modified to:
0,, + B,.41 a
(35)
Considering the fact that 04,1 = Ai,1 = 1, based on 135 we can write
In( 04,4+1 + (1,T+1
2 In(o + 1n(3,,41)- - -ln())
(36)
0, + Br,1
q
According to the update rule in
29
36 is equivalent to
T
T
In 04,7+1 + BB.T-11 2
C(f 0.), Vie)
Be
Σ
C(f(ase) va.)) In(2). (37)
01,1 + +BB
q tm1
t=
16

---

Combining 33
with 37
we arrive at
T
+
Ocit + Bi,
+ Bi,
t=1
1 C(i(as vi.))
In(2)
neT
(ai Me))
+
(38)
9
1/c
2
Due to the convexity of L(,,),, we can write
[(fcs((s.))) =
as,t
s(xi))))) 0c) +
0ii,t + Big
ait + Big
T
Bea T
041
Σ
<
ai, Bi,t t=1
ai, + Bi, t=1
(39)
Combining 38
with
39
get
(.).
Σ
9 1=1
(2) neT
(40)
1/e 2
Substituting p oo and q = 1 in (40). we obtain
In(2) neT
Σ ((r v1.)) E (ul)a
(41)
Since Fed-POE updates Φh. locally using online gradient descent as outlined in ( for any can write
according to
6
| 2m 2
Σ Clf(arioo.))..)
I
7
GTT.
(42)
Substituting of along with combining
with 42
we can conclude that
T Σ C(fre(aan) Σ ()).) mt.))
In(2)
+ G
neT
(43)
2m
Te
2
which proves the personalized regret upper bound of Fed-POE in 12 Furthermore, plunging in = 1 and q = xo into (40), we get
T Σ Clf.,(E.)..... v.d) > c(f(z, 0.).
T
1n(2) +
noT 2
vi.e) -
(44)
tml
Summing 44 over all clients, obtain
NIn
N.
C(f(o. v.).
(45)
1/c
2
Combining 45 with
28
we arrive at
T ΣΣ
((s 0*), Vc.)
1=1
1-11
N (n())
1/NN
G2T + No
+
(46)
2n
nec
2
2
which proves the Theorem.
17

---

C
Proof of Theorem 3
According to assumption A3 that S (2; 0), S 1, it can be written that (ffa( vc.))
U N
U
N
1
C(h (2.,,), S U.
(47)
N
N
Let lig,e denote the importance sampling loss estimate, which is expressed as
listt (( Vs.)) 1jEM,..
(48)
4ij.A
Let the total number of model parameters stored by the server after time step U is D and Wi, E Wiitt For any t > U, considering [7). we can write
Wh.+1 W
Σ
Wiitt exp( (pp(-nelij.) = Wi
Σ
Pija exp(-nelij.))
(49)
1=1
3=1
Employing the inequality
S 1 - x + f , Vr 2 0, from 449
we obtain
W
n (.))
S Σ Pisa(1 nolysa
(50)
W..
Taking the logarithm from both sides of (50) and using the inequality 1 1 x S e",, we arrive at
Wis+1 In
Pij.e (00(-nnnee noliz + T 2 C.y)
< >
(51)
W
2
5=1
Summing (51) over time, get
W1T+1
D Σ 1=U 1=1
+ ne (.ge).
In
Σ
(52)
W.
2
Considering the fact that the weights {wij,t] are initialized as way,1 1, Vj [D]. it can concluded that Way S D. Theefore, for any [D], the left hand side of (52) is bounded from below as
WiTH1 In
In 2 WiJTT1
WigT++
Σ
noloss D.
(53)
W.
Wi
Combining (53) with 52 yields
T
Pija
Σ
life
In D
Te
ΣΣ
D
Pipal
(54)
T]e
t=
1=
:=U j=1
The expected values of lijee and Ci given observed losses up until time step be obtained
C(f(x, sh.e)
E, [[j.M..] = [(f(xce))))) Vi.s)
(55a)
Digut
((f(x) Ps) Mr.e)²
E. [[)e,,,)) C(f(xi P;), wi.e)
E, [12
(55b)
4ij.t
Tij.t
Taking the expectation from
54 arrive
Σ Σ
D
Pij, (( (.).
E
InD
πe
Σ
D
Pija
(56)
nc
q:j.t
1=1
t=
j=1
18

---

Since 4ij,1 =1- -1-Pis,)) Pij,o(1 + Pij.a) + (1 Pis,,) t... + (1-Pij.) - be concluded that 4ij,t 2 Pij.t- Therefore, (56) can be relaxed to
M-1), it can
Σ ((( In Te D(T U). (57) t=
Tc
According model selection procedure adopted by Fed-POE presented in Algorithm client i chooses a subset of models by sampling them in M rounds with replacement. Let dij,t 2 0 denote the number of times that the model in Di, is chosen by client i at time step f. The number of different situations for selected subset models M4, is equal to the number of solutions for the linear equation
011, + a,D.t = M, "iy. 2 O, Vj E [D].
(58)
Let A denote the set of all possible solutions for (58) such that if a E A. where [a
@D)
d1....;a0 ap satisfies 588 Therefore, for the expected loss of the ensemble (a, in 14
write
|&| D
(.(C(,i,((a) w.
EI
Oakk Paja C(S (Ia1)),
((M)
(59)
k=
where ft. (x);.) denote the k-th possible ensemble model generated by client i using Fed-POE. Using the Jensen inequality and convexity of the loss function, we can relax (59) to
IAI
D
Eg (.
ask
Pojit
k=1
[A]
D
II
asa Poja
Σ mEM
Wim,t ((k 5,8
(i)
(60)
k=1
(-(k) where M/ and w.t the k-th possible model subset and weight summations, respectively. Rearranging the right hand side of (60). we can write
D
D Brack Pim,t
C.I.(faaa.... S
Σ P
Σ
π m==1
Wist [(((xt,)))) 9t.))
(61)
w
3=1
A=1
where iB, is the set of possible solutions for the linear equation in 58 Since for any k. we have Wajet wi(k)
condition on aija 2 1.
61
be relaxed to
BBI
D
(F (2..) v.li <
Pija
II
bm.kk Pim,t
(62)
1=1
Since
DiEy
(I
bm Pim.t
includes
all possibilities in By,
conclude that
Σ
k
R2
D
bmak Pimit)
1. Combining this with
62
obtain
(.( va.) <
Σ Pija ((S vee)-
(63)
Combining 63 with 57 arrive
E, (().))))))) v..)) - Σ (cf(muss))))vuu))) In + te D(T - U). 2
Σ
(64)
1=
t=
Since 0 < ((f(x;)),))) < can be written that
V
E, B.[C(fea(R.) c(((p.))))) $ U.
(65)
19

---

Combining 65 with 64
we get
T
S In D -
E. (.I(( ya.)l - Σ P))
U) +
(66)
de
Since (-) similar to Si.e()) is the ensemble of two models, following the same derivation steps from 29) 441 by substituting ft.(( (a.(())))) and fa,(( with (x,,() /f ( 02) and f ( rir,), respectively, we can conclude that
In(2)
neT
(aL)) (Uas(aa)) S
+
(67a)
1/c
2
In(2) noT
Σ ((Fas(ao)))
Σ C(Fa,()))) m..))
(67b)
T) 2
Taking the expectation from both sides of (67b) with respect to randomization in model selection along with combining (67b with (66) yields
T
E, (CCC(Fa(
Σ (().
2D
Le (D+ (D+
Ne D)U.
(68)
Furthermore, combining selection randomization, obtain
677) with
41 and
44 and taking the expectation with respect to model
In(4) - neT,
E, E.(C(f..((au)....)
- (rt),
(69a)
t7c
In(4) n.T.
E, (.
Σ (ff(a,,;:0)),00 =
(69b)
17c
Recall that (.) associated with the j-th model parameter in D, be defined as (a,,)) s( P,) while hge0 and h.aa(-) correspond to the local and federated models, respectively, defined as hoce (20, f(x,, 1),)) and les (x.,,)
f 0.). Also recall that H := {h, Vi 1 s
j S [rr U { (hwer h Comparing the right hand sides of (69a) and (69b) with that of (68) and write
considering the fact that D 2 2, for any HH
T EeIC(f.(EE.))
Σ ((N(Z.)).
In2D
(D 1JT+1)T++
Te 2
DYU. (70)
By substituting 1(-)) with (-) as defined in 18b) and considering the fact that D < U/n, we obtain the personalized regret upper bound for client as shown in (20). Moreover, taking the average of 70 across clients and substituting h(-) with h" h-),,, arrive at
E, (IC((..((EW))....))
ΣΣΣ (a.,)) v1.))
In 2D
+
The 2
(D 1)+ D)U
(71)
which proves the theorem.
D Supplementary Experimental Details
This section presents supplementary experimental results and details about experimental setup. All experiments were carried out using Intel(R) Core(TM) 17-10510U CPU @ 1.80 GHz 2.30 GHz processor with 64-bit Windows operating system.
20

---

D.1 Regression Data Distribution
As it is pointed out in section 5] the present paper tests the performance of algorithms for online regression task on Air and WEC datasets. These datasets are downloaded from UCI Machine Learning Repository [27]. Each data sample in Air dataset includes air quality information such as concentration of some chemicals in the air. Data samples in Air dataset collected from four different geographical locations. Moreover, data samples in WFC, collected from wave energy converters in four different geographical locations. In order to distribute data, clients are partitioned into 4 groups. For each group, 70% of data samples observed by each client in the group belongs to specific geographical location while 10% of observed data samples belong to each of the rest 3 locations.
D.2 Random Feature Kernel-based Models
As it is pointed in section the proposed Fed-POE and all baselines utilize random feature kernel- based model to perform online regression task. In what follows we explain random feature-based kernel models. Let ii::-) be a positive-definite function called kernel such that s(xr, x²) measures the similarity between æ and x'. In online kernel learning context, at time step + 1, the following
prediction is made for x (see e.g. [52 25 48):
E
N
fa at)
04,7 (x, Di,r)
(72)
where a, = [a1,1.... CNN.,........................... 01.11 [NN.I] denotes the learnable parameters. Therefore, the number of parameters that should be learned grows with time. In order to alleviate the computational complexity of online kernel learning, random feature approximation 44 can be employed. In fact, using random feature approximation, the number of parameters that needs be learned is time- invariant and is selected by the algorithm. Assume that x(-) is a shift-invariant kernel function such that x(æ, x²) = <(x x²). Also, suppose that <(-) is scaled such that x(0) 1. Let E(-) denotes the Fourier transform of x(-). According to definition of inverse Fourier transform <(0)= = - ((w)dw 1. Therefore, it can be concluded that E(-) a probability density function (PDF). Let Wi...., wD be drawn randomly from E(-) and called random features. Using the random
features wi...., WD. the representation 2xx) is defined
z (xx) = VD
[ ,in...,,in(([]]]) cos(w] [o)
(73)
Given random features W1,, wD and using the proposed Fed-POE, at time step t, client i makes prediction f ( 0,) = 0 z 2(x,,)). Clients and the server employ the proposed Fed-POE to learn the parameter 9,. As it can be inferred since the model f(: is linear with respect to model parameter 0,, using convex loss functions, the loss (x 0,), vt,e) is convex as well.
Furthermore, in section 5] the proposed Fed-POF utilizes three Gaussian kernels for online regression on Air and WEC datasets. In order to implement multi-kernel learning for Fed-POF, the prediction of kernels are linearly combined and the weights for linear combination is learned locally by each client. Let fo. (2t..), (xx..) and fio( represent predictions of Gaussian kernels with variances of 0.1, 1 and 10, respectively. Then time step t, client i makes prediction wo. fo (zx,,) W1.:t f1 (x.,,) + w1o.a. fno((;,)) In order to update weights Wo.1,at W1.at and W10,st. client employs multiplicative update rule. As an example after observing the loss ((f1(x,)) vi.s), client updates Wiat as (1)++11 = W1. exp(-r;[(fi (z Vs,e)) where is a learning rate specified by client i.
D.3 Image Classification Experimental Setup
The pre-trained CNN used by Fed-POE and other baselines is biased toward class label 0. For CIFAR-IO, the pre-trained CNN is trained on a subset of the CIFAR-10 training data, consisting of 5, 000 samples with label 0 and 500 samples from each of the other 9 class labels. For FMNIST, the model is trained on a subset of the FMNIST training data, consisting of 6, 000 samples with label 0 and 500 samples from each of the other 9 class labels. The CNN models
trained using Tensorflow 1. We used the SGD optimizer with a learning rate of 10-3 3 and momentum of 0.9. The models for CIFAR- 10 and FMNIST were trained for 100 epochs and 10 epochs, respectively.
Clients receive test data samples sequentially and make prediction for the newly received sample. To distribute test data samples of CIFAR-10 among clients, we split clients into 10 groups For
21

---

Table 4: Average accuracy and standard deviation across clients employing Fed-POE for image classification with varying values of M
Datasets CIFAR-IO 10 FMNIST
M = 0
M 4
M = 8 66.54%
M = 16
65.55% 8.77%
66.50% 8.00%
8.08%
66.46% + 7.98% 79.18% + 1.85%
79.03% = 1.76%
79.12% + 1.87%
79.23% + 1.88%
500
Local
Average Cumulative Pgret
17.5
local
15.0
400
Fed Rop
Fed Rop
12.5
300
Fed ALA
10.0
elt KDFF
200
Fed-POE
fed-Pot
5.0
100
0
100
200
300
400
500
0
50
100
150
200
250
Time
(a) CIFAR-10
(b) WEC
Figure 1: Cumulative regret over time on CIFAR-10 and WEC datasets.
CIFAR- 10, 55% of samples observed by a client belongs to a specific class label while only 5% of received samples belong to each of other class labels. For FMNIST, client data distribution is time-variant. Since the number of test samples is 10, 000 and the number of clients is 20, it can be concluded that time horizon T is 500. In the first half of time steps (i.e. / S 250), each client observes 200 samples from the first 5 class labels and 50 samples from other class labels. In the second half, this is reversed: clients observe 200 samples from the last 5 class labels and 50 samples from the other class labels. In each half, each client is biased toward one of the five majority classes. For example, if a client biased toward class 0 in the first half, it observes 100 samples from class label 0, 25 samples from each of class labels 1 to 4, and 10 samples from each of class labels 5 to 9. Therefore, in each half, each client observes 100 samples from one class, 25 samples from each of four other classes, and 10 samples from each of the remaining five classes
To implement Fed-OMD for both regression and image classification, used the fg-norm regularizer function for mirror descent. For implementing Ditto for both regression and image classification, we the regularization factor À to 1. In the case of image classification using Fed-Rep, clients locally fine-tune the last two layers of the CNN model, while the rest of the network used as the global backbone to generate representations. Furthermore. Fed-POE is compatible with any federated learning method and can utilize any federated algorithm. For CIFAR-10, Fed-POE uses Fed-OMD, while for FMNIST, Fed-POE uses Fed-Rep. fine-tune the CNN model, Fed-POE and all baselines use the SGD optimizer and the cross-entropy loss function.
D.4 Supplementary Results
Table presents the accuracy of clients for image classification using Fed-POE with varying values of M. As can be seen for both CIFAR-10 and FMNIST, when M > 0, the accuracy is higher than in the case where M 0. The case where M 0 corresponds clients not using models stored by the server in their ensemble. Therefore, these results show that constructing the ensemble using previous federated model parameters stored by the server improves the accuracy of Fed-POE. This indicates the effectiveness of the model selection procedure of Fed-POE presented in Algorithm[] []] Additionally, these results show that increasing M does not necessarily lead to further accuracy improvement. This implies the effectiveness of Fed-POE's model selection in pruning model parameters from the ensemble that have relatively lower accuracy. Figure Tigureeillustraan illustraates the average cumulative global regret of clients over time using Fed-POE and all other baselines. As depicted, Fed-POF. achieves sublinear regret for both CIFAR-10 and WEC datascts. This corroborates the theoretical results in Theorems 2 and 3
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction reflect that the contribution of this paper is the proposal of a novel personalized federated learning algorithm for online prediction and model fine-tuning.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In Section 4 the paper makes it clear that Fed-POR utilizes multiple models instead of a single model for prediction. This requires more computations than the case where clients use only one model. Computational efficiency of Fed-POE is discussed in Section 4
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
23

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: The paper provides the full set of assumptions in Section and the complete proofs of all theorems can be found in Appendices A B and
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: In Section and Appendix D we provide detailed information needed to reproduce the results.
Guidelines:
The answer NA means that the paper does not include experiments.
the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
24

---

reproducible, they 5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
We provide open access the data and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
public/guides
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
7/nips cc/ wtc/publicc/ htppllic//iiesssee gww.///ieee//eee// Codeeobmiiiie /esd/gueuude//oeuubbiiiiii SanSSubbissss iebbbbissssioi ssisnnoiccyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: In Section 5 and Appendix D we provide all the training and test details necessary understand the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: In Tables ! and 4
We reported the standard deviation of the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
25

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative еттоr rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: In Appendix D we clarify that all experiments were carried out using Intel(R) Core(TM) 17-10510U CPU 1.80 GHz 2.30 GHz processor with a 64-bit Windows
operating system.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't Itdn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htttpp::/eeurrpp..ccpuupbbii//// iet.eiiiceeeiiccccciieeeeieeee
Answer: [Yes]
Justification: We believe the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper studies the problem of personalized federated learning. While personalized federated learning, in general, has societal impact, we do not foresee any specific societal impact resulting from our work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
26

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We believe that the paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
In Section 5 and Appendix D we properly cited the original owners of assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URI.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
27

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
28

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "gVs2y7BKsz", "reviewer": "Reviewer_Lvm5", "summary": "This paper introduces Fed-POE, a novel personalized federated learning algorithm tailored for online prediction and model fine-tuning. Fed-POE creates an ensemble by integrating local models with those periodically contributed by the server over time. Theoretical analysis confirms that Fed-POE attains sublinear regret. Empirical results demonstrate that Fed-POE consistently surpasses the performance of both local and federated models across all evaluated datasets, which indicates that Fed-POE effectively leverages the advantages of both local and federated models.", "questions": "See weaknesses.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The technical content of the paper appears to be accurate, although I did not check all the details carefully.\n- This paper is generally well-written and structured clearly.\n- The experiments substantiate the main theoretical analysis, and the proposed algorithm demonstrates superior performance over the baseline methods", "weaknesses": "My primary concern is that the assertion the proposed algorithm can effectively harness the combined advantages of federated and local models is not clearly demonstrated within the theoretical bounds. The paper presents two principal theoretical results: Theorem 2 provides the regret upper bound for the proposed algorithm in convex scenarios, while Theorem 3 addresses non-convex cases. Both theorems establish sublinear regret bounds that are consistent with those for federated learning using a straightforward online gradient descent approach.  I recommend enhancing the clarity of the proposed method's advantages in the theorems by incorporating assumptions about the data distributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for taking the time to review our paper and providing your valuable comments. Please find below our response to your review.\n\nThe main advantage of the proposed Fed-POE compared to the straightforward online gradient descent approach is its ability to provide sublinear regret upper bounds for both global and personalized regret. The conventional method, as presented in Theorem 1, cannot guarantee a sublinear regret upper bound for personalized regret. The conventional online gradient descent approach only guarantees global regret. Fed-POE achieves sublinear regret upper bounds for personalized regret, as shown in Equations (12) and (20). Moreover, global regret upper bounds for Fed-POE are presented in Equations (11) and (19).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X6ubCodu8V", "reviewer": "Reviewer_NAbk", "summary": "The paper introduces an interesting perspective about the role of ensembles of models in federated learning. The provocative claim is the fact that federated learning is not always better than locally-trained models. This is contextualized in the field of not IID data and time-varying data generating processes. To address this issue the paper introduces from the theoretical point of view how to quantify the regret in federated and locally trained models. In addition it includes an analysis of non-convex models by managing an history of models. The overall impression about the paper is positive even if some points could have been better explored (in particular the part related to not IID data that is somehow the core of the paper).", "questions": "See Weaknesses box", "limitations": "See Weaknesses box", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper introduces a theoretical evaluation of the gain produced by federated models w.r.t. locally trained models. This results show that federated learning is relevant only when models can be considered iid (hence averaging providing better results). This is somehow a know results but I appreciated the theoretical analysis\n- The proposed solution is to combine with a convex mean a locally-trained models with the federated models\n- This is further extended in case of non-convex models by considering an \"history\" of models to be used when needed (i.e., according to the loss)", "weaknesses": "- The federated models somehow includes the locally-trained model. I would have appreciated a further analysis about the fact that the two \"sides\" of the average model are related each other. \n- The setting in which eta and eta_c scales with T prevents adaptation in the long run (which is somehow the core of the paper). How to deal with that?\n- Federated learning typically takes also into account the complexity of the learning phase (i.e., the amount of info to be transmitted, e.g., the models). This is not quantified here. And this could be also a weak point in the fed-poe algorithm.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for taking the time to review our paper and providing your valuable comments. Please find below our responses to your comments and questions.\n\n## Relations between Federated Models and Local Models\nThe federated model can differ significantly from the local models, especially when the data distribution among clients is heterogeneous. The degree of data heterogeneity influences the similarity between the federated model and the local models. Although the federated model is trained on local models, if the gradient trajectories of other clients differ significantly from those of a particular client, it is expected that the federated model will be substantially different from the local model of that outlier client. We can add this to the paper.\n\n## Learning Rates in the Long Run\nIf the time horizon $T$ is unknown which may often be the case in the long run, the doubling trick technique (see e.g., [R1]) can be effectively used to set the learning rates $\\eta$ and $\\eta_c$ while maintaining theoretical guarantees. The doubling trick is a well-known technique in online learning that adaptively sets the learning rates without knowing the time horizon. We add a note about this to the paper.\n\n[R1] N. Alon, N. Cesa-Bianchi, C. Gentile, S. Mannor, Y. Mansour, and O. Shamir, “Nonstochastic multi-armed bandits with graph-structured feedback,” SIAM J. Comput., vol. 46, no. 6, pp. 1785–1826, 2017.\n\n## Complexity\nLet $C_F$ denote the number of computations required to fine-tune the model $f$, and let $C_I$ represent the number of computations required to make an inference with model $f$. Assume that the complexity of model selection in Algorithm 1 is negligible compared to fine-tuning and making inferences with model $f$. According to Algorithm 2, each client performs $2C_F + (M+2)C_I$ computations per time step. Therefore, the computational complexity of Fed-POE for each client is $\\mathcal{O}(C_F + MC_I)$. Typically, fine-tuning deep neural networks with backpropagation requires significantly more computations than making inferences with them. Thus, if the model $f$ is a deep neural network, $C_I$ is negligible compared to $C_F$. In this case, the computational complexity for each client using Fed-POE is $\\mathcal{O}(C_F)$, which is comparable to most state-of-the-art federated learning methods.\n\nMoreover, at each time step, each client sends one updated model to the server. Let $P_f$ denote the number of parameters in model $f$. Therefore, the amount of information that needs to be transmitted to the server is $\\mathcal{O}(P_f)$, which is the same as most state-of-the-art federated learning methods.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TnmL2ZwCZG", "reviewer": "Reviewer_extd", "summary": "This paper proposes a novel personalized federated learning algorithm, Fed-POE, which is designed for adaptive prediction and model fine-tuning in dynamic environments. It addresses the challenge of real-time predictions on streaming data by constructing a personalized model that combines a locally fine-tuned model with multiple federated models. Theoretical analysis and experiments on real datasets demonstrate its effectiveness in achieving sublinear regret bounds and improved online prediction accuracy.", "questions": "1. The method in this paper does not significantly improve accuracy and even has a larger standard deviation. Can you give me more reasons to support your methods？\n\n2. The method is designed with two parts to mitigate catastrophic forgetting (old data replay and integration of multiple old models), The complex model updating process is unreasonable for real-time prediction. Can you design more ablation experiments to analyze these two parts？", "limitations": "see the weaknesses and questions", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper proposes a unique ensemble method that dynamically combines local and federated models, which is a novel approach in the field of federated learning.\n\n2. It provides a solid theoretical analysis, demonstrating sublinear regret bounds for convex models.\n\n3. The paper is well organized.", "weaknesses": "1. Although the presented method is novel, it is simply a combination of the previous personalized federated learning approaches as well as ensemble learning and provides comparably little conceptual originality.  The contribution's main novelty seems to be that integrating results from prior models would be beneficial in mitigating catastrophic forgetting in online federated learning.\n\n2. Experimental results show that the improvement in the accuracy of Fed-POE compared to other methods is not significant, but ensemble learning inevitably increases the computational overhead increase. The paper needs to analyze whether this trade-off is reasonable.\n\n3. The paper needs more experiments to prove the effectiveness of the method, for example, for real-time predictions, the size of the old data replay is crucial, and the authors should design experiments to analyze the effect of batch size b on the experimental results. This paper also needs experiment results on the accuracy over the time step.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for taking the time to review our paper and providing your valuable comments. Please find below our responses to your comments and questions.\n\nWe would like to briefly review the main contributions of this paper, which we believe are of interest and utility to the community. We hope the respected reviewer will consider these contributions.\n\nSection 3 of the paper highlights the challenge of choosing between local models and federated models in online federated learning when there is no prior information about data distribution. In Section 4.1, the paper proposes an ensemble federated learning algorithm that effectively leverages both local and federated models to provide more reliable performance for clients. Moreover, Section 4.2 demonstrates how federated learning can be employed to address catastrophic forgetting in real-time decision making.\n\n## Performance Gain\nWe believe the performance of the proposed Fed-POE in Section 5 demonstrates its advantage in real-time prediction tasks. The main challenge in real-time prediction arises when there is no prior information about the streaming data, making it difficult to evaluate model performance before the task begins. As discussed in Section 3.2 of the paper, it is theoretically unclear whether federated or local models perform better under such conditions. The performance largely depends on the dataset.\n\nResults presented in Section 5 confirm that the relative performance of algorithms varies with the dataset. Tables 1 and 2 show that for the CIFAR-10 dataset, federated models outperform local models, while for the Air dataset, local models perform better than federated models. Furthermore, for WEC and FMNIST, both local models and personalized federated learning models outperform non-personalized federated models. This variability complicates the decision between local and federated models. However, the results in Tables 1 and 2 indicate that Fed-POE consistently outperforms all baselines, albeit marginally on some datasets. This suggests that Fed-POE's performance is robust across different datasets, making it a reliable choice for real-time prediction tasks in the absence of prior information.\n\n## Further Ablation Study\nTo address your concerns, we performed additional ablation studies to analyze the effect of batch size $b$ on the Fed-POE performance. We conducted experiments on the CIFAR-10 dataset, varying the batch size $b$ and the number of models $M$ selected by each client to construct the ensemble model. The table below illustrates the results. As observed, the batch size $b=1$ results in the worst accuracy, mainly due to the forgetting process where models overfit to the most recently observed data. However, increasing the batch size from $b=10$ or $b=20$ to $b=30$ does not significantly improve the accuracy. Larger batch sizes may lead the model to perform better on older data, as the model is trained on older data over more iterations. Therefore, from this study, we conclude that a moderate batch size is optimal, considering that an increase in batch size leads to an increase in computational complexity. Based on these findings, we choose $b=10$. Furthermore, in Figure 1 in Appendix D, we illustrate the regret of Fed-POE and other baselines over time for the CIFAR-10 and WEC datasets.\n\n| CIFAR-10    | M=0 | M=4 | M=8 | M=16 |\n| -------- | ------- | -------- | -------- | -------- |\n| $b=1$  | 53.80% $\\pm$ 6.71%    | 62.73%$\\pm$8.29% | 62.73%$\\pm$8.29% | 62.73%$\\pm$8.26% |\n| $b = 10$ | 65.55%$\\pm$8.77%     | 66.50%$\\pm$8.00% | 66.54%$\\pm$8.08% | 66.46%$\\pm$7.98% |\n| $b = 20$    | 65.72%$\\pm$8.62%   | 66.13%$\\pm$8.20% | 66.64%$\\pm$7.94% | 66.53%$\\pm$8.00% |\n| $b = 30$ | 65.83%$\\pm$8.54% | 66.32%$\\pm$7.92% | 66.24%$\\pm$8.05% | 66.39%$\\pm$8.02% |\n\n## Complexity\nSimilar to other federated learning algorithms, using Fed-POE each client only updates one federated model per step. Therefore, we believe that using Fed-POE does not impose significant additional updating complexity compared to other federated learning counterparts. Please find below the computational complexity analysis of Fed-POE.\n\nLet $C_F$ denote the number of computations required to fine-tune the model $f$, and let $C_I$ represent the number of computations required to make an inference with model $f$. Assume that the complexity of model selection in Algorithm 1 is negligible compared to fine-tuning and making inferences with model $f$. According to Algorithm 2, each client performs $2C_F + (M+2)C_I$ computations per time step. Therefore, the computational complexity of Fed-POE for each client is $\\mathcal{O}(C_F + MC_I)$. Typically, fine-tuning deep neural networks with backpropagation requires significantly more computations than making inferences with them. Thus, if the model $f$ is a deep neural network, $C_I$ is negligible compared to $C_F$. In this case, the computational complexity for each client using Fed-POE is $\\mathcal{O}(C_F)$, which is comparable to most state-of-the-art federated learning methods.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iV0gcbweRE", "reviewer": "Reviewer_rUKC", "summary": "This paper introduces a personalized federated learning algorithm to address the challenges of real-time predictions in non-stationary environments. Clients fine-tune models online, combining their locally fine-tuned models with multiple federated models learned over time. This approach ensures efficient adaptation to evolving data streams, with theoretical analysis and experiments on real datasets demonstrating its effectiveness.", "questions": "No", "limitations": "Yes", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The proposed algorithm effectively addresses the challenge of making real-time predictions in non-stationary environments by allowing clients to fine-tune models online, ensuring continuous adaptation to evolving data streams.\n- By combining locally fine-tuned models with multiple federated models, the approach enhances personalization and leverages the strengths of both local and federated learning, resulting in improved performance.\n- The paper provides a solid theoretical analysis alongside experimental validation on real datasets, demonstrating the practical effectiveness and robustness of the proposed algorithm in real-world scenarios.", "weaknesses": "1. Contributions are suggested to list by items for clear summaries.\n2. The baselines in Table 1 are all before the 2022 year, more latest related methods published in 2023 should be compared.\n3. Fed-POE has limited improvements on Air and FMNIST datasets.\n4. The process of combining locally fine-tuned models with multiple federated models may introduce significant computational overhead for clients, especially those with limited resources.\n5. As the number of clients increases, managing and integrating multiple personalized models can become complex, posing scalability challenges for the proposed algorithm.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for taking the time to review our paper and providing your valuable comments. Please find below our responses to your comments and questions.\n\nWe will revise the presentation of the contributions section in the introduction by breaking down the last paragraph into itemized points.\n\n## Baselines in Table 1\nFor the results in Table 1, we used online kernel-based models to evaluate the performance of Fed-POE in a convex setting. We believe that we compared the performance of Fed-POE against all state-of-the-art online federated kernel learning algorithms which provide rigorous theoretical guarantees. However, all these works were for the year 2022 and before. Moreover, if one employs Fed-DS, which is a baseline from 2023, for the online federated kernel learning problem investigated in Table 1, it is equivalent to Fed-OMD in this specific case. Therefore, our results in Table 1 are relevant to the baselines from 2023.\n\n## Improvements on Air and FMNIST Datasets\nWe believe the performance of the proposed Fed-POE on the Air and FMNIST datasets demonstrates its advantage in real-time prediction tasks. The main challenge in real-time prediction arises when there is no prior information about the streaming data, making it difficult to evaluate model performance before the task begins. As discussed in Section 3.2 of the paper, it is theoretically unclear whether federated or local models perform better under such conditions. The performance largely depends on the dataset.\n\nResults presented in Section 5 confirm that the relative performance of algorithms varies with the dataset. Tables 1 and 2 show that for the CIFAR-10 dataset, federated models outperform local models, while for the Air dataset, local models perform better than federated models. Furthermore, for WEC and FMNIST, both local models and personalized federated learning models outperform non-personalized federated models. This variability complicates the decision between local and federated models. However, the results in Tables 1 and 2 indicate that Fed-POE consistently outperforms all baselines, albeit marginally on some datasets. This suggests that Fed-POE's performance is robust across different datasets, making it a reliable choice for real-time prediction tasks in the absence of prior information.\n\n## Complexity\nWe would like to clarify that at each time step, Fed-POE fine-tunes only one federated model while making inferences with multiple federated models. Given that the computational complexity of making inferences is usually considerably less than that of model fine-tuning, we believe Fed-POE does not introduce significant additional computational overhead. Furthermore, the number of federated models used by clients can be adjusted to ensure that the required computation and memory costs remain manageable for the clients. We analyze the computational complexity of Fed-POE.\n\nLet $C_F$ denote the number of computations required to fine-tune the model $f$, and let $C_I$ represent the number of computations required to make an inference with model $f$. Assume that the complexity of model selection in Algorithm 1 is negligible compared to fine-tuning and making inferences with model $f$. According to Algorithm 2, each client performs $2C_F + (M+2)C_I$ computations per time step. Therefore, the computational complexity of Fed-POE for each client is $\\mathcal{O}(C_F + MC_I)$. Typically, fine-tuning deep neural networks with backpropagation requires significantly more computations than making inferences with them. Thus, if the model $f$ is a deep neural network, $C_I$ is negligible compared to $C_F$. In this case, the computational complexity for each client using Fed-POE is $\\mathcal{O}(C_F)$, which is comparable to most state-of-the-art federated learning methods.\n\n## Increase in the Number of Clients\nAn increase in the number of clients does **not** pose scalability issues for Fed-POE compared to other state-of-the-art federated learning methods. As with most other federated learning algorithms, each client using Fed-POE only fine-tunes one model at each step and sends the update to the server.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a personalized federated learning algorithm, Fed-POE, which is designed for adaptive prediction and model fine-tuning in dynamic environments, based on the provocative claim that federated learning is not always better than locally-trained models. It is established on some sound theoretical analysis, and also evaluated to show robust performance in experiments. In the final version, we hope that the authors can improve the presentation as suggested by some reviewers, and incorporate more helpful details in the rebuttal into the main paper.
Decision: Accept (poster)