Title: On the Necessity of Collaboration for Online Model Selection with Decentralized Data
Venue: NeurIPS 2024
Accepted: yes
Keywords: online learning

model selection

federated learning

kernel methods
Abstract: We consider online model selection with decentralized data over $M$ clients, and study the necessity of collaboration among clients. Previous work proposed various federated algorithms without demonstrating their necessity, while we answer the question from a novel perspective of computational constraints. We prove lower bounds on the regret, and propose a federated algorithm and analyze the upper bound. Our results show (i) collaboration is unnecessary in the absence of computational constraints on clients; (ii) collaboration is necessary if the computational cost on each client is limited to $o(K)$, where $K$ is the number of candidate hypothesis spaces. We clarify the unnecessary nature of collaboration in previous federated algorithms for distributed online multi-kernel learning, and improve the regret bounds at a smaller computational and communication cost. Our algorithm relies on three new techniques including an improved Bernstein's inequality for martingale, a federated online mirror descent framework, and decoupling model selection and prediction, which might be of independent interest.

On the Necessity of Collaboration for Online Model Selection with Decentralized Data

Abstract

We consider online model selection with decentralized data over M clients, and study the necessity of collaboration among clients. Previous work proposed various federated algorithms without demonstrating their necessity, while we answer the question from a novel perspective of computational constraints. We prove lower bounds on the regret, and propose a federated algorithm and analyze the upper bound. Our results show (i) collaboration is unnecessary in the absence of computational constraints on clients; (ii) collaboration is necessary if the computational cost on each client is limited to o ( K ) , where K is the number of candidate hypothesis spaces. We clarify the unnecessary nature of collaboration in previous federated algorithms for distributed online multi-kernel learning, and improve the regret bounds at a smaller computational and communication cost. Our algorithm relies on three new techniques including an improved Bernstein's inequality for martingale, a federated online mirror descent framework, and decoupling model selection and prediction, which might be of independent interest.

1 Introduction

Model selection which is a fundamental problem for offline machine learning focuses on how to select a suitable hypothesis space for a machine learning algorithm [Mitchell, 1997, Bartlett et al., 2002, Mohri et al., 2018]. Model selection for online machine learning is called online model selection (OMS), such as model selection for online supervised learning [Foster et al., 2017, Zhang and Liao, 2018, Zhang et al., 2021, Li and Liao, 2022], model selection for online active learning [Karimi et al., 2021], and model selection for contextual bandits [Foster et al., 2019, Pacchiano et al., 2020, Ghosh and Chowdhury, 2022]. We consider model selection for online supervised learning. Let F = {F 1 , . . . , F K } contain K hypothesis spaces and glyph[lscript] ( · , · ) be a loss function. For a sequence of examples { ( x t , y t ) } t =1 ,...,T , we aim to adapt to the case that the optimal hypothesis space F i ∗ ∈ F is given by an oracle and we run an online learning algorithm in F i ∗ . OMS can be defined by minimizing the regret , i.e.,

where f t ∈ ∪ K i =1 F i is the hypothesis used by an OMS algorithm at the t -th round. The optimal value of the regret depends on the complexity of F i ∗ [Foster et al., 2017, 2019].

In this work, we consider online model selection with decentralized data (OMS-DecD) over M clients, in which each client observes a sequence of examples {( x ( j ) t , y ( j ) t )} t =1 ,...,T , j = 1 , . . . , M , and but does not share personalized data with others. There is a central server that coordinates the clients by sharing personalized models or gradients [Koneˇ cný et al., 2016, Kairouz et al., 2021, Zeng et al., 2023a,b]. OMS-DecD captures some real-world applications where the data may be collected by sensors on M different remote devices or mobile phones [Li et al., 2020, Patel et al., 2023, Kwon et al., 2023], or a local device can not store all of data due to low storage and thus it is necessary to store the data on more local devices [Slavakis et al., 2014, Bouboulis et al., 2018]. OMS-DecD can be defined by minimizing the following regret,

where f ( j ) t ∈ ∪ K i =1 F i is the hypothesis adopted by the j -th client at the t -th round. Solving OMSDecD must achieve two goals: G 1 minimizing the regret, and G 2 providing privacy protection.

A trivial approach is to use a noncooperative algorithm that independently runs a copy of an OMS algorithm on the M clients. It naturally provides strong privacy protection, that is, it achieves G 2, but suffers a regret bound that increases linearly with M . It is unknown whether it achieves G 1. Another approach is federated learning which is a framework of cooperative learning with privacy protection and is provably effective in stochastic convex optimization [McMahan et al., 2017, Woodworth et al., 2020b, Wang et al., 2021, Reddi et al., 2021]. It is natural to ask:

Question 1. Whether collaboration is effective for OMS-DecD.

The question reveals the hardness of OMS-DecD and is helpful to understand the limitations of federated learning. Previous work studied a special instance of OMS-DecD called distributed online multi-kernel learning (OMKL) where F i is a reproducing kernel Hilbert space (RKHS), and proposed three federated OMKL algorithms including vM-KOFL, eM-KOFL [Hong and Chae, 2022] and POF-MKL [Ghari and Shen, 2022]. The three algorithms also suffer regret bounds that increase linearly with M , and thus can not answer the question. If K = 1 , then OMS-DecD is equivalent to distributed online learning [Mitra et al., 2021, Kwon et al., 2023, Patel et al., 2023]. A noncooperative algorithm that independently runs online gradient descent (OGD) on each client achieves the two goals simultaneously [Patel et al., 2023]. Collaboration is unnecessary in the case of K = 1 .

In summary, previous work can not answer the question well. On one hand, previous work can not answer the question in the case of K > 1 . On the other hand, in the case of K = 1 , previous work has answered the question only using the statistical property of algorithms, i.e., the worst-case regret, but omitted the computational property which is very important for real-world applications.

1.1 Main Results

In this paper, we will answer the question from a new perspective of computational constraints on the problem (Section 5.5). Our main results are as follows.

An upper bound on the regret. We propose a federated algorithm, FOMD-OMS, and prove an upper bound on the regret (Theorem 2). Besides, if F 1 , ..., F K are RKHSs, then our algorithm improves the regret bounds of FOP-MKL [Ghari and Shen, 2022] and eM-KOFL [Hong and Chae, 2022] at a smaller computational and communication cost. Table 1 summarizes the results.

Lower bounds on the regret. We separately prove a lower bound on the regret of any (possibly cooperative) algorithm and any noncooperative algorithm (Theorem 3).

A new perspective of computational constraints for Question 1. By the upper bound and lower bounds, we conclude that (i) collaboration is unnecessary when there are no computational constraints on clients, thereby generalizing the result for distributed online learning, i.e., K = 1 ; (ii) collaboration is necessary if the computational cost on each client is limited to o ( K ) where irrelevant parameters are omitted. Our results clarify the unnecessary nature of collaboration in previous federated algorithms for distributed OMKL.

1.2 Technical Challenges

There are two main technical challenges on designing a federated online model selection algorithm.

The first challenge lies in obtaining high-probability regret bounds that adapt to the complexity of individual hypothesis space, a fundamental problem in online model selection [Foster et al., 2017]. While acquiring expected regret bounds that adapt to the complexity of individual hypothesis spaces is straightforward, the crux is to derive high-probability bounds from expected bounds. To this end, we introduce a new Bernstein's inequality for martingale (Lemma 1), which might be of independent interest.

The second challenge involves achieving a per-round communication cost of o ( K ) . To tackle this challenge, we propose two techniques: (i) decoupling model selection and prediction; (ii) an algorithmic framework, named FOMD-No-LU, which might be of independent interest. Specifically, when clients execute model selection, server must broadcast an aggregated probability distribution, denoted by p ∈ R K , to clients, naturally incurring a O ( K ) download cost. Our algorithm conducts model selection on server and makes predictions on clients, thereby eliminating the need to broadcast the aggregated probability distribution to clients. Additionally, if we use the local updating approach [Mitra et al., 2021, Patel et al., 2023], then server must broadcast K aggregated models to clients, also resulting in a O ( K ) download cost [Ghari and Shen, 2022]. By utilizing FOMD-No-LU, our algorithm only broadcasts the selected models to clients and can achieve a o ( K ) download cost.

2 Related work

Previous work has studied the necessity of collaboration for distributed bandit convex optimization [Patel et al., 2023], where a federated algorithmic framework named FEDPOSGD was proposed. Although the regret bounds of FEDPOSGD are smaller than some noncooperative algorithms, there is not a lower bound on the regret of any noncooperative algorithm [Patel et al., 2023]. Moreover, the regret analysis of FEDPOSGD is based on the analysis for federated online gradient descent that is not applicable to our algorithm, FOMD-OMS. The regret analysis of FOMD-OMS requires the analysis for federated online mirror descent with negative entropy regularizer.

Our work is also different from federated bandits, such as federated K -armed bandits [Wang et al., 2020] and federated linear contextual bandits [Huang et al., 2021]. For OMS-DecD, we do not assume that the examples ( x ( j ) t , y ( j ) t ) , t = 1 , . . . , T , on each client are independent and identically distributed (i.i.d.). In contrast, in both federated K -armed bandits or federated linear contextual bandits, the rewards must be i.i.d., thereby making collaboration effective. This is similar to the approach used in federated stochastic optimization. However, this may not hold true for OMS-DedD. Therefore, it is a distinctive problem for OMS-DecD to study whether collaboration is effective.

3 Problem Setting

Notations Let X = { x ∈ R d |‖ x ‖ 2 < ∞} be an instance space, Y = { y ∈ R : | y | < ∞} be an output space, and { ( x t , y t ) } t ∈ [ T ] be a sequence of examples, where [ T ] = { 1 , . . . , T } , x t ∈ X and y t ∈ Y . Let S = { s 1 , s 2 , . . . } be a finite set, Uni( S ) be the uniform distribution over the elements in S and s [ T ] be the abbreviation of the sequence s 1 , s 2 , . . . , s T . Denote by P [ A ] the probability that an event A occurs, a ∧ b = min { a, b } , a ∨ b = max { a, b } and log( a ) = log 2 ( a ) . Let ψ t ( · ) : Ω → R , t ∈ [ T ] be a sequence of time-variant strongly convex regularizers defined on a domain Ω . The Bregman divergence denoted by D ψ t ( · , · ) , associated with ψ t ( · ) is defined by

3.1 Online Model Selection (OMS)

Let F = {F 1 , ..., F K } contain K hypothesis spaces where

Let F i ∗ ∈ F be the optimal but unknown hypothesis space for a given { ( x t , y t ) } t ∈ [ T ] . OMS can be defined as follows: generating a sequence of hypotheses f [ T ] that minimizes the following regret ,

where f t ∈ ∪ K i =1 F i . The optimal hypothesis space F i ∗ must contain a good hypothesis and has a low complexity [Foster et al., 2017, 2019], and is defined by

where C i measures the complexity of F i , such as U i and d i .

OMS is more challenge than online learning, since we not only learn the optimal hypothesis space, but also learn the optimal hypothesis in the space. Next we give some examples of OMS.

Example 1 (Online Hyper-parameters Tuning) . Let F i consist of linear functions of the form

where U i > 0 is a regularization parameter. Let U = { U i , i ∈ [ K ] : U 1 < U 2 < . . . < U K } . The hypothesis spaces are nested, i.e., F 1 ⊆ F 2 ⊆ . . . ⊆ F K . The optimal regularization parameter U i ∗ ∈ U corresponds to the optimal hypothesis space F i ∗ ∈ F .

Example 2 (Online Kernel Selection [Shen et al., 2019, Li and Liao, 2022]) . Let κ i ( · , · ) : R d × R d → R be a positive semidefinite kernel function, and φ i : R d → R d i be the associated feature mapping. F i is the RKHS associated with κ i , i.e.,

The optimal kernel function κ i ∗ ∈ { κ 1 , . . . , κ K } corresponds to the optimal RKHS F i ∗ ∈ F .

Example 3 (Online Pre-trained Classifier Selection [Karimi et al., 2021]) . Generally, F i can be a well-trained machine learning model. Let F contain K pre-trained classifiers. For a new instance x t , we select a (combinational) pre-trained classifier and make a prediction. The selection of a pre-trained classifier has an important implication in practical scenarios.

3.2 Online Model Selection with Decentralized Data (OMS-DecD)

We formally define OMS-DecD as follows. Assuming that there are M clients and a server. At any round t , each client observes an instance x ( j ) t , and selects a hypothesis f ( j ) t ∈ ∪ K i =1 F i , j ∈ [ M ] . Then clients output predictions { f ( j ) t ( x ( j ) t ) } j M =1 . The goal is to minimize the following regret

where y ( j ) t is the label or true output. Each client can not share personalized data with others, but can share personalized models or gradients via the central server.

4 FOMD-No-LU

In this section, we propose a federated algorithmic framework, FOMD-No-LU (Federated Online Mirror Descent without Local Updating) for online collaboration.

Let Ω be a convex and bounded decision set. At any round t , each client j ∈ [ M ] selects a decision u ( j ) t ∈ Ω , and then observes a loss function l ( j ) t ( · ) : Ω → R . The client computes the loss l ( j ) t ( u ( j ) t ) and an estimator of the gradient denoted by ˜ g ( j ) t (or the gradient denoted by g ( j ) t ). To reduce the communication cost, we adopt the intermittent communication (IC) protocol [Woodworth et al., 2021] where clients communicate with server every N rounds. Assuming that T = N × R where N,R ∈ Z , the IC protocol limits the rounds of communication to R .

We divide [ T ] into R disjoint sub-intervals denoted by { T r } R r =1 , in which

For any t ∈ T r , all clients always select the initial decision, i.e.,

At the end of the rN -round, all of clients send 1 N ∑ t ∈ T r ˜ g ( j ) t , j ∈ [ M ] to server. Then server updates the decision within online mirror descent framework [Bubeck and Cesa-Bianchi, 2012],

(4)-(5) is called model averaging [McMahan et al., 2017] and shows the collaboration among clients. Finally, server may broadcast u t +1 to all clients. Let the initial decision u ( j ) 1 = u 1 for all j ∈ [ M ] , then it must be u ( j ) t = u t for all t ∈ [ T ] . Thus clients do not transmit u ( j ) t to server. The pseudo-code of FOMD-No-LU is shown in Algorithm 1.

4.1 Regret Bound

Theorem 1. Let E = { N, 2 N,...., RN } where N = T R and R ∈ [ T ] . Assuming that l ( j ) t ( · ) , t ∈ [ T ] , j ∈ [ M ] , are convex loss functions. Let g ( j ) t = ∇ u ( j ) t l ( j ) t ( u ( j ) t ) and ˜ g ( j ) t be an estimator of g ( j ) t . At any round t ∈ E , let q t +1 and r t +1 be two auxiliary decisions defined as follows,

Then FOMD-No-LU guarantees that,

It is intriguing that the regret bound comprises two components: the first part, Ξ 1 , cannot be reduced by collaboration, while the second part, Ξ 2 , highlights the benefits of collaboration. Ξ 1 is the regret induced by exact gradients, while Ξ 2 is the regret induced by estimated gradients and shows how collaboration controls the regret. It is worth mentioning that Theorem 1 gives a general regret bound, from which various types of regret bounds can be readily derived by instantiating the decision set Ω and the regularizer ψ t ( · ) . For instance, if Ω = F i where F i follows Example 1, ψ t ( v ) = 1 2 λ ‖ v ‖ 2 2 and E [ ‖ ˜ g ( j ) t ‖ 2 2 ] ≤ C ‖ g ( j ) t ‖ 2 2 , then FOMD-No-LU becomes a federated online descent descent. It is easy to give a O ( MU i √ (1 + C M ) T ) expected regret from Theorem 1. Besides, N > 1 increases the regret and shows the trade-off between communication cost and regret bound.

Theorem 1 requires a novel analysis on how the bias of estimators, i.e., ∑ M j =1 ‖ ˜ g ( j ) t -g ( j ) t ‖ 2 2 , is controlled by cooperation. To this end, we introduce two virtual decisions q t +1 and r t +1 that are updated by 2 ∑ M j =1 ˜ g ( j ) t -g ( j ) t M and 2 ∑ M j =1 g ( j ) t M , respectively. Previous federated online mirror descent uses exact gradients g ( j ) t , j ∈ [ M ] [Mitra et al., 2021]. Thus its analysis is different from ours.

Algorithm 1 FOMD-No-LU

Require:

Ω

.

Ensure:

u

(

j

)

1

, j

∈

[

M

]

1:

for

r

= 1

,

2

, . . . , R

do

2:

for

t

= (

r

-

1)

N

+1

, . . . , rN

do

3:

for

j

= 1

, . . . , M

in parallel

do

4:

Client selects

u

(

j

)

(

r

-

1)

N

+1

∈

Ω

5:

Client observes loss function

l

(

j

)

t

(

·

)

6:

Client computes estimated gradient

˜

g

(

j

)

t

7:

end for

8:

if

t

==

rN

then

9:

Clients transmit

1

N

∑

t

∈

T

r

˜

g

(

j

)

t

,

j

∈

[

M

]

10:

Server computes

u

t

+1

following (4)-(6)

11:

Server may broadcast

u

t

+1

12:

end if

13:

end for

14:

end for

Algorithm 2

FOMD-OMS (

R

=

T

)

Require:

T

,

J

,

η

1

,

{

U

i

, λ

1

,i

, i

∈

[

K

]

}

Ensure:

f

(

j

)

1

,i

= 0

,

p

1

,i

,

i

∈

[

K

]

,

j

∈

[

M

]

1:

for

t

= 1

,

2

, . . . , T

do

2:

for

j

= 1

, . . . , M

do

3:

Server samples

O

(

j

)

t

following (7)

4:

Server broadcasts

f

(

j

)

t,i

, i

∈

O

(

j

)

t

to client

5:

end for

6:

for

j

= 1

, . . . , M

in parallel

do

7:

Client outputs

f

(

j

)

t,A

t,

1

(

x

(

j

)

t

)

8:

Client computes

∇

(

j

)

t,i

, c

(

j

)

t,i

, i

∈

O

(

j

)

t

9:

Client transmits

∇

(

j

)

t,i

, c

(

j

)

t,i

, i

∈

O

(

j

)

t

10:

end for

11:

Server computes

p

t

+1

following (8)

12:

Server computes

w

t

+1

,i

, i

∈

[

K

]

following (9)

13:

end for

4.2 Comparison with Previous Work

In fact, FOMD-No-LU adopts the batching technique [Dekel et al., 2011], that is, it divides [ T ] into R sub-intervals and executes (3) during each sub-intervals. The batching technique (also known as mini-batch) has been used in the multi-armed bandit problem [Arora et al., 2012] and distributed stochastic convex optimization [Karimireddy et al., 2020, Woodworth et al., 2020a]. We use the batching technique for the first time to distributed online learning.

glyph[negationslash]

FOMD-No-LU is different from FedOMD (federated online mirror descent) [Mitra et al., 2021]. (i) FedOMD only transmits exact gradients, while FOMD-No-LU can transmit estimated gradients. Thus the regret bound of FedOMD did not contain Ξ 2 in Theorem 1. (ii) FedOMD uses local updating, such as local OGD [Patel et al., 2023] and local SGD [McMahan et al., 2017, Reddi et al., 2021]. Thus FedOMD induces client drift, i.e., u ( j ) t = u t . Besides, if we use FedOMD to design a federated online model selection algorithm, then the download cost is in O ( MK ) bits.

5 A Federated Algorithm for OMS-DecD

In this section, we just consider the case R = T , that is, there is no communication constraints. Due to space constraints, we have deferred the algorithm and result of R < T to the appendix.

At a high level, our algorithm comprises two components both of which are critical for achieving a communication cost in o ( K ) : (i) decoupling model selection and online prediction; (ii) collaboratively updating decisions within the framework of FOMD-No-LU.

5.1 Decoupling Model Selection and Prediction

Model Selection on Server At any round t , server maintains K hypotheses { f ( j ) t,i ∈ F i } K i =1 and a probability distribution p ( j ) t over the K hypotheses for all j ∈ [ M ] . The model selection process aims to select a hypothesis from { f ( j ) t,i } K i =1 and then predicts the output of x ( j ) t . An intuitive idea is that, for each j ∈ [ M ] , the client samples a hypothesis following p ( j ) t . However, such an approach requires that server broadcasts p ( j ) t to clients, and will cause a download cost in O ( K ) .

The sampling operation (or model selection process) can be executed on server. Specifically, server just broadcasts the selected hypotheses, and thus saves the communication cost. For each j ∈ [ M ] , server selects J ∈ [2 , K ] hypotheses denoted by f ( j ) t,A t,a , a ∈ [ J ] where A t,a ∈ [ K ] . For simplicity, let O ( j ) t = { A t, 1 , . . . , A t,J } . We instantiate u t = p t in FOMD-No-LU. Then FOMD-No-LU ensures

p ( j ) t = p t for all j ∈ [ M ] . We sample A t, 1 , . . . , A t,J in order and follow (7).

Server samples O ( j ) t for all j ∈ [ M ] and thus must independently execute (7) M times which only pays an additional computational cost in O ( M log K ) . The factor log K arises from the process of sampling a number from { 1 , ..., K } . Server only sends f ( j ) A t,a , a ∈ [ J ] to the j -th client. It is worth mentioning that server does not send p t . The total download cost is O ( ∑ M j =1 ∑ J a =1 ( d A t,a +log K )) . If J is independent of K , then the download cost is only O ( M log K ) .

Prediction on Clients For each j ∈ [ M ] , the j -th client receives f ( j ) A t,a , a ∈ [ J ] , and uses f ( j ) t,A t, 1 to output a prediction, i.e.,

where we assume that f ( j ) t,i is parameterized by w ( j ) t,i ∈ R d i (see (1)). After observing the true output y ( j ) t , the client suffers a loss glyph[lscript] ( f ( j ) t,A t, 1 ( x ( j ) t ) , y ( j ) t ) .

It is worth mentioning that the other J -1 hypotheses f ( j ) t,A t,a , a ≥ 2 are just used to obtain more information on the loss function. We will explain more in the following subsection. Thus we do not cumulate the loss glyph[lscript] ( f ( j ) t,A t,a ( x ( j ) t ) , y ( j ) t ) , a ≥ 2 .

5.2 Online Collaboration Updating

Updating sampling probabilities For each j ∈ [ M ] , let c ( j ) t = ( c ( j ) t, 1 , . . . , c ( j ) t,K ) where c ( j ) t,i = glyph[lscript] ( f ( j ) t,i ( x ( j ) t ) , y ( j ) t ) is the loss of f ( j ) t,i , i ∈ [ K ] . The j -th client will send c ( j ) t,i , i ∈ O ( j ) t , to server. Since c ( j ) t,i , i / ∈ O ( j ) t can not be observed, it is necessary to construct an estimated loss vector ˜ c ( j ) t = (˜ c ( j ) t, 1 , . . . , ˜ c ( j ) t,K ) where ˜ c ( j ) t,i = c ( j ) t,i P [ i ∈ O ( j ) t ] · I i ∈ O ( j ) t , i ∈ [ K ] . It is easy to prove that E t [ ˜ c ( j ) t,i ] = c ( j ) t,i and E t [ (˜ c ( j ) t,i ) 2 ] ≤ K -1 J -1 ( c ( j ) t,i ) 2 where E t [ · ] := E [ ·| O ( j ) [ t -1] ] . Thus sampling A t,a , a ≥ 2 reduces the variance of the estimators which is equivalent to obtain more information on the true loss.

Server aggregates ˜ c ( j ) t , j ∈ [ M ] and updates p t following (4)-(6). Let ∆ K be the ( K -1) -dimensional simplex, Ω = ∆ K and ˜ g ( j ) t = ˜ c ( j ) t . Then the server executes (8).

where ψ t ( p ) is the weighted negative entropy regularizer [Bubeck et al., 2017], η t > 0 is a timevariant learning rate and C i > 0 satisfies that max t,j c ( j ) t,i ≤ C i . It is obvious that server does not broadcast p t +1 .

Updating hypotheses For each j ∈ [ M ] and i ∈ [ K ] , let ∇ ( j ) t,i = ∇ w ( j ) t,i glyph[lscript] (〈 w ( j ) t,i , φ i ( x ( j ) t ) 〉 , y ( j ) t ) . Since ∇ ( j ) t,i , i / ∈ O ( j ) t are unknown, it is necessary to construct an estimator of the gradient, denoted by ˜ ∇ ( j ) t,i = ∇ ( j ) t,i P [ i ∈ O ( j ) t ] · I i ∈ O ( j ) t for all j ∈ [ M ] , i ∈ [ K ] . Clients send {∇ ( j ) t,i , i ∈ O ( j ) t } , j ∈ [ M ] to server. Then server aggregates { ˜ ∇ ( j ) t,i , i ∈ [ K ] } , j ∈ [ M ] and updates the hypotheses following

(4)-(6). For each i ∈ [ K ] , let Ω = F i and ˜ g ( j ) t = ˜ ∇ ( j ) t,i . Server executes (9).

where ψ t,i ( w ) is the Euclidean regularizer and λ t,i is a time-variant learning rate.

We name this algorithm FOMD-OMS (FOMD-No-LU for OMS-DecD) and show it in Algorithm 2.

5.3 Regret bounds

To obtain high-probability regret bounds that adapt to the complexity of individual hypothesis space, we establish a new Bernstein's inequality for martingale.

Lemma 1. Let X 1 , . . . , X n be a bounded martingale difference sequence w.r.t. the filtration H = ( H k ) 1 ≤ k ≤ n and with | X k | ≤ a . Let Z t = ∑ t k =1 X k be the associated martingale. Denote the sum of the conditional variances by Σ 2 n = ∑ n k =1 E [ X 2 k |H k -1 ] ≤ v, where v ∈ [0 , B ] is a random variable and B ≥ 2 is a constant. Then for any constant a > 0 , with probability at least 1 -2 glyph[ceilingleft] log B glyph[ceilingright] δ ,

Note that v is a random variable in Lemma 1, while it is a constant in standard Bernstein's inequality for martingale (see Lemma A.8 [Cesa-Bianchi and Lugosi, 2006]). Lemma 1 is derived from the standard Bernstein's inequality along with the well-known peeling technique [Bartlett et al., 2005].

Assumption 1. For each i ∈ [ K ] , there is a constant b i such that ‖ φ i ( x ) ‖ 2 ≤ b i where φ i ( · ) is defined in (1) .

Lemma 2. Under Assumption 1, for each i ∈ [ K ] , there are two constants C i > 0 , G i > 0 that depend on U i or b i such that max t,j c ( j ) t,i ≤ C i and max t,j ‖∇ ( j ) t,i ‖ 2 ≤ G i .

Theorem 2. Let R = T . Under Assumption 1, denote by A m = argmin i ∈ [ K ] C i and C = max i ∈ [ K ] C i . Assuming that glyph[lscript] ( · , · ) is convex and K ≥ J ≥ 2 . Let g K,J = K -J J -1 ,

With probability at least 1 -Θ( M log( T ) + log( KT/M )) · δ , the regret of FOMD-OMS satisfies:

Both C i and G i depend on U i or b i (see Lemma 2). Let C i = Θ( U i G i + C i ) . Thus C i measures the complexity of F i . Then our regret bound adapts to √ CC i where C = max i ∈ [ K ] C i , while previous regret bounds depend on C [Ghari and Shen, 2022, Hong and Chae, 2022], that is, they can not adapt to the complexity of individual hypothesis space. If C i ∗ glyph[lessmuch] C , then our regret bound is much better.

The regret bound in Theorem 2 is also called multi-scale regret bound [Bubeck et al., 2017]. However, previous regret analysis can not yield a high-probability multi-scale bound. The reason is the lack of the new Bernstein's inequality for martingale (Lemma 1). If we use the new Freedman's inequality for martingale [Lee et al., 2020], then a high-probability bound can still be obtained, but is worse than the bound in Theorem 2 by a factor of order O (poly(ln T )) .

5.4 Time Complexity and Communication Complexity Analysis

For each j ∈ [ M ] , the j -th client makes prediction and computes gradients in time O ( ∑ i ∈ O ( j ) t d i ) .

Server samples O ( j ) t , j ∈ [ M ] , aggregates gradients and updates global models. The per-round time complexity on server is O ( ∑ M j =1 ∑ i ∈ O ( j ) t d i + ∑ K i =1 d i + JM log K ) .

Upload At any round t ∈ [ T ] , the j -th client transmits c ( j ) t,i , ∇ ( j ) t,i , i ∈ O ( j ) t and the corresponding indexes to server. It requires J ( ∑ i ∈ O ( j ) t d i +1) floating-point numbers and J integers. If we use 32 bits to represent a float, and use log K bits to represent an integer in [ K ] . Each client transmits (32 J ( ∑ i ∈ O ( j ) t d i +1) + J log K ) bits to server.

Download Server broadcasts w t,i ∈ R d i , i ∈ O ( j ) t and the corresponding indexes to clients. The total download cost is (32 MJ ( ∑ i ∈ O ( j ) t d i +1) + MJ log K ) bits.

5.5 Answers to Question 1

Before discussing Question 1, we give two lower bounds on the regret.

Theorem 3 (Lower Bounds) . Assuming that 5 ≤ K ≤ min { d, T } . For each i ∈ [ K ] , let F i = { f i ( x ) = e glyph[latticetop] i x } and D i = [min x ∈X f i ( x ) , max x ∈X f i ( x )] , where e i is the standard basis vector in R d . Denote by sup the supremum over all examples.

(i) There are no computational constraints on clients. Let glyph[lscript] ( v, y ) = | v -y | . The regret of any algorithm for OMS-DecD satisfies: lim T →∞ sup max i ∈ [ K ] Reg D ( F i ) ≥ 0 . 25 M √ T ln K ;

(ii) The per-round time complexity on each client is limited to O ( J ) . Let glyph[lscript] ( v, y ) = 1 -v · y . The regret of any, possibly randomized, noncooperative algorithm with outputs in ∪ i ∈ [ K ] D i for OMS-DecD satisfies: sup E [max i ∈ [ K ] Reg D ( F i )] ≥ 0 . 1 M √ KTJ -1 , where the expectation is taken over the randomization of algorithm.

The assumption that the outputs of any noncooperative algorithm belong to ∪ i = ∈ [ K ] D i is natural, and can be removed in the case of J = 1 . Next we define a noncooperative algorithm, NCO-OMS.

Definition 1 (NCO-OMS) . NCO-OMS independently samples O ( j ) t following (7) and executes

where the definitions of ˜ c ( j ) t and ˜ ∇ ( j ) t,i follow FOMD-OMS.

It is easy to prove the regret of NCO-OMS satisfies: with probability at least 1 -Θ( M log( KT )) · δ ,

where B i, 1 = U i G i + C i √ ln( KT ) , B i, 2 = C + U i G i and B i, 3 = U i G i + √ CC i . We leave the pseudo-code of NCO-OMS and the corresponding regret analysis in appendix.

Next we discuss Question 1 by considering two cases.

Case 1 : There are no computational constraints on clients. Collaboration is unnecessary.

Let J = Θ( K ) in FOMD-OMS and NCO-OMS. By Theorem 2, both FOMD-OMS and NCO-OMS enjoy a O ( MU i G i √ T + MC i √ T ln( KT )) regret. By Theorem 3, FOMD-OMS and NCO-OMS are nearly optimal in terms of the dependence on M and T . Thus collaboration is unnecessary.

Case 2 : The per-round time complexity on each client is limited to o ( K ) . Collaboration is necessary.

Let J = o ( K ) in FOMD-OMS and Theorem 3. By Theorem 2, FOMD-OMS enjoys a O ( MB i, 1 √ T + B i, 3 √ MKTJ -1 ln δ -1 ) regret, which is smaller than the lower bound on the regret of any noncooperative algorithm (see Theorem 3). Thus collaboration is necessary.

6 Application to Distributed OMKL

We will apply the proposed FOMD-OMS to a special instance of OMS-DecD, known as distributed OMKL, in which F i is a RKHS. Then we contrast our results with those from earlier studies, highlighting the unnecessary nature of collaboration in prior federated algorithms.

Theorem 4. Let F i be a RHKS for all i ∈ [ K ] and R ≤ T . With probability at least 1 -Θ( TM log( R ) + T log( KR/M )) · δ , the regret of FOMD-OMS satisfies, ∀ i ∈ [ K ] ,

where the notation ˜ O ( · ) hides polylogarithmic factor in δ -1 and D = d i follows (1) .

We defer the algorithm in appendix. Let R = T and J = 2 . We compare FOMD-OMS with eM-KOFL [Hong and Chae, 2022] and POF-MKL [Ghari and Shen, 2022]. Table 1 gives the results.

We observe that FOMD-OMS significantly improves the computational complexity of eM-KOFL and POF-MKL (by a factor of O ( K ) ) on each client. The per-round time complexity of the two algorithms is O ( DK ) . Recalling the answer to Question 1 (see Section 5.5), collaboration in eM-KOFL and POF-MKL is unnecessary.

Next we compare the regret bound of the three algorithms. Recalling that C i ≤ C . The regret bounds of eM-KOFL and POF-MKL can not adapt to the complexity of individual hypothesis space. (i) The regret bound of FOMD-OMS is better than that of POF-MKL in relation to its dependence on M and C i . (ii) In the case of K = O ( C C i M · ln K ) , the regret bound of FOMD-OMS is better than that of eM-KOFL. (iii) In the case of K = Ω ( C C i M · ln K ) , the regret bound of FOMD-OMS is worse than that of eM-KOFL. If K is sufficiently large, the regret bound of eM-KOFL is better than that of FOMD-OMS by a factor of O ( √ K ) .

7 Conclusion

In this paper, we have studied the necessity of collaboration for OMS-DecD from the perspective of computational constraints. We demonstrate that collaboration is unnecessary when there are no computational constrains on clients, while it becomes necessary if the time complexity on each client is limited to o ( K ) . Our work clarifies the unnecessary nature of collaboration in previous algorithms for the first time, gives conditions under which collaboration is necessary, and provides inspirations for studying the problem from constraints beyond computational constrains.

References

Raman Arora, Ofer Dekel, and Ambuj Tewari. Online bandit learning against an adaptive adversary: from regret to policy regret. In Proceedings of the 29th International Conference on Machine Learning , pages 1503-1510, 2012.

Peter L. Bartlett, Stéphane Boucheron, and Gébor Lugosi. Model selection and error estimation. Machine Learning , 48(1):85-113, 2002.

Peter L. Bartlett, Olivier Bousquet, and Shahar Mendelson. Local rademacher complexities. Annals of Statistics , 33(4):1497-1537, 2005.

Pantelis Bouboulis, Symeon Chouvardas, and Sergios Theodoridis. Online distributed learning over networks in RKH spaces using random fourier features. IEEE Transactions on Signal Processing , 66(7):1920-1932, 2018.

Stephen Boyd and Lieven Vandenberghe. Convex Optimization . Cambridge University Press, New York, NY, 2004.

Sébastien Bubeck and Nicolò Cesa-Bianchi. Regret analysis of stochastic and nonstochastic multiarmed bandit problems. Foundations and Trends glyph[circleR] in Machine Learning , 5(1):1-122, 2012.

Sébastien Bubeck, Nikhil R. Devanur, Zhiyi Huang, and Rad Niazadeh. Online auctions and multiscale online learning. In Proceedings of the 2017 ACM Conference on Economics and Computation , pages 497-514, 2017.

Nicolò Cesa-Bianchi and Gábor Lugosi. Prediction, Learning, and Games. Cambridge University Press, New York, NY, 2006.

Ofer Dekel, Ran Gilad-Bachrach, Ohad Shamir, and Lin Xiao. Optimal distributed online prediction. In Proceedings of the 28th International Conference on Machine Learning , pages 713-720, 2011.

Dylan J Foster, Satyen Kale, Mehryar Mohri, and Karthik Sridharan. Parameter-free online learning via model selection. Advances in Neural Information Processing Systems , 30:6022-6032, 2017.

Dylan J Foster, Akshay Krishnamurthy, and Haipeng Luo. Model selection for contextual bandits. Advances in Neural Information Processing Systems , 32:14741-14752, 2019.

Pouya M. Ghari and Yanning Shen. Personalized online federated learning with multiple kernels. Advances in Neural Information Processing Systems , 35:33316-33329, 2022.

Avishek Ghosh and Sayak Ray Chowdhury. Model selection in reinforcement learning with general function approximations. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases , pages 148-164, 2022.

Mark A. Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. The WEKA data mining software: an update. SIGKDD Explorations , 11(1):10-18, 2009.

Songnam Hong and Jeongmin Chae. Communication-efficient randomized algorithm for multi-kernel online federated learning. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44 (12):9872-9886, 2022.

Ruiquan Huang, Weiqiang Wu, Jing Yang, and Cong Shen. Federated linear contextual bandits. Advances in Neural Information Processing Systems , 34:27057-27068, 2021.

Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista A. Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D'Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaïd Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Koneˇ cný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Hang Qi, Daniel Ramage, Ramesh Raskar, Mariana Raykova, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma,

Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, and Sen Zhao. Advances and open problems in federated learning. Foundations and Trends in Machine Learning , 14(1-2): 1-210, 2021.

Mohammad Reza Karimi, Nezihe Merve Gürel, Bojan Karlas, Johannes Rausch, Ce Zhang, and Andreas Krause. Online active model selection for pre-trained classifiers. In Proceedings of the 24th International Conference on Artificial Intelligence and Statistics , pages 307-315, 2021.

Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, and Ananda Theertha Suresh. SCAFFOLD: stochastic controlled averaging for federated learning. In Proceedings of the 37th International Conference on Machine Learning , pages 5132-5143, 2020.

Jakub Koneˇ cný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. CoRR , arXiv:1610.05492v2, 2016.

Dohyeok Kwon, Jonghwan Park, and Songnam Hong. Tighter regret analysis and optimization of online federated learning. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45 (12):15772-15789, 2023.

Chung-Wei Lee, Haipeng Luo, Chen-Yu Wei, and Mengxiao Zhang. Bias no more: high-probability data-dependent regret bounds for adversarial bandits and mdps. Advances in Neural Information Processing Systems , 33:15522-15533, 2020.

Junfan Li and Shizhong Liao. Improved regret bounds for online kernel selection under bandit feedback. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases , pages 333-348, 2022.

Junfan Li and Shizhong Liao. Improved regret bounds for online kernel selection under bandit feedback. CoRR , arXiv:2303.05018v2, 2023.

Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine , 37(3):50-60, 2020.

Zhu Li, Jean-Francois Ton, Dino Oglic, and Dino Sejdinovic. Towards a unified analysis of random Fourier features. In Proceedings of the 36th International Conference on Machine Learning , pages 3905-3914, 2019.

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics , pages 1273-1282, 2017.

Tom M. Mitchell. Machine Learning . McGraw-Hill Science, New York, NY, 1997.

Aritra Mitra, Hamed Hassani, and George J. Pappas. Online federated learning. In Proceedings of the 60th IEEE Conference on Decision and Control , pages 4083-4090, 2021.

Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of Machine Learning (second edition) . MIT Press, Cambridge, MA, 2018.

Aldo Pacchiano, My Phan, Yasin Abbasi-Yadkori, Anup Rao, Julian Zimmert, Tor Lattimore, and Csaba Szepesvári. Model selection in contextual stochastic bandit problems. Advances in Neural Information Processing Systems , 33:10328-10337, 2020.

Kumar Kshitij Patel, Lingxiao Wang, Aadirupa Saha, and Nathan Srebro. Federated online and bandit convex optimization. In Proceedings of the 40th International Conference on Machine Learning , pages 27439-27460, 2023.

Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. Advances in Neural Information Processing Systems , 20:1177-1184, 2007.

Ali Rahimi and Benjamin Recht. Weighted sums of random kitchen sinks: Replacing minimization with randomization in learning. Advances in Neural Information Processing Systems , 21:13131320, 2008.

Sashank J. Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Koneˇ cný, Sanjiv Kumar, and Hugh Brendan McMahan. Adaptive federated optimization. In Proceedings of the 9th International Conference on Learning Representations , 2021.

Yevgeny Seldin, Peter L. Bartlett, Koby Crammer, and Yasin Abbasi-Yadkori. Prediction with limited advice and multiarmed bandits with paid observations. In Proceedings of the 31st International Conference on Machine Learning , pages 280-287, 2014.

Yanning Shen, Tianyi Chen, and Georgios B. Giannakis. Random feature-based online multi-kernel learning in environments with unknown dynamics. Journal of Machine Learning Research , 20 (22):1-36, 2019.

Konstantinos Slavakis, Georgios B. Giannakis, and Gonzalo Mateos. Modeling and optimization for big data analytics: (statistical) learning tools for our era of data deluge. IEEE Signal Processing Magazine , 31(5):18-31, 2014.

Roman Vershynin. High-dimensional probability: An introduction with applications in data science . Cambridge University Press, New York, NY, 2018.

Jianyu Wang, Zachary Charles, Zheng Xu, Gauri Joshi, H. Brendan McMahan, Blaise Agüera y Arcas, Maruan Al-Shedivat, Galen Andrew, Salman Avestimehr, Katharine Daly, Deepesh Data, Suhas N. Diggavi, Hubert Eichner, Advait Gadhikar, Zachary Garrett, Antonious M. Girgis, Filip Hanzely, Andrew Hard, Chaoyang He, Samuel Horváth, Zhouyuan Huo, Alex Ingerman, Martin Jaggi, Tara Javidi, Peter Kairouz, Satyen Kale, Sai Praneeth Karimireddy, Jakub Koneˇ cný, Sanmi Koyejo, Tian Li, Luyang Liu, Mehryar Mohri, Hang Qi, Sashank J. Reddi, Peter Richtárik, Karan Singhal, Virginia Smith, Mahdi Soltanolkotabi, Weikang Song, Ananda Theertha Suresh, Sebastian U. Stich, Ameet Talwalkar, Hongyi Wang, Blake E. Woodworth, Shanshan Wu, Felix X. Yu, Honglin Yuan, Manzil Zaheer, Mi Zhang, Tong Zhang, Chunxiang Zheng, Chen Zhu, and Wennan Zhu. A field guide to federated optimization. CoRR , abs/2107.06917, 2021.

Yuanhao Wang, Jiachen Hu, Xiaoyu Chen, and Liwei Wang. Distributed bandit learning: Nearoptimal regret with efficient communication. In Proceedings of the 8th International Conference on Learning Representations , 2020.

Blake E. Woodworth, Kumar Kshitij Patel, and Nati Srebro. Minibatch vs local SGD for heterogeneous distributed learning. Advances in Neural Information Processing Systems , 33:6281-6292, 2020a.

Blake E. Woodworth, Kumar Kshitij Patel, Sebastian U. Stich, Zhen Dai, Brian Bullins, H. Brendan McMahan, Ohad Shamir, and Nathan Srebro. Is local SGD better than minibatch sgd? In Proceedings of the 37th International Conference on Machine Learning , pages 10334-10343, 2020b.

Blake E. Woodworth, Brian Bullins, Ohad Shamir, and Nathan Srebro. The min-max complexity of distributed stochastic convex optimization with intermittent communication. In Proceedings of the 34th Annual Conference on Learning Theory , pages 4386-4437, 2021.

Dun Zeng, Siqi Liang, Xiangjing Hu, Hui Wang, and Zenglin Xu. Fedlab: A flexible federated learning framework. Journal of Machine Learning Research , 24(100):1-7, 2023a.

Dun Zeng, Zenglin Xu, Yu Pan, Xu Luo, Qifan Wang, and Xiaoying Tang. Enhanced federated optimization: Adaptive unbiased client sampling with reduced variance. CoRR , arXiv:2310.02698v3, 2023b. URL https://arxiv.org/abs/2310.02698 .

Xiao Zhang and Shizhong Liao. Online kernel selection via incremental sketched kernel alignment. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence , pages 3118-3124, 2018.

Xiao Zhang, Shizhong Liao, Jun Xu, and Ji-Rong Wen. Regret bounds for online kernel selection in continuous kernel space. In Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence , pages 10931-10938, 2021.

Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the Twentieth International Conference on Machine Learning , pages 928-936, 2003.

A Limitations

The main limitation of this work lies in the first lower bound in Theorem 3 that holds in the case of T →∞ . Although this lower bound nearly matches our upper bound asymptotically and is enough to answer Question 1, it is desired to establish a non-asymptotical lower bound, that is, the lower bound holds for any value of T .

B Broader Impact

This work can be potentially applied to distributed online prediction tasks. We aim to address a fundamental problem whether collaboration among clients is necessary and under what conditions collaboration is necessary. A predictable economic benefit of our work is to give a guidance on the usage of federated learning and save unnecessary communication overhead.

Our work can also instruct the machine learning engineers to tune the hyper-parameter of online learning algorithms. Thus our work can alleviate the burden of machine learning engineers and improve the utility of online leaning algorithms in industrial applications.

C Experiments

In this section, we aim to verify the following three goals which are our main results.

G 1 Collaboration is unnecessary if we allow the computational cost on each client to be O ( K ) . We set R = T and J = K in FOMD-OMS. In this case, the per-round running time on each client is O ( K ) . We aim to verify that FOMD-OMS enjoys similar prediction performance with the noncooperative algorithm, NCO-OMS with J = K (see Definition 1).

G 2 Collaboration is necessary if we limit the computational cost on each client to o ( K ) . We set R = T and J = 2 in FOMD-OMS. In this case, the per-round running time on each client is O (1) . We aim to verify that FOMD-OMS enjoys better prediction performance than

NCO-OMS with J = 2 .

G 3 FOMD-OMS improves the regret bounds of algorithms for distributed OMKL.

FOMD-OMS with R = T and J = 2 enjoys similar prediction performance with eM-KOFL [Hong and Chae, 2022], and enjoys better prediction performance than POF-MKL [Ghari and Shen, 2022] at a smaller computational cost on each client.

Although there are more baseline algorithms, such as vM-KOFL [Hong and Chae, 2022], pMKOFL [Hong and Chae, 2022] and OFSKL [Ghari and Shen, 2022], we do not compare with the three algorithms since they do not perform as well as eM-KOFL and POF-MKL.

C.1 Experimental setting

We will execute three experiments and each one verifies a goal. For simplicity, we do not measure the actual communication cost and use serial implementation to simulate the distributed implementation.

To verify G 1 and G 2, we use the instance of online model selection given in Example 1. The first experiment verifies G 1. We construct 10 nested hypothesis spaces (i.e., K = 10 ) as follows

where U i = i 10 . We set R = T and J = K in FOMD-OMS. Since J = K , we have O ( j ) t = [ K ] and P [ i ∈ O ( j ) t ] = 1 . The learning rates η t , λ t,i , i ∈ [ K ] of FOMD-OMS follow Theorem 2. For NCO-OMS, we set J = K and set the learning rate η t , λ t,i , i ∈ [ K ] following Theorem 2 in which M = 1 , i.e.,

We use the square loss function glyph[lscript] ( f ( x ) , y ) = ( f ( x ) -y ) 2 . For both FOMD-OMS and NCO-OMS, we tune G i = ( U i +1) ×{ 1 , 2 , 4 , 6 , 8 , 10 } and set C i = ( U i +1) 2 .

The second experiment verifies G 2. We set R = T and J = 2 in FOMD-OMS. The learning rates of FOMD-OMS also follow Theorem 2. For NCO-OMS, we also set J = 2 and set the learning rate η t , λ t,i , i ∈ [ K ] following Theorem 2 in which M = 1 , i.e.,

Similar to the first experiment, we tune G i = ( U i +1) ×{ 1 , 2 , 4 , 6 , 8 , 10 } and set C i = ( U i +1) 2 .

The third experiment verifies G 3. We consider online kernel selection (as known as online multikernel learning) which is an instance of online model selection given in Example 2. We select the Gaussian kernel with 8 different kernel widths (i.e., K = 8 ),

and construct the corresponding hypothesis space F i and H i following (21) in which we set U i = U and D i = D for all i ∈ [ K ] and tune U ∈ { 1 , 2 , 4 } . Note that U i is same for all i ∈ [ K ] . We replace the initial distribution p 1 in Theorem 2 with a uniform distribution ( 1 K , . . . , 1 K ) . We set D = 100 for FOMD-OMS, eM-KOFL and POF-MKL. D is the number of random features. We set R = T , J = 2 and C = U +1 in FOMD-OMS. Thus the per-round time complexity on each client is O ( D ) and the per-round communication cost is O ( MD + M log K ) . There are three hyper-parameters in eM-KOFL, i.e., η g , η l and λ . η g is the global learning rate, η l is the local learning rate and λ is a regularization parameter. There are 2 M +3 hyper-parameters in POF-MKL, i.e., η g , η j , ξ j , j ∈ [ M ] , m , λ in which M/m plays the same role with J in FOMD-OMS. η g is the global learning rate, η j is the local learning rate, ξ j is called exploration rate and λ is a regularization parameter. Since J = 2 in FOMD-OMS, we can set m = M/ 2 for FOMD-OMS. Following the original paper [Ghari and Shen, 2022], we set ξ j = 1 . For a fair comparison, we change the learning rates of FOMD-OMS, eM-KOFL and POF-MKL. Following the parameter setting of eM-KOFL [Hong and Chae, 2022], we tune η g , η l , η j ∈ { 0 . 1 , 0 . 5 , 1 , 4 , 8 , 16 } and λ ∈ { 0 . 1 , 0 . 001 , 0 . 0001 } for eM-KOFL and POF-MKL. For FOMD-OMS, we also tune η t , λ t,i ∈ { 0 . 1 , 0 . 5 , 1 , 4 , 8 , 16 } .

For all of the three experiments, we set 10 clients, i.e., M = 10 . We use 8 regression datasets shown in Table 2 from WEKA 2 [Hall et al., 2009] and UCI machine learning repository 3 , and rescale the target variables and features of all datasets to fit in [0,1] and [-1,1] respectively. For each dataset, we randomly divide it into 10 subsets and each subset simulates the data on a client. We randomly permutate the instances in the datasets 10 times and report the average results. All algorithms are implemented with R on a Windows machine with 2.8 GHz Core(TM) i7-1165G7 CPU 4 .

We use the square loss function and define the mean squared error (MSE) of all algorithms, i.e.,

We record the mean of MSE over 10 random experiments, and the standard deviation of the mean of MSE. We also record the mean of of the total running time on each client, and the standard deviation of the mean of running time.

C.2 Results of the First and the Second Experiment

We summary the experimental results of the first and the second experiments in Table 3.

In Table 3, ∆ is defined as the difference of MSE between NCO-OM and FOMD-OMS. Thus ∆ shows whether collaboration improves the prediction performance of the noncooperative algorithm. Times (s) records the total running time on all clients.

We first consider the case J = K in which the per-round time complexity on each client is O ( K ) . It is obvious that the MSE of NCO-OMS is similar with that of FOMD-OMS. Although there are four datasets on which FOMD-OMS performs better than NCO-OMS, such as the elevator , bank , Year and Slice datsets, the improvement is very limited. Beside, the value of ∆ is very small. Thus collaboration does not significantly improve the prediction performance of the noncooperative algorithm. The results verify the first goal G 1.

Next we consider the case J = 2 in which the per-round time complexity on each client is O (1) . It is obvious that FOMD-OMS performs better than NCO-OMS on all datasets. Besides, the value of ∆ in the case of J = 2 is much larger than that in the case of J = K , such as the elevators , ailerons , ailerons and Year datasets. Thus collaboration indeed improves the prediction performance of the noncooperative algorithm. The results verify the second goal G 2.

Finally we compare the running time of all algorithms. It is obvious that FOMD-OMS with J = 2 runs faster than the other algorithms. The results coincide with our theoretical analysis. NCO-OMS runs slower than FOMD-OMS. The reason is that NCO-OMS must solve the sampling probability p t using an additional binary search on each client (see Section G.1). In other words, NCO-OMS must execute binary search M times at each round. FOMD-OMS only executes one binary search on server at each round. The improvement on the computational cost is benefit from decoupling model selection and prediction.

C.3 Results of the Third Experiment

We summary the experimental results of the third experiment in Table 4.

We first compare FOMD-OMS with eM-KOFL. As a whole, the MSE of the two algorithms is similar. On the TomsHardware , Twitter and ailerons datasets, eM-KOFL enjoys slightly better prediction performance than FOMD-OMS. However, the running time of eM-KOFL is much larger than that of FOMD-OMS. The results coincide with the theoretical observations that FOMD-OMS enjoys a similar regret bound with eM-KOFL at a much smaller computational cost on the clients.

Next we compare FOMD-OMS with POF-MKL. Both the MSE and running time of FOMD-OMS are much smaller than that of POF-MKL. The results coincide with the theoretical observations that FOMD-OMS enjoys a smaller regret bound than POF-MKL at a much smaller computational cost on the clients.

Thus the results in Table 4 verifies the third goal G 3.

Finally, we explain that why POF-MKL performs worse than FOMD-OMS. There are three reasons.

POF-MKL does not use federated learning to learn a global probability distribution denoted by p t , but learns a personalized probability distribution denoted by p t,j on each client. Thus POF-MKL converges to the best kernel function at a lower rate.

POF-MKL uniformly samples two kernel functions and then learns two global hypotheses, while FOMD-OMS uses p t to sample a kernel function and learns a global hypothesis. Thus POF-MKL can learn a better global hypothesis.

On each client, POF-MKL executes model selection and combines the predictions of K hypotheses using p t,j . Thus the time complexity is in O ( DK ) . FOMD-OMS executes model selection on server, and only uses the sampled hypothesis to make prediction. Thus the time complexity on each client is in O ( D ) .

D A Federated Algorithm for OMS-DecD with Communication Constraints

Let R < T . Clients communicate with server every N rounds. At the r -th communication, clients transmit { 1 N ∑ t ∈ T r ∇ ( j ) t,i , 1 N ∑ t ∈ T r c ( j ) t,i } i ∈ O ( j ) t to server at the last round in T r . Then server updates sampling probabilities and hypotheses. We give the pseudo-code in Algorithm 3.

Theorem 5. Let R < T . For any t ∈ [ R ] , let

Under the condition of Theorem 2, with probability at least 1 -Θ ( T R M log( R ) + T R log( KR/M ) ) · δ , the regret of FOMD-OMS satisfies: ∀ i ∈ [ K ] ,

The regret bound depends on O ( 1 √ R ) . Thus FOMD-OMS explicitly balances the prediction performance and the communication cost.

Proof. If FOMD-OMS runs on a sequence of examples with length T = R , then Theorem 2 gives, with probability at least 1 -Θ( M log( CR ) + log( CKR/M )) · δ ,

According to Theorem 1, the regret bound of FOMD-OMS with R < T satisfies, with probability at least 1 -Θ ( T R M log( CR ) + T R log( CKR/M ) ) · δ ,

which concludes the proof.

E Proof of Theorem 1

We first state a technical lemma.

Lemma3 (Boyd and Vandenberghe [2004]) . Assuming that ψ ( · ) : X → R is a convex and differential function, and X is a convex domain. Let f ∗ = argmin f ∈X ψ ( f ) . Then it must be

Lemma 3 gives the first-order optimality condition.

Proof of Theorem 1. We first consider the case R = T .

The main idea is to give an lower bound and upper bound on 〈 ¯ g t , u t +1 -v 〉 , respectively. Next we give an upper bound.

The last inequality comes from Lemma 3.

Then we give a lower bound.

where u ( j ) t = u t .

Next we analyze Ξ 1 and Ξ 2 .

To analyze Ξ 1 , we introduce an auxiliary variable r t +1 defined as follows

Then we have

Before analyzing Ξ 2 , we also introduce an auxiliary variable q t +1 defined as follows

Now we can analyze Ξ 2 . We have

Combining the lower bound and upper bound gives

Using the convexity of l ( j ) t , that is, l ( j ) t ( u ( j ) t ) -l ( j ) t ( v ) ≤ 〈 g ( j ) t , p ( j ) t -v 〉 , we further obtain

which concludes the proof.

Now we consider the case R < T .

For any batch T r , r = 1 , . . . , R , we define a new loss function ¯ l ( j ) rN ( · ) at the end of this batch,

During each batch, our algorithmic framework does not change the decision, i.e.,

Thus the regret can be decomposed as follows,

Now we can use FOMD-No-LU with R = T to the new loss functions { ¯ l (1) rN , . . . , ¯ l ( M ) rN } r =1 ,...,R , and use Theorem 1 to obtain

which concludes the proof.

F Proof of Lemma 1

Lemma 4 (Bernstein's inequality for martingale) . Let X 1 , . . . , X n be a bounded martingale difference sequence w.r.t. the filtration H = ( H k ) 1 ≤ k ≤ n and with | X k | ≤ a . Let Z t = ∑ t k =1 X k be the associated martingale. Denote the sum of the conditional variances by

Then for all constants a, v > 0 , with probability at least 1 -δ ,

Note that v must be a constant. Lemma 4 is derived from Lemma A.8 in [Cesa-Bianchi and Lugosi, 2006].

Proof. Let v ∈ [0 , B ] be a random variable and B ≥ 2 is a constant. We use the well-known peeling technique [Bartlett et al., 2005]. We divide the interval [0 , B ] as follows

First, we consider the case v > 2 -glyph[ceilingleft] log B glyph[ceilingright] . Let

We decompose the random event as follows,

in which we use Lemma 4 for each sub-event.

Then we consider the case v ≤ 2 -glyph[ceilingleft] log B glyph[ceilingright] ≤ 1 B . Lemma 4 yields, with probability at least 1 -δ ,

Combining the two cases, with probability at least 1 -2 glyph[ceilingleft] log B glyph[ceilingright] δ ,

which concludes the proof.

G Properties of Online Mirror Descent (OMD)

G.1 OMDwith the weighted negative entropy regularizer

Let Ω = ∆ K and ψ t ( p ) = ∑ K i =1 C i η t p i ln p i . Then we have

The Bregman divergence associated with the negative entropy regularizer is

Denote by ¯ c t = 1 M ∑ M j =1 ˜ c ( j ) t . Recalling that the OMD is defined as follows,

Substituting into the gradient of ψ t , the mirror updating can be simplified.

Now we use the Lagrangian multiplier method to solve the projection associated with Bregman divergence.

Let p t +1 , λ ∗ and { β ∗ i } K i =1 be the optimal solution. By the KKT conditions, we have

in which the last equality comes from p t +1 ,i > 0 . The reason is that by the fact p 1 ,i > 0 for all i ∈ [ K ] , we can iteratively prove that p t,i > 0 and p t +1 ,i > 0 , satisfying the condition under which β ∗ i = 0 . Then we can obtain the solution p t +1 , i.e.,

Next we prove that λ ∗ can be found by the binary search.

∗

Thus it must be -max i ¯ c t,i ≤ λ ≤ 0 . For any 0 ≥ λ 1 ≥ λ 2 ≥ -max i ¯ c t,i , we can obtain

Thus ∑ K i =1 p t,i · exp ( -η t ( λ ∗ +¯ c t,i ) C i ) is non-increasing w.r.t. λ ∗ .

We can use the binary search to find λ ∗ .

G.2 OMDwith the Euclidean regularizer

Let Ω = F i and ψ t,i ( w ) = 1 2 λ t,i ‖ w ‖ 2 2 . Then we have

Recalling that the OMD is defined as follows,

The mirror updating is as follows,

Thus OMD with the Euclidean regularizer is online gradient descent [Zinkevich, 2003].

H Proof of Lemma 2

Recalling that c ( j ) t,i = glyph[lscript] ( f ( j ) t,i ( x ( j ) t ) , y ( j ) t ) , in which

Since | y ( j ) t | is uniformly bounded for all j ∈ [ M ] and t ∈ [ T ] , there is a constant C i that depends on U i and b i such that c ( j ) t,i ≤ C i .

Recalling that ∇ ( j ) t,i = glyph[lscript] ′ ( f ( j ) t,i ( x ( j ) t ) , y ( j ) t ) · φ i ( x ( j ) t ) . Since glyph[lscript] ( f ( j ) t,i ( x ( j ) t ) , y ( j ) t ) can be upper bounded by C i and ‖ φ i ( x ( j ) t ) ‖ 2 ≤ b i , there is a constant G i that depends on U i and b i such that ‖∇ ( j ) t,i ‖ 2 ≤ G i .

I Proof of Theorem 2

The regret w.r.t. any f ∈ F i can be decomposed as follows.

Next we separately give an upper bound on Ξ 4 and Ξ 5 .

I.1 Analyzing Ξ 4

We start with Lemma 1 and instantiate some notations.

Lemma 1 gives, ∀ v ∈ ∆ K ,

Recalling that

In (13), we redefine Ω = ∆ K and ψ t ( p ) = ∑ K i =1 C i η t p i ln p i , and in (14), we redefine Ω = ∆ K and ψ t ( p ) = ∑ K i =1 2 C i η t p i ln p i . Using the results in Section G.1, we can obtain

It can be verified that δ t,i ∈ [ -2 C i , 2 K -J J -1 C i ] and ˆ c t,i ∈ [0 , 2 C i ] .

Recalling the definition of learning rate η t in Theorem 2. We can obtain η t δ t,i C i ≥ -1 and η t ˆ c t,i 2 C i ≥ -1 . Next we use (10) and (15) to analyze the following two Bregman divergences.

and

in where we use the fact exp( -x ) ≤ 1 -x + x 2 for all x ≥ -1 . Substituting the two upper bounds into (12) gives

Bounding Ξ 4 , 1

We define a random variable X t as follows,

Let H t = { O (1) t , . . . , O ( M ) t } . Then we have E [ X t | H [ t -1] ] = 0 and | X t | ≤ C where C = max i C i . Thus X [ T ] is a bounded martingale difference sequence w.r.t. the filtration H [ T ] . The sum of condition variance satisfies

The upper bound is a random variable. Lemma 1 yields, with probability at least 1 -M log( C 2 T ) δ ,

where the fail probability comes from the union-of-events, and the lower order term M √ 2 C 2 T ln 1 δ is omitted.

Bounding Ξ 4 , 2

According to (10), we have

Bounding Ξ 4 , 3

We define a random variable X t as follows,

It can be verified that E [ X t | H [ t -1] ] = 0 and | X t | ≤ K -J J -1 C . Next we upper bound the sum of condition variance.

where we use the fact ˜ c ( j ) t,i = c ( j ) t,i P [ i ∈ O ( j ) t ] ≥ K -1 J -1 c ( j ) t,i . Lemma 1 yields, with probability at least 1 -log( C 2 K 3 T/M ) δ ,

where g K,J = K -J J -1 and the lower order term √ 2 M g 3 K,J C 2 T ln 1 δ is omitted.

Bounding Ξ 4 , 4

We define a random variable X t as follows,

{ X t } T t =1 is a bounded martingale difference sequence and | X t | ≤ g K,J C . We further have glyph[negationslash]

where 〈 c ( j ) t ⊗ c ( j ) t , v 〉 = ∑ K i =1 v i ( c ( j ) t,i ) 2 .

With probability at least 1 -log(4 C 2 KT/M ) δ ,

For simplicity, we introduce some new notations

Combining all

Combining all gives, with probability at least 1 -Θ(log( CKT/M )) · δ ,

Rearranging terms gives

Recalling that, the solution of the following inequality

is x ≤ a 2 + b + a √ b . Solving for ¯ L T gives

Denote by A m = argmin i ∈ [ K ] C i . Let the learning rate and initial distribution p 1 satisfy glyph[negationslash]

Then we have

We further simplify ¯ L T -¯ L T ( v ) .

in which we omit the lower order terms such as O ( T 1 4 ) and O ( √ g K,J C ln 1 δ ) .

Finally, using the upper bound on Ξ 4 , 1 gives, with probability at least 1 -Θ( M log( CT ) + log( CKT/M )) · δ , Ξ 4

I.2 Analyzing Ξ 5

We also start with Lemma 1.

We just a fixed i ∈ F i . We instantiate some notations.

Lemma 1 gives

where the Bregman divergence is

Thus we have: ∀ w ∈ F i ,

1

M

Ξ

5

Next we separately give a high-probability upper bound on Ξ 4 , 1 and Ξ 4 , 2 .

Bounding Ξ 5 , 2

We define a random variable X t as follows,

X [ T ] is a bounded martingale difference sequence w.r.t. H [ T ] and | X t | ≤ 2 g K,J G i U i . We further have

The upper bound on the sum of conditional variance is a constant. Lemma 4 gives, with probability at least 1 -δ ,

Bounding Ξ 5 , 1

Recalling that

Besides, (13) and (14) can be instantiated as follows

It can be found that λ t,i ≤ U i 2 g K,J G i . Case 1 : T > g 2 K,J . We decompose Ξ 5 , 1 as follows,

We separately analyze Ξ 5 , 1 , 1 and Ξ 5 , 1 , 2 . Let

X [ T ] is a martingale difference sequence and satisfies | X t | ≤ λ t,i · g 2 K,J G 2 i ≤ g K,J U i G i 2 . We further have

With probability at least 1 -2 δ ,

Combining with all results gives, with probability at leat 1 -3 δ ,

Case 2 : T ≤ g 2 K,J .

In this case, we do not decompose Ξ 5 , 1 and λ t,i = U i 2 G i √ ( 1+ g K,J M ) g 2 K,J . With probability at least 1 -δ ,

Furthermore, with probability at least 1 -2 δ ,

Combining the two cases gives, with probability at least 1 -( M +5) δ ,

I.3 Combining all

Combining the upper bounds on Ξ 4 and Ξ 5 gives an upper bound on the regret. With probability at least 1 -Θ( M log( CT ) + log( CKT/M )) · δ ,

Omitting the constant terms and lower order terms concludes the proof.

J Proof of Theorem 3

We first establish a technical lemma.

Lemma 5. Let X 1 , ..., X K be a sequence of independent standard normal random variables. Let Z K = max { X 1 , ..., X K } . If K ≥ 5 , then E [ Z K ] ≥ ( 1 -1 √ e ) √ 2 ln K .

Proof of Lemma 5. Proposition 2.1.2 in Vershynin [2018] gives a lower bound on the tail probability of standard normal distribution.

Then we have

Let ε = √ 2 ln K . If K > 5 , then we have

Substituting into the lower bound of E [ Z K ] concludes the proof.

J.1 Proof of the First Lower Bound

Proof. Let d ≥ K , X ⊆ R d and Y ∈ { 0 , 1 } . We use the absolute loss function glyph[lscript] ( f ( x t ) , y t ) = | f ( x t ) -y t | . Recalling that

where e i is the standard basis vector in R d . It is obvious that the time complexity of computing f i ( x ) = x i is O (1) . At each client j , let the selected hypothesis be f ( j ) t and the prediction be f ( j ) t ( x ( j ) t ) . Since there are no computational constraints on each client, f ( j ) t ( x ( j ) t ) can be a weighted combination of K predictions, i.e., f ( j ) t ( x ( j ) t ) = ∑ K i =1 w ( j ) t,i f i ( x ( j ) t ) . The time complexity of computing f ( j ) t ( x ( j ) t ) is O ( K ) . We will follow the techniques used in the proof of Theorem 3.1 in Patel et al. [2023] and Theorem 3.7 in Cesa-Bianchi and Lugosi [2006].

Following the proof of Theorem 3.1 in Patel et al. [2023], the adversary gives a sequence of same examples for each client. To be specific, we define

where x t = ( b t, 1 , b t, 2 , ..., b t,K , 0 , . . . , 0) ∈ R d , and b t, 1 , b t, 2 , ..., b t,K , y t is a sequence of symmetric i.i.d. Bernoulli random variables, i.e., P [ y t = 1] = P [ y t = 0] = 1 2 .

At any round t , the minimax regret against the best hypothesis can be simplified as follows

in which f i ( x t ) = b t,i is a Bernoulli random variable and

We further obtain

where both { Z t,i } t ∈ [ T ] ,i ∈ [ K ] and { σ t } t ∈ [ T ] are i.i.d. Rademacher random variables.

By Lemma A.11 in Cesa-Bianchi and Lugosi [2006], we obtain

where G 1 , ..., G N are independent standard normal random variables. By Lemma 5, we obtain

which concludes the proof.

J.2 Proof of the Second Lower Bound

We mainly use the techniques in the proof of Theorem 2 in Seldin et al. [2014], but also require a new technique. The idea of our proof is to reduce the online model selection on each client to multi-armed bandit problem with additional observations.

Proof. Now we prove the second lower bound in Theorem 3.

Let d ≥ K , X ⊆ R d and Y ∈ { 0 , 1 } . We use a linear loss function glyph[lscript] ( f ( x t ) , y t ) = 1 -y t f ( x t ) . Recalling that

It is obvious that the time complexity of computing f i ( x ) = x i is O (1) . Under the constraint that the time complexity on each client is limited to O ( J ) , on each client, any algorithm can only select J hypotheses and then output a prediction.

One of challenges is that the prediction may be a combination of J predictions. To be specific, for each client j ∈ [ M ] , f ( j ) t ( x ( j ) t ) = ∑ i ∈ O ( j ) t w t,i f i ( x ( j ) t ) , where O ( j ) t contains the index of selected J hypotheses by some algorithm. To address this challenge, we introduce a virtual strategy that randomly selects a hypothesis f ( j ) I ( j ) t ∈ { f A t, 1 , f A t, 2 , ..., f A t,J } following the distribution

( w t,A t, 1 , w t,A t, 2 , ..., w t,A t,J ) where A t,a ∈ O ( j ) t , a = 1 , ..., J . Since the loss function is a linear function, it is easy to prove that,

where the expectation is taken over I ( j ) t . Assuming that glyph[lscript] ( f i ( x ( j ) t ) , y ( j ) t ) ≤ C for all i = 1 , ..., K . Lemma A.7 in Cesa-Bianchi and Lugosi [2006] gives, with probability at least 1 -δ ,

Note that we assume w t,i ≥ 0 and ∑ i ∈ O ( j ) t w t,i = 1 for all t = 1 , ..., T . Recalling that Theorem 3 assumes the outputs of algorithm belong to [min i ∈ [ K ] , x ∈X f i ( x ) , max i ∈ [ K ] , x ∈X f i ( x )] . If w t,i < 0 or ∑ i ∈ O ( j ) t w t,i > 1 , we can still find a weight vector w ′ t,i ≥ 0 and ∑ i ∈ O ( j ) t w ′ t,i = 1 , such that

Then we sample I ( j ) t following ( w ′ t,A t, 1 , w ′ t,A t, 2 , ..., w ′ t,A t,J ) . We can replace ( w t,A t, 1 , w t,A t, 2 , ..., w t,A t,J ) with ( w ′ t,A t, 1 , w ′ t,A t, 2 , ..., w ′ t,A t,J ) .

Since the algorithm is non-cooperative, the total regret can be decomposed into the summation of the regret on each client. With probability at least 1 -Mδ ,

Note that if we take expectation w.r.t. the randomness of algorithm, then the additional term C √ T 2 ln 1 δ in (17) can be omitted following (16).

If the prediction is not a combination of J predictions, but just f ( j ) I ( j ) t ( x ( j ) t ) , then we have

Combining with the two cases, we just need to analyze Reg D ( F i ) .

The adversary first uniformly samples a same h ∈ [ K ] for all clients, and then constructs { ( x ( j ) t , y t ) } T t =1 as follows

in which b t,i satisfies

glyph[negationslash]

Let E h [ · ] and P h [ · ] separately be the expectation and probability operator conditioned on h is selected. Then we have glyph[negationslash]

It is obvious that online model selection can be reduced to a K -armed bandit problem, in which f i is the i -th arm. At each round t , let I ( j ) t be the selected arm. Besides, any algorithm can select another J -1 arms. Thus any algorithm can observe J losses. Let O ( j ) t be the set of the selected J arms. Note that f ( j ) I ( j ) t = f I ( j ) t for any I ( j ) t ∈ [ K ] .

Assuming that the algorithm is deterministic, that is, O ( j ) t is determined by { O ( j ) τ } t -1 τ =1 and the observed losses. Let N T,i = ∑ T t =1 I I ( j ) t = i . Taking expectation w.r.t. ( b t, 1 , ..., b t,K ) T t =1 yields glyph[negationslash]

Following the techniques in the proof of Theorem 2 in Seldin et al. [2014], we have

Recalling that T ≥ K ≥ 5 . Let ρ = √ K 3 √ JT . We further have

For any deterministic algorithm, we can prove

Algorithm 4 NCO-OMS

where the last inequality comes from (19). As claimed in the proof of Theorem 6.11 in CesaBianchi and Lugosi [2006], the lower bound of any randomized algorithm is same with that of any deterministic algorithm, i.e.,

in which the expectation is taken over the internal randomness of algorithm. Substituting into (17), or (18) concludes the proof.

K Regret Analysis of NCO-OMS

Algorithm 4 gives the pseudo-code of NCO-OMS.

Following the definition of NCO-OMS and Algorithm 4, it is obvious that the regret bound of NCO-OMS on each client is same with Theorem 2 in which we set M = 1 . The regret bound on M clients is M times of that of a client. Thus we have Theorem 6.

Theorem 6 (Regret Bound of NCO-OMS) . Let the learning rate η , λ t,i and the initial distribution p 1 be same for each client j ∈ [ M ] . The values of η , λ t,i and p 1 follow Theorem 2 in which M = 1 . With probability at least 1 -Θ( M log( KT )) · δ , the regret of NCO-OMS satisfies:

L Proof of Theorem 4

L.1 Algorithm

We give the pseudo-code in Algorithm 5.

To implement Algorithm 5, we require one more technique, i.e., the random features [Rahimi and Recht, 2007]. We will use the random features to construct an approximation of the implicity kernel mapping. The are two reasons. The first one is that we can avoid transferring the data itself and thus the privacy is protected. The second one is that we can avoid the O ( T ) computational cost on the clients.

For any i ∈ [ K ] , we consider the kernel function κ i ( x , v ) that has an integral representation, i.e.,

where ϕ i : X × Γ → R is the eigenfunctions and µ i ( · ) is a distribution function on Γ . Let p i ( · ) be the density function of µ i ( · ) . We sample { ω j } D j =1 ∼ p i ( ω ) independently and compute

For any f ( x ) = ∫ Γ α ( ω ) ϕ i ( x , ω ) p i ( ω )d ω . We can approximate f ( x ) by ˆ f ( x ) = 1 D ∑ D j =1 α ( ω j ) ϕ i ( x , ω j ) . It can be verified that E [ ˆ f ( x )] = f ( x ) . Such an approximation scheme also defines an explicit feature mapping denoted by

For each κ i , i ∈ [ K ] , we define two hypothesis spaces [Rahimi and Recht, 2008, Li and Liao, 2022] as follows

in which F i is exact the hypothesis space defined in (1).

It can be verified that ‖ w ‖ 2 2 ≤ U 2 i . Let W i = { w ∈ R D : ‖ w ‖ ∞ ≤ U i √ D } . We replace (9) with (22),

L.2 Regret Analysis

We first give an assumption and a technique lemma.

Assumption 2 (Li et al. [2019]) . For any i ∈ [ K ] , if κ i satisfies (20) , then there is a bounded constant b i such that, ∀ x ∈ X , | ϕ i ( x , ω ) | ≤ b i .

Under Assumption 2, we have | f ( x ) | ≤ U i b i for any f ∈ H i and f ∈ F i . It is worth mentioning that if Assumption 2 holds, then Assumption 1 holds with the same b i .

Lemma 6. For any i ∈ [ K ] , let F i and H i follow (21) . With probability at least 1 -δ , ∀ f ∈ F i , there is a ˆ f ∈ H i such that | f ( x ) -ˆ f ( x ) | ≤ Ub i √ D √ 2 ln 1 δ .

The lemma is adopted from Lemma 5 in [Li and Liao, 2023]. Thus we omit the proof.

Now we begin to prove Theorem 4.

Algorithm 5 FOMD-OMS for Distributed OMKL

Proof of Theorem 4. The regret w.r.t. any f ∈ F i can be decomposed as follows.

Reg D ( H i ) is the regret that we run FOMD-OKS with hypothesis spaces { H i } K i =1 . ˆ f ∈ H i satisfies Lemma 6. In other words, Ξ 6 is induced by the approximation error that we use ˆ f to approximate f . Reg D ( H i ) has been given by Theorem 5. Next we analyze Ξ 6 .

Using the convexity of glyph[lscript] ( · , · ) , with probability at least 1 -TMδ ,

Under Assumption 2, there is a constant g i such that ∣ ∣ ∣ ∣ d glyph[lscript] ( ˆ f ( x ( j ) t ) ,y ( j ) t ) d ˆ f ( x ( j ) t ) ∣ ∣ ∣ ∣ ≤ g i . The last inequality comes from the definition of Lipschitz constant (see Lemma 2). Combining the upper bounds on Reg D ( H i ) and Ξ 6 concludes the proof.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We have clearly stated our main results, technical contributions and scope in the abstract and introduction. The main results are also supported by our theoretical and experimental results.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have discussed the limitations of this work in appendix (Section A).

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We have give all assumptions in the main paper. Due to the limited space, we do not include proof sketches in the main paper, but give a complete and correct proof for each theoretical result (including lemma and theorem) in appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We have clearly state the experimental setting and datasets, and give the datasets and code in supplemental material.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: All of the datasets used in this paper are public. The datasets and code of all algorithms are available at https://github.com/JunfLi-TJU/OMS-DecD.git .

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We have clearly specified the experimental setting in appendix (Section C) including basic information of datasets, the type of hyperparameters, the value of hyperparameters and so on.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We have reported the error bar which is the standard deviation of the mean. We run each experiment 10 times, record the mean of 10 random results (including MSE and running time) and the standard deviation of the mean.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We have stated the compute resources in the experimental setting (Section C). All algorithms are implemented with R on a Windows machine with 2.8 GHz Core(TM) i7-1165G7 CPU.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have reviewed the NeurIPS Code of Ethics and our research conforms with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We have included the broader impacts in appendix (Section B)

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We use public datasets and give the URL from which the datasets are downloaded.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We have provided the code of our algorithms with a simplified documentation available at https://github.com/JunfLi-TJU/OMS-DecD.git .

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
The paper studies the problem of regret minimization $ \min_{f_1, \ldots, f_T} \left( \sum_{t=1}^T \ell(f_t(x_t), y_t) - \min_{f \in \mathcal{F}^*} \sum_{t=1}^T \ell(f(x_t), y_t) \right)$ in distributed settings with $M$ clients. The paper proves lower bounds on regret and introduces a federated algorithm with an upper bound analysis. The paper proves collaboration is unnecessary without computational constraints on sites and necessary if the computation per client is bounded by $o(K)$ where $K$ is the number of hypotheses ($F$).

STRENGTHS:
The paper made a strong contribution. It proposes a new algorithm that improves previous bounds on computation and communication costs and establishes interesting lower bounds on algorithms. The work presents three new techniques: improved Bernstein's inequality for martingales, a federated online mirror descent framework, and decoupling model selection and prediction. The technical contribution is sold, and the paper is well-written. The paper contains extensive experimental studies that support theoretical bounds.

WEAKNESSES:
I didn't see any significant weaknesses.

QUESTIONS:
N/A

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-10

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper discusses the necessity of collaboration among clients in a decentralized data setting for online model selection. The authors analyze this necessity from the perspective of computational constraints and propose a new federated learning algorithm. They prove the upper and lower bounds of this algorithm. The main conclusion is that collaboration is unnecessary if there are no computational constraints on clients, but necessary if each client's computational cost is limited to o(K), where K is the number of candidate hypothesis spaces. The key contributions include establishing conditions under which collaboration is required, designing an improved federated learning algorithm (FOMD-OMS) separating model selection from client prediction, which improves regret bounds and enhances model selection performance at lower computational and communication costs.\n\nSTRENGTHS:\n1. The paper provides a rigorous theoretical analysis of the necessity of collaboration in federated learning for online model selection, considering computational constraints.\n2. The algorithm demonstrates improved regret bounds and performance in model selection while maintaining low computational and communication costs.\n\nWEAKNESSES:\n1. The motivation for federated learning in the context of online model selection with decentralized data is not clearly articulated. It would be beneficial to provide more details to justify the need for federated learning for practical applications, such as the online hyperparameter tuning mentioned in the example.\n2. As shown in Section 4.4, the exact communication cost for $o(K)$ is $O(MJ\\log K)$, where $J$ may have the same order with $K$ and introduce a large communication cost. It would be better to discuss the choice of $J$ and provide the exact communication cost in the abstract and introduction part.\n3. The paper does not sufficiently discuss the difference from previous works on federated online bandits. Both settings involve selection among K options (arms or hypothesis spaces), and a detailed comparison would highlight the unique aspects of this work.\n3. When considering setting $K=1$, [1] achieve results $O(M\\sqrt{T}+\\sqrt{MT})$ result for federated adversarial linear bandits setting, which has a similar form compared to the result of this work. It would be better to discuss the technique challenge in the proof process.\n\nQUESTIONS:\nPlease see the weaknesses part.\n\nQuestion 1: How does this work specifically differ from and improve upon federated online bandit algorithms? What are the unique challenges introduced with the online model selection setting?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations and societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

comments: SUMMARY:
The paper proposes a new online multi-kernel federated learning algorithm. The proposed method works based on estimating the gradients by the clients to make collaborations among clients for online federated learning helpful. The paper analyzes the regret bound of the proposed algorithm.

STRENGTHS:
1. The paper tries to address the important and interesting problem of benefit of collaboration in online model selection and online multi-kernel learning.
2. The paper provides complete comparisons between regret upper of the proposed algorithm and those of other works.
3. The paper is well-written and clear.

WEAKNESSES:
The proposed algorithm improves the regret of online multi-kernel learning by constant. Maybe adding some assumptions can help to achieve tighter regret bounds. Finding such conditions can be an interesting direction for future research.

QUESTIONS:
The paper states that some algorithms suffer from $\mathcal O(K)$ download cost. However, based on my understanding download cost may not be that significant. Can you explain the importance of download cost?

LIMITATIONS:
Limitations adequately discussed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-13

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper looks at the problem of Online Model Selection in a Decentralized context (OMS-DECD). The authors show the relationship of this problem class with multiple problem classes included Online Model Selection, federated learning, online multi-kernel learning etc. They consider two aspects of the problem 1. whether they can get a minimized regret for the OMS-DECD problem and 2. whether collaboration is necessary (between clients) to get privacy preserving version of regret. \nThe authors show initially a federated version of online mirror descent with no Local update which provides a bound on the regret. Additionally they develop a federated OMS algorithm which provides an upper bound on the regret and recreates the result of historical work. They also develop lower bound results for cooperative and non-cooperative algorithms. Finally they show that collaboration is unnecessary ( the best bounds are obtained without collaboration) when there are no computational constraints on the clients, thus generalizing the results of distributed online learning and collaboration is necessary if the computation cost is constrained to o(K) where K is the number of clients. Along the way they develop a new Berstein's inequality for a Martingale to show the high probability bounds on the regret making it a stronger bounds than in expectation.\n\nSTRENGTHS:\nI have not checked the mathematical accuracy of the proofs but the results look novel and are well motivated in the context of recent work and connection with other recent results. This helps us provide a unique perspective about when collaboration is actually necessary (the authors claim that they are not required if we focus on only the statistical perspective and not the computational aspects of the algorithms) and also gives us high probability bounds on the regret.\n\nWEAKNESSES:\nThe paper is difficult to follow at times due to the mathematical density. It would be better presented by motivating the actual theorems in the paper and pushing the mathematical proofs to the appendix possibly. In particular, it was a bit difficult to follow the connection between FOMD-no-LU and FOMD-OMS and why the authors promoted the first algorithm. Having these details summarized in a table/section would make the paper much more readable.\n\nQUESTIONS:\nAddressed in the previous section. \nAdditionally I had a slightly naive question regarding how to compare the regret bound of the three algorithms in table 1. It seems like the FOMD-OMS is adding an additional term and can potentially have poorer regret if C and C_i are comparable. Are these regret bounds not to be compared as they might be due to different function classes?\n\nLIMITATIONS:\nNo specific limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 1, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-22"}