Communication Efficient Federated Learning for Generalized Linear Bandits
Authors: Chuanhao Li
Hongning Wang
Venue: NeurIPS 2022 Accept (2022)
Keywords: contextual bandit
generalized linear model
federated learning
communication efficiency

Abstract: Contextual bandit algorithms have been recently studied under the federated learning setting to satisfy the demand of keeping data decentralized and pushing the learning of bandit models to the client side. But limited by the required communication efficiency, existing solutions are restricted to linear models to exploit their closed-form solutions for parameter estimation. Such a restricted model choice greatly hampers these algorithms' practical utility. 
In this paper, we take the first step to addressing this challenge by studying generalized linear bandit models under the federated learning setting. We propose a communication-efficient solution framework that employs online regression for local update and offline regression for global update. We rigorously proved, though the setting is more general and challenging, our algorithm can attain sub-linear rate in both regret and communication cost, which is also validated by our extensive empirical evaluations.

Communication Efficient Federated Learning for Generalized Linear Bandits
Chuanhao Li Hongning Wang'
Department of Computer Science, University of Virginia {c15ev, hwwp::/wwwwwwiiiie..a @virginia, edu
Abstract
Contextual bandit algorithms have been recently studied under the federated learn- ing setting to satisfy the demand of keeping data decentralized and pushing the learning of bandit models to the client side. But limited by the required communi- cation efficiency, existing solutions are restricted to linear models to exploit their closed-form solutions for parameter estimation. Such restricted model choice greatly hampers these algorithms' practical utility. In this paper, we take the first step addressing this challenge by studying generalized linear bandit models un- solution
der the federated learning setting. We propose a Communication-effice framework that employs online regression for local update and offline regression for global update. We rigorously proved, though the setting is more general and chal- lenging, our algorithm can attain sub-linear rate in both regret and communication cost, which is also validated by our extensive empirical evaluations.
1 Introduction
As a classic model for sequential decision making problems, contextual bandit has been widely used for variety of real-world applications, including recommender systems [19], display advertisement [1 and clinical trials [7]. While most existing bandit solutions are designed under a centralized setting (i.e., data is readily available at a central server), in response the increasing application scale and public concerns of privacy, there is increasing research effort on federated bandit learning [71. where N clients collaborate with limited communication bandwidth to minimize the overall cumulative regret incurred over a finite time horizon T, while keeping each client's raw data local. Compared with standard federated learning 23 13
lately (28 6 27
that works with fixed datasets, federated bandit learning is characterized by its online interactions with the environment,
which continuously provides new data samples the clients over time. This brings in new challenges in addressing the conflict between the need of timely data/model aggregation for regret minimization and the need of communication efficiency with decentralized data. carefully designed model update method and communication strategy become vital to strike this balance.
Existing federated bandit learning solutions only partially addressed this challenge by considering simple bandit models, like context-free bandit (27] and contextual linear bandit (28 6 17,, where
closed-form solution for both local and global model update exists. Therefore, efficient communica- tion for global bandit model update is realized by directly aggregating local sufficient statisties, such that the only concern left is how control the communication frequency over time horizon T. How- ever, such a solution framework does not apply to the more complicated bandit models that are often preferred in practice, such generalized linear bandit (GLB) [8] or neural bandit [30], where only iterative solutions exist for parameter estimation (c.g., gradient-bassee optimization). To enable joint model estimation, now the learning system needs to solve distributed optimization for multiple times as new data is collected from the environment, and each requires iterative gradient/model aggregation among clients. This is much more expensive compared with linear models, and naturally leads to the question: whether a communication efficient solution to this challenging problem is still possible?
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

In this paper, we answer this question affirmatively by proposing the first provably communication efficient algorithm for federated GLB that only requires ((VT) communication cost, while still attaining the optimal order of regret. Our proposed algorithm employs a combination of online and offline regression, with online regression adjusting each client's model using its newly collected data, and offline (distributed) regression occasionally soliciting local gradients from all N clients for joint model estimation when sufficient amount of new data has been accumulated. In order to balance exploration and exploitation in arm selection, we propose a novel way to construct the confidence set based on the sequence of aolline-add-olinee model updates that each client has received. The initialization of online regression with offline regression introduces dependencies that break the standard martingale argument, which requires proof techniques unique to this paper.
We also explored other non-trivial solution ideas to further justify our current design. Specifically, in practice, a common way to update the deployed model for applications with streaming data is to set schedule and periodically re-train the model using iterative optimization methods. For comparison. we propose and rigorously analyze a federated GLB algorithm designed based on this idea, as well as a variant that further enables online updates on the clients. We also consider another solution idea motivated by distributed/batched online convex optimization, which is characterized by lazy online updates over batches of data. Moreover, extensive empirical evaluations on both synthetic and real-world datasets are performed to validate the effectiveness of our algorithm.
2
Related Work
GLB, as an important extension of linear bandit models, has demonstrated encouraging performance in modeling binary rewards (such as clicks) that are ubiquitous in real-world applications [18) The study of GLB under a centralized setting dates back to Filippi et al. [8), who proposed UCB-type algorithm that achieved ((dVT) regret. Li et al. (00 later proposed two improvements: a similar UCB-type algorithm that improves the result of by a factor of Olog T),, which has been popularly used in practice as it avoids the projection step needed in 8 and another impractical algorithm that further improves the result by a factor of ((vd) assuming fixed number of arms. To improve the time and space complexity of the aforementioned GLB algorithms, followup works adopted online regression methods. In particular, motivated by the online-to-contee to-confidencc-see conversion technique from (2). Jun et al. [22 proposed both UCB and Thompsan sampling algorithms with online Newton step, and Ding et al. proposed a Thompson sampling algorithm with online gradient descent, which, however, requires an additional context regularity assumption to obtain a sub-linear regret.
GLB under fedtrraed//issriiuttee setting still remains under-explored. The most related works are the federated/distributeed linear bandits [16] 28 6 [7]. In these works, thanks to the existence of closed-fore solution for linear models, the clients only communicate their local sufficient statistics for global model update. Korda et al. [16) considered a peer-to-peer (P2P) communication network and assumed the clients form clusters, i.c., each cluster is associated with a unique bandit problem. But as they only focused on reducing per-round communication, the communication cost is still linear over time. Huang et al. [II considered a star-shaped communication network as in our paper, but their proposed phase- based elimination algorithm only works in fixed arm set setting. The closest works to ours are 2x 6 17 which uses event-triggered communication protocols to obtain sub-linear communication cost over time for federated linear bandit with a time-varying arm set.
Another related line of research is the standard federated learning that considers offline supervised learning problems [3]. Since its debut in (23. FedAvg has become the most popularly used algorithm for offline federated learning. However, despite its popularity, several works [22 14] (4] identified that FedAvg suffers from a client-drift problem when the clients' data are non- (which is an important signature of our case), i.e., local iterates in each client drift towards their local minimum. This leads sub-optimal convergence rate of FedAvg: for example, one has to suffer a sub-linear convergence rate for strongly convex and smooth losses, though a linear convergence rate is expected under a centralized setting. To alleviate this, Pathak and Wainwright (26) proposed an operator splitting procedure to guarantee linear convergence to neighborhood of the global minimum. Later, Mitra et al. (24 introduced variance reduction techniques to guarantee exact linear convergence to the global minimum.
2

---

3 Preliminaries
In this section, we first introduce the general problem formulation of federated bandit learning, and discuss the existing solutions under the linear reward assumption. Then we formulate the federated GLB problem considered in this paper, followed by detailed discussions about the new challenges compared with its linear counterpart.
3.1 Federated Bandit Learning
Consider learning system with 1) N clients responsible for taking actions and receiving correspond- ing reward feedback from the environment, e.g., each client being an edge device directly interacting with a user, and 2) a central server responsible for coordinating the communication between the clients for joint model estimation.
At each time step manner, i.e., each client i E [M] chooses an arm X from its time-varying candidate set Ar,s = (x (2) "........ x 1 where x!") E Rd denotes the context vector associated with the a-th arm a,
2,2,..., all N clients interact with the environment in round-robin
for client at time t. Without loss of generality, we assume ||x] ll2 I 1, a,t, Then client receives the corresponding reward y/r, E IR from the environment, which is drawn from the reward
distribution governed by an unknown parameter 0, R2 (assume ||o,]| s S), i.e. yt.s ~ Pe. (v/xx (a)) The interaction between the learning system and the enviromment repeats itself, and the goal of the learning system is to minimize the cumulative (pseudo) regret over all N clients in the finite time horizon T, i..., R Ei= E T',.s, where rt, A., Ely[x]
E Ex,..
In federated learning setting, the clients cannot directly communicate with each other, but through the central server, i.c., star-shaped communication network. Raw data collected by each client i [N]. i.e., {(x..;: )).,))) ee[T] is stored locally and cannot be shared with anyone else. Instead, the clients can only communicate the parameters of the learning algorithm, e.g., models, gradients, or sufficient statistics; and the communication cost measured by the total number of times data being transferred across the system up to time I", which is denoted as Cr.
3.2 Federated Linear Bandit
Prior works have studied communication- federated linear bandit 228 6 i.e., the reward function a linear model Vi,, x_, + Tt,s, where Mt. denotes pero-mean nab-Gaussian noise. Consider an imaginary centralized agent that has direct access to the data of all clients. so that it can compute the global sufficient statistics A, EiENN) Evete x,
EvENN Eveta 3s, Then the cumulative regret incurred by this distributed learning system can match that under centralized setting, if all N clients select arms based on the global sufficient statistics {.A, d,). However, requires N2T communication cost for the immediate sharing each client's update to the sufficient statistics with all other clients, which is expensive for most applications.
To ensure communication efficiency, prior works like DisLinUCB 28 let each client maintain a local copy (A,-1, 6,-1, } for arm selection, which receives immediate local update using each newly collected data sample, i.c., Ar,i Ar + X,, bb.5 = b + Xt,iBt, Then client i det Ass
checks whether the event (t - t1s)) log( Tet. Aises
> D is true, where tast denotes the time step of last global update is triggered, such that the server will collect all clients'
global update. If true, local update since hast aggregate them to compute (A,, bt), and then synchronize the local sufficient statistics of all clients, i.c., set {Ar, br.i) = (A,, by Vi E [N].
3.3 Federated Generalized Linear Bandit
In this paper, we study federated bandit learning with generalized linear models, i.e., the conditional distribution reward y given context vector is drawn from the exponential family 20]
yx 0. mmxTxx O.)) + n((..))
po. (2/x) exp
(I)
g(T)
3

---

Client 1
pall Xy
02 020. Osu
Otinzel Orteze 41 S2g
Oxyest
Faviromentt
Server
get reward
61 Pzn ... P
Scasst
Btass No N... Poaee
Arm Selection
Local Updates
Giobaal Update
Local Updates
regressoon Oua
02-11 online regresion Ba
synchmolze P
Ocun
for 2 Ngur +
Figure 1: Illustration of FedGLB-UCB algorithm, which uses online regression for local update, i.e., immediately update each client's local model o,, using its newly collected data sample, and uses offline regression for global update, i.e., synchronize all N clients to globally updated model Ozw. using all the data samples collected far.
where 7 E IR is a known scale parameter. Given function f: R - R, we denote its first and second derivatives by and f, respectively. It is known that m 0.) = E xx] = ux( 0.), which called the inverse link function, and ri(x 0,) = V(ylx 0.). Based on Eq. the reward yr observed by client at time can be equivalently represented as Yt,i = ,xx,,, 0+) + T), where q), denotes the Gaussan noise. Then we denote the negative log-likelihood of Vi.t given X,,t as 1xxT,,0,
0, xt, = - log Ps. (x,, -,,,,,,,, 0, + m(x,,ee 0.). In addition, we adopt the following
two assumptions about the reward, which are standard for GLB [
Assumption 1. The link function μ continuously differentiable on (-S, S), Lipsschch on [-S, S). and inf=e[-5.s] ji(=) Cp 0.
Assumption 2. E[/r,,/FF,, = 0, VI, where Ft.i = o ( (Xx.j Us de)escnnn))))) denotes the o-algebra generated by client i's previously pulled arms and observed rewards, and max,, |Tt S
Rmax for some constant Rymxx > 0.
New Challenges Compared with federated linear bandit discussed in Section 3.2, arise in designing a coammunication-effiiie algorithm for federated GLB due the absence of closed form solution:
new challenges
Iterative communication for global update: compared with the global update for federated linear bandit that only requires one round of communication to share the sufficient statistics, now it takes multiple iterations of gradient aggregation obtain converged global optimiza- tion. Moreover, as the clients collect more data samples over time during bandit learning, the required number of iterations for convergence also increases.
Drifting issue with local update: during local model update, iterative optimization using only local gradient can push the updated model away from the global model, i.e., forget the knowledge gained during previous communications [1].
4 Methodology
In this section we propose the first algorithm for federated GLB that addresses the aforementioned challenges. We rigorously prove that attains sub-linear rate in T for both regret and communication cost. addition, we propose and analyze different variants of our algorithm to facilitate understanding of our algorithm design.
4.1 FedGLB-UCR Algorithm
To ensure communication-efficin model updates for federated GLB, we propose to use online regression for local update, i.e., update each client's local model only with its newly collected data samples, and use offline regression for global update, i.e., solicit all clients' local gradients for joint model estimation. Based on the resulting sequence of offline-anie andee--nd--niine model updates, the

---

confidence ellipsoid for 0, is constructed for each client to select arms using the OFUL principle. We name this algorithm Federated Generalized Linear Bandit with Upper Confidence Bound, or BedGLBBUCCE for short. We illustrate its key components in Figure and describe its procedures in Algorithm In the following, we discuss about each component of WedGLB-UCG in details.
Algorithm 1 FedGLB-UCB
Input: threshold D, regularization parameter À > E (0, and Cu. 2: Initialize Vi E [N] Ao,:
E Rddd bo. 0 E IR Po.:
Rd AAD RRdd,
Ag 2n= Rddd bo E R2 0o E IR blatt O Sp 3: for t f= 1, T do
4: for client i = 1, 2,.... N. do
5: Observe arm set Ab.; for client i
6: Select arm X4,; Ae,s by E5. and observe reward Ve,
7:
Update client Ar, A1-1,5 + x A AA--1 + x1,
8:
if (L has) log det den((Ar, AA,,) < D then
det(AA.))
9:
Client i: perform local update 0t,s ONS-Update(B--1111 Ae. Vl(x7,00011 Ve,s))
br, bb,1, ; ,
10:
else
11:
Clients Vi E [N]: send A to server, and reset AA,,
12:
Server: compute A; = Alm. + + AAri
13:
Server: perform global update 0, AGDD-UU Updatepdaue..... : Jt) (see E3. for the choice of J:), by = b + 2i=1 Ar,,0,, and set lans = Clients vi E [N]: set Oz,; = O2, Ar,i = Ae, be,i bt
14:
Local update. As mentioned earlier, iterative optimization over local dataset {(x,,i Vs.i)) sejo leads the drifting issue that pushes the updated model the local optimum. Due to the small size of this local dataset, the confidence ellipsoid centered at the converged model has increased width, which leads to increased regret bandit learning. However, as will prove in Section 4.2 completely disabling local update and restricting all clients to use the previous globally updated model for arm selection is also bad choice, because the learning system will then need more frequent global updates to adapt to the growing dataset.
To enable local update while alleviating the drifting issue, adopt online regression in each client, such that the local model estimation 0..; is only updated for one step using the sample (xx, Vr,s) collected time f. Prior works [2] [2] showed that UCB-type algorithms with online regression can attain comparable cumulative regret to the standard UCB-type algorithms 20), as long as the selected online regression method guarantees logarithmic online regret. As the negative log- likelihood loss defined in Section 3.33io exp-concave and online Newton step (ONS) is known to attain 12], ONS chosen for the local update of FedGLB-UCB and its description is given in Algorithm 2 At time step I, after client i pulls an arm X E Ar,
logarithmic online regret this case [10
and observes the reward y/i, its model 0,- 1,4 is immediately updated by the ONS update rule (line
in Algorithm 1. where Vl(xx,, [(xx,,0t_,, covariance matrix for client at time f.
-,,, Mt,a) denotes the gradient w.r.t. 0 and Ar,s denotes the
Algorithm 2 ONS-Update
Input: 0, Ag,s, V1(xx1,,0-111 M(,s)
2: 0r = 0,-1, - 1 Vx, 00-,, Ve.s)
3: Or,: arg (in) 110 - 02,e1 11.11
4: Output: 0r,
Global update The global update of FedGLB-UCB requires communication among the N clients, which imposes communication cost two aspects: 1) each global update for federated GLB requires multiple rounds of communication among N clients, i.e., iterative aggregation of local gradients; and 2) global update needs to be performed for multiple times over time horizon T, in order to adapt to the growing dataset collected by each client during bandit learning, Consider particular time step
5

---

t E [T] when global update happens, the distributed optimization objective is:
N
min Fi(O) = E
Fi,(0)
(2)
N 1=1
where (o) 3 1( 8, ys.s) + likelihood loss for client [N]. and X O denotes the regularization parameter. Based Assumption 1 (F): are NO -oosro and (kx + No. smooohh in & (proof in Appendix A). and we denote the unique minimizer of Eq.42) as OMLE. In this case, it is known that the number of communication rounds J, required to attain specified sub-optimality et, such that
À 1012 denotes the average regularized negative log- 21
F((0) - minsge (i) S Et, has lower bound J = ? ( (KkkNt)/A + log + B which means
Jr increases at least at the rate of VNt. This lower bound is matched by the distributed version of accelerated gradient descent (AGD) 25
J. < 1+ (ktNN + log (k4 + (10) 6
2E,
where the superscript (t) denotes the i-th iteration of AGD.
In order to minimize the number of communication rounds in one global update, AGD is chosen as the offline regression method for BaGGLB-UCCE and its description is given in Algorithm (subscript t is omitted for simplicity). However, other federated/distribute optimization methods can be readily used in place of AGD, as our analysis only requires the convergence result of the adopted method. We should note that E, is essential to the regret-communicattio trade-off during the global update at time t: a larger e, leads to a wider confidence ellipsoid, which increases regret, while a smaller et requires more communication rounds Jr, which increases communication cost. In Section 4.2 will discuss the proper choice of e to attain desired trade-off between the two conflicting objectives.
Algorithm 3 AGD-Update
1: Input: : initial 0, number of inner iterations
2: Initialization: set ((1) = ((1) = 6, and define the sequences {uy := 1+ 14-407
hE([] (with
ey 0), and {7, 1-0: 1 ]e())
3: for j jo 1,2,, J do
4: Clients compute and send local gradient (VF((OO)) (i())) to the server
5: Server aggregates local gradients VF(06)) = & E VF((())), and execute the following
update rule to get g(++1)
xU+1) 0(5)
ky, VF(D())
1
(6+1) (1 - -11-)))(-t) (10(tt )(0
6: Output: arg mngimnesssss)l (B(J+1)) (0))
To reduce the total number of global updates over time horizon T, we adopt the event-triggered communication from (28), such that global update triggered if the following event true for any client i [N] (line 8):
det(AA,))
- tast log det(Ar, AA > D
(4)
where A denotes client i's local update to its covariance matrix since last global update at fast and D > is the chosen threshold for the event-trigger. During the global update, the model estimation oe,i, covariance matrix Ar, and vector b,, for all clients E [[] will be updated (line 11-14). We should note that the LHS of Eq. is essentially
upper bound of the cumulative regret that client i's locally updated model has incurred since fast- Therefore, this event-trigger guarantees that a global update only happens when effective regret reduction is possible.
Arm selection To balance exploration and exploitation during bandit learning, FedGLB- uses the OFUL. principle for arm selection D]. which requires the construction of confidence ellipsoid for each client We propose novel construction of the confidence ellipsoid based on the sequence of model updates that each client i has received up to time t: basically, there are I)

---

one global update at Muss, i.., the joint offline regression across all clients' accumulated data till tast ((x,,i (ea.lei) which resets all clients' local models to fr..: and 2) multiple local updates from bast + 1 i.c., the online regression on client i's own data sequence {(x,,' ya,a) Jeltun + to get [ step by step. This can be more easily understood by the illustration in Figure
i The resulting confidence ellipsoid is centered at the ridge regression estimator dr,5 = A br,s [2[[2)). which is computed using the predicted rewards given by the past sequence of model updates 10 (0s.)))eeet. 1.4] (see the update of by,; in line and 13 of Algorithm D.. Then at time step t, client i selects the arm that maximizes the UCB score:
Xt, arg max x T 01-1,1+ + 10-1, xEAr,;
(5)
where 0t-1,; is the parameter of the confidence ellipsoid given in Lemma 4.2 Note that compared with standard federateeddissrriutee learning where clients only need communicate gradients for joint model estimation, in our problem, due to the time-varying arm set, it is also necessary to communicate the confidence ellipsoid among clients, i.e., A, E Redxd and by E TRd (line 14 in Algorithm|1 1), as the clients need to be prepared for all possible arms E Rd that may appear in future for the sake of regret minimization.
4.2 Theoretical Analysis
In this section, we construct the confidence ellipsoid based on the offlin--and-onliee estimators described in Section I.T Then analyze the cumulative regret and communication cost of FedGLB- UCB, followed by theoretical comparisons with its different variants.
Construction of confidence ellipsoid Compared with prior works that convert sequence of online regression estimators to confidence ellipsoid [2] 12], our confidence ellipsoid is built on the combination of an offline regression estimator t.... for global update, and the subsequent online regression estimators {0,-1.5 h +E[ for local updates on each client i. This construction is new and requires proof techniques unique our proposed solution. In the following, we highlight the key steps, and refer our readers to the appendix for details.
To simplify the use of notations, we assume without loss of generality that the global update at tast is triggered by the N-th client, such that no more new data will be collected at flast i.c. the first data sample obtained after the global update has index tast + 1. We start our construction by considering the following loss difference introduced by the global and local model updates: (x, v., where the first term is the loss difference between the globally updated model 6 and 6., and the second term is between the sequence of locally updated models {0.-1. 1, s 1,t] and 0.. This ; and due to the existence of
Ese Em [ O Ma.a) 1(xJ, vo.:)] Ei ((=x,,00=1... Va.s)
extends the definition of online regret used in the construction in [2 12):
offline regression, the obtained upper bounds in Lemma 4.1 are unique our solution.
Lemma 4.1 (Upper Bound of Loss Difference). Denote the sub-optimality of the global model update procedure at time step tiast as Crwe such that Fre (0) mingeBa(s)) Fia. S Ctur Then under Assumption and we have
N
[x Uo.i) 1xx Wo.t]] S Bi
(6)
where B₁ = Ntiast tio S², and with probability at least 1 - 5,
[[x] 1,5. O.x3...-1.. 1(x,, 0n, On, Monil S B2
(7)
s 1
where Bu
20p Ex=h |IVI(( 0s Vo.:) 12-1+ Su cu Rmax / log (1 + N Ni /dA) 2 2log (1/5)+
Et
2Ni
+ Niwery Et + Xea
2
respectively.
Specifically, Bi corresponds to the convergence of the offline (distributed) optimization in previous global update; B2 is essentially the online regret upper bound of ONS, with the major difference that it is initialized using the globally updated model Oewes instead of an arbitrary model as in standard
7

---

ONS. Then due to the
of [22, y) w.n. 2, i... 1(x, 0, vo) 1(x, A. Mo) 2
[u(x] 0.) Y/ x (0- 0.) -/2 [0 0.) 2 and by rearranging terms Eq. (6) and Eq. 7
we have: Σ Iue Ex=
N (0 0.)l S B1 2 E. EN 7,, 10 0.) and
Ex=teet1 xx,, (0,-,. 0.)]² S 2 B2+
6u Ex=ta+t1 X0. 0.), whose LHS is quadratic in 0.. To further upper bound the RHS, we should note that the term 2 Ee X, (0.-1, 0.) standard in
Sp
Gp.
as x (0, 8x) is F meesurraae for online estimator 0_ i However,
this is not true for the term E/m Ei=1 x (0 6+) as the offline regression estimator oram depends on all data samples collected till hust: and thus we have to develop different approach to bound it. This leads to Lemma 4.22 which provides the confidence ellipsoid for 0..
Lemma 4.2 (Confidence Ellipsoid of FedGIB-UUB)) With probability at least 1 26, for all t te [ E [N].
10 - 0.11,, S SB,s > A55 1| 1B 0 ai.
cp
where Zt,i denotes the vector of predicted rewards [xI, 0 x x X Orsmn Xun N Otim
T
Bant +, Oumer 1, taar 2, t -......... x, 01 and Br,s
sn
log(} $ v /dot(T 1 Eben EN
xx,xx T.))
of
B1
4Rmax
2 (2log(3 2 Elur En xx.))Ti)) (10mmllz
110112 vBi)
4B2
or
8. log
N
4+ cq
B2 64 +1
of.
Regret and communication cost From Lemma 4.2 we can see that 04 grows at rate of N / Caa through its dependence on the B2 term. To make sure the growth rate of og,; matches that in standard GLB algorithms (20 [2], we set Elgn
N which leads to the following corollary.
Corollary 4.2.1 (Order of Br,i). With Etam
Be = 0 dllo [12 + R)
N
Then using similar argument as the proof for Theorem 4 of (28) we obtain the following upper bounds on Rr and Cr for FedGLB- UCB (proof in Appendix D)
Theorem 4.3 (Regret and Communication Cost Upper Bound of FedGLB-UCB). Under Assumption and by setting a Nap Vt and D Ndlog(NTY) the cumulative regret Ry has upper bound
RT 00=0( [yplky + Rmmax) d dVNTlog(NT//)
Cp
with probability at least
25. The corresponding communication cost Cr upper bound
Cr (O(NNNTTlogg
Theorem 4.3 shows that FedGLB-UCB recovers the standard O (dv NT (og()))) rate in regret as in the centralized setting, while only incurring a communication cost that is sub-linear in T. Note that, log(
to obtain log (NT) regret for federated linear bandit, the DisLin algorithm incurs communication cost of (odN1 [og(NN)) (28), which smaller than that of FedGLB-UCB by factor of VNT (og(NT). . the frequency of global updates is the same for both algorithms (due to their use of the same event-trigger), this additional communication cost is caused by the iterative optimization procedure for the global update, which required for GL.B model estimation. Moreover, as we mentioned in Section 4. there is not much room for improvement here the use of AGD already matches the lower bound up to a logarithmic factor.
To facilitate the understanding of our algorithm design and investigate the impact of different components of FedGLB-UGE on its regret and communication efficiency trade-off, we propose
28 measure Cr by the total mumber of scalars transferred, in which case, we have Cr ( N log( NT) + d N2 INT)(N
This measured by the total number of times data transferred. Some works

---

Table 1: Comparison between FedGLB-UCB and its variants with different design choices.
Global Upd.
Local Lpd.
Setting
Ry
Cr dN VT (NV)) N9Tlog(NT)
AGD
ONS
D Nalog(NTy
Butly + limasl log(NT))
AGD
no update
B = /NT
Ky Auons de N/NT (NT(
on Bullg Bonaw) dy NT (NR( loR(T)
AGD
ONS
B d N (oog(NT)
&N (NT)) NIVT
ONS
ONS
B = VNT
Nelkki Rmar) d( ((NT) (Tg(
and analyze three variants, which are also of independent interest, and report the results in Table I Detailed descriptions, as well as proof for these results can be found in Appendix D Note that all three variants perform global update according to fixed schedule S = (tz
T B 1. :=
21 l.........t tB= == B| ff J}. where B denotes the total number of global updates specified in advance to trade-off between Rr and Cr, and these variants differ in their global and local update strategies. This comparison demonstrates that solution is proven to achieve a better regret- communicatinn trade-off against these reasonable alternatives. For example, when using standard federated learning methods (which assume fixed dataset) for streaming data in real-world applications, it is common practice to set some fixed schedule to periodically retrain the global model to fit the new dataset, and FedGL B implements such behaviors. The design of BUGG, is motivated by distributed online convex optimization that also deals with streaming data in a distributed setting.
5 Experiments
We performed extensive empirical evaluations of FedGLB-UCB both synthetic and real-world datasets, and the results (averaged over 10 runs) are reported in Figure 2 We included the three variants of FedGLB-UCB (listed in Table D. One-UCB-GLM, N-UCB-GLM (20) and N-ONS-GLM
12] as baselines, where One-UCB-GLM, learns a shared bandit model across all clients, and N-
UCB-GLM and N-ONS-GI.M learn separated bandit model for each client with no communication.
Additional results and discussions about experiments can be found in Appendix F
Synthetic Dataset We simulated the federated GLB setting defined in Section
3.3 with T
2000, .V- = 200, d = 10,5 = 1, A, (K = 25) uniformly sampled from a &g unit sphere, and reward Vt.. (ermoulli(((xx____e 0.)). with (==)
To compare the algorithms' Rr and Cr under different trade-off settings, we run FedGLB-UCB with different threshold value D (logarithmically spaced between 10-1 and 10º) and its variants with different number of global updates B. Note that each dot in the result figure illustrates the Cr (x-axis) and Rr (y-axis) that a particular instance of FedGLB-UCB or its variants obtained by time T, and the corresponding value for D or B is labeled next to the dot. Ry of One-UCB-GLMM is illustrated as the red horizontal line, and Rr of -LUBB-LMM and N-ONS-GLM are labeled on the top of the figure. We can observe that for FedGLB-UCB and its variants, Rr decreases as Or increases, interpolating between the two extreme cases: independently learned bandit models by N-UCB-GLM, N-ONS-GI -LMM; and the jointly learned bandit model by Ine-UCB-GLI low Ry, i.e., its regret even comparable with Ine-UCB-CC G.MM that requires at least Cr- N2T (8 x 10" in this simulation) for gradient aggregation at each time step.
(1 exp(-2))
FedGLB-UCE significantly reduces Cy, while attaining
Real-world Dataset The results above demonstrate the effectiveness of FedGLB-UCB when data is generated by a well-specified generalized linear model. To evaluate its performance in a more challenging and practical scenario. we performed experiments using real-world datasets: CoverType, MagicTelesscoe and Mushroom from the UCT Machine Learning Repository [5] To convert them to contextual bandit problems, we pre-processed these datasets following the steps in prior works (8) with T = 2000 and N = 20. Moreover, to demonstrate the advantage of GLB over linear model, we included DisLinUCB (28] as an additional baseline. Since the parameters being communicated in DnCL and FedGLB- BUC are different, to ensure a fair comparison of Cr in this experiment, we measure communication cost (x-axis) by the number of integers or real numbers transferred across the learning system (instead of the frequency of communications). Note that DisLinUCB has no Cr 2 x 10 Figure 2 because its global update already happening in every round and cannot be increased further. As mentioned earlier, due to the difference in messages being sent, the communication Disl DisliinUB's per global update is much smaller than that in FedGLB BC- However, because linear models failed to capture the complicated reward mappings in these three
9

---

8000
20
M-OKS-CLM Ry 30452
8000
7302
Fedoo 30-00
Fedo (0-000--00
x
6000
Cumuarive Rogre
6000
30.0
00000.0
7OO.0
Cumulastee Nagpi::
5,00
$co.0
4200
4000
x
20000.0
120 0
50
2000
#0000
2000
ga.
1000
107
Communication Cost
(a) Synthetic
(b) CoverType
3000
MUD GUM 2044
W-U''s GL So 765
4273
7002
2500
2000
5000
gone
Dumuatoo
1500
x
20
3000
1000
grnng
2000
3
500
0.0
0.2
0.4
0.6
0.8
1.0
1.2
Communicstion Cost
Commurication Cost
MagicTelescope
(d) Mushroom
Figure 2: Experiment results synthetic and real world datasets.
datasets, we can see that DisLinUCB is clearly outperformed by FedGLB-UCB and its variants. This shows that, by offering larger variety of modeling choices, e.g., linear, Poisson, logistic regression, etc., FedGLB-UCB has more potential in dealing with the complicated data in real-world applications.
Conclusion
In this paper, we take the first step to address the new challenges in communication efficient federated bandit learning beyond linear models, where closed-form solutions do not exist, and propose solution framework for federated GLB that employs online regression for local update and offline regression for global update. For arm selection, we propose novel confidence ellipsoid construction based on the sequence offline-and-onllin algorithm attains sub-linear rate for both regret and communication cost, and also analyze the impact of each component of our algorithm via thcoretical comparison with different variants. In addition, extensive empirical evaluations are performed to validate the effectiveness of our algorithm.
model estimations. We rigorously prove that the proposed
important further direction of this work is the lower bound analysis for the communication cost, analogous to the communication lower bound for standard distributed optimization by Arjevani and Shamir [3] Moreover, in our algorithm, clients' locally updated models are not utilized for global model update, so that another interesting direction is to investigate whether using such knowledge, e.g., by model aggregation, can further improve communication efficiency.
Acknowledgement
This work is supported by NSF grants IIS IIS-2213700, 11S-2128019 and 113-155615.
10

---

References
[1] Yasin Abbasi-Yadkori, Dávid Pál. and Csaba Szepesvári. Improved algorithms for linear stochastic bandits. Advances neural information processing systems, 24:2312-2320, 2011.
[2] Yasin Sobasi-Yadkong David Pal, and Csaba Szepesvari. Online-t--oonideencesst conversions and application to sparse stochastic bandits. In Artificial Intelligence and Statistics, pages 1-9. PMLR, 2012.
[3] Yossi Arjevani and Ohad Shamir. Communication complexity of distributed convex learning and optimization. arXiv preprint
2015.
[4] Qin Ding, Cho-Jui Hsieh, and James Sharpnack. An efficient algorithm for generalized linear bandit: Online stochastic gradient descent and thompson sampling. In International Conference on Artificial Intelligence and Statistics, pages 1585-1593, PMLR, 2021.
[5] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.
[6] Abhimanyu Dubey and AlexSandy' Pentland. Differe federated linear bandits. Advances in Neural Information Processing Systems, 33, 2020.
[7] Audrey Durand, Charis Achilleos, Demetris lacovides, Katerina Strati, Georgios D Mitsis, and Joelle Pineau. Contextual bandits for adapting treatment mouse model of de novo careinogenesis. In Machine learning for healthcare conference, pages 67-82. PMLR, 2018.
[8] Sarah Filippi, Olivier Cappe, Aurélien Garivier, and Csaba Szepesvári. Parametric bandits: The generalized linear case. In NIPS, volume 23, pages 586-594, 2010.
[9] Elad Hazan. Introduction to online convex optimization. arXiv preprint arXiv:1909.088 1909.0507, 2019.
[10] Elad Hazan, Amit Agarwal, and Satyen Kale. Logarithmic regret algorithms for online convex optimization. Machine Learning, 696--1);16--19,, 2007.
[1I] Ruiquan Huang, Weiqiang Wu, Jing Yang, and Cong Shen. Federated linear contextual bandits. Advances in Neural Information Processing Systems, 34, 2021.
[12] Kwang-Sung Jun, Aniruddha Bhargava, Robert Nowak. and Rebecca Willett. Scalable gen- eralized linear bandits: Online computation and hashing. arXiv preprint arXlv:1706.001.66 2017.
[13] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Ar- jun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. arXiv preprint arXiv: 1912.0977 2019.
[14] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International Conference on Machine Learning, pages 51 5132-5143 PMLR, 2020.
[15] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Saabbsaa-Brrwiskk.. et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 1 114((3):3521-333
2017.
[16] Nathan Korda, Balazs Szorenyi, and Shuai Li. Distributed clustering linear bandits in peer to peer networks. In International conference on machine learning, pages 1301-1309, PMLR, 2016.
[17] Chuanhao Li and Hongning Wang. Asynchronous upper confidence bound algorithms for federated linear bandits. International Conference on Artificial Intelligence and Statisties, pages 6529-6553. PMLR, 2022.

---

[18] Lihong Li, Wei Chu, John Langford, Taesup Moon, and Xuanhui Wang, An unbiased offline evaluation of contextual bandit algorithms with generalized linear models. In Proceedings of the Workshop on On-line Trading of Exploration and Exploitation 2, pages 19-36. JMLR Workshop and Conference Proceedings, 2012.
[19] Lihong Li, Wei Chu, John Langford, and Robert Schapire. A contextual bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, pages 661 661-670, 2010.
[20] Lihong Li, Yu Lu, and Dengyong Zhou. Provably optimal algorithms for generalized linear contextual bandits, In International Conference on Machine Learning, pages 2071-2080. PMLR, 2017.
[21] Wei Li, Xuerui Wang, Ruofei Zhang, Ying Cui, Jianchang Mao, and Rong Jin. Exploitation and exploration in performance based contextual advertising system. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pages
27-36, 2010.
[22] Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of fedavg on non-iid data. arXiv preprint arXiv:1907.02183) 2019.
[23] Brendan McMahan, Fider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficee learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273-1282. PMLR, 2017.
[24] Aritra Mitra, Rayana Jaafar, George Pappas, and Hamed Hassani. Achieving linear con- vergence federated learning under objective and systems heterogeneity. arXiv preprint arXiv:2102.07053 2021.
[25] Yurii Nesterov. Introductory lectures on convex optimization: A basic course, volume 87. Springer Science Business Media, 2003.
[26] Reese Pathak and Martin J Wainwright. Fedsplit: an algorithmic framework for fast federated optimization. Advances in Neural Information Processing Systems, 33:7057-7066, 2020.
[27] Chengshuai Shi, Cong Shen, and Jing Yang. Federated multi-armed bandits with personalization. In International Conference on Artificial Intelligence and Statistics, pages 2917-2925. PMLR,
2021.
[28] Yuanhao Wang, Jiachen Hu, Xiaoyu Chen, and Liwei Wang. Distributed bandit learning: Near-optimal regret with efficient communication. In International Conference on Learning Representations, 2019.
[29] Lijun Zhang, Tianbao Yang, Rong Jin, Yichi Xiao, and Zhi-Hua Zhou. Online stochastic linear optimization under one-bit feedback. In International Conference on Machine Learning, pages 392 401. PMLR, 2016.
[30] Dongruo Zhou, Lihong Li, and Quanquan Gu. Neural contextual bandits with ucb-based exploration. International Conference on Machine Learning, pages 11492-11502. PMLR, 2020.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]. see Section 4.2
(c) Did you discuss any potential negative societal impacts of your work? [No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
12

---

(a) Did you state the full set of assumptions of all theoretical results? [Yes], see Section
(b) Did you include complete proofs of all theoretical results? [Yes], the complete proof given in appendix
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [No]
4. If you using cxisting assets (c.g., code, data, models) curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]. in Section 5
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(c) Did you discuss whether the data you using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "pPZWTtMfsPb", "reviewer": "Reviewer_vtVx", "summary": "This paper considers federated learning for generalized linear bandits. The key difference compared to federated linear bandits is that it requires an iterative process for the global update rather than relying on sufficient statistics. The authors propose an efficient algorithm that is able to achieve a trade-off between communication and regret. Some variants of the proposed algorithm are also studied. Experiments are also conducted to corroborate the theoretical results.", "questions": "The reviewer has the following concerns.\n- The first one is about the technical contributions. It seems to me that this paper is a simple combination of existing techniques. That is, online-to-confidence bound + distributed optimization. The authors might need to highlight the new technical challenges in this paper. \n- The authors point out the issue of filtration in existing work (e.g., Lemma H.1 [28]) in the appendix. But it seems to the reviewer that the current explanation is not clear. Can the authors elaborate on this possible gap?\n- The presentation for the main part, especially Section 4.2 needs improvement. The current presentation makes readers difficult to appreciate new insights in this paper compared to existing work [2,12]. The authors can try to only give an informal version of the lemma and results by only highlighting the key terms and discarding all other similar terms in [2,12]. \n- Another question is about the measure of the communication overhead. It would also be important to measure the cost in terms of real numbers rather than the communication times.\n- Also, the current communication is a synchronized one, i.e., when one local client triggered it, all the clients need to communicate. This could be impractical. Can the authors comment on how to handle an asynchronous communication scheme?", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- A first study on the new problem\n- Both theoretical results and empirical experiments\n\n**Weaknesses**\n- Technical contributions are limited\n- The presentation may be improved", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "nGVi9ij1uJM", "reviewer": "Reviewer_btTD", "summary": "This paper studies the generalized linear bandit in the federated setting. In contrast to prior work on federated/distributed bandits that consider the linear bandit setting, the generalized linear bandit is more challenging since the lack of a closed-form solution prevents naively broadcasting sufficient statistics in the generalized linear bandit. The authors propose a novel algorithm $\\texttt{FedGLB-UCB}$ for the generalized linear bandit in the federated setting that combines offline and online regression in order to balance communication and computation. The authors demonstrate that $\\texttt{FedGLB-UCB}$ obtains competitive regret with a moderate communication budget, and also demonstrate its efficiency on a synthetic and real-world benchmark.", "questions": "Please see above for questions.", "limitations": "The authors have discussed their limitations in the conclusion, however it would be great if they could provide more substantial remarks in those directions.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Federated bandits are an upcoming area of interest within the bandit community, and hence the topic of this paper is very relevant to that community.\n- The authors are correct in that most prior work focuses on linear bandits in the federated setting, and extending those algorithms to the generalized linear bandit is non-trivial and hence this is an important challenge. \n- The paper is well-written and easy to understand. The central algorithm design components are easy to grasp.\n- The presented algorithm is no-regret with efficient communication and works well on benchmarks, outperforming prior work.\n\nWeaknesses:\n- The authors do not discuss optimality within the problem setting: what is the optimal level of communication for no-regret learning in federated generalized linear bandits?\n- It appears that the suboptimality (e.g., compared to the linear bandit) arises from the global update communication. Do the authors have any concrete remarks regarding this? The update does nullify the value in local updates (as pointed out by the authors in the conclusion as well), are there any assumptions that can allow reusing local updates in the global aggregation step?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "kmTeKZVriV0", "reviewer": "Reviewer_Uv78", "summary": "This paper studies the federated generalized linear bandits problem where each client faces a generalized linear bandits model. The parameters and the link functions across clients are the same. The learning objective is to minimize the cumulative regret of those clients. The paper proposes an algorithm called FedGLB-UCB with regret bound $O(d\\sqrt{T})$ and communication cost at most $O(dN^2\\sqrt{T})$, where $d$ is the dimension of parameter space, $T$ is the time horizon, and $N$ is the number of clients. Experiments are performed on both synthetic and real world dataset. ", "questions": "FedGLB-UCB adopts online regression for local update. While this reduces the computation complexity in each round, it may lead to larger estimation error. Is it possible to relace online regression with offline regression in each round for local update? Meanwhile, the global update takes many iterations of communications for each update. If the local update is already offline, can the computation complexity (hence the communication cost) for global update be reduced?\n\nAs mentioned above, the communication cost scales in $O(\\sqrt{T})$ other than $\\log(T)$. Is it possible to further reduce the communication cost to $O(\\log(T))$ under federated generalized linear bandits? The scaling in $N$ is also not promising, as it increases as $N$ increases, which limits the scalability of the algorithm. Can this be reduced as well?\n\nIn Li, et al, (2017), it assumes that the link function is strictly increasing. Does this paper have similar assumption?\n\nIs there applicable lower bound on regret for this problem?\n", "limitations": "The paper discusses limitations of current analysis and potential future directions.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The GLB setting introduces new challenges in the federated setting, since the estimation of the parameter does not have closed form and should be iteratively updated. This imposes challenges for efficient communication, and causes drifting issues at local updates.\n\nThe proposed FedGLB-UCB algorithm addresses such challenges by combining online local updating and offline regression for global update. It rigorously proves that  FedGLB-UCB achieves the same learning regret order as in the centralized setting, with a sublinear communication cost. The paper is in general well written and easy to follow.\n\nAlthough  the communication cost is sub-linear, the order of $O(N^2\\sqrt{T})$ still seems a little bit large, since federated linear bandits can achieve $O(N\\log(T))$. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept