Cluster Agnostic Network Lasso Bandits
Authors: Sofien Dhouib
Steven Bilaj
Behzad Nourani-Koliji
Setareh Maghsudi
Venue: Accepted by TMLR ()

Abstract: We consider a multi-task contextual bandit setting, where the learner is given a graph encoding relations between the bandit tasks. The tasks' preference vectors are assumed to be piecewise constant over the graph, forming clusters. At every round, we estimate the preference vectors by solving an online network lasso problem with a suitably chosen, time-dependent regularization parameter. We establish a novel oracle inequality relying on a convenient restricted eigenvalue assumption. Our theoretical findings highlight the importance of dense intra-cluster connections and sparse inter-cluster ones. That results in a sublinear regret bound significantly lower than its counterpart in the independent task learning setting. Finally, we support our theoretical findings by experimental evaluation against graph bandit multi-task learning and online clustering of bandits algorithms.

Published in Transactions on Machine Learning Research (12/2025)
Cluster Agnostic Network Lasso Bandits
Sofien Dhouib'
Department of Computer Science, Technical University of Darmstadt Faculty of Science, University of Tübingen
Steven Bilaj
steven. Siliiniblla@ruur-uue www.uuur-uun--ohumm.ddd
Faculty of Science, University of Tübingen
Faculty of Electrical Engineering and Information Technology, Ruhr University Bochum
Behzad Nourani-Koliji
behzad hww.r//owouuoniiiiiuntttttiieieo
Paculty of Science, University of Tübingen
Setareh Maghsudi
haturrhmmmggssut@rrrrruuuui---o de
Facully of Electrical Engineering and Information Technology, Ruho University Bochum
Reviewed on OpenReview: https: // openreviee.........mm net/ forum? id- QjAyyMPPDDD
Abstract
We consider multi-task contextual bandit setting, where the learner is given a graph encoding relations between the bandit tasks. The tasks' preference vectors are assumed lo be piecewise constant over the graph, forming clusters. At every round, we estimate the preference vectors by solving an online network lasso problem with a suitably chosen, time-dependent regularization parameter. We establish a novel oracle incquality relying on a convenient restricted eigenvalue assumption. Our theoretical findings highlight the importance of dense intra-cluster connections and sparse inter-cluster ones. That results in sublinear regret bound significantly lower than its counterpart in the independent task learning setting. Finally, we support our theoretical findings by experimental evaluation against graph bandit multi-task learning and online clustering of bandits algorithins.
1 Introduction
Online commercial websites aim to recommend their products their customers properly, and the per- formance of these recommendations depends on the knowledge of users' preferences. Unlike traditional collaborative--llleiin-- based methods (Su & Khoshgoftaar 2009), such knowledge initially unavailable. Therefore, the online recommender systems need to recommend various items to the users and observe their ratings to explore their preferences. At the same time, the recommender system should be able to rec- ommend items that attract users' attention and receive high ratings by exploiting the learned knowledge. The contextual bandit frameworks (Li et al. 2010) have been popularly used to formalize and address this opploration-eeplloitation trade-off.
However, the classical form of contextual bandits (Li ct al. 2010 Chu et al. 2011 Yadkkoi et al. 2011) ignores the availability of social networks amongst users and solves the problem for each user separately. Consequently, such algorithms have some drawbacks when applied to problems with large number of users. First, such large number hinders their computational efficiency. Second, the partial feedback of the bandit settings exposes the algorithms to having weak estimations and impairing their decision- decision-making ability (Yang et al. 2020). Consequently, to improve bandit algorithms' performance for large-scale applications, utructural
assumptions that link the different users are usually integrated within bandit algorithms (Cesa- Bianchi et al.
2013 Gentile al. 2014
Li et al. 2019 Herbster et al. 2021
Equal contribution

---

Published in Transactions on Machine Learning Research (12/2025)
et al. (2013) and Yang et al. (2020) use the prior knowledge of social networks into their contextual bandit algorithms. Both papers propose UCB-style algorithme and exhibit the importance of using the social network graph to achieve lower regrets using Laplacian regularization. The latter regularization promotes smoothness among the preference vectors of users, allowing the transfer of the collected information between them. However, the Laplacian regularization does not account. for the smoothness heterogeneity introduced by a piecewise constant behavior over the graph (Wang et al., 2016). On the other hand, algorithms of online clustering of bandits (Gentile et al.) 2014 Li al. 2019) tackle such piecewise constant behavior by explicitly estimating user clusters.
In this paper, we assume access to a graph encoding relations between bandit tasks, and that the task parameter vectors are piecewise constant over the graph. We propose an algorithm that integrates the prior knowledge of the piecewise constant structure to update tasks rather than finding the clusters explicitly. That way, we mitigate the limitations mentioned above: the piecewise constant smoothness naturally integratod into our regularizer, and we do not estimate the clusters so our algorithm does not suffer from overconfidence drawbacks.
More precisely, we provide the following contributions
We analyze an instance of the Network Lasso problem (Hallac et al. 2015), estimating every vertex's preference vector using data generated during the interaction between users and the bandit. We provide the first oracle inequality in this setting and link it to fundamental quantities characterizing the relation between the graph and the true preference vectors of the users. Our result relies on our novel restricted eigenvalue (RE) condition, which we assume for our setting. This result is of independent interest and can be applied to i.i.d. data special case.
We prove that the empirical multi-task Gram matrix of the data inherits the RE condition from its true counterpart. Both this result and the previous one depend on the sparsity of inter-cluster connections and the density of intra-cluster ones.
We provide a regret upper bound for our setting. Our bound highlights the advantage of our algorithm in high dimensional settings, and for large graphs.
We support our theoretical findings by extensive numerical experiments on simulated data that prove the advantage of our algorithm over other related approaches.
The rest of the paper is organized as follows. Section discusses the relation of our work to the literature. We formulate our problem and state some of our assumptions in Section then present our bandit algorithm in Soction We analyze the problem theoretically in Scction and demonstrate its practical interest experimentally in Section G
2 Related work
Lasso contextual bandits. To address the high dimensional setting for linear bandits, several multi- armed bandit papers solve a LASSO (Tibshiran] 1996) problem under different assumptions (Bastani & Bayati 2019 Kim & Paik 2019 Oh al.] 2021; Ariu et al. 2022). They all rely on previously established compatibility RE condition Bühlmann & van de Geer 2011 that they adapt to the non-ii.d case
resulting from the context selection procedure across rounds. Such assumptions were also used in the multi- task setting by (Cella & Pontil (2021) with Group Lasso regularization YYuan & Lin 2006), and to impose a low-rank structure on the task preference vectors in Cella et al. (2023). In our case, we establish a novel oracle inequality, rather than only generalize an existing one to the non-i.i.d setting, with a newly introduced RE assumption, which can be of independent interest.
Clustering of bandits. Gentile et al. (2014) introduced sequential clustering of bandits with the CLUB algorithm. The latter starts with fully connected graph, and then an iterative graph learning process is performed, where edges between users deleted if their preference vectors significantly different. As
2

---

Published in Transactions on Machine Learning Research (12/2025)
result, any connected component is seen cluster and only one recommendation per cluster is developed.
The SCLUB a algorithm of Li et al. (2019) generalizes CL.UB via including merging operations in addition to splitting. In contrast to these approaches, Nguyen & Lauw (2014) groups users via K-means clustering, and
Cheng et al. (2023) rely on hedonic games for online clustering of bandits. Furthermore, Yang & Toni (2018) make use community detection techniques on graphs to find user clusters. Gentile et al. (2017 sludy the clustering of the contextual bandit problem where their proposed algorithm, named CAB, adaptively matches user preferences in the face of constantly evolving items. Our work fundamentally differs from the previous ones on two aspects. First, assume access to graph encoding relations between users, which is more informative than a complete graph. Second, we do not keep track of model for each cluster, but rather we integrate a prior over the graph via a graph total variation regularizer that enforces a piecewise constant behavior for the estimated preference vectors.
Multi-task learning. Several contributions assume that the bandit tasks share some underlying structure. In Cella & Pontil (021), task preference vectors are assumed to be sparse and to share their sparsity support, implying that they lie in a low-dimensional subspace with dimensions aligning with the canonical basis vectors. This idea is further generalized in Cella et al.] (2023). where the tasks are assumed to be confined to arbitrary unknown low-dimensional subspace. That work improves upon Hu et al. (2021) by not requiring the knowledge of the small dimension of the task space. It can be considered to solve our problem if the number of clusters is smaller than the dimension, resulting in low-rank structure. However, our work does not rely on any assumption between the number of clusters and the dimension. The underlying structure linking tasks can also be a graph encoding relations between them (Cesa-Bianch) et al.) 2013; Yang & Toni 2018), which is our case. However, while they assume smoothness as prior, we assume piecewise constant behavior.
Homophily and modularity in social networks Given the large number of users on social networks, one may be able to learn their preferences more quickly by leveraging the similarities between them. This idea relies the notion of homophily in social networks (MePherson et al. 2001; Fasley et al. 2010). In modelling social networks, users' preferences relationships are encoded in graph, where neighboring nodes are users with similar preferences. This graph can be known priori or it can be inferred from previously collected feedback <Dong et. al. 2019). Exploiting this information and integrating them into bandit algorithms can lead to a significant increase in performance Yang et al. 2220).. Indeed, the knowledge of user relations allows the algorithm to tackle the data sparsity issue that is inherent to bandit settings. Another fundamental point that can be used to integrate information from social networks is that, social networks show large modularity measures (Newman) 2006 Borge-Holthoed of edges within clusters and low density of edges between clusters. As result, users can be clustered based on the graph topology and a preference vector can be learned for each cluster, substantially reducing the dimensionality of the problem. In other words, discovering the clustering structure of users can reduce the computational burden of large social networks. Consoquently, there have been attempts in exploiting the clustered structures social networks in bandit algorithms (Gentile et al. 2014 Nguyen & Lauw 2014
et al.) 2011). This implies that we have high density
Yang & Toni 2018 Li et al. 2019: Nourani-Kolln et al. 2023; Cheng et al. 2023).
Bandit meta-learning In contrast to the multi-task setting, meta learning deals with sequentially arriving tasks that have to be learnt and generalizing the gained information to improve performance for future tasks. Here, as in the multi-task setting, it is assumed that the tasks share some common structure that is ought to be learnt and exploited. Bilaj et al. (2024) assume that the tasks are sampled from common distribution and concentrated around an affine subspace learned through PCA algorithm. The resulting projection matrices could then be exploited to improve learning for new tasks in an adapted UCB and Thompson sampling approach.
Other lines of work are Cella et al. (2020) Kveton et al. (2021): ; Basu et al. (001), which learns the mean of the distribution under the assumption that the covariance of the prior is known or Peleg et al. (2022) which generalizes this assumption and attempts to learn the covariance well.
3

---

Published in Transactions on Machine Learning Research (12/2025)
3 Problem setting
We consider a linear bandit setting, with finite number of tasks representing users in recommendation system for example. For each task the agent has to choose among K arms, each associated to a d-dimensional context vector. All interactions over horizon of T time steps. We further assume that we have access to an undirected graph 9 (V,E), with vertex set V representing the tasks and edge set & encoding the relationships bet ween them. We identify the vertex set V with the set of verlex indices [|V]]- Thus, we consider to be a subset of V2, where every edge (m, n) E & has weight Wmn > 0, with m < n. The tasks' preference vectors are denoted by {fm) mee C IR verifying ||0mll S Am E V, which we concatenate as row vectors into matrix O RIvixd, The latter represents graph vector signal, assumed to be piecewise constant over G.
At a round t E N", user m(t) E V is selected uniformly at random and served an arm with context vector xtt) from a finite action set A(t) Rd with size K, depending their estimated preference vector O (m(() (t) e R We assume the expected reward to be linear, with an additive o-sub- Gaub--a noise conditionally on the past. Formally, denoting by Fo the trivial sigma-algebra, and for all t 2 1, by Fo the sigma-algebra generated by history set x(1), v(1)," mm(t) x+)), v(), mlt + 1)}, the received reward y(() is given by yt) = (t), xt)) + nt),, where (t) is F, F measurable and Vt 2 1, Va E E,
E [+t =
(exp( ((exp(s((())))) (1)) exp ( (
(2020))
The performance of our policy assessed by the expected regret over the T interaction rounds for all tasks:
(0 (0m(t), x) - (0met)) 2(1)) BEA(T)
R(T) E
max
(2)
The Optimization problem in Equation (4) is an instance of the Network Lasso (Hallac al. 2015 Several instances of the same type were studied by Jung al. (2018); Jung & Vesselinoval (2019) Jung (2020)
He et al. (201)). The objective is characterized by its second term which, while being just the Laplacian regularization without squaring the norms, promotes piecewise constant behavior rather than smoothness. For real-valued signals (d = 1), this regularization has been extensively studied for image and graph signal denoising, for the problem of trend filtering on graphs (Wang et al. 2016). According to Wang et al. (2016). that regularization better adapts to the heterogeneity structure in the data: similar users will not only have similar models but the same model, which offers compression of the overall model over the graph. Note that setting is cluster agnostic; algorithm does not aim to learn the cluster structure explicitly but to exploit it implicitly using the total variation semi-norm as regularization. The strength of the latter is controlled via timo-dependent regularization coefficient a<)), which we will express later in the analysis.
smoothness of the signal and induces cluster
We formalize our assumption on the context generation as follows.
Assumption 1 (i.i.d action sets). Context sets {A(t))T= are generated i.i.d. from distribution p over kx such that |>]|| < 1A a E A(t) VI 1.
In addition to the i.id assumption, assume more regularity follows.
Assumption 2 (Relaxed symmetry and balancod covariance). There erists constant 2 1 such that for all
X E B p(-X) vp(X). Purthermore, there exists w > 0, such that for any permulation (a.,-.. a0k
of [K], for any (2..... -11, IRdd have.
E La, I 1ww Dai < ... < w Taal]] & WE [(Ia, IT, + Bax x )1/ Ia, ... < w Iox|]
where M of N means that N M is PSD matria.
4

---

Published in Transactions on Machine Learning Research (12/2025)
This assumption was introduced in Oh al (2021 and has already been used in multi-task setting by Cella et al. (2023). Parameter v controls the skewness, as = corresponds to symmetric distribution. w decreases with increasing positive correlation between arms. It verifies w = O(1) for multi-variate Gaussians and uniform distributions ovcr the unit sphere (Oh et al. 2021 The piccowise constant behavior of the graph signal is formalized in the next assumption.
Assumption 3 (Piecewise constant signal). There exists portition P V, such that for any cluster C P, signal O is constant on C, that is, 0m On for all m, n E C. The graph obtained by taking the vertices in C and the edges linking them is connected.
Assumption 3 basically states that the true preference vectors are clustered and that the given graph induces the cluster structure. is required for our approach to be beneficial, as we will detail in the analysis section. For the sake of clarily, defer the statement of other technical assumptions to Section
4 Algorithm
Our policy in Algorithm 1 follows greedy arm selection rule in multi-task setting, in the same vein those presented in Oh et al. (2021): Cella et al. (2023). Indeed, as pointed out in Oh et al. (2021), exploration is implicitly incorporated into regularization parameter a(t)'s time dependence. It has the following expression
a(t) ag Vt+ az((),
1
(t) 2
Im (t)l² log 8(t)²
mEV
02 (l) 2 max
ITm (4) log 8(1)
mEV
where og 0. The set of time steps task has been selected up to time t is denoted by Tm (t). At the end of a round t, all preference vectors are updated into a new estimation @(t) while leveraging the structure of graph 9, formally by solving the following network lasso optimization problem:
(t)
arg min
1
Σ ((0m(r), v(T))² a(t)
Σ (m,n)EE
Wmn 1m 0m|l
(4)
SERIV 2t
where ||-|| denotes the Euclidean norm for vectors. At each time step the network Lasso problem is solved via the primal-dual algorithm (Jung 2020).
Algorithm 1: Network Lasso Policy
Input: 7,00 > 0,9, Initialization: (() = 0 E R xd
for t E (l....,T) T} do
Draw a user mm()) E V uniformly at random Observe context set A(t)
Select xtt) E aarg mrgmax (0
(0m(a-1)) x)
Receive payoff y()
Update a(t) via equation 3
Update (() via equation
5 Analysis
This section provides the main steps of the analysis. One of the paper's contribution lies finding an oracle inequality of the network lasso problem given restricted eigenvalue condition holding for the true multi- task Gram matrix. In this regard, the next major challenge and contribution is to show that the empirical
5

---

Published in Transactions on Machine Learning Research (12/2025)
multi-task Gram matrix, estimated in the algorithm, satisfies the restricted eigenvalue condition. We start by proving an oracle inequality for the estimation error of A Then, we prove that the latter assumption holds with high probability given that the true multi-task Gram matrix satisfies it, We end this section by establishing a regret bound for our algorithm.
5.1 Notation and technical assumptions
We provide additional notations required for the analysis. We denote by OP the set of all edges in E connecting vertices from different clusters from partition P (Assumption 3 and we call it the boundary of P. Thus, Op", the complementary set of åP, is formed by edges connecting vertices of the same cluster. The total weight of the boundary, i.ethhe sum of its edges' weights, referred to as w(PP). Given a signal Z E Rivixd, we denote by Zp the signal obtained by setting row vectors of Z to their mean- per-cluster value w.r.t. P. For any edge subset [ E, we denote the following norms: ||-Ilp the Frobenius norm and ||0|| = - (m.n)e WmnllOn. 0z as the total variation semi-norm of O E RRl xd over I. Thus, the regularization term of Problem Equation (4) equal to ||O|| Also, we define the incidence matrix B, C
restricted to I _ & to be mull except at rows with index i E corresponding to edge (m,,), where it equals Wmm(em. en), where em is the mh canonical basis vector of R We define Ay(t)= = diag (X1(t)T (i(t)...., (t) (((t))XX((()) (t))(t) E [ddV)x and subsequently the empirical multi-task Gram matrix up to time step is given by Aty))). The following definition introduces quantities related to the clusters defined by partition P, with crucial roles that we will elucidate throughout the analysis.
Definition 1 (Cluster content constants). Let C P be cluster.
We denote by a,C the vertices of C that are connected to its complementary. We define the inner isoperi- metric ratio of C as (g(C)= = (8,c|
|C|
By abuse of notation, we denote as Be the incidence matria restricted edges linking vertices of C, its associaled Laplacian matrir by Le B Be, and its pseudo-inverse by Le. The lopological centrality index of node m E C w.r.t C is equal to (I)) We define the topological centrality index of by (C) = minmec( (1))
The inner isoperimetric ratio of a cluster measures how many "interior" nodes a cluster contains, in the sense that they are not connected to its complementary. It is at most equal to the isoperimetric ratio for weightless graphs as the size of the inner boundary is at most equal to that of the edge boundary, the latter being connected to the algebraic connectivity via the Cheeger inequality (Cheeger) 1970).
The topological contrality index measures the overall connectodness of a vertex in a network and indicates how robust a node is to edge failures (Ranjan & Zhang, 2013). Also, it can be tied to electricity spreading in network according to Van Mieghem et al. (2017). We refer the interested reader the two previously mentioned works for a detailed account of the properties of the topological centrality index. In the appendix, show that for binary weights graphs the minimum topological centrality index is at least equal to the algebraic connectivity theoretically and experimentally, where we showcase that the difference between the two can be significant.
Remark 1. Both the topological centrality index and inner isoperimetric ratio are key parameters of the cluster structure and the graph. They determine the "quality' of the given graph. An optimal graph and cluster structure yield many intra-cluster connections and few inter cluster connections iee. a high topological centrality inder and low inner isoperimetric ratio for any cluster. This will later be highlighted in the oracle inequality and the regret bound.
To proceed, we will need the following definition that introduces several notations to reduce the clutter.
Definition 2 (Restricted Eigenvalue (RE) condition and norm). A PSD matris M e [ verifies the
RE condition with constants x 2 1, v > 0 and $ > 0 if
2 2 & (eo(2T) M veo(ZT) VZ E S,
(5)
6

---

Published in Transactions on Machine Learning Research (12/2025)
where 8 is the cone defined by:
8 = Z E ; 9 vw(dp)
| 1 ape Sa2 g, v
1| }
+ (xP(
1
a1 01 0, an) = ag
a2 a2(9, 0,0
+ (0P) max CEP
Vig(C),
min V cC((C) CEP
ao
and the R.E. semi-norm defined by Z RE RE == Z
For the rest of the paper, when clutter. For our main results, we cover the case of 2 but treat the more general case in the supplementary material. For such simplification to be valid, we need to assume that the graph satisfies min /cg(CC) > 2w(OP), such that we can always find constants k 2 and 1b 0 that guarantee CEP
d1 and d2 without arguments, set og =((P)) in order reduce
the proofs
0
To explain the RE condition, if we had S = gD and |R-E = |1-Il then M would be invertible with minimum cigenvalue at least Φ². In comparison, our requirement is weaker since it only needs to hold for signals Z E and for the RE semi-norm. the Lasso problem in (Bhhlmann & van de Geer 2011 Oh al.] 2021) or the restricted strong convexity assumption (Cella al. 2023).
has the samne form as the compatibility assumption for
We further make the following assumption on the true multi-task Gram matrix:
Assumption 4 (RE condition for the true multi-task Gram matrix). For ls [K], let Eg = B [arar]]] be the Gram matrix of the kth contect vector's marginal distribution, let Ey be the true multi-task Gram matrix of the contert vector generaling distribulion, given by
Ey = =1=1 E,
where Σ
Σ..
K
We assume that Ey verifies RE condition (Definition with some problem dependent constonts k E
1, (0( CEP
min v (C)(C)) ab E (0, w(PP)
min V cc( 2 and > CEP
This assumption common to several Lasso-like bandit problems Oh el al. 2021 Ariu et al. 2022 ct al. 2023).
Cella
We will later show that it can be transferred to the empirical multi-task Gram matrix.
Remark 2. The previous assumplion implies the invertibility of the true mean yram matris of any cluster. For the special case that all clusters are known, i.e. the graph has no inter cluster connections, we can fully exploit this with since all users willvin cluster share observed conlext veclors and can be viewed as singular user. In order for the individual OLS problems to be soluable the true mean grom matria per cluster needs to be invertàble.
oo, which amounts to the task learning of [P] instead of [V] independent OLS problems,
We provide further intuition the constant ó within the RE condition. We can show that o has an upper bound:
Proposition 1 (On the RE constant Φ). Let M, E Rdxd be the brue mulliilask gram matric of user i. Assume 2 1. Then the constant of the RE condition can be upper bounded as:
Eiec M, C
Amin
where Amiu (.) gields the minimum eigenvalue of a given matria.

---

Published in Transactions on Machine Learning Research (12/2025)
Since the true multi-task gram matrix per cluster is always invertible, we always have non-null minimal eigenvalue.
Remark 3. The minimal eigenvalue in Proposition could be further bounded using the trace of the co- variances i.e. the sum of all the eigenvalues over the dimension. This would result into upper bound of o² S d'
5.2 Oracle inequality
This section is dedicated to provide bound on the estimation error of the Network Lasso problem given in Equation (4b at particular step t of Algorithm [ We assume fixed design, meaning that the context vectors are given and fixed, and we are not concerned by their randomness (due to the context generating distribution), nor by the randomness of their number for each user (due to random selection al each time
step).
For a time step t, we deliver the oracle incquality controlling the deviation between the estimated preference vectors (() and the true ones O.
Theorem (Oracle inequality). Assume that the RE assumplion holds for the empirical mulli-task Gram
malric t Av( with constants &. E 1, 11,2(0pp min v cc(C)) & E 0, (OPT CEP
min v cc(CC) and & > 0. CEP Suppose that maxmev (Tm S bt for some b > O and oo 2 vwloPj Then, with probability at least 1- 5(1); have.
( <2 oau
f(g, O, ao)
1 1 2
|o| 8(t) + 26 log 8(t)
0²
where
02(9, O a.0)
#1)
f(0,0,a))==(((() Oa)))
(, 0,0 min Veg(C)
CEP
The proof relies decomposing the estimation error signal into sum of two terms. The first term amounts to laking its mean per cluster, that is, every node within the same cluster is mapped to the mean estimation error of its cluster. The second term is proven to be related to the incidence matrices of each cluster. The probabilistic statement comes from a high probability bound on the Euclidean norm of an empirical vector process associated with our problem, using a generalization of the Hanson- Hanson-Wrightl inequality to the subgaussian case (Hsu al. 2012] Theorem 2.1). Compared to the bound of Jung (2020) Theorem 1), we bound norm of the estimation error rather than just the total variation semi-norm. Besides, due to the expressions of d, a.)) and az(O, G, aw), the bound significantly decreases with the products w(PP) minger vi(C) and w(PP) maxcer eg (C)-,, 2 which are small enough for dense intra-cluster edge links and sparse inter-cluster ones. The bound on the oracle inequality clearly grows with x and v thus it is most beneficial if k is close to and &i close to zero.
5.3 RE condition for the empirical multi-task Gram matrix
To establish the oracle inequality, we assumed that the RE condition holds for the empirical multi-task Gram matrix. In this section, we prove that this holds with high probability. To this end, we use the same strategy as in Oh et al. (2021]; Cella et al. (2023). We prove that on the one hand, the empirical multi-task Gram matrix inherits the RE condition from its adapted counterpart since it concentrates around it. On the other hand, we show that the adapted Gram matrix verifies the RE condition due Assumption
and Theorem 2 (RE condition holding for the empirical multi-task Gram matrix). Under assumptions and let t 2 1, and let K, & be the constants from Assumption 4 Assume that maxmey (t)(t) < bt. Then, for any
E 0, (1 1 22) )²)) the empirical malti-task Gram matrix - At) verifies the RE condition with constants
-2)
8

---

Published in Transactions on Machine Learning Research (12/2025)
K, vo and é, where
3 = 01
02
-
01
-3 (mincer (C) A ög (C))))
with probability at least equal to 1 6d| exp
where
and
6b + 2/
V2v
(c(C) = cg(C) A P.
The proof follows similar approach as in Oh et al. 2021 Cella el al. 2023 1 prove that the RE condition transfers from the true multi-task Gram matrix to its adapted counterpart (t),), defined as follows:
(1) diag (Vi TM (i(()))
(8)
where
V (1) = Σ [0(T)
TET.(())
This transfer relies on the work of Oh et al. (2021 lemma 10). The other step of the proof is showing that the empirical multi-task Gram matrix and (t(t) become close to each other with high probability after sulficiently many lime sleps, in the sense of a matrix norm induced by the RE semi-norm and the restriction set & (Definition 2). The bound showcases a dependence on mincer cg(C) [C], which is of the same order |C) for fully connected cluster with vertices C. It is also clear that the probability of satisfying the RE condition increases with higher minimum centrality of cluster.
5.4 Regret bound
To bound the regret, bound the expected instantaneous regret for each round 2 1. This bound relies the oracle inequality holding and the RE condition being satisfied for the empirical Gram matrix, both with high probability. Thanks to Theorem and Theorem 2 these two conditions are ensured.
Thcorem 3. Let the mean horizon per node be T = the Network Lasso Bandit algorithm is upper bounded as follows:
T [V] Under assumptions to the expected regret of
R(T) S 0 [g (on
G,oov (VIVI+ +il++iogg Vlog(T|V)) v log [
1 + log(d[V)) vivi).
Φ²
with
A 3 mincer (Tx(C) (g(())
-2 a1
=
1
6 log VD
22
VIy
Our regret is mainly formed of two parts. The first one is the sublinear time-dependent term and represents the bulk of horizon dependence. Interestingly, it decreases the topological centrality index grows with the graph size, which proves the importance of intra-cluster high connectivity.
The second significant term comes from ensuring the RE condition for the empirical multi-task Gram matrix, and can be interpreted as the number of time steps necessary for to hold, as pointed out by Oh et al.) (2021b. It has a logarithmic dependence the graph size and in the dimension, which is characteristic of regret bound of the "lasso type'. Also noteworthy is that the regret grows explicitly with log(d) only in the time-independdet lerm, making our policy useful in high-dimensional settings. Though from Proposition [ we can expect an implicit dependency on the dimension in the RE constant 6. Specifically, the lower bound on o is an open problem that appears unsolved in other lasso based works such as Oh et al. (2011) Cella
ct al. 2023).
Both the regret bound and the oracle inequality presented in Theorem I hold only for the of graphs that least satisfy the condition min (C( > 2w(0P) and even though our results hold for a large set CEP

---

Published in Transactions on Machine Learning Research (12/2025)
of graphs, the individual role of graph-related constants, encapsulated in f(g, oo); is not obvious. By further restricting the set of graphs, are able to provide simplified bound
Corollary Assume w(OP)(+22) S Ω2 with some positive constant Q < 1, then under assumptions to min cC((C)
CEP the expected regret of the Nelwork Lasso Bandil algorithm is upper bounded follows:
R(T) = (
w(OP) maxcep (G()) VT [VV + (log(T + VDylog (FIV))
(2 - 22) mincer v (C)
w(P))² max Lg (C) CEP
log log(d[V) + vIM1)
+ (1 Q)² mincer((g (C) & (C))
The simplified bound in Corollary exhibits the typical multi-task learning dependency /T rather than the independent task learning case W/VT and highlights the role of graph related properties such the total weight of the boundary, the maximal inner isoperimetric ratio and the minimal topological contrality index. Furthermore with f we can see the influence on the regret bound, when w(OP) changes relative to min (C)(C). Compared to the GraphUCB algorithm (Yang & Toni 2018), that yields regret CEP
bound of (dVIVT) and the trace norm bandit algorithm Cella et al. 2023 with a regret bound of
( (VrodiM IVIV ToT with TO as true rank of the multi task structure, our result shows an improvement with respect to the number of nodes and horizon. The bound of the SCLUB algorithm given in Li et al. (2019) yields an improved regret bound of Ô d d/IPPIN(T) that depends on the total number of clusters [P] While our algorithm exhibits a similar result, it does not explicitly depend on the number of clusters but with the properties of the graph and the compatibility with the underlying cluster structure. We present two additional specific instances: a graph with no boundary (i.e., w(OP) = 0), and a graph where full connectivity achieved within each individual cluster.
Corollary 2. Assume w(dP)= 0, then under assumptions Bandit algorithm is upper bounded as follows:
to 4 the expected regret of the Network Lasso
R(T)= - ( V Vlog(TIVI) V V (TIVI) mincer A Eg(C)) log(d[V) + VIDI)
[III+
Corollary 3. Assume (()))
min / cC( < Ω with some positive constant Ω < and assume any two modes
CEP within the same cluster are connected through an edge in the given graph, then under assumptions to the expected regret of the Network Lasso Bandit algorithm upper bounded as follows:
R(T)= 0 1 w(OP) maxcer (g(C), VT (VIV++ (oo((TTV) 1oy (TIVI)
(1 Ω2
mincer VIC
w(PP)²
max (C(C) CEP
log log(d[V) + VIDI
+
(1 - 22)
mincer [C]
6
Experiments
We compare our algorithm with ag to several baselines of the literature. On the one hand, we consider baselines relying given graph, GOBLin (Cesa-Bianchi et al. 2013 and GraphUCB (Yang al.) 2020
10

---

Published in Transactions on Machine Learning Research (12/2025)
that use the Laplacian smooth the preference vectors. On the other hand, compare to clustering of bandits baselines, namely CLUB (Gentile et al. 2014), SCLUB (Li et al. 2019), OLS-ITT. (Bastani et al. 2021)) and LOCB (Ban & He 2021). We provided CLUB with graph & rather than fully connected graph for a fair comparison. We also include the trace norm bandit algorithm (Cella et al. 2023 which is relevant when the number of clusters is smaller than d. Indeed, the cluster structure of θ can mathematically written as O Ecep 1cf6, where 1c the indicator vector of cluster (coordinates equal nodes belonging to C and zeros elsewhere) and Oc is the true vector of every node in C. The range of O is equal to the span of 1c; C E P, implying that its rank is at most equal to min(d, (P]). It will then satisfy the low-rank assumption for [P] < d. As a sanity check, we compare to the independent task learning case with LinUCB (LinUcbITL) where each task is solved independently. The graph used is weightless and generated using a stochastic block model lo ensure a cluster structure, where an edge is constructed with probability within clusters and between clusters.
on the
Experimentally, we found that normalizing the weights as Wmn (deg(m) deg(n)))= where dog(m) denotes the degree of node m, yields significantly better results. Indeed, such normalization makes the algorithm focus more edges between low-degree nodes, which improves the propagation of the collected information within the graph.
2000
CLUB
500
CLUB
GOBLin
GOBLin
1750
GraphUCB
GraphtUCB
400
LOCB
1500
LinUcblTT
LinUichTT
carslativ nogntt
1250
NetLassso
canssaaive rigatd
300
Lasso
O1.8-ITI.
O1.5-ITI.
1000
Trace: Narm
SCLUB
200
Troco-Norm
750
500
100
250
0
300
1000
1500
2000
2500
3000
3500
1000
500
1000
1800 line steps
2000
2500
3000
time sleps
(a.) | 200. [P] 25, d 10, 0.5, 0.05
(b) |V| 50, [P] 5. 80. 0.8, g 0.2
1000
CLUB
CLUB
GODLis
GODLin
GapphUBB
GraphUCB
800
LOCB
LOCB
LinUcbTTL
LinUlb/TL
300
comnleive regad
800
NetLasso
OLS-IITI
OL.-TIT.
SCLUB
SCLUB
200
Tracs-Norm
400
Trace-Norm
200
0
200
400
600
800
1000
500
1030
1500
2000
2500
3000
steps
steps
(c) | 100, [P] 8, d= 10,p = 0.5, 7 0.1
(d) |V] 100, IP| 8, = 20, 0.4,
Figure 1: Synthetic data experiments showing the cumulative regret of Network Lasso Policy as function of time-steps compared to other baselines, for different choices of |P|,|P|, d, and q.
11

---

Published in Transactions on Machine Learning Research (12/2025)
Our results clearly demonstrate improvement compared to the other baselines. Our policy performs significantly better than the rest beyond the error margins, covering one standard deviation at ten repetitions. We provide results for up to [ = 200 nodes showing the effective transfer of knowledge between nodes.
7 Conclusion and future perspectives
In this work, we proposed a multi-task bandit framework that solves the case where the task preference vectors are piecewise constant over a graph. To this end, we used the Network Lasso policy to estimate the task parameters, which bypasses explicit clustering procedures. We established sublinear regret bound and proved a novel oracle inoquality that relies on the small size of the boundary and the high value of the topological centrality index of each node within its cluster. Our experimental evaluations highlight the advantage of our method, especially when either the number of dimensions or nodes increases.
Due to the technical similarity of our problem with the Lasso, a natural extension would be to extend it to a thresholded approach, in the same vein as (Ariu et al. 2022 Another possible extension would be to use regularization with higher order total variation terms that impose a piecewise polynomial signal on a graph, as explained for scalar signals in Wang et al. (2016); Ortelli & van de Geer (019
Acknowledgments
This work was supported by the German Research Foundation (DFG) under Grant Nos. MA7111/6-1 and MA7111/7-1. The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Steven Bilaj.
References
Yasin Abbasi-Yadkony Dávid Pál, and Csaba Szepesvári. Improved algorithms for lincar stochastic bandits. In Advances in Newral Information Processing Systems, 2011.
Kaito Ariu, Kenshi Abe, and Alexandre Proutiere. Thresholded Lasso Bandit. In Proceedings of the 39th International Conference on Machine Learning, 2022.
Yikun Ban and Jingrui He. Local clustering in contextual multi-armed bandits. In Proceedings of the Web Conference 2021, 2021.
Hamsa Bastani and Mohsen Bayati. Online Decision Making with High-DDmensional Covariates. Operations Research, 2019.
Hamsa Bastani, Mohsen Bayati, and Khashayar Khosravi. Mostly exploration-free algorithms for contextual bandits. Manage. Sci., 2021.
Sounya Basu, Branislav Kveton, Manzil Zaheer, and Csaba Szepesvari. No Regrets for Learning the Prior in Bandits. In Advances in Neural Information Processing Systems, 2021.
Steven Bilaj, Soficn Dhouib, and Sctarch Maghsudi. Meta learning in bandits within shared affine subspaces. In Proceedings of The 27th International Conference on Artificial Intelligence and Stalistics, 2024.
Javier Horgo-Holthoefer, Alejandro Rivero, Iñigo Gareía, Elisa Cauhé, Alfredo Ferrer, Darío Forrer, David Francos, David Iniguez, María Pilar Pérez, Gonzalo Ruiz, al. Structural and dynamical patterns on online social networks: the spanish may 15th movement case study. PloS one, 2011.
Peter Bühlmann and Sara van de Geer. Statislics for high-dimennsionll data. Springer Series in Statistics. Springer, Heidelberg, 2011.
Leonardo Cella and Massimiliano Pontil. Multi-task and meta-learning with sparse linear bandits. In Uncertainty in Artificial Intelligence, 2021.
Leonardo Cella, Alessandro Lazaric, and Massimiliano Pontil. Meta-learning with stochastic linear bandits.
Tn International Conference on Machine Learning, 2020.
12

---

Published in Transactions on Machine Learning Research (12/2025)
Leonardo Cella, Karim Lounici, Grégoire Pacreau, and Massimiliano Pontil. Multi-task representation learn- ing with stochastic linear bandits. In International Conference on Artificial Intelligence and Stalistics, 2023.
Nicolo Cesa-Bianchi, Claudio Gentile, and Giovanni Zappella. A gang of bandits. Advances in neural information processing systems, 2013.
Jell Cheeger. A lower bound for the smallest eigenvalue of the laplacian. Problems in analysis, 1970.
Xiaotong Cheng, Cheng Pan, and Setareh Maghsudi. Parallel online clustering of bandits via hedonic game. In International Conference on Machine Learning, 2023.
Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff functions. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, 2011.
Xiaowen Dong, Dorina Thanou, Michael Rabbat, and Pascal Frossard. Learning graphs from data: A signal representation perspective. IEEE Signal Processing Magazine, 2019.
David Easley, Jon Kleinberg, et al. Networks, crowds, and markels: Reasoning about a highly connected world. Cambridge university press Cambridge, 2010.
Angela Fontan and Claudio Altafini. On the properties of laplacian pseudoinverses. In 2021 60th IEEE Conference on Decision and Control (CDC), 2021.
Claudio Gentile, Shuai Li, and Giovanni Zappella. Online clustering of bandits. In International Conference on Machine Learning, 2014.
Claudio Gentile, Shuai Li, Purushottam Kar, Alexandros Karatzoglou, Giovanni Zappella, and Evans Etrue. On context-dependent clustering of bandits. In International Conference on machine learning, 2017.
David Hallac, Jure Leskovec, and Stephen Boyd. Network lasso: Clustering and optimization in large graphs. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and dala mining, 2015.
Xiao He, Francesco Alesiani, and Ammar Shaker, Efficient and scalable multi-task regression massive number of tasks. In Proceedings of the AAAI Conference on Artificial Intelligence, 2019.
Mark Herbster, Stephen Pastoris, Fabio Vitale, and Massimiliano Pontil. gang adversarial bandits. Advances in Neural Information Processing Systems, 2021.
Daniel Hsu, Sham Kakade, and Tong Zhang. A tail inequality for quadratic forms of subgaussian random vectors. Flectronic Communications in Probability, 2012.
Jiachen Hu, Xiaoyu Chen, Chi Jin, Lihong Li, and Liwei Wang. Near-optimal representation learning for linear bandits and linear rl. In International Conference on Machine Learning, 2021.
Alexander Jung. Networked Exponential Families for Big Data Over Networks. IEEE Access, 2020.
Alexander Jung and Natalia Vessclinova. Analysis of network lasso for semi-supervised regression. In The 22nd International Conference on Arlificial Intelligence und Statistics, 2019.
Alexander Jung, Nguyen Tran, and Alexandru Mara. When Is Network Lasso Accurate? Frontiers in Applied Mathematics and Statistics, 2018.
Gi-Soo Kim and Myunghee Cho Paik. Doubly-robust lasso bandit. Advances in Neural Information Pro- cessing Systems, 2019.
Branislav Kveton, Mikhail Konobeev, Manzil Zaheer, Chih-wei Hsu, Martin Mladenov, Craig Boutilier, and Csaba Szepesvari. Meta-thompson sampling. Tn International Conference Machine Learning, 2021.
13

---

Published in Transactions on Machine Learning Research (12/2025)
Lihong Li, Wei Chu, John Langford, and Robert E Schapire. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, 2010.
Shuai Li, Wei Chen, and Kwong-Sak Leung. Improved algorithm on online clustering of bandits. preprint xrxiv:1900 09162, 2019.
arXiv
Miller McPherson, Lynn Smith- Lovin, and James M Cook. Birds of feather: Homophily in social networks. Annual review of sociology, 2001.
Mark EJ Newman. Modularity and community structure in networks. Proceedings of the national academy of sciences, 2006.
Trong T Nguyen and Hady W Lauw. Dynamic chustering of contextual multi-armed bandits. Proceedings of the 23rd ACM international conference on conference on information and knowledge management, 2014.
Behzad Nourani- Koliji, Steven Bilaj, Amir Rezaei Balef, and Setareh Maghsudi. Piecewise-stationary com- binatorial semi-bandit with causally related rewards. In ECAI 2023. IOS Press, 2023.
Min-Hwan Oh, Garud Tyengar, and Assal Zeevi. Sparsity-Agnostic Lasso Bandit. In Proceedings of the 38th International Conference on Machine Learning, 2021.
Francesco Ortelli and Sara van de Geer. Synthesis and analysis in total variation regularization. arXiv preprint trXiv:1901.06411, 2019.
Amit Peleg, Naama Pearl, and Ron Meir. Metalearning linear bandits by prior update. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, 2022.
Gyan Ranjan and Zhi-Li Zhang. Geometry of complex networks and topological centrality. Physica A: Statistical Mechanics and its Applications, 2013.
Xiaoyuan Su and Taghi M Khoshgoftaar. A survey of collaborative filtering techniques. Advances in arlificial intelligence, 2009.
Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 1996.
Joel Tropp. Freedman's inequality for matrix martingales. Electronic Communications in Probability, 2011.
Pict Van Mieghem, Karel Devriendt, and H Cotinay. Pscudoinverse of the laplacian and best spreader node in network. Physicul Review E, 2017.
Yu-Xiang Wang, James Sharpnack, Alexander J. Smola, and Ryan J. Tibshirani. Trend filtering on graphs. Journal of Machine Learning Research, 2016.
Kaige Yang and Laura Toni. Graph-based recommendation system. In 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2018.
Kaige Yang, Laura Toni, and Xiaowen Dong. Laplacian-regularized graph bandits: Algorithms and thcoret- ical analysis. In International Conference on Artificial Intelligence and Statistics, 2020.
Ming Yuan and Yi Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Sociely Series B: Statislical Methodology, 2006.
14

---

Published in Transactions on Machine Learning Research (12/2025)
A
Some helper results
Proposition 2 (Bounds norms of matrix products). Let M E =R and N E Ruxp, Then
[MN 4,1 - | N [MN| F S |MINI
4q E [1,
|MN] F S VIMM M c0.1
JN 2,1 S |
Proof.
First incquality For any [1, oo], we have:
le |N
M Σ e N
max 18j5n
lee Me,
Σ |le] Ml1,
max 1555n
(M)
|| 1|
Second inequality
We have
| Σ |MNe, 1|
M |el IMII
Third inequality
We have
|MN = Tr(MNNNTTT MT) s ||M M lo0.000 NN !|
Elements of (i.j) entry of matrix NNT is the inner product
(e, Ne,,Nee,,M N). Hence, have
lee leTne N/| = || INN = EKee,N, Σ
4
Fourth inequality We have
|MN 2.1 = Σ eeN S Σ leM IN = |M [|
Proposition 3 (Decomposition of signal over a graph). For any C P
Let Z E RlVixd be graph signal. Let us denote by Ze the signal obtained from Z by setting rows of verlices outside of C to zeros, and lel Z E Ridlxd be the signal oblained from Ze by vemoving the rows of vertices outside of C. Also, let Bic |elel be the matriz obtained by taking Be, and removing rows of edges that link C to its outside, and the resulting null columns. It is clear that
B BeZe BeZ
(10)
15

---

Published in Transactions on Machine Learning Research (12/2025)
Let Qe == B.c Then
MM = Σ Jc + Qc
(11)
CEP
Qape = Bbp- Bop = Σ Qe
(12)
CEP
where Jc
1c1 - Qc= E and Qap = B Baps.
While Ecep Jc projects each entry of graph signal onto the mean vector value of ils respective cluster, its residual Qape can be interpreted as the projection onto the respective entries deviation from its cluster mean value.
Proof. Since the proof of the first point is trivial, directly treat. the second point. Denoting B/c the pseudo-inverse of Bie it is a well-known linear algebra result that the matrix Qie Bie is the projector onto the null space e Since C is connected, the null space Bic is unidimensional, and is generated by vector Lee glcl having only ones coordinates. Since the projector into that null space is Jie we deduce that
Le
Z Jic 4 Q/eZIc
Ze Jc + 2eZe
JeZ + QeZ
where in the last line, Qe == Be. Consequently, have
Z ' Ze
CEP
Σ JeZ+Q0t + QcZ
CEP
To prove the second point, we recall that Bops is the incidence matrix obtained by setting rows corresponding to edges 0P to zero. In other words, Bap= is the incidence matrix of the graph after removing the boundary odges, and having exactly |P| connected components. Hence, Byp= has a null space spanned by the {1e]ceP, and the orthogonal projector onto this mull space is Ecep Jc. Combining this fact with the fact that Qape is the projector onto the orthogonal of the null space of Bops, we arrive at the second point.
J
Proposition 4 (On the minimum topological centrality index of graph vertex). Let G be connected B. Let c(G) denote the mindmum
graph with incidence matria B and verter set size N, and let L = B B.B. value of inverses of diagonal element of Lt, called its månimum topological centrality index. Also let a(g) be its algebraic connectivity, defined as the minimum non null eigenvalue of L. Then
((G) = |
c(g) a(g).
g is weightless, then c(g) <
N² N-I'
Proof. Since L PSD, Lt is PSD and hence
Lt 60,00 equal to the maximum diagonal entry of Lt
Taking the inverse proves the first point. Also, this implies that
<())
|=
> |-||
=
(13)
16

---

Published in Transactions on Machine Learning Research (12/2025)
where we used the fact that 11 |1-1l for matrices. This proves the second point of the proposition.
For the last point, assume G is weightless, let comp be the Laplacian of complete graph built on the vertices of G. Then we have Loomp = [ - Jx), where J is the square matrix of dimension N having 1/N as Lemma 4), we have
entries. From Fontan & Altafini 2021
(eomp + +NJN) 1 N
IN N
1 N
comp
JN
JN
(14)
which has diagonal elements N
N
On the other hand, L < Leomp Hence, by Fontan Altafini (2021 lemma 4) we have for any +
L L + aJN) JN/a /ax > (Ecomp + aJN) JN/en L comp
This implies that the maximum diagonal entry of L is at least equal to that of Lompp: i.eto 1 N N². the inverse that entry finishes the proof.
Taking
B Proofs of the different claims
B.1 Additional notation
The regularization term can be written more compactly using the incidence matrix of the graph B E [[X] corresponding to an arbitrary orientation under the following form
Σ
Wmn Wmn| On - -O.l = || 22,1 = |01|e
(15)
ISM<RSM
where the 11-112,1 norm denotes the sum of the La norms the rows of a matrix We provide notations that we use in the proofs of the different statements, in order to reduce the clutter. We define E = as the error signal and its rows by {em)
While Σi=1 projects each entry of a graph signal onto the mean vector value of its respective cluster, its residual Qap- can be interpreted the projection onto the respective entries deviation from its cluster mean value.
Let 7n be vector, vertically concatenated by noise terms of rewards received by node m, then we define E xd as the matrix of vertically concatenated row vectors n Xm-
N.B. Except for the results concerning the regret bound, we consider the case rather than our proofs.
B.2 Oracle inequality
In this section, we present all intermediary theoretical results leading to Theorem 5 stating the oracle
inequality. To reduce clutter, omit the dependence on of several quantities. For instance, write a and θ instead of a(t) and @tt).
Definition 3 (Restricted Eigenvalue (RE) condition and norm, generalization of Definition Let (M, c Raxd a set of positive semi-definite matrices. We say that the matria My = diag(M,.... (MM) verifies the restricted eigenvalue condition with constants and & if
Z
Σ TEV
1
ZZ E S with vows (zihev,
It. possible that the notation ||,11 denotes the sum 2-norms columns in the literature.
17

---

Published in Transactions on Machine Learning Research (12/2025)
Table Notation table.
Notation
Mcaning
Independent of time
V
set of graph vertices
of graph edges
By E RR E
Graph incidence Matrix obtained by setting rows of edges outside I zeros cf. Definition B
Be RIIXIIM
L E RIVIIIMI
On E Rd
preference vector of user/bandit m
@ RIvIxd
matrix of vertically concatenated row preferences vectors Boundary of P: set of edges connecting nodes from different clusters Minimum topological centrality index of a node in C restricted to the graph in C Total weight of 0P, i.e. sum of weights of edges in P Euclidean norm for vectors, largest singular value for matrices Semi-norm defined by PSD malrix A: || x Aa matrix Frobenius norm q-norm of the vector with coordinates equal to the p- p-norm of rows Total variation norm of signal over edges of I
8P 33
(C(C)
((P))
H
F
I. E
Moore-Penrose pseudo-inverse of matrix
vec
vectorization operator consisting in concatenating the columns vertically Kronecker product
1c RII
Vector with entries corresponding to vertices in equal to and elsewhere
Jc VIXIV
equal 1 ICI
Qc RIIII
equal to BC
Q E I
equal BJ
ek
elementary vectors of dimension depending on the context.
Subgaussianity constant variance proxy
Dependent on time t
(t)
set of time steps user m has been encountered before time t
0m E Rd
estimated preference vector of user aser/bandit estimation error for user/ beerbbandit m θ.. 0,
6m E TRd
E R Vixd
vertical concatenation of row vectors cm
7m E (1)
vector subgaussian noise of user m context vector received time time
att) Rdd
m(l) N
X (olxx
data matrix of user m data matrix of context vectors of all users
X E Rod
A. E Bddd
X Xm (potentially associated to time ()
Ay RdDixddII
diag(A,, (A,,... .Am)
K E Rlvixd
matrix of vertically concatenated row vectors 7h X..
where S is the cone defined by:
(G
= IZ R Mixd; aa
1 alp <az G. (w((p)
|Z + (1- (1-x))+1 x2) 111pp)
vw((
(WP)
1
(1 0, = a
02(9, 2 an) = + v 2kw (PP (PP)
max
CEP (C)(C).
min
V (C)
ao
CEP
18

---

Published in Transactions on Machine Learning Research (12/2025)
and the RE semi-norm is defined by || ZrDle v (1 (1--a (1- Bop Bop (i
We have the structure
dependent unknown constants vo and n. for which we assume they guarantee a1 (g, θ v 0.
Proof of Proposition Let Z lcv" be constant per cluster signal, with 1. R as indicator vector with the ith entry equal to 1 if i E C and 0 otherwise. Then Z is contained in any cone S defined in the RE
condition and we have:
Zc
| = || Ico o
1 1c|
= |||||||?
For the right hand side of the RE condition we have:
Σ 1EV
1 M. veo(Z") T M vec(Z")
vec(1cu ) M vec(lcu))
= =) T Eeie, @ @M, (1c8v) iEV
-10 80 Eeee @ M.
(1c (1c
NEV
Σ 1 1e 1 M2
VEV
M.U = T
Σ Mi
iEC
Plugging the results into the RE condition, get:
ΣM, M,
VEE
o² z T (Eiec Mi) 1
||01| ICl
Eiec M;
0 S Amin
C
Lemma 1 (A first deterministic incquality).
Let t be time step. We have
1
Σ
|| em 11 | I (K. E)
1
+
E or
(16)
2ta mev
ta
Proof. By optimality of θ we have
1
Σ
X 6m
ym
+ al
0 S Σ
1m Vm
(17)
2f
mEV
meV
19

---

Published in Transactions on Machine Learning Research (12/2025)
where the second line holds by definition of the observed rewards.
On the one hand, given a user index m E V, and since by definition of the observed rewards we have we have for the least squared terms
Xm 6m ym =
X dm
X - 7m
Cm mmll"
Xml + Xm 0m ym 1
7m Xm
where used the fact that Yo 0m + Mm: which holds by definition of the observed rewards. Summing over the users, and using the definition of K, we have
1
Σ X MEV
ym
Σ
ymll"
1
Σ
1 (K, E)) |m
(18)
2t
2t
28 mEV
mEV
On the other hand, we have for the estimated preference vectors
|0 = Σ
θm.
Wman
(m.n)EE
Σ
Wmn Om 0nl
0ul|+
Σ
0m - 0m1
Wmn
(mp
(m.n)Eap:
+
opp
ape
For the true ones, and for any C E P, let & denote the edges linking the nodes of set of nodes C. It is clear that Op = Ucep Ec as disjoint union, hence
|
Σ (m,
Wmn 0
1
Σ (m,
Wmen 10m - 0nl +
Σ
Wma 0m - 0,.
(m.
|Ollar
Σ
Wmn 1mm 0,1
CEP (m.n)eEe
|
where the last equality holds due to the cluster assumption. Hence, we have
|0 - ||01| = =101l
0p
ap
E
(19)
uppe
where the first incquality holds due to the triangle incquality, and the last one since ||Olaap = 0. Combining 19 we obtain the result of the statement.
Equations 17
In the proof for the oracle inequality, we utilize projection operators on the graph signal, which we define as follows:
While D Jc projects cach entry of a graph signal onto the mean vector value of its respective cluster, its residual Qape can be interpreted the projection onto the respective entries deviation from its cluster mean value.
20

---

Published in Transactions on Machine Learning Research (12/2025)
Lemma 2 (Bounding the error restricted to the boundary). The total variation of E restricted to the boundary verifies
(v2max CEP
E yyp
E EBlor S sw(PP)
/c((C) Epl| +
(20)
min CEP
cg (C)
Proof. The proof relies on decomposition of the || term from Proposition 3
We have
E
Σ JeE + Q&E
up
CEP
8P
Epp + Blype Baps E
OP
Ep Eoloop + Blype Bap E
(21)
0P
where Ep is obtained by setting the error signal on every cluster to its mean.
For the first term the right-hand side, let denote by eg the value of any of Ep belonging to cluster C, which is equal to the mean of errors E over that cluster. Also, denote by (Ep)ap the signal obtained from Ep by setting its rows corresponding to nodes that are not adjacent to any edge in the boundary dp to zeros. Also, let O.C denote the inner boundary set of nodes C,i.c. nodes of that connect it to its complementary. Then holds that:
E p jsp
Bap Ep ||
Bap Epp lap 2,1
Bar 2.1
(Ep)apl
(by Proposition 2
< Bop la.
Ep)op F
Bap 2,,1
|
CEP
10,C|
= B l2.1
Σ CEP
C|
CCI
< | 12,1 max
V(((C)
CI
CEP
CEP
= V2w(OP) max Vig(C) Eplla
(22)
CEP
21

---

Published in Transactions on Machine Learning Research (12/2025)
For the second term, we have
Blppe Bap E
Bop Byps Bap E
up
2,1
Bap Bjpp
E
b0.1
Bap B upc
E
18p
< B
B
llape
BBg 12,
BBppe (B) (ppc))
E Bllape (by Proposition 2
18p 1,1
E appe
min v (C) CEP
= w(PP)
E llae.
(23)
min Veg(C) CEP
The result obtained by combining Equations to
Theorem 4 (Theorem 2.1 of Hsu et al. 2012 ) At time step t, let A E Bebt where E N and let v R a random vector such that for some a 2 0, we have
E [exp( (z,, w))] exp [|
Vu R
Then for any ō E (0, 1), we have with probability at least 1 6:
1 + +1 log 1 | 0z² [1 Allpy log 5
Lemma 3 (Empirical process bound). Let Xm E Tml xd denotes the matriz of collected contert vectors for task m V, then, given collected context matrices (Xmlmev for any (0, 1) we have with probability of least 5:
K 5 as(t)
4
where
as(t) = ago
++2 2
Σ
IT (t) log x
+ max
Tma (f) log
(24)
t
me)
8
mEV
Proof. We recall that K E Bx is the matrix obtained by stacking the row vectors ni X,m vertically. On the one hand, we have
K
Σ x 7m 21 || m|1?
(25)
where Xy diag( (inng(X,".. XI, .,, ,1111) RexdiV]
22

---

Published in Transactions on Machine Learning Research (12/2025)
On the other hand, for any u (u1,..) .4) E R', denoting P(t) = exp (Er=1 un77.). we have
E [[(()) = I. [R [exp(um))) P(1 F(il) (by the law of total expectation)
13 (P 1)13 (exp(uune)FFF-ll (because (ne) are Fi-1 measurable.)
I exp u E
Pt 1)
(by the conditional subgaussianity assumption)
exp
2 _
(by induction)
1 f
exp
(26)
From Equations
25 and
26 we can apply Theorem
4 to matrix Xy and random vector n, which implies
that with a probability at least
8, we have
v S
Σ Am MEE
+2 +2V
Σ mEV
|| Amall log 1 x
+ 2 max
||
Am log g.
meV
where we used the equalities ||
= Emev Tr(A)): | max
Am and | 1|
mEV
x = Eee |l ll.. To arrive the statement of the theorem, we use the fact that the context vectors have Euclidean norms of at most.
Proposition 5 (Probabilistic inequality). With probability at least 1 - 8, we have
Σ
X em + (n) S a2 ap) Ep |r + (1 x:) El)
(27)
2ta mEV
min cc(C) CEP 2w(DP)
where 05 <
Proof.
The proof is a combination of the results of Lemmas
to
We have
X Em 2 + llap-
2tos
mEV
(K, E) + E laa
(by Lemmn
< [Elr +
kl|l Elap + (1
x]] E Ellor
(by Lemma 3
|EPle
E Booo
(vZ max CEP
Vig(C)|
Epl1,
| |
Papp /C)(C)
x)]
E Elaap
a0
an min CEP
(C)(C)
min
CEP
where the last line is an application of Lemma 2 finishes the proof.
Grouping the terms by the type of norm applied
Theorem 5 (Oracle inequality, generalization of Theorem Assume that the RE assumption
holds for the empirical multi-task Gram matrix with constants K. E 1 [1 2w1opp
min V /cs(C) v e CEP
0, I min Vcg(C)- -2) and o > 0. Suppose that max Im (t) S bt for some > 0 and 0g 2 ve(OP) W(PP CEP
23

---

for Published in Transactions on Machine Learning Research (12/2025)
Then, with probobility at least - 5(t), have
6(0) 2
o00
[(0, O 000)
1 2 [ollo8 8(1)
+ 2b log 8(1)
Φ2 22
where
[22(C, 0) + 21 (x))(()))
0(9,0) (02(9, 6) 221 v21 (211+(())()))))
a1 Q) min v (C) +1}
CEP
Proof. Using the previously established results, we obtain
1
X Em 2|2 aa| Bloops
21
Saza2(
E0,a2((,9)) () Ep p + ass( - x)
E Blpp
(by Proposition
5
=0.a2 (9,9 Ep + as(1 =** Bop Byp Bop
(by properties of the pscudo-inverse)
2,1
Sasaz(0,9) Ep + asl Bap -s (1) - x)+
Byp Bap
(by Proposition 2
Sas(as (0, 9) (+
E RE
(by definition of the ||ll norm)
a2( (0, + 1: (x)
Sa
||Em 1l
(using the RE assumption)
ov
mEV
(a2 ( 9) 1s)(K) (BP 12
mm
(28)
2Ø
28t
mEV
u² + v² where the last inequality holds for any 8> O, and is a consequence of the property that ue < any u E R. In the second to last inequality we used the RE assumption, here it is important to mention that the assumption does not hold for any choice of ag- In the definition Sf i.e. the set matrices for which the RE condition holds, we have dgp vu(opy. We can also observe that this set is non increasing for the inclusion operator i.c. the RE condition would become weaker, for increasing og- Thus for any og 2 (w((P) the respective set of the RE assumption is contained in S and any matrix contained in the smaller set is automatically contained in S, allowing us use the RE condition in the proof due our lower bound on ao (b(()))
2
As result, we can bound the norm of Qyp as follows:
11up F
B Byp E
$
L Lbpp
Eloope ope
2as ( 9, ap) + Isi (x) (aPll,),)
(Equation 28 with 3 1).
(29)
0² ( 9, 0o) min (C)
CEP
We can also bound the norm of Ep follows:
Epl' 2
(by RE assumption on empirical multi-task Gram matrix)
MEV
40z O (,0 + (x))
(by Equation 28 with 2)-
(30)
0²
24

---

Published in Transactions on Machine Learning Research (12/2025)
The result is then obtained by combining Equations 29 and (30) along with using the fact that E
Ep Qap E and the expressions of an ( and @2 ( 9, on), and bounding os(() as follows:
at)3
o²
Σ MEV
Xm
1p +
Σ mEV
X
log
+
max mEv
X 1| log
t
mi
0²
I
1 8)
2
Σ
Im (t)2
log
+ 2 max meV
(t) [t) log
mEV
0²
1
+
log
+ log
8
0² -2 t
1
1
log
B.3 Inheriting the RE condition from the true to the empirical data Gram matrix
B.3.1 From the adapted the empirical multi-task Gram matrix
Lemma 4 (Bounding a quadratic form using projections). Let M Mp. E Rdddd symmetric matrices, and let J 11 and Q I-J. J. Then, for any Z Roxd with rows {z, 1=) we have:
p
P Σ 2, M,
S Σ
1Z13+
M
M
1Z1
+2
1 Σ
M
M
7 0
+
max ISiSP
M Z
p
Proof. We have
P
P
2, M,
Σ
z M, +2
(2) - 2) M, Σ (z3 T M,,(2 z)
i=1
Miž
+2 Σ
T. QZM,
Σ
e
T
QZM,2 Qc
(31)
where we used the fact that ZTe;
Je; Z Ze
Let us examine every term on the right-hand side of Equation
3
For the first term, have
P
Z
M
<
Σ
M
M
1
(32)
1=1
i=1
1=1
25

---

Published in Transactions on Machine Learning Research (12/2025)
For the second term, we have
e,
QZM,
S
Σ M,Z Qe: |
i=1
Σ (e, @ M,) vec Z Q) |
e @ M,
vec 2)
i=1
e @ M,)
Q22 |
P M))
e
Σ
c M,)
Q 1
P
Σ
Ce_ Mi))(e, & M,)
1
==
P
(eTe, ej M.M,)
| 1|
P
Σ
M
QQZ |||
(33)
Finally, for the last term, we have
P
Σ
e
QZM,Z Qe
M, TT zTQe;ll²
max 1sisp
M
Σ
I Qeill
i=1
max issP
|
(34)
Combining Equations
32 to 34 yields the result.
We also define an operator norm that is induced by the [llRe introduced in Definition 3
Definition 4 ((RE.S)-induce) operator norm). Let (Mm tmev C Rddd be symmetric matrices associated to the graph nodes V, and let My == diag (M,...,MMM) E [ddV)xd For any cluster C E P, lel the cluster mean and mean of squares associated to those matrices be given by
Mc = Σ Mm
M =
Σ M
C
CC
MEC
MEC
The R.E-induced operator norm of My is defined
M RE,S = max
Mc
V
min
(C(c
max
M c
V min CEP
(C
max mEV
Mm
(35)
CEP
CEP
CEP
26

---

Published in Transactions on Machine Learning Research (12/2025)
B.3.2 Linking the adapted to the empirical Gram
We first start by establishing that given the closeness of two PSD matrices in a certain sense, the RE condition can be transferred between them. For the sake of readability we remove the arguments of the constants: 01 = an ( θ vo(opp) 02 02 (G, θ pw(oPT)
Proposition (Restricted spectral norm). Let Z E RRll verifying
@1 Z op S 02 Zrl1F F + +1 - 11)
Let {Mm Jev S Rdxdd be symmelric matrices associaled to the graph. modes V, and let My = diag( MM,.... MDr1) E [dMVixdIV) Then we have:
02 + (1 x)+ Bar (1+ 1+
Σ mEV
m T Mn Zm
$ | RE,S
| 2 Re
(36)
Proof. For any cluster C, we denote by Bc the incidence matrix obtained by setting the rows of B outside the edges linking nodes in to mull vectors. The latter's 's mull space is the span of the vector 1c having coordinates at nodes in C and zeros elsewhere. Hence, the projector onto the orthogonal of le is Qe == B.
On the one hand, for any signal Z R Mi have
Σ
BeZl1a 2,1
ap:
CEP
B
CEP
00,00
min v cC(C)
Σ
12
CEP
CEP
Hence, by the proposition's assumptions, Z verifies
min
(C)
Z Qo < (az Zp F + (1 - x)]l 2| (p))
CEP
CEP
Zp <
F + (1 xx) Bap
B Dy Bar
< (a2 + (1 x) B 2, ) 2 RE
From Lemma
we have
Σ mEV
T Mm Zm
2 Mn Zm
CEP
Mc
1 +2 Σ
M
12 1211, + Σ
max |M 1|
(37)
CEP
CEP
CEP
where we used Equation 10
27

---

Published in Transactions on Machine Learning Research (12/2025)
This allows us to bound every term in Equation 37
For the second term on the right-hand side, we have
M c
Z Qc
2
CEP
S max
Zpll
Σ CEP
Z
Qc
CEP
min
cg(C)
CEP
max
M
(02
(1 -
(38)
81
CEP
As for the third term, we have
2
Σ max mee
Mm
|l||
l..
I max mey
Mm |l
Σ
||
Z
Qu
CEP
CEP
min og (C) -1 CEP
S max Mm meV
(a2 + (1 1 |B 2 1
B
(39)
02 + (1-
Consoquently, denoting
2,1 and combining Equations (37) to (30), we obtain
Q1
Σ MEV
T M mm
m Zm
max CEP
Mc
2c max CEP
M
+ x²
max iEV
|M
|
max CEP
Mcll)
min CEP
(C)
max CEP
M
v min (c) max
max |MMaI)) (1 + (1+))211211) 1
CEP
iEV
which finishes the proof.
Proposition 7 (Inheritance of a RE condition from close matrix). Assume that the matriz Vy
Ay
verifies the RE condition with constant ó 0, and that
Vv
I go² for some r e
t
oo.
(0, (1 (2+(1-x)+ (2u(8p) (2 - Then b
Ay
verifies the RE condition with consland
3 001-1 711 a2 + (1 x)+ 22w (8P)
(40)
a1
Proof. From Proposition know that
28

---

Published in Transactions on Machine Learning Research (12/2025)
1
Ey Apey= =
[V] cV Vpey + Ev PEv
t
z
cy Vyey - 15 Aee
M
1+ 02 + (1 - x)*1 B 12,1 2,1
Φ²
max mEV
A Ayllop,RE
| RE
a1
Barll2. kt) - ))
02 (1 (1-k))
1 70²
|
RE
a1
where the third inequality is an applicaiton of Proposition 6
Theorem 6 (Matrix Freedman Inequality, Tropp 2011 Consider a matrix martingale {M with dimension di x d2- Let (N())) be the associaled differenc sequence. Assume that for some > 0, we have [|N(())] & A Vt 2 almost surely. Define for any 2 1:
(t)(t)) = Σ E (T(
W (row(() =
Σ T=1
E (NT))
Then, for any u, u, > O,
3u²
P 1: M(]l 2 a and (W Wrow S S (d1 + d2) exp
6U + 2Auu
Corollary 4. Let (N(()) by sequence of matrices of dimension d1 x d2, adapted to filtration {F Let (tx) an increasing sequence with elements in [t] for some < t. Consider the sequence (M(n) of random matrices defoned by
M(n) = Σ BIN
(41)
Then {M) is martingale adapted to the filtration {Fr, IN 18 Moreover, if [[N(()] < b Wr E [t] for some b > 0, then we have
3u
P[]M((NNI 2 u) (di + d2) [66N2> + 2v 2bu
(42)
Proof. We denote E [-FFs) E. [-] for any 8 E N. Also, let C(s) = E5-1 [N()), which is meesuuualrall by construction. We have for any n E [N],
EA, (C((n) Bt,.1 [R, [N()a]]] F.,, [N()))
(43)
Er, (N())) C(Ln) =0
(44)
where the first equality is due to the tower rule since Fin-, Ft,--11- Also, we have for any 2
-V(7)-C(FI = [[(N(T) - 1
(45)
S TT((N(T) C(T))
(46)
-T(((((T) C(T))
(47)
= - (TICCCF)NN))++T(((((C)
(48)
-
S [[N(+)) - +IF(C)))) 26
(49)
29

---

Published in Transactions on Machine Learning Research (12/2025)
Hence (()) C(r) is integrable for any T 1. This shows that (n) is sequence of partial sums of matrix martingale differences, hence it is a matrix martingale.
The second part of the corollary statement is a consequence of Theorem The boundedness of the sequence of martingale differences has already been established above. To verify the second requirement the theorem, let us compute bounds on the norms of Weol and Wrow from Theorem 0 Notice that the two matrices are equal since the difference sequence matrices (ta) are symmetric. Hence, for any [N], we have
W (N) |[ row (N)Il S Tr(W col (N)) V Tr(Wrow (N))
(50)
N
= Tr
Σ
E (tt(tn) - -
(51)
n=1
B
[IN((w))13]
- -E,,-1 (+A--I +T(()) Tr(C(t,,))
(52)
BAG -1 [IN(( TT((C(1,)))
(53)
E
[IN((+))1
SN6..
(54)
By Theorem 6 we have for any > 0
3u
2d exp
An 2 1; 1: and [Wa]l Nb2
(55)
6 + 22nbu
[N) 2 u and [N]]] s Nb² ([] m]
(56)
(57)
where the last line holds because we showed that the inequality [N)](l)] S N62 holds almost surely. (Concentration of the empirical multi-task Gram matrix around the adapted one). Let 2 1, > 0. Then we have:
Proposition
(1 P
V3
max Im (t) S bt mEV
S d(2)Ple Ast ((V) (P]) -22
A2t + 221V/--As0 Ast),
t
op.RE
where
3-2 min CC|t
A == CEP
+ 2
3 min eg (C)t
A2 =
min og (C)
6b + 2/27
CEP
min [C] CEP
3-2² min eg (C)
Ag 6b + 2- 2y min (C) CEP
CEP
Proof. For > 0, let us define
Ay
and G = ( I $7 $
Vy
30

---

Published in Transactions on Machine Learning Research (12/2025)
where By is block diagonal matrix formed by men) We also define De and A in the same pattern of Definition 4 We can express the complementary of this event as the disjunction of a finite number of events as follows:
Geram,y
(58)
max CEP
V
min (c(C) CEP
max CEP
A
V min cg(C)
max mey
Aml > mn
(59)
CEP
U
h)
U
A
> pr
min CEP (C))
U U
A
> by min Co(C)
(60)
CEP
CEP
CEP
mEV
The first and third event can be bounded by considering the sequence xx: (T) adapted to the filtration (Fr).
verifying [x T (T) S.
Bounding the probability of the first event
Let C E: P be cluster. By definition, we have
((Ac(() Σ
xx(()) 2x (T)
mEC TET (t)
Σ
x xx(
TEU meE (m)())
We will apply Corollary for the sequence of time indices in C, i.e. UMEV Tm (t). Hence C is martingale sequence, and we have
-3,2101212
PP
A Ac(t)]
yt
mev (t) s bt
< 2dxxp
Emec [mi(t)] 2//2//lt
3/
$ 2d
6|C|bt + 2/21/Clt
-3 Clt
2dexp
6b + 2/27/
-7²2 min |C CEP
< 2dexp
(61)
6b + 2//27
Bounding the probability of the third event Let m be task index. We apply Corollary for the sequence of time steps in Tm (l)... We have
(t) = Σ
am(() E [ [+]
TE)
is a martingale sequence, hence
-3 min cg 2C CEP
[m)(()]] > min (C) max
ITm (t) < bt
< 2dc
CEP
mEV
6)Tm (1) 2 27 min og (C),
CEP
-3y² min cg 2C) CEP
< exp
6bt 2/222 min co cc(()) CEP
-3 min cg (C) CEP
exp
(62)
6b 2 2 min (C) CEP
31

---

Published in Transactions on Machine Learning Research (12/2025)
Bounding the probability of the second event canonical vector of IR Cl We have
Let C E P be cluster, and let us denote em the m
21
e(t))
Σ MEC
xx(T)
E a (T)
C
(T)
1
Σ
c &
T
Σ xx((T) - E Xx [T
cc
MEC
TETm(()
2
1
Σ
mm(r) @ (xxx(T) E [ (r=((T)(_-,1))
C
rEU mec (t)
1
Σ
em(r) @ xx((T) - E (mm(r) xxx(T)]FF-il
C
FEU
mec (t)
where the last equality holds since m(T) is measurable w.r.t. F+-1- We will apply the Corollary 4 to the set of time steps Unec Tm (t) and the adapted sequence m((r) @ xx((T) of matrices in [ddx[c] Hence we have
P
A c(t)
CEP yt min V cC(
max mEV
Imm (t) S bt
-3>²
min eg (C))² CEP
$ dd(1 exp
6 Emec Im (1) + 2
/IC)
min cg CEP
-3-2 min ca (C) CEP
$ d(1 + exp
exp
6 + 2/27 /
min og (C) CEP
322 min cc(C)t CEP
d(1 + (C]) exp exp
min (C)
6b + 2v27
CEP
C
-3-² min og (C)t CEP
d(1 + exp
exp
(63)
min
(C)
6b + 2/27,
CEP
min CT
CEP
Union bound We conclude the result of the statement via a union bound using Equation
660
Proposition 9 (Concentration of the empirical multi-task Gram matrix around the adapted one, simplified).
Let t 2 1, b> 0. Assume that maxmey 1Tm (t) < bt. Then we have:
yy
3- (mincep((g(C) A (c)2)))
IP
V
>
S 6d Vl exp
2b+2.27
op.RE
where og(C) cg(C) A AC
AC E.
32

---

Published in Transactions on Machine Learning Research (12/2025)
Proof. The proof will rely simple calculus inequalitees. Hence, let mincer cg(C), mincer [C], 37 9 = 6b, h 2/2y, which are all positive. Then, we have
fu (uAv)s (uΛv) (LAUAU)S ftg
A
ftg
ft ((LAAA))))
fu
()A
(DAu)f
(LAUAU)S
A2 f + +
2 f+g 2 (uue)) [+ Miu
f gh
fu²
2 [+ 2 (uAv) (LAUAU)S
(vAu)²
A3 = + go
(IAAAAD
where used the fact that functions of the form Biotpa for positive $1, B2 increasing IR+-
(1Ax)S
x xΛ
As final step, we the incquality ft (1Ax)g function and we use the result of Proposition S
taken for u A u, apply the exp(-.
2)g
f+g
we deduce the result.
B.3.3 From the true to the adapted Gram matrix
For all of the proofs in this subsoction, we follow approach similar to that of et al. (2021). In particular, we use their Lemma 10.
Theorem 7 (Lemma 10 of Oh al. let t 2 1. We have for any 0 E R.:
2021 Under Assumplion on the contert generaling distribulion,
|
Σ E
ET T1
x E arg max (0, .t) #EA(1)
Σ
(64)
21w
(EE)((
Proposition 10 (RE condition from the true to the adapted Gram matrix). Under Assumption 2 for any
121, 1, the adapted Grom matrix Vv(t) verífies the computibility condition with constants & and v 22w
Proof. For 2 1, we have
E [+()<x(t)) [[F-1] =E E
Σ
x())a(1)11F)))
(65)
xEA(1)
Let m V. We have
(1) 1
Σ (t)
E [<((Tx<(7) [F [F=
E (T())(())))))) (1,, [
(law of total expectation)
(e)
Σ
(T. 1)] (xx(T) is fully determined by Om - 1))
TET..(()
(0.0))mm77 - 1
Σ
Σ
xx
x arg max (0, (0,z) &EA((1)
110m(T-
TE)
TEA(TT
Σ (by Theorem
(66)
21n
33

---

Published in Transactions on Machine Learning Research (12/2025)
Now, let Z S, where S is defined with constant of Assumption 4
Then
Σ mEV
|= 2 20w
Σ
by Equation (66
mEV
Z
|IRE (by Assumption
2vw
which finishes the proof.
Theorem (RE condition holding for the empirical multi-task Gram matrix, generalization of Theorem
Under assumplions 2 and let 2 1, and let k. be the constants from Assumplion 4 Assume that
max (t) bt. Then, for any r (o.(1+ (0. -=t(t==)tt/aae ( ) a
-2)
the empirical multi-task Gram
matria verifies the RE condition with constants n. and &, with
02 (1 k)) (2w(())))))) 1 1-7 1
3 à= 0/
(67)
a1
-3722 (minceerr(()(( (C))))
with probability at least equal to 1 6d]V/ exp
where क = 2ees
and
6b + 2v/222
i=(C)== = cg(C) V.P..
Proof. For the sake of readability, let x-
the compatibilitt constant of the adapted Gram matrix,
V2ww
according Proposition
T0
Then:
-3 24 (mince (Cg ög (C))))
6dlV exp
(68)
66 + 2//262
Ay
<P
Vv
< 3
(by Proposition 9
(69)
t
Ro..E
Ay
satisfies the RE condition with constant k and @ (by Proposition 7
<P
(70)
t
where 3 = /011 - (1+ #1+(1-x) (X+KK)))-Vx((PP))
//11>7
B.4 Regret bound
Lemma (Concentration of the fraction of observations per task). Assume that [V] 2 have with a probability at least 1 6:
Then for 8 e (0, 1),
T (t)
V
V 8
max MEV
+
log
log
(71)
V
5
3/
Proof. We have (t)) = Dt (m( m]. where Vt, Vm V,P (,P(m(() m| - mcaning that the binary
variable (m(t) m] follows a Bernoulli distribution B( ( Then, the random variable X, = [m(() m] 1 has mean 0, variance D (1 ) and verifies |X, < M
M
since [V] 2 2. As result, via the Bernstein
incquality, we have for any m E V, and for any w 2 0,
Ti (t)
tw2
tw
+ w
< exp
< exp
V
2(1
M )
2(
M
34

---

Published in Transactions on Machine Learning Research (12/2025)
For the right-hand side to hold with probability at most d (0, 1), it is sufficient to have
w
t
log
2(
[2]
w
M
log
w
2w log
2
and
2
2
2
3/
M
log
4 log
21
1
36
Hence, and via union bound, we get
T (t)
1
1
IP
2
log
+
log
2 8
8
0
P
max
ITm (t)l
1
21
M
log 8
4 log
S 1216
meV
M
3t
The result obtained by adjusting the value of 6.
Theorem 9 (Regret bound, generalization of Theorem 3 Let the mean horizon per node be T = T
VV Under
assumptions7 to and K 0, the expected regret of the Network Lasso Bandit algoritim is upper bounded as follows:
[oof(G, (0m 0, au.)T
R(T) R(T)=O 0
1 (VIVI + Vlog(TIV)) (/ (1V111 log(d[V)) vIPI
<VIV++Vlog(TIVV
2²
3x mincer (C) (C)) with A
6 log' VI
+ +2
V
Proof. For any time step I, we will define a list of good events under which the Oracle inequality and the RE condition for the empirical multi-task Gram matrix both hold with high probability. Then, we will use those bounds to sum up over time steps until horizon
Good events We formalize these requirements three families of time-depending 'good' events.
Gtro(t) is the event that the mean of the empirical process bounded by a(t) up to constant c, which is equivalent to saying that it converges:
Gpro(t) = (+ S
a(l) ag
(72)
(sal()) is the event that the number of selections of all tasks is bounded by its expected value up to small constant p(t)
Tm
(t)
p(t)
Gtel(t) = max mEV
< V
+
(73)
t
GRE(t) is the event that the empirical multi-tan task Gram matrix Av((t satisfies the RE condition.
GRE(t) = { Av(t) verifies the RE condition with constants k, o
(74)
Event Gpro (t) is the most straightforward cover since our bound on the empirical process given in Lemma 3 holds with probability of at least 5(t), thus:
35

---

Published in Transactions on Machine Learning Research (12/2025)
PP (ppoo())9)Ga 8(1),
(75)
where we included the time dependency 0(1) in contrast to the previous section. This way we emphasize to adjust 8(t) after each round, to guarantee sub linear regret bound. The probability of event Gze( be determined using Bernstein's inequality:
From Lemma we can select p(() = 2 M log (a) + 3 log (1)
[v] as well as IP (() S (le)
B.4.1 Instantaneous regret decomposition
Now, given the event probabilities, condition the instantancous regret r(t) on the good events time > lo- We have for its expectation:
E [r(()) & (a(t))saa((t + 2PP (1) $ [r(1)] (1) n GRE(() n (a)
+ 2 (IP pro (t) See (f)] P (t) + P[GGa((t)))
(76)
where we used the worst case bound r(1) 2 if any one of the good events does not hold.
Bounding the regret Inserting results of the event probabilities, the oracle inequality and the de- composition of the expected instantancous regret in Equation (66) and bounding the sum over rounds, yields the final result. Thus, we start by bounding the sum over the first term iee the expected regret in case all good events hold:
T
T
D E Brr(t)]Gpoo(tn GRE(t) n (te)
<
Σ
Taking the result of our oracle inequality in Theorem 5 we point out that only act) is time dependent such that the rest of the terms can be pulled outside the sum:
I
Σ - 1=1
6(t)
and
flG O())
1 1+26 2b
|| 8(t)
26 log 8(t)
I2 Vt"
1=1
T Σ
2000
199, O 0p)
2b
/21V1 log(() + log(!)
4b
1=1
200
f(9, 0,,o)) 2
1
+
VT
2b
(V2[D) log(T) + 2 log dt
VE
2000
1(9, O 00)
(2VT+
V8T
+ 4
32 loggg///00) (DVTT
+
V
16
log( (VT( log(T)
32
[
V1
3
2222 log(T) V2log(T))
0
[oo an))
(vIVT+ v log (T|() + (oggoT](/))
36

---

Published in Transactions on Machine Learning Research (12/2025)
where
[22lG, 0,,) + 221 (x) +1).
f(9, = (a2(G, ao) (21g((()w(0P))
a1(9, 0, min V CC)(C) CEP
We upper bounded the sum with an integralie. Ei= (t) S So g(t)dt for monotonically decreasing functions s(t) in the last inequality. Also b is the bound the concentration of the fraction of observation per task provided by Lemma For to vIV we find by inserting the result to Lemma for all > to:
1 V
V
4
V
2
[VI
8 llog (PIVIN)
2
log
log
I
+
2
8
31.
V
V
3/11
+ VI
log((V)) + log((1))
[
log((V)
= o
= b.
VIVI
Finally we bound the sum over the instantancous regret term for the bad events:
Σ CP
(()))
By construction, we have max(F (F { = (2 Hence,
1
dt 12
IP (()) (1)9 22
2
1+
< 4
(77)
1
t=1
3x2 mincer ( (C)) A
As for the RE condition event, letting A
we have for any to 2
6b 22
P (GGee()")Gaa(()
< 6
exp(-At)
(by Theorem
tu
t=fo
Ato
I <6d[Vle- Ao (1+ 1
1 A
$ [d
e-A
(1 (+ A
s Gd|V/e -Ar
Alo
where the last line, we used the inequality exp(A) 2 1. Hence, for any a > 0, choosing
6d 6dl + 1 1
to = [IVV
log
A
u
implies that EL, P(Gne(t)((Gaa(t) < Now, we simply have to insert all our results into the sum of
instantaneous regrets:
37

---

Published in Transactions on Machine Learning Research (12/2025)
onf 0,00)TT
R(T)= to + 2u +8+0
VIVI+ (Tg(() + 1/yilog(T
8²
1 < Vivi +
6dlvi(1 + 11
log
+ 2u +8
A
+0 coflG, O Voo'T
(VIVI+ + (lo((TIV)) + ((Vllos(((V)))
< M
+
log(12d[V1(1 A1)
+ + +8
1
A
[of a.oVT
(VIVI + (lo((TVV)) + (1Pllo8(TIV))
+0
2²
< |
log(12d[V1(1 +1))
1 +
+ +8
+
A
A
aof(G, + ao)VT
+0
VIVl+ + V log log (Tl)) + 1/ (TIV))
VII + log(d[V)) + ool(9, O og)vT A
[vIl + (og + (1vilo0(
3²
VM + Ā log(d[V1) + flo O vo)
(VIb]+ + (og(T + / (1M1oo((Ti))
d²
where we sot u 24 in the third incquality.
Proof of Corollary 7
Assuming (0 < Ω with some positive constant Ω < 1 and sctting ag min (C) CEP
Bw( then the term f(g, θ og) can be bounded
0,00,00 )w(Op))
02
01 min CEP Veg(C) +1
a2
w(OP) 5 + 22n max v Vio(C)
(w((P)) 2n CEP
max Vis(c))
CEP
/ee(C) - w(BP) 2x) +1
min CEP
u(Pp)² max ig (C)
CEP
(1 - [2) min /cg(C)
CEP
Ω acts as a threshold for the quality of any graph the satisfy this bound. Similarly we can find bound on the term :
38

---

Published in Transactions on Machine Learning Research (12/2025)
-2
log( [V)
1+
VMVI
v2
01
A
-4
3
+
d2
mincep((ag (C) ^ (2((()
-2
w(8P)
1/ 2x max v (s((C) CEP
1
1+
VIvI
-2
1- Ω
VI
(x + 22k max Vcs(C) CEP 1-
mincep(Cs(( Ag)())
2
(8P) 1b + 2xx max Vig(C) CEP 1-Ω
1+
0
mincep mincep(Co(C) (co(C)
w(PP) max (c) CEP
0
11 ?)2 (( A ((()))
Insorting the terms into the regret bound yiclds the final result.
Proof of Corollary 2 With ag vu(oP] we have for aof (9, O ao) first:
w((P)) ④b + 22x max Vis(C) CEP
1 v/w(OP)
(w(OP) (++ 22x max v /o(C)) CEP
0o (G, O,
+1
min CEP
Veg(C) w + 2x)
vb + 2x max Vis(C) CEP 4b
w(PP) 16
22 max V /c((C) CEP
+1
min CEP
v /c((C) w(PP) + 2x)
22x max Vig(C) CEP
11
Where we used max V (C) in the last equality for the zero boundary case. For 4 we have:
CEP
39

---

Published in Transactions on Machine Learning Research (12/2025)
-2
Q2
1
VIVI
v2
a1
A
02
1
1
minc
4
01
glog(
v/29
vIvI
(incee((o(C)) A (C))
Inserting the terms into the regret bound yields the final result.
Proof of Corollary 3 In the case of fully connected clusters, we know from Proposition 4 that the topologival
centrality index of any cluster is given by:
C|2
ca (C)
|C|
[C] -
This allows us to lower bound the following term:
min(er(C) |C]
CEP CEP
Inscrting both lower bounds into our result of Corollary yields the final result.
C Additional experimental details
C.1 About experiments of the main paper
The experiments have been conducted with an intel i7 CPU with 12 2.6 GHz cores and 32 GB of RAM. The two experiments with the highest number tasks (200) and dimension (80) take about hours, parallelized over the 12 cores.
To generate clusters, generate [P] variables Wee from the uniform distribution, then we them to construct a categorical distribution with probabilities proportional to ex These probabilities defines the cluster proportions.
C.2 Solving the Network Lasso problem
We implement the Primal-Dual algorithm proposed in Jung (2020) to solve the Network Lasso problem but we do not vectorize the matrices (in the sense of stacking their columns into a vector), which speeds up computation.
C.3 Algebraic connectivity vs topological centrality index
Given two fully connocted graphs weightless 9, and 92 with size 100 cach, we progressively link them by edges and construct the Laplacian of the resulting graph g. We measure the minimum topological contrality index min [ and the algebraic connectivity, i.e. the minimum non-null eigenvalue of L.
40

---

Published in Transactions on Machine Learning Research (12/2025)
algebraic connectivity
140
minimum topological centrality index
120
100
80
60
40
20
0
0.0
0.2
0.4
0.6
0.8
1.0
fraction of all edges linking the two complete graphs
Figure 2: two fully connected initial graphs G1, G2 with size 100 each.
Minimum Topological centrality index Algebraic Connectivity, for graph formed by connecting
C.4 Fusion rate of clusters
We can plot the fusion rate of clusters, iee. the fraction edges linking nodes with identical feature vector estimations and the respoctive regret as function of time steps:
True fusion rate
1.0
200
150
Ission nto
0.6
0.4
0.2
0.0
Network Lasgo
1000
2000
3000
4000
5000
6000
1000
2000
3000
4000
5000
6000
time
time
(a) Fusion rate
(b) Regret
Figure 3: Fusion rate and regret for the case of non-zero noise within clusters. number of users 100, dimension: 10, number of clusters: 8, imbalance: 1.0, pt 0.5, qE 0.1,
41

---

Published in Transactions on Machine Learning Research (12/2025)
Fusion rate
80-
1.0
True fusion rase
0.8
60
fuxion ale
0.6
cururaaive
0.0
Network
1000
2000
2500
3000
500
1000
1500
2000
2500
3000
steps
(a) Fusion rate
(b) Regret
Figure 4: Fusion rate and regret for the case of non-zero noise within clusters. number of users 100. dimension: 10, number of clusters: 8, imbalance: 1.0, p: 0.9, 0.05,
42


Reviews:
[{"review_id": "xqmYxJWmNU", "reviewer": "Reviewer_7sKd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths.\n\nThe loop is simple and coherent: decide, refit with TV, repeat. When clusters are truly constant, TV is a good inductive bias, it pulls neighbors together while keeping boundaries sharp. The theory surfaces graph effects through $c_{\\mathcal G}(C)$ and $w(\\partial\\mathcal P)$, which is helpful.\n\nPoints that need clarification.\n\n1. Checking the safe regime. \n   The key inequality in Assumption 4\n   $$\n   \\min_{C\\in\\mathcal P}\\sqrt{c_{\\mathcal G}(C)} > 2 w(\\partial\\mathcal P)\n   $$\n   depends on the true partition $\\mathcal P$, which we do not know before deployment. On the same graph, plausible but different partitions can flip the inequality. How should a practitioner test or approximate this gate without $\\mathcal P$? Is there a proxy that uses only the graph and the observed data, for example spectral or commute-time surrogates, or a fusion-stability diagnostic under small edge perturbations?\n \n2. Exact versus near-constant clusters. \n   Exact equality within a cluster feels too strong. A more realistic model is $\\theta_m=\\theta_C+\\varepsilon_m$ with $\\|\\varepsilon_m\\|\\le\\rho$. Can you provide a finite-sample statement for this near-constant case? For instance, if the inter-cluster margin $\\Delta_\\star>2\\rho$ and the per-edge TV threshold $\\tau_{mn}(t)=\\frac{\\alpha(t)w_{mn}}{2(n_m+n_n)}$ shrinks with $t$, can we bound the probability of cross-cluster fusion as\n\n$\\mathbb{P}\\left[\\text{fusion across clusters at time } t\\right] \\le \\exp\\Big(-c \\big(\\Delta_\\star - 2\\rho - \\tau_{mn}(t)\\big)^{2} n_{\\min}(t)\\Big) ,.$\n\n   If not, please state clearly what blocks such a result, for example global KKT coupling or the lack of time-uniform concentration.\n\n3. Tuning and robustness. \n   How should one choose $\\alpha_0$ and the schedule $\\alpha(t)$ without knowing margins or $w(\\partial\\mathcal P)$? How sensitive is the method to heavier cross edges or thin chain-like clusters, that is small $c_{\\mathcal G}$? A compact set of stress tests would help: vary $w(\\partial\\mathcal P)$ by adding or reweighting cross edges, add a small intra-cluster noise radius $\\rho$, and plot regret and fusion rate.\n\n\n4. Typo: “To arrive the the statement of the theorem.”", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies multi-task linear contextual bandits on a fixed user graph. Each round the algorithm selects a user, picks the arm that maximizes $\\langle \\hat\\theta_{m(t)}, x\\rangle$, then refits all user parameters with a Network Lasso objective.\nThe main idea is to assume user parameters are piecewise constant on the graph and to use TV regularization to fuse connected users. The paper provides an oracle inequality, a transfer from population RE to empirical RE as rounds grow, and a regret bound with graph-aware constants.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Provide a way to diagnose or approximate the gate without $\\mathcal P$, for example a proxy from graph or data statistics. \n2. Give a near-constant result, theoretical or empirical, with an explicit bound on mis-fusion probability under a margin condition $\\Delta_\\star>2\\rho$. \n3. Add robustness experiments that vary $w(\\partial\\mathcal P)$ and the intra-cluster noise radius $\\rho$, and report regret and fusion rates.\n4. Clarify $f(\\mathcal G,\\Theta)$ and the components of $\\alpha(t)$, and fix the typo. \n5. Add an ablation on $\\alpha_0$ and the TV weight, and report the fraction of fused edges versus performance. \n6. Test on a noisy or misspecified graph such as a perturbed stochastic block model.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "89RXFflbTg", "reviewer": "Reviewer_WjeX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n1. The paper provides an algorithm which only assumes a cluster structure between the preference vector and does not know what the exact cluster structure.\n\n2. Theoretical results comply with the intuitions that the more \"interior nodes\" a cluster contains and more disconnected the clusters are, the cleaner it is to \"lasso\" the parameters.\n\n\nWeakness:\n\n1. It is not clear why would this approach help as compared to the fact that one can simply cluster the nodes earlier and then run the learning algorithm independently. Having a result on the clustering error and probability of identifying a bad cluster given the graph and then bounding that with $\\mathcal{O}(T)$ would significantly improve the paper and the motivation of using the proposed lasso regression on the preference vectors with edge weights as regression parameters.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper considers the problem of minimizing regret while serving $\\mathcal{V}$ tasks. Every task has their own preference which makes the learning difficult, but assuming task share common behavior can reduce the difficulty. The paper focusses on the shared common by assuming that the preference vectors for the common behavior tasks are close (in some metric space).  This setup holds importance in various use cases such as multi-task learning for robots, personalized recommendations, etc.\n\nFor this setup, the paper provides a Lasso regression algorithm which utilizes the graph, $\\mathcal{G} = (\\mathcal{V} , \\mathcal{E})$, between the tasks/users. The paper assumes that the tasks form clusters by sharing the preference vectors, and this is represented by connection in the graph $\\mathcal{G}$.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "In the evaluation, the authors construct graphs with stochastic block cluster:\n\n\"The graph used is weightless and generated using a stochastic block model to ensure a cluster structure, where an edge is constructed with probability $p$ within clusters and $q$ between clusters\" with $p\\ge 4q$ as evident from Figure 1. \n\n1. What happens if we pre-cluster the graph and solve the problem independently for the identified clusters? \n2. What happens when we bring p to as close as q as possible? When which cases other algorithm start showing to be more useful?\n\nI would request authors to add experimental result with the above two points.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "x242UyswGt", "reviewer": "Reviewer_Tyz5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The strength of the paper is in formulating a (possibly) more realistic model of the collaborative filtering with cold-start problem. The main weakness lies in the lack of clarity and poor exposition.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a novel algorithm for multi-task contextual bandit problems that leverages graph structures to improve preference estimation and reduce regret in recommendation systems. The authors also provide detailed experiments comparing with existing algorithms.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The lack of clarity was glaring in the paper \n\n1. Right from the abstract, the following sentence is used multiple times throughout the paper without any explanation \n\n\"The tasks’ preference vectors are assumed to be piecewise constant over the graph, forming clusters\" - What does piecewise constant mean in this context? What does cluster mean? What do two tasks being in the same cluster share? Are the clusters hard? These are many questions which remain unanswered throughout. \n\n2. However, the Laplacian regularization does not account for the smoothness heterogeneity introduced by a piecewise constant behavior over the graph - this sentence does not make any sense to me and seems like a lot of keywords bundled together\n\n3. However, their clustering can cause overconfidence in the constructed clusters, potentially leading to error accumulation - What is the exact criticism here? Theoretical guarantees have been established in the cited papers? Are the authors claiming that their model is not a good representation of practice? It is important for the authors to point out exactly what is missing in the model of the cited papers - that is because, under the modeling assumption, the algorithms are good provably!\n\n4. First, we assume access to a graph encoding relations between users, which is more informative than a complete graph - Any graph encodes relations - so I am not sure what this sentence mean\n\n5. Regarding the related work on multi-task learning, It needs to be clarified what is the vanilla regret ... what is the regret when clusters is smaller than dimension..what is their regret when that condition does not hold and what do the authors get in the regime when clusters are smaller than dimension? In general, the ambiguity in writing is too high. \n\n6. How should I think about K? Why does it need to be finite? \n\n7. The definition of \"piecewise constant\" comes in Page 4 but it makes me more confused - How is it different from standard online clustering of bandits? What is the point of the weighted graph which is supposed to carry more information than a complete graph? Also, how are the weights learnt? It is not possible for the weights to be known apriori in any practical scenario, right?\n\n8. The definition of RE looks extremely complex and unreadable. Why so? How is this definition different from the standard RE definition in the following paper? \n\nhttps://arxiv.org/abs/1106.1151", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "tCJcaaOdlO", "reviewer": "Reviewer_K6nF", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### **Strengths:**\n\n1. The algorithm leverages user similarity through a weighted graph, allowing for more efficient preference estimation compared to learning each user independently.\n\n2. By using lasso regularization to implicitly model similarity between users, the method avoids reliance on hard clustering. This helps reduce cumulative regret due to mis-specified or imperfect cluster assignments.\n\n3. The paper provides both regret bounds and simulation results to validate the algorithm’s effectiveness in various settings.\n\n---\n\n### **Weaknesses:**\n\n1. While the setting introduces a weighted user network, it is unclear how the edge weights reflect user similarity. The weights appear arbitrary and are not explicitly incorporated into the assumptions on the preference vectors $\\theta$ or into the regret definition (e.g., Equation (2)). This lack of clarity raises concerns about how crucial the network structure is to the algorithm’s theoretical guarantees.\n\n2. The assumption regarding balanced covariance is not clearly defined. It is possible that an indicator function or additional condition is missing, making it difficult to interpret or verify in practice.\n\n3. The algorithm appears to rely solely on exploitation. Although the authors mention that exploration is implicitly handled via the time-dependent regularization parameter $ \\alpha(t) $, this mechanism merely enforces similarity across connected users. In extreme cases (e.g., when all edge weights are zero), the algorithm reduces to greedy per-user learning without any exploration. This raises concerns about the ability to discover near-optimal policies in the absence of sufficient exploration.\n\n4. It is not evident how the regret benefits from leveraging the network. If we simply learn each user independently, the regret remains on the order of $ \\sqrt{|V|} \\cdot \\sqrt{\\bar{T}} $. The paper should clarify what gains are achieved by introducing the network and how the regret scales with the dimensionality $ d $ of the preference vectors. The current analysis lacks transparency in its dependence on the feature space dimension.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work studies the contextual bandit problem under a network structure, where users with similar preferences are connected by weighted edges. By leveraging this network, the learner can accelerate preference estimation through shared information across connected users. The authors propose a novel algorithm, Network Lasso Policy, which, unlike previous methods, does not rely on an explicit clustering of users. Instead, it uses a lasso regularization on the differences across edges, implicitly encouraging similar behavior between connected users. The authors also provide both theoretical regret guarantees and empirical results to demonstrate the effectiveness of their approach.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Clarify how edge weights reflect user similarity  \n   It's not clear how the weights $w_{mn}$ are defined or used. They don't appear in the assumptions on $\\theta$ or in the regret definition. Please clarify their role and how they affect the algorithm or analysis.\n\n2. Fix the statement of Assumption 2  \n   The balanced covariance assumption is unclear. It may be missing an indicator function or needs to be written more precisely. Please revise the assumption clearly, with a formal definition.\n\n3. Clarify whether the algorithm includes exploration  \n   The algorithm seems to only exploit current estimates. The claim that exploration is handled through $\\alpha(t)$ is not convincing. Please explain how the algorithm explores, or discuss this limitation.\n\n4. State the regret improvement from using the network  \n   It's unclear what benefit the network provides in terms of regret. Please compare the regret to the case where users are learned independently and explain the dependence on dimension $d$.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)