SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Assumption 1 (Assumptions on  $\mathbf{X}$ ). Let  $\mathbf{X} = \mathbf{U}\pmb{\Sigma}\mathbf{V}^{\top}$  be the SVD of  $\mathbf{X}$ . Also, let  $\mathbf{X}$  satisfy the following: 1) Condition number:  $\mathbf{X}$  is full-rank and has non zero singular values  $\lambda_1 > \dots >\lambda_C$  with condition number  $\lambda_{1} / \lambda_{\mathsf{C}} = O(1)$ , 2)  $\mu$ -incoherence:  $||\mathbf{V}||_{2,\infty}\leq \sqrt{\mu\mathsf{C} / \mathsf{N}}$ , 3) Subset Strong Convexity: For some  $\alpha$  satisfying  $\alpha \log \mathsf{M} = \Omega (1)$ ,  $\gamma = \widetilde{O} (1)$  for all subset of indices  $S\subseteq [\mathsf{M}],|S|\geq \gamma C$ , the minimum non zero singular value of  $\mathbf{V}_S$  must be at least  $\sqrt{\alpha|\mathcal{S}| / \mathsf{M}}$ .
Remark 2 (Discussion on Assumption 1). We need Assumption 1 only to bound the incoherence factors and condition numbers of sub-matrices whose rows correspond to a nice subset of users and the columns comprise of at least  $\mathsf{T}^{1/3}$  golden items for each user in that nice subset. This implies that information is well spread-out across the entries instead of being concentrated in a few. Thus, we invoke Lemma 1 for those sub-matrices and utilize existing guarantees for low rank matrix completion. Our theorem statement can hold under significantly weaker assumptions but for clarity, we used the theoretically clean and simple to state Assumption 1. Note that the assumption is not required to run our algorithm or any offline matrix completion oracle - the only purpose of the assumption is to invoke existing theoretical guarantees of offline low rank matrix completion algorithms with  $\ell_{\infty}$  error guarantees.
Assumption 2. We will assume that  $\mu, \sigma, \tau, ||\mathbf{P}||_{\infty}, C$  are positive constants and do not scale with the number of users  $M$ , items  $N$  or the number of rounds  $T$ .
Assumption 2 is only for simplicity of exposition/analysis and is standard in the literature. As in [25], we can easily generalize our guarantees if any of the parameters in Assumption 2 scale with In that case, the regret has additional polynomial factors of  $\mu, \sigma, \tau, ||\mathbf{P}||_{\infty}$ , C - moreover these quantities (or a loose upper bound) are assumed to be known to the algorithm. Now, we present our main theorem:
Theorem 1. Consider the blocked collaborative bandits problem with  $\mathsf{M}$  users,  $\mathsf{N}$  items,  $\mathsf{C}$  clusters,  $\mathsf{T}$  rounds with blocking constraint  $\mathsf{B} = \Theta (\log \mathsf{T})^3$  such that at every round  $t\in [\mathsf{T}]$ , we observe reward  $\mathbf{R}^{(t)}$  as defined in eq. (1) with noise variance proxy  $\sigma^2 >0$ . Let  $\mathbf{P}\in \mathbb{R}^{\mathsf{M}\times \mathsf{N}}$  be the expected reward matrix and  $\mathbf{X}\in \mathbb{R}^{\mathsf{C}\times \mathsf{M}}$  be the sub-matrix of  $\mathbf{P}$  with distinct rows. Suppose Assumption 1 is satisfied by  $\mathbf{X}$  and Assumption 2 is true. Then Alg. B-LATTICE initialized with phase index  $\ell = 1$ ,  $\mathcal{M}^{(1)} = [[\mathsf{M}]],\mathcal{N}^{(1)} = [[\mathsf{N}]],\mathsf{C},\mathsf{T},t_0 = 1,t_{\mathrm{exploit}} = 0,\widetilde{\mathbf{P}} = \mathbf{0}$  and  $\epsilon_{1} = c(\log \mathsf{M})^{-1}$  for some constant  $c > 0$  guarantees the regret  $\operatorname {Reg}(\mathsf{T})$  defined under the blocking constraint (eq. 2) to be:
$$
\operatorname {R e g} (T) = \widetilde {O} \left(\sqrt {T \max  \left(1 , \frac {N}{M}\right)}\right) \tag {4}
$$
Note that the regret guarantee in Theorem 1 is small as long as  $\mathsf{N} \approx \mathsf{M}$  (but  $\mathsf{M}, \mathsf{N}$  can be extremely large). The dependence of the regret on number of rounds scales as  $\sqrt{\mathsf{T}}$ . This result also matches the  $\sqrt{\mathsf{T}}$  dependence on the number of rounds without the blocking constraint [17, 25]. Furthermore, Theorem 1 also highlights the importance of collaboration across users - compared with standard multi-armed bandits, the effective number of arms is now the number of items per user.
Remark 3. Note that in our model, we have assumed that users in the same latent cluster have identical true rewards across all items. Although we have made this assumption for simplicity as in other works in the literature [25, 2, 3, 5], the assumption of each user having exactly 1 of C different mean vectors can be relaxed significantly. In fact, in [25], a similar cluster relaxation was done in the following way for a known value of  $\nu >0$  - there are C clusters such that 1) users in the same cluster have same best item and mean reward vectors that differ entry-wise by at most  $\nu 2)$  any 2 users in different clusters have mean reward vectors that differ entry-wise by at least  $20\nu$ . Our analysis can be extended to the relaxed cluster setting with an addition cost in regret in terms of  $\nu$ .
We now move on to a rigorous lower bound in this setting.
Theorem 2. Consider the blocked collaborative bandits problem with M users, N items, C = 1 cluster, reward matrix  $\mathbf{P} \in [0,1]^{\mathsf{M} \times \mathsf{N}}$ , T rounds with blocking constraint B such that at every round  $t \in [\mathsf{T}]$ , we observe reward  $\mathbf{R}^{(t)}$  as defined in eq. (1) with noise random variables  $\{\mathbf{E}_{u\rho_u(t)}^{(t)}\}_{\substack{u \in [\mathsf{M}] \\ t \in [\mathsf{T}]}}$
generated i.i.d according to a zero mean Gaussian with variance 1. In that case, any algorithm must suffer a regret of at least
$$
\operatorname {R e g} (T) = \Omega \left(\max  \left(\sqrt {\frac {N B}{M}}, \frac {T \sqrt {\log (N / T)}}{B \sqrt {M}}\right)\right). \tag {5}
$$
Note that the main difficulty in proving tight lower bounds in the blocked collaborative bandits setting is that, due to the blocking constraint of B, a single item having different rewards can only cause a
It suffices for us if  $\mathsf{B} = c\log \mathsf{T}$  for any constant  $c > c'$  where  $c'$  is a constant independent of all other model parameters. This is because, we simply want  $B$  to be at least the number of phases. Since our phase lengths increase exponentially, the number of phases is  $O(\log \mathsf{T})$ .
difference in regret of at most  $\mathsf{B}$  in two separate instances constructed in standard measure change arguments [19]. However, we prove two regret lower bounds on this problem, out of which the latter is the technically more interesting one.
The first term in the lower bound in Thm. 2 follows from a simple reduction from standard multi-armed bandits. Intuitively, if we have  $\mathsf{T} / \mathsf{B}$  known identical copies of each item, then the blocking constraint is void - but with  $C = 1$ , this is (almost) equivalent to a standard multi-armed bandit problem with MT rounds,  $\mathsf{NBT}^{-1}$  distinct arms (up to normalization with  $\mathsf{M}$ ). The lower bound follows from invoking standard results in MAB literature. For  $\mathsf{B} = \mathsf{T}$ , we recover the matching regret lower bound in [25] without the blocking constraint. Furthermore, for  $\mathsf{B} = \Theta (\log \mathsf{T})$ , the first term is tight up to log factors when number of rounds is small for example when  $\mathsf{T} = O(1)$ . (Appendix E)
The second term in the lower bound is quite non-trivial. Note that the second term is tight up to log factors when the number of rounds  $\mathsf{T} = \Theta(\mathsf{N})$  is very large (close to the number of items) and  $\mathsf{B} = O(\log \mathsf{T})$  is small. We obtain this bound via reduction of the regret problem to a multiple hypothesis testing problem with exponentially many instances and applying Fano's inequality [28] (commonly used in proving statistical lower bounds in parameter estimation) for approximate recovery. Our approach might be of independent interest in other online problems as well. (Appendix E)
Remark 4. We leave the problem of extending our guarantees for  $\mathsf{B} = 1$  as future work. However we make progress by assuming a cluster structure over items as well. More precisely, suppose both items and users can be grouped into a constant number of disjoint clusters as such that (user, item) pairs in the same cluster have the same expected reward. Then, we can sidestep the statistical dependency issues in analysis of Alg. 1 for  $\mathsf{B} = 1$  and provide similar guarantees as in Thm. 1 (see Appendix F).
4 B-LATTICE Algorithm
B-LATTICE is a recursive algorithm and runs in  $O(\log T)$  phases of exponentially increasing length. B-LATTICE assume the knowledge of the following quantities - users M, items N, clusters C, rounds T, blocking constraint B, maximum true reward  $||\mathbf{P}||_{\infty}$ , incoherence factor  $\mu$ , cluster size ratio  $\tau$ , noise variance  $\sigma^2 > 0$  and other hyper-parameters in Assumption 1. At any phase, the algorithm maintains a partitioning of users into crude clusters, and a set of active items for each such crude cluster. Let  $\mathcal{M}^{(\ell)}$  be the partitioning of users in the  $\ell^{th}$  phase, and  $\mathcal{N}^{(\ell)}$  be the list containing the set of active items for each group of users in  $\mathcal{M}^{(\ell)}$ . In the first phase, we place all users into a single group and keep all items active for every user; that is,  $\mathcal{M}^{(1)} = [[\mathsf{M}]]$ ,  $\mathcal{N}^{(1)} = [[\mathsf{N}]]$ . There are three key components in each phase: (a) exploration, (b) exploitation, and (c) user clustering. In what follows, we explain these components in detail. For simplicity of exposition, suppose Assumption 2 is true namely the parameters  $\mu, \sigma, \tau, ||\mathbf{P}||_{\infty}$ ,  $C$  are constants.
Exploration (Alg. 2). The goal of the Explore sub-routine is to gather enough data to obtain a better estimate of the reward matrix. To be precise, let  $\mathcal{M}^{(\ell,i)}$  be the users in the  $i^{th}$  group at phase  $\ell$ , and  $\mathcal{N}^{(\ell,i)}$  be the set of active items for this group. Let  $\mathbf{P}_{\mathcal{M}^{(\ell,i)},\mathcal{N}^{(\ell,i)}}$  be the sub-matrix of  $\mathbf{P}$  corresponding to rows  $\mathcal{M}^{(\ell,i)}$  and columns  $\mathcal{N}^{(\ell,i)}$  (recall, this sub-matrix has rank at most  $\mathbb{C}$ ). Our goal is to get an estimate  $\widetilde{\mathbf{P}}_{\mathcal{M}^{(\ell,i)},\mathcal{N}^{(\ell,i)}}$  of this matrix that is entry-wise  $O(2^{-\ell})$  close to the true matrix. To do this, for each user in  $\mathcal{M}^{(\ell,i)}$ , we randomly recommend items from  $\mathcal{N}^{(\ell,i)}$  with probability  $p = O\left(\frac{2^{2\ell}\log d_1}{d_2}\right)$  (Line 6). Here,  $d_1 = \max(|\mathcal{M}^{(\ell,i)}|, |\mathcal{N}^{(\ell,i)}|)$ ,  $d_2 = \min(|\mathcal{M}^{(\ell,i)}|, |\mathcal{N}^{(\ell,i)}|)$ . We then rely on low-rank matrix completion algorithm (Algorithm 4) to estimate the low rank sub-matrix. By relying on Lemma 1, we show that our estimate is entry-wise  $O(2^{-\ell})$  accurate. This also shows that our algorithm gets more accurate estimates of  $\mathbf{P}_{\mathcal{M}^{(\ell,i)},\mathcal{N}^{(\ell,i)}}$  as we go to the later phases.
User Clustering (lines 7-8 of Alg. 1). After the exploration phase, we refine the user partition to make it more fine-grained. An important property we always want to ensure in our algorithm is that  $\mathcal{M}^{(\ell ,i)}$ , the  $i^{th}$  group/crude cluster in phase  $\ell$ , is a nice subset for all  $(\ell ,i)$  (see Definition 1 for a definition of nice subset). To this end, we build a user-similarity graph whose nodes are users in  $\mathcal{M}^{(\ell ,i)}$ , and draw an edge between two users if they have similar rewards for all the arms in  $\mathcal{N}^{(\ell ,i)}$  (Line 8 in Alg. 1). Next, we partition  $\mathcal{M}^{(\ell ,i)}$  based on the connected components of the graph. That is, we group all the users in a connected component into a single cluster. In our analysis, we show that each connected component of the graph is a nice subset.
Algorithm 1 B-LATTICE (Blocked Latent bAndiTs via maTrIx ComplEtion)
Require: Phase index  $\ell$ , List of disjoint nice subsets of users  $\mathcal{M}^{(\ell)}$ , list of corresponding subsets of active items  $\mathcal{N}^{(\ell)}$ , clusters  $\mathsf{C}$ , rounds  $\mathsf{T}$ , round index  $t_0$ , exploit rounds  $t_{\mathrm{exploit}}$ , estimate  $\widetilde{\mathbf{P}}$  of  $\mathbf{P}$ , entry-wise error guarantee  $\epsilon_{\ell}$  of  $\widetilde{\mathbf{P}}$  restricted to users in  $\mathcal{M}^{(\ell)}$  and all items in  $\mathcal{N}^{(\ell)}$ , count matrices  $\mathbf{K}, \mathbf{L} \in \mathbb{N}^{\mathsf{M} \times \mathsf{N}}$ . // 1)  $\mathbf{K}, \mathbf{L}$  are global variables 2)  $\mathcal{M}^{(\ell)}$  is a list of disjoint subset of users, each of which are proved to be nice w.h.p.
Exploitation (Alg. 3). The main goal in the Exploit sub-routine of our algorithm is to identify common golden items for a group of users jointly. Consider a group of users  $\mathcal{M}$  with active items  $\mathcal{N}$  at the beginning of exploit sub-routine invoked in the  $\ell^{\mathrm{th}}$  phase of Algorithm 1. We perform the following test in Line 1 of the Exploit sub-routine: for every user  $u\in \mathcal{M}$ , we check if  $\widetilde{\mathbf{P}}_{u\widetilde{\pi}_u(1)|\mathcal{N}} - \widetilde{\mathbf{P}}_{u\widetilde{\pi}_u(|\mathcal{N}|)|\mathcal{N}}\geq c'2^{-\ell}$  for some constant  $c^\prime >0$  that is if the estimated highest rewarding and lowest rewarding items of the user  $u$  have a significant gap. For all the users that satisfy the above property, we take a union of the items close to the estimated highest rewarding item for each of them (Line 3). We can show that these identified items are actually golden items for all users in the nice subset  $\mathcal{M}$ . Hence, we immediately recommend these identified golden items to every user in the nice subset B times. In case a golden item is blocked for a user, we recommend an unblocked active item (Line 5). Subsequently we remove the golden items from the active set of items (Line 8) and prune it. We go on repeating the process with the pruned set of items until we can find no user that satisfy the above gap property between highest and lowest estimated rewarding items in the active set.
To summarize, in each phase of Algorithm 1, we perform the exploration, exploitation and user clustering steps described above. As the algorithm proceeds, we get more fine grained clustering of users, and more accurate estimates of the rewards of active items. Using this information, the algorithm tries to identify golden items and recommends the identified golden items to users.
Implementation Details: The actual implementation of the algorithm described above is a bit more involved due to the fact that every user has to be recommended an item at every time step (see
Algorithm 2 Explore (Explore Component of a phase)
Require: Phase index  $\ell$ , nice subset of users  $\mathcal{M}$ , active items  $\mathcal{N}$ , round index  $t_0$ , sampling probability  $p$ . //Takes a particular set of users (nice w.h.p.), their corresponding set of active items and returns an estimate of corresponding reward sub-matrix. Unblocked items are (almost) randomly recommended to obtain data - recommendations are kept track of in global variables  $\mathbf{K}, \mathbf{L}$ . Data from previous phases are not used to maintain independence.
1: For each  $(i,j)\in \mathcal{M}\times \mathcal{N}$ , independently set  $\delta_{ij} = 1$  with probability  $p$  and  $\delta_{ij} = 0$  with probability  $1 - p$ .
2: Denote  $\Omega = \{(i,j)\in \mathcal{M}\times \mathcal{N}  |  \delta_{ij} = 1\}$  and  $m = \max_{i\in \mathcal{M}}|j\in \mathcal{N}|(i,j)\in \Omega |$  to be the maximum number of index tuples in a particular row. Initialize observations corresponding to indices in  $\Omega$  to be  $\mathcal{A} = \phi$  Set  $m_{u} = |j\in \mathcal{N}|(u,j)\in \Omega |$
// Create Bernoulli Mask  $\Omega$  with the idea of recommending all items in  $\Omega$
3: for each user  $u \in \mathcal{M}$  do
4: for rounds  $t = t_0 + 1, t_0 + 2, \ldots, t_0 + m_u$  do
5: Find an item  $z$  in  $\{j\in \mathcal{N}\mid (u,j)\in \Omega ,\delta_{uj} = 1\}$  . Set  $\delta_{uj} = 0$  // Find item in  $\Omega$  for the user that has not been recommended yet in this function instantiation
6: If  $\mathbf{K}_{uz} + \mathbf{L}_{uz} < \mathsf{B}$  ( $z$  is unblocked), set  $\rho_{u}(t) = z$  and recommend  $z$  to user  $u$ . Observe  $\mathbf{R}_{u\rho_u(t)}^{(t)}$  and update  $\mathcal{A} = \mathcal{A} \cup \{\mathbf{R}_{u\rho_u(t)}^{(t)}\}$ ,  $\mathbf{K}_{uz} \gets \mathbf{K}_{uz} + 1$ . // Recommend unblocked item in  $\Omega$
7: If  $\mathbf{K}_{uz} + \mathbf{L}_{uz} = \mathsf{B}$  ( $z$  is blocked), recommend any unblocked item  $\rho_u(t)$  in  $\mathcal{N}$  s.t.  $(u,\rho_u(t))\not\in\Omega$ . Update  $\mathbf{L}_{u\rho_u(t)}\gets \mathbf{L}_{u\rho_u(t)} + 1$ . Set  $\mathcal{A} = \mathcal{A}\cup \{\mathbf{R}_{u\rho_u(t')}^{(t')}\}$  where  $t^{\prime} < t$  is the last round when  $\rho_u(t') = z$  was recommended to user  $u$  but the observation  $\mathbf{R}_{uz}^{(t')}$  has not been used in an invocation of the function Estimate. Update  $\mathbf{K}_{uz}\gets \mathbf{K}_{uz} + 1$  and  $\mathbf{L}_{uz}\gets \mathbf{L}_{uz} - 1$ . //Found item is blocked but some observation can be re-used
8: end for
9: for rounds  $t = t_0 + m_u + 1, \ldots, t_0 + m$  do
10: Recommend any unblocked item  $\rho_{u}(t)$  in  $\mathcal{N}$  s.t.  $(u,\rho_u(t))\notin \Omega$  Update  $\mathbf{L}_{u\rho_u(t)}\gets \mathbf{L}_{u\rho_u(t)} + 1$  // No available items in  $\Omega$  to recommend. Recommend some other item
11: end for
12: end for
13: Compute the estimate  $\widetilde{\mathbf{P}} = \mathrm{Estimate}(\mathcal{M},\mathcal{N},\sigma^2,\mathsf{C},\Omega ,\mathcal{A})$  and return  $\widetilde{\mathbf{P}},t_0 + m$  // Low Rank Matrix Completion of Reward Sub-matrix  $\mathbf{P}_{\mathcal{M},\mathcal{N}}$  using observations in  $\Omega$
problem setting in Section 2). To see this, consider the following scenario. Suppose, we identified item  $j$  as a golden item for users in the nice subset  $\mathcal{M}$  (crude cluster) in the Exploit sub-routine invoked in some phase of Alg. 1. Moreover, suppose there are two users  $u_{1}, u_{2}$  in the cluster for whom the item has been recommended  $n_{1}, n_{2}$  times respectively, for some  $n_{1} < n_{2}$ . So, for the final  $n_{2} - n_{1}$  iterations during which the algorithm recommends item  $j$  to  $u_{1}$ , it has to recommend some other item to  $u_{2}$ . In our algorithm, we randomly recommend an item from the remaining active set of items for  $u_{2}$  during those  $n_{2} - n_{1}$  rounds. We store the rewards from these recommendations and use them in the exploration component of future phases. A similar phenomenon also occurs in the Explore sub-routine where we sometimes need to recommend items outside the sub-sampled set of entries in  $\Omega$  (Line 7 and 10 in Alg. 2) To this end, we introduce matrices  $\mathbf{K}, \mathbf{L} \in \mathbb{R}^{M \times N}$  which perform the necessary book-keeping for us.
$\mathbf{K}$  tracks number of times an item has been recommended to a user and the corresponding observation has been already used in computing an estimate of some reward sub-matrix (Line 2 of Alg. 2). Since these estimates are used to cluster users and eliminate items (Lines 7-10 in Alg. 1), these observations are not reused in subsequent phases to avoid statistical dependencies.  $\mathbf{L}$  tracks the number of times an item has been recommended to a user and the corresponding observation has not been used in computing an estimate of reward sub-matrix so far. These observations can still be used once in Line 2 of Alg. 2. Observe that  $\mathbf{K}_{ij} + \mathbf{L}_{ij}$  is the total times user  $i$  has been recommended item  $j$ . In practice, eliminating observations is unnecessary and we can reuse observations whenever required. Hence Alg. 1 can work even when  $\mathsf{B} = 1$  (see Alg. 6 for a simplified and practical version).
Handling very few active items (Line 5 in Alg. 1). Recall, in the exploration step of phase  $\ell$ , we randomly recommend active items with probability  $p = O\left(\frac{2^{2\ell}\log d_1}{d_2}\right)$ . If  $p > 1$ , then we simply recommend all the remaining unblocked items for each user until the end. In our analysis, we can
Algorithm 3 Exploit (Exploit Component of a phase)
Require: Phase index  $\ell$  , nice subset of users  $\mathcal{M}$  , active items  $\mathcal{N}$  , round index  $t_0$  , exploit rounds  $t_{\mathrm{exploit}}$  , estimate  $\tilde{\mathbf{P}}$  of  $\mathbf{P}$  and error guarantee  $\Delta_{\ell}$  such that  $\left|\widetilde{\mathbf{P}}_{\mathcal{M},\mathcal{N}} - \mathbf{P}_{\mathcal{M},\mathcal{N}}\right|_{\infty}\leq 88C\Delta_{\ell}$  with high probability. //Takes a particular set of users (nice w.h.p.), their corresponding set of active items and an estimate of corresponding reward sub-matrix as input. Identifies common golden items for all aforementioned users in this module and recommends them jointly until exhausted - recommendations are kept track of in global variables K,L and active items are pruned.   
1: while there exists  $u\in \mathcal{M}$  such that  $\widetilde{\mathbf{P}}_{u\widetilde{\pi}_u(1)|\mathcal{N}} - \widetilde{\mathbf{P}}_{u\widetilde{\pi}_u(|\mathcal{N}|)|\mathcal{N}}\geq 64C\Delta_\ell \}$  do   
2: Compute  $\mathcal{R}_u = \{j\in \mathcal{N}|\tilde{\mathbf{P}}_{uj}\geq \tilde{\mathbf{P}}_{u\widetilde{\pi}_u(1)|\mathcal{N}} - 2\Delta_{\ell +1}\}$  for every user  $u\in \mathcal{M}$  . Compute  $S =$ $\cup_{u\in \mathcal{M}}\mathcal{R}_u$  // Find common set of golden items for all users in  $\mathcal{M}$    
3: for rounds  $t = t_0 + 1,t_0 + 2,\ldots ,t_0 + |S|\mathsf{B}$  do   
4: for each user  $u\in \mathcal{M}$  do   
5: Denote by  $x$  the  $\lceil (t - t_0 / \mathsf{B})\rceil$  item in  $S$  . If  $\mathbf{K}_{ux} + \mathbf{L}_{ux} <   B$  ( $x$  is unblocked), then recommend  $x$  to user  $u$  and update  $\mathbf{L}_{ux}\gets \mathbf{L}_{ux} + 1$  . If  $\mathbf{K}_{ux} + \mathbf{L}_{ux} = \mathsf{B}$  ( $x$  is blocked), recommend any unblocked item  $y$  in  $\mathcal{N}$  (i.e  $\mathbf{K}_{uy} + \mathbf{L}_{uy} <   \mathsf{B}$ ) for the user  $u$  and update  $\mathbf{L}_{uy}\gets \mathbf{L}_{uy} + 1$  //Recommend all identified golden items   
6: end for   
7: end for   
8: Update  $\mathcal{N}\gets \mathcal{N}\backslash S$  .// Remove golden items and prune active items  $\mathcal{N}$    
9: Update  $t_0\gets t_0 + |S|\mathsf{B}$  and  $t_{\mathrm{exploit}}\gets t_{\mathrm{exploit}} + |S|\mathsf{B}$    
10: end while   
11: Return  $\mathcal{N},t_0,t_{\mathrm{exploit}}$
show that this happens only if the size of surviving items is too small, and when the number of remaining rounds is very small. Hence the regret is small if we follow this approach (Lemma 5). Similarly, when remaining rounds become smaller than  $T^{1/3}$ , we follow the same approach.
Running Time: Computationally speaking, the main bottleneck of our algorithm is the matrix completion function Estimate invoked in Line 13 of the Explore Component (Algorithm 2). All the remaining steps have lower order run-times. Note that the Estimate function is invoked at most  $O(\mathsf{C}\log \mathsf{T})$  times since there are can be at most  $\mathsf{C}$  disjoint nice subsets of users at a time and the number of phases is  $\log \mathsf{T}$ . Moreover, note that the Estimate function (Algorithm 4) solves a convex objective in Line 3 - this has a time complexity of  $O(\mathsf{M}^2\mathsf{N} + \mathsf{N}^2\mathsf{M})$  which is slightly limiting because of the quadratic dependence. However, a number of highly efficient techniques have been proposed for optimizing the aforementioned objective even when  $\mathsf{M}, \mathsf{N}$  are in the order of millions (see [14]).