SlowGuess/NeurIPS_2023_OCR huggingface dataset:
6: Choose a random item  $j \in \mathcal{N}^{(\ell, i)}$  for recommendation to user  $u$ . If  $j$  is not blocked, recommend  $j$  to user  $u$  and observe  $\mathbf{Z}_{uj}$  to be the feedback from user  $u$  for item  $j$ . If  $j$  is blocked, then recommend any  $j' \in \mathcal{N}^{(\ell, i)}$  that is unblocked for user  $u \in \mathcal{M}^{(\ell, i)}$ . Observe  $\mathbf{Z}_{uj'}$  to be the feedback from user  $u$  for item  $j'$ .
7: Update  $\Omega^{(i)} = \Omega^{(i)}\cup \{(u,j)\}$
8: end for
9: end for
11: for  $i^{\mathrm{th}}$  nice subset of users  $\mathcal{M}^{(\ell ,i)}\in \mathcal{M}^{(\ell)}$  with active items  $\mathcal{N}^{(\ell ,i)}$  ( $i^{\mathrm{th}}$  set in list  $\mathcal{N}^{(\ell)}$ ) do
12: Initialize  $\mathcal{M}^{(\ell + 1)} = \varnothing$  and  $\mathcal{N}^{(\ell + 1)} = \varnothing$ .
13: Compute  $\mathbf{T} \in \mathbb{R}^{M \times N}$  by solving the convex program
$$
\min  _ {\mathbf {T} \in \mathbb {R} ^ {N \times M}} \frac {1}{2} \sum_ {(i, j) \in \Omega^ {(i)}} \left(\mathbf {Z} _ {i j} - \mathbf {T} _ {i j}\right) ^ {2} + \lambda \| \mathbf {T} _ {\mathcal {M} ^ {(\ell , i)}, \mathcal {N} ^ {(\ell , i)}} \| _ {\star}, \tag {7}
$$
14: Solve  $k$ -means for users in  $\mathcal{M}^{(\ell, i)}$  using the vector embedding formed by the rows in  $\mathbf{T}_{\mathcal{M}^{(\ell, i)}, \mathcal{N}^{(\ell, i)}}$ . Choose best  $k \leq C$  by using ELBOW method. Denote the cluster of users by  $\{\mathcal{M}^{(\ell, i, j)}\}_{j}$ .
15: for each cluster of users  $\mathcal{M}^{(\ell, i, j)}$  do
16: Compute set of active arms  $\mathcal{N}^{(\ell, i, j)}$  as  $\{s \in \mathcal{N}^{(\ell, i)} \mid \mathbf{T}_{us} \geq \mathbf{T}_{u\widetilde{\pi}_u(T)} - \nu_\ell$  for some  $u \in \mathcal{M}^{(\ell, i, j)}$ . Here  $\widetilde{\pi}_u$  is the permutation of the surviving items in  $\mathcal{N}^{(\ell, i)}$  in descending order of estimated reward for user  $u$ .
17: Append  $\mathcal{M}^{(\ell, i, j)}$  to  $\mathcal{M}^{(\ell + 1)}$  and  $\mathcal{N}^{(\ell, i, j)}$  to  $\mathcal{N}^{(\ell + 1)}$ .
18: end for
19: Compute  $m_{\ell + 1}, \nu_{\ell + 1}$  as a function of  $\ell$ . Set  $t = \min(t_0 + m, T)$ .
20: If  $t < \mathsf{T}$ , invoke Algorithm PB-LATTICE (phase  $\ell + 1$ , phase length  $m_{\ell + 1}$ , list of users  $\mathcal{M}^{(\ell + 1)}$ , list of items  $\mathcal{N}^{(\ell + 1)}$ , clusters  $C$ , rounds  $T$ , noise  $\sigma^2$ , round index  $t_0 + m$ , gap factor  $\nu_{\ell + 1}$ ).
21: end for
1.  $(D1)$  Each entry of  $\mathbf{V}$  is sampled from a gaussian distribution  $\mathcal{N}(0,25)$  with mean zero and variance 25. User  $u$  on being recommended item  $j$  provides a random feedback distributed according to  $\mathcal{N}(\mathbf{P}_{uj},0.25)$ .
2.  $(D2:)$  Each entry of  $\mathbf{V}$  is sampled from a uniform distribution  $\mathcal{U}(0,5)$  with range in [0, 5]. User  $u$  on being recommended item  $j$  provides a random feedback distributed according to  $\mathcal{N}(\mathbf{P}_{uj},0.25)$ .
3.  $(D3:)$  Each entry of  $\mathbf{V}$  is sampled from a uniform distribution [0.05, 0.95] with equal probability. User  $u$  on being recommended item  $j$  provides a like  $(+1)$  with probability  $\mathbf{P}_{uj}$  and a dislike  $(-1)$  with probability  $1 - \mathbf{P}_{uj}$ .
For the PB-LATTICEalgorithm (Alg. 6), we choose the hyper-parameters on the phase lengths and gap parameters as  $m_{\ell} = 10 + 2\ell$  and  $\nu_{\ell} = ||\mathbf{P}||_{\infty} / (8\cdot 2^{\ell})$ . Since PB-LATTICEis a recursive algorithm, we initialize PB-LATTICEwith phase index 1, phase length 12, list of users [[M]], list of items [[N]], clusters C, rounds T, noise  $\sigma^2$ , round index 1, gap factor  $||\mathbf{P}||_{\infty} / 16$ . For the greedy algorithm (Alg. 5), we experiment with two exploration periods ( $m = 10$  and  $m = 30$ ). Finally, for the Collaborative-Greedy algorithm in [4], we choose  $\theta = 0.5$  and  $\alpha = 0.5$ .
Results and Insights: The cumulative reward until round  $t$  (defined as  $\mathsf{M}^{-1}\sum_{j = 1}^{t}\sum_{u = 1}^{\mathsf{M}}\mathbf{P}_{u\rho_u(j)}$ ) and the round wise reward at round  $t$  (defined as  $\mathsf{M}^{-1}\sum_{u = 1}^{\mathsf{M}}\mathbf{P}_{u\rho_u(t)}$ ) is plotted for all the three datasets D1, D2, D3. In Figures 1a, 1b, 1c, 1d, 1e and 1f. Apart from obtaining good empirical rewards, PB-LATTICEhas several practically relevant properties which are demonstrated through our experiments:
1. Note from Figures 1b and 1d that the greedy algorithm has a large cold-start period for good exploitation properties. If the exploration is too small, then the estimation becomes poor. On the other hand, PB-LATTICEimproves in phases - therefore, it has a small cold-start period i.e. it starts recommending relevant items very quickly and also has good estimation guarantees
2. Collaborative-greedy in [4] proposes exploration rounds throughout the entire course of T rounds. This is often impractical as users will demand good recommendations throughout. This is demonstrated in Fig. 1f where the red curve (round-wise reward of Collaborative-Greedy) has several dips but the round-wise regret of PB-LATTICEstays high.
D Detailed Proof of Theorem 1
Next, we characterize some properties namely the condition number and incoherence of sub-matrices of  $\mathbf{P}$  restricted to a nice subset of users. The following Lemmas 2 and 3 can be found in [25]
Lemma 2. Suppose Assumption 1 is true. Consider a sub-matrix  $\mathbf{P}_{\mathrm{sub}}$  of  $\mathbf{P}$  having non-zero singular values  $\lambda_1' > \dots > \lambda_{\mathbb{C}'}'$  (for  $C' \leq C$ ). Then, if the rows of  $\mathbf{P}_{\mathrm{sub}}$  correspond to a nice subset of users, we have  $\frac{\lambda_1'}{\lambda_{C'}'} \leq \frac{\lambda_1}{\lambda_C} \sqrt{\tau}$ .
Lemma 3. Suppose Assumption 1 is true. Consider a sub-matrix  $\mathbf{P}_{\mathrm{sub}} \in \mathbb{R}^{\mathbf{B}' \times \mathbf{A}'}$  (with SVD decomposition  $\mathbf{P}_{\mathrm{sub}} = \widetilde{\mathbf{U}} \widetilde{\boldsymbol{\Sigma}} \widetilde{\mathbf{V}}$ ) of  $\mathbf{P}$  whose rows correspond to a nice subset of users. Then,  $\left\| \widetilde{\mathbf{U}} \right\|_{2,\infty} \leq \sqrt{\frac{C\tau}{N'}}$  and  $\left\| \widetilde{\mathbf{V}} \right\|_{2,\infty} \leq \sqrt{\frac{\mu C}{\alpha M'}}$ .
Lemmas 2 and 3 allow us to apply low rank matrix completion (Lemma 1) to relevant sub-matrices of the reward matrix  $\mathbf{P}$ .
D.1 Main Analysis
Blocked LATTICE is run in phases consisting of exploit component and explore component indexed by  $\ell = 1,2,\ldots$ . Note that the exploit component of a phase is followed by the explore component. However, for the first phase, the exploit component has a length of zero rounds. Importantly note that any phase (say  $\ell$ ) for distinct nice subsets of users  $\mathcal{M}^{(\ell,i)}, \mathcal{M}^{(\ell,j)}$  will be run asynchronously as reward observations corresponding to one subset is not used to determine the policy for any other subset of users. At the beginning of the explore component of each phase  $\ell$ , we have the following set of desirable properties:
(A) We will run the explore component of phase  $\ell$  asynchronously and separately for a list of disjoint nice subsets of users  $\mathcal{M}^{(\ell)} \equiv \{\mathcal{M}^{(\ell,i)}, \dots, \mathcal{M}^{(\ell,a_{\ell})}\}$  and respective sets of arms  $\mathcal{N}^{(\ell)} \equiv \{\mathcal{N}^{(\ell,1)}, \dots, \mathcal{N}^{(\ell,a_{\ell})}\}$  where  $a_{\ell} \leq \mathsf{C}$  such that  $\cup_{i \in [a_{\ell}]} \mathcal{M}^{(\ell,i)} \subseteq [\mathsf{M}]$ ,  $\mathcal{M}^{(\ell,i)} \cap \mathcal{M}^{(\ell,j)} = \emptyset$  for any  $i,j \in [a_{\ell}]$  (i.e. the groups of users are nice subsets of  $[\mathsf{M}]$  with no overlap) and  $\mathcal{N}^{(\ell,i)} \subseteq [\mathsf{N}]$  (i.e. the active set of items  $\mathcal{N}^{(\ell,i)}$  for users in  $\mathcal{M}^{(\ell,i)}$  are a subset of  $[\mathsf{N}]$  but the active sets can overlap). The sets  $\{(\mathcal{M}^{(\ell,i)}, \mathcal{N}^{(\ell,i)})_{i \in [a_{\ell}]}$  remain unchanged during the explore component of phase  $\ell$ . As mentioned before, the round at which the explore component of the phase  $\ell$  starts is different for each subset of nice users  $\mathcal{M}^{(\ell,i)}$ .
(B) At any round  $t \in [\mathsf{T}]$  in a particular phase  $\ell$ , let us denote by  $\mathcal{O}_{\mathcal{M}(\ell, i)}^{(\ell, t)}$  to be the set of items chosen for recommendation (see Step 5 in Alg. 3) in exploit components so far (from round  $t = 1$ ) for users in  $\mathcal{M}^{(\ell, i)}$ . In other words, the items in  $\mathcal{O}_{\mathcal{M}^{(\ell, i)}}^{(\ell, t)}$  have already been recommended to all users in  $\mathcal{M}^{(\ell, i)}$  up to  $\ell^{\text{th}}$  phase. In that case, we also maintain that
$$
\mathcal {O} _ {\mathcal {M} ^ {(\ell , i)}} ^ {(\ell , t)} \subseteq \{\pi_ {u} (t) \} _ {t = 1} ^ {\mathsf {T} / \mathsf {B}}.
$$
the best  $\mathsf{T} / \mathsf{B}$  items (golden items) not chosen for recommendation so far in the exploit phases for users in  $\mathcal{M}^{(\ell, i)}$  are contained in the set of surviving items i.e. if  $\mathcal{Z} = [\mathsf{N}] \setminus \mathcal{O}_{\mathcal{M}^{(\ell, i)}}^{(\ell, t)}$ , then it must happen that
$$
\mathcal {N} ^ {(\ell , i)} \supseteq \bigcup_ {u \in \mathcal {M} ^ {(\ell , i)}} \left\{\pi_ {u} \left(t ^ {\prime}\right) \mid \mathcal {Z} \right\} _ {t ^ {\prime} = 1} ^ {\mathrm {T} / \mathrm {B} - \left| \mathcal {O} _ {\mathcal {M} ^ {(\ell , i)}} ^ {(\ell , t)} \right|} \text {f o r a l l} i \in [ a _ {\ell} ] \tag {8}
$$
Note that eq. 8 implies that for every user  $u \in \mathcal{M}^{(\ell, i)}$ , there are sufficient golden items in  $\mathcal{N}^{(\ell, i)}$  to be recommended for the remaining rounds. To see this, note that there are  $T / B$  golden items at the beginning and no golden items which are unblocked are eliminated. Hence the number of remaining rounds at any point must be smaller than the number of possible allowed recommendations of golden items belonging to the surviving set of items.
(C) Furthermore, for all  $i \in [a_{\ell}]$ , the set  $\mathcal{N}^{(\ell,i)}$  must also satisfy the following:
$$
\left| \mathbf {P} _ {u \pi_ {u} (1) | \mathcal {N} ^ {(\ell , i)}} - \min  _ {j \in \mathcal {N} ^ {(\ell , i)}} \mathbf {P} _ {u j} \right| \leq \epsilon_ {\ell} \text {f o r a l l} u \in \mathcal {M} ^ {(\ell , i)} \tag {9}
$$
where  $\epsilon_{\ell}$  is a fixed exponentially decreasing sequence in  $\ell$  (in particular, we choose  $\epsilon_1 = ||\mathbf{P}||_{\infty}$  and  $\epsilon_{\ell} = C^{\prime}2^{-\ell}\min \left(\| \mathbf{P}\|_{\infty},\frac{\sigma\sqrt{\mu}}{\log\mathsf{N}}\right)$  for  $\ell >1$  for some constant  $C^\prime >0$ ).
Next, at the beginning of the exploit component of phase  $\ell + 1$ , we will have the following set of desirable properties:
(a) At every round  $t$  in the exploit component of phase  $\ell + 1$ , we maintain a list of disjoint nice subsets of users  $\mathcal{M}^{(\ell + 1)} \equiv \{\mathcal{M}^{(\ell + 1, 1)}, \ldots, \mathcal{M}^{(\ell + 1, a_{\ell + 1})}\}$  (where  $\cup_{i \in [a_{\ell + 1}]} \mathcal{M}^{(\ell + 1, i)} \subseteq [\mathsf{M}]$ ) and corresponding sets of items  $\mathcal{N}^{(\ell + 1, t)} \equiv \{\mathcal{N}^{(\ell + 1, t, 1)}, \ldots, \mathcal{N}^{(\ell + 1, t, a_{\ell + 1})}\}$  where  $a_{\ell + 1} \leq C$  and  $\cup_{i \in [a_{\ell + 1}]} \mathcal{N}^{(\ell + 1, t, i)} \subseteq [\mathsf{N}]$ . Note that in the exploit component, we also use the round index in the superscript for item subsets as they can change during the exploit component (unlike the explore component).
(b) We ensure that for any user  $u \in \mathcal{M}^{(\ell + 1, i)}$ , the set of items chosen for recommendation in the exploit component of phase  $\ell + 1$  belongs to the set of best  $\top$  items i.e.  $\{\pi_u(t)\}_{t = 1}^{\top}$ .
Since LATTICE is random, we will say that our algorithm is  $(\epsilon_{\ell},\ell)$ -good if at the beginning of the explore component of the  $\ell^{\mathrm{th}}$  phase the algorithm can maintain a list of users and items satisfying properties A-C. Let us also define the event  $\mathcal{E}_2^{(\ell)}$  to be true if properties (A-C) are satisfied at the beginning of the explore component of phase  $\ell$  by the phased elimination algorithm. We can show that if our algorithm is  $(\epsilon_{\ell},\ell)$ -good then the low rank matrix completion step (denoted by the event  $\mathcal{E}_3^{(\ell)}$ ) in the explore component of phase  $\ell$  is successful (i.e. the event  $\mathcal{E}_3^{(\ell)}$  is true) with high probability.
Conditioned on the aforementioned two events  $\mathcal{E}_2^{(\ell)},\mathcal{E}_3^{(\ell)}$ , with probability 1, during all rounds  $t$  of the exploit component in phase  $\ell +1$ , properties  $a - b$  are satisfied and the event  $\mathcal{E}_2^{(\ell +1)}$  is going to be true. We are going to prove inductively that our algorithm is  $(\epsilon_{\ell},\ell)$ -good for all phases indexed by  $\ell$  for our choice of  $\{\epsilon_{\ell}\}_{\ell}$  with high probability.
Base Case: For  $\ell = 1$  (the first phase), the number of rounds in the exploit component is zero and we start with the explore component. We initialize  $\mathcal{M}^{(1,1)} = [\mathsf{N}],\mathcal{N}^{(1,1)} = [\mathsf{M}]$  and therefore, we have
$$
\left| \max  _ {j \in \mathcal {N} ^ {(\ell , 1)}} \mathbf {P} _ {u j} - \min  _ {j \in \mathcal {N} ^ {(\ell , 1)}} \mathbf {P} _ {u j} \right| \leq | | \mathbf {P} | | _ {\infty} \text {f o r a l l} u \in [ \mathsf {M} ].
$$
Clearly,  $[\mathsf{M}]$  is a nice subset of users and finally for every user  $u\in [\mathsf{M}]$ , the best  $\mathsf{T} / \mathsf{B}$  items (golden items)  $\{\pi_u(t)\}_{t = 1}^{\mathsf{T} / \mathsf{B}}$  belong to the entire set of items. Thus for  $\ell = 1$ , conditions A-C are satisfied at the beginning of the explore component and therefore the event  $\mathcal{E}_2^{(1)}$  is true. Hence, our initialization makes the algorithm  $(||\mathbf{P}||_{\infty},1)$ -good.
Inductive Argument: Suppose, at the beginning of the phase  $\ell$ , we condition on the events  $\bigcap_{j=1}^{\ell} \mathcal{E}_2^{(j)}$  that Algorithm is  $(\epsilon_j, j)$ -good for all  $j \leq \ell$ . This means that conditions (A-C) are satisfied at the beginning of the explore component of all phases up to and including that of  $\ell$  for each
reward sub-matrix (indexed by  $i \in [a_{\ell}]$ ) corresponding to the users in  $\mathcal{M}^{(\ell, i)}$  and items in  $\mathcal{N}^{(\ell, i)}$ . Next, our goal is to run low rank matrix completion in order to estimate each of the sub-matrices corresponding to  $\left\{(\mathcal{M}^{(\ell, i)}, \mathcal{N}^{(\ell, i)})\right\}_{i \in [a_{\ell}]}$ .
Exploration Strategy: Consider a particular nice subset of users  $\mathcal{M}^{(\ell, i)}$  and corresponding active items  $\mathcal{N}^{(\ell, i)}$  (s.t.  $|\mathcal{N}^{(\ell, i)}| \geq T^{1/3}$ ) at the beginning of the explore component of phase  $\ell$  for  $\mathcal{M}^{(\ell, i)}$ . For each user  $u \in \mathcal{M}^{(\ell, i)}$ , we are going to sample each item  $j \in \mathcal{N}^{(\ell, i)}$  with probability  $p$  (to be determined based on the desired error guarantee). Suppose the set of indices sampled in the explore component of phase  $\ell$  is denoted by  $\Omega^{(\ell)} \subseteq \mathcal{M}^{(\ell, i)} \times \mathcal{N}^{(\ell, i)}$ . Now, for each user  $u \in \mathcal{M}^{(\ell, i)}$  we recommend all the unblocked items in the set  $\mathcal{A}_u^{(\ell)} \equiv \{j \in \mathcal{N}^{(\ell, i)} \mid (u, j) \mid \Omega^{(\ell)}\}$  and obtain the corresponding noisy reward (note that for blocked items in the aforementioned set, we have already obtained the corresponding noisy rewards).
For simplicity, we intend to complete the explore component at the same round for all users in  $\mathcal{M}^{(\ell ,i)}$ . If, for two users  $u,v\in \mathcal{M}^{(\ell ,i)}$ , it happens that recommendation of all items in  $\mathcal{A}_u^{(\ell)}$  is complete for user  $u$  but recommendation of all items in  $\mathcal{A}_v^{(\ell)}$  is incomplete for user  $v$ , then for the remaining rounds we recommend to user  $u$  arbitrary unblocked items from  $\mathcal{N}^{(\ell ,i)}$ . Note that this is always possible since the set  $\mathcal{N}^{(\ell ,i)}$  has sufficiently many unblocked items allowing recommendations in the remaining rounds at beginning of explore component of phase  $\ell$ . We start with the following lemma to characterize the round complexity of estimating the sub-matrix  $\mathbf{P}_{\mathcal{M}^{(\ell ,i)},\mathcal{N}^{(\ell ,i)}}$  up to entry-wise error  $\Delta_{\ell +1}$  with high probability using the noisy observations corresponding to the subset of indices  $\Omega^{(\ell)}$ :
Lemma 4. Consider a particular subset of nice users  $\mathcal{M}^{(\ell, i)}$  and corresponding active items  $\mathcal{N}^{(\ell, i)}$  (such that  $\min \left( \mathsf{M} / (\tau \mathsf{C}), |\mathcal{N}^{(\ell, i)}| \right) \geq \mathsf{T}^{1/3}$ ) at the beginning of the explore component of phase  $\ell$ . Suppose  $d_1 = \max(|\mathcal{M}^{(\ell, i)}|, |\mathcal{N}^{(\ell, i)}|)$  and  $d_2 = \min(|\mathcal{M}^{(\ell, i)}|, |\mathcal{N}^{(\ell, i)}|)$ . Let us fix  $\Delta_{\ell+1} = \Omega(\sigma \sqrt{\mu^3 \log d_1} / \sqrt{d_2})$  and condition on the event  $\mathcal{E}_2^{(\ell)}$ . Suppose Assumptions 1 and 2 are satisfied. In that case, in explore component of phase  $\ell$ , by choosing  $1 \geq p = c \left( \frac{\sigma^2 \tilde{\mu}^3 \log d_1}{\Delta_{\ell+1}^2 d_2} \right)$  (for some constant  $c > 0$ ) and using
$$
m _ {\ell} = O \Big (\frac {\sigma^ {2} \widetilde {\mu} ^ {3} \log (M \lor N)}{\Delta_ {\ell + 1} ^ {2}} \max \Big (1, \frac {N \tau}{M} \Big) \log T \Big) \Big)
$$
rounds under the blocked constraint, we can compute an estimate  $\widetilde{\mathbf{P}}^{(\ell)}\in \mathbb{R}^{M\times N}$  such that with probability  $1 - O(\mathsf{T}^{-3})$ , we have
$$
\left| \left| \widetilde {\mathbf {P}} _ {\mathcal {M} ^ {(\ell , i)}, \mathcal {N} ^ {(\ell , i)}} ^ {(\ell)} - \mathbf {P} _ {\mathcal {M} ^ {(\ell , i)}, \mathcal {N} ^ {(\ell , i)}} \right| \right| _ {\infty} \leq \Delta_ {\ell + 1}. \tag {10}
$$
where  $\sigma^2$  is the noise variance,  $\mu$  is the incoherence of reward matrix  $\mathbf{P}$  and  $\widetilde{\mu}$  is the incoherence factor of reward sub-matrix  $\mathbf{P}_{\mathcal{M}^{(\ell ,i)},\mathcal{N}^{(\ell ,i)}}$ .
Proof of Lemma 4. We are going to use Lemma 1 in order to compute an estimate  $\widetilde{\mathbf{P}}_{\mathcal{M}^{(\ell,i)},\mathcal{N}^{(\ell,i)}}^{(\ell)}$  of the sub-matrix  $\mathbf{P}_{\mathcal{M}^{(\ell,i)},\mathcal{N}^{(\ell,i)}}$  satisfying  $\left|\left|\widetilde{\mathbf{P}}_{\mathcal{M}^{(\ell,i)},\mathcal{N}^{(\ell,i)}}^{(\ell)} - \mathbf{P}_{\mathcal{M}^{(\ell,i)},\mathcal{N}^{(\ell,i)}}\right|\right|_{\infty} \leq \Delta_{\ell+1}$ . Since  $\mathcal{M}^{(\ell,i)}$  is a nice subset of users, the cardinality of  $|\mathcal{M}^{(\ell,i)}|$  must be larger than  $\mathsf{M}/(\tau\mathsf{C})$ . Recall that  $\tau$  is the ratio of the maximum cluster size and the minimum cluster size;  $\mathsf{M}/\mathsf{C}$  being the average cluster size implies that the minimum cluster size is bounded from above by  $\mathsf{M}/(\tau\mathsf{C})$ . From Lemma 1, we know that by using  $m_{\ell} = O\left(p|\mathcal{N}^{(\ell,i)}| + \sqrt{| \mathcal{N}^{(\ell,i)}| p \log(|\mathcal{M}^{(\ell,i)}| \delta^{-1})}\right)$  rounds (see Lemma 1) restricted to users in  $\mathcal{M}^{(\ell,i)}$  such that with probability at least  $1 - (\delta + d_2^{-12})$  (see Remark 5),
$$
\left| \left| \widetilde {\mathbf {P}} _ {\mathcal {M} ^ {(\ell , i)}, \mathcal {N} ^ {(\ell , i)}} ^ {(\ell)} - \mathbf {P} _ {\mathcal {M} ^ {(\ell , i)}, \mathcal {N} ^ {(\ell , i)}} \right| \right| _ {\infty} = O \left(\frac {\sigma \sqrt {\tilde {\mu} ^ {3} \log d _ {1}}}{\sqrt {p d _ {2}}}\right).
$$
where  $d_{1} = \max (|\mathcal{M}^{(\ell ,i)}|,|\mathcal{N}^{(\ell ,i)}|),d_{2} = \min (|\mathcal{M}^{(\ell ,i)}|,|\mathcal{N}^{(\ell ,i)}|)$  and  $\tilde{\mu}$  is the incoherence factor of the matrix  $\mathbf{P}_{\mathcal{M}^{(\ell ,i)},\mathcal{N}^{(\ell ,i)}}$ . In order for the right hand side to be less than  $\Delta_{\ell +1}$ , we can set
$p = c\left(\frac{\sigma^2\tilde{\mu}^3\log d_1}{\Delta_{\ell + 1}^2d_2}\right)$  for some appropriate constant  $c > 0$ . Since the event  $\mathcal{E}_2^{(\ell)}$  is true, we must have that  $\left|\mathcal{M}^{(\ell ,i)}\right|\geq \mathsf{M} / (\tau \mathsf{C})$ ; hence  $d_{2}\geq \min \left(\mathsf{M} / (\tau \mathsf{C}),\left|\mathcal{N}^{(\ell ,i)}\right|\right)$ . Therefore, we must have that
where we substitute  $\delta^{-1} = 1 / \mathrm{poly}(\mathsf{T})$  and furthermore, the condition of the Lemma statement implies that  $d_2^{-12} = O(\mathsf{T}^{-3})$ . Hence, we complete the proof of the lemma.
Remark 5 (Remark 1 in [8]). The error probability can be reduced from  $O(\delta + d_2^{-3})$  to  $O(\delta + d_2^{-c})$  for any constant  $c > 0$  with only constant factor changes in the round complexity  $m$  and the estimation guarantees  $\| \mathbf{P} - \widetilde{\mathbf{P}} \|_{\infty}$ . We will use  $c = 12$  in rest of the paper.
Note that although  $\widetilde{\mu}$ , the incoherence factor of the sub-matrix  $\mathbf{P}_{\mathcal{M}(\ell, i), \mathcal{N}(\ell, i)}$  is unknown, from Lemma 3, we know that  $\widetilde{\mu}$  is bounded from above by  $O(\mu)$  (recall that  $C, \alpha, \tau = O(1)$ ).
Exploration Strategy continued: In particular, we choose  $\Delta_{\ell +1} = \epsilon_{\ell} / 176C$  at the beginning of the explore component of phase  $\ell$  for the set of users  $\mathcal{M}^{(\ell ,i)}$  with active items  $\mathcal{N}^{(\ell ,i)}$ . One edge case scenario is when for a particular set of nice users  $\mathcal{M}^{(\ell ,i)}$ , at the beginning of the explore component of phase  $\ell$ , we have  $|\mathcal{N}^{(\ell ,i)}|\leq T^{1 / 3}$ . In this case, this set of nice users  $\mathcal{M}^{(\ell ,i)}$  do not progress to the next phase and in the explore component, we simply recommend arbitrary items in  $\mathcal{N}^{(\ell ,i)}$  to users in  $\mathcal{M}^{(\ell ,i)}$  for the remaining rounds. Another interesting edge case scenario is when  $d_{2} = \min (\left|\mathcal{M}^{(\ell ,i)}\right|,\left|\mathcal{N}^{(\ell ,i)}\right|)$  is so small that the requisite error guarantee  $\Delta_{\ell +1}$  (eq. 10) cannot be achieved even by setting  $p = 1$  i.e. we recommend all the items in the active set. Recall that by our induction assumption,  $\mathcal{N}^{(\ell ,i)}$  is sufficiently large so that it is possible to recommend unblocked items for the number of remaining rounds (say  $\mathsf{T} - t_{\ell}$  with  $t_\ell$  being the round at which the explore component of phase  $\ell$  starts). In that case, the set of users  $\mathcal{M}^{(\ell ,i)}$  do not progress to the subsequent phase; we simply recommend arbitrary unblocked items in the set  $\mathcal{N}^{(\ell ,i)}$  to the users in  $\mathcal{M}^{(\ell ,i)}$  for the remaining rounds. In both the above edge case scenarios, the explore component of phase  $\ell$  for users in  $\mathcal{M}^{(\ell ,i)}$  would last for the remaining rounds from where it starts. We can now show the following lemma:
Lemma 5. Consider a particular subset of nice users  $\mathcal{M}^{(\ell, i)}$  and corresponding surviving items  $\mathcal{N}^{(\ell, i)}$  at the beginning of the explore component of phase  $\ell$ . Suppose  $d_1 = \max(|\mathcal{M}^{(\ell, i)}|, |\mathcal{N}^{(\ell, i)}|)$  and  $d_2 = \min(|\mathcal{M}^{(\ell, i)}|, |\mathcal{N}^{(\ell, i)}|)$ . Now  $\Delta_{\ell+1} = \epsilon_\ell / 176\mathsf{C}$  is such that there does not exist any  $p \in [0,1]$  for which RHS in eq. 3 can be  $\Delta_{\ell+1}$  for estimation of matrix  $\mathbf{P}$  restricted to the rows in  $\mathcal{M}^{(\ell, i)}$  and columns in  $\mathcal{N}^{(\ell, i)}$ . In that case, we must have for all users  $u \in \mathcal{M}^{(\ell, i)}$ ,