SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Next, we apply it to our setting to prove an estimation error lower bound for our designed estimator  $\widehat{X}$ . In our setting,  $G_{\mathrm{max}}$  corresponds to the maximum possible number of sets in  $\mathcal{T}$  that have intersection of size more than  $\mathrm{T} / 4\mathrm{B}$  with some fixed set  $S \in \mathcal{T}$ . Clearly we have
$$
G _ {\max } \leq \sum_ {t = T / 4 B + 1} ^ {T} \binom {N - T} {T - t} \binom {T} {t} \leq T \binom {N - T} {T - T / 4 B} \binom {T} {T / 4} \leq T \Big (\frac {2 N e}{T} \Big) ^ {T - T / 4 B} \Big (4 e \Big) ^ {T / 4 B}.
$$
Furthermore, in our setting, we also have that
$$
\begin{array}{l} \max  _ {\mathcal {S}, \mathcal {S} ^ {\prime} \in \mathcal {T}} \mathrm {K L} (P _ {\mathcal {S}} | | P _ {\mathcal {S} ^ {\prime}}) \\ \leq \sum_ {i \in \mathcal {S} \cup \mathcal {S} ^ {\prime}} \mathbb {E} _ {\mathcal {S}} \mathsf {R} (\{i \}) \max  (\mathsf {K L} (\mathcal {N} (0, 1) | | \mathcal {N} (\Delta , 1)), \mathsf {K L} (\mathcal {N} (\Delta , 1) | | \mathcal {N} (0, 1))) \leq 2 \mathsf {M B T} \Delta^ {2} \\ \end{array}
$$
- this follows from the fact that the distributions  $P_{S}, P_{S'}$  are most separated in KL-Divergence if  $S \cap S' = \emptyset$  and by using the fact that each arm can be pulled at most MB times. Therefore, we must have (provided  $N = c\overline{T}$  for some large enough constant  $c > 0$ , and  $T$  is large enough) for some constant  $c' > 0$
$$
\begin{array}{l} \mathbb {P} (\text {E r r o r}) \geq 1 - \frac {I (\mathcal {S} ; \widehat {S}) + \log 2}{\log \frac {| \mathcal {T} |}{G _ {\max }}} \geq 1 - \frac {2 M B T \Delta^ {2} + \log 2}{\log \left(\frac {(N / T) ^ {T}}{T \left(\frac {2 N e}{T}\right) ^ {T - T / 4 B} (4 e) ^ {T / 4 B}}\right)} \\ \geq 1 - \frac {2 M B ^ {2} T \Delta^ {2} + B \log 2}{c ^ {\prime} T \log (N / T)} \geq 0. 9 - \frac {2 M B \Delta^ {2}}{c ^ {\prime} \log (N / T)}. \\ \end{array}
$$
Therefore, substituting  $\Delta = \frac{c' \sqrt{\log(N / T)}}{B \sqrt{M}}$ , we have that for some constant  $c'' \geq 0$
$$
\inf_{\pi \in \Pi}\sup_{\nu \in \mathcal{E}}\operatorname{Reg}_{\mathsf{MAB}}(\mathsf{MT};\mathcal{E})\geq c^{\prime \prime}\mathsf{T}B^{-1}\sqrt{\mathsf{M}\log(\mathsf{N} / \mathsf{T})}
$$
$$
\inf_{\pi \in \Pi}\sup_{\nu \in \mathcal{E}}\operatorname {Reg}(\mathsf{T};\mathcal{E})\geq \frac{1}{\mathsf{M}}\inf_{\pi \in \Pi}\sup_{\nu \in \mathcal{E}}\operatorname{Reg}_{\mathsf{MAB}}(\mathsf{MT};\mathcal{E}) = \Omega \Big(\frac{\mathsf{T}\sqrt{\log(\mathsf{N} / \mathsf{T})}}{\mathsf{B}\sqrt{\mathsf{M}}}\Big).
$$
Algorithm 7 BBUIC (Blocked Latent Bandits with User and Item Clusters)
Require: Phase index  $\ell$ , List of disjoint nice subsets of users  $\mathcal{M}^{(\ell)}$ , list of corresponding subsets of active items  $\mathcal{N}^{(\ell)}$ , clusters  $C$ , rounds  $T$ , noise  $\sigma^2 > 0$ , round index  $t_0$ , exploit rounds  $t_{\mathrm{exploit}}$ , estimate  $\widetilde{\mathbf{P}}$  of  $\mathbf{P}$ , incoherence  $\mu$ , entry-wise error guarantee  $\epsilon_{\ell}$  of  $\widetilde{\mathbf{P}}$  restricted to all users in  $\mathcal{M}^{(\ell)}$  and all items in  $\mathcal{N}^{(\ell)}$ , count matrix  $\mathbf{K} \in \mathbb{N}^{\mathsf{M} \times \mathsf{N}}$ .
F Restricted Bandits having User and Item Clusters with blocking constraint
B = 1
Recall that in this setting, the  $\mathsf{N}$  items can be grouped into  $C'$  disjoint clusters  $\mathsf{D}^{(1)}, \mathsf{D}^{(2)}, \ldots, \mathsf{D}^{(C')}$  that are unknown. The expected reward for user  $u$  (belonging to cluster  $a \in [C]$ ) on being recommended item  $j$  (belonging to cluster  $b \in [C']$ ) is  $\mathbf{P}_{ij} = \mathbf{Q}_{ab}$  where  $\mathbf{Q} \in \mathbb{R}^{C \times C'}$  is the small core reward matrix (unknown). In this setting, we provide our theoretical guarantees with  $B = 1$  i.e. any item can be recommended to a user only once under the blocking constraint.
Recall that the main reason our theoretical analysis required  $\mathsf{B} = \Theta (\log \mathsf{T})$  for Thm. 1 setting is that if the observations that were used to compute estimates of some reward sub-matrix in a certain phase with certain guarantees that hold with some probability, then conditioning on such estimates with the said guarantees make the aforementioned observations dependent in analysis of future phases. In the BBIC setting, we show that in each phase, the possibility of the nice subsets of users and their corresponding active items is actually bounded and small. On the other hand, the possibilities were exponentially large in the number of items in the GBB setting. Therefore, we can use a for all argument here implying that for a set of already used observations, we can show that for all possible nice subsets of users and their active items, they can be used again to compute acceptable estimates. Such an analysis allows us to provide theoretical guarantees even with  $\mathsf{B} = 1$  for the BBIC setting.
Algorithm 8 Exploit_Item_Cluster(Exploit Component of a phase)
Require: Phase index  $\ell$ , nice subset of users  $\mathcal{M}$ , active items  $\mathcal{N}$ , round index  $t_0$ , estimate  $\widetilde{\mathbf{P}}$  of  $\mathbf{P}$  and error guarantee  $\Delta_{\ell}$  such that  $\left|\left|\widetilde{\mathbf{P}}_{\mathcal{M},\mathcal{N}} - \mathbf{P}_{\mathcal{M},\mathcal{N}}\right|\right|_{\infty} \leq 88C\Delta_{\ell}$  with high probability, similarity graph  $\mathcal{G}_{\mathrm{item}}$  over the set of items  $\mathcal{N}$ .
1: while there exists  $u\in \mathcal{M}^{(\ell ,i)}$  such that  $\tilde{\mathbf{P}}_{u\tilde{\pi}_{u}(1)|\mathcal{N}} - \tilde{\mathbf{P}}_{u\tilde{\pi}_{u}(\mathsf{T} - t_{\mathrm{exploit}})|\mathcal{N}}\geq 64C\Delta_{\ell}\}$  do
2: Compute  $\mathcal{R}_u = \{j\in \mathcal{N}\mid \tilde{\mathbf{P}}_{uj}\geq \tilde{\mathbf{P}}_{u\widetilde{\pi}_u(1)|\mathcal{N}} - 2\Delta_{\ell +1}\}$  for every user  $u\in \mathcal{M}$ . Compute  $\mathcal{S} = \cup_{u\in \mathcal{M}}\mathcal{R}_u$
3: Update  $S \gets \{v \in \mathcal{G}_{\mathrm{item}} \mid v \text{ is connected with some node in } S\}$ .
4: for rounds  $t = t_0 + 1, t_0 + 2, \ldots, t_0 + |\mathcal{S}|\mathsf{B}$  do
5: for each user  $u\in \mathcal{M}$  do
6: Denote by  $x$  the  $(t - t_0)^{\mathrm{th}}$  item in  $\mathcal{S}$ . If  $\mathbf{K}_{ux} == 0$  ( $x$  is unblocked), then recommend  $x$  to user  $u$  and update  $\mathbf{K}_{ux} \gets 1$ . If  $\mathbf{K}_{ux} == 1$  ( $x$  is blocked), recommend any unblocked item  $y$  in  $\mathcal{N}$  (i.e  $\mathbf{K}_{uy} = 0$ ) for the user  $u$  and update  $\mathbf{K}_{uy} \gets 1$
7: end for
8: end for
9: Update  $\mathcal{N} \gets \mathcal{N} \setminus S$ . Update  $t_0 \gets t_0 + |\mathcal{S}|$ .
10: end while
11: Return  $\mathcal{N}, t_0$ .
Algorithm 9 Explore_Item_Cluster(Explore Component of a phase)
Require: Phase index  $\ell$ , nice subset of users  $\mathcal{M}$ , active items  $\mathcal{N}$ , round index  $t_0$ , sampling probability  $p$ .
1: For each tuple of indices  $(i,j) \in \mathcal{M} \times \mathcal{N}$ , independently set  $\delta_{ij} = 1$  with probability  $p$  and  $\delta_{ij} = 0$  with probability  $1 - p$ .
2: Denote  $\Omega = \{(i,j) \in \mathcal{M} \times \mathcal{N} \mid \delta_{ij} = 1\}$  and  $m = \max_{i \in \mathcal{M}} |j \in \mathcal{N}| (i,j) \in \Omega|$  to be the maximum number of index tuples in a particular row. Initialize observations corresponding to indices in  $\Omega$  to be  $\mathcal{A} = \phi$ .
3: for rounds  $t = t_0 + 1, t_0 + 2, \dots, t_0 + m$  do
4: for each user  $u\in \mathcal{M}$  do
5: Find an item  $z$  in  $\{j\in \mathcal{N}\mid (u,j)\in \Omega ,\delta_{uj} = 1\}$  . If  $\mathbf{K}_{uz} == 0$  ( $z$  is unblocked), set  $\rho_u(t) = z$  and recommend  $z$  to user  $u$  . Observe  $\mathbf{R}_{u\rho_u(t)}^{(t)}$  and update  $\mathcal{A} = \mathcal{A}\cup \{\mathbf{R}_{u\rho_u(t)}^{(t)}\}$ $\mathbf{K}_{uz}\gets 1$
6: If  $\mathbf{K}_{uz} == 1$  ( $z$  is blocked), recommend any unblocked item  $\rho_u(t)$  in  $\mathcal{N}$  s.t.  $(u, \rho_u(t)) \notin \Omega$ . Update  $\mathbf{K}_{u\rho_u(t)} \gets 1$ . Set  $\mathcal{A} = \mathcal{A} \cup \{\mathbf{R}_{u\rho_u(t')}^{(t')} \}$  where  $t' < t$  is the round when  $\rho_u(t') = z$  was recommended to user  $u$ .
7: end for
8: end for
9: Compute the estimate  $\widetilde{\mathbf{P}} = \mathrm{Estimate}(\mathcal{M},\mathcal{N},\sigma^2,\mathsf{C},\Omega ,\mathcal{A})$  and return  $\widetilde{\mathbf{P}},t_0 + m$
Therefore in Algorithm 7 for the BBIC setting, we only have a single counter matrix  $\mathbf{K} \in \{0,1\}^{\mathsf{M} \times \mathsf{N}}$  which is binary. The matrix  $\mathbf{K}$  is initialized to be a zero matrix. Whenever an item  $j$  is recommended to user  $i$ , we set  $\mathbf{K}_{ij} = 1$ . If, in a future phase, we need to recommend item  $j$  to user  $i$  again, due to the blocking constraint, we simply reuse the observation (see Step 6 in Alg. 9).
F.1 Algorithm and Discussion
We start with a definition for nice subsets of items analogous to the nice subset of users (see Definition 1).
Definition 4. A subset of items  $S \subseteq [\mathsf{N}]$  will be called "nice" if  $S \equiv \bigcup_{j \in \mathcal{A}} \mathcal{D}^{(j)}$  for some  $\mathcal{A} \subseteq [\mathsf{C}']$ . In other words,  $S$  can be represented as the union of some subset of clusters of items.
As in the analysis of GBB setting, we will have the desirable properties A-C that should be satisfied with high probability at the beginning of the explore component of phase  $\ell$ . Recall that we defined the event  $\mathcal{E}_2^{(\ell)}$  to be true if properties (A-C) are satisfied at the beginning of the explore component
of phase  $\ell$  by the phased elimination algorithm. Here, we stipulate a further property D that the corresponding surviving set of items for each nice subset of users  $\mathcal{M}^{(\ell ,i)}\in \mathcal{M}^{(\ell)}$  is also a nice subset of items.
Furthermore, we defined the event  $\mathcal{E}_3^{(\ell)}$  when eq. 10 is true for all nice subsets  $\mathcal{M}^{(\ell,i)} \in \mathcal{M}'^{(\ell)}$  in the explore component of phase  $\ell$ . Algorithm 7 is a similar recursive algorithm as Algorithm 1 and the only modification is the addition of Step 11. As in Alg. 1, we instantiate Alg. 7 with phase index 1, list of nice subsets of users having a single element comprising all users i.e.  $\mathcal{M}^{(1)} = [[\mathsf{M}]]$  and corresponding list of active items  $\mathcal{N}^{(1)} = [[\mathsf{N}]]$ , clusters C, rounds T, blocking constraint B, noise  $\sigma^2$ , round index 1, exploit rounds 0, estimate  $\widetilde{\mathbf{P}}$  to be  $\mathbf{0}^{\mathsf{M} \times \mathsf{N}}$ , incoherence  $\mu$  and  $\epsilon_1 = O\left(\| \mathbf{P}\|_{\infty}, \frac{\sigma \sqrt{\mu}}{\log \mathsf{M}}\right)$ . Here, we will have a single count matrix  $\mathbf{K}$  which is binary and is initialized to be an all zero matrix. We are going to show recursively that conditioned on  $\mathcal{E}_2^{(\ell)}, \mathcal{E}_3^{(\ell)}$ , properties A-D will be satisfied at the beginning of phase  $\ell + 1$ .
Clearly,  $[\mathsf{M}]$  is a nice subset of users,  $[\mathsf{N}]$  is a nice subset of items and finally for every user  $u\in [\mathsf{M}]$ , the best  $\mathsf{T} / \mathsf{B}$  items (golden items)  $\{\pi_u(t)\}_{t = 1}^{\mathsf{T} / \mathsf{B}}$  belong to the entire set of items. Thus for  $\ell = 1$ , conditions A-D are satisfied at the beginning of the explore component and therefore the event  $\mathcal{E}_2^{(1)}$  is true. Furthermore, from Lemma 4, eq. 10 is true for the first phase with probability  $1 - o(T^{-12})$  implying that the event  $\mathcal{E}_3^{(1)}$  is true with high probability.
Inductive Argument: Suppose, at the beginning of the phase  $\ell$ , we condition on the events  $\bigcap_{j=1}^{\ell} \mathcal{E}_2^{(j)} \bigcap_{j=1}^{\ell} \mathcal{E}_2^{(j)}$  that Algorithm is  $(\epsilon_j, j)$ -good for all  $j \leq \ell$ . This means that conditions (A-D) are satisfied at the beginning of the explore component of all phases up to and including that of  $\ell$  for each reward sub-matrix (indexed by  $i \in [a_\ell]$ ) corresponding to the users in  $\mathcal{M}^{(\ell,i)}$  and items in  $\mathcal{N}^{(\ell,i)}$ . Furthermore, we also condition on the event eq. 10 is true for all nice subsets of users  $\mathcal{M}^{(\ell,i)} \in \mathcal{M}^{(\ell)}$  and their corresponding set of items  $\mathcal{N}^{(\ell,i)}$  - implying that the event  $\mathcal{E}_3^{(\ell)}$  is true. Recall that for a user  $v$ , the set  $\mathcal{T}_v^{(\ell)}$  was constructed as
$$
\mathcal {T} _ {v} ^ {(\ell)} \equiv \left\{j \in \mathcal {N} ^ {(\ell , i)} \mid \widetilde {\mathbf {P}} _ {v j} \geq \widetilde {\mathbf {P}} _ {v \widetilde {\pi} _ {v} \left(s ^ {\prime}\right) \mid \mathcal {N} ^ {(\ell , i)}} - 2 \Delta_ {\ell + 1} \right\} \text {w h e r e} s ^ {\prime} = \mathrm {T B} ^ {- 1} - \left| \mathcal {O} _ {\mathcal {M} ^ {(\ell , i)}} ^ {(\ell)} \right| \tag {33}
$$
which implies that every item in  $\mathcal{T}_v^{(\ell)}$  is close to one of the golden items in  $\mathcal{N}^{(\ell,i)}$ . At the end of Step 10 in Alg. 7, we can still show that Lemma 9 holds for each set of users  $\mathcal{M}^{(\ell,i,j)}$  as they form a connected component. Notice that in Step 11 in Lemma 9, for each set of users  $\mathcal{M}^{(\ell,i,j)}$ , we construct a graph  $\mathsf{G}_{\text{item}}^{(\ell,i,j)}$  whose nodes correspond to the items in  $\mathcal{N}^{(\ell,i)}$  and an edge exists between two items  $u,v\in \mathcal{N}^{(\ell,i)}$  if  $\left|\widetilde{\mathbf{P}}_{xu}^{(\ell)} - \widetilde{\mathbf{P}}_{xv}^{(\ell)}\right|\leq 16C\Delta_{\ell+1}$  for all users  $x\in \mathcal{M}^{(\ell,i,j)}$ . Analogous to the proof of Lemma 7, we can conclude here as well that items in the same item cluster form a clique. Therefore, any connected component in the graph  $\mathsf{G}_{\text{item}}^{(\ell,i,j)}$  must correspond to a nice subset of items since condition D is true and  $\mathcal{N}^{(\ell,i)}$  is already a nice subset of items. Hence the set of modified items  $\mathcal{N}^{(\ell,i,j)}$  constructed at the end of Step 11 in Alg. 7 is a nice subset of items. Furthermore, we can also prove the corresponding version of Lemma 9:
Lemma 19. Condition on the events  $\mathcal{E}_2^{(\ell)},\mathcal{E}_3^{(\ell)}$  being true. Consider a nice subset of users  $\mathcal{M}^{(\ell ,i,j)}$  and their corresponding set of active items  $\mathcal{N}^{(\ell ,i,j)}$  for which guarantees in eq. 11 holds. Fix any set  $\mathcal{V} = \mathcal{N}^{(\ell ,i,j)}\setminus \mathcal{I}$  for some  $\mathcal{J}$  such that for every user  $u\in \mathcal{G}$ , we have  $\widetilde{\pi}_u(s')\mid \mathcal{N}^{(\ell ,i,j)}\equiv \widetilde{\pi}_u(s)\mid \mathcal{V}$  for  $s^{\prime} = TB^{-1} - \left|\mathcal{O}_{\mathcal{M}^{(\ell ,i)}}^{(\ell)}\right|$  and some common index  $s$ . In that case, we must have
$$
\max  _ {v \in \mathcal {G}} \left(\max  _ {x, y \in \mathcal {Y}} \widetilde {\mathbf {P}} _ {v x} - \widetilde {\mathbf {P}} _ {v y}\right) \leq \max  _ {u \in \mathcal {G}} \left(\widetilde {\mathbf {P}} _ {u \widetilde {\pi} _ {u} (1) | \mathcal {Y}} - \widetilde {\mathbf {P}} _ {u \widetilde {\pi} _ {u} (s ^ {\prime}) | \mathcal {N} ^ {(\ell , i)}}\right) + 2 4 (\mathsf {C} + \mathsf {C} ^ {\prime}) \Delta_ {\ell + 1}
$$
Proof. Note that with the analysis of Lemma 9, the conclusion was true for the constructed  $\mathcal{N}^{(\ell ,i,j)}$  at the end of Step 10 in Alg. 7. However, in the modified set of items  $\mathcal{N}^{(\ell ,i,j)}$ , we are only adding
items in  $\mathcal{N}^{(\ell, i)}$  that either belong to the same cluster (in which case there is no added gap) or if we are adding items belonging to a different cluster, then the added gap on the RHS can be at most  $C' \Delta_{\ell+1}$ . The above statement follows from a similar argument as in Lemma 13 where we showed that gap in expected reward between two users in  $\mathcal{M}^{(\ell, i, j)}$  (or rather two users connected by a path in  $\mathcal{G}^{(\ell, i)}$ ) for the same item is at most  $O(C \Delta_{\ell+1})$ .
Hence, we have shown that each subset of users  $\mathcal{M}^{(\ell,i,j)}$  is a nice subset of users and furthermore, the corresponding subset of items constructed at the end of Step 11 is nice subset of items. Next, we move on to the exploit component of phase  $\ell + 1$  where we use a similar trick (See Step 3 in Alg. 8) to ensure that at the end, we are left with a nice subset of items. As before, we can show that the constructed set of items  $S$  is itself a nice subset of items. We can again prove the following modified version of Corollary 2
Corollary 3. Condition on the events  $\mathcal{E}_2^{(\ell)},\mathcal{E}_3^{(\ell)}$  being true. Consider a nice subset of users  $\mathcal{M}^{(\ell ,i)}\in$ $\mathcal{M}'(\ell)$  and their corresponding set of active items  $\mathcal{N}^{(\ell ,i)}$  for which guarantees in eq. 11 holds. Fix any subset  $\mathcal{V}\subseteq \mathcal{N}^{(\ell ,i)}$  . Consider two users  $u,v\in \mathcal{M}^{(\ell ,i)}$  having a path in the graph  $\mathcal{G}^{(\ell ,i)}$  . Conditioned on the events  $\mathcal{E}_2^{(\ell)},\mathcal{E}_3^{(\ell)}$  , we must have
$$
\max _ {x, y \in \mathcal {S}} | \mathbf {P} _ {u x} - \mathbf {P} _ {u y} | \leq 1 6 (\mathsf {C} + \mathsf {C} ^ {\prime}) \Delta_ {\ell + 1} a n d \max _ {x, y \in \mathcal {S}} | \mathbf {P} _ {v x} - \mathbf {P} _ {v y} | \leq 1 6 (\mathsf {C} + \mathsf {C} ^ {\prime}) \Delta_ {\ell + 1}
$$
where  $S\equiv \{z\in \mathcal{N}^{(\ell ,i,j)}\mid z$  is connected with  $\mathcal{R}_u^{(\ell)}\cup \mathcal{R}_v^{(\ell)}\}$
Proof. The proof again follows from the fact that adding items belonging to a different cluster but connected via a path to the original items in  $\mathcal{R}_u^{(\ell)}\cup \mathcal{R}_V^{(\ell)}$  can only add a term of at most  $16C^{\prime}\Delta_{\ell +1}$  in the RHS.
Therefore, at the beginning of the explore component of phase  $\ell + 1$  for a particular nice subset of users  $\mathcal{M}^{(\ell + 1, z)}$ , the corresponding set of items  $\mathcal{N}^{(\ell + 1, z)}$  must be a nice subset of items as well. Most importantly, what this implies is that for the low rank matrix completion step in the explore component of phase  $\ell + 1$ , we can re-use observations from previous phases and provide theoretical guarantees as well. This is because, in eq. 10 in Lemma 4, we can take a union bound over all possible nice subsets of users and all possible nice subsets of items. Since the number of clusters  $C$ ,  $C' = O(1)$ , the total possibilities is  $O(1)$  as well (although exponential in the number of clusters). Hence, the complex dependencies of the previously made observations used to compute prior estimates resulted by the conditioning on surviving items and users is no longer a problem - we have simply made a for all argument. The rest of the analysis follows as in the GBB setting and we can arrive at a similar result as in Theorem 1 but with  $B = 1$  when the set of items can be clustered into  $C' = O(1)$  disjoint clusters.