SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. Recall from the construction of  $\mathcal{G}^{(\ell, i)}$  that conditioned on  $\mathcal{E}_3^{(\ell)}$  users  $u, v \in \mathcal{M}^{(\ell, i)}$  have an edge if  $\left|\widetilde{\mathbf{P}}_{ux}^{(\ell)} - \widetilde{\mathbf{P}}_{vx}^{(\ell)}\right| \leq 2\Delta_{\ell+1}$  (and therefore  $|\mathbf{P}_{ux} - \mathbf{P}_{vx}| \leq 4\Delta_{\ell+1}$ ) for all  $x \in \mathcal{N}^{(\ell, i)}$ . Again, for simplicity of notation, let us denote  $\mathcal{R}_u$  to be the set  $\mathcal{R}_u^{(\ell)} \mid \mathcal{Y}$ . Suppose  $\mathbf{P}_{u\pi_u(1)|\mathcal{Y}} = a$  and  $\mathbf{P}_{v\pi_v(1)|\mathcal{Y}} = b$ . Consider any pair of items  $x \in \mathcal{R}_u, y \in \mathcal{R}_v$  respectively. Therefore, for  $x \in \mathcal{R}_u$  this must mean that  $\mathbf{P}_{ux} \geq a - 4\Delta_{\ell+1}$ . For  $y \in \mathcal{R}_v$ , we must similarly have  $\mathbf{P}_{vy} \geq b - 4\Delta_{\ell+1}$ . Since  $u, v$  are connected by an path of length  $\mathsf{L}$  (say  $a_1, a_2, \ldots, a_{(\mathsf{L}-1)}$ ), we must have  $\mathbf{P}_{a_1x} \geq a - 8\Delta_{\ell+1}$ ,  $\mathbf{P}_{a_2x} \geq a - 12\Delta_{\ell+1}$  and finally  $\mathbf{P}_{vx} \geq a - 4(\mathsf{L} + 1)\Delta_{\ell+1}$ . By a similar analysis  $\mathbf{P}_{uy} \geq b - 4(\mathsf{L} + 1)\Delta_{\ell+1}$ . Now, we have  $\mathbf{P}_{v\pi_v(1)|\mathcal{Y}} \geq \mathbf{P}_{vx}$  implying that  $b \geq a - 4(\mathsf{L} + 1)\Delta_{\ell+1}$ ; hence
$$
\mathbf {P} _ {u x} - \mathbf {P} _ {u y} \leq \mathbf {P} _ {u \pi_ {u} (1) | \mathcal {R} _ {u}} - \mathbf {P} _ {u y} \leq a - (b - 4 (\mathsf {L} + 1) \Delta_ {\ell + 1}) \leq 8 (\mathsf {L} + 1) \Delta_ {\ell + 1}.
$$
Again, a similar analysis for  $v$  shows that  $\mathbf{P}_{vy} - \mathbf{P}_{vx} \leq 8(\mathsf{L} + 1)\Delta_{\ell + 1}$ . This completes the proof of the lemma.
Corollary 2. Condition on the events  $\mathcal{E}_2^{(\ell)},\mathcal{E}_3^{(\ell)}$  being true. Consider a nice subset of users  $\mathcal{M}^{(\ell ,i)}\in$ $\mathcal{M}'(\ell)$  and their corresponding set of active items  $\mathcal{N}^{(\ell ,i)}$  for which guarantees in eq. 11 holds. Fix any subset  $\mathcal{V}\subseteq \mathcal{N}^{(\ell ,i)}$  . Consider two users  $u,v\in \mathcal{M}^{(\ell ,i)}$  having a path in the graph  $\mathcal{G}^{(\ell ,i)}$  . Conditioned on the events  $\mathcal{E}_2^{(\ell)},\mathcal{E}_3^{(\ell)}$  , we must have
$$
\max_{x\in \mathcal{R}_{u}^{(\ell)}|\mathcal{Y},y\in \mathcal{R}_{v}^{(\ell)}|\mathcal{Y}}| \mathbf{P}_{ux} - \mathbf{P}_{uy}|\leq 16\mathsf{C}\Delta_{\ell +1}a n d\max_{x\in \mathcal{R}_{u}^{(\ell)}|\mathcal{Y},y\in \mathcal{R}_{v}^{(\ell)}|\mathcal{Y}}| \mathbf{P}_{vx} - \mathbf{P}_{vy}|\leq 16\mathsf{C}\Delta_{\ell +1}
$$
Proof. The proof follows from the fact that any two users  $u, v \in \mathcal{M}^{(\ell,i)}$  connected via a path must have a shortest path of length at most  $2\mathsf{C} - 1$  conditioned on the events  $\mathcal{E}_2^\ell, \mathcal{E}_3^{(\ell)}$ . This is because, from Lemma 13, we know that users in the same cluster form a clique and since there are at most  $\mathsf{C}$  clusters, the shortest path must be of length at most  $2\mathsf{C} - 1$ .
Hence, for a particular set of users  $\mathcal{M}^{(\ell ,i)}\in \mathcal{M}'^{(\ell)}$ , consider the  $j^{\mathrm{th}}$  connected component of  $\mathcal{G}^{(\ell ,i)}$  comprising of users  $\mathcal{M}^{(\ell ,i,j)}$ . If we consider the union of items  $\mathcal{S}\equiv \cup_{u\in \mathcal{M}^{(\ell ,i,j)}}\mathcal{R}_u^{(\ell)}\mid \mathcal{V}$  for any subset  $\mathcal{V}\mid \mathcal{N}^{(\ell ,i)}$ , then from Corollary 2, we must have that for any user  $u\in \mathcal{M}^{(\ell ,i,j)}$ ,
$$
\max  _ {x, y \in \mathcal {S}} \left| \mathbf {P} _ {u x} - \mathbf {P} _ {u y} \right| \leq 1 6 \mathrm {C} \Delta_ {\ell + 1}. \tag {15}
$$
This follows from the fact that every element  $s \in S$  must exist in  $\mathcal{R}_v^{(\ell)} \mid \mathcal{V}$  for some  $v \in \mathcal{M}^{(\ell, i, j)}$ ; moreover,  $v$  is connected to  $u$  and therefore the shortest path joining them must be of length at most  $2C - 1$ . Finally we use Corollary 2 to conclude equation 15.
Lemma 14. Condition on the events  $\mathcal{E}_2^{(\ell)},\mathcal{E}_3^{(\ell)}$  being true. Consider a nice subset of users  $\mathcal{M}^{(\ell ,i)}\in$ $\mathcal{M}'(\ell)$  and their corresponding set of active items  $\mathcal{N}^{(\ell ,i)}$  for which guarantees in eq. 11 holds. Consider the  $j^{\mathrm{th}}$  connected component of the graph  $\mathcal{G}^{(\ell ,i)}$  comprising of users  $\mathcal{M}^{(\ell ,i,j)}$ . Let  $\mathcal{N}^{(\ell ,i,j)}\equiv$ $\cup_{u\in \mathcal{M}^{(\ell ,i,j)}}\mathcal{T}_u^{(\ell)}$  denote the union of good items for users in  $\mathcal{M}^{(\ell ,i,j)}$ . In that case,
1. For every user  $u \in \mathcal{M}^{(\ell, i, j)}$ , the items  $\pi_u(s) \mid [\mathsf{N}] \backslash \mathcal{O}_{\mathcal{M}^{(\ell, i)}}^{(\ell)}$  for  $s \in [\mathsf{T}\mathsf{B}^{-1} - \left| \mathcal{O}_{\mathcal{M}^{(\ell, i)}}^{(\ell)} \right|]$  must belong to the set  $\mathcal{N}^{(\ell, i, j)}$  i.e. the golden items that have not been chosen for recommendation to users in  $\mathcal{M}^{(\ell, i, j)}$  in exploit components of previous phases must belong to the surviving set of items  $\mathcal{N}^{(\ell, i, j)}$ .
2. For any subset  $\mathcal{V} \subseteq \mathcal{N}^{(\ell, i, j)}$  and any  $s \leq |\mathcal{V}|$ , we must have the following for any  $\mathsf{A} > 0$ :
$$
\begin{array}{l} I f \widetilde {\mathbf {P}} _ {u \widetilde {\pi} _ {u} (1) | \mathcal {Y}} - \widetilde {\mathbf {P}} _ {u \widetilde {\pi} _ {u} (s) | \mathcal {Y}} \geq \mathrm {A} f o r s o m e u \in \mathcal {M} ^ {(\ell , i, j)} \\ t h e \quad \mathbf {P} _ {v \pi_ {v} (1) | \mathcal {Y}} - \mathbf {P} _ {v \pi_ {v} (s) | \mathcal {Y}} \\ \geq \widetilde {\mathbf {P}} _ {v \widetilde {\pi} _ {v} (1) | \mathcal {Y}} - \widetilde {\mathbf {P}} _ {v \widetilde {\pi} _ {v} (s) | \mathcal {Y}} - 4 \Delta_ {\ell + 1} \geq \mathsf {A} - (2 \mathsf {C} - 1) 4 \Delta_ {\ell + 1} f o r a l l v \in \mathcal {M} ^ {(\ell , i, j)}. \\ \end{array}
$$
Proof. The proof of the first part follows directly from Lemma 11 where we showed that for a particular user  $u \in \mathcal{M}^{(\ell, i)}$ , the items  $\mathbf{P}_{u\pi_u(s) | [\mathsf{N}] \setminus \mathcal{O}_{\mathcal{M}^{(\ell, i)}}^{(\ell)}}$  for  $s \in [\mathsf{T}\mathsf{B}^{-1} - \left|\mathcal{O}_{\mathcal{M}^{(\ell, i)}}^{(\ell)}\right|]$  must belong to the set  $\mathcal{T}_u^{(\ell)}$  (and the fact that  $\mathcal{N}^{(\ell, i, j)} \supseteq \mathcal{T}_u^{(\ell)}$ ).
We move on to the proof of the second part of the lemma. Recall that any two users  $u, v \in \mathcal{M}^{(\ell,i)}$  have an edge in the graph  $\mathcal{G}^{(\ell,i)}$  if  $\left|\widetilde{\mathbf{P}}_{ux}^{(\ell)} - \widetilde{\mathbf{P}}_{vx}^{(\ell)}\right| \leq 2\Delta_{\ell+1}$  for all  $x \in \mathcal{N}^{(\ell,i)}$ . In that case, we have
$$
\begin{array}{l} \widetilde {\mathbf {P}} _ {v \widetilde {\pi} _ {v} (1) | \mathcal {Y}} - \widetilde {\mathbf {P}} _ {v \widetilde {\pi} _ {v} (s) | \mathcal {Y}} \geq \widetilde {\mathbf {P}} _ {v \widetilde {\pi} _ {u} (1) | \mathcal {Y}} - \widetilde {\mathbf {P}} _ {v \widetilde {\pi} _ {u} (s) | \mathcal {Y}} \\ \geq \widetilde {\mathbf {P}} _ {u \widetilde {\pi} _ {u} (1) | \mathcal {Y}} - \widetilde {\mathbf {P}} _ {u \widetilde {\pi} _ {u} (s) | \mathcal {Y}} - 4 \Delta_ {\ell + 1} \geq \mathsf {A} - 4 \Delta_ {\ell + 1}. \\ \end{array}
$$
We are going to prove the Lemma statement by induction on length of the shortest path joining the users  $u, v$ . The base case (when the path length is 1 i.e.  $u, v$  are joined by an edge) is proved above. Suppose the statement is true when length of the shortest path is  $\mathsf{L} - 1$ . In that case, we have the following set of inequalities (suppose  $w$  is the neighbor of  $v$  and the length of the shortest path joining  $u, w$  is  $\mathsf{L} - 1$ )
$$
\begin{array}{l} \widetilde {\mathbf {P}} _ {v \widetilde {\pi} _ {v} (1) | \mathcal {Y}} - \widetilde {\mathbf {P}} _ {v \widetilde {\pi} _ {v} (s) | \mathcal {Y}} \geq \widetilde {\mathbf {P}} _ {v \widetilde {\pi} _ {w} (1) | \mathcal {Y}} - \widetilde {\mathbf {P}} _ {v \widetilde {\pi} _ {w} (s) | \mathcal {Y}} \\ \geq \widetilde {\mathbf {P}} _ {w \widetilde {\pi} _ {w} (1) | \mathcal {Y}} - \widetilde {\mathbf {P}} _ {w \widetilde {\pi} _ {w} (s) | \mathcal {Y}} - 4 \mathsf {L} \Delta_ {\ell + 1} \geq \mathsf {A} - 4 \mathsf {L} \Delta_ {\ell + 1}. \\ \end{array}
$$
The lemma statement follows from the fact that the length of the shortest path between users  $u, v$  in the same connected component is at most  $2C - 1$ . This completes the proof of the lemma.
Partition of  $\mathcal{M}^{(\ell, i)}$  in phase  $\ell + 1$  into nice subsets of users and their corresponding active subsets of items: Consider a nice set of users  $\mathcal{M}^{(\ell, i)}$  at the end of the explore component of phase  $\ell$  i.e. the start of the subsequent phase  $\ell + 1$  for the users in  $\mathcal{M}^{(\ell, i)}$ . At this point, conditioned on events  $\mathcal{E}_2^{(\ell)}, \mathcal{E}_3^{(\ell)}$ , our goal is to further partition the users in  $\mathcal{M}^{(\ell, i)}$  into more nuanced nice subsets. Suppose the number of rounds is at least  $T^{1/3}/B = \widetilde{O}(T^{1/3})$  implying that the active set of items  $\mathcal{N}^{(\ell, i)}$  is of size at least  $\widetilde{\Omega}(T^{1/3})$  (induction assumption property B). Suppose, we index the connected components of the graph  $\mathcal{G}^{(\ell, i)}$  formed by the users in  $\mathcal{M}^{(\ell, i)} \in \mathcal{M}'^{(\ell)}$ . Now, for each index  $j$ , the set of users corresponding to the  $j^{\text{th}}$  connected component of the graph  $\{\mathcal{G}^{(\ell, i)}\}$  forms the  $j^{\text{th}}$  nice subset of users (Lemma 7) stemming from users in  $\mathcal{M}^{(\ell, i)} \in \mathcal{M}'^{(\ell)}$  - let us denote this set of users by  $\mathcal{M}^{(\ell + 1, z)}$  for some index  $s > 0$ . For this set of users  $\mathcal{M}^{(\ell + 1, z)}$ , at the start of the exploit component of phase  $\ell + 1$  (round  $t$ ), we define the active set of items  $\mathcal{N}^{(\ell + 1, t, z)}$  to be  $\bigcup_{u \in \mathcal{M}^{(\ell + 1, z)}} T_u^{(\ell)}$ . Hence  $\{(\mathcal{M}^{(\ell + 1, z)}, \mathcal{N}^{(\ell + 1, t, z)})_z$  forms the family of nice sets of users (that progress to the  $(\ell + 1)^{\text{th}}$  phase) and their corresponding active set of items at the beginning (exploit component) of phase  $\ell + 1$  for users stemming from  $\mathcal{M}'^{(\ell)}$ . Next, we discuss our recommendation strategy for  $\mathcal{M}^{(\ell + 1, z)}$  (a nice subset of users) in the exploit component of phase  $\ell + 1$ .
Strategy in exploit component of phase  $\ell + 1$ : Note that in the exploit component of phase  $\ell + 1$  for users in  $\mathcal{M}^{(\ell, i, j)} \equiv \mathcal{M}^{(\ell + 1, z)}$  (new notation indicating that  $\mathcal{M}^{(\ell, i, j)} \equiv \mathcal{M}^{(\ell + 1, z)}$  is a nice subset of users at phase  $\ell + 1$ ) for some indices  $i, j$  and  $z$ , we follow a recursive approach to identify and recommend items in  $\{\pi_u(t)\}_{t=1}^{\mathrm{T}\mathsf{B}^{-1}}$  for all users  $u \in \mathcal{M}^{(\ell + 1, z)}$ . At the beginning of the exploit component of phase  $\ell + 1$  (say round  $t$ ), we will also initialize  $\mathcal{O}_{\mathcal{M}^{(\ell + 1, z)}}^{(\ell + 1, t)}$  to be the set of items  $\mathcal{O}_{\mathcal{M}^{(\ell, i)}}^{(\ell)}$  (i.e. the golden items that have been chosen for recommendation in the exploit components of previous phases  $(1 - \ell)$  to users in  $\mathcal{M}^{(\ell, i)}$  and recommended sufficiently enough number of times to be blocked). At start of the exploit component of phase  $\ell + 1$  (round  $t$ ), recall that the active set of items is given by  $\mathcal{N}^{(\ell + 1, t, z)} \equiv \bigcup_{u \in \mathcal{M}^{(\ell + 1, z)}} \mathcal{T}_u^{(\ell)}$ . We reiterate here that  $t$  corresponds to the index of the starting round in the exploit component of phase  $\ell + 1$  for users in the nice subset  $\mathcal{M}^{(\ell + 1, z)}$ .
Let us denote  $s = \mathsf{TB}^{-1} - \left|\mathcal{O}_{\mathcal{M}(\ell +1,z)}^{(\ell +1,t)}\right|$  and  $\mathcal{V} = \mathcal{N}^{(\ell +1,t,z)}$ . First of all, note that from Lemma 14, for all users  $u\in \mathcal{M}^{(\ell +1,z)}$  the items  $\pi_u(r)\mid [\mathsf{N}]\setminus \mathcal{O}_{\mathcal{M}^{(\ell +1,z)}}^{(\ell +1,t)}$  for  $r\in [s]$  must belong to the set  $\mathcal{V}$  i.e. the best  $s$  items among those that are not in the set  $\mathcal{O}_{\mathcal{M}^{(\ell +1,z)}}^{(\ell +1,t)}$  must survive in  $\mathcal{V}$  (Lemma 14). Now, we look at two possibilities:
1. (Possibility A): For all users  $u \in \mathcal{M}^{(\ell + 1, z)}$ , we have that  $\widetilde{\mathbf{P}}_{u\widetilde{\pi}_u(1)|\mathcal{Y}} - \widetilde{\mathbf{P}}_{u\widetilde{\pi}_u(s)|\mathcal{Y}} \leq 64C\Delta_{\ell + 1}$ . In that case, we stop the exploit component of phase  $\ell + 1$  and move on to the explore component of phase  $\ell + 1$  for users in  $\mathcal{M}^{(\ell + 1, z)}$  and active items  $\mathcal{V}$ . Conditioned on events  $\mathcal{E}_2^{(\ell)}, \mathcal{E}_3^{(\ell)}$ , in Lemma 15, we show that the above condition implies for every user  $u \in \mathcal{M}^{(\ell + 1, z)}$ , we must have
$$
\max  _ {x, y \in \mathcal {Y}} \leq 8 8 \mathsf {C} \Delta_ {\ell + 1}.
$$
Furthermore, for  $\mathcal{Z} = [\mathsf{N}] \setminus \mathcal{O}_{\mathcal{M}(\ell + 1, z)}^{(\ell + 1)}$ , it must happen that  $\mathcal{Y} \supseteq \{\pi_u(s) | \mathcal{Z}\}_{s=1}^{\mathsf{T} / \mathsf{B}} - \left|\mathcal{O}_{\mathcal{M}(\ell + 1, z)}^{(\ell + 1)}\right|$  for every user  $u \in \mathcal{M}^{(\ell + 1, z)}$ . In other words, for each user  $u \in \mathcal{M}^{(\ell + 1, z)}$ , the set  $\mathcal{V}$  must contain all the top  $\mathsf{T}\mathsf{B}^{-1}$  golden items ( $\{\pi_u(r)\}_{r=1}^{\mathsf{T}\mathsf{B}^{-1}}$ ) that were not recommended in the exploit components so far.
2. (Possibility B): For some user  $u \in \mathcal{M}^{(\ell + 1, z)}$ , we have that  $\widetilde{\mathbf{P}}_{u\widetilde{\pi}_u(1)}|y - \widetilde{\mathbf{P}}_{u\widetilde{\pi}_u(s)}|y \geq 64C\Delta_{\ell + 1}$ . In that case, from Lemma 14, we know that for every user  $v \in \mathcal{M}^{(\ell + 1, z)}$ , we must have  $\mathbf{P}_{v\pi_v(1)}|y - \mathbf{P}_{v\pi_v(s)}|y \geq 56C\Delta_{\ell + 1}$ . In that case, if we consider the set of items  $S \equiv \cup_{u \in \mathcal{M}^{(\ell + 1, z)}} \mathcal{R}_u^{(\ell)} \mid y$ , then from Lemma 13 (or see eq. 15), we must have that for every user  $u \in \mathcal{M}^{(\ell + 1, z)}$ ,
$$
\max  _ {x, y \in \mathcal {S}} \left| \mathbf {P} _ {u x} - \mathbf {P} _ {u y} \right| \leq 1 6 \mathrm {C} \Delta_ {\ell + 1} \tag {16}
$$
For every user  $u \in \mathcal{M}^{(\ell + 1, z)}$ , recall that in Lemma 11, we showed that  $\pi_u(1) \mid \mathcal{Y} \in \mathcal{R}_u^{(\ell)} \mid \mathcal{Y}$  and in Lemma 14, we showed that  $\pi_u(1) \mid \mathcal{Y} = \pi_u(1) \mid [\mathsf{N} \setminus \left|\mathcal{O}_{\mathcal{M}^{(\ell + 1, z)}}^{(\ell + 1, t)}\right|]$ . Hence,  $\pi_u(1) \mid \mathcal{Y} = \pi_u(1) \mid [\mathsf{N} \setminus \left|\mathcal{O}_{\mathcal{M}^{(\ell + 1, z)}}^{(\ell + 1, t)}\right|]$  belongs to the set  $\mathcal{S}$  for every user  $u \in \mathcal{M}^{(\ell + 1, z)}$ . Hence we must have that the items  $\mathcal{S}$  is a subset of  $\{\pi_u(r) \mid [\mathsf{N}] \setminus \mathcal{O}_{\mathcal{M}^{(\ell + 1, z)}}^{(\ell + 1, t)}\}_{r=1}^s$ . Suppose we index the items in  $\mathcal{S}$ . For each of the subsequent  $\mathsf{B} \mid \mathcal{S}$  rounds (indexed by  $b \in [\mathsf{B}\mathcal{S}]$ ), for every user  $u \in \mathcal{M}^{(\ell + 1, z)}$ , we go to the  $\lceil (b / \mathsf{B}) \rceil^{\text{th}}$  item in  $\mathcal{S}$  and recommend it to user  $u$  if unblocked. On the other hand, if the  $\lceil (b / \mathsf{B}) \rceil^{\text{th}}$  item in  $\mathcal{S}$  is blocked (or becomes blocked) for the user  $u \in \mathcal{M}^{(\ell + 1, z)}$ , then we simply recommend any unblocked item in  $\mathcal{N}^{(\ell + 1, t, z)}$ . This is always possible because we will prove via induction (see Lemma 15 and in particular eq. 23) that at every round in the exploit component of phase  $\ell + 1$ , the number of unblocked items for any user  $u \in \mathcal{M}^{(\ell + 1, z)}$  in the set  $\mathcal{N}^{(\ell + 1, t, z)}$  (where  $t$  is the previous decision round for  $\mathcal{M}^{(\ell + 1, z)}$  on whether possibility A or B is true) is always larger than the number
of remaining rounds. We make the following updates:
$$
\mathcal {O} _ {\mathcal {M} ^ {(\ell + 1, z)}} ^ {(\ell + 1, t + | \mathcal {S} |)} \leftarrow \mathcal {O} _ {\mathcal {M} ^ {(\ell + 1, z)}} ^ {(\ell + 1, t)} \cup \mathcal {S} \tag {17}
$$
$$
\mathcal {N} ^ {(\ell + 1, t + | \mathcal {S} |, z)} \leftarrow \mathcal {N} ^ {(\ell + 1, t, z)} \backslash \mathcal {O} _ {\mathcal {M} ^ {(\ell + 1, z)}} ^ {(\ell + 1, t + | \mathcal {S} |)} \tag {18}
$$
$$
t \leftarrow t + | \mathcal {S} | \text {a n d} \mathcal {Y} \leftarrow \mathcal {N} ^ {(\ell + 1, t, s)} \tag {19}
$$
i.e we update the set  $\mathcal{O}_{\mathcal{M}(\ell +1,z)}^{(\ell +1,t)}$  by taking union with the set of  $|\mathcal{S}|$  identified items in  $\{\pi_u(t)\}_{t = 1}^T$  for all users  $u\in \mathcal{M}^{(\ell +1,z)}$ . After these  $|\mathcal{S}|$  rounds, for the set of users  $\mathcal{M}^{(\ell +1,z)}$ , the set of active items  $\mathcal{N}^{(\ell +1,t,z)}$  is pruned by removing the items in  $\mathcal{S}$  and the time index is increased from  $t$  to  $t + |\mathcal{S}|$ .
At this point, we repeat the same process again for users in  $\mathcal{M}^{(\ell +1,z)}$  with the pruned set of active items  $\mathcal{N}^{(\ell +1,t,s)}$  i.e. we check for possibility  $A$  or possibility  $B$ . If we encounter possibility  $B$ , then we again find the set of items  $\mathcal{S}\equiv \cup_{u\in \mathcal{M}^{(\ell +1,z)}}\mathcal{R}_u^{(\ell)}\mid \mathcal{V}$  and recommended it to all users in  $\mathcal{M}$  in  $|\mathcal{S}|$  B steps as outlined above. We do this step recursively until we encounter Step A for the users in  $\mathcal{M}^{(\ell +1,z)}$  and at that point we exit the exploit component of phase  $\ell +1$  and enter the explore component of phase  $\ell +1$ .
As before, at the beginning of the explore component of phase  $\ell + 1$  for the nice subset of users  $\mathcal{M}^{(\ell + 1, z)}$ , let us denote the set of active items by  $\mathcal{N}^{(\ell + 1, z)}$  and the set of items considered for recommendation in the exploit phases including the  $(\ell + 1)^{\text{th}}$  one by  $\mathcal{O}_{\mathcal{M}^{(\ell + 1, z)}}^{(\ell + 1)}$  (i.e. we remove the  $t$  in the superscript for simplicity). Therefore, at the end of the explore component of phase  $\ell + 1$  for the nice subset of users  $\mathcal{M}^{(\ell + 1, z)}$ , the set of active items  $\mathcal{N}^{(\ell + 1, z)}$  satisfy the following:
Lemma 15. Consider a nice subset of users  $\mathcal{M}^{(\ell +1,z)}$  and their corresponding set of active items  $\mathcal{N}^{(\ell +1,z)}$  at the end of the exploit stage of phase  $\ell +1$  i.e. for all users  $u\in \mathcal{M}^{(\ell +1,z)}$ , we have  $\widetilde{\mathbf{P}}_{u\widetilde{\pi}_u(1)|\mathcal{N}^{(\ell +1,z)}} - \widetilde{\mathbf{P}}_{u\widetilde{\pi}_u(s)|\mathcal{N}^{(\ell +1,z)}}\leq 64C\Delta_{\ell +1}$  for  $s = \mathsf{T}\mathsf{B}^{-1} - \left|\mathcal{O}_{\mathcal{M}^{(\ell +1,z)}}^{(\ell +1)}\right|$ . Suppose  $\mathcal{M}^{(\ell +1,z)}$  is comprised of the users in a connected component of the graph  $\mathcal{G}^{(\ell ,i)}$  which in turn is formed by the users in  $\mathcal{M}^{(\ell ,i)}$  for which guarantees in eq. 11 holds true i.e. we condition on the events  $\mathcal{E}_2^{(\ell)},\mathcal{E}_3^{(\ell)}$  being true. In that case, we must have that
1. for all users  $u \in \mathcal{M}^{(\ell + 1, z)}$ ,
$$
\max  _ {x, y \in \mathcal {N} ^ {(\ell + 1, z)}} \left| \mathbf {P} _ {u x} - \mathbf {P} _ {u y} \right| \leq 8 8 \mathrm {C} \Delta_ {\ell + 1}
$$
i.e. the best and worst items in the set  $\mathcal{N}^{(\ell +1,z)}$  for any user  $u\in \mathcal{M}^{(\ell +1,z)}$  has close rewards.
2. Denote  $\mathcal{Z} = [\mathsf{N}] \setminus \mathcal{O}_{\mathcal{M}^{(\ell + 1, z)}}^{(\ell + 1)}$  to be a set of items not chosen for recommendation to users in  $\mathcal{M}^{(\ell + 1, z)}$  in the exploit component of phases until (and including) phase  $\ell + 1$ . Then it must happen that
$$
\mathcal {O} _ {\mathcal {M} ^ {(\ell + 1, z)}} ^ {(\ell + 1)} \subseteq \left\{\pi_ {u} (t) \right\} _ {t = 1} ^ {\mathrm {T B} ^ {- 1}} \tag {20}
$$
$$
\mathcal {N} ^ {(\ell + 1, z)} \supseteq \bigcup_ {u \in \mathcal {M} ^ {(\ell + 1, z)}} \left\{\pi_ {u} \left(t ^ {\prime}\right) \mid \mathcal {Z} \right\} _ {t ^ {\prime} = 1} ^ {\mathrm {T B} ^ {- 1} - \left| \mathcal {O} _ {\mathcal {M} (\ell + 1, z)} ^ {(\ell + 1)} \right|}. \tag {21}
$$
Proof. Suppose at the end of the explore component of phase  $\ell$ ,  $\mathcal{M}^{(\ell +1,z)}\subseteq \mathcal{M}^{(\ell ,i)}$ . We will prove a more general statement. Consider the rounds  $t_1,t_2,\ldots$  at which we check for possibility  $A$  or possibility  $B$  (this includes the starting and ending rounds of the exploit component of phase  $\ell +1$ ). At any such round  $t_r$ , for all users  $u\in \mathcal{M}^{(\ell +1,z)}$ , we must have that  $\widetilde{\pi}_u(TB^{-1} - \left|\mathcal{O}_{\mathcal{M}^{(\ell ,i)}}^{(\ell ,t_r)}\right|)\mid \mathcal{N}^{(\ell ,i)}\in \mathcal{N}^{(\ell +1,t_r,z)}$  and
$$
\max  _ {x, y \in \mathcal {N} ^ {(\ell + 1, t _ {r}, z)}} \left| \mathbf {P} _ {u x} - \mathbf {P} _ {u y} \right| \leq \max  _ {v \in \mathcal {M} ^ {(\ell + 1, z)}} \left(\widetilde {\mathbf {P}} _ {u \widetilde {\pi} _ {u} (1) | \mathcal {N} ^ {(\ell + 1, t _ {r}, z)}} - \widetilde {\mathbf {P}} _ {u \widetilde {\pi} _ {u} (s) | \mathcal {N} ^ {(\ell + 1, t _ {r}, z)}}\right) + 2 4 \mathrm {C} \Delta_ {\ell + 1} \tag {22}
$$
$$
\mathcal {N} ^ {(\ell + 1, t _ {r}, z)} \supseteq \bigcup_ {u \in \mathcal {M} ^ {(\ell + 1, z)}} \left\{\pi_ {u} \left(t ^ {\prime}\right) \mid \mathcal {Z} \right\} _ {t ^ {\prime} = 1} ^ {s} \text {w h e r e} \mathcal {Z} \equiv [ N ] \backslash \mathcal {O} _ {\mathcal {M} ^ {(\ell + 1, z)}} ^ {(\ell + 1, t _ {r})} \tag {23}
$$
$$
\mathcal {O} _ {\mathcal {M} ^ {(\ell + 1, z)}} ^ {(\ell + 1, t _ {r})} \subseteq \left\{\pi_ {u} (t) \right\} _ {t = 1} ^ {\mathrm {T B} ^ {- 1}} \tag {24}
$$
for  $s = \mathsf{T} - \left|\mathcal{O}_{\mathcal{M}^{(\ell +1,z)}}^{(\ell +1,t_r)}\right|$  We will prove the statement above via induction on the recursions performed in the exploit component of the phase  $\ell +1$  . For the base case, we consider the round  $t_1$  which corresponds to the beginning of the exploit component of phase  $\ell +1$  . At this round, recall that  $\mathcal{N}^{(\ell +1,t_1,z)} = \cup_{u\in \mathcal{M}^{(\ell +1,z)}}\mathcal{T}_u^{(\ell)}$  . Of course, with  $s = \mathsf{T} - \left|\mathcal{O}_{\mathcal{M}^{(\ell ,i)}}^{(\ell)}\right|$  , for every user  $u\in \mathcal{M}^{(\ell +1,z)}$  we must have  $\{\widetilde{\pi}_u(r)\mid \mathcal{N}^{(\ell ,i)}\}_{r = 1}^s\in \mathcal{T}_u^{(\ell)}$  (Lemma 11) and since  $\mathcal{N}^{(\ell +1,z)}$  is a union of the sets  $\mathcal{T}_u^{(\ell)},\widetilde{\pi}_u(r)\mid \mathcal{N}^{(\ell ,i)} = \widetilde{\pi}_u(r)\mid \mathcal{N}^{(\ell +1,z)}$  for all  $r\in [s]$  . Hence by invoking Lemma 9 (in the statement of Lemma 9, we have  $\mathcal{J} = \phi$  i.e.  $\mathcal{V} = \mathcal{N}^{(\ell +1,z)}$  and  $\widetilde{\pi}_u(s)\mid \mathcal{N}^{(\ell ,i)} = \widetilde{\pi}_u(s)\mid \mathcal{N}^{(\ell +1,z)}$  for  $s = \mathsf{T} - \left|\mathcal{O}_{\mathcal{M}^{(\ell ,i)}}^{(\ell ,t_r)}\right|$  ), we obtain the statement of the Lemma for round  $t_1$