SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$\mathcal{N}_m(t)\}$ . Moreover, for client  $j \in \mathcal{N}_m(t)$  with  $n_{j,i}(t) < \mathcal{N}_{j,i}(t) - K$ , i.e.  $\mathcal{N}_{j,i}(t + 1)$  will not be affected by  $n_{j,i}(t + 1) \leq n_{j,i}(t) + 1$ . Thus, the value of  $N_{m,i}(t + 1) = \max \{n_{m,i}(t + 1), N_{j,i}(t), j \in \mathcal{N}_m(t)\}$  is independent of such clients. We observe that for client  $j \in \mathcal{N}_m(t)$  with  $n_{j,i}(t) > \mathcal{N}_{j,i}(t) - K$ , the value  $N_{j,i}(t)$  will be the same if the client does not sample arm  $i$ , which leads to a decrease of 1 in the difference  $-n_{m,i}(t) + N_{m,i}(t)$ . Otherwise, if such a client samples arm  $i$  which brings an increment of 1 to  $N_{m,i}(t)$ , the difference between  $n_{m,i}(t)$  and  $N_{m,i}(t)$  will remain the same. However, the latter has just been discussed and must be the cases as in Case 2 and Case 3, the total length of which has already been upper bounded by  $\frac{\pi^2}{3}$  as shown in (32).
Therefore, the gap is at most  $K(K + 2M) - K + \frac{\pi^2}{3}$ , i.e.
$$
\sum_ {t = 1} ^ {T} P \left(\text {C a s e 1}, a _ {t} ^ {m} = i, n _ {m, i} (t - 1) \geq l | A\right) \leq K (K + 2 M) - K + \frac {\pi^ {2}}{3}. \tag {33}
$$
Subsequently, we derive that
$$
\begin{array}{l} E [ n _ {m, i} (T) | A _ {\epsilon , \delta} ] \leq l + \frac {\pi^ {2}}{3} + K (K + 2 M) - K + \frac {\pi^ {2}}{3} + 0 \\ = l + \frac {2 \pi^ {2}}{3} + K ^ {2} + (2 M - 1) K \\ = \max \left\{\left[ \frac {4 C _ {1} \log T}{\Delta_ {i} ^ {2}} \right], 2 (K ^ {2} + M K + M) \right\} + \frac {2 \pi^ {2}}{3} + K ^ {2} + (2 M - 1) K \\ \end{array}
$$
where the inequality results from (30), (31), (32), and (33).
This completes the proof steps.
Next, we establish the concentration inequalities of the network-wide estimators when the rewards follow sub-exponential distributions, i.e. in setting  $S_{1}, S_{2}$ , and  $S_{3}$ .
Proposition 11. Assume the parameter  $\delta$  satisfies that  $0 < \delta < c = f(\epsilon, M, T)$ . In setting  $S_{1}, S_{2}$ , and  $S_{3}$ , for any  $m, i, \lambda$  and  $t > L$  where  $L$  is the length of the burn-in period, the global estimator  $\tilde{\mu}_{i}^{m}(t)$  is sub-exponentially distributed. Moreover, the conditional moment generating function satisfies that with  $P(A_{\epsilon, \delta}) = 1 - 7\epsilon$ , for  $|\lambda| < \frac{1}{\alpha}$
$$
w h e r e \sigma^ {2} = \max _ {j, i} (\tilde {\sigma} _ {i} ^ {j}) ^ {2} a n d C = \max \{\frac {4 (M + 2) (1 - \frac {1 - c _ {0}}{2 (M + 2)}) ^ {2}}{3 M (1 - c _ {0})}, (M + 2) (1 + 4 M d _ {m, t} ^ {2}) \}.
$$
Proof. Assume that parameter  $|\lambda| < \frac{1}{\alpha}$ . We prove the statement on the conditional moment generating function by induction. Let us start with the basis step.
Note that the definition of  $A$  and the choice of  $\delta$  again guarantee that for  $t \geq L$ ,  $|P_t - cE| < \delta < c$  on event  $A$ . This implies that for any  $t \geq L$ ,  $P_t > 0$  and thereby obtaining that if  $t = L$
$$
P _ {m, j} ^ {\prime} (t) = \frac {1}{M} \tag {34}
$$
$$
P _ {m, j} ^ {\prime} (t) = \frac {M - 1}{M ^ {2}}. \tag {35}
$$
Consider the time step  $t \leq L + 1$ . The quantity
$$
\begin{array}{l} E \left[ \exp \left\{\lambda \left(\tilde {\mu} _ {i} ^ {m} (t) - \mu_ {i}\right) \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ = E \left[ \exp \left\{\lambda \left(\tilde {\mu} _ {i} ^ {m} (L + 1) - \mu_ {i}\right) \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ = E \left[ \exp \left\{\lambda \left(\sum_ {j = 1} ^ {M} P _ {m, j} ^ {\prime} (L) \hat {\tilde {\mu}} _ {i, j} ^ {m} \left(h _ {m, j} ^ {L}\right) - \mu_ {i}\right) \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ = E \left[ \exp \left\{\lambda \left(\sum_ {j = 1} ^ {M} \frac {1}{M} \bar {\mu} _ {i} ^ {j} \left(h _ {m, j} ^ {L}\right) - \mu_ {i}\right) \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ = E \left[ \exp \left\{\lambda \sum_ {j = 1} ^ {M} \frac {1}{M} \left(\bar {\mu} _ {i} ^ {j} \left(h _ {m, j} ^ {L}\right) - \mu_ {i} ^ {j}\right) \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ \leq \Pi_ {j = 1} ^ {M} \left(E \left[ \left(\exp \left\{\left(\lambda \frac {1}{M} \left(\bar {\mu} _ {i} ^ {j} \left(h _ {m, j} ^ {L}\right) - \mu_ {i} ^ {j}\right) \right\} 1 _ {A _ {\epsilon , \delta}}\right) ^ {M} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right)\right) \right]\right) ^ {\frac {1}{M}} \tag {36} \\ \end{array}
$$
where the third equality holds by (34), the fourth equality uses the definition  $\mu_{i} = \frac{1}{M}\sum_{i = 1}^{M}\mu_{i}^{j}$  and the last inequality results from the generalized hoeffding inequality as in Lemma 3.
Note that for any client  $j$ , by the definition of  $\bar{\mu}_i^j (h_{m,j}^L)$  we have
$$
\begin{array}{l} E \left[ \left(\exp \left\{\left(\lambda \frac {1}{M} \left(\bar {\mu} _ {i} ^ {j} \left(h _ {m, j} ^ {L}\right) - \mu_ {i} ^ {j}\right) \right\} 1 _ {A _ {\epsilon , \delta}}\right) ^ {M} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right)\right) \right] \\ = E \left[ \exp \left\{\left(\lambda \left(\bar {\mu} _ {i} ^ {j} \left(h _ {m, j} ^ {L}\right) - \mu_ {i} ^ {j}\right)\right) 1 _ {A _ {\epsilon , \delta}} \right| \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ = E \left[ \exp \left\{\left(\lambda \frac {\sum_ {s} \left(r _ {i} ^ {j} (s) - \mu_ {i} ^ {j}\right)}{n _ {j , i} \left(h _ {m , j} ^ {L}\right)} \right\} 1 _ {A _ {\epsilon , \delta}} \right. \right\rvert   \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) ] \\ = E \left[ \exp \left\{\sum_ {s} \left(\lambda \frac {\left(r _ {i} ^ {j} (s) - \mu_ {i} ^ {j}\right)}{n _ {j , i} \left(h _ {m , j} ^ {L}\right)} \right\} 1 _ {A _ {\epsilon , \delta}}\right) \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right]. \tag {37} \\ \end{array}
$$
It is worth noting that given  $s$ ,  $r_i^j(s)$  is independent of everything else, which gives us
$$
\begin{array}{l} (3 7) = \Pi_ {s} E [ \exp \left\{\lambda \frac {\left(r _ {i} ^ {j} (s) - \mu_ {i} ^ {j}\right)}{n _ {j , i} \left(h _ {m , j} ^ {L}\right)} \right\} 1 _ {A _ {\epsilon , \delta}} | \sigma (\left\{n _ {m, i} (t) \right\} _ {t, i, m}) ] \\ = \Pi_ {s} E \left[ \exp \left\{\lambda \frac {\left(r _ {i} ^ {j} (s) - \mu_ {i} ^ {j}\right)}{n _ {j , i} \left(h _ {m , j} ^ {L}\right)} \right\} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \cdot E \left[ 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ = \Pi_ {s} E _ {r} \left[ \exp \left\{\lambda \frac {\left(r _ {i} ^ {j} (s) - \mu_ {i} ^ {j}\right)}{n _ {j , i} \left(h _ {m , j} ^ {L}\right)} \right\} \right] \cdot E \left[ 1 _ {A _ {\epsilon , \delta}} | \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ \leq \Pi_ {s} \exp \left\{\frac {\left(\frac {\lambda}{n _ {j , i} \left(h _ {m , j} ^ {L}\right)}\right) ^ {2} \sigma^ {2}}{2} \right\} \cdot E \left[ 1 _ {A _ {\epsilon , \delta}} | \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ \leq \left(\exp \left\{\frac {\left(\frac {\lambda}{n _ {j , i} \left(h _ {m , j} ^ {L}\right)}\right) ^ {2} \sigma^ {2}}{2} \right\}\right) ^ {n _ {j, i} \left(h _ {m, j} ^ {L}\right)} \\ = \exp \left\{\frac {\frac {\lambda^ {2}}{n _ {j , i} \left(h _ {m , j} ^ {L}\right)} \sigma^ {2}}{2} \right\} \leq \exp \left\{\frac {\lambda^ {2} \sigma^ {2}}{2 \min  _ {j} n _ {j , i} \left(h _ {m , j} ^ {L}\right)} \right\} \tag {38} \\ \end{array}
$$
where the first inequality holds by the definition of sub-exponential random variables  $r_i^j (s) - \mu_i^j$  with mean 0, the second inequality again uses  $1_{A_{\epsilon ,\delta}}\leq 1$ , and the last inequality is by the fact that  $n_{j,i}(h_{m,j}^{L})\geq \min_{j}n_{j,i}(h_{m,j}^{L})$ .
Therefore, we obtain that by plugging (38) into (36)
$$
\begin{array}{l} (3 6) \leq \Pi_ {j = 1} ^ {M} (\exp \left\{\frac {\lambda^ {2} \sigma^ {2}}{2 \min _ {j} n _ {j , i} (h _ {m , j} ^ {L})} \right\}) ^ {\frac {1}{M}} \\ = ((\exp \left\{\frac {\lambda^ {2} \sigma^ {2}}{2 \min _ {j} n _ {j , i} (h _ {m , j} ^ {L})} \right\}) ^ {\frac {1}{M}}) ^ {M} = \exp \left\{\frac {\lambda^ {2} \sigma^ {2}}{2 \min _ {j} n _ {j , i} (h _ {m , j} ^ {L})} \right\} \\ \end{array}
$$
Now we proceed to the induction step. Suppose that for any  $s < t + 1$  where  $t + 1 > L + 1$ , we have
$$
\begin{array}{l} E \left[ \exp \left\{\lambda \left(\tilde {\mu} _ {i} ^ {m} (s) - \mu_ {i}\right) \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (s) \right\} _ {s, i, m}\right) \right] \\ \leq \exp \left\{\frac {\lambda^ {2}}{2} \frac {C \sigma^ {2}}{\min  _ {j} n _ {j , i} (s)} \right\} \tag {39} \\ \end{array}
$$
The update rule of  $\tilde{\mu}_i^m$  again and (25) implies that
$$
\begin{array}{l} E \left[ \exp \left\{\lambda \left(\tilde {\mu} _ {i} ^ {m} (t + 1) - \mu_ {i}\right) \right\} 1 _ {A _ {e, \delta}} \mid \sigma \left(\left\{n _ {m, i} (s) \right\} _ {s, i, m}\right) \right] \\ \leq \Pi_ {j = 1} ^ {M} (\exp \left\{\frac {\lambda^ {2} (P _ {t} ^ {\prime} (m , j)) ^ {2} (M + 2) ^ {2}}{2} \frac {C \sigma^ {2}}{\min _ {j} n _ {j , i} (t _ {m , j})} \right\}) ^ {\frac {1}{M + 2}}. \\ \Pi_ {j \in N _ {m} (t)} \Pi_ {s} \big (E _ {r} [ \exp \{\lambda d _ {m, t} (M + 2) \frac {(r _ {i} ^ {j} (s) - \mu_ {i} ^ {j})}{n _ {j , i} (t)} \} ] \cdot E [ 1 _ {A _ {\epsilon , \delta}} | \sigma (\{n _ {m, i} (t) \} _ {t, i, m}) ] \big) ^ {\frac {1}{M + 2}}. \\ \Pi_ {j \notin N _ {m} (t)} \Pi_ {s} \left(E _ {r} \left[ \exp \left\{\lambda d _ {m, t} (M + 2) \frac {\left(r _ {i} ^ {j} (s) - \mu_ {i} ^ {j}\right)}{n _ {j , i} \left(t _ {m , j}\right)} \right\} \right] \cdot E \left[ 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right]\right) ^ {\frac {1}{M + 2}}. \tag {40} \\ \end{array}
$$
We continue bounding the last two terms by using the definition of sub-exponential random variables  $(r_i^j (s) - \mu_i^j)$  and obtain
$$
\begin{array}{l} (4 0) \leq (\exp \{\frac {\lambda^ {2} (P _ {t} ^ {\prime} (m , j)) ^ {2} (M + 2) ^ {2}}{2} \frac {C \sigma^ {2}}{\min _ {j} n _ {j , i} (t _ {m , j})} \}) ^ {\frac {M}{M + 2}}. \\ \Pi_ {j \in N _ {m} (t)} \Pi_ {s} (\exp \frac {\lambda^ {2} d _ {m , t} ^ {2} (M + 2) ^ {2} \sigma^ {2}}{2 n _ {j , i} ^ {2} (t)} \cdot E [ 1 _ {A _ {\epsilon , \delta}} | \sigma (\{n _ {m, i} (t) \} _ {t, i, m}) ]) ^ {\frac {1}{M + 2}}. \\ \Pi_ {j \not \in N _ {m} (t)} \Pi_ {s} (\exp {\frac {\lambda^ {2} d _ {m , t} ^ {2} (M + 2) ^ {2} \sigma^ {2}}{2 n _ {j , i} ^ {2} (t _ {m , j})}} \cdot E [ 1 _ {A _ {\epsilon , \delta}} | \sigma (\{n _ {m, i} (t) \} _ {t, i, m}) ]) ^ {\frac {1}{M + 2}} \\ = (\exp \left\{\frac {\lambda^ {2} (P _ {t} ^ {\prime} (m , j)) ^ {2} (M + 2) ^ {2}}{2} \frac {C \sigma^ {2}}{\min _ {j} n _ {j , i} (t _ {m , j})} \right\}) ^ {\frac {M}{M + 2}}. \\ \Pi_ {j \in N _ {m} (t)} \exp \left\{\frac {n _ {j , i} (t)}{M + 2} \frac {\lambda^ {2} d _ {m , t} ^ {2} (M + 2) ^ {2} \sigma^ {2}}{2 n _ {j , i} ^ {2} (t)} \right\} \cdot E [ 1 _ {A _ {\epsilon , \delta}} | \sigma (\{n _ {m, i} (t) \} _ {t, i, m}) ]. \\ \Pi_ {j \not \in N _ {m} (t)} \exp \left\{\frac {n _ {j , i} (t _ {m , j})}{M + 2} \frac {\lambda^ {2} d _ {m , t} ^ {2} (M + 2) ^ {2} \sigma^ {2}}{2 n _ {j , i} ^ {2} (t _ {m , j})} \right\} \cdot E [ 1 _ {A _ {\epsilon , \delta}} | \sigma (\{n _ {m, i} (t) \} _ {t, i, m}) ]. \\ \end{array}
$$
$$
\begin{array}{l} (4 0) \leq (\exp \left\{\frac {\lambda^ {2} \left(P _ {t} ^ {\prime} (m , j)\right) ^ {2} (M + 2) ^ {2}}{2} \frac {C \sigma^ {2}}{\operatorname* {m i n} _ {j} n _ {j , i} \left(t _ {m , j}\right)} \right\}) ^ {\frac {M}{M + 2}}. \\ \left. \right.\left(\exp \left\{\frac {\lambda^ {2} d _ {m , t} ^ {2} (M + 2) \sigma^ {2}}{2 \min  _ {j} n _ {j , i} (t)} \right\}\right) ^ {| N _ {m} (t) |} \cdot E \left[ 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right)\right] \cdot \\ \left. \right.\left(\exp \left\{\frac {\lambda^ {2} d _ {m , t} ^ {2} (M + 2) \sigma^ {2}}{2 \min  _ {j} n _ {j , i} \left(t _ {m , j}\right)} \right\}\right) ^ {| M - N _ {m} (t) |} \cdot E \left[ 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right)\right] \\ = E [ (\exp \{\frac {\lambda^ {2} (P _ {t} ^ {\prime} (m , j)) ^ {2} M (M + 2)}{2} \frac {C \sigma^ {2}}{\min _ {j} n _ {j , i} (t _ {m , j})} \}) \cdot (\exp \{\frac {\lambda^ {2} d _ {m , t} ^ {2} (M + 2) | N _ {m} (t) |}{2 \min _ {j} n _ {j , i} (t)} \}) \\ \cdot \left(\exp \left\{\frac {\lambda^ {2} d _ {m , t} ^ {2} (M + 2) \sigma^ {2} | M - N _ {m} (t) |}{2 \min  _ {j} n _ {j , i} (t _ {m , j})} \right\}\right) 1 _ {A _ {\epsilon , \delta}} \left| \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ \leq E [ (\exp \{\frac {\lambda^ {2} (P _ {t} ^ {\prime} (m , j)) ^ {2} M (M + 2)}{2 (1 - c _ {0})} \frac {C \sigma^ {2}}{\min _ {j} n _ {j , i} (t + 1)} \}) \cdot (\exp \{\frac {\lambda^ {2} d _ {m , t} ^ {2} (M + 2) | N _ {m} (t) | \sigma^ {2}}{2 \frac {L / K}{L / K + 1} \min _ {j} n _ {j , i} (t + 1)} \}) \\ \cdot \left(\exp \left\{\frac {\lambda^ {2} d _ {m , t} ^ {2} (M + 2) | M - N _ {m} (t) | \sigma^ {2}}{2 \left(1 - c _ {0}\right) \min  _ {j} n _ {j , i} (t + 1)} \right\}\right) 1 _ {A _ {\epsilon , \delta}} \left| \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ \end{array}
$$
where the first inequality uses the fact that  $n_{j,i}(t) \geq \min_j n_{j,i}(t)$  and  $n_{j,i}(t_{m,j}) \geq \min_j n_{j,i}(t_{m,j})$ . For the second inequality, the first term is a result of  $\frac{\min_j n_{j,i}(t)}{\min_j n_{j,i}(t + 1)} \geq \frac{\min_j n_{j,i}(t)}{\min_j n_{j,i}(t) + 1} \geq \frac{L / K}{L / K + 1}$  since  $n_{j,i}(t) > n_{j,i}(L) = L / K$  and the ratio is monotone increasing in  $n$ , and the second term is bounded through applying Proposition 6, which holds on event  $A_{\epsilon,\delta}$  and leads to
$$
\begin{array}{l} \min  _ {j} n _ {j, i} \left(t _ {m, j}\right) \geq \min  _ {j} n _ {j, i} \left(t + 1 - t _ {0}\right) \\ \geq \min  _ {j} n _ {j, i} (t + 1) - t _ {0} \\ \geq \min  _ {j} n _ {j, i} (t + 1) - c _ {0} \min  _ {j} n _ {j, i} (t + 1) \\ = (1 - c _ {0}) \min  _ {j} n _ {j, i} (t + 1). \\ \end{array}
$$
$$
\begin{array}{l} (4 0) = E [ (\exp \{\frac {\lambda^ {2} \sigma^ {2}}{2 \min _ {j} n _ {j , i} (t + 1)} \cdot (\frac {C \lambda^ {2} (P _ {t} ^ {\prime} (m , j)) ^ {2} M (M + 2)}{2 (1 - c _ {0})} + \\ \frac {d _ {m , t} ^ {2} (M + 2) | N _ {m} (t) |}{\frac {L / K}{L / K + 1}} + \frac {d _ {m , t} ^ {2} (M + 2) | M - N _ {m} (t) |}{(1 - c _ {0})}) \} 1 _ {A _ {\epsilon , \delta}} | \sigma (\{n _ {m, i} (t) \} _ {t, i, m}) ] \\ \leq E \left[ \exp \left\{\frac {C \lambda^ {2} \sigma^ {2}}{2 \min  _ {j} n _ {j , i} (t + 1)} \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ \leq \exp \left\{\frac {C \lambda^ {2} \sigma^ {2}}{2 \min  _ {j} n _ {j , i} (t + 1)} \right\} \\ \end{array}
$$
where the first inequality holds again by the choice of  $P_{t}^{\prime}(m,j), d_{m,t}, L$  and  $c_{0}$  and the second inequality uses the fact that  $1_{A_{\epsilon, \delta}} \leq 1$ .
Proposition 12. Assume the parameter  $\delta$  satisfies that  $0 < \delta < c = f(\epsilon, M, T)$ . In setting  $S_{1}, S_{2}$ , and  $S_{3}$ , for any  $m, i$  and  $t > L$  where  $L$  is the length of the burn-in period, the deviation of  $\tilde{\mu}_{m,i}(t)$
satisfies that if  $n_{m,i}(t) \geq 2(K^2 + KM + M)$ , then with  $P(A_{\epsilon, \delta}) = 1 - 7\epsilon$ ,
$$
P (\tilde {\mu} _ {m, i} (t) - \mu_ {i} \geq \sqrt {\frac {C _ {1} \log t}{n _ {m , i} (t)}} + \frac {C _ {2} \log t}{n _ {m , i} (t)} | A _ {\epsilon , \delta}) \leq \frac {1}{P (A _ {\epsilon , \delta})} \frac {1}{T ^ {4}},
$$
$$
P (\mu_ {i} - \tilde {\mu} _ {m, i} (t) \geq \sqrt {\frac {C _ {1} \log t}{n _ {m , i} (t)}} + \frac {C _ {2} \log t}{n _ {m , i} (t)} | A _ {\epsilon , \delta}) \leq \frac {1}{P (A _ {\epsilon , \delta})} \frac {1}{T ^ {4}}.
$$
Proof. By Proposition 7, we have  $E[\tilde{\mu}_{m,i}(t) - \mu_i|A_{\epsilon,\delta}] = 0$ , which allows us to consider the tail bound of the global estimator  $\tilde{\mu}_i^m(t)$  conditional on event  $A_{\epsilon,\delta}$ . It is worth mentioning that by the choice of  $C_1$  and  $C_2$ , we have
$$
C _ {1} ^ {2} \cdot \frac {\alpha^ {2}}{\tilde {\sigma} ^ {4}} \leq C _ {2} ^ {2}.
$$
where  $\tilde{\sigma}^2$  is  $\frac{2C\sigma^2}{n_{m,i}(t)}$ .
Note that since we set  $\text{Rad} = \sqrt{\frac{C_1 \ln T}{n_{m,i}(t)}} + \frac{C_2 \ln T}{n_{m,i}(t)}$ , we obtain
$$
P \left(\left| \tilde {\mu} _ {i} ^ {m} (t) - \mu_ {i} \right| > R a d \mid A _ {\epsilon , \delta}\right) <   P \left(\left| \tilde {\mu} _ {i} ^ {m} (t) - \mu_ {i} \right| > \sqrt {\frac {C _ {1} \ln T}{n _ {m , i} (t)}} \mid A _ {\epsilon , \delta}\right), \tag {41}
$$
$$
P \left(\left| \tilde {\mu} _ {i} ^ {m} (t) - \mu_ {i} \right| > R a d \mid A _ {\epsilon , \delta}\right) <   P \left(\left| \tilde {\mu} _ {i} ^ {m} (t) - \mu_ {i} \right| > \frac {C _ {2} \ln T}{n _ {m , i} (t)} \mid A _ {\epsilon , \delta}\right) \tag {42}
$$
On the one hand, if  $\frac{\sqrt{\frac{C_1\log T}{n_{m,i}(t)}}}{\tilde{\sigma}^2} >\frac{1}{\alpha}$ , i.e.  $n_{m,i}(t)\leq C_1\log T\frac{\alpha^2}{(\tilde{\sigma})^4}$ , we have
$$
\begin{array}{l} P \left(\left| \tilde {\mu} _ {i} ^ {m} (t) - \mu_ {i} \right| > \frac {C _ {2} \ln T}{n _ {m , i} (t)} \mid A _ {\epsilon , \delta}\right) \\ = E [ 1 _ {| \tilde {\mu} _ {i} ^ {m} (t) - \mu_ {i} | > \frac {C _ {2} \ln T}{n _ {m , i} (t)}} | A _ {\epsilon , \delta} ] \\ = \frac {1}{P (A _ {\epsilon , \delta})} E [ 1 _ {| \tilde {\mu} _ {i} ^ {m} (t) - \mu_ {i} | > \frac {C _ {2} \ln T}{n _ {m , i} (t)}} 1 _ {A _ {\epsilon , \delta}} ] \\ = \frac {1}{P (A _ {\epsilon , \delta})} E [ 1 _ {\exp \{\lambda (n) (| \tilde {\mu} _ {m, i} (t) - \mu_ {i} |) \} \geq \exp \{\lambda (n) \frac {C _ {2} \ln T}{n _ {m , i} (t)} \}} 1 _ {A _ {\epsilon , \delta}} ] \\ \leq \frac {1}{P \left(A _ {\epsilon , \delta}\right)} E \left[ \frac {\exp \left\{\lambda (n) \left(\left| \tilde {\mu} _ {m , i} (t) - \mu_ {i} \right|\right) \right\}}{\exp \left\{\lambda (n) \frac {C _ {2} \ln T}{n _ {m , i} (t)} \right\}} 1 _ {A _ {\epsilon , \delta}} \right] \tag {43} \\ \end{array}
$$
where the last inequality is by the fact that  $1_{\exp \{\lambda (n)(|\tilde{\mu}_{m,i}(t) - \mu_i|)\}}\geq \exp \{\lambda (n)\frac{C_2\ln T}{n_{m,i}(t)}\} \leq$
$$
\frac {\exp \{\lambda (n) (| \tilde {\mu} _ {m , i} (t) - \mu_ {i} |) \}}{\exp \{\lambda (n) \frac {\mathcal {C} _ {2} \ln T}{n _ {m , i} (t)} \}}.
$$
By the assumption that  $\delta < c$ , we have Proposition 11 holds. Subsequently, by Proposition 11 and Lemma 2 which holds since  $n_{m,i}(t) \geq 2(K^2 + KM + M)$ , we have for any  $|\lambda| < \frac{1}{\alpha}$
$$
E \left[ \exp \left\{\lambda \left(\tilde {\mu} _ {i} ^ {m} (t) - \mu_ {i}\right) \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \leq \exp \left\{\frac {\lambda^ {2}}{2} \tilde {\sigma} ^ {2} \right\}. \tag {44}
$$
Likewise, we obtain that by taking  $\lambda = -\lambda$
$$
E \left[ \exp \left\{\lambda \left(- \tilde {\mu} _ {i} ^ {m} (t) + \mu_ {i}\right) \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \leq \exp \left\{\frac {\lambda^ {2}}{2} \tilde {\sigma} ^ {2} \right\}. \tag {45}
$$
With (44) and (45), we arrive at for any  $|\lambda| < \frac{1}{\alpha}$  that
$$
E \left[ \exp \left\{\lambda \left(\left| \tilde {\mu} _ {i} ^ {m} (t) - \mu_ {i}\right) \right| \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \leq 2 \exp \left\{\frac {\lambda^ {2}}{2} \tilde {\sigma} ^ {2} \right\}. \tag {46}
$$
Again, we utilize the law of total expectation and further obtain that  $|\lambda (n)| < \frac{1}{\alpha}$
$$
\begin{array}{l} (4 3) = \frac {1}{P \left(A _ {\epsilon , \delta}\right)} E \left[ E \left[ \frac {\exp \left\{\lambda (n) \left(\left| \tilde {\mu} _ {m , i} (t) - \mu_ {i} \right|\right) \right\}}{\exp \left\{\lambda (n) \frac {C _ {2} \ln T}{n _ {m , i} (t)} \right\}} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {m, i, t}\right) \right] \right] \\ = \frac {1}{P (A _ {\epsilon , \delta})} E \left[ \frac {1}{\exp \{\lambda (n) \frac {C _ {2} \ln T}{n _ {m , i} (t)} \}} E [ \exp \{\lambda (n) (| \tilde {\mu} _ {m, i} (t) - \mu_ {i} |) \} 1 _ {A _ {\epsilon , \delta}} | \sigma (\{n _ {m, i} (t) \} _ {m, i, t}) ] \right] \\ \leq 2 \frac {1}{P \left(A _ {\epsilon , \delta}\right)} E \left[ \frac {1}{\exp \left\{\lambda (n) \frac {C _ {2} \ln T}{n _ {m , i} (t)} \right\}} \cdot \exp \left\{\frac {\lambda^ {2} (n)}{2} \tilde {\sigma} ^ {2} \right\} \right] \tag {47} \\ \end{array}
$$
where the first inequality holds by (46).
Note that the condition  $\frac{\sqrt{\frac{C_1\log T}{n_{m,i}(t)}}}{\tilde{\sigma}^2} >\frac{1}{\alpha}$  implies that  $n_{m,i}(t) < \frac{C_1\ln T}{\frac{\tilde{\sigma}^2}{\alpha}}$  which is the global optima of the function in (47). This is true since  $n_{m,i}(t)\leq C_1\log T\frac{\alpha^2}{(\tilde{\sigma})^4}\leq \frac{(C_2\log T)^2}{C_1}$ . Henceforth, (47) is monotone decreasing in  $\lambda (n)\in (0,\frac{1}{\alpha})$  and we obtain a minima when choosing  $\lambda (n) = \frac{1}{\alpha}$  and using the continuity of (47).
Formally, it yields that
$$
\begin{array}{l} (4 7) \leq 2 \frac {1}{P (A _ {\epsilon , \delta})} E [ \frac {1}{\exp \left\{\frac {1}{\alpha} \frac {C _ {2} \ln T}{n _ {m , i} (t)} \right\}} \cdot \exp \left\{\frac {\frac {1}{2 \alpha^ {2}}}{1} \tilde {\sigma} ^ {2} \right\} ] \\ = 2 \frac {1}{P (A _ {\epsilon , \delta})} E [ \exp \left\{\frac {1}{2 \alpha^ {2}} \tilde {\sigma} ^ {2} - \frac {1}{\alpha} \frac {C _ {2} \ln T}{n _ {m , i} (t)} \right\} ] \\ \leq 2 \frac {1}{P \left(A _ {\epsilon , \delta}\right)} \exp \{- 4 \log T \} = \frac {2}{P \left(A _ {\epsilon , \delta}\right) T ^ {4}} \tag {48} \\ \end{array}
$$
where the last inequality uses the choice of  $C_2$  and the condition that  $\frac{1}{2\alpha^2}\tilde{\sigma}^2 -\frac{1}{\alpha}\frac{C_2\ln T}{n_{m,i}(t)}\leq -4\ln T$  which holds by the following derivation. Notably, we have
$$
\begin{array}{l} \frac {1}{2 \alpha^ {2}} \tilde {\sigma} ^ {2} - \frac {1}{\alpha} \frac {C _ {2} \ln T}{n _ {m , i} (t)} \leq \frac {1}{2 \alpha^ {2}} \tilde {\sigma} ^ {2} - \frac {C _ {2} \ln T}{\alpha} \frac {\frac {\tilde {\sigma} ^ {2}}{\alpha}}{C _ {1} \ln T} \\ = \frac {1}{2 \alpha^ {2}} \tilde {\sigma} ^ {2} - \frac {C _ {2}}{C _ {1}} \frac {\tilde {\sigma} ^ {2}}{\alpha^ {2}} \\ = \left(\frac {1}{2} - \frac {C _ {2}}{C _ {1}}\right) \left(\tilde {\sigma} ^ {2} \cdot \frac {\frac {C _ {1} \log T}{n _ {m , i} (t)}}{\tilde {\sigma} ^ {4}}\right) = \left(\frac {1}{2} - \frac {C _ {2}}{C _ {1}}\right) \left(\frac {\frac {C _ {1} \log T}{n _ {m , i} (t)}}{\tilde {\sigma} ^ {2}}\right) \\ = \left(\frac {1}{2} - \frac {C _ {2}}{C _ {1}}\right) \cdot \frac {C _ {1} \log T}{n _ {m , i} (t)} \cdot \frac {1}{\frac {2 C \sigma^ {2}}{n _ {m , i} (t)}} = \left(\frac {1}{2} - \frac {C _ {2}}{C _ {1}}\right) \frac {C _ {1}}{2 C \sigma^ {2}} \log T \leq - 4 \log T \\ \end{array}
$$
where the first inequality uses  $n_{m,i}(t) < \frac{C_1 \ln T}{\frac{\sigma^2}{\alpha}}$  and the last inequality is by the choices of parameters  $\left(\frac{1}{2} - \frac{C_2}{C_1}\right) \frac{C_1}{2C\sigma^2} \leq -4$ .
On the other hand, if  $\frac{\sqrt{\frac{C_1\log T}{n_{m,i}(t)}}}{\tilde{\sigma}^2} < \frac{1}{\alpha}$ , i.e.  $n_{m,i}(t) \geq C_1\log T\frac{\alpha^2}{(\tilde{\sigma})^4}$ , we observe for  $|\lambda (n)| < \frac{1}{\alpha}$