SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Case 2.1's happening means that arm  $k$  is not eliminated in or before phase  $p_k$ , which can only happen when the arm's empirical mean  $\hat{\mu}_k^{(M)}$  lies outside its corresponding confidence interval. This event in or before phase  $p_k$  is with a probability no greater than  $2 \times \frac{2^2p_k\lambda^{(M)}}{\Lambda}$ . Since this event may happen to any suboptimal arm  $k \in \mathcal{K}'$ , then the regret of this case is upper bounded by
$$
\begin{array}{l} \sum_ {k \in \mathcal {K} ^ {\prime}} \frac {2 ^ {2 p _ {k} + 1} \lambda^ {(M)}}{\Lambda} \frac {\Lambda}{\lambda^ {(M)}} \left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {k} ^ {(M)}\right) = \sum_ {k \in \mathcal {K} ^ {\prime}} 2 ^ {2 p _ {k} + 1} \left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {k} ^ {(M)}\right) \\ \stackrel {(a)} {\leqslant} \sum_ {k \in \mathcal {K} ^ {\prime}} \frac {3 2}{(\Delta_ {k} ^ {(M)}) ^ {2}} \left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {k} ^ {(M)}\right), \\ \end{array}
$$
If Case 2.2 happens, the optimal arm 1 is not in the candidate arm set  $\mathcal{C}_{p_k}$  in phase  $p_k$ . We denote  $p_1$  as the phase that the optimal arm 1 is eliminated, and it is at this phase that some arms' empirical means lie outside their confidence interval so that this mis-elimination happens. The probability of this event is upper bounded by  $2 \times \frac{2^{2p_1} \lambda^{(M)}}{\Lambda}$ . We assume that arms  $k$  with  $p_k < p_1$  are eliminated in or before phase  $p_i$  properly; otherwise, the regret is counted in Case 2.1. Therefore, the optimal arm 1 eliminated in phase  $p_1$  should be eliminated by an arm  $k$  with  $p_k \geqslant p_1$ . Consequently, the maximal per time slot regret in Case 2.2 is among arms with  $p_k \geqslant p_1$ . Denote  $p_\varepsilon \coloneqq \min \{p | 2^{-p} < \frac{\varepsilon}{2}\}$ . We bound the cost of Case 2.2 as follows,
$$
\begin{array}{l} \sum_ {p _ {1} = 0} ^ {\max  _ {i \in \mathcal {K} ^ {\prime}} p _ {i}} \sum_ {k > 1: p _ {k} \geqslant p _ {1}} \frac {2 ^ {2 p _ {1} + 1} \lambda^ {(M)}}{\Lambda} \frac {\Lambda}{\lambda^ {(M)}} \cdot \max  _ {k ^ {\prime} > 1: p _ {k ^ {\prime}} \geqslant p _ {1}} \left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {k ^ {\prime}} ^ {(M)}\right) \\ = \sum_ {p _ {1} = 0} ^ {\max  _ {i \in \mathcal {K} ^ {\prime}} p _ {i}} \sum_ {k > 1: p _ {k} \geqslant p _ {1}} 2 ^ {2 p _ {1} + 1} \cdot \left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {1} ^ {(M)} + \max  _ {k ^ {\prime} > 1: p _ {k ^ {\prime}} \geqslant p _ {1}} \Delta_ {k ^ {\prime}} ^ {(M)}\right) \\ \stackrel {(a)} {\leqslant} \sum_ {p _ {1} = 0} ^ {\max  _ {i \in \mathcal {K} ^ {\prime}} p _ {i}} \sum_ {k > 1: p _ {k} \geqslant p _ {1}} 2 ^ {2 p _ {1} + 1} \cdot \left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {1} ^ {(M)} + 4 \times 2 ^ {- p _ {1}}\right) \\ \stackrel {(b)} {\leqslant} \sum_ {k > 1} \sum_ {p _ {1} = 0} ^ {\min  \left\{p _ {k}, p _ {\varepsilon} \right\}} 2 ^ {2 p _ {1} + 1} \cdot \left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {1} ^ {(M)} + 4 \times 2 ^ {- p _ {1}}\right) \\ \leqslant \sum_ {k > 1} \left(\left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {1} ^ {(M)}\right) \sum_ {p _ {1} = 0} ^ {\min  \{p _ {k}, p _ {\varepsilon} \}} 2 ^ {2 p _ {1} + 1} + \sum_ {p _ {1} = 0} ^ {\min  \{p _ {k}, p _ {\varepsilon} \}} 2 ^ {p _ {1} + 3}\right) \\ \leqslant \sum_ {k > 1} \left(\left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {1} ^ {(M)}\right) \frac {2 ^ {2 \min  \{p _ {k} , p _ {\varepsilon} \} + 1}}{3} + 2 ^ {\min  \{p _ {k}, p _ {\varepsilon} \} + 4}\right) \\ \leqslant \sum_ {k \in \mathcal {K} ^ {\prime}} \left(\left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {1} ^ {(M)}\right) \frac {2 ^ {2 p _ {k} + 1}}{3} + 2 ^ {p _ {k} + 4}\right) + \sum_ {k > 1, k \notin \mathcal {K} ^ {\prime}} \left(\left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {1} ^ {(M)}\right) \frac {2 ^ {2 p _ {\varepsilon} + 1}}{3} + 2 ^ {p _ {\varepsilon} + 4}\right) \\ \stackrel {(c)} {\leqslant} \sum_ {k \in \mathcal {K} ^ {\prime}} \left(\left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {1} ^ {(M)}\right) \frac {3 2}{3 \left(\Delta_ {k} ^ {(M)}\right) ^ {2}} + \frac {6 4}{\Delta_ {k} ^ {(M)}}\right) + \sum_ {k > 1, k \notin \mathcal {K} ^ {\prime}} \left(\left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {1} ^ {(M)}\right) \frac {3 2}{3 \varepsilon^ {2}} + \frac {6 4}{\varepsilon}\right) \\ \leqslant \sum_ {k \in \mathcal {K} ^ {\prime}} \left(\left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {k} ^ {(M)}\right) \frac {3 2}{3 \left(\Delta_ {k} ^ {(M)}\right) ^ {2}} + \frac {6 4}{\Delta_ {k} ^ {(M)}}\right) + \sum_ {k > 1, k \notin \mathcal {K} ^ {\prime}} \left(\left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {k} ^ {(M)}\right) \frac {3 2}{3 \varepsilon^ {2}} + \frac {6 4}{\varepsilon}\right) \tag {32} \\ \end{array}
$$
where (a) and (c) are due to Eq.(27), and (b) is due to the property of swapping two summations.
Summing up the costs in Eq.(29), Eq.(30), Eq.(31), and Eq.(32) concludes the proof.
Next, we derive the problem-independent regret bound from Eq.(26). When  $\Lambda$  is large, the  $O(\log \Lambda)$  and  $O(\Lambda)$  terms dominate other terms in Eq.(26). For any given  $\varepsilon$ , if  $\Lambda$  is large enough, we always have  $\varepsilon > 4\sqrt{\frac{\lambda^{(M)}e}{\Lambda}}$ , which, with some calculus, guarantees that  $\frac{16}{(\Delta_k^{(M)})^2}\log \frac{\Lambda(\Delta_k^{(M)})^2}{16\lambda^{(M)}} < \frac{16}{\varepsilon^2}\log \frac{\Lambda\varepsilon^2}{16\lambda^{(M)}}$  for all  $\Delta_k^{(M)} > \varepsilon$ . Therefore, we can scale all logarithmic arm pulling times as  $\frac{16}{\varepsilon^2}\log \frac{\Lambda\varepsilon^2}{16\lambda^{(M)}}$ , and upper bound the pulling cost  $\left(\frac{\lambda^{(M)}}{\lambda^{(1)}}\mu_1^{(M)} - \mu_k^{(M)}\right)$  by  $\frac{\lambda^{(M)}}{\lambda^{(1)}}\mu_1^{(M)}$ . We derive the problem-independent regret upper bound as follows,
$$
\begin{array}{l} \mathbb {E} [ R (\Lambda) ] \leqslant \frac {K \mu_ {1} ^ {(M)} \cdot \lambda^ {(M)}}{\lambda^ {(1)}} \frac {1 6}{\varepsilon^ {2}} \log \frac {\Lambda \varepsilon^ {2}}{1 6 \lambda^ {(M)}} + \frac {\Lambda}{\lambda^ {(1)}} \varepsilon \\ \leqslant \frac {K \mu_ {1} ^ {(M)} \cdot \lambda^ {(M)}}{\lambda^ {(1)}} \frac {1 6}{\varepsilon^ {2}} \log \frac {\Lambda}{1 6 \lambda^ {(M)}} + \frac {\Lambda}{\lambda^ {(1)}} \varepsilon \\ \stackrel {(a)} {\leqslant} 2 \left(\frac {1 6 K \mu_ {1} ^ {(M)} \lambda^ {(M)}}{\lambda^ {(1)}} \log \frac {\Lambda}{1 6 \lambda^ {(M)}}\right) ^ {\frac {1}{3}} \left(\frac {\Lambda}{\lambda^ {(1)}}\right) ^ {\frac {2}{3}}, \\ \end{array}
$$
where the equation of (a) holds when  $\varepsilon = \left(\frac{K\mu_1^{(M)}\log(\Lambda / 16\lambda^{(M)})}{(\Lambda / 16\lambda^{(M)})}\right)^{\frac{1}{3}}$