SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Po-An Wang
KTH, Stockholm, Sweden
wang9@kth.se
Ruo-Chun Tzeng
rctzeng@kth.se
Alexandre Proutiere
EECS and Digital Futures
alepro@kth.se
We consider the problem of identifying the best arm in stochastic Multi-Armed Bandits (MABs) using a fixed sampling budget. Characterizing the minimal instance-specific error probability for this problem constitutes one of the important remaining open problems in MABs. When arms are selected using a static sampling strategy, the error probability decays exponentially with the number of samples at a rate that can be explicitly derived via Large Deviation techniques. Analyzing the performance of algorithms with adaptive sampling strategies is however much more challenging. In this paper, we establish a connection between the Large Deviation Principle (LDP) satisfied by the empirical proportions of arm draws and that satisfied by the empirical arm rewards. This connection holds for any adaptive algorithm, and is leveraged (i) to improve error probability upper bounds of some existing algorithms, such as the celebrated SR (Successive Rejects) algorithm (Audibert et al., 2010), and (ii) to devise and analyze new algorithms. In particular, we present CR (Continuous Rejects), a truly adaptive algorithm that can reject arms in any round based on the observed empirical gaps between the rewards of various arms. Applying our Large Deviation results, we prove that CR enjoys better performance guarantees than existing algorithms, including SR. Extensive numerical experiments confirm this observation.
We study the problem of best-arm identification in stochastic bandits in the fixed budget setting. In this problem, abbreviated by BAI-FB, a learner faces  $K$  distributions or arms  $\nu_{1},\ldots ,\nu_{K}$  characterized by their unknown means  $\pmb {\mu} = (\mu_1,\dots ,\mu_K)$  (we restrict our attention to distributions taken from a one-parameter exponential family). She sequentially pulls arms and observes samples of the corresponding distributions. More precisely, in round  $t\geq 1$ , she pulls an arm  $A_{t} = k$  selected depending on previous observations and observes  $X_{k}(t)$  a sample of a  $\nu_{k}$ -distributed random variable.  $(X_{k}(t),t\geq 1,k\in [K])$  are assumed to be independent over rounds and arms. After  $T$  arm draws, the learner returns  $\hat{\imath}$ , an estimate of the best arm  $1(\pmb {\mu})\coloneqq \arg \max_k\mu_k$ . We assume that the best arm is unique, and denote by  $\Lambda$  the set of parameters  $\pmb{\mu}$  such that this assumption holds. The objective is to devise an adaptive sampling algorithm minimizing the error probability  $\mathbb{P}_{\pmb{\mu}}[\hat{\imath}\neq 1(\pmb {\mu})]$ . This learning task is one of the most important problems in stochastic bandits, and despite recent research efforts, it remains largely open (Qin, 2022). In particular, researchers have so far failed
at characterizing the minimal instance-specific error probability. This contrasts with other basic learning tasks in stochastic bandits such as regret minimization (Lai and Robbins, 1985) and BAI with fixed confidence (Garivier and Kaufmann, 2016), for which indeed, asymptotic instance-specific performance limits and matching algorithms have been derived. In BAI-FB, the error probability typically decreases exponentially with the sample budget  $T$ , i.e., it scales as  $\exp(-R(\mu)T)$  where the instance-specific rate  $R(\mu)$  depends on the sampling algorithm. Maximizing this rate over the set of adaptive algorithms is an open problem.
Instance-specific error probability lower bound. To guess the maximal rate at which the error probability decays, one may apply the same strategy as that used in regret minimization or BAI in the fixed confidence setting: (i) derive instance-specific lower bound for the error probability for some notion of uniformly good algorithms; (ii) devise a sampling strategy mimicking the optimal proportions of arm draws identified in the lower bound. Here the notion of uniformly good algorithms is that of consistent algorithms. Under such an algorithm, for any  $\pmb{\mu} \in \Lambda$ ,  $\mathbb{P}_{\pmb{\mu}}[i = 1(\pmb{\mu})] \to 1$  as  $T \to \infty$ . (Garivier and Kaufmann, 2016) conjectures the following asymptotic lower bound satisfied by any consistent algorithm (refer to Appendix J for details): as  $T \to \infty$ ,
$$
\frac {1}{T} \log \frac {1}{\mathbb {P} _ {\boldsymbol {\mu}} [ \hat {t} \neq 1 (\boldsymbol {\mu}) ]} \leq \max  _ {\boldsymbol {\omega} \in \Sigma} \inf  _ {\boldsymbol {\lambda} \in \operatorname {A l t} (\boldsymbol {\mu})} \Psi (\boldsymbol {\lambda}, \boldsymbol {\omega}), \tag {1}
$$
where  $\Sigma$  is the  $(K - 1)$ -dimensional simplex,  $\Psi (\lambda ,\omega) = \sum_{k = 1}^{K}\omega_{k}d(\lambda_{k},\mu_{k})$ ,  $\mathrm{Alt}(\pmb {\mu}) = \{\pmb {\lambda}\in \Lambda :$ $1(\pmb {\mu})\neq 1(\pmb {\lambda})\}$  is the set of confusing parameters (those for which  $1(\pmb {\mu})$  is not the best arm), and  $d(x,y)$  denotes the KL divergence between two distributions of parameters  $x$  and  $y$ . Interestingly, the solution  $\omega^{\star}\in \Sigma$  of the optimization problem  $\max_{\omega \in \Sigma}\inf_{\lambda \in \mathrm{Alt}(\pmb {\mu})}\Psi (\lambda ,\omega)$  provides the best static proportions of arm draws. More precisely, an algorithm selecting arms according to the allocation  $\omega^{\star}$ , i.e., selecting arm  $k$ $\omega_k^\star T$  times and returning the best empirical arm after  $T$  samples, has an error rate matching the lower bound (1). This is a direct consequence of the fact that, under a static algorithm with allocation  $\omega$ , the empirical reward process  $\{\hat{\mu} (t)\}_{t\geq 1}$  satisfies a LDP with rate function  $\lambda \mapsto \Psi (\lambda ,\omega)$ , see (Glynn and Juneja, 2004) and refer to Section 3 for more details.
Adaptive sampling algorithms and their analysis. The optimal allocation  $\omega^{\star}$  depends on the instance  $\mu$  and is initially unknown. We may devise an adaptive sampling algorithm that (i) estimates  $\omega^{\star}$  and (ii) tracks this estimated optimal allocation. In the BAI with fixed confidence, such tracking scheme exhibits asymptotically optimal performance (Garivier and Kaufmann, 2016). Here however, the error made estimating  $\omega^{\star}$  would inevitably impact the overall error probability of the algorithm. To quantify this impact or more generally to analyze the performance of adaptive algorithms, one would need to understand the connection between the statistical properties of the arm selection process and the asymptotic statistics of the estimated expected rewards.
To be more specific, any adaptive algorithm generates a stochastic process  $\{Z(t)\}_{t\geq 1} = \{(\omega (t),\hat{\mu} (t))\}_{t\geq 1}$ .  $\omega (t) = (\omega_1(t),\dots ,\omega_K(t))$  represents the allocation realized by the algorithm up to round  $t$ $(\omega_{k}(t) = N_{k}(t) / t$  and  $N_{k}(t)$  denotes the number of times arm  $k$  has been selected up to round  $t$ .  $\hat{\mu} (t) = (\hat{\mu}_1(t),\ldots ,\hat{\mu}_K(t))$  denotes the empirical average rewards of the various arms up to round  $t$ . Now assuming that at the end of round  $T$ , the algorithm returns the arm with the highest empirical reward, the error probability is  $\mathbb{P}_{\pmb{\mu}}[\hat{i}\neq 1(\pmb {\mu})] = \mathbb{P}_{\pmb{\mu}}[\hat{\mu} (T)\in \mathrm{Alt}(\pmb {\mu})]$ . Assessing the error probability at least asymptotically requires understanding the asymptotic behavior of  $\hat{\mu} (t)$  as  $t$  grows large. Ideally, one would wish to establish the Large Deviation properties of the process  $\{Z(t)\}_{t\geq 1}$ . This task is easy for algorithms using static allocations (Glynn and Juneja, 2004), but becomes challenging and open for adaptive algorithms. Addressing this challenge is the main objective of this paper.
Contributions. In this paper, we develop and leverage tools towards the analysis of adaptive sampling algorithms for the BAI-FB problem. More precisely, our contributions are as follows.
(a) We establish a connection between the LDP satisfied by the empirical proportions of arm draws  $\{\pmb {\omega}(t)\}_{t\geq 1}$  and that satisfied by the empirical arm rewards. This connection holds for any adaptive algorithm. Specifically, we show that if the rate function of  $\{\pmb {\omega}(t)\}_{t\geq 1}$  is lower bounded by  $\pmb {\omega}\mapsto I(\pmb {\omega})$  , then that of  $(\hat{\mu} (t))_{t\geq 1}$  is also lower bounded by  $\lambda \mapsto \min_{\pmb {\omega}\in \Sigma}\max \{\Psi (\pmb {\lambda},\pmb {\omega}),I(\pmb {\omega})\}$  . This result has interesting interpretations and implies the following asymptotic upper bound on the error probability of the algorithm considered: as  $T\to \infty$
$$
\frac {1}{T} \log \frac {1}{\mathbb {P} _ {\boldsymbol {\mu}} [ \hat {i} \neq 1 (\boldsymbol {\mu}) ]} \geq \inf  _ {\boldsymbol {\omega} \in \Sigma , \boldsymbol {\lambda} \in \operatorname {A l t} (\boldsymbol {\mu})} \max  \left\{\Psi (\boldsymbol {\lambda}, \boldsymbol {\omega}), I (\boldsymbol {\omega}) \right\}. \tag {2}
$$
The above formula, when compared to the lower bound (1), quantifies the price of not knowing  $\omega^{\star}$  initially, and relates the error probability to the asymptotic statistics of the sampling process used by the algorithm.
(b) We show that by simply applying our generic Large Deviation result, we may improve the error probability upper bounds of some existing algorithms, such as the celebrated SR algorithm (Audibert et al., 2010). Our result further opens up opportunities to devise and analyze new algorithms with a higher level of adaptiveness. In particular, we present CR (Continuous Rejects), an algorithm that, unlike SR, can eliminate arms in each round. This sequential elimination process is performed by comparing the empirical rewards of the various candidate arms using continuously updated thresholds. Leveraging the LDP tools developed in (a), we establish that CR enjoys better performance guarantees than SR. Hence CR becomes the algorithm with the lowest instance-specific and guaranteed error probability. We illustrate our results via numerical experiments, and compare CR to other BAI algorithms.
We distinguish two main classes of algorithms to solve the best arm identification problem in the fixed budget setting. Algorithms from the first class, e.g. Successive Rejects (SR) (Audibert et al., 2010) and Sequential Halving (SH) (Karnin et al., 2013), split the sampling budget into phases of fixed durations, and discard arms at the end of each phase. Algorithms from the second class, e.g. UCB-E (Audibert et al., 2010) and UGapE (Gabillon et al., 2012) sequentially sample arms based on confidence bounds of their empirical rewards. It is worth mentioning that algorithms from the second class usually require some prior knowledge about the problem, for example, an upper bound of  $H = \sum_{k\neq 1}(\mu)\frac{1}{(\mu_1(\mu) - \mu_k)^2}$ . Without this knowledge, the parameters can be chosen in a heuristic way, but the performance gets worse.
Algorithms from the first class exhibit better performance numerically and are also those with the best instance-specific error probability guarantees. SR had actually the best performance guarantees so far: for example, when the reward distributions are supported on  $[0,1]$ , the error probability of SR satisfies:  $\lim \inf_{T\to \infty}\frac{1}{T}\log \frac{1}{\mathbb{P}_{\mu}[i\neq 1(\mu)]}\geq \frac{1}{H_2\log K}$ , where  $H_{2} = \max_{k\neq 1(\mu)}\frac{k}{(\mu_{1}(\mu) - \mu_{k})^{2}}$ . In this paper, we strictly improve this guarantee (see Section 3.4). Recently, (Barrier et al., 2022) also refined and extended the analysis of (Audibert et al., 2010) by replacing, in the analysis, Hoeffding's inequality by a large deviation result involving KL-divergences. Again here, we further improve this new guarantee. We also devise CR, an algorithm with error probability provably lower than our improved guarantees for SR.
Fundamental limits on the error probability have also been investigated. In the minimax setting, (Carpentier and Locatelli, 2016) established that for any algorithm, there exists a problem within the class of instances with given complexity  $H$  such that the error probability is greater than  $\exp \left( -\frac{400T}{H\log K} \right) \geq \exp \left( -\frac{400T}{H_2\log K} \right)$ . Up to a universal constant (here 400), SR is hence minimax optimal. This lower bound was also recently revisited in (Ariu et al., 2021; Degenne, 2023; Wang et al., 2023) to prove that the instance-specific lower bound (1) cannot be achieved on all instances by a single algorithm. Deriving tight instance-specific lower bounds remains open (Qin, 2022).
We conclude this section by mentioning two interesting algorithms. In (Komiyama et al., 2022), the authors propose DOT, an algorithm trying to match minimax error probability lower bounds. To this aim, the algorithm requires to periodically call an oracle able to determine an optimal allocation, solution of an optimization problem with high and unknown complexity. DOT has minimax guarantees but is computationally challenging if not infeasible (numerically, the authors cannot go beyond simple instances with 3 arms). Finally, researchers have also looked at the best arm identification problem from a Bayesian perspective. For example, (Russo, 2016) devise variants of the celebrated Thompson Sampling algorithm, that could potentially work well in practice. Nevertheless, as discussed in (Komiyama, 2022), Bayesian algorithms cannot be analyzed nor provably perform well in the frequentist setting.
3 Large Deviation Analysis of Adaptive Sampling Algorithms
In this section, we first recall key concepts in Large Deviations (refer to the classical textbooks (Budhiraja and Dupuis, 2019; Dembo and Zeitouni, 2009; Dupuis and Ellis, 2011; Varadhan, 2016) for a more detailed exposition). We then apply these concepts to the performance analysis of adaptive sampling algorithms. Finally, we exemplify the analysis and apply it to improve existing performance guarantees for the SR algorithm (Audibert et al., 2010).
3.1 Large Deviation Principles
Consider the stochastic process  $\{Y(t)\}_{t\geq 1}$  with values in a separable complete metric space (i.e., a Polish space)  $\mathcal{V}$ . Large Deviations are concerned with the probabilities of rare events related to  $\{Y(t)\}_{t\geq 1}$  that decay exponentially in the parameter  $t$ . The asymptotic decay rate is characterized by the rate function  $I:\mathcal{V}\to \mathbb{R}_+$  defined so that essentially  $-\frac{1}{t}\log \mathbb{P}[Y(t)\in B]$  converges to  $\min_{x\in B}I(x)$  for any Borel set  $B$ . We provide a more rigorous definition below.
Definition 1. [Large Deviation Principle (LDP)] The stochastic process  $\{Y(t)\}_{t\geq 1}$  satisfies a LDP with rate function  $I$  if:
(i)  $I$  is lower semicontinuous, and  $\forall s\in [0,\infty ]$  , the set  $\mathcal{K}_s = \{y\in \mathcal{V}:I(y)\leq s\}$  is compact; (ii) for every closed (resp. open) set  $C\subset \mathcal{V}$  (resp.  $O\subset \mathcal{V}$
$$
\lim  _ {t \rightarrow \infty} \frac {1}{t} \log \frac {1}{\mathbb {P} [ Y (t) \in C ]} \geq \inf  _ {y \in C} I (y), \tag {3}
$$
$$
\varlimsup_ {t \rightarrow \infty} \frac {1}{t} \log \frac {1}{\mathbb {P} [ Y (t) \in O ]} \leq \inf  _ {y \in O} I (y). \tag {4}
$$
LDPs have been derived earlier in stochastic bandit literature. (Glynn and Juneja, 2004) have used Gartner-Ellis Theorem (Ellis, 1984; Gartner, 1977) to establish that under a static sampling algorithm with allocation  $\omega \in \Sigma$  (i.e., each arm  $k$  is selected  $\omega_{k}T$  times up to round  $T$ ), the process  $\{\hat{\mu}(T)\}_{T \geq 1}$  satisfies an LDP with rate function  $\lambda \mapsto \Psi(\lambda, \omega) = \sum_{k=1}^{K} \omega_{k} d(\lambda_{k}, \mu_{k})$ . Our objective in the next subsection is to investigate how to extend this result to the case of adaptive sampling algorithms.
3.2 Analysis of adaptive sampling algorithms
An adaptive sampling algorithm generates a stochastic process  $\{Z(t)\}_{t\geq 1} = \{(\boldsymbol {\omega}(t),\hat{\boldsymbol{\mu}} (t))\}_{t\geq 1}$ . When the sampling budget  $T$  is exhausted, should the algorithm returns the arm with the highest empirical reward, the error probability is  $\mathbb{P}_{\pmb{\mu}}[i\neq 1(\pmb {\mu})] = \mathbb{P}_{\pmb{\mu}}[\hat{\pmb{\mu}} (T)\in \mathrm{Alt}(\pmb {\mu})]$ . To assess the rate at which this probability decays with the budget, we may try to establish a LDP for the empirical reward process  $\{\hat{\pmb{\mu}} (t)\}_{t\geq 1}$ . Due to the intricate dependence between the sampling and the empirical reward processes, deriving such an LDP is very challenging. Instead, we establish a connection between the LDPs satisfied by these processes. This connection will be enough for us to derive tight upper bounds on the error probability. We present our main result in the following theorem.
Theorem 1. Assume that under some adaptive sampling algorithm,  $\{\omega(t)\}_{t\geq 1}$  satisfies the LDP upper bound (3) with rate function  $I$ . Then  $\{\hat{\mu}(t)\}_{t\geq 1}$  satisfies the LDP upper bound (3) with rate function  $\lambda \mapsto \min_{\boldsymbol{\omega} \in \Sigma} \max \{\Psi(\boldsymbol{\lambda},\boldsymbol{\omega}), I(\boldsymbol{\omega})\}$ . Moreover, we have: for any bounded Borel subset  $\mathcal{S}$  of  $\mathbb{R}^K$  and any Borel subset  $W$  of  $\Sigma$ ,
$$
\lim  _ {t \rightarrow \infty} \frac {1}{t} \log \frac {1}{\mathbb {P} _ {\boldsymbol {\mu}} [ \hat {\boldsymbol {\mu}} (t) \in \mathcal {S} , \boldsymbol {\omega} (t) \in W ]} \geq \inf  _ {\boldsymbol {\omega} \in \operatorname {c l} (W)} \max  \left\{F _ {\mathcal {S}} (\boldsymbol {\omega}), I (\boldsymbol {\omega}) \right\},
$$
where  $F_{\mathcal{S}}(\omega)\coloneqq \inf_{\boldsymbol {\lambda}\in \operatorname {cl}(\mathcal{S})}\Psi (\boldsymbol {\lambda},\omega)$  , and  $\operatorname {cl}(S)$  denotes the closure of  $\mathcal{S}$
Before proving the above theorem, we make the following remarks and provide a simple corollary that will lead to improved upper bound on the error probability of the SR algorithm.
(a) Not a complete LDP. To upper bound the error probability of a given algorithm, we do not actually need to establish that  $\{\hat{\mu}(t)\}_{t\geq 1}$  satisfies a complete LDP. Instead, deriving a LDP upper bound is enough. Theorem 1 provides such an upper bound, but does not yield a complete LDP. We conjecture if  $\{\omega(t)\}_{t\geq 1}$  satisfies an LDP with rate function  $I$ ,  $\{\hat{\mu}(t)\}_{t\geq 1}$  satisfies an LDP with rate function  $\lambda \mapsto \inf_{\omega \in W} \max \{\Psi(\lambda, \omega), I(\omega)\}$ . The conjecture holds for static sampling algorithms as shown
below. If it holds for adaptive algorithms, we show, in Appendix I, that when  $K > 3$ , no algorithm can attain the instance-specific lower bound (1) for all parameters.
(b) Theorem 1 is tight for static sampling algorithms. When the sampling rule is static, namely  $\omega(t) = \omega \in \Sigma$ , then  $\{\omega(t)\}_{t \geq 1}$  satisfies a LDP with rate function  $I$  defined as  $I(\omega) = 0$  and  $\infty$  elsewhere. Theorem 1 with  $\bar{W} = \Sigma$  states that  $\{\hat{\mu}(t)\}_{t \geq 1}$  satisfies the LDP upper bound (3) with rate function  $F_{S}$ . In fact, as shown by (Glynn and Juneja, 2004),  $\{\hat{\mu}(t)\}_{t \geq 1}$  satisfies a complete LDP with this rate function.
(c) A useful corollary. From Theorem 1, we have:
$$
\lim  _ {t \rightarrow \infty} \frac {1}{t} \log \frac {1}{\mathbb {P} _ {\boldsymbol {\mu}} [ \hat {\boldsymbol {\mu}} (t) \in \mathcal {S} , \boldsymbol {\omega} (t) \in W ]} \geq \inf  _ {\boldsymbol {\omega} \in \operatorname {c l} (W)} F _ {\mathcal {S}} (\boldsymbol {\omega}).
$$
From there, we will be able to improve the performance guarantee for SR.
3.3 Proof of Theorem 1
Proof. Observe that when  $S$  or  $W$  is empty, the result holds. Now recall that  $F_{S}(\cdot) = \inf_{\lambda \in \operatorname{cl}(S)} \Psi(\lambda, \cdot)$  is the infimum of a family of linear functions on a compact set,  $\Sigma$ , hence it is upper bounded. Denote  $u > 0$  such an upper bound.  $F_{S}(\cdot)$  is also continuous (see Appendix F for details). For each integer  $N \in \mathbb{N}$ , we define a collection of closed sets:
$$
W _ {n} ^ {N} = \left\{\boldsymbol {\omega} \in \operatorname {c l} (W): \frac {u (n - 1)}{N} \leq F _ {\mathcal {S}} (\boldsymbol {\omega}) \leq \frac {u n}{N} \right\}, \quad \forall n \in [ N ]. \tag {5}
$$
We observe that:
$$
\begin{array}{l} \mathbb {P} _ {\boldsymbol {\mu}} [ \hat {\boldsymbol {\mu}} (t) \in \mathcal {S}, \boldsymbol {\omega} (t) \in W ] \leq \sum_ {n = 1} ^ {N} \mathbb {P} _ {\boldsymbol {\mu}} [ \hat {\boldsymbol {\mu}} (t) \in \mathcal {S}, \boldsymbol {\omega} (t) \in W _ {n} ^ {N} ] \\ \leq N \max _ {n \in [ N ]} \mathbb {P} _ {\boldsymbol {\mu}} [ \hat {\boldsymbol {\mu}} (t) \in \mathcal {S}, \boldsymbol {\omega} (t) \in W _ {n} ^ {N} ]. \\ \end{array}
$$
Taking the logarithm on both sides and dividing them by  $-t$  yields that
$$
\begin{array}{l} \lim  _ {t \rightarrow \infty} \frac {1}{t} \log \frac {1}{\mathbb {P} _ {\boldsymbol {\mu}} [ \hat {\boldsymbol {\mu}} (t) \in \mathcal {S} , \boldsymbol {\omega} (t) \in W ]} \geq \lim  _ {t \rightarrow \infty} \min  _ {n \in [ N ]} \frac {1}{t} \log \frac {1}{\mathbb {P} _ {\boldsymbol {\mu}} [ \hat {\boldsymbol {\mu}} (t) \in \mathcal {S} , \boldsymbol {\omega} (t) \in W _ {n} ^ {N} ]} \\ = \min  _ {n \in [ N ]} \lim  _ {t \rightarrow \infty} \frac {1}{t} \log \frac {1}{\mathbb {P} _ {\boldsymbol {\mu}} \left[ \hat {\boldsymbol {\mu}} (t) \in \mathcal {S} , \boldsymbol {\omega} (t) \in W _ {n} ^ {N} \right]} \\ \geq \min  _ {n \in [ N ]} \max  \left\{\frac {u (n - 1)}{N}, \inf  _ {\boldsymbol {\omega} \in W _ {n} ^ {N}} I (\boldsymbol {\omega}) \right\}, \tag {6} \\ \end{array}
$$
where the last inequality follows from Lemma 1. Since for all  $n \in [N]$ ,
$$
\max \left\{\frac {u (n - 1)}{N}, \inf _ {\boldsymbol {\omega} \in W _ {n} ^ {N}} I (\boldsymbol {\omega}) \right\} = \inf _ {\boldsymbol {\omega} \in W _ {n} ^ {N}} \max \left\{\frac {u (n - 1)}{N}, I (\boldsymbol {\omega}) \right\},
$$
the r.h.s. of (6) is equal to
$$
\begin{array}{l} \min  _ {n \in [ N ]} \inf  _ {\boldsymbol {\omega} \in W _ {n} ^ {N}} \max  \left\{\frac {u (n - 1)}{N}, I (\boldsymbol {\omega}) \right\} \geq \min  _ {n \in [ N ]} \inf  _ {\boldsymbol {\omega} \in W _ {n} ^ {N}} \max  \left\{F _ {\mathcal {S}} (\boldsymbol {\omega}), I (\boldsymbol {\omega}) \right\} - u / N \\ = \inf  _ {\boldsymbol {\omega} \in \operatorname {c l} (W)} \max  \left\{F _ {S} (\boldsymbol {\omega}), I (\boldsymbol {\omega}) \right\} - u / N, \\ \end{array}
$$
where the first inequality is due to (5). As  $N$  can be taken arbitrarily large, we conclude this theorem.
Lemma 1. For any  $N\in \mathbb{N}$ $n\in [N]$
$$
\lim  _ {t \rightarrow \infty} \frac {1}{t} \log \frac {1}{\mathbb {P} _ {\boldsymbol {\mu}} [ \hat {\boldsymbol {\mu}} (t) \in \mathcal {S} , \boldsymbol {\omega} (t) \in W _ {n} ^ {N} ]} \geq \max  \left\{\frac {u (n - 1)}{N}, \inf  _ {\boldsymbol {\omega} \in W _ {n} ^ {N}} I (\boldsymbol {\omega}) \right\}
$$
Proof. Recall  $F_{\mathcal{S}}(\cdot) = \inf_{\boldsymbol{\lambda} \in \operatorname{cl}(\mathcal{S})} \Psi(\boldsymbol{\lambda}, \cdot)$ . We deduce that
$$
\mathbb {P} _ {\boldsymbol {\mu}} \left[ \hat {\boldsymbol {\mu}} (t) \in \mathcal {S}, \boldsymbol {\omega} (t) \in W _ {n} ^ {N} \right] \leq \mathbb {P} _ {\boldsymbol {\mu}} \left[ X \geq t F _ {S} (\boldsymbol {\omega} (t)), \boldsymbol {\omega} (t) \in W _ {n} ^ {N} \right],
$$
where  $X$  denotes  $t\Psi (\hat{\mu} (t),\omega (t))$  for short. Let  $\alpha \in (0,1)$ , Markov's inequality implies that