SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathbb {P} \left\{E _ {1} \cap E _ {2} \right\} \geq 1 - \frac {4 \log \mathsf {T}}{\mathsf {T}}.
$$
Proof. From Lemma 15 we have  $\mathbb{P}\{E_1\} \geq 1 - \frac{6}{T}$ . Furthermore, from Lemma 16 we have  $\mathbb{P}\{E_2\} \geq 1 - \frac{3\log T}{T}$ . Applying a union bound on the complements of these two events establishes the corollary.
Lemma 18. Consider any bandit instance with  $\langle x^{*},\theta^{*}\rangle \geq 192\sqrt{\frac{d\nu\log(T|\mathcal{X}|)}{T}}$ . If event  $E_{1}$  holds, then any arm with mean  $\langle x,\theta^{*}\rangle \leq 10\sqrt{\frac{d\nu\log(T|\mathcal{X}|)}{T}}$  is eliminated after Part I of Algorithm 2.
Proof. We will show that in the given bandit instance and under the event  $E_{1}$ , for each arm  $x \in \mathcal{X}$  with mean  $\langle x, \theta^{*} \rangle \leq 10\sqrt{\frac{d\nu\log(\mathsf{T}|\mathcal{X}|)}{\mathsf{T}}}$  the upper Nash confidence bound (see equation (7)) is less than the lower confidence bound of the optimal arm  $x^{*}$ . Hence, all such arms  $x$  are eliminated from consideration in Line 8 of Algorithm 2. This will establish the lemma.
The upper Nash confidence bound of arm  $x$  at the end of Part I is defined as
$$
\begin{array}{l} \mathrm {U N C B} \left(x, \widehat {\theta}, \widetilde {\mathsf {T}} / 3\right) = \langle x, \widehat {\theta} \rangle + 6 \sqrt {\frac {3 \langle x , \widehat {\theta} \rangle d \nu \log (\mathsf {T} | \mathcal {X} |)}{\widetilde {\mathsf {T}}}} \\ \leq 2 0 \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T}}} + 6 \sqrt {\frac {3 \langle x , \widehat {\theta} \rangle d \nu \log (\mathsf {T} | \mathcal {X} |)}{\widetilde {\mathsf {T}}}} \quad \text {(v i a e v e n t E _ {1})} \\ \leq 2 0 \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T}}} + 6 \sqrt {\frac {3 \cdot 2 0 \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T}}} d \nu \log (\mathsf {T} | \mathcal {X} |)}{3 \sqrt {\mathsf {T} \nu d \log (\mathsf {T} | \mathcal {X} |)}}} \quad (\text {s u b s t i t u t i n g} \widetilde {\mathsf {T}}) \\ \leq 4 7 \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T}}} \tag {27} \\ \end{array}
$$
In the given bandit instance and under event  $E_{1}$ , for the optimal arm  $x^{*}$ , we have
$$
\begin{array}{l} \langle x ^ {*}, \widehat {\theta} \rangle \leq \langle x ^ {*}, \theta^ {*} \rangle + 3 \sqrt {\frac {d \nu \langle x ^ {*} , \theta^ {*} \rangle \log (\mathrm {T} | \mathcal {X} |)}{\widetilde {\mathrm {T}}}} \\ = \langle x ^ {*}, \theta^ {*} \rangle \left(1 + 3 \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{\langle x ^ {*} , \theta^ {*} \rangle 3 \sqrt {\mathsf {T} d \nu \log (\mathsf {T} | \mathcal {X} |)}}}\right) \quad \text {(s u b s t i t u t i n g} \widetilde {\mathsf {T}}) \\ \leq \langle x ^ {*}, \theta^ {*} \rangle \left(1 + 3 \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{1 9 2 \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T}}} 3 \sqrt {\mathsf {T} \nu d \log (\mathsf {T} | \mathcal {X} |)}}}\right) \\ (\text {u s i n g} \langle x ^ {*}, \theta^ {*} \rangle \geq 1 9 2 \sqrt {\frac {d \nu \log (\mathrm {T} | \mathcal {X} |)}{\mathrm {T}}}) \\ = \frac {1 7}{1 6} \left\langle x ^ {*}, \theta^ {*} \right\rangle . \tag {28} \\ \end{array}
$$
Therefore, the lower Nash confidence bound of  $x^{*}$  satisfies
$$
\operatorname {L N C B} \left(x ^ {*}, \widehat {\theta}, \widetilde {\mathsf {T}} / 3\right) = \langle x ^ {*}, \widehat {\theta} \rangle - 6 \sqrt {\frac {3 \langle x ^ {*} , \widehat {\theta} \rangle d \nu \log (\mathsf {T} | \mathcal {X} |)}{\widetilde {\mathsf {T}}}}
$$
$$
\begin{array}{l} \geq \langle x ^ {*}, \theta^ {*} \rangle - 3 \sqrt {\frac {d \nu \langle x ^ {*} , \theta^ {*} \rangle \log (\mathsf {T} | \mathcal {X} |)}{\widetilde {\mathsf {T}}}} - 6 \sqrt {\frac {3 \langle x ^ {*} , \widehat {\theta} \rangle d \nu \log (\mathsf {T} | \mathcal {X} |)}{\widetilde {\mathsf {T}}}} \quad (\text {v i a (1 9) i n e v e n t} E _ {1}) \\ \geq \langle x ^ {*}, \theta^ {*} \rangle - \left(3 + 6 \sqrt {\frac {5 1}{1 6}}\right) \sqrt {\frac {d \nu \langle x ^ {*} , \theta^ {*} \rangle \log (\mathsf {T} | \mathcal {X} |)}{\widetilde {\mathsf {T}}}} \\ \left(\text {s i n c e} \langle x ^ {*}, \widehat {\theta} \rangle \leq \frac {1 7}{1 6} \langle x ^ {*}, \theta^ {*} \rangle \right. (28) \\ \geq \langle x ^ {*}, \theta^ {*} \rangle \left(1 - 1 4 \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{\langle x ^ {*} , \theta^ {*} \rangle \widetilde {\mathsf {T}}}}\right) \\ \geq \langle x ^ {*}, \theta^ {*} \rangle \left(1 - 1 4 \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{1 9 2 \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T}}} 3 \sqrt {\mathsf {T} d \nu \log (\mathsf {T} | \mathcal {X} |)}}}\right) \\ \geq \frac {5}{1 2} \langle x ^ {*}, \theta^ {*} \rangle \\ \geq 8 0 \sqrt {\frac {d \nu \log (\mathrm {T} | \mathcal {X} |)}{\mathrm {T}}} (29) \\ \end{array}
$$
Equations (29) and (27) imply
$$
\operatorname {U N C B} \left(x, \widehat {\theta}, \widetilde {\mathrm {T}} / 3\right) <   \operatorname {L N C B} \left(x ^ {*}, \widehat {\theta}, \widetilde {\mathrm {T}} / 3\right) \tag {30}
$$
As mentioned previously, Line 8 in Algorithm 2 eliminates all arms  $x$  that satisfy inequality (30). Hence, the lemma stands proved
C.2 Proofs of Lemmas 7 and 8
Proof. We will show that, under events  $E_{1}$  and  $E_{2}$ , throughout the execution of Algorithm 2 the UNCB of the optimal arm  $x^{*}$  is never less than the LNCB of any arm  $x$ . Hence, then the optimal arm  $x^{*}$  never satisfies the elimination criterion in Algorithm 2 and, hence,  $x^{*}$  always exists in the surviving set of arms.
First, we consider arms  $x$  with the property that  $\langle x, \theta^* \rangle < 10\sqrt{\frac{d\nu\log(T|\mathcal{X}|)}{T}}$ . For any such arm  $x$ , at the end of Part I of the algorithm we have
$$
\operatorname {L N C B} \left(x, \widehat {\theta}, \widetilde {\mathrm {T}} / 3\right) \leq \operatorname {U N C B} \left(x, \widehat {\theta}, \widetilde {\mathrm {T}} / 3\right) _ {\text {v i a (3 0)}} <   \operatorname {L N C B} \left(x ^ {*}, \widehat {\theta}, \widetilde {\mathrm {T}} / 3\right) \leq \operatorname {U N C B} \left(x ^ {*}, \widehat {\theta}, \widetilde {\mathrm {T}} / 3\right).
$$
Hence, at the end of Part I, arm  $x^{*}$  is not eliminated via the LNCB of any  $x$  which satisfies  $\langle x, \theta^{*} \rangle < 10\sqrt{\frac{d\nu\log(\mathsf{T}|\mathcal{X}|)}{\mathsf{T}}}$ . Further, note that, under event  $E_{1}$ , such arms are eliminated at the end of Part I (Lemma 18). Hence, the LNCB of such arms are not even considered in the phases of Part II. To complete the proof, we next show that the UNCB of the optimal arm  $x^{*}$  is at least the LNCB of all arms  $x$  which bear  $\langle x, \theta^{*} \rangle \geq 10\sqrt{\frac{d\nu\log(\mathsf{T}|\mathcal{X}|)}{\mathsf{T}}}$ . Below, we will consider the Nash confidence bounds for a general  $\mathsf{T}'$ . Replacing  $\mathsf{T}'$  by  $\widetilde{\mathsf{T}}$  gives us the desired confidence-bounds comparison for the end of Part I – this repetition is omitted.
Under events  $E_{1}$  and  $E_{2}$ , for any arm  $x$  with  $\langle x,\theta^{*}\rangle \geq 10\sqrt{\frac{d\nu\log(\mathsf{T}|\mathcal{X}|)}{\mathsf{T}}}$ , it holds that
$$
\begin{array}{l} \operatorname {L N C B} (x, \widehat {\theta}, \mathsf {T} ^ {\prime}) = \langle x, \widehat {\theta} \rangle - 6 \sqrt {\frac {\langle x , \widehat {\theta} \rangle d \nu \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T} ^ {\prime}}} \\ \leq \langle x, \theta^ {*} \rangle + 3 \sqrt {\frac {d \nu \langle x , \theta^ {*} \rangle \log (\mathrm {T} | \mathcal {X} |)}{\mathrm {T} ^ {\prime}}} - 6 \sqrt {\frac {\langle x , \widehat {\theta} \rangle d \nu \log (\mathrm {T} | \mathcal {X} |)}{\mathrm {T} ^ {\prime}}} \quad (\text {v i a (2 1)}) \\ \leq \langle x, \theta^ {*} \rangle - \left(\frac {6}{\sqrt {2}} - 3\right) \sqrt {\frac {d \nu \langle x , \theta^ {*} \rangle \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T} ^ {\prime}}} \quad (\langle x, \widehat {\theta} \rangle \geq \frac {1}{2} \langle x, \theta^ {*} \rangle \text {v i a (2 2)}) \\ \leq \langle x, \theta^ {*} \rangle . \tag {31} \\ \end{array}
$$
Complementarily, for optimal arm  $x^{*}$  we have
$$
\begin{array}{l} \mathrm {U N C B} (x ^ {*}, \widehat {\theta}, \mathsf {T} ^ {\prime}) = \langle x ^ {*}, \widehat {\theta} \rangle + 6 \sqrt {\frac {\langle x ^ {*} , \widehat {\theta} \rangle d \nu \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T} ^ {\prime}}} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle - 3 \sqrt {\frac {d \nu \langle x ^ {*} , \theta^ {*} \rangle \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T} ^ {\prime}}} + 6 \sqrt {\frac {\langle x ^ {*} , \widehat {\theta} \rangle d \nu \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T} ^ {\prime}}} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle + \left(\frac {6}{\sqrt {2}} - 3\right) \sqrt {\frac {d \nu \langle x ^ {*} , \theta^ {*} \rangle \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T} ^ {\prime}}} \quad (\text {s i n c e} \langle x ^ {*}, \widehat {\theta} \rangle \geq \frac {\langle x ^ {*} , \theta^ {*} \rangle}{2}) \\ \geq \langle x ^ {*}, \theta^ {*} \rangle \tag {32} \\ \end{array}
$$
Since  $\langle x^{*},\theta^{*}\rangle \geq \langle x,\theta^{*}\rangle$  for all arms  $x$ , inequalities (31) and (32) lead to the confidence-bounds comparison:
$$
\mathrm {U N C B} \left(x ^ {*}, \widehat {\theta}, \mathrm {T} ^ {\prime}\right) \geq \mathrm {L N C B} \left(x, \widehat {\theta}, \mathrm {T} ^ {\prime}\right).
$$
Hence, if events  $E_{1}$  and  $E_{2}$  hold, then the optimal arm  $x^{*}$  is never eliminated from Algorithm 2. Further, Corollary 17 ensures that the events  $E_{1}$  and  $E_{2}$  hold with probability at least  $1 - \frac{4\log T}{T}$ . Hence, the lemma stands proved.
Proof. For the analysis, assume that events  $E_{1}$  and  $E_{2}$  hold. Lemma 7 ensures that the optimal arm is contained in the surviving set of arms  $\widetilde{\mathcal{X}}$ . Furthermore, if an arm  $x \in \widetilde{\mathcal{X}}$  at the beginning of the  $\ell^{\mathrm{th}}$  phase, then it must be the case that arm  $x$  was not eliminated in the previous phase (which executed for  $T'/2$  rounds); in particular, we have  $\mathrm{UNCB}(x, \widehat{\theta}, T'/2) \geq \mathrm{LNCB}(x^{*}, \widehat{\theta}, T'/2)$ . This inequality reduces to
$$
\langle x, \widehat {\theta} \rangle + 6 \sqrt {\frac {\langle x , \widehat {\theta} \rangle d \nu \log (\mathsf {T} | \mathcal {X} |)}{\frac {\mathsf {T} ^ {\prime}}{2}}} \geq \langle x ^ {*}, \widehat {\theta} \rangle - 6 \sqrt {\frac {\langle x ^ {*} , \widehat {\theta} \rangle d \nu \log (\mathsf {T} | \mathcal {X} |)}{\frac {\mathsf {T} ^ {\prime}}{2}}}.
$$
Rearranging the terms, we obtain
$$
\begin{array}{l} \langle x, \widehat {\theta} \rangle \geq \langle x ^ {*}, \widehat {\theta} \rangle - 6 \sqrt {\frac {\langle x ^ {*} , \widehat {\theta} \rangle d \nu \log (\mathrm {T} | \mathcal {X} |)}{\frac {\mathrm {T} ^ {\prime}}{2}}} - 6 \sqrt {\frac {\langle x , \widehat {\theta} \rangle d \nu \log (\mathrm {T} | \mathcal {X} |)}{\frac {\mathrm {T} ^ {\prime}}{2}}} \\ \geq \langle x ^ {*}, \widehat {\theta} \rangle - 6 \sqrt {\frac {4 \langle x ^ {*} , \theta^ {*} \rangle d \nu \log (\mathsf {T} | \mathcal {X} |)}{3 \mathsf {T} ^ {\prime} / 2}} - 6 \sqrt {\frac {4 \langle x , \theta^ {*} \rangle d \nu \log (\mathsf {T} | \mathcal {X} |)}{3 \mathsf {T} ^ {\prime} / 2}} \\ \left(\langle x, \widehat {\theta} \rangle \leq \frac {4}{3} \langle x, \theta^ {*} \rangle \text {v i a} (2 2)\right) \\ \geq \langle x ^ {*}, \widehat {\theta} \rangle - 2 0 \sqrt {\frac {\langle x ^ {*} , \theta^ {*} \rangle d \nu \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T} ^ {\prime}}}. \\ \end{array}
$$
Further, invoking equation (21) for  $x^{*}$  leads to
$$
\begin{array}{l} \langle x, \theta^ {*} \rangle \geq \langle x ^ {*}, \theta^ {*} \rangle - 2 0 \sqrt {\frac {\langle x ^ {*} , \theta^ {*} \rangle d \nu \log (T | \mathcal {X} |)}{T ^ {\prime}}} - 3 \sqrt {\frac {\langle x ^ {*} , \theta^ {*} \rangle d \nu \log (T | \mathcal {X} |)}{\frac {T ^ {\prime}}{2}}} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle - 2 5 \sqrt {\frac {\langle x ^ {*} , \theta^ {*} \rangle d \nu \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T} ^ {\prime}}}. \\ \end{array}
$$
Substituting  $\mathsf{T}' = 2^{\ell}\widetilde{\mathsf{T}} /3$ , the above inequality reduces to the desired bound in (10). From Corollary 17, we have that the events  $E_{1}$  and  $E_{2}$  hold with probability at least  $1 - \frac{4\log\mathsf{T}}{\mathsf{T}}$ . Hence, the lemma stands proved.
Proof. We will assume, without loss of generality, that  $\langle x^{*},\theta^{*}\rangle \geq 192\sqrt{\frac{d\nu}{T}\log(T|\mathcal{X}|)}$ , otherwise the stated Nash Regret bound directly holds (see equation (1)). Write  $E$  to denote the 'good' event identified in Lemma 8; the lemma ensures that  $\mathbb{P}\{E\} \geq 1 - \frac{4\log T}{T}$ .
During Part I of Algorithm 2, the product of expected rewards, conditioned on  $E$ , satisfies
$$
\begin{array}{l} \prod_ {t = 1} ^ {\tilde {\tau}} \mathbb {E} [ \langle X _ {t}, \theta^ {*} \rangle \mid E ] ^ {\frac {1}{\top}} \geq \left(\frac {\langle x ^ {*} , \theta^ {*} \rangle}{2 (d + 1)}\right) ^ {\frac {\tilde {\tau}}{\top}} \tag {via Lemma 5} \\ = \left\langle x ^ {*}, \theta^ {*} \right\rangle^ {\frac {\widetilde {\tau}}{\mathsf {T}}} \left(1 - \frac {1}{2}\right) ^ {\frac {\log (2 (d + 1)) ^ {\widetilde {\tau}}}{\mathsf {T}}} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle^ {\frac {\widetilde {\tau}}{\mathsf {T}}} \left(1 - \frac {\log (2 (d + 1)) \widetilde {\mathsf {T}}}{\mathsf {T}}\right). \\ \end{array}
$$
For analyzing Part II, we will utilize Lemma 8. Write  $\mathcal{B}_{\ell}$  to denote all the rounds  $t$  that belong to  $\ell^{\mathrm{th}}$  phase (in Part II). Also, let  $\mathsf{T}_{\ell}^{\prime}$  denote the associated phase-length parameter, i.e.,  $\mathsf{T}_{\ell}^{\prime} = 2^{\ell}\widetilde{\mathsf{T}} /3$ . Note that in each phase  $\ell$  (i.e., in the for-loop at Line 12 of Algorithm 2), every arm  $a$  in  $\operatorname{Supp}(\lambda)$  (the support of D-optimal design) is pulled  $\lceil \lambda_a\mathsf{T}_{\ell}^\prime \rceil$  times. Given that  $|\operatorname {Supp}(\lambda)|\leq d(d + 1) / 2$ , we have  $|\mathcal{B}_{\ell}|\leq \mathsf{T}_{\ell}^{\prime} + \frac{d(d + 1)}{2}$ . By construction  $\mathsf{T}_{\ell}^{\prime}\geq \frac{d(d + 1)}{2}$  and, hence,  $|\mathcal{B}_{\ell}|\leq 2\mathsf{T}_{\ell}^{\prime}$ . Since the phase length parameter,  $\mathsf{T}_{\ell}^{\prime}$ , doubles after each phase, the algorithm would have at most log T phases. Hence, the product of expected rewards in Part II satisfies
$$
\begin{array}{l} \prod_ {t = \widetilde {\mathbb {T}} + 1} ^ {\mathbb {T}} \mathbb {E} [ \langle X _ {t}, \theta^ {*} \rangle \mid E ] ^ {\frac {1}{\mathbb {T}}} = \prod_ {\mathcal {B} _ {\ell}} \prod_ {t \in \mathcal {B} _ {\ell}} \mathbb {E} [ \langle X _ {t}, \theta^ {*} \rangle \mid E ] ^ {\frac {1}{\mathbb {T}}} \\ \geq \prod_ {\mathcal {B} _ {\ell}} \left(\langle x ^ {*}, \theta^ {*} \rangle - 2 5 \sqrt {\frac {d \nu \langle x ^ {*} , \theta^ {*} \rangle \log (\mathrm {T} | \mathcal {X} |)}{\mathrm {T} _ {\ell} ^ {\prime}}}\right) ^ {\frac {| \mathcal {B} _ {\ell} |}{\mathrm {T}}} \tag {Lemma 8} \\ \geq \left\langle x ^ {*}, \theta^ {*} \right\rangle^ {\frac {\tau - \tilde {\tau}}{\tau}} \prod_ {\ell = 1} ^ {\log \tau} \left(1 - 2 5 \sqrt {\frac {d \nu \log (\Upsilon | \mathcal {X} |)}{\langle x ^ {*} , \theta^ {*} \rangle \Upsilon_ {\ell} ^ {\prime}}}\right) ^ {\frac {| \mathcal {B} _ {\ell} |}{\Upsilon}} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle^ {\frac {\tau - \widetilde {\tau}}{\tau}} \prod_ {\ell = 1} ^ {\log \mathsf {T}} \left(1 - 5 0 \frac {| \mathcal {B} _ {\ell} |}{\mathsf {T}} \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{\langle x ^ {*} , \theta^ {*} \rangle \mathsf {T} _ {\ell} ^ {\prime}}}\right). \\ \end{array}
$$
The last inequality follows from the fact that  $(1 - x)^r \geq (1 - 2rx)$ , for any  $r \in [0,1]$  and  $x \in [0,1/2]$ . Note that the term  $\sqrt{\frac{d\nu\log(\mathsf{T}|\mathcal{X}|)}{\langle x^*,\theta^*\rangle\mathsf{T}_{\ell}^{\prime}}} \leq 1/2$ , since  $\langle x^*,\theta^*\rangle \geq 192\sqrt{\frac{d\nu}{\mathsf{T}}}\log (\mathsf{T}|\mathcal{X}|)$  along with  $\mathsf{T}_{\ell}' \geq 2\sqrt{\mathsf{T}d\nu\log\mathsf{T}|\mathcal{X}|}$  and  $\mathsf{T} \geq e^4$ . We further simplify the expression as follows
$$
\prod_ {\ell = 1} ^ {\log \mathsf {T}} \left(1 - 5 0 \frac {| \mathcal {B} _ {\ell} |}{\mathsf {T}} \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{\langle x ^ {*} , \theta^ {*} \rangle \mathsf {T} _ {\ell} ^ {\prime}}}\right) \geq \prod_ {\ell = 1} ^ {\log \mathsf {T}} \left(1 - 1 0 0 \frac {\sqrt {\mathsf {T} _ {\ell} ^ {\prime}}}{\mathsf {T}} \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{\langle x ^ {*} , \theta^ {*} \rangle}}\right)
$$
$$
(s i n c e \left| \mathcal {B} _ {\ell} \right| \leq 2 \mathrm {T} _ {\ell} ^ {\prime})
$$
$$
\geq 1 - \frac {1 0 0}{T} \sqrt {\frac {d \nu \log (T | \mathcal {X} |)}{\langle x ^ {*} , \theta^ {*} \rangle}} \left(\sum_ {\ell = 1} ^ {\log T} \sqrt {T _ {\ell} ^ {\prime}}\right)
$$
$$
(s i n c e (1 - a) (1 - b) \geq 1 - a - b f o r a, b \geq 0)
$$
$$
\geq 1 - \frac {1 0 0}{T} \sqrt {\frac {d \nu \log (T | \mathcal {X} |)}{\langle x ^ {*} , \theta^ {*} \rangle}} \left(\sqrt {T \log T}\right)
$$
(via Cauchy-Schwarz inequality)
$$
\geq 1 - 1 0 0 \sqrt {\frac {d \nu}{\mathsf {T} \langle x ^ {*} , \theta^ {*} \rangle}} \log (\mathsf {T} | \mathcal {X} |).
$$
Combining the lower bound for the expected rewards in the two parts we get
$$
\begin{array}{l} \prod_ {t = 1} ^ {\mathsf {T}} \mathbb {E} [ \langle X _ {t}, \theta^ {*} \rangle ] ^ {\frac {1}{\mathsf {T}}} \geq \prod_ {t = 1} ^ {\mathsf {T}} \left(\mathbb {E} [ \langle X _ {t}, \theta^ {*} \rangle \mid E ] \mathbb {P} \{E \}\right) ^ {\frac {1}{\mathsf {T}}} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle \left(1 - \frac {\log (2 (d + 1)) \widetilde {\mathsf {T}}}{\mathsf {T}}\right) \left(1 - 1 0 0 \sqrt {\frac {d \nu}{\mathsf {T} \langle x ^ {*} , \theta^ {*} \rangle}} \log (\mathsf {T} | \mathcal {X} |\right) \mathbb {P} \{E \} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle \left(1 - \frac {\log (2 (d + 1)) \widetilde {\mathsf {T}}}{\mathsf {T}} - 1 0 0 \sqrt {\frac {d \nu}{\mathsf {T} \langle x ^ {*} , \theta^ {*} \rangle}} \log (\mathsf {T} | \mathcal {X} |\right) \mathbb {P} \{E \} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle \left(1 - \frac {\log (2 (d + 1)) \widetilde {T}}{T} - 1 0 0 \sqrt {\frac {d \nu}{T \langle x ^ {*} , \theta^ {*} \rangle}} \log (T | \mathcal {X} |)\right) \left(1 - \frac {4 \log T}{T}\right) \\ \geq \langle x ^ {*}, \theta^ {*} \rangle \left(1 - \frac {\log (2 (d + 1)) 3 \sqrt {\mathsf {T} d \nu \log (\mathsf {T} | \mathcal {X} |)}}{\mathsf {T}} - 1 0 0 \sqrt {\frac {d \nu}{\mathsf {T} \langle x ^ {*} , \theta^ {*} \rangle}} \log (\mathsf {T} | \mathcal {X} |) - \frac {4 \log \mathsf {T}}{\mathsf {T}}\right) \\ \geq \langle x ^ {*}, \theta^ {*} \rangle - 1 0 0 \sqrt {\frac {\langle x ^ {*} , \theta^ {*} \rangle d \nu}{T}} \log (\mathsf {T} | \mathcal {X} |) - 6 \langle x ^ {*}, \theta^ {*} \rangle \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{T}} \log (2 (d + 1)). \\ \end{array}
$$
Therefore, the Nash Regret can be bounded as
$$
\begin{array}{l} \mathrm {N R} _ {\mathsf {T}} = \left\langle x ^ {*}, \theta^ {*} \right\rangle - \left(\prod_ {t = 1} ^ {\mathsf {T}} \mathbb {E} [ \left\langle X _ {t}, \theta^ {*} \right\rangle ]\right) ^ {1 / \mathsf {T}} \\ \leq 1 0 0 \sqrt {\frac {\langle x ^ {*} , \theta^ {*} \rangle d \nu}{T}} \log (T | \mathcal {X} |) + 6 \sqrt {\frac {d \nu \log (T | \mathcal {X} |)}{T}} \log (2 (d + 1)) \langle x ^ {*}, \theta^ {*} \rangle (33) \\ \leq \left(1 0 0 \sqrt {\langle x ^ {*} , \theta^ {*} \rangle} + 6 \log (2 (d + 1)) \langle x ^ {*}, \theta^ {*} \rangle\right) \sqrt {\frac {d \nu}{T}} \log (T | \mathcal {X} |) (34) \\ \end{array}
$$
Hence, with  $\beta = \max \left\{1,\sqrt{\langle x^{*},\theta^{*}\rangle},\langle x^{*},\theta^{*}\rangle \log d\right\} = \max \left\{1,\langle x^{*},\theta^{*}\rangle \log d\right\}$ , from equation (34) we obtain the desired bound on Nash regret  $\mathrm{NR}_{\mathsf{T}} = O\left(\beta \sqrt{\frac{d\nu}{\mathsf{T}}}\log (\mathsf{T}|\mathcal{X}|)\right)$ . The theorem stands proved.
D Algorithm 3 and Regret Analysis
Algorithm 3 LINNASH (Nash Confidence Bound Algorithm for Infinite Set of Arms)
Input: Arm set  $\mathcal{X}$  and horizon of play  $T$ .  
1: Initialize matrix  $\mathbf{V} \gets [0]_{d,d}$  and number of rounds  $\widetilde{\mathsf{T}} = 3\sqrt{\mathsf{T}d^{2.5}\nu\log(\mathsf{T})}$ .  
Part I  
2: Generate arm sequence  $S$  for the first  $\widetilde{\mathsf{T}}$  rounds using Algorithm 1.  
3: for  $t = 1$  to  $\widetilde{\mathsf{T}}$  do  
4: Pull the next arm  $X_{t}$  from the sequence  $S$ .  
5: Observe reward  $r_t$  and update  $\mathbf{V} \gets \mathbf{V} + X_t X_t^T$ .  
6: end for  
7: Set estimate  $\widehat{\theta} \coloneqq \mathbf{V}^{-1}\left(\sum_{t=1}^{\widetilde{\mathsf{T}}} r_t X_t\right)$   
8: Find  $\gamma = \max_{z \in \mathcal{X}} \langle z, \widehat{\theta} \rangle$   
9: Update  $\widetilde{\mathcal{X}} \gets \{x \in \mathcal{X} : \langle x, \widehat{\theta} \rangle \geq \gamma - 16\sqrt{\frac{3\gamma d^{\frac{5}{2}}\nu\log(\mathsf{T})}{\widetilde{\mathsf{T}}}}\}$   
10:  $\mathsf{T}' \gets \frac{2}{3}\widetilde{\mathsf{T}}$   
Part II  
11: while end of time horizon  $\mathsf{T}$  is reached do  
12: Initialize  $V = [0]_{d,d}$  to be an all zeros  $d \times d$  matrix and  $s = [0]_d$  to be an all-zeros vector. // Beginning of new phase.  
13: Find the probability distribution  $\lambda \in \Delta(\widetilde{\mathcal{X}})$  by maximizing the following objective  
    log Det( $\mathbf{V}(\lambda)$ ) subject to  $\lambda \in \Delta(\widetilde{\mathcal{X}})$  and  $\operatorname{Supp}(\lambda) \leq d(d+1)/2$ .  
14: for each arm  $a$  in  $\operatorname{Supp}(\lambda)$  do  
15: Pull arm  $a$  for the next  $[\lambda_a T']$  rounds.  
16: Observe rewards and Update  $\mathbf{V} \gets \mathbf{V} + [\lambda_a T'] \cdot aa^T$ .  
17: Observe  $[\lambda_a T']$  corresponding rewards  $z_1, z_2, \ldots$  and update  $s \gets s + (\sum_j z_j)a$ .  
18: end for  
19: Estimate  $\widehat{\theta} = \mathbf{V}^{-1}\left(\sum_{t \in \mathcal{E}} r_t X_t\right)$   
20: Find  $\gamma = \max_{z \in \mathcal{X}} \langle z, \widehat{\theta} \rangle$   
21:  $\widetilde{\mathcal{X}} \gets \{x \in \mathcal{X} : \langle x, \widehat{\theta} \rangle \geq \gamma - 16\sqrt{\frac{\gamma d^{\frac{5}{2}}\log(\mathsf{T})}{\widetilde{\mathsf{T}}'}}\}$   
22:  $\mathsf{T}' \gets 2 \times \mathsf{T}' // End$  of phase.  
23: end while
Instead of ensuring probability bounds on individual arms, we construct a confidence ellipsoid around  $\theta^{*}$ . In the context of Algorithm 3, we define the following events for the regret analysis:
$G_{1}$  In Part I, arms from the D-optimal design are chosen at least  $\widetilde{\mathsf{T}} /3$  times. If  $\langle x^{*},\theta^{*}\rangle \geq 196\sqrt{\frac{d^{2.5}\nu}{\mathsf{T}}}\log \mathsf{T}$ , then  $\widehat{\theta}$  calculated at the end of Part I satisfies
$$
\left\| \widehat {\theta} - \theta^ {*} \right\| _ {\mathbf {V}} \leq 7 \sqrt {\langle x ^ {*} , \theta^ {*} \rangle d ^ {\frac {3}{2}} \nu \log \mathsf {T}}.
$$
$G_{2}$  In Part II, for every phase, if  $\langle x^{*},\theta^{*}\rangle \geq 196\sqrt{\frac{d^{2.5}\nu}{T}}\log T$ , the estimators  $\widehat{\theta}$  satisfy:
Without loss of generality, we assume throughout that  $\langle x^{*},\theta^{*}\rangle \geq 196\frac{d^{1.25}\sqrt{\nu}}{\sqrt{T}}\log T$ . Otherwise, the regret bound in Theorem 2 trivially holds. Let  $\mathcal{B}$  denote the unit ball in  $\mathbb{R}^d$ . We have