SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \left\| \widehat {\theta} - \theta^ {*} \right\| _ {\mathbf {V}} = \left\| \mathbf {V} ^ {\frac {1}{2}} \left(\widehat {\theta} - \theta^ {*}\right) \right\| _ {2} \\ = \max _ {y \in \mathcal {B}} \left\langle y, \mathbf {V} ^ {\frac {1}{2}} (\widehat {\theta} - \theta^ {*}) \right\rangle . \\ \end{array}
$$
We construct an  $\varepsilon$ -net for the unit ball, denoted as  $\mathcal{C}_{\varepsilon}$ . For any  $y \in \mathcal{B}$ , we define  $y_{\varepsilon} \coloneqq \arg \min_{b \in \mathcal{C}_{\varepsilon}} \| b - y \|_2$ . We can now write
$$
\begin{array}{l} \left\| \widehat {\theta} - \theta^ {*} \right\| _ {\mathbf {V}} = \max  _ {y \in \mathcal {B}} \langle y - y _ {\varepsilon}, \mathbf {V} ^ {\frac {1}{2}} (\widehat {\theta} - \theta^ {*}) \rangle + \langle y _ {\varepsilon}, \mathbf {V} ^ {\frac {1}{2}} (\widehat {\theta} - \theta^ {*}) \rangle \\ \leq \max _ {y \in \mathcal {B}} \| y - y _ {\varepsilon} \| _ {2} \left\| \mathbf {V} ^ {\frac {1}{2}} (\widehat {\theta} - \theta^ {*}) \right\| _ {2} + | \langle y _ {\varepsilon}, \mathbf {V} ^ {\frac {1}{2}} (\widehat {\theta} - \theta^ {*}) \rangle | \\ \leq \varepsilon \left\| (\widehat {\theta} - \theta^ {*}) \right\| _ {\mathbf {V}} + | \langle y _ {\varepsilon}, \mathbf {V} ^ {\frac {1}{2}} (\widehat {\theta} - \theta^ {*}) \rangle |. \\ \end{array}
$$
$$
\left\| \widehat {\theta} - \theta^ {*} \right\| _ {\mathbf {V}} \leq \frac {1}{1 - \varepsilon} | \langle y _ {\varepsilon} \mathbf {V} ^ {\frac {1}{2}}, \widehat {\theta} - \theta^ {*} \rangle |. \tag {36}
$$
In the following lemmas, we show that  $|\langle y_{\varepsilon} \mathbf{V}^{\frac{1}{2}}, \widehat{\theta} - \theta^{*} \rangle|$  is small for all values of  $y_{\varepsilon}$ .
Lemma 19. Let  $x_{1}, x_{2}, \ldots, x_{n}$  be a sequence of fixed arm pulls (from a set  $\mathcal{X}$ ) such that each arm  $x$  in the support  $\lambda$  from  $D$ -optimal design (for  $\mathcal{X}$ ) is pulled at least  $\lceil \lambda_x \tau \rceil$  times. Consider the matrix  $\mathbf{V} = \sum_{j=1}^{n} x_j x_j^\top$  and let  $z$  be a vector such that  $\|z\|_2 \leq 1$  and  $\langle z \mathbf{V}^{\frac{1}{2}}, \theta^* \rangle \geq 6\nu \sqrt{\frac{d}{\tau}} \log(\mathsf{T}|\mathcal{C}_{\varepsilon}|)$ . Then, with probability greater than  $1 - \frac{2}{\Gamma|\mathcal{C}_{\varepsilon}|}$ , we have,
$$
\left| \langle z \mathbf {V} ^ {\frac {1}{2}}, \theta^ {*} - \widehat {\theta} \rangle \right| \leq \left(3 \nu \sqrt {\frac {n d}{\tau}} \log (\mathrm {T} | \mathcal {C} _ {\varepsilon} |) \langle x ^ {*}, \theta^ {*} \rangle\right) ^ {\frac {1}{2}}
$$
Proof. We begin by utilizing Lemma 6. First, we determine the  $\gamma$  parameter in the lemma as follows, for any  $t\in [n]$  we have
$$
\begin{array}{l} \left(z \mathbf {V} ^ {\frac {1}{2}}\right) ^ {T} \mathbf {V} ^ {- 1} x _ {t} \leq \left\| z \mathbf {V} ^ {\frac {1}{2}} \right\| _ {\mathbf {V} ^ {- 1}} \left\| \mathbf {V} ^ {- 1} x _ {t} \right\| _ {\mathbf {V}} \\ \leq \left\| z \right\| _ {2} \left\| x _ {t} \right\| _ {\mathbf {V} ^ {- 1}} \\ \leq \left\| x _ {t} \right\| _ {\mathbf {V} ^ {- 1}}. \quad (\text {s i n c e} \| z \| _ {2} \leq 1) \\ \end{array}
$$
Let  $A_{\lambda}$  be the optimal design matrix. Since  $\mathbf{V} \succ \tau A_{\lambda}$ , we have
$$
\begin{array}{l} \left\| x _ {t} \right\| _ {\mathbf {V} ^ {- 1}} \leq \left\| x _ {t} \right\| _ {\frac {1}{\tau} A _ {\lambda} ^ {- 1}} \\ \leq \sqrt {\frac {d}{\tau}}. \quad (\text {b y} \\ \end{array}
$$
Now, we use Corollary 9 with  $\gamma = \sqrt{\frac{d}{\tau}}$  and  $\delta = \left(3\sqrt{\frac{d}{\tau}}\frac{\nu\log\left(\mathsf{T}|\mathcal{C}_{\varepsilon}|\right)}{\langle z\mathbf{V}^{\frac{1}{2}},\theta^{*}\rangle}\right)^{\frac{1}{2}}$ . Note that  $\delta \in [0,1]$  since  $\langle z\mathbf{V}^{\frac{1}{2}},\theta^{*}\rangle \geq 6\sqrt{\frac{d}{\tau}}\nu \log \left(\mathsf{T}|\mathcal{C}_{\varepsilon}|\right)$ . We obtain the following probability bound
$$
\begin{array}{l} \mathbb {P} \left\{\left| \langle z \mathbf {V} ^ {\frac {1}{2}}, \theta^ {*} - \widehat {\theta} \rangle \right| \geq \left(3 \nu \sqrt {\frac {d}{\tau}} \log \left(\mathsf {T} | \mathcal {C} _ {\varepsilon} |\right) \langle z \mathbf {V} ^ {\frac {1}{2}}, \theta^ {*} \rangle\right) ^ {\frac {1}{2}} \right\} \leq 2 \exp \left(- \frac {3 \sqrt {\frac {d}{\tau}} \frac {\nu \log \left(\mathsf {T} | \mathcal {C} _ {\varepsilon} |\right)}{\langle z \mathbf {V} ^ {\frac {1}{2}} , \theta^ {*} \rangle} \langle z \mathbf {V} ^ {\frac {1}{2}}, \theta^ {*} \rangle}{3 \nu \sqrt {\frac {d}{\tau}}}\right) \\ \leq \frac {2}{T \left| \mathcal {C} _ {\varepsilon} \right|}. \tag {37} \\ \end{array}
$$
Finally, we establish an upper bound on the term  $\langle z\mathbf{V}^{\frac{1}{2}},\theta^{*}\rangle$  as follows
$$
\begin{array}{l} \langle z \mathbf {V} ^ {\frac {1}{2}}, \theta^ {*} \rangle \leq \| z \| _ {2} \left\| \mathbf {V} ^ {\frac {1}{2}} \theta^ {*} \right\| _ {2} \\ \leq \sqrt {\theta^ {* T} \mathbf {V} \theta^ {*}} \quad (\text {s i n c e} \| z \| _ {2} \leq 1) \\ = \sqrt {\left(\sum_ {i \in [ n ]} \theta^ {* T} x _ {i} x _ {i} ^ {T} \theta^ {*}\right)} \\ = \sqrt {n} \langle x ^ {*}, \theta^ {*} \rangle . \quad (\langle x _ {i}, \theta^ {*} \rangle \leq \langle x ^ {*}, \theta^ {*} \rangle) \\ \end{array}
$$
Substituting in (37) we get the lemma statement. This completes the proof of the lemma.
Lemma 20. Consider the same notation as in Lemma 19. If  $\langle z\mathbf{V}^{\frac{1}{2}},\theta^{*}\rangle \in \left[0,6\nu \sqrt{\frac{d}{\tau}}\log \left(\mathsf{T}|\mathcal{C}_{\varepsilon}|\right)\right]$ , then with probability greater than  $1 - \frac{2}{\mathsf{T}|\mathcal{X}|}$  we have
$$
| \langle z \mathbf {V} ^ {\frac {1}{2}}, \theta^ {*} - \widehat {\theta} \rangle | \leq 1 2 \nu \sqrt {\frac {d}{\tau}} \log (\mathsf {T} | \mathcal {C} _ {\varepsilon} |).
$$
Proof. Utilizing Lemma 10, with  $\delta = 1$ ,  $\alpha = 6\nu \sqrt{\frac{d}{\tau}}\log (\mathsf{T}|\mathcal{C}_{\varepsilon}|)$ , and  $\gamma = \sqrt{\frac{d}{\tau}}$ , we have  $\langle z\mathbf{V}^{\frac{1}{2}},\widehat{\theta}\rangle \leq 12\nu \sqrt{\frac{d}{\tau}}\log (\mathsf{T}|\mathcal{C}_{\varepsilon}|)$ . Since  $\langle z\mathbf{V}^{\frac{1}{2}},\theta^{*}\rangle \geq 0$ , it follows, with probability greater than  $1 - \frac{1}{\mathsf{T}|\mathcal{X}|}$ , that
$$
\langle z \mathbf {V} ^ {\frac {1}{2}}, \widehat {\theta} - \theta^ {*} \rangle \leq 1 2 \nu \sqrt {\frac {d}{\tau}} \log (\mathsf {T} | \mathcal {C} _ {\varepsilon} |).
$$
Next, applying Lemma 11 with  $\delta = 1$  and  $\alpha = 6\nu \sqrt{\frac{d}{\tau}}\log (\mathsf{T}|\mathcal{C}_{\varepsilon}|)$ , we have, with probability greater than  $1 - \frac{1}{\mathsf{T}|\mathcal{X}|}$ ,
$$
\langle z \mathbf {V} ^ {\frac {1}{2}}, \theta^ {*} - \widehat {\theta} \rangle \leq 6 \nu \sqrt {\frac {d}{\tau}} \log (\mathsf {T} | \mathcal {C} _ {\varepsilon} |) \leq 1 2 \nu \sqrt {\frac {d}{\tau}} \log (\mathsf {T} | \mathcal {C} _ {\varepsilon} |).
$$
Lemma 21. If  $\langle x^{*},\theta^{*}\rangle \geq 196\sqrt{\frac{d^{2.5\nu}}{\mathsf{T}}}\log \mathsf{T}$ , then
$$
\mathbb {P} \left\{G _ {1} \right\} \geq 1 - \frac {3}{T} \tag {38}
$$
Proof. First, we note (from Lemma 13) that arms from the solution of the D-optimal design problem are selected (with probability greater than  $1 - \frac{1}{\mathbb{T}}$ ) at least  $\widetilde{\mathsf{T}} / 3$  times. Hence, we can use Lemmas 19 and 20 with  $\tau = \widetilde{\mathsf{T}} / 3$ .
Let us consider the case where  $\langle y_{\varepsilon} \mathbf{V}^{\frac{1}{2}}, \theta^{*} \rangle \geq 6 \sqrt{\frac{3d}{T}} \log (\mathsf{T}|\mathcal{C}_{\varepsilon}|)$ . We have that the following holds with probability greater than  $1 - \frac{1}{T|\mathcal{C}_{\varepsilon}|}$ :
$$
\begin{array}{l} \left\| \widehat {\theta} - \theta^ {*} \right\| _ {\mathbf {V}} \leq \frac {1}{1 - \varepsilon} \langle y _ {\varepsilon} \mathbf {V} ^ {\frac {1}{2}}, \widehat {\theta} - \theta^ {*} \rangle \tag {from(36)} \\ \leq \frac {1}{1 - \varepsilon} \left(3 \nu \sqrt {\frac {\widetilde {\mathrm {T}} d}{\frac {\widetilde {\mathrm {T}}}{3}}} \log \left(\mathrm {T} \mid \mathcal {C} _ {\varepsilon} |\right) \langle x ^ {*}, \theta^ {*} \rangle\right) ^ {\frac {1}{2}} \quad (\text {u s i n g L e m m a 1 9}) \\ \leq \frac {1}{1 - \varepsilon} \left(3 \sqrt {3 d} \nu \log \left(\mathsf {T} | \mathcal {C} _ {\varepsilon} |\right) \langle x ^ {*}, \theta^ {*} \rangle\right) ^ {\frac {1}{2}}. \\ \end{array}
$$
Next, we note that  $|\mathcal{C}_{\varepsilon}| \leq \left(\frac{3}{\varepsilon}\right)^d$  [17], and by choosing  $\varepsilon = 1/2$  we get
$$
\left\| \widehat {\theta} - \theta^ {*} \right\| _ {\mathbf {V}} \leq 7 \left(\nu d ^ {\frac {3}{2}} \log (\mathsf {T}) \langle x ^ {*}, \theta^ {*} \rangle\right) ^ {\frac {1}{2}}
$$
Taking a union bound over all elements in  $\mathcal{C}_{\varepsilon}$  gives a probability bound of  $1 - \frac{1}{\Gamma}$ .
Now, for the case where  $\langle y_{\varepsilon}\mathbf{V}^{\frac{1}{2}},\theta^{*}\rangle \in \left[0,6\sqrt{\frac{3d}{\mathsf{T}}}\log \left(\mathsf{T}|\mathcal{C}_{\varepsilon}|\right)\right]$ , substituting  $\tau = \widetilde{\mathsf{T}} /3$  in Lemma 20 we have, with probability greater than  $1 - \frac{1}{\mathsf{T}|\mathcal{C}_{\varepsilon}|}$
$$
\begin{array}{l} \left\| \widehat {\theta} - \theta^ {*} \right\| _ {\mathbf {V}} \leq \frac {1}{1 - \varepsilon} \langle y _ {\varepsilon} \mathbf {V} ^ {\frac {1}{2}}, \widehat {\theta} - \theta^ {*} \rangle \\ \leq \frac {1 2 \nu}{1 - \varepsilon} \sqrt {\frac {d}{\tau}} \log \left(\mathrm {T} \mid \mathcal {C} _ {\varepsilon} \right|) \quad (\text {u s i n g L e m m a 2 0}) \\ \leq 2 4 \nu \sqrt {\frac {3 d ^ {3}}{\tilde {\mathsf {T}}}} \log (\mathsf {T}) \quad \text {(s u b s t i t u t i n g} \varepsilon = 0. 5) \\ \leq 7 \left(d ^ {\frac {3}{2}} \nu \log (\mathsf {T}) \langle x ^ {*}, \theta^ {*} \rangle\right) ^ {\frac {1}{2}} \\ \end{array}
$$
The last inequality is due to the fact that  $\langle x^{*},\theta^{*}\rangle \geq 196\sqrt{\frac{d^{2.5}\nu}{T}}\log T$  and  $\widetilde{T} = 3\sqrt{T\nu d^{2.5}\log T}$ . We again take a union bound over all elements in  $\mathcal{C}_{\varepsilon}$  to get a probability bound of  $1 - \frac{1}{T}$ .
Finally, a union bound over the two cases and the event in Lemma 13 proves the lemma.
Lemma 22. If  $\langle x^{*},\theta^{*}\rangle \geq 196\sqrt{\frac{d^{2.5}\nu}{T}}\log T$  then
$$
\mathbb {P} \left\{G _ {2} \right\} \geq 1 - \frac {\log \mathrm {T}}{\mathrm {T}}. \tag {39}
$$
Proof. To prove Lemma 22, we follow the same steps as in the proof of Lemma 21. Utilizing Lemma 19 and Lemma 20 with  $\tau = \mathsf{T}'$ , we establish that for any fixed phase, the following inequality holds with probability greater than  $1 - \frac{1}{\mathsf{T}}$ :
$$
\left\| \widehat {\theta} - \theta^ {*} \right\| _ {\mathbf {V}} \leq 7 \left(d ^ {\frac {3}{2}} \nu \log \mathrm {T} \langle x ^ {*}, \theta^ {*} \rangle\right) ^ {\frac {1}{2}}.
$$
Taking a union bound over all - at most  $\log T -$  phases in Part II of Algorithm 3 gives us the desired lower bound on  $\mathbb{P}\{G_2\}$
Corollary 23. If  $G_{1}$  holds, then for all  $x \in \mathcal{X}$ ,  $\widehat{\theta}$  calculated at the end of Part I satisfies
$$
| \langle x, \widehat {\theta} \rangle - \langle x, \theta^ {*} \rangle | \leq 7 \sqrt {\frac {3 \langle x ^ {*} , \theta^ {*} \rangle d ^ {2 . 5} \nu \log \mathsf {T}}{\widetilde {\mathsf {T}}}}
$$
Consider any phase  $\ell$  in Part II. If  $G_{2}$  holds, then for every arm in the surviving arm set  $\widetilde{\mathcal{X}}$ ,  $\widehat{\theta}$  calculated at the end of the phase satisfies
$$
| \langle x, \widehat {\theta} \rangle - \langle x, \theta^ {*} \rangle | \leq 7 \sqrt {\frac {3 \langle x ^ {*} , \theta^ {*} \rangle d ^ {2 . 5} \nu \log \mathsf {T}}{2 ^ {\ell} \widetilde {\mathsf {T}}}}.
$$
Proof. First we use Hölder's inequality
$$
\left| \langle x, \theta^ {*} - \widehat {\theta} \rangle \right| \leq \| x \| _ {\mathbf {V} ^ {- 1}} \left\| \theta^ {*} - \widehat {\theta} \right\| _ {\mathbf {V}}. \tag {40}
$$
Since  $G_{1}$  holds, arms from the optimal design matrix are selected at least  $\widetilde{\mathsf{T}} /3$  times; we have by Lemma 3
$$
\left\| x \right\| _ {\mathbf {V} ^ {- 1}} \leq \sqrt {\frac {3 d}{\widetilde {\mathbf {T}}}}.
$$
Similarly, for every phase in Part II with  $\mathsf{T}' = 2^{\ell}\widetilde{\mathsf{T}} /3$  we have
$$
\| x \| _ {\mathbf {V} ^ {- 1}} \leq \sqrt {\frac {d}{\mathsf {T} ^ {\prime}}}.
$$
Finally, using bounds on  $\left\| \theta^{*} - \widehat{\theta}\right\|_{\mathbf{V}}$  from events  $G_{1}$  and  $G_{2}$ , and substituting in (40), we get the desired bound.
Corollary 24. If  $\langle x^{*},\theta^{*}\rangle \geq 196\sqrt{\frac{d^{2.5}\nu}{T}}\log T$
$$
\frac {7}{1 0} \langle x ^ {*}, \theta^ {*} \rangle \leq \max  _ {x \in \mathcal {X}} \langle x, \widehat {\theta} \rangle \leq \frac {1 3}{1 0} \langle x ^ {*}, \theta^ {*} \rangle
$$
Proof. Since  $\mathsf{T}' \geq 2\widetilde{\mathsf{T}} / 3$ , via Corollary 23 any  $\widehat{\theta}$  calculated in Part I or during any phase of Part II satisfies
$$
\begin{array}{l} \max  _ {x \in \mathcal {X}} \langle x, \widehat {\theta} \rangle \geq \langle x ^ {*}, \widehat {\theta} \rangle \\ \geq \langle x ^ {*}, \theta^ {*} \rangle - 7 \sqrt {\frac {\langle x ^ {*} , \theta^ {*} \rangle d ^ {2 . 5} \nu \log \mathsf {T}}{\widetilde {\mathsf {T}}}} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle \left(1 - 7 \sqrt {\frac {d ^ {2 . 5} \nu \log \mathsf {T}}{\langle x ^ {*} , \theta^ {*} \rangle \widetilde {\mathsf {T}}}}\right) \\ \geq \frac {7}{1 0} \langle x ^ {*}, \theta^ {*} \rangle \quad (\text {s i n c e} \langle x ^ {*}, \theta^ {*} \rangle \geq 1 9 6 \sqrt {\frac {d ^ {2 . 5} \nu}{\mathsf {T}}} \log \mathsf {T} \text {a n d} \widetilde {\mathsf {T}} = 3 \sqrt {\mathsf {T} d ^ {2 . 5} \nu \log (\mathsf {T}))} \\ \end{array}
$$
Now, for any  $x\in \mathcal{X}$
$$
\begin{array}{l} \langle x, \widehat {\theta} \rangle \leq \langle x, \theta^ {*} \rangle + 7 \sqrt {\frac {\langle x ^ {*} , \theta^ {*} \rangle d ^ {2 . 5} \nu \log \mathsf {T}}{\tau}} \\ \leq \langle x ^ {*}, \theta^ {*} \rangle \left(1 + 7 \sqrt {\frac {d ^ {2 . 5} \nu \log \mathsf {T}}{\langle x ^ {*} , \theta^ {*} \rangle \tau}}\right) \\ \leq \frac {1 3}{1 0} \langle x ^ {*}, \theta^ {*} \rangle \\ \end{array}
$$
Lemma 25. If events  $G_{1}$  and  $G_{2}$  hold then the optimal arm  $x^{*}$  always exists in the surviving set  $\widetilde{X}$  in every phase in Part II of Algorithm 3
Proof. Let  $\tau = \widetilde{\mathsf{T}} /3$  for Part I and  $\tau = \mathsf{T}'$  for every phase of Part II. From Corollary 23 we have
$$
\begin{array}{l} \langle x ^ {*}, \widehat {\theta} \rangle \geq \langle x ^ {*}, \theta^ {*} \rangle - 7 \sqrt {\frac {\langle x ^ {*} , \theta^ {*} \rangle d ^ {2 . 5} \nu \log \mathsf {T}}{\tau}} \\ \geq \langle x, \theta^ {*} \rangle - 7 \sqrt {\frac {\langle x ^ {*} , \theta^ {*} \rangle d ^ {2 . 5} \nu \log \mathsf {T}}{\tau}} \quad (\text {s i n c e} \langle x ^ {*}, \theta^ {*} \rangle \geq \langle x, \theta^ {*} \rangle) \\ \geq \langle x, \widehat {\theta} \rangle - 1 4 \sqrt {\frac {\langle x ^ {*} , \theta^ {*} \rangle d ^ {2 . 5} \nu \log \mathsf {T}}{\tau}} \quad (u s i n g C o r o l l a r y 2 3) \\ \geq \langle x, \widehat {\theta} \rangle - 1 6 \sqrt {\frac {\operatorname* {m a x} _ {x \in \widetilde {\chi}} \langle x , \theta^ {*} \rangle d ^ {2 . 5} \nu \log \mathsf {T}}{\tau}}. \quad (u s i n g C o r o l l a r y 2 4) \\ \end{array}
$$
Hence, the best arm will never satisfy the elimination criteria in Algorithm 3.
Lemma 26. Given that events  $G_{1}$  and  $G_{2}$  hold, consider any phase index  $\ell$  in Part II of Alg. 3. For the surviving set of arms  $\widetilde{\mathcal{X}}$  at the beginning of that phase, and for  $\widetilde{\mathsf{T}} = \sqrt{d^{2.5}\nu\mathsf{T}\log(\mathsf{T})}$ , the following inequality holds for all  $x \in \widetilde{\mathcal{X}}$
$$
\langle x, \theta^ {*} \rangle \geq \langle x ^ {*}, \theta^ {*} \rangle - 2 6 \sqrt {\frac {3 d ^ {2 . 5} \nu \langle x ^ {*} , \theta^ {*} \rangle}{2 ^ {\ell} \cdot \widetilde {T}}}. \tag {41}
$$
Proof. Lemma 25 ensures that the optimal arm is contained in the surviving set of arms  $\widetilde{\mathcal{X}}$ . Furthermore, if an arm  $x \in \widetilde{\mathcal{X}}$  is pulled in the  $\ell^{\mathrm{th}}$  phase, then it must be the case that arm  $x$  was not eliminated in the previous phase (with a phase length parameter  $\frac{T'}{2}$ ); in particular the arms  $x$  does not satisfy the inequality on Line 21 of Algorithm 3. This inequality reduces to
$$
\begin{array}{l} \langle x, \widehat {\theta} \rangle \geq \langle x ^ {*}, \widehat {\theta} \rangle - 1 6 \sqrt {\frac {\max _ {x \in \widetilde {\mathcal {X}}} \langle x , \widehat {\theta} \rangle d ^ {2 . 5} \nu \log (\mathsf {T})}{\frac {\mathsf {T} ^ {\prime}}{2}}} \\ \geq \langle x ^ {*}, \widehat {\theta} \rangle - 2 6 \sqrt {\frac {\langle x ^ {*} , \theta^ {*} \rangle d ^ {2 . 5} \nu \log (\mathsf {T})}{\mathsf {T} ^ {\prime}}} \\ \end{array}
$$
(via Corollary 24)
Substituting  $\mathsf{T}' = 2^{l}\widetilde{\mathsf{T}} /3$  in the above inequality proves the Lemma.
Proof. Without loss of generality, we assume that  $\langle x^{*},\theta^{*}\rangle \geq 196\sqrt{\frac{d^{2.5}\nu}{T}}\log T$ . Otherwise, the Nash Regret bound is trivially true. For Part I, the product of expected rewards satisfies
$$
\begin{array}{l} \prod_ {t = 1} ^ {\tilde {\tau}} \mathbb {E} [ \langle X _ {t}, \theta^ {*} \rangle \mid G _ {1} \cap G _ {2} ] ^ {\frac {1}{\tau}} \geq \left(\frac {\langle x ^ {*} , \theta^ {*} \rangle}{2 (d + 1)}\right) ^ {\frac {\tilde {\tau}}{\tau}} \tag {From Lemma 5} \\ = \left\langle x ^ {*}, \theta^ {*} \right\rangle^ {\frac {\bar {\tau}}{\bar {\tau}}} \left(1 - \frac {1}{2}\right) ^ {\frac {\log (2 (d + 1)) ^ {\bar {\tau}}}{\bar {\tau}}} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle^ {\frac {1}{T}} \left(1 - \frac {\log (2 (d + 1)) \widetilde {T}}{T}\right). \\ \end{array}
$$
For Part II, we use Lemma 8. Let  $\mathcal{E}_i$  denote the time interval of the  $i^{th}$  phase, and let  $\mathsf{T}_i^\prime$  be the phase length parameter in that phase. Recall that  $|\mathcal{E}_i| \leq \mathsf{T}_i' + \frac{d(d+1)}{2}$ . Also, the algorithm runs for at most  $\log \mathsf{T}$  phases. Hence, we have
$$
\begin{array}{l} \prod_ {t = \tilde {\mathbb {T}} + 1} ^ {\mathbb {T}} \mathbb {E} [ \langle X _ {t}, \theta^ {*} \rangle \mid G _ {1} \cap G _ {2} ] ^ {\frac {1}{\mathbb {T}}} = \prod_ {\mathcal {E} _ {j}} \prod_ {t \in \mathcal {E} _ {j}} \mathbb {E} [ \langle X _ {t}, \theta^ {*} \rangle \mid G _ {1} \cap G _ {2} ] ^ {\frac {1}{\mathbb {T}}} \\ \geq \prod_ {\varepsilon_ {j}} \left(\langle x ^ {*}, \theta^ {*} \rangle - 2 6 \sqrt {\frac {d ^ {2 . 5} \nu \langle x ^ {*} , \theta^ {*} \rangle \log (\mathsf {T})}{\mathsf {T} _ {j} ^ {\prime}}}\right) ^ {\frac {| \varepsilon_ {j} |}{\mathsf {T}}} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle^ {\frac {\mathrm {T} - \tilde {\mathrm {T}}}{\mathrm {T}}} \prod_ {i = 1} ^ {\log \mathrm {T}} \left(1 - 2 6 \sqrt {\frac {d ^ {2 . 5} \nu \log (\mathrm {T})}{\langle x ^ {*} , \theta^ {*} \rangle \mathrm {T} _ {j} ^ {\prime}}}\right) ^ {\frac {| \mathcal {E} _ {j} |}{\mathrm {T}}} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle^ {\frac {\tau - \widetilde {\tau}}{\tau}} \prod_ {i = 1} ^ {\log \tau} \left(1 - 5 2 \frac {| \mathcal {E} _ {j} |}{\mathsf {T}} \sqrt {\frac {d ^ {2 . 5} \nu \log (\mathsf {T})}{\langle x ^ {*} , \theta^ {*} \rangle \mathsf {T} _ {j} ^ {\prime}}}\right) \\ \end{array}
$$
The last inequality is due to the fact that  $(1 - x)^r \geq (1 - 2rx)$  where  $r \in [0,1]$  and  $x \in [0,1/2]$ . Note that the term  $\sqrt{\frac{d^{2.5}\nu\log(\mathsf{T})}{\langle x^*,\theta^*\rangle\mathsf{T}_j'}} \leq 1/2$  for  $\langle x^*,\theta^*\rangle \geq 196\sqrt{\frac{d^{2.5}\nu}{\mathsf{T}}}\log \mathsf{T}$ ,  $\mathsf{T}' \geq 2\sqrt{\mathsf{T}d^{2.5}\nu\log\mathsf{T}}$ , and  $\mathsf{T} \geq e^6$ . We can further simplify the expression as follows
$$
\prod_ {j = 1} ^ {\log \mathsf {T}} \left(1 - 5 2 \frac {| \mathcal {E} _ {j} |}{\mathsf {T}} \sqrt {\frac {d ^ {2 . 5} \nu \log (\mathsf {T})}{\langle x ^ {*} , \theta^ {*} \rangle \mathsf {T} _ {j} ^ {\prime}}}\right) \geq \prod_ {j = 1} ^ {\log \mathsf {T}} \left(1 - 5 2 \frac {\mathsf {T} _ {j} ^ {\prime} + \frac {d (d + 1)}{2}}{\mathsf {T}} \sqrt {\frac {d ^ {2 . 5} \nu \log (\mathsf {T})}{\langle x ^ {*} , \theta^ {*} \rangle \mathsf {T} _ {j} ^ {\prime}}}\right)
$$
$$
\geq \prod_ {j = 1} ^ {\log \mathsf {T}} \left(1 - 7 8 \frac {\sqrt {\mathsf {T} _ {j} ^ {\prime}}}{\mathsf {T}} \sqrt {\frac {d ^ {2 . 5} \nu \log (\mathsf {T})}{\langle x ^ {*} , \theta^ {*} \rangle}}\right)
$$
(assuming  $\mathsf{T}_j^\prime \geq d(d + 1)$ )
$$
\geq 1 - 78 \frac {1}{T} \sqrt {\frac {d ^ {2 . 5} \nu \log (T)}{\langle x ^ {*} , \theta^ {*} \rangle}} \left(\sum_ {j = 1} ^ {\log T} \sqrt {T _ {j} ^ {\prime}}\right)
$$
$$
\geq 1 - 7 8 \frac {1}{T} \sqrt {\frac {d ^ {2 . 5} \nu \log (T)}{\langle x ^ {*} , \theta^ {*} \rangle}} \left(\sqrt {T \log T}\right)
$$
(using Cauchy Schwarz)
$$
\geq 1 - 7 8 \sqrt {\frac {d ^ {2 . 5} \nu}{\mathsf {T} \left\langle x ^ {*} , \theta^ {*} \right\rangle}} \log (\mathsf {T}).
$$
Combining the lower bound for rewards in Part I and Part II of the algorithm, we obtain
$$
\begin{array}{l} \prod_ {t = 1} ^ {\mathsf {T}} \mathbb {E} [ \langle X _ {t}, \theta^ {*} \rangle ] ^ {\frac {1}{\mathsf {T}}} \geq \prod_ {t = 1} ^ {\mathsf {T}} \left(\mathbb {E} [ \langle X _ {t}, \theta^ {*} \rangle \mid G _ {1} \cap G _ {2} ] \cdot \mathbb {P} \{G _ {1} \cap G _ {2} \}\right) ^ {\frac {1}{\mathsf {T}}} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle \left(1 - \frac {\log (2 (d + 1)) \widetilde {T}}{T}\right) \left(1 - 7 8 \sqrt {\frac {d ^ {2 . 5} \nu}{T \langle x ^ {*} , \theta^ {*} \rangle}} \log (T)\right) \mathbb {P} \left\{G _ {1} \cap G _ {2} \right\} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle \left(1 - \frac {\log (2 (d + 1)) \widetilde {\mathsf {T}}}{\mathsf {T}} - 7 8 \sqrt {\frac {d ^ {2 . 5} \nu}{\mathsf {T} \langle x ^ {*} , \theta^ {*} \rangle}} \log (\mathsf {T})\right) \mathbb {P} \{G _ {1} \cap G _ {2} \} \\ \geq \langle x ^ {*}, \theta^ {*} \rangle \left(1 - \frac {\log (2 (d + 1)) \widetilde {T}}{T} - 7 8 \sqrt {\frac {d ^ {2 . 5} \nu}{T \langle x ^ {*} , \theta^ {*} \rangle}} \log (T)\right) \left(1 - \frac {2 \log T}{T}\right) \\ \geq \langle x ^ {*}, \theta^ {*} \rangle \left(1 - \frac {\log (2 (d + 1)) 3 \sqrt {\mathsf {T} d \nu \log (\mathsf {T})}}{\mathsf {T}} - 7 8 \sqrt {\frac {d ^ {2 . 5} \nu}{\mathsf {T} \langle x ^ {*} , \theta^ {*} \rangle}} \log (\mathsf {T}) - \frac {2 \log \mathsf {T}}{\mathsf {T}}\right) \\ \geq \langle x ^ {*}, \theta^ {*} \rangle - 7 8 \sqrt {\frac {\langle x ^ {*} , \theta^ {*} \rangle d ^ {2 . 5} \nu}{T}} \log (T) - 2 \frac {\langle x ^ {*} , \theta^ {*} \rangle \log (2 (d + 1)) 3 \sqrt {d \log (T)}}{\sqrt {T}}. \\ \end{array}
$$
Hence, the Nash Regret can be bounded as
$$
\begin{array}{l} \mathrm {N R} _ {T} = \left\langle x ^ {*}, \theta^ {*} \right\rangle - \left(\prod_ {t = 1} ^ {T} \mathbb {E} [ \langle X _ {t}, \theta^ {*} \rangle ]\right) ^ {1 / T} \\ \leq 78 \sqrt {\frac {\langle x ^ {*} , \theta^ {*} \rangle d ^ {2 . 5} \nu}{T}} \log (T) + 2 \frac {\langle x ^ {*} , \theta^ {*} \rangle \log (2 (d + 1)) 3 \sqrt {d \nu \log (T)}}{\sqrt {T}}. \\ \end{array}
$$
The theorem stands proved.
We conduct experiments to compare the performance of our algorithm LINNASH with Thompson Sampling on synthetic data. For a comparison, we select Thompson Sampling (Algorithm 1 in
Figure 1: Nash Regret comparison of LINNASH and Thompson Sampling
Figure 3: Round-wise reward for Thompson Sampling
Figure 2: Round-wise reward for LINNASH
[2]), instead of UCB/OFUL, since randomization is essential to achieve meaningful Nash Regret guarantees.