SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[22] Eric M Schwartz, Eric T Bradlow, and Peter S Fader. Customer acquisition via display advertising using multi-armed bandit experiments. Marketing Science, 36(4):500-522, 2017.
[23] Ambuj Tewari and Susan A Murphy. From ads to interventions: Contextual bandits in mobile health. Mobile Health: Sensors, Analytic Methods, and Applications, pages 495-517, 2017.
[24] William R Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3-4):285-294, 1933.
[25] Michael J Todd. Minimum-volume ellipsoids: Theory and algorithms. SIAM, 2016.
[26] Yogatheesan Varatharajah and Brent Berry. A contextual-bandit-based approach for informed decision-making in clinical trials. *Life*, 12(8):1277, 2022.
[27] Sofia Villar, Jack Bowden, and James Wason. Multi-armed bandit models for the optimal design of clinical trials: Benefits and challenges. Statistical Science, 30:199–215, 05 2015.
Limitations: The main contributions of our works are theoretical. From a theoretical point of view, the limitations of our paper are discussed in Section 5. In particular, we believe that tightening the gap between the upper and lower bounds in Nash regret for an infinite set of arms will require novel and non-trivial algorithmic ideas - we leave this as an important direction of future work.
Broader Impact: Due to the theoretical nature of this work, we do not foresee any adverse societal impact of this work.
A Proof of Lemmas 1 and 2
Lemma 1. Any non-negative random variable  $X \in [0, \mathsf{B}]$  is  $\mathsf{B}$ -sub-Poisson, i.e., if mean  $\mathbb{E}[X] = \mu$ , then for all  $\lambda \in \mathbb{R}$ , we have  $\mathbb{E}[e^{\lambda X}] \leq \exp \left(B^{-1} \mu(e^{B\lambda} - 1)\right)$ .
Proof. For random variable  $X$  we have
$$
\begin{array}{l} \mathbb {E} \left[ \exp (\lambda X) \right] = \mathbb {E} \left[ \exp \left(\lambda B \frac {X}{B} + (1 - \frac {X}{B}) 0\right) \right] \\ \leq \mathbb {E} \left[ \frac {X}{B} e ^ {(\lambda B)} + \left(1 - \frac {X}{B}\right) e ^ {0} \right] \quad (\text {d u e t o c o n v e x i t y o f} e ^ {x}) \\ = 1 + \frac {\mathbb {E} [ X ]}{\mathsf {B}} \left(e ^ {\lambda \mathsf {B}} - 1\right) \\ \leq 1 + \frac {\mu}{B} \left(e ^ {\lambda B} - 1\right) \\ \leq \exp \left(\frac {\mu}{\mathsf {B}} \left(e ^ {\lambda \mathsf {B}} - 1\right)\right). \\ \end{array}
$$
Proof. Since  $X$  is a  $\sigma$ -sub-Gaussian random variable, for any non-negative scalar  $s \geq 0$ , we have
$$
\begin{array}{l} \mathbb {E} \left[ e ^ {s X} \right] \leq \exp \left(s \mu + \frac {(s \sigma) ^ {2}}{2}\right) \\ = \exp \left(\frac {\mu^ {2}}{\sigma^ {2}} \left(\frac {s \sigma^ {2}}{\mu} + \frac {1}{2} \left(\frac {s \sigma^ {2}}{\mu}\right) ^ {2}\right)\right) \tag {12} \\ \end{array}
$$
The fact that  $X$  is a positive random variable implies that the mean  $\mu > 0$ . Also, the considered scalar  $s \geq 0$  and, hence, the term  $\frac{s\sigma^2}{\mu} > 0$ . Also, recall that  $e^x \geq 1 + x + \frac{x^2}{2}$ , for any non-negative  $x$ . Using these observations and equation (12), we obtain
$$
\mathbb {E} \left[ e ^ {s X} \right] \leq \exp \left(\frac {\mu^ {2}}{\sigma^ {2}} \left(e ^ {\frac {s \sigma^ {2}}{\mu}} - 1\right)\right) \tag {13}
$$
For random variable  $X$ , inequality (13) ensures that the required mgf bound (equation (2)) holds for all non-negative  $s$  and with sub-Poisson parameter equal to  $\frac{\sigma^2}{\mu}$ .
We next complete the proof by showing that the mgf bound holds for negative  $s$  as well. Towards this, write  $B := \frac{\sigma^2}{\mu}$  and define random variable  $Y := \mathbf{1}_{\{X \leq B\}} X + \mathbf{1}_{\{X > B\}} B$ . Note that  $Y$  is a positive, bounded random variable. Furthermore, for any negative  $s$ , we have  $\exp(sY) \geq \exp(sX)$ . Therefore, for a negative  $s$ , it holds that  $\mathbb{E}[\exp(sX)] \leq \mathbb{E}[\exp(sY)]$ . Since positive random variables  $Y \in [0, B]$ , the mgf bound obtained in Lemma 1 gives us
$$
\mathbb {E} \left[ e ^ {s X} \right] \leq \mathbb {E} \left[ e ^ {s Y} \right] \leq \exp \left(\frac {\mu}{\mathsf {B}} \left(e ^ {s \mathsf {B}} - 1\right)\right).
$$
Since  $B \coloneqq \frac{\sigma^2}{\mu}$ , the mgf bound (equation (2)) on  $X$  holds for negative  $s$  as well. This, overall, shows that  $X$  is a  $\left(\frac{\sigma^2}{\mu}\right)$ -sub-Poisson random variable. The lemma stands proved.
B Proof of Concentration Bounds
Proof. We use the Chernoff method to get an upper bound on the desired probabilities, as shown below
$$
\begin{array}{l} \mathbb {P} \left\{\langle z, \widehat {\theta} \rangle \geq (1 + \delta) \langle z, \theta^ {*} \rangle \right\} = \mathbb {P} \left(\exp (c \langle z, \widehat {\theta} \rangle) \geq \exp (c (1 + \delta) \langle z, \theta^ {*} \rangle)\right) \quad (\text {f o r s o m e c o n s t a n t} c) \\ \leq \frac {\mathbb {E} \left[ \exp \left(c z ^ {T} \mathbf {V} ^ {- 1} \left(\sum_ {t} r _ {t} x _ {t}\right)\right) \right]}{\exp (c \left(1 + \delta\right) \langle z , \theta^ {*} \rangle)} \\ = \frac {\prod_ {t = 1} ^ {s} \mathbb {E} \left[ \exp \left(c r _ {t} \mathbf {V} ^ {- 1} x _ {t}\right) \right]}{\exp (c (1 + \delta) \langle z , \theta^ {*} \rangle)} \quad (r _ {t} \text {’ s a r e i n d e p e n d e n t}) \\ \leq \frac {\prod_ {t = 1} ^ {s} \exp \left(\frac {\mathbb {E} [ r _ {t} ]}{\nu} \left(e ^ {c \nu z ^ {T} \mathbf {V} ^ {- 1} x _ {t}} - 1\right)\right)}{\exp (c (1 + \delta) \langle z , \theta^ {*} \rangle)} \quad (r _ {t} \text {i s s u b P o i s s o n}) \\ = \exp \left(- c \langle z, \theta^ {*} \rangle (1 + \delta) + \sum_ {t = 1} ^ {s} \frac {\langle x , \theta^ {*} \rangle}{\nu} \left(e ^ {c \nu z ^ {T} \mathbf {V} ^ {- 1} x _ {t}} - 1\right)\right). \\ \end{array}
$$
Substituting  $c = \frac{\log(1 + \delta)}{\nu\gamma}$ , we get
$$
\mathbb {P} \left\{\langle z, \widehat {\theta} \rangle \geq (1 + \delta) \langle z, \theta^ {*} \rangle \right\} \leq \exp \left(- \frac {\langle z , \theta^ {*} \rangle}{\nu \gamma} (1 + \delta) \log (1 + \delta) + \sum_ {t = 1} ^ {s} \frac {\langle x _ {t} , \theta^ {*} \rangle}{\nu} \left((1 + \delta) ^ {\frac {1}{\gamma} z ^ {T}} \mathbf {V} ^ {- 1} x _ {t} - 1\right)\right). \tag {14}
$$
Since  $\frac{1}{\gamma} z^T \mathbf{V}^{-1} x_t \leq 1$  we have  $(1 + \delta)^{\frac{1}{\gamma} z^T \mathbf{V}^{-1} x_t} \leq 1 + \delta \cdot \frac{1}{\gamma} z^T \mathbf{V}^{-1} x_t$ . Substituting in (14) we get
$$
\begin{array}{l} \mathbb {P} \left\{\langle z, \widehat {\theta} \rangle \geq (1 + \delta) \langle z, \theta^ {*} \rangle \right\} \\ \leq \exp \left(- \frac {1}{\nu \gamma} \langle z, \theta^ {*} \rangle (1 + \delta) \log (1 + \delta) + \sum_ {t = 1} ^ {s} \langle x _ {t}, \theta^ {*} \rangle \cdot \frac {\delta}{\nu \gamma} z ^ {T} \mathbf {V} ^ {- 1} x _ {t}\right) \\ = \exp \left(- \frac {1}{\nu \gamma} \langle z, \theta^ {*} \rangle (1 + \delta) \log (1 + \delta) + \frac {\delta}{\nu \gamma} \sum_ {t = 1} ^ {s} \theta^ {* T} x _ {t} x _ {t} ^ {T} \mathbf {V} ^ {- 1} z\right) \quad (\text {r e a r r a n g i n g t e r m s}) \\ = \exp \left(- \frac {1}{\nu \gamma} \langle z, \theta^ {*} \rangle (1 + \delta) \log (1 + \delta) + \frac {\delta}{\nu \gamma} \langle z, \theta^ {*} \rangle\right). \quad (\sum_ {t = 1} ^ {s} x _ {t} x _ {t} ^ {T} = \mathbf {V}) \\ \end{array}
$$
Using the logarithmic inequality  $\log (1 + \delta) \geq \frac{2\delta}{2 + \delta}$ , we further simplify as
$$
\begin{array}{l} \mathbb {P} \left\{\langle z, \widehat {\theta} \rangle \geq (1 + \delta) \langle z, \theta^ {*} \rangle \right\} \leq \exp \left(- \frac {\langle z , \theta^ {*} \rangle}{\nu \gamma} ((1 + \delta) \log (1 + \delta) - \delta)\right) \\ \leq \exp \left(\frac {- \delta^ {2} \langle z , \theta^ {*} \rangle}{(2 + \delta) \nu \gamma}\right) \\ \leq \exp \left(\frac {- \delta^ {2} \langle z , \theta^ {*} \rangle}{3 \nu \gamma}\right). \quad (\text {s i n c e} \delta \in [ 0, 1 ]) \\ \end{array}
$$
Following similar steps and substituting  $c = \frac{\log(1 - \delta)}{\nu\gamma}$ , we obtain a bound on the lower tail (inequality 9):
$$
\mathbb {P} \left\{\langle z, \widehat {\theta} \rangle \leq (1 - \delta) \langle z, \theta^ {*} \rangle \right\} \leq \exp \left(- \frac {1}{\nu \gamma} \langle z, \theta^ {*} \rangle (1 - \delta) \log (1 - \delta) - \frac {\delta}{\nu \gamma} \langle z, \theta^ {*} \rangle\right).
$$
Now, using the logarithmic inequality  $(1 - \delta)\log (1 - \delta)\geq -\delta +\frac{\delta^2}{2}$ , we get
$$
\mathbb {P} \left\{\langle z, \widehat {\theta} \rangle \leq (1 - \delta) \langle z, \theta^ {*} \rangle \right\} \leq \exp \left(\frac {- \delta^ {2} \langle z , \theta^ {*} \rangle}{2 \nu \gamma}\right)
$$
Combining (9) and (8) we get the following Corollary.
Corollary 9. Using the notations as in Lemma 6, we have
$$
\mathbb {P} \left\{\left| \langle z, \widehat {\theta} \rangle - \langle z, \theta^ {*} \rangle \right| \geq \delta \langle z, \theta^ {*} \rangle \right\} \leq 2 \exp \left(- \frac {\delta^ {2} \langle z , \theta^ {*} \rangle}{3 \gamma}\right). \tag {15}
$$
The next two lemmas are variants of Lemma 6 where we bound the error in terms of an upper bound on  $\langle z, \theta^* \rangle$ .
Lemma 10. Let  $x_{1}, x_{2}, \ldots, x_{s} \in \mathbb{R}^{d}$  be a fixed set of vectors and let  $r_{1}, r_{2}, \ldots, r_{s}$  be independent  $\nu$ -sub Poisson random variables satisfying  $\mathbb{E}r_{s} = \langle x_{s}, \theta^{*} \rangle$  for some unknown  $\theta^{*}$ . In that case, let matrix  $\mathbf{V} = \sum_{j=1}^{s} x_{j}x_{j}^{T}$  and  $\widehat{\theta} = \mathbf{V}^{-1}\left(\sum_{j} r_{j}x_{j}\right)$  be the least squares estimator of  $\theta^{*}$ . Consider any  $z \in \mathbb{R}^{d}$  that satisfies  $z^{T}\mathbf{V}^{-1}x_{j} \leq \gamma$  for all  $j \in [s]$  and  $\langle z, \theta^{*} \rangle \leq \alpha$ . Then for any  $\delta \in [0,1]$  we have
$$
\mathbb {P} \left\{\langle z, \widehat {\theta} \rangle \geq (1 + \delta) \alpha \right\} \leq e ^ {- \frac {\delta^ {2} \alpha}{3 \gamma \nu}}. \tag {16}
$$
Proof. Following the same approach as in the proof of Lemma 6, we have
$$
\mathbb {P} \left\{\langle z, \widehat {\theta} \rangle \geq (1 + \delta) \alpha \right\} \leq \frac {\mathbb {E} [ \exp (c z ^ {T} \mathbf {V} ^ {- 1} (\sum_ {t} r _ {t} x _ {t})) ]}{\exp (c (1 + \delta) \alpha)}
$$
$$
\leq \exp \left(- c \alpha (1 + \delta) + \sum_ {t = 1} ^ {s} \frac {\langle x _ {t} , \theta^ {*} \rangle}{\nu} \left(e ^ {c \nu z ^ {T} \mathbf {V} ^ {- 1} x _ {t}} - 1\right)\right)
$$
$(r_t$  are sub-poisson and independent)
Now, substituting  $c = \frac{1}{\nu \gamma} \log (1 + \delta)$  and using  $(1 + \delta)^{\frac{1}{\gamma} z^T \mathbf{V}^{-1} x_t} \leq 1 + \delta \cdot \frac{1}{\gamma} z^T \mathbf{V}^{-1} x_t$  we have
$$
\begin{array}{l} \mathbb {P} \left\{\langle z, \widehat {\theta} \rangle \geq (1 + \delta) \alpha \right\} \leq \exp \left(- \frac {1}{\gamma \nu} \alpha (1 + \delta) \log (1 + \delta) + \sum_ {t = 1} ^ {s} \frac {\langle x _ {t} , \theta^ {*} \rangle}{\nu} \theta^ {*} \left((1 + \delta) ^ {\frac {1}{\gamma} z ^ {T} \mathbf {V} ^ {- 1} x _ {t}} - 1\right)\right) \\ \leq \exp \left(- \frac {1}{\nu \gamma} \alpha (1 + \delta) \log (1 + \delta) + \frac {\delta}{\nu \gamma} \sum_ {t = 1} ^ {s} \theta^ {* T} x _ {t} x _ {t} ^ {T} \mathbf {V} ^ {- 1} Z\right) \\ = \exp \left(- \frac {1}{\nu \gamma} \alpha (1 + \delta) \log (1 + \delta) + \frac {\delta}{\nu \gamma} \langle z, \theta^ {*} \rangle\right) \\ \leq \exp \left(- \frac {1}{\nu \gamma} \alpha (1 + \delta) \log (1 + \delta) + \frac {\delta}{\nu \gamma} \alpha\right) \quad (\alpha \geq \langle z, \theta^ {*} \rangle) \\ \leq \exp \left(\frac {- \delta^ {2} \alpha}{(2 + \delta) \nu \gamma}\right) \quad (\text {u s i n g} \log (1 + \delta) \geq \frac {2 \delta}{2 + \delta}) \\ \end{array}
$$
Since  $\delta \in [0,1]$ , we have the desired result.
Lemma 11. Using the same notations as in Lemma 10, for any  $\delta \in [0,1]$ , the following holds
$$
\mathbb {P} \left\{\langle z, \widehat {\theta} \rangle \leq \langle z, \theta^ {*} \rangle - \delta \alpha \right\} \leq \exp \left(- \frac {\delta^ {2} \alpha}{2 \gamma \nu}\right) \tag {17}
$$
Proof. Using steps similar to the previous lemmas, we obtain
$$
\begin{array}{l} \mathbb {P} \left\{\langle z, \widehat {\theta} \rangle \leq \langle z, \theta^ {*} \rangle - \delta \alpha \right\} \leq \frac {\mathbb {E} [ \exp (c z ^ {T} \mathbf {V} ^ {- 1} (\sum_ {t} r _ {t} x _ {t})) ]}{\exp (c (\langle z , \theta^ {*} \rangle - \delta \alpha))} \\ \leq \exp \left(c \alpha \delta + c \langle z, \theta^ {*} \rangle + \sum_ {t = 1} ^ {s} \frac {\langle x _ {t} , \theta^ {*} \rangle}{\nu} \left(e ^ {c \nu z ^ {T} \mathbf {V} ^ {- 1} x _ {t}} - 1\right)\right) \\ \end{array}
$$
Substituting  $c = \frac{\log(1 - \delta)}{\nu\gamma}$  and simplifying we get
$$
\mathbb {P} \left\{\langle z, \widehat {\theta} \rangle \leq \langle z, \theta^ {*} \rangle - \delta \alpha \right\} \leq \exp \left(- \frac {\langle z , \theta^ {*} \rangle}{\nu \gamma} (\log (1 - \delta) + \delta) + \frac {\alpha}{\nu \gamma} \delta \log (1 - \delta)\right)
$$
Note that since  $\log (1 - \delta) + \delta$  is negative, we can upper bound the above expression by replacing  $\langle z,\theta^{*}\rangle$  with  $\alpha$ .
$$
\begin{array}{l} \mathbb {P} \left\{\langle z, \widehat {\theta} \rangle \leq \langle z, \theta^ {*} \rangle - \delta \alpha \right\} \leq \exp \left(- \frac {\alpha}{\nu \gamma} (\log (1 - \delta) + \delta - \delta \log (1 - \delta))\right) \\ \leq \exp \left(- \frac {\delta^ {2} \alpha}{2 \nu \gamma}\right). \quad (\text {s i n c e} (1 - \delta) \log (1 - \delta) \geq - \delta + \frac {\delta^ {2}}{2}) \\ \end{array}
$$
Hence, the lemma stands proved.
C Regret Analysis of Algorithm 2
We will first define events  $E_{1}$  and  $E_{2}$  for each phase of the algorithm and show that they hold with high probability. We will use the events in the regret analysis.
- Event  $E_{1}$ : At the end of Part I, let  $\widehat{\theta}$  be the unbiased estimator of  $\theta^{*}$  and  $\widetilde{\mathsf{T}}$  be as defined in Algorithm 2. All arms  $x\in \mathcal{X}$  with  $\langle x,\theta^{*}\rangle < 10\sqrt{\frac{d\nu\log(\mathsf{T}|\mathcal{X}|)}{\mathsf{T}}}$  satisfy
$$
\langle x, \widehat {\theta} \rangle \leq 2 0 \sqrt {\frac {d \nu \log (\mathrm {T} | \mathcal {X} |)}{\mathrm {T}}} \tag {18}
$$
In addition, all arms  $x \in \mathcal{X}$  with  $\langle x, \theta^* \rangle \geq 10\sqrt{\frac{d\nu\log(\mathsf{T}|\mathcal{X}|)}{\mathsf{T}}}$  satisfy
$$
| \langle x, \theta^ {*} \rangle - \langle x, \widehat {\theta} \rangle | \leq 3 \sqrt {\frac {d \nu \langle x , \theta^ {*} \rangle \log (\mathrm {T} | \mathcal {X} |)}{\widetilde {\mathrm {T}}}} \quad \text {a n d} \tag {19}
$$
$$
\frac {1}{2} \langle x, \theta^ {*} \rangle \leq \langle x, \widehat {\theta} \rangle \leq \frac {4}{3} \langle x, \theta^ {*} \rangle . \tag {20}
$$
- Event  $E_{2}$ : Let  $\widetilde{\mathcal{X}}$  denote the surviving set of arms at the start of a phase in Part II, and  $T'$  be as defined in Algorithm 2. For all phases and for all  $x \in \widetilde{\mathcal{X}}$  such that  $\langle x, \theta^{*} \rangle \geq 10\sqrt{\frac{d\nu\log(\mathsf{T}|\mathcal{X}|)}{\mathsf{T}}}$ , the estimator  $\widehat{\theta}$  (calculated at the end of a phase) satisfies
$$
\left| \langle x, \theta^ {*} \rangle - \langle x, \widehat {\theta} \rangle \right| \leq 3 \sqrt {\frac {d \nu \langle x , \theta^ {*} \rangle \log (\mathrm {T} | \mathcal {X} |)}{\mathrm {T} ^ {\prime}}} \quad \text {a n d} \tag {21}
$$
$$
\frac {1}{2} \langle x, \theta^ {*} \rangle \leq \langle x, \widehat {\theta} \rangle \leq \frac {4}{3} \langle x, \theta^ {*} \rangle . \tag {22}
$$
Lemma 12 (Chernoff Bound). Let  $Z_{1},\ldots ,Z_{n}$  be independent Bernoulli random variables. Consider the sum  $S = \sum_{r = 1}^{n}Z_{r}$  and let  $\mu = \mathbb{E}[S]$  be its expected value. Then, for any  $\varepsilon \in [0,1]$ , we have
$$
\mathbb {P} \left\{S \leq (1 - \varepsilon) \mu \right\} \leq \exp \left(- \frac {\mu \varepsilon^ {2}}{2}\right).
$$
Lemma 13. During Part I, arms from  $D$ -optimal design are added to  $S$  at least  $\widetilde{T} / 3$  times with probability greater than  $1 - \frac{1}{\widetilde{T}}$ .
Proof. We use Lemma 12 with  $Z_{i}$  as indicator random variables that take value one when an arm from  $\mathcal{A}$  (the support of  $\lambda$  in the optimal design) is chosen. By setting  $\varepsilon = \frac{1}{3}$  and  $\mu = \frac{\widetilde{\Gamma}}{2}$ , we obtain the required probability bound.
Lemma 14. Using the notations in Algorithm 1, if the event in Lemma 13 holds, then for each  $x \in \mathcal{X}$  and each round  $t$  in Part I of the algorithm, we have
$$
x ^ {T} \mathbf {V} ^ {- 1} X _ {t} \leq \frac {3 d}{\widetilde {\mathbf {T}}},
$$
where  $X_{t}$  is the arm pulled in round  $t$ .
Proof. Let  $\mathbf{U}(\lambda)$  and  $\lambda$  denote the optimal design matrix (as defined in (4)) and the solution to the D-optimal design problem in Algorithm 1, respectively. That is,  $\lambda$  is the solution of the optimization problem stated in equation (5) and  $\mathbf{U}(\lambda) = \sum_{x\in \mathcal{X}}\lambda_{x}xx^{T}$ . Lemma 3 implies that  $\| x\|_{\mathbf{U}(\lambda)^{-1}}\leq \sqrt{d}$  for all  $x\in \mathcal{X}$ .
Next, note that the construction of the sequence  $S$  in Part I (Subroutine GenerateArmSequence) and the event specified in Lemma 13 give us  $\mathbf{V} \succ \frac{\widetilde{T}}{3}\mathbf{U}(\lambda)$ . Hence,
$$
\begin{array}{l} x ^ {T} \mathbf {V} ^ {- 1} X _ {t} \leq \| x \| _ {\mathbf {V} ^ {- 1}} \left\| \mathbf {V} ^ {- 1} X _ {t} \right\| _ {\mathbf {V}} \quad (\text {v i a H o l d e r s i n e q u a l i t y}) \\ = \| x \| _ {\mathbf {V} ^ {- 1}} \| X _ {t} \| _ {\mathbf {V} ^ {- 1}} \\ \leq \| x \| _ {\left(\frac {\tilde {\tau}}{3} \mathbf {U} (\lambda)\right) ^ {- 1}} \| X _ {t} \| _ {\left(\frac {\tilde {\tau}}{3} \mathbf {U} (\lambda)\right) ^ {- 1}} \quad \text {(s i n c e} \mathbf {V} \succ \frac {\tilde {\tau}}{3} \mathbf {U} (\lambda)) \\ = \sqrt {\frac {3}{\widetilde {\mathbf {T}}}} \| x \| _ {\mathbf {U} (\lambda) ^ {- 1}} \sqrt {\frac {3}{\widetilde {\mathbf {T}}}} \| X _ {t} \| _ {\mathbf {U} (\lambda) ^ {- 1}} \\ \leq \sqrt {\frac {3 d}{\widetilde {T}}} \sqrt {\frac {3 d}{\widetilde {T}}} \quad (\text {b y} \\ = \frac {3 d}{\widetilde {\Upsilon}}. \\ \end{array}
$$
The next lemma lower bounds the probability of event  $E_{1}$  (see equations (18), (19), and (20)).
Lemma 15. Event  $E_{1}$  holds with probability at least  $1 - \frac{6}{T}$ .
Proof. First, consider all arms  $x \in \mathcal{X}$  for which  $\langle x, \theta^* \rangle < 10\sqrt{\frac{d\nu\log(\mathsf{T}|\mathcal{X}|)}{\mathsf{T}}}$ . Here, we invoke Lemma 10, with  $\gamma = \frac{3d}{\widetilde{\mathsf{T}}}$  (as derived in Lemma 14),  $\alpha = 10\sqrt{\frac{d\nu\log(\mathsf{T}|\mathcal{X}|)}{\mathsf{T}}}$ , and  $\delta = 1$ , to obtain
$$
\begin{array}{l} \mathbb {P} \left\{\langle x, \widehat {\theta} \rangle \leq 2 0 \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T}}} \right\} \leq \exp \left(- \frac {\delta^ {2} \alpha}{3 \gamma \nu}\right) \\ \leq \exp \left(- \frac {1 0 \sqrt {\frac {d \nu \log (\mathrm {T} | \mathcal {X} |)}{\mathrm {T}}} 3 \sqrt {T d \nu \log (\mathrm {T} | \mathcal {X} |)}}{3 \nu d}\right) \\ \leq \frac {1}{T | \mathcal {X} |} \tag {23} \\ \end{array}
$$
Next, we consider arms  $x \in \mathcal{X}$  such that  $\langle x, \theta^* \rangle \geq 10\sqrt{\frac{d\nu\log(\Upsilon|\mathcal{X}|)}{\Upsilon}}$  and for such arms establish equations (19) and (20). Towards this, we invoke Lemma 6, with parameters  $\gamma = \frac{3d}{\Upsilon}$  and  $\delta =$
$3\sqrt{\frac{d\nu\log(\mathsf{T}|\mathcal{X}|)}{\langle x,\theta^{*}\rangle\widetilde{\mathsf{T}}}}$  It is relevant to note that here  $\delta \in [0,1]$  - this containment follows from the condition  $\langle x,\theta^{*}\rangle \geq 10\sqrt{\frac{d\nu\log(\mathsf{T}|\mathcal{X}|)}{\mathsf{T}}}$  and  $\widetilde{\mathsf{T}} = 3\sqrt{\mathsf{T}d\nu\log(\mathsf{T}|\mathcal{X}|)}$ . Therefore,
$$
\mathbb {P} \left\{\left| \langle x, \theta^ {*} \rangle - \langle x, \widehat {\theta} \rangle \right| \geq 3 \sqrt {\frac {d \nu \langle x , \theta^ {*} \rangle \log (\mathsf {T} | \mathcal {X} |)}{\widetilde {\mathsf {T}}}} \right\} = \mathbb {P} \left\{\left| \langle x, \theta^ {*} \rangle - \langle x, \widehat {\theta} \rangle \right| \geq \delta \langle x, \theta^ {*} \rangle \right\}
$$
$$
(s i n c e \delta = 3 \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{\langle x , \theta^ {*} \rangle \widetilde {\mathsf {T}}}})
$$
$$
\leq 2 \exp \left(- \frac {\frac {9 d \nu \log (\Upsilon | \mathcal {X} |)}{\langle x , \theta^ {*} \rangle \widetilde {\Upsilon}} \langle x , \theta^ {*} \rangle}{3 \nu \frac {3 d}{\widetilde {\Upsilon}}}\right)
$$
(Lemma 6)
$$
= \frac {2}{\overline {{T}} | \mathcal {X} |} \tag {24}
$$
For establishing equation (20), we invoke Lemma 6 again, now with  $\gamma = \frac{3d}{\tilde{\Gamma}}$  and  $\delta = \frac{1}{3}$ :
$$
\begin{array}{l} \mathbb {P} \left\{\langle x, \widehat {\theta} \rangle \geq \frac {4}{3} \langle x, \theta^ {*} \rangle \right\} \leq \exp \left(- \frac {3 \sqrt {\mathsf {T} \nu d \log (\mathsf {T} | \mathcal {X} |)} \langle x , \theta^ {*} \rangle}{2 7 \nu d}\right) \\ \leq \exp \left(- \frac {3 \sqrt {\mathsf {T} \nu d \log (\mathsf {T} | \mathcal {X} |)} 1 0 \sqrt {\frac {d \nu \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T}}}}{2 7 \nu d}\right) \\ \leq \frac {1}{T | \mathcal {X} |} \tag {25} \\ \end{array}
$$
Similarly, with  $\delta = \frac{1}{2}$ , Lemma 6 gives us
$$
\mathbb {P} \left\{\langle x, \widehat {\theta} \rangle \leq \frac {1}{2} \langle x, \theta^ {*} \rangle \right\} \leq \frac {1}{\mathrm {T} | \mathcal {X} |} \tag {26}
$$
Finally, we combine (23), (24), (25) and (26), and apply a union bound over all arms in  $\mathcal{X}$ . Then, conditioning on the event in Lemma 13 leads to the stated probability bound. The lemma stands proved.
The next lemma shows that event  $E_{2}$  (see equations (21) and (22)) holds with high probability Lemma 16. Event  $E_{2}$  holds with probability at least  $1 - \frac{3\log T}{T}$ .
Proof. Consider any phase in Part II and let  $\mathbf{U}(\lambda)$  be the optimal design matrix obtained after solving the D-optimal design problem at the start of the phase. By Lemma 3, for all  $x,z\in \widetilde{\mathcal{X}}$  we have
$$
\begin{array}{l} z ^ {T} \mathbf {V} ^ {- 1} x \leq \| z \| _ {\mathbf {V} ^ {- 1}} \| \mathbf {V} ^ {- 1} x \| _ {\mathbf {V}} \quad (\text {v i a H o l d e r ' s i n e q u a l i t y}) \\ \leq \| z \| _ {\mathbf {V} ^ {- 1}} \| x \| _ {\mathbf {V} ^ {- 1}} \\ \leq \sqrt {\frac {d}{T ^ {\prime}}} \sqrt {\frac {d}{T ^ {\prime}}} = \frac {d}{T ^ {\prime}} \\ \end{array}
$$
First, we address equation (21). In particular, we instantiate Lemma 6 with  $\delta = 3\sqrt{\frac{d\nu\log(T|\mathcal{X}|)}{\langle x,\theta^{*}\rangle T^{\prime}}}$  and  $\gamma = \frac{d}{T^{\prime}}$ . Note that given the lower bound on  $\langle x,\theta^{*}\rangle$  and the inequality  $T^{\prime} \geq 2\sqrt{Td\nu\log(T|\mathcal{X}|)}$  ensure that  $\delta$  lies in [0, 1]. Hence, substituting these values of  $\delta$  and  $\gamma$  in Lemma 6, we obtain
$$
\begin{array}{l} \mathbb {P} \left\{| \langle x, \theta^ {*} \rangle - \langle x, \widehat {\theta} \rangle | \geq 3 \sqrt {\frac {d \nu \langle x , \theta^ {*} \rangle \log (\mathsf {T} | \mathcal {X} |)}{\mathsf {T} ^ {\prime}}} \right\} \leq 2 \exp \left(- \frac {\frac {9 d \nu \log (\mathsf {T} | \mathcal {X} |)}{\langle x , \theta^ {*} \rangle \mathsf {T} ^ {\prime}} \cdot \langle x , \theta^ {*} \rangle}{3 \frac {d \nu}{\mathsf {T} ^ {\prime}}}\right) \\ \leq \frac {2}{(\mathsf {T} | \mathcal {X} |) ^ {3}} \\ \end{array}
$$
Next, following a similar approach as in the proof of Lemma 15, we use Lemma 6 with  $\delta = \frac{1}{3}$  and  $\delta = \frac{1}{2}$  to establish the upper and lower bounds of equation (22), respectively. Applying a union bound across arms in  $\widetilde{\mathcal{X}}$  and over all—at most  $\log T$ —phases, we obtain the desired probability bound of  $1 - \frac{3\log T}{T}$ .
Corollary 17.