SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \sum_ {i = 1} ^ {K} \mathbb {E} (T _ {i}) \geq \log (1 / 2. 4 \delta) \min  _ {\lambda \in P _ {\mathcal {X}}} \max  _ {\tilde {\theta} \in \mathcal {C}} \frac {1}{\sum_ {i = 1} ^ {K} \lambda_ {i} K L (\nu_ {\theta^ {*} , i} | | \nu_ {\tilde {\theta} , i})} \\ \geq \log (1 / 2.4\delta)\min_{\lambda \in P_{\mathcal{X}}}\max_{j = 2,\dots ,m}\frac{1}{\sum_{i = 1}^{K}\lambda_{i}KL(\nu_{\theta^{*},i}||\nu_{\theta_{j}(\epsilon,\lambda),i})} \\ \geq \log (1 / 2. 4 \delta) \min _ {\lambda \in P _ {\mathcal {X}}} \max _ {j = 2, \dots , m} \frac {\left\{w _ {j} ^ {\top} A (\lambda) ^ {- 1} w _ {j} \right\} ^ {2}}{\sum_ {i = 1} ^ {K} (w _ {j} ^ {\top} \theta^ {*} + \epsilon) ^ {2} w _ {j} ^ {\top} A (\lambda) ^ {- 1} \lambda_ {i} \frac {x _ {i} x _ {i} ^ {\top}}{2 \sigma_ {x _ {i}} ^ {2}} A (\lambda) ^ {- 1} w _ {j}} \\ \stackrel {(a)} {=} 2 \log (1 / 2. 4 \delta) \min _ {\lambda \in P _ {\mathcal {X}}} \max _ {j = 2, \dots , m} \frac {\left\{w _ {j} ^ {\top} A (\lambda) ^ {- 1} w _ {j} \right\} ^ {2}}{(w _ {j} ^ {\top} \theta^ {*} + \epsilon) ^ {2} w _ {j} ^ {\top} A (\lambda) ^ {- 1} \left(\sum_ {i = 1} ^ {K} \lambda_ {i} \frac {x _ {i} x _ {i} ^ {\top}}{\sigma_ {x _ {i}} ^ {2}}\right) A (\lambda) ^ {- 1} w _ {j}} \\ = 2\log (1 / 2.4\delta)\min_{\lambda \in P_{\mathcal{X}}}\max_{j = 2,\dots ,m}\frac{\left\{w_{j}^{\top}A(\lambda)^{-1}w_{j}\right\}^{2}}{(w_{j}^{\top}\theta^{*} + \epsilon)^{2}}, \\ \end{array}
$$
where  $(a)$  uses the fact that  $\sum_{i=1}^{K} \lambda_i \frac{x_i x_i^\top}{\sigma_{x_i}^2} = A(\lambda)$ . Letting  $\epsilon \to 0$  establishes the result.
E.2 Upper Bound
We now prove Theorem 4.2 by first establishing that Alg. 2 is  $\delta$ -PAC for the transductive linear bandits best-arm identification problem.
E.2.1 Proof of Correctness
Overview of Correctness Proof. Lemma D.1 along with Lemmas E.1 through E.3 are combined to prove that Algorithm 2 is  $\delta$ -PAC for the best-arm identification problem. Lemma E.1 leverages Lemma D.1 and begins by establishing that in any round  $\ell \in \mathbb{N}$ , the estimation error for the difference in mean rewards between  $z \in \mathcal{Z}$  and  $z^{*} = \arg \max_{z \in \mathcal{Z}} z^{\top} \theta^{*}$  is bounded by  $\epsilon_{\ell} = 2^{-\ell}$  with probability  $1 - \delta$  for  $\delta \in (0,1)$ . Lemma E.2 uses this result to prove that starting at round  $\ell$ ,  $z^{*} \in \mathcal{Z}_{\ell + 1}$  with probability  $1 - \delta$ . This establishes that  $z^{*}$  is not eliminated from the set of items we are considering at any step with high probability. Lastly, Lemma E.3 uses Lemma E.1 to show that Algorithm 2 will eventually identify  $z^{*}$ . This is done by proving that Alg. 2 eliminates items with mean rewards that are outside a shrinking radius (halves at every step of the algorithm) of the highest reward,  $z^{*^{\top}} \theta^{*}$ . In other words, with probability  $1 - \delta$  for  $\delta \in (0,1)$ , we prove that items in  $\mathcal{Z}_{\ell}$  with expected rewards that are  $2\epsilon_{\ell}$  away from  $z^{*^{\top}} \theta^{*}$  will not be in  $\mathcal{Z}_{\ell + 1}$ . Since  $\epsilon_{\ell} \to 0$  as  $\ell \to \infty$ , we know that  $\lim_{\ell \to \infty} \mathbb{P}(\mathcal{Z}_{\ell} = \{z^{*}\}) = 1 - \delta$ .
Lemma E.1. With probability at least  $1 - \delta$  for  $\delta \in (0,1)$ ,
$$
| \langle z - z ^ {*}, \widehat {\theta} _ {\ell} - \theta^ {*} \rangle | \leq \epsilon_ {\ell}
$$
for all  $\ell \in \mathbb{N}$  and  $z\in \mathcal{Z}$
Proof. Define  $\sigma_x^2 = x^\top \Sigma^* x$ ,  $\widehat{\theta}_{\ell}$  as the weighted least squares estimator constructed in Algorithm 2 at stage  $\ell$ , and events
$$
\mathcal {E} _ {z, \ell} = | \langle z - z ^ {*}, \widehat {\theta} _ {\ell} - \theta^ {*} \rangle | \leq \epsilon_ {\ell}
$$
$$
\mathcal {J} = \left\{\sigma_ {x} ^ {2} \leq \widehat {\sigma} _ {x} ^ {2} (1 + C _ {\Gamma , \delta}) = \frac {3}{2} \widehat {\sigma} _ {x} ^ {2}, \forall x \in \mathcal {X} \right\}.
$$
Label  $n_{\ell}$  as the number of data points collected during stage  $\ell$ ,  $\Upsilon = \mathrm{diag}(\{\sigma_i^2\}_{i=1}^{n_\ell})$ ,  $\widehat{\Upsilon} = \mathrm{diag}(\{\widehat{\sigma}_i^2\}_{i=1}^{n_\ell})$ ,  $X_{\ell}$  as a matrix whose rows are composed of the arms  $x_{i,\ell}$  drawn during round  $\ell$ ,  $Y_{\ell}$  as the vector of rewards drawn during round  $\ell$ ,  $A_{\ell} := \sum_{i=1}^{n_\ell} \widehat{\sigma}_i^{-2} x_{\ell,i} x_{\ell,i}^\top$ , and  $\widetilde{\Omega} = A_{\ell}^{-1} X_{\ell}^\top \widehat{\Upsilon}^{-1} \Upsilon \widehat{\Upsilon}^{-1} X_{\ell} A_{\ell}^{-1}$  as the weighted least squares estimator's covariance. We first establish that for  $z \in \mathbb{R}^d$
$$
\mathbb {E} \left\{z ^ {\top} (\widehat {\theta} _ {\ell} - \theta^ {*}) \right\} = z ^ {\top} \left\{A _ {t} ^ {- 1} X _ {\ell} \widehat {\Upsilon} ^ {- 1} \mathbb {E} (Y _ {\ell}) - \theta^ {*} \right\} = z ^ {\top} \left(A _ {t} ^ {- 1} X _ {\ell} \widehat {\Upsilon} ^ {- 1} X _ {\ell} \theta^ {*} - \theta^ {*}\right) = 0,
$$
$$
\mathbb {V} \left\{z ^ {\top} (\widehat {\theta} _ {\ell} - \theta^ {*}) \right\} = z ^ {\top} \widetilde {\Omega} z.
$$
Furthermore, defining  $u^{\top} = z^{\top}A_{\ell}^{-1}X_{\ell}^{\top}\widehat{\Upsilon}^{-1 / 2}\in \mathbb{R}^{n_{\ell}}$  and conditioning on event  $\mathcal{J}$
$$
\begin{array}{l} z ^ {\top} \widetilde {\Omega} z = z ^ {\top} A _ {\ell} ^ {- 1} X _ {\ell} ^ {\top} \widehat {\Upsilon} ^ {- 1} \Upsilon \widehat {\Upsilon} ^ {- 1} X _ {\ell} A _ {\ell} ^ {- 1} z \\ = \left(z ^ {\top} A _ {\ell} ^ {- 1} X _ {\ell} ^ {\top} \widehat {\Upsilon} ^ {- 1 / 2}\right) \widehat {\Upsilon} ^ {- 1 / 2} \Upsilon \widehat {\Upsilon} ^ {- 1 / 2} \left(\widehat {\Upsilon} ^ {- 1 / 2} X _ {\ell} A _ {\ell} ^ {- 1} z\right) \\ = u ^ {\top} \hat {\Upsilon} ^ {- 1 / 2} \Upsilon \hat {\Upsilon} ^ {- 1 / 2} u \\ = \sum_ {i = 1} ^ {n _ {\ell}} u _ {i} ^ {2} \frac {\sigma_ {x _ {i}} ^ {2}}{\widehat {\sigma} _ {i} ^ {2}} \\ \stackrel {(a)} {\leq} \frac {3}{2} \sum_ {i = 1} ^ {n _ {\ell}} u _ {i} ^ {2} \\ = \frac {3}{2} z ^ {\top} A _ {\ell} ^ {- 1} X _ {\ell} ^ {\top} \hat {\Upsilon} ^ {- 1} X _ {\ell} A _ {\ell} ^ {- 1} z \\ = \frac {3}{2} z ^ {\top} A _ {\ell} ^ {- 1} z, \\ \end{array}
$$
where  $(a)$  is the result of event  $\mathcal{I}$ . Using this result and Proposition B.1, with probability at least  $1 - \frac{\delta}{4\ell^2|\mathcal{Z}|}$ ,
$$
\begin{array}{l} \mathcal {E} _ {z, \ell} = | \langle z - z ^ {*}, \widehat {\theta} _ {\ell} - \theta \rangle | \leq | | z - z ^ {*} | | _ {\widetilde {\Omega}} \sqrt {2 \log (8 \ell^ {2} | \mathcal {Z} | / \delta)} \\ \leq \frac {3}{2} | | z - z ^ {*} | | _ {A _ {\ell} ^ {- 1}} \sqrt {2 \log (8 \ell^ {2} | \mathcal {Z} | / \delta)}. \\ \end{array}
$$
Conditioned on event  $\mathcal{J}$ , we have with probability  $1 - \frac{\delta}{4\ell^2|\mathcal{Z}|}$
$$
\begin{array}{l} \left| \langle z - z ^ {*}, \widehat {\theta} _ {\ell} - \theta \rangle \right| \leq \frac {3}{2} \| z - z ^ {*} \| \left(\sum_ {x \in \mathcal {V}} \lceil \tau_ {\ell} \lambda_ {\ell , x} \rceil \frac {x x ^ {\top}}{\hat {\sigma} _ {x} ^ {2}}\right) ^ {- 1} \sqrt {2 \log (8 \ell^ {2} | \mathcal {Z} | / \delta)} \\ \leq \frac {3}{2} \| z - z ^ {*} \| \left(\sum_ {x \in \mathcal {X}} \left\lceil \tau_ {\ell} \lambda_ {\ell , x} \right\rceil \frac {x x ^ {\top}}{\hat {\sigma} _ {x} ^ {2}}\right) ^ {- 1} \sqrt {2 \log (8 \ell^ {2} | \mathcal {Z} | / \delta)} \\ \leq \frac {\frac {3}{2} | | z - z ^ {*} | |}{\sqrt {\tau_ {\ell}}} \sqrt {2 \log (8 \ell^ {2} | \mathcal {Z} | / \delta)} \\ = \sqrt {\frac {\frac {3}{2} | | z - z ^ {*} | | ^ {2}}{\left(\sum_ {x \in \mathcal {X}} \lambda_ {\ell , x} \frac {x x ^ {\top}}{\hat {\sigma} _ {x} ^ {2}}\right) ^ {- 1}}} \sqrt {2 \log (8 \ell^ {2} | \mathcal {Z} | / \delta)} \\ \leq \epsilon_ {\ell}. \\ \end{array}
$$
Applying a union bound, we find that
$$
\begin{array}{l} \mathbb {P} \left\{\left(\cap_ {\ell = 1} ^ {\infty} \cap_ {z \in \mathcal {Z} _ {\ell}} \mathcal {E} _ {z, \ell} | \mathcal {J}\right) \cap \mathcal {J} \right\} = 1 - \mathbb {P} \left\{\left(\cup_ {\ell = 1} ^ {\infty} \cup_ {z \in \mathcal {Z} _ {\ell}} \mathcal {E} _ {z, \ell} ^ {c} | \mathcal {J}\right) \cup \mathcal {J} ^ {c} \right\} \\ \leq 1 - \left\{\sum_ {\ell = 1} ^ {\infty} \sum_ {z \in \mathcal {Z} _ {\ell}} \mathbb {P} (\mathcal {E} _ {z, \ell} ^ {c} | \mathcal {J}) + \mathbb {P} (\mathcal {J} ^ {c}) \right\} \\ = 1 - \left(\sum_ {\ell = 1} ^ {\infty} \sum_ {z \in \mathcal {Z} _ {\ell}} \frac {\delta}{4 \ell^ {2} | \mathcal {Z} |} + \frac {\delta}{2}\right) \\ \leq 1 - \delta , \\ \end{array}
$$
which proves the result.
Now we establish that, with probability  $1 - \delta$  for  $\delta \in (0,1)$ , the best arm  $z^{*}$  will not be eliminated from  $\mathcal{Z}_{\ell}$  for any round  $\ell \in \mathbb{N}$ .
Lemma E.2. With probability at least  $1 - \delta$  for  $\delta \in (0,1)$ , for any  $\ell \in \mathbb{N}$  and  $z' \in \mathcal{Z}_{\ell}$ ,
$$
\langle z ^ {\prime} - z ^ {*}, \widehat {\theta} _ {\ell} \rangle \leq \epsilon_ {\ell}.
$$
Proof. We know that with probability  $1 - \delta$  for all  $\ell \in \mathbb{N}$  and  $z' \in \mathcal{Z}_{\ell}$ ,
$$
\begin{array}{l} \langle z ^ {\prime} - z ^ {*}, \widehat {\theta} _ {\ell} \rangle = \langle z ^ {\prime} - z ^ {*}, \widehat {\theta} _ {\ell} - \theta^ {*} \rangle + \langle z ^ {\prime} - z ^ {*}, \theta^ {*} \rangle \\ \leq \left\langle z ^ {\prime} - z ^ {*}, \widehat {\theta} _ {\ell} - \theta^ {*} \right\rangle \\ \leq \epsilon_ {\ell}, \\ \end{array}
$$
where the last inequality follows from Lemma E.1.
Then, we establish that in round  $\ell$ , items in  $\mathcal{Z}_{\ell}$  that have mean rewards farther than  $2\epsilon_{\ell}$  from the highest reward will be eliminated.
Lemma E.3. With probability at least  $1 - \delta$  for  $\delta \in (0,1)$ , for any  $\ell \in \mathbb{N}$  and  $z\in \mathcal{Z}_{\ell}$  such that  $\langle z^{*} - z,\theta^{*}\rangle >2\epsilon_{\ell}$
$$
\max _ {z ^ {\prime} \in \mathcal {Z} _ {\ell}} \langle z ^ {\prime} - z, \widehat {\theta} _ {\ell} \rangle > \epsilon_ {\ell}.
$$
Proof. We know that with probability  $1 - \delta$  for all  $\ell \in \mathbb{N}$
$$
\begin{array}{l} \max  _ {z ^ {\prime} \in \mathcal {Z} _ {\ell}} \langle z ^ {\prime} - z, \widehat {\theta} _ {\ell} \rangle \geq \langle z ^ {*} - z, \widehat {\theta} _ {\ell} \rangle \\ = \left\langle z ^ {*} - z, \widehat {\theta} _ {\ell} - \theta^ {*} \right\rangle + \left\langle z ^ {*} - z, \theta^ {*} \right\rangle \\ \stackrel {(a)} {>} - \epsilon_ {\ell} + 2 \epsilon_ {\ell} \\ = \epsilon_ {\ell}, \\ \end{array}
$$
where  $(a)$  follows from Lemma E.1.
E.2.2 Proof of Sample Complexity
Overview of Sample Complexity Proof. In this section, we prove an upper bound on the sample complexity of Algorithm 2 for transductive best-arm identification with linear bandits. We begin by establishing some basic facts about the partial ordering of Hermetian positive definite matrices in Lemmas E.4 and E.5. Let  $A$  and  $B$  be Hermitian positive definite matrices in  $\mathbb{R}^{d \times d}$  and define  $A \succeq B$  if  $(A - B)$  is positive definite.
Lemma E.4. Let  $A$  and  $B$  be Hermetian Positive Definite matrices in  $\mathbb{R}^{d\times d}$
$$
A \succeq B \iff I \succeq A ^ {- 1 / 2} B A ^ {- 1 / 2}.
$$
Proof. For any  $x \in \mathbb{R}^d$  we define  $y = A^{1/2}x$  and
$$
\begin{array}{l} x ^ {\top} A x = x ^ {\top} A ^ {1 / 2} A ^ {- 1 / 2} A A ^ {- 1 / 2} A ^ {1 / 2} x \\ = y ^ {\top} I y \\ = y ^ {\top} A ^ {- 1 / 2} A A ^ {- 1 / 2} y \\ \geq x ^ {\top} B x \\ = y ^ {\top} A ^ {- 1 / 2} B A ^ {- 1 / 2} y \\ \end{array}
$$
Lemma E.5. Let  $A$  and  $B$  be Hermitian positive definite matrices in  $\mathbb{R}^{d\times d}$
$$
A \succeq B \iff B ^ {- 1} \succeq A ^ {- 1}.
$$
Proof. We use Lemma E.4 twice in the following proof.
$$
\begin{array}{l} A \succeq B \iff I \succeq A ^ {- 1 / 2} B A ^ {- 1 / 2} \\ \iff I \succeq B ^ {1 / 2} A ^ {- 1 / 2} A ^ {- 1 / 2} B ^ {1 / 2} \\ \iff I \succeq B ^ {1 / 2} A ^ {- 1} B ^ {1 / 2} \\ \iff B ^ {- 1} \succeq A ^ {- 1}. \\ \end{array}
$$
where line 2 follows because  $A^{-1/2}BA^{-1/2}$  is similar to  $B^{1/2}A^{-1/2}A^{-1/2}B^{1/2}$  and both are Hermetian positive definite.
We now use Lemmas E.4 and E.5 in our problem context. For  $\mathcal{V} \subset \mathbb{R}^d$ , define  $g: (\mathcal{V}, P_{\mathcal{X}}) \to \mathbb{R}$  such that  $g(v, \lambda) = \|v\|_{(\sum_x \lambda_x xx^\top / \hat{\sigma}_x^2)^{-1}}$ ,  $f: (\mathcal{V}, P_{\mathcal{X}}) \to \mathbb{R}$  such that  $f(v, \lambda) = \|v\|_{(\sum_x \lambda_x xx^\top / \sigma_x^2)^{-1}}$  and
$$
q^{*}(\mathcal{V}) = \min_{\lambda \in P_{\mathcal{X}}}\max_{v\in \mathcal{V}}f(v,\lambda),\quad q(\mathcal{V}) = \min_{\lambda \in P_{\mathcal{X}}}\max_{v\in \mathcal{V}}g(v,\lambda).
$$
Note that the function,  $q$ , is also defined and leveraged in Algorithm 2. Lemma E.6 establishes a multiplicative bound of the form
$$
(1 - C _ {\Gamma , \delta}) g (v, \lambda) \leq f (v, \lambda) \leq (1 + C _ {\Gamma , \delta}) g (v, \lambda)
$$
using Lemmas D.1 and E.5.
Lemma E.6. Define  $\widehat{\sigma}_x^2 = \min \left\{\max \left\{x^\top \widehat{\Sigma}_\Gamma x, \sigma_{\min}^2\right\}, \sigma_{\max}^2\right\}$ . For a given  $\lambda$ , define
$$
\Upsilon = \mathrm {d i a g} (\{\sigma_ {x} ^ {2} \} _ {x \in \mathcal {X}}) \times \mathrm {d i a g} (\{1 / \lambda_ {x} \} _ {x \in \mathcal {X}}), \widehat {\Upsilon} ^ {- 1} = \mathrm {d i a g} (\{\widehat {\sigma} _ {x} ^ {2} \} _ {x \in \mathcal {X}}) \times \mathrm {d i a g} (\{1 / \lambda_ {x} \} _ {x \in \mathcal {X}}).
$$
Assume  $C_{\Gamma, \delta} < 1$  and  $v \in \mathbb{R}^d$ , then with probability  $1 - \delta / 2$  for  $\delta \in (0, 1)$  the following holds true:
$$
\begin{array}{l} (1 - C _ {\Gamma , \delta}) v ^ {\top} \left(X ^ {\top} \hat {\Upsilon} ^ {- 1} X\right) ^ {- 1} v \leq v ^ {\top} \left(X ^ {\top} \Upsilon^ {- 1} X\right) ^ {- 1} v \\ \leq \left(1 + C _ {\Gamma , \delta}\right) v ^ {\top} \left(X ^ {\top} \hat {\Upsilon} ^ {- 1} X\right) ^ {- 1} v, \\ \end{array}
$$
$$
(1 - C _ {\Gamma , \delta}) g (v, \lambda) \leq f (v, \lambda) \leq (1 + C _ {\Gamma , \delta}) g (v, \lambda).
$$
Proof. Using Lemma D.1, we establish that
$$
\begin{array}{l} v ^ {\top} X ^ {\top} \widehat {Y} ^ {- 1} X v = v ^ {\top} X ^ {\top} \widehat {Y} ^ {- 1} \Upsilon \Upsilon^ {- 1} X v \\ = v ^ {\top} X ^ {\top} \Upsilon^ {- 1 / 2} \hat {\Upsilon} ^ {- 1} \Upsilon \Upsilon^ {- 1 / 2} X v \\ \leq \left(1 + C _ {\Gamma , \delta}\right) v ^ {\top} X ^ {\top} \Upsilon^ {- 1} X v \\ \end{array}
$$
$$
\begin{array}{l} v ^ {\top} X ^ {\top} \hat {\Upsilon} ^ {- 1} X v = v ^ {\top} X ^ {\top} \hat {\Upsilon} ^ {- 1} \Upsilon \Upsilon^ {- 1} X v \\ = v ^ {\top} X ^ {\top} \Upsilon^ {- 1 / 2} \hat {\Upsilon} ^ {- 1} \Upsilon \Upsilon^ {- 1 / 2} X v \\ \geq \left(1 - C _ {\Gamma , \delta}\right) v ^ {\top} X ^ {\top} \Upsilon^ {- 1} X v. \\ \end{array}
$$
Leveraging the partial ordering of Hermitian positive definite matrices and Lemma E.5, we can then represent these inequalities as
$$
\begin{array}{l} \left(1 - C _ {\Gamma , \delta}\right) X ^ {\top} \Upsilon^ {- 1} X \preceq X ^ {\top} \widehat {\Upsilon} ^ {- 1} X \preceq \left(1 + C _ {\Gamma , \delta}\right) X ^ {\top} \Upsilon^ {- 1} X \iff \\ \frac {1}{1 - C _ {\Gamma , \delta}} \left(X ^ {\top} \Upsilon^ {- 1} X\right) ^ {- 1} \succeq \left(X ^ {\top} \widehat {\Upsilon} ^ {- 1} X\right) ^ {- 1} \succeq \frac {1}{1 + C _ {\Gamma , \delta}} \left(X ^ {\top} \Upsilon^ {- 1} X\right) ^ {- 1}. \\ \end{array}
$$
$$
\left(X ^ {\top} \Upsilon^ {- 1} X\right) ^ {- 1} \succeq \left(1 - C _ {\Gamma , \delta}\right) \left(X ^ {\top} \widehat {\Upsilon} ^ {- 1} X\right) ^ {- 1}
$$
$$
\left(X ^ {\top} \Upsilon^ {- 1} X\right) ^ {- 1} \preceq \left(1 + C _ {\Gamma , \delta}\right) \left(X ^ {\top} \widehat {\Upsilon} ^ {- 1} X\right) ^ {- 1}.
$$
By combining these inequalities we arrive at the result.
We now want to use the multiplicative bound of Lemma E.6 to define the relationship between  $q(\mathcal{V})$  and  $q^{*}(\mathcal{V})$  and eventually relate the sample complexity upper bound of Alg. 2 to the lower bound established in Theorem 4.1. In order to accomplish this goal, we prove some general results concerning minimax problems in Lemmas E.7 and E.8.
Lemma E.7. For  $c > 0$ ,  $v \in \mathbb{R}^d$  and  $\lambda \in P_{\mathcal{X}}$ , if
$$
(1 - c) g (v, \lambda) \leq f (v, \lambda) \leq (1 + c) g (v, \lambda),
$$
$$
\begin{array}{l} (1 - c) \max  _ {v \in \mathcal {V}} g (v, \lambda) \leq \max  _ {v \in \mathcal {V}} f (v, \lambda) \\ \leq (1 + c) \max  _ {v \in \mathcal {V}} g (v, \lambda). \\ \end{array}
$$
Proof. For a given  $\lambda \in P_{\mathcal{X}}$ , define  $v^{\dagger}(\lambda) = \arg \max_{v\in \mathcal{V}}g(v,\lambda)$  and  $v^{*}(\lambda) = \arg \max_{v\in \mathcal{V}}f(v,\lambda)$ , then
$$
\begin{array}{l} (1 - c) g \left\{v ^ {\dagger} (\lambda), \lambda \right\} \leq f \left\{v ^ {\dagger} (\lambda), \lambda \right\} \\ \leq f \left\{v ^ {*} (\lambda), \lambda \right\} \\ \leq (1 + c) g \left\{v ^ {*} (\lambda), \lambda \right\} \\ \leq (1 + c) g \left\{v ^ {\dagger} (\lambda), \lambda \right\}. \\ \end{array}
$$
Lemma E.8. For  $c > 0$ ,  $v \in \mathbb{R}^d$  and  $\lambda \in P_{\mathcal{X}}$ , if
$$
\begin{array}{l} (1 - c) \min  _ {\lambda \in P _ {\mathcal {X}}} \max  _ {v \in \mathcal {V}} g (v, \lambda) \leq \min  _ {\lambda \in P _ {\mathcal {X}}} \max  _ {v \in \mathcal {V}} f (v, \lambda) \\ \leq (1 + c) \min  _ {\lambda \in P _ {\mathcal {X}}} \max  _ {v \in \mathcal {V}} g (v, \lambda). \\ \end{array}
$$
Proof. For a given  $\lambda \in P_{\mathcal{X}}$ , define  $v^{\dagger}(\lambda) = \arg \max_{v\in \mathcal{V}}g(v,\lambda)$ ,  $v^{*}(\lambda) = \arg \max_{v\in \mathcal{V}}f(v,\lambda)$ ,  $\lambda^{*} = \arg \min_{\lambda \in P_{\mathcal{X}}}\max_{v\in \mathcal{V}}f(v,\lambda)$  and  $\lambda^{\dagger} = \arg \min_{\lambda \in P_{\mathcal{X}}}\max_{v\in \mathcal{V}}g(v,\lambda)$ , then
$$
\begin{array}{l} (1 - c) g \left\{v ^ {\dagger} \left(\lambda^ {\dagger}\right), \lambda^ {\dagger} \right\} \leq (1 - c) g \left\{v ^ {\dagger} \left(\lambda^ {*}\right), \lambda^ {*} \right\} \\ \stackrel {(a)} {\leq} f \left\{v ^ {*} \left(\lambda^ {*}\right), \lambda^ {*} \right\} \\ \leq f \left\{v ^ {*} \left(\lambda^ {\dagger}\right), \lambda^ {\dagger} \right\} \\ \leq (1 + c) g \left\{v ^ {*} \left(\lambda^ {\dagger}\right), \lambda^ {\dagger} \right\} \\ \stackrel {(b)} {\leq} (1 + c) g \left\{v ^ {\dagger} \left(\lambda^ {\dagger}\right), \lambda^ {\dagger} \right\} \\ \end{array}
$$
where lines  $(a)$  and  $(b)$  follows from Lemma E.7.
Leveraging Lemma E.8, we bound  $q^{*}(\mathcal{V})$  in terms of  $q(\mathcal{V})$  in Lemma E.9.
Lemma E.9. With probability  $1 - \delta /2$  for  $\delta \in (0,1)$
$$
(1 - C _ {\Gamma , \delta}) q (\mathcal {V}) \leq q ^ {*} (\mathcal {V}) \leq (1 + C _ {\Gamma , \delta}) q (\mathcal {V}).
$$
Proof. From Lemma E.6, we know that
According to Lemma E.8, this implies
$$
(1 - C_{\Gamma ,\delta})\min_{\lambda \in P_{\mathcal{X}}}\max_{v\in \mathcal{V}}g(v,\lambda)\leq \min_{\lambda \in P_{\mathcal{X}}}\max_{v\in \mathcal{V}}f(v,\lambda)\leq (1 + C_{\Gamma ,\delta})\min_{\lambda \in P_{\mathcal{X}}}\max_{v\in \mathcal{V}}g(v,\lambda).
$$
In round  $\ell$  of Algorithm 2, we sample arms in  $\mathcal{X}$  according to a given design,  $\widehat{\lambda}_{\ell}$ . Although other rounding procedures are possible (see Proposition A.1), Theorem 4.2 assumes that we take  $\lceil \tau_{\ell} \widehat{\lambda}_{\ell, x} \rceil$  samples of arm  $x \in \mathcal{X}$  to accomplish this goal. This results in more than  $\tau_{\ell}$  samples being taken in round  $\ell$ . We define the sparsity of any  $\widehat{\lambda}_{\ell}$  design using Caratheodory's Theorem to motivate the number of extra samples needed by sampling in this manner.
Lemma E.10. [Appendix 8, [44]] In Algorithm 2, the support of  $\widehat{\lambda}_{\ell}$  for all  $\ell \in \mathbb{N}$  is  $d(d + 1) / 2 + 1$
Proof. Any design matrix in  $\mathbb{R}^{d\times d}$  is symmetric and can consequently be expressed as a point in  $\mathbb{R}^M$ ,  $M = d(d + 1)/2$ . The design matrices leveraged in this paper belong to a convex hull of a subset of points since they are a convex combination of  $xx^\top$  for  $x \in \mathcal{X}$ . According to Caratheodory's theorem, any point in the convex hull of any subset of points in  $\mathbb{R}^D$  can be defined as a convex combination of  $D + 1$  points. The optimal experimental designs in this paper can therefore be represented using only  $M + 1$  points [44]. This sparsity holds regardless of the design's form and so applies to both the homoskedastic and heteroskedastic variance settings.
Finally, we connect the sample complexity upper bound of Algorithm 2 to the lower bound for  $\delta$ -PAC algorithms established in Theorem 4.2 for transductive best-arm identification.
Proof of Theorem 4.2 for transductive best-arm identification. Assume that  $\max_{z\in \mathcal{Z}_\ell}|\langle z-z^*,\theta^*\rangle |\leq 2$ . Define  $\mathcal{S}_{\ell} = \{z\in \mathcal{Z}_{\ell}:\langle z^{*} - z,\theta^{*}\rangle \leq 4\epsilon_{\ell}\}$ . Note that by assumption  $\mathcal{Z} = \mathcal{Z}_1 = S_1$ . Lemma E.3 implies that with probability at least  $1 - \delta$  for  $\delta \in (0,1)$ , we know  $\mathcal{Z}_{\ell}\subset \mathcal{S}_{\ell}$  for all  $\ell \in \mathbb{N}$ . This means that
$$
\begin{array}{l} q(\mathcal{Y}(\mathcal{Z}_{\ell})) = \inf_{\lambda \in P_{\mathcal{X}}}\max_{z,z^{\prime}\in \mathcal{Z}_{\ell}}||z - z^{\prime}||_{(\sum_{x\in \mathcal{X}}\lambda_{x}\frac{x x^{\top}}{\delta_{x}^{2}})^{-1}}^{2} \\ \leq \inf _ {\lambda \in P _ {\mathcal {X}}} \max _ {z, z ^ {\prime} \in S _ {\ell}} | | z - z ^ {\prime} | | _ {(\sum_ {x \in \mathcal {X}} \lambda_ {x} \frac {x x ^ {\top}}{\hat {\sigma} _ {x} ^ {2}})} ^ {2} - 1 \\ = q \left(\mathcal {S} _ {\ell}\right). \\ \end{array}
$$
For  $\ell \geq \lceil \log_2(4\Delta^{-1})\rceil$  , we know that  $S_{\ell} = \{z^{*}\}$  . Thus, the sample size needed to identify  $z^{*}$  is upper bounded as follows.
$$
\begin{array}{l} \sum_ {\ell = 1} ^ {\lceil \log_ {2} (4 \Delta^ {- 1}) \rceil} \sum_ {x \in \mathcal {X}} \lceil \tau_ {\ell} \widehat {\lambda} _ {\ell , x} \rceil \stackrel {(a)} {=} \sum_ {\ell = 1} ^ {\lceil \log_ {2} (4 \Delta^ {- 1}) \rceil} \left\{\frac {d (d + 1)}{2} + \tau_ {\ell} \right\} \\ = \sum_ {\ell = 1} ^ {\lceil \log_ {2} (4 \Delta^ {- 1}) \rceil} \left[ \frac {d (d + 1)}{2} + 2 \left(\frac {3}{2}\right) \epsilon_ {\ell} ^ {- 2} q \left\{\mathcal {Y} (\mathcal {Z} _ {\ell}) \right\} \log (8 \ell^ {2} | \mathcal {Z} | / \delta) \right] \\ \leq \frac {d (d + 1)}{2} \left[ \log_ {2} \left(4 \Delta^ {- 1}\right) \right] + \sum_ {\ell = 1} ^ {\lceil \log_ {2} \left(4 \Delta^ {- 1}\right) \rceil} 3 \epsilon_ {\ell} ^ {- 2} q (\mathcal {S} _ {\ell}) \log \left(8 \ell^ {2} | \mathcal {Z} | / \delta\right) \\ \leq \frac {d (d + 1)}{2} \lceil \log_ {2} (4 \Delta^ {- 1}) \rceil \\ { + } { 3 \log \left\{ \frac { 8 \lceil \log _ { 2 } ( 4 \Delta ^ { - 1 } ) \rceil ^ { 2 } | \mathcal { Z } | } { \delta } \right\} ^ { \lceil \log _ { 2 } ( 4 \Delta ^ { - 1 } ) \rceil } \sum _ { \ell = 1 } 2 ^ { 2 \ell } q ( \mathcal { S } _ { \ell } ) , } \\ \end{array}
$$
where  $(a)$  follows from Lemma E.10.
We relate this quantity to the lower bound in Theorem 4.1 below. With probability  $1 - \delta$ ,