SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {P} \left(\boldsymbol {x} _ {t} ^ {[ b ]} \in \mathcal {X} \setminus S _ {t} ^ {\prime} | \mathcal {F} _ {t - 1}, \mathcal {F} _ {t - 1} ^ {\prime}\right) \geq \mathbb {P} \left(h _ {t} ^ {[ b ]} (\boldsymbol {x} ^ {*}) > h _ {t} ^ {[ b ]} (\boldsymbol {x}), \forall \boldsymbol {x} \in S _ {t} ^ {\prime} | \mathcal {F} _ {t - 1}, \mathcal {F} _ {t - 1} ^ {\prime}\right) \\ \geq \mathbb {P} \left(h _ {t} ^ {[ b ]} \left(\boldsymbol {x} ^ {*}\right) > h \left(\boldsymbol {x} ^ {*}\right) \mid \mathcal {F} _ {t - 1}, \mathcal {F} _ {t - 1}\right) - \mathbb {P} \left(\overline {{E ^ {f _ {t}} (t)}} \cup \overline {{E ^ {g _ {t}} (t)}} \mid \mathcal {F} _ {t - 1}, \mathcal {F} _ {t - 1} ^ {\prime}\right) \\ \geq p ^ {2} - 1 / t ^ {2}, \tag {34} \\ \end{array}
$$
which holds for all  $b \in [b_t]$ . The second inequality follows from considering the following two events separately:  $E^{f_t}(t) \cap E^{g_t}(t)$  and  $\overline{E^{f_t}(t) \cap E^{g_t}(t)}$ , and the last inequality follows since  $\mathbb{P}\left(\overline{E^{f_t}(t)} \cup \overline{E^{g_t}(t)} | \mathcal{F}_{t-1}, \mathcal{F}_{t-1}'\right) \leq \mathbb{P}\left(\overline{E^{f_t}(t)} | \mathcal{F}_{t-1}, \mathcal{F}_{t-1}'\right) + \mathbb{P}\left(\overline{E^{g_t}(t)} | \mathcal{F}_{t-1}, \mathcal{F}_{t-1}'\right) = 1 / (2t^2) + 1 / (2t^2) = 1 / t^2.$
Next, we will need the following Lemma which is a counterpart to Lemma B.6.
Lemma G.6. Define  $C_2 = \frac{2}{\log(1 + \lambda^{-1})}$ . Denote all  $\tau_{t-1}$  observed empirical means from iterations (batches) 1 to  $t - 1$  as  $\pmb{y}_{1:t-1}$ , and the  $b_t$  observed empirical means in the  $t^th$  batch as  $\pmb{y}_t$ . Also denote all  $\tau_{t-1}$  observed noise variances from iterations (batches) 1 to  $t - 1$  as  $\pmb{y}_{1:t-1}'$ , and the  $b_t$  observed noise variances in the  $t^th$  batch as  $\pmb{y}_t'$ . Choose  $C$  as an absolute constant such that  $\max_{A \subset \mathcal{X}, |A| \leq \mathbb{B}} \mathbb{I}\left(f; \pmb{y}_A | \pmb{y}_{1:t-1}\right) \leq C, \forall t \geq 1$  and  $\max_{A \subset \mathcal{X}, |A| \leq \mathbb{B}} \mathbb{I}\left(g; \pmb{y}_A' | \pmb{y}_{1:t-1}'\right) \leq C, \forall t \geq 1$ . Then we have that
$$
\sum_ {b = 1} ^ {b _ {t}} \sigma_ {t - 1} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) \leq e ^ {C} \sqrt {C _ {2} b _ {t} \mathbb {I} (f ; \boldsymbol {y} _ {t} | \boldsymbol {y} _ {1 : t - 1})},
$$
$$
\sum_ {b = 1} ^ {b _ {t}} \sigma_ {t - 1} ^ {\prime} (\boldsymbol {x} _ {t} ^ {[ b ]}) \leq e ^ {C} \sqrt {C _ {2} b _ {t} \mathbb {I} (g ; \boldsymbol {y} _ {t} ^ {\prime} | \boldsymbol {y} _ {1 : t - 1} ^ {\prime})}.
$$
Proof. Note that we have assumed that both  $f$  and  $g$  are associated with the SE kernels  $k$  and  $k'$ , respectively. Denote the length scales for  $k$  and  $k'$  by  $\theta$  and  $\theta'$  respectively. Note that the maximum information gain is decreasing in the length scale, i.e., a smaller length scale leads to a larger maximum information gain [1]. Therefore, we can run the initialization stage via uncertainty sampling to observe  $y_{\mathrm{init}}$  using the kernel with a smaller length scale.
For example, if  $\theta < \theta'$ , we use the kernel  $k$  to run the uncertainty sampling algorithm for  $T_{\mathrm{init}}$  iterations to collect the initial set of inputs  $D_{\mathrm{init}}$ , such that we can guarantee that
$$
\max  _ {A \subset \mathcal {X}, | A | \leq \mathbb {B}} \mathbb {I} (f; \boldsymbol {y} _ {A} | \boldsymbol {y} _ {1: t - 1}) \leq \max  _ {A \subset \mathcal {X}, | A | \leq \mathbb {B}} \mathbb {I} (f; \boldsymbol {y} _ {A} | \boldsymbol {y} _ {\text {i n i t}}) \leq C, \forall t \geq 1, \tag {35}
$$
where the first inequality follows from the submodularity of conditional information gain, and the second inequality is a consequence of Lemma 4 of [19]. Note that given the same set of initial inputs  $D_{\mathrm{init}}$ , the maximum conditional information gains  $\max_{A \subset \mathcal{X}, |A| \leq \mathbb{B}} \mathbb{I}\left(f; \boldsymbol{y}_A | \boldsymbol{y}_{\mathrm{init}}\right)$  and  $\max_{A \subset \mathcal{X}, |A| \leq \mathbb{B}} \mathbb{I}\left(g; \boldsymbol{y}_A' | \boldsymbol{y}_{\mathrm{init}}'\right)$  differ by only the lengthscales of the kernels  $k$  and  $k'$ , denoted as  $\theta$  and  $\theta'$ , respectively. Therefore, since we have assumed that  $\theta < \theta'$  in the discussion here, we have that
$$
\max  _ {A \subset \mathcal {X}, | A | \leq \mathbb {B}} \mathbb {I} (g; \boldsymbol {y} _ {A} ^ {\prime} \mid \boldsymbol {y} _ {\text {i n i t}} ^ {\prime}) <   \max  _ {A \subset \mathcal {X}, | A | \leq \mathbb {B}} \mathbb {I} (f; \boldsymbol {y} _ {A} \mid \boldsymbol {y} _ {\text {i n i t}}). \tag {36}
$$
This further tells us that
$$
\max  _ {A \subset \mathcal {X}, | A | \leq \mathbb {B}} \mathbb {I} (g; \boldsymbol {y} _ {A} ^ {\prime} \mid \boldsymbol {y} _ {1: t - 1} ^ {\prime}) \leq \max  _ {A \subset \mathcal {X}, | A | \leq \mathbb {B}} \mathbb {I} (g; \boldsymbol {y} _ {A} ^ {\prime} \mid \boldsymbol {y} _ {\text {i n i t}} ^ {\prime}) \leq \max  _ {A \subset \mathcal {X}, | A | \leq \mathbb {B}} \mathbb {I} (f; \boldsymbol {y} _ {A} \mid \boldsymbol {y} _ {\text {i n i t}}) \leq C, \forall t \geq 1. \tag {37}
$$
Subsequently, the proof is completed by applying the proof techniques of Lemma B.6 to  $\sigma_{t - 1}$  and  $\sigma_{t - 1}^{\prime}$  separately.
Lemma G.7. Define  $\widetilde{B} = \omega B + (1 - \omega)B'$ . For any filtrations  $\mathcal{F}_{t-1}$  and  $\mathcal{F}_{t-1}'$ , conditioned on the events  $E^f(t)$  and  $E^g(t)$ , we have that
$$
\begin{array}{l} \mathbb {E} \left[ \min  _ {b \in [ b _ {t} ]} r _ {t} ^ {[ b ]} \mid \mathcal {F} _ {t - 1} \right] \leq e ^ {C} \left(1 + \frac {2 8}{p ^ {2}}\right) \left[ \omega c _ {t} \mathbb {E} \left[ \frac {1}{b _ {t}} \sqrt {C _ {2} b _ {t} \mathbb {I} (f ; \boldsymbol {y} _ {t} | \boldsymbol {y} _ {1 : t - 1})} \mid \mathcal {F} _ {t - 1}, \mathcal {F} _ {t - 1} ^ {\prime} \right] \right. \\ \left. + (1 - \omega) c _ {t} ^ {\prime} \mathbb {E} \left[ \frac {1}{b _ {t}} \sqrt {C _ {2} b _ {t} \mathbb {I} \left(g ; \boldsymbol {y} _ {t} ^ {\prime} \mid \boldsymbol {y} _ {1 : t - 1} ^ {\prime}\right)} \mid \mathcal {F} _ {t - 1}, \mathcal {F} _ {t - 1} ^ {\prime} \right] \right] + \frac {2 \widetilde {B}}{t ^ {2}}, \\ \end{array}
$$
in which  $r_t^{[b]} = h(\pmb{x}^*) - h(\pmb{x}_t^{[b]})$ .
Proof. To begin with, we define  $\overline{x}_t$  as the unsaturated input at iteration  $t$  with the smallest weighted posterior standard deviation:
$$
\overline {{\boldsymbol {x}}} _ {t} = \arg \min  _ {\boldsymbol {x} \in \mathcal {X} \backslash S _ {t} ^ {\prime}} \left(\omega c _ {t} \sigma_ {t - 1} (\boldsymbol {x}) + (1 - \omega) c _ {t} ^ {\prime} \sigma_ {t - 1} ^ {\prime} (\boldsymbol {x})\right). \tag {38}
$$
Following this definition, if both  $E^{f}(t)$  and  $E^{g}(t)$  hold  $\forall b \in [b_{t}]$ , then we have that
$$
\begin{array}{l} \mathbb {E} \left[ \omega c _ {t} \sigma_ {t - 1} (\pmb {x} _ {t} ^ {[ b ]}) + (1 - \omega) c _ {t} ^ {\prime} \sigma_ {t - 1} ^ {\prime} (\pmb {x} _ {t} ^ {[ b ]}) | \mathcal {F} _ {t - 1}, \mathcal {F} _ {t - 1} ^ {\prime} \right] \\ \geq \mathbb {E} \left[ \omega c _ {t} \sigma_ {t - 1} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) + (1 - \omega) c _ {t} ^ {\prime} \sigma_ {t - 1} ^ {\prime} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) \mid \mathcal {F} _ {t - 1}, \mathcal {F} _ {t - 1} ^ {\prime}, \boldsymbol {x} _ {t} ^ {[ b ]} \in \mathcal {X} \setminus S _ {t} ^ {\prime} \right] \mathbb {P} \left(\boldsymbol {x} _ {t} ^ {[ b ]} \in \mathcal {X} \setminus S _ {t} ^ {\prime} \mid \mathcal {F} _ {t - 1}, \mathcal {F} _ {t - 1} ^ {\prime}\right) \\ \geq \left[ \omega c _ {t} \sigma_ {t - 1} (\bar {\boldsymbol {x}} _ {t}) + (1 - \omega) c _ {t} ^ {\prime} \sigma_ {t - 1} ^ {\prime} (\bar {\boldsymbol {x}} _ {t}) \right] \left(p ^ {2} - 1 / t ^ {2}\right) \tag {39} \\ \end{array}
$$
Now we condition on all events  $E^{f}(t), E^{ft}(t), E^{g}(t)$  and  $E^{gt}(t)$ , and analyze the instantaneous regret as:
$$
\begin{array}{l} \min  _ {b \in \left[ b _ {t} \right]} r _ {t} ^ {[ b ]} \leq \frac {1}{b _ {t}} \sum_ {b = 1} ^ {b _ {t}} r _ {t} ^ {[ b ]} = \frac {1}{b _ {t}} \sum_ {b = 1} ^ {b _ {t}} \Delta \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) = \frac {1}{b _ {t}} \sum_ {b = 1} ^ {b _ {t}} \left[ h \left(\boldsymbol {x} ^ {*}\right) - h (\overline {{\boldsymbol {x}}} _ {t}) + h (\overline {{\boldsymbol {x}}} _ {t}) - h \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) \right] \\ \leq \frac {1}{b _ {t}} \sum_ {b = 1} ^ {b _ {t}} \left[ \Delta (\overline {{\boldsymbol {x}}} _ {t}) + h _ {t} ^ {[ b ]} (\overline {{\boldsymbol {x}}} _ {t}) + \omega c _ {t} \sigma_ {t - 1} (\overline {{\boldsymbol {x}}} _ {t}) + (1 - \omega) c _ {t} ^ {\prime} \sigma_ {t - 1} ^ {\prime} (\overline {{\boldsymbol {x}}} _ {t}) \right. \\ \left. \right.\left. - h _ {t} ^ {[ b ]} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) + \omega c _ {t} \sigma_ {t - 1} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) + (1 - \omega) c _ {t} ^ {\prime} \sigma_ {t - 1} ^ {\prime} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right)\right] \\ \leq \frac {1}{b _ {t}} \sum_ {b = 1} ^ {b _ {t}} \left[ \omega c _ {t} \sigma_ {t - 1} (\bar {\boldsymbol {x}} _ {t}) + (1 - \omega) c _ {t} ^ {\prime} \sigma_ {t - 1} ^ {\prime} (\bar {\boldsymbol {x}} _ {t}) \right. \\ + h _ {t} ^ {[ b ]} (\overline {{\boldsymbol {x}}} _ {t}) + \omega c _ {t} \sigma_ {t - 1} (\overline {{\boldsymbol {x}}} _ {t}) + (1 - \omega) c _ {t} ^ {\prime} \sigma_ {t - 1} ^ {\prime} (\overline {{\boldsymbol {x}}} _ {t}) \\ \left. - h _ {t} ^ {[ b ]} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) + \omega c _ {t} \sigma_ {t - 1} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) + (1 - \omega) c _ {t} ^ {\prime} \sigma_ {t - 1} ^ {\prime} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) \right] \\ \leq \frac {1}{b _ {t}} \sum_ {b = 1} ^ {b _ {t}} \left[ \omega c _ {t} \left(2 \sigma_ {t - 1} (\overline {{\boldsymbol {x}}} _ {t}) + \sigma_ {t - 1} (\boldsymbol {x} _ {t} ^ {[ b ]})\right) + (1 - \omega) c _ {t} ^ {\prime} \left(2 \sigma_ {t - 1} ^ {\prime} (\overline {{\boldsymbol {x}}} _ {t}) + \sigma_ {t - 1} ^ {\prime} (\boldsymbol {x} _ {t} ^ {[ b ]})\right) \right. \\ \left. + h _ {t} ^ {[ b ]} \left(\overline {{\boldsymbol {x}}} _ {t}\right) - h _ {t} ^ {[ b ]} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) \right] \\ \leq \frac {1}{b _ {t}} \sum_ {b = 1} ^ {b _ {t}} \left[ \omega c _ {t} \left(2 \sigma_ {t - 1} (\overline {{\boldsymbol {x}}} _ {t}) + \sigma_ {t - 1} (\boldsymbol {x} _ {t} ^ {[ b ]})\right) + (1 - \omega) c _ {t} ^ {\prime} \left(2 \sigma_ {t - 1} ^ {\prime} (\overline {{\boldsymbol {x}}} _ {t}) + \sigma_ {t - 1} ^ {\prime} (\boldsymbol {x} _ {t} ^ {[ b ]})\right) \right] \\ = \frac {1}{b _ {t}} \sum_ {b = 1} ^ {b _ {t}} \left[ 2 \left(\omega c _ {t} \sigma_ {t - 1} (\overline {{\boldsymbol {x}}} _ {t}) + (1 - \omega) c _ {t} ^ {\prime} \sigma_ {t - 1} ^ {\prime} (\overline {{\boldsymbol {x}}} _ {t})\right) + \omega c _ {t} \sigma_ {t - 1} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) + (1 - \omega) c _ {t} ^ {\prime} \sigma_ {t - 1} ^ {\prime} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) \right], \tag {40} \\ \end{array}
$$
in which the second inequality follows from Lemmas B.1, B.2, G.1 and G.2, the third inequality results from Definition G.3 and the fact that  $\overline{\boldsymbol{x}}_t$  is unsaturated, and the last inequality follows from the way in which  $\boldsymbol{x}_t^{[b]}$  is selected:  $\boldsymbol{x}_t^{[b]} = \arg \max_{\boldsymbol{x} \in \mathcal{X}} h_t^{[b]}(\boldsymbol{x})$ .
The second inequality follows from equation (39), the third inequality follows since  $1 / (p^2 - 1 / t^2) \leq 14 / p^2$ , and the last inequality makes use of Lemma G.6.
Definition G.8. Define  $Y_0 = 0$ , and for all  $t = 1, \ldots, T$
$$
\overline {{r}} _ {t} = \mathbb {I} \{E ^ {f} (t) \cap E ^ {g} (t) \} \min  _ {b \in [ b _ {t} ]} r _ {t} ^ {[ b ]},
$$
$$
\begin{array}{l} X _ {t} = \bar {r} _ {t} - e ^ {C} \left(1 + \frac {2 8}{p ^ {2}}\right) \left[ \omega c _ {t} \frac {1}{b _ {t}} \sqrt {C _ {2} b _ {t} \mathbb {I} (f ; \boldsymbol {y} _ {t} | \boldsymbol {y} _ {1 : t - 1})} + (1 - \omega) c _ {t} ^ {\prime} \frac {1}{b _ {t}} \sqrt {C _ {2} b _ {t} \mathbb {I} (g ; \boldsymbol {y} _ {t} ^ {\prime} | \boldsymbol {y} _ {1 : t - 1} ^ {\prime})} \right] - \frac {2 \widetilde {B}}{t ^ {2}} \\ Y _ {t} = \sum_ {s = 1} ^ {t} X _ {s}. \\ \end{array}
$$
Following the proof of Lemma B.9, we can easily show that  $(Y_{t}:t = 0,\dots ,T)$  is a super-martingale. Now we are finally ready to prove an upper bound on the batch cumulative regret of our Mean-VarBTS-RED:
Lemma G.9. Define  $C_0 \triangleq \frac{1}{\mathbb{B} / \lceil \frac{\sigma_{\max}^2}{R^2} \rceil - 1}$ . Given  $\delta \in (0,1)$ , then with probability of at least  $1 - \delta$
$$
\begin{array}{l} R _ {T} ^ {M V} \leq e ^ {C} \left(1 + \frac {2 8}{p ^ {2}}\right) \sqrt {C _ {2} C _ {0}} \sqrt {T} \left[ \omega c _ {T} \sqrt {\Gamma_ {T \mathbb {B}}} + (1 - \omega) c _ {T} ^ {\prime} \sqrt {\Gamma_ {T \mathbb {B}} ^ {\prime}} \right] \\ + \frac {\widetilde {B} \pi^ {2}}{3} + \left(6 \widetilde {B} + e ^ {C} \sqrt {C} \left(1 + \frac {2 8}{p ^ {2}}\right) \sqrt {C _ {2} C _ {0}} \left[ \omega c _ {T} + (1 - \omega) c _ {T} ^ {\prime} \right]\right) \sqrt {2 \log (4 / \delta) T}, \\ \end{array}
$$
in which  $\Gamma_{T\mathbb{B}}$  is the maximum information gain about  $f$  obtained from any set of  $T\mathbb{B}$  observations, and  $\Gamma_{T\mathbb{B}}^{\prime}$  is the maximum information gain about  $g$  obtained from any set of  $T\mathbb{B}$  observations.
Proof. To begin with, let's derive a lower bound on  $b_{t}$ . Firstly, note that  $n_t \leq \left\lceil \frac{\sigma_{\mathrm{max}}^2}{R^2} \right\rceil$ . This implies that  $b_{t} \geq \mathbb{B} / \left\lceil \frac{\sigma_{\mathrm{max}}^{2}}{R^{2}} \right\rceil - 1$ .
$$
\begin{array}{l} \left| Y _ {t} - Y _ {t - 1} \right| = \left| X _ {t} \right| \\ = | \bar {r} _ {t} | + e ^ {C} \left(1 + \frac {2 8}{p ^ {2}}\right) \left[ \omega c _ {t} \frac {1}{b _ {t}} \sqrt {C _ {2} b _ {t} \mathbb {I} (f ; \boldsymbol {y} _ {t} | \boldsymbol {y} _ {1 : t - 1})} \right. \\ \left. + (1 - \omega) c _ {t} ^ {\prime} \frac {1}{b _ {t}} \sqrt {C _ {2} b _ {t} \mathbb {I} \left(g ; \boldsymbol {y} _ {t} ^ {\prime} \mid \boldsymbol {y} _ {1 : t - 1} ^ {\prime}\right)} \right] + \frac {2 \widetilde {B}}{t ^ {2}} \tag {42} \\ \leq 2 \widetilde {B} + e ^ {C} \left(1 + \frac {2 8}{p ^ {2}}\right) \left[ \omega c _ {t} \sqrt {\frac {C _ {2} C}{b _ {t}}} + (1 - \omega) c _ {t} ^ {\prime} \sqrt {\frac {C _ {2} C}{b _ {t}}} \right] + 2 \widetilde {B} \\ = 4 \widetilde {B} + e ^ {C} \left(1 + \frac {2 8}{p ^ {2}}\right) \left[ \omega c _ {t} \sqrt {\frac {C _ {2} C}{b _ {t}}} + (1 - \omega) c _ {t} ^ {\prime} \sqrt {\frac {C _ {2} C}{b _ {t}}} \right]. \\ \end{array}
$$
$$
\sum_ {t = 1} ^ {T} \sqrt {\mathbb {I} (f ; \boldsymbol {y} _ {t} | \boldsymbol {y} _ {1 : t - 1})} \leq \sqrt {T \sum_ {t = 1} ^ {T} \mathbb {I} (f ; \boldsymbol {y} _ {t} | \boldsymbol {y} _ {1 : t - 1})} = \sqrt {T \mathbb {I} (f ; \boldsymbol {y} _ {1 : T})} \leq \sqrt {T \Gamma_ {T \mathbb {B}}}. \tag {43}
$$
Applying the Azuma-Hoeffding's inequality using an error probability of  $\delta / 3$  leads to
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \bar {r} _ {t} \leq e ^ {C} \left(1 + \frac {2 8}{p ^ {2}}\right) \left[ \omega \sum_ {t = 1} ^ {T} c _ {t} \sqrt {\frac {1}{b _ {t}} C _ {2} \mathbb {I} (f ; \boldsymbol {y} _ {t} | \boldsymbol {y} _ {1 : t - 1})} + (1 - \omega) \sum_ {t = 1} ^ {T} c _ {t} ^ {\prime} \sqrt {\frac {1}{b _ {t}} C _ {2} \mathbb {I} (g ; \boldsymbol {y} _ {t} ^ {\prime} | \boldsymbol {y} _ {1 : t - 1} ^ {\prime})} \right] \\ + \sum_ {t = 1} ^ {T} \frac {2 \widetilde {B}}{t ^ {2}} + \sqrt {2 \log (\frac {3}{\delta}) \sum_ {t = 1} ^ {T} \left(4 \widetilde {B} + e ^ {C} \left(1 + \frac {2 8}{p ^ {2}}\right) \left[ \omega c _ {t} \sqrt {\frac {C _ {2} C}{b _ {t}}} + (1 - \omega) c _ {t} ^ {\prime} \sqrt {\frac {C _ {2} C}{b _ {t}}} \right]\right) ^ {2}} \\ \leq e ^ {C} \left(1 + \frac {2 8}{p ^ {2}}\right) \sqrt {\frac {C _ {2}}{\mathbb {B} / \lceil \frac {\sigma_ {\max} ^ {2}}{R ^ {2}} \rceil - 1}} \sqrt {T} \left[ \omega c _ {T} \sqrt {\Gamma_ {T \mathbb {B}}} + (1 - \omega) c _ {T} ^ {\prime} \sqrt {\Gamma_ {T , \mathbb {B}} ^ {\prime}} \right] \\ + \frac {\widetilde {B} \pi^ {2}}{3} + \left(4 \widetilde {B} + e ^ {C} \sqrt {C} \left(1 + \frac {2 8}{p ^ {2}}\right) \sqrt {\frac {C _ {2}}{\mathbb {B} / \lceil \frac {\sigma_ {\max} ^ {2}}{R ^ {2}} \rceil - 1}} \left[ \omega c _ {T} + (1 - \omega) c _ {T} ^ {\prime} \right]\right) \sqrt {2 \log (3 / \delta) T} \\ = e ^ {C} \left(1 + \frac {2 8}{p ^ {2}}\right) \sqrt {C _ {2} C _ {0}} \sqrt {T} \left[ \omega c _ {T} \sqrt {\Gamma_ {T \mathbb {B}}} + (1 - \omega) c _ {T} ^ {\prime} \sqrt {\Gamma_ {T \mathbb {B}} ^ {\prime}} \right] \\ + \frac {\widetilde {B} \pi^ {2}}{3} + \left(4 \widetilde {B} + e ^ {C} \sqrt {C} \left(1 + \frac {2 8}{p ^ {2}}\right) \sqrt {C _ {2} C _ {0}} \left[ \omega c _ {T} + (1 - \omega) c _ {T} ^ {\prime} \right]\right) \sqrt {2 \log (3 / \delta) T}. \tag {44} \\ \end{array}
$$
The second inequality makes use of equation (43) and the lower bound on  $b_{t}$ :  $b_{t} \geq \mathbb{B} / \left\lceil \frac{\sigma_{\max}^{2}}{R^{2}} \right\rceil - 1$ . Now, note that  $\overline{r}_{t} = r_{t}, \forall t \geq 1$  with probability of  $\geq 1 - \delta / 3 - \delta / 3$  because both events  $E^{f}(t)$  and  $E^{g}(t)$  hold with probability of  $\geq 1 - \delta / 3$  respectively. As a result, taking into account the error probability of  $\delta / 3$  from the Azuma-Hoeffding's inequality, the regret upper bound holds with probability of  $\geq 1 - \delta / 3 - \delta / 3 - \delta / 3 = 1 - \delta$ .
Figure 4: Groundtruth noise variance function and an estimated upper bound (Sec. 3.2).
Finally, we can simplify the regret upper bound from Lemma G.9 into asymptotic notation:
$$
R _ {T} ^ {\mathrm {M V}} = \widetilde {\mathcal {O}} \left(e ^ {C} \frac {1}{\sqrt {\mathbb {B} / \lceil \frac {\sigma_ {\operatorname* {m a x}} ^ {2}}{R ^ {2}} \rceil - 1}} \sqrt {T} \left[ \omega R \sqrt {\Gamma_ {T \mathbb {B}}} (\sqrt {\Gamma_ {T \mathbb {B}}} + \sqrt {C}) + (1 - \omega) R ^ {\prime} \sqrt {\Gamma_ {T \mathbb {B}} ^ {\prime}} (\sqrt {\Gamma_ {T \mathbb {B}} ^ {\prime}} + \sqrt {C}) \right]\right) \tag {45}
$$
H More Experimental Details
Since the theoretical value of  $\beta_{t}$  and  $\beta_{t}^{\prime}$  is usually too conservative [41, 3], we follow the common practice and set them to a constant:  $\beta_{t} = \beta_{t}^{\prime} = 1$ . In every experiment, we use the same set of initial inputs selected via random search for all methods to ensure a fair comparison. For all methods based on TS (including all our methods), i.e., all methods which require sampling functions from the GP posterior (e.g., line 5 of Algo. 1, line 5 of Algo. 3 and line 5 of Algo. 2), we follow the common practice and sample the functions from the GP posterior through random Fourier features approximation [39, 46, 34]. Again following the common practice in BO, we optimize the GP hyperparameters by maximizing the marginal likelihood after every 10 iterations. Our experiments are run on a computer server with 128 CPUs, with the AMD EPYC 7543 32-Core Processor. The server has 8 NVIDIA GeForce RTX 3080 GPUs.
H.1 Synthetic Experiments
In the synthetic experiments, for both mean and mean-variance optimization, we firstly use a sampled function from a GP with the SE kernel (lengthscale  $= 0.04$ ) as the objective function  $f$  (normalized into the range  $[0,1]$ ), and then sample another function from a GP with the SE kernel (lengthscale  $= 0.15$ ) as the noise variance function  $\sigma^2$  (normalized into the range  $[0.0001,0.2]$ ).
Fig. 4 shows an example of the upper bound  $U_{t}^{\sigma^{2}}(\pmb{x}) = -\mu_{t-1}^{\prime}(\pmb{x}) + \beta_{t}^{\prime}\sigma_{t-1}^{\prime}(\pmb{x})$  constructed by our BTS-RED-Unknown (Sec. 3.2), which is an effective approximation of the groundtruth  $\sigma^{2}(\cdot)$ .
Here we also use the synthetic experiments to explore the impact of the uncertainty sampling (US) initialization, which is required by our theoretical results (Sec. 3.1.2), affects the empirical performance of our algorithms. The results in Fig. 8 show that using US and random search as the initialization method lead to similar performances. This provides an empirical justification for our choice of using the simpler random search as the initialization method in our main experiments.
H.2 Real-world Experiments on Plant Growth
As we have mentioned in the main text (Sec. 5.2), we perform real-world experiments using the input conditions from a regular a 2-D grid within the 2-D domain. Specifically, the 2-D grid is constructed using the pH values of  $\{2.5,3.0,3.5,4.0,4.5,5.5,6.5\}$  and NH3 concentrations of  $\{0,1,5,10,15,20,25,30\} (\times 1000~\mathrm{uM})$ . In other words, the size of the grid is  $7\times 8 = 56$ . Every tested input condition is replicated 6 times. For every tested input condition, the leaf area and tipburn area after harvest are observed, both measured in the unit of  $\mathrm{mm}^2$ . The resulting observations are
Figure 5: Results for the mean optimization problem for the synthetic experiment (Sec. 5.1) after adding comparisons with some sequential BO algorithms. Sequential BO algorithms are unable to perform competitively with other algorithms which leverage batch evaluations.
Figure 6: (a) The synthetic function used in the experiments for mean-variance optimization in Sec. 5.1. (b) The mean and mean-variance objective functions  $(\omega = 0.3)$ .
Figure 7: (a) Results using more values of  $\kappa$  for BTS-RED-Known in the synthetic experiment. (b) Results for the synthetic experiment using the LCB as the report metric.
Figure 8: Comparisons of results using uncertainty sampling (US) and random search as the initialization method.
Figure 9: Groundtruth (a) mean-variance objective function and (b) noise variance functions for the leaf area, including some selected queries (white stars) and the corresponding  $n_t$ 's.
(a)
(b)
Figure 10: Mean optimization for (a) the leaf area and (b) the weighted combination of leaf area and negative tipburn area. The results for some sequential BO algorithms are included here, i.e., GP-TS, GP-UCB and GP-UCB with a heteroscedastic GP. Similar to Fig. 5 for the synthetic experiment, sequential BO algorithms fail to achieve competitive performances with other algorithms which are able to exploit batch evaluations.
used to learn a heteroscedastic GP model from GPglow (https://gpflow.readthedocs.io/en/develop/ notebooks/advanced/heteroskedastic.html), which is then used to produce the groundtruth mean and variance function used in our experiments.
H.3 Real-world Experiments on AutoML
In this experiment, we aim to tune two hyperparameters of SVM: the penalty parameter (denoted as  $C$ ) and RBF kernel parameter (referred to as gamma), both within the range of [0.0001, 2]. To obtain the groundtruth mean and variance functions, we firstly construct a uniform 2-D grid of size  $80 \times 80$  using the 2-D domain, and then evaluate every input hyperparameter configuration on the grid using 100 different classification tasks (i.e., using the images of hand-written characters from 100 different individuals in the EMNIST dataset). The EMNIST dataset is under the CC0 license. For every tested input hyperparameter configuration on the grid, we record the empirical mean and variance as the groundtruth mean and variance at the corresponding input location. As a result, this completes our construction of the groundtruth mean and variance functions with the input domain being discrete with a size of  $80 \times 80 = 6400$ .