SlowGuess/NeurIPS_2023_OCR huggingface dataset:
6 Related Works
BO has been extended to the batch setting in recent years [9, 11, 19, 22, 38, 44, 47]. The work of [28] proposed a simple batch TS method by exploiting the inherent randomness of TS. Interestingly, as we discussed in Sec. 3.1.2, the method from [28] is equivalent to a reduced version of our BTS-RED-Known with homoscedastic noise, and our Theorem 3.1 provides a theoretical guarantee on its frequentist regret (in contrast to the Bayesian regret analyzed in [28]). The work of [2] aimed to adaptively choose whether to explore a new query or to replicate a previous query. However, their method requires additional heuristic techniques to achieve replications and hence has no theoretical guarantees, in stark contrast to our simple and principled way for replication selection (Sec. 3). Moreover, their method does not support batch evaluations, and is unable to tackle risk-averse optimization. Recently, [43] proposed to select a batch of queries while balancing exploring new queries and replicating existing ones. However, unlike our simple and principled algorithms, their method requires complicated heuristic procedures for query/replication selection and batch construction, and hence does not have theoretical guarantees. Moreover, their method also only focuses on standard mean optimization and cannot be easily extended for risk-averse optimization.
The work of [23] used a heteroscedastic GP [29] as the surrogate model for risk-averse optimization. The works of [6, 26, 36, 37, 42] considered risk-averse BO, however, these works require the ability to observe and select an environmental variable, which is usually either not explicitly defined or uncontrollable in practice (e.g., our experiments in Sec. 5). The recent work of [33] modified BO to maximize the mean-variance objective and derived theoretical guarantees using results from [30]. Their method uses a heteroscedastic GP as the surrogate model and employs another (homoscedastic) GP to model the observation noise variance, in which the second GP is learned by replicating every query for a fixed predetermined number of times. Importantly, all of these works on risk-averse BO have focused only on the sequential setting without support for batch evaluations. Replicating the selected inputs in BO multiple times has also been adopted by the recent works of [7, 13], which have shown that replication can lead to comparable or better theoretical and empirical performances of BO.
We have introduced the BTS-RED framework, which can trade-off between evaluating more unique conditions vs. replicating each condition more times and can perform risk-averse optimization. We derive theoretical guarantees for our methods to show that they are no-regret, and verify their empirical effectiveness in real-world precision agriculture and AutoML experiments. A potential limitation is that we use a heuristic (rather than principled) technique to handle unused budgets in an iteration (last paragraph of Sec. 3.1.1). Another interesting future work is to incorporate our technique of using an adaptive number of replications (depending on the noise variance) into other batch BO algorithms [18, 22] to further improve their performances. Moreover, it is also interesting to combine our method with the recent line of work on neural bandits [16, 17], which may expand the application of our method to more AI4Science problems.
This research/project is supported by A\*STAR under its RIE2020 Advanced Manufacturing and Engineering (AME) Programmatic Funds (Award A20H6b0151) and its RIE2020 Advanced Manufacturing and Engineering (AME) Industry Alignment Fund - Pre Positioning (IAF-PP) (Award A19E4a0101).
[1] F. Berkenkamp, A. P. Schoellig, and A. Krause. No-regret Bayesian optimization with unknown hyperparameters. Journal of Machine Learning Research, 2019.
[2] M. Binois, J. Huang, R. B. Gramacy, and M. Ludkovski. Replication or exploration? sequential design for stochastic simulation experiments. Technometrics, 61(1):7-23, 2019.
[3] I. Bogunovic, J. Scarlett, S. Jegelka, and V. Cevher. Adversarially robust optimization with Gaussian processes. In Proc. NeuIPS, 2018.
[4] G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. OpenAI Gym. arXiv:1606.01540, 2016.
[5] X. Cai and J. Scarlett. On lower bounds for standard and robust Gaussian process bandit optimization. In Proc. ICML, pages 1216-1226. PMLR, 2021.
[6] S. Cakmak, R. Astudillo, P. Frazier, and E. Zhou. Bayesian optimization of risk measures. In Proc. NeurIPS, 2020.
[7] D. Calandriello, L. Carratino, A. Lazaric, M. Valko, and L. Rosasco. Scaling Gaussian process optimization by evaluating a few unique candidates multiple times. In Proc. ICML, pages 2523-2541. PMLR, 2022.
[8] S. R. Chowdhury and A. Gopalan. On kernelized multi-armed bandits. In Proc. ICML, pages 844–853, 2017.
[9] S. R. Chowdhury and A. Gopalan. On batch Bayesian optimization. arXiv:1911.01032, 2019.
[10] G. Cohen, S. Afshar, J. Tapson, and A. Van Schaik. EMNIST: Extending MNIST to handwritten letters. In Proc. IJCNN, pages 2921-2926. IEEE, 2017.
[11] E. Contal, D. Buffoni, A. Robicquet, and N. Vayatis. Parallel Gaussian process optimization with upper confidence bound and pure exploration. In Proc. ECML/PKDD, pages 225-240, 2013.
[12] A. I. Cowen-Rivers, W. Lyu, R. Tutunov, Z. Wang, A. Grosnit, R. R. Griffiths, A. M. Maraval, H. Jianye, J. Wang, J. Peters, et al. An empirical study of assumptions in Bayesian optimisation. arXiv:2012.03826, 2020.
[13] Z. Dai, G. K. R. Lau, A. Verma, Y. Shu, B. K. H. Low, and P. Jaillet. Quantum Bayesian optimization. In Proc. NeurIPS, 2023.
[14] Z. Dai, B. K. H. Low, and P. Jaillet. Federated Bayesian optimization via Thompson sampling. In Proc. NeurIPS, 2020.
[15] Z. Dai, B. K. H. Low, and P. Jaillet. Differentially private federated Bayesian optimization with distributed exploration. In Proc. NeurIPS, volume 34, 2021.
[16] Z. Dai, Y. Shu, B. K. H. Low, and P. Jaillet. Sample-then-optimize batch neural Thompson sampling. In Proc. NeurIPS, 2022.
[17] Z. Dai, Y. Shu, A. Verma, F. X. Fan, B. K. H. Low, and P. Jaillet. Federated neural bandits. In Proc. ICLR, 2023.
[18] E. A. Daxberger and B. K. H. Low. Distributed batch Gaussian process optimization. In Proc. ICML, pages 951-960, 2017.
[19] T. Desautels, A. Krause, and J. W. Burdick. Parallelizing exploration-exploitation tradeoffs in Gaussian process bandit optimization. Journal of Machine Learning Research, 15:3873-3923, 2014.
[20] D. Eriksson, M. Pearce, J. Gardner, R. D. Turner, and M. Poloczek. Scalable global optimization via local Bayesian optimization. In Proc. NeurIPS, 2019.
[45] Z. Wang, C. Gehring, P. Kohli, and S. Jegelka. Batched large-scale Bayesian optimization in high-dimensional spaces. In Proc. AISTATS, pages 745-754. PMLR, 2018.
[46] Z. Wang and S. Jegelka. Max-value entropy search for efficient Bayesian optimization. In Proc. ICML, pages 3627-3635. PMLR, 2017.
[47] J. Wu and P. Frazier. The parallel knowledge gradient method for batch Bayesian optimization. In Proc. NeurIPS, volume 29, pages 3126-3134, 2016.
A Expressions of GP Posterior
Following the notations in the main text (Sec. 2), we index every queried input  $\boldsymbol{x}_t^{[b]}$  and observed output  $y_t^{[b]}$  via an iteration index  $t$  and a batch index  $b$ . Define  $\mathcal{I}_{t-1}$  the collection of indices of the queried inputs in the first  $t-1$  iterations:  $\mathcal{I}_{t-1} = \{(t', b)\}_{t' \in [t-1], b \in [b_{t'}]}$ . Note that according to our notations in the main text, the cardinality of  $\mathcal{I}_{t-1}$  is  $|\mathcal{I}_{t-1}| = \tau_{t-1} = \sum_{t'=1}^{t-1} b_{t'}$ . Then, the GP posterior for the objective function  $f$  in iteration  $t$  can be represented as  $\mathcal{GP}(\mu_{t-1}(\cdot), \sigma_{t-1}^2(\cdot, \cdot))$ , where
$$
\begin{array}{l} \mu_ {t - 1} (\boldsymbol {x}) \triangleq \boldsymbol {k} _ {t - 1} (\boldsymbol {x}) ^ {\top} \left(\boldsymbol {K} _ {t - 1} + \lambda \mathbf {I}\right) ^ {- 1} \boldsymbol {y} _ {t - 1}, \\ \sigma_ {t - 1} ^ {2} \left(\boldsymbol {x}, \boldsymbol {x} ^ {\prime}\right) \triangleq k \left(\boldsymbol {x}, \boldsymbol {x} ^ {\prime}\right) - \boldsymbol {k} _ {t - 1} (\boldsymbol {x}) ^ {\top} \left(\boldsymbol {K} _ {t - 1} + \lambda \mathbf {I}\right) ^ {- 1} \boldsymbol {k} _ {t - 1} \left(\boldsymbol {x} ^ {\prime}\right), \tag {3} \\ \end{array}
$$
in which  $\pmb{k}_{t-1}(\pmb{x}) \triangleq [k(\pmb{x},\pmb{x}_{t'}^{[b]})]_{(t',b) \in \mathcal{I}_{t-1}}^{\top}$ .  $\pmb{y}_{t-1} \triangleq (y_{t'}^{[b]})_{(t',b) \in \mathcal{I}_{t-1}}^{\top}$  in which  $y_{t'}^{[b]} = (1/n_{t'}^{[b]}) \sum_{n=1}^{n_{t'}^{[b]}} y_{t',n}^{[b]}$  represents the empirical mean at the input  $\pmb{x}_{t'}^{[b]}$  calculated using the  $n_{t'}^{[b]}$  replications.  $\mathbf{K}_{t-1} \triangleq (k(\mathbf{x}_{t'}^{[b]},\mathbf{x}_{t''}^{[b']}))_{(t',b) \in \mathcal{I}_{t-1},(t'',b') \in \mathcal{I}_{t-1}}$  and  $\lambda > 0$  is a regularization parameter and will need to be set to  $\lambda = 1 + 2 / T$  in order for our theoretical results to hold [8].
Denote by  $\tau_{t-1}$  the total number of observations (input-output pairs) up to and including iteration  $t-1$ :  $\tau_{t-1} \triangleq \sum_{t'=1}^{t-1} b_{t'}$ . This immediately implies that  $t-1 \leq \tau_{t-1} \leq \mathbb{B}(t-1)$ . We use  $\mathcal{F}_{t-1}$  to denote the history of all  $\tau_{t-1}$  observations up to iteration  $t-1$ . Denote by  $b_t$  the batch size in iteration  $t$ . Note that conditioned on  $\mathcal{F}_{t-1}$ ,  $b_t$  is a random variable, which is in contrast with standard batch BO in which the batch size is usually fixed. Here we use  $\mu_{t-1}(\boldsymbol{x})$  and  $\sigma_{t-1}(\boldsymbol{x})$  to denote the GP posterior mean and standard deviation conditioned on all  $\tau_{t-1}$  observations up to (and including) iteration  $t-1$ . Moreover, denote by  $\sigma_{t-1,b'}(\boldsymbol{x})$  the GP posterior standard deviation after additionally conditioning on the first  $b' = 0, \ldots, b_t - 1$  selected inputs in iteration  $t$ . Note that  $\sigma_{t-1,0}(\boldsymbol{x}) = \sigma_{t-1}(\boldsymbol{x})$  according to our definitions. Define  $\beta_t \triangleq B + R \sqrt{2(\Gamma_{\tau_{t-1}} + 1 + \log(2/\delta))}$  and  $c_t \triangleq \beta_t(1 + \sqrt{2\log(\mathbb{X}|t^2)})$ .
Lemma B.1. Let  $\delta \in (0,1)$ . Define  $E^{f}(t)$  as the event that  $|\mu_{t - 1}(\pmb {x}) - f(\pmb {x})|\leq \beta_{t}\sigma_{t - 1}(\pmb {x})$  for all  $\pmb {x}\in \mathcal{X}$ . We have that  $\mathbb{P}\left[E^{f}(t)\right]\geq 1 - \delta /2$  for all  $t\geq 1$ .
Lemma B.1 is a consequence of Theorem 2 of the work of [8].
Lemma B.2. Define  $E^{f_t}(t)$  as the event:  $|f_t^{[b]}(\pmb{x}) - \mu_{t-1}(\pmb{x})| \leq \beta_t \sqrt{2\log(\mathbb{B}|\mathcal{X}|t^2)} \sigma_{t-1}(\pmb{x}), \forall \pmb{x} \in \mathcal{X}, \forall b \in [b_t]$ . We have that  $\mathbb{P}\left[E^{f_t}(t)|\mathcal{F}_{t-1}\right] \geq 1 - 1/t^2$  for any possible filtration  $\mathcal{F}_{t-1}$ .
Proof. According to Lemma B4 of [24], in iteration  $t$ , for a particular  $b$  and  $\pmb{x}$ , we have that
$$
\left| f _ {t} ^ {[ b ]} (\boldsymbol {x}) - \mu_ {t - 1} (\boldsymbol {x}) \right| \leq \beta_ {t} \sqrt {2 \log (1 / \delta)} \sigma_ {t - 1} (\boldsymbol {x}), \tag {4}
$$
with probability of  $\geq 1 - \delta$ . Replacing  $\delta$  by  $\delta / (\mathbb{B}|\mathcal{X}|)$  and taking a union bound over all  $\pmb{x} \in \mathcal{X}$  and all  $b \in [b_{\ell}]$  gives us:
$$
\left| f _ {t} ^ {[ b ]} (\boldsymbol {x}) - \mu_ {t - 1} (\boldsymbol {x}) \right| \leq \beta_ {t} \sqrt {2 \log (\mathbb {B} | \mathcal {X} | / \delta)} \sigma_ {t - 1} (\boldsymbol {x}), \quad \forall \boldsymbol {x} \in \mathcal {X}, b \in [ b _ {t} ], \tag {5}
$$
which holds with probability of  $\geq 1 - \frac{\delta}{\mathbb{B}|\mathcal{X}|}\times |\mathcal{X}|b_t\geq 1 - \delta$ , because  $b_{t}\leq \mathbb{B}$ . Further replacing  $\delta$  by  $1 / t^{2}$  completes the proof.
Next, we define the set of saturated points.
Definition B.3. Define the set of saturated points at iteration  $t$  as
$$
S _ {t} = \left\{\boldsymbol {x} \in \mathcal {X}: \Delta (\boldsymbol {x}) > c _ {t} \sigma_ {t - 1} (\boldsymbol {x}) \right\},
$$
in which  $\Delta (\pmb {x}) = f(\pmb{x}^{*}) - f(\pmb {x})$  and  $\pmb {x}^{\ast}\in \arg \max_{\pmb {x}\in \mathcal{X}}f(\pmb {x})$
The next auxiliary lemma will be needed shortly to lower-bound the probability that an unsaturated point is selected.
Lemma B.4. For any filtration  $\mathcal{F}_{t - 1}$ , conditioned on the events  $E^{f}(t)$ , we have that  $\forall \pmb {x}\in \mathcal{X},b\in [b_t]$
$$
\mathbb {P} \left(f _ {t} ^ {[ b ]} (\boldsymbol {x}) > f (\boldsymbol {x}) \mid \mathcal {F} _ {t - 1}\right) \geq p, \tag {6}
$$
in which  $p = \frac{1}{4e\sqrt{\pi}}$ .
Proof. For any  $b \in [b_t]$ , we have that
$$
\begin{array}{l} \mathbb {P} \left(f _ {t} ^ {[ b ]} (\boldsymbol {x}) > f (\boldsymbol {x}) | \mathcal {F} _ {t - 1}\right) = \mathbb {P} \left(\frac {f _ {t} ^ {[ b ]} (\boldsymbol {x}) - \mu_ {t - 1} (\boldsymbol {x})}{\beta_ {t} \sigma_ {t - 1} (\boldsymbol {x})} > \frac {f (\boldsymbol {x}) - \mu_ {t - 1} (\boldsymbol {x})}{\beta_ {t} \sigma_ {t - 1} (\boldsymbol {x})} \Big | \mathcal {F} _ {t - 1}\right) \\ \geq \mathbb {P} \left(\frac {f _ {t} ^ {[ b ]} (\boldsymbol {x}) - \mu_ {t - 1} (\boldsymbol {x})}{\beta_ {t} \sigma_ {t - 1} (\boldsymbol {x})} > \frac {| f (\boldsymbol {x}) - \mu_ {t - 1} (\boldsymbol {x}) |}{\beta_ {t} \sigma_ {t - 1} (\boldsymbol {x})} \mid \mathcal {F} _ {t - 1}\right) \tag {7} \\ \geq \mathbb {P} \left(\frac {f _ {t} ^ {[ b ]} (\boldsymbol {x}) - \mu_ {t - 1} (\boldsymbol {x})}{\beta_ {t} \sigma_ {t - 1} (\boldsymbol {x})} > 1 \mid \mathcal {F} _ {t - 1}\right) \\ \geq \frac {e ^ {- 1}}{4 \sqrt {\pi}}. \\ \end{array}
$$
The second last inequality results from Lemma B.1, and the last inequality follows because  $f_{t}^{[b]}(\pmb{x})$  follows a Gaussian distribution because  $f_{t}^{[b]} \sim \mathcal{GP}(\mu_{t-1}(\cdot), \beta_{t}^{2}\sigma_{t-1}^{2}(\cdot))$ . Lastly, since all  $f_{t}^{[b]}$ 's are sampled in the same way:  $f_{t}^{[b]} \sim \mathcal{GP}(\mu_{t-1}(\cdot), \beta_{t}^{2}\sigma_{t-1}^{2}(\cdot))$ , the proof above holds for all  $b \in [b_t]$ .
The next lemma shows that the probability that an unsaturated input is selected can be lower-bounded.
Lemma B.5. For any filtration  $\mathcal{F}_{t - 1}$ , conditioned on the event  $E^{f}(t)$ , we have that
$$
\mathbb {P} \left(\boldsymbol {x} _ {t} ^ {[ b ]} \in \mathcal {X} \setminus S _ {t}, | \mathcal {F} _ {t - 1}\right) \geq p - 1 / t ^ {2}, \quad \forall b \in [ b _ {t} ].
$$
Proof. For every  $b \in [b_t]$ ,
$$
\mathbb {P} \left(\boldsymbol {x} _ {t} ^ {[ b ]} \in \mathcal {X} \backslash S _ {t} | \mathcal {F} _ {t - 1}\right) \geq \mathbb {P} \left(f _ {t} ^ {[ b ]} \left(\boldsymbol {x} ^ {*}\right) > f _ {t} ^ {[ b ]} (\boldsymbol {x}), \forall \boldsymbol {x} \in S _ {t} | \mathcal {F} _ {t - 1}\right), \tag {8}
$$
which holds  $\forall b\in [b_t]$ . The validity of the inequality above can be seen by noting that  $\pmb{x}^*$  is always unsaturated, because  $\Delta (\pmb{x}^{*}) = f(\pmb{x}^{*}) - f(\pmb{x}^{*}) = 0 < c_{t}\sigma_{t - 1}(\pmb {x})$ . As a result, if the event on the right hand side holds (i.e., if  $f_{t}^{[b]}(\pmb{x}^{*}) > f_{t}^{[b]}(\pmb {x}),\forall \pmb {x}\in S_{t})$ , then the event on the left hand side is guaranteed to hold because  $\pmb{x}_{t}^{[b]}$  is selected by  $\pmb{x}_{t}^{[b]} = \arg \max_{\pmb {x}\in \mathcal{X}}f_{t}^{[b]}(\pmb {x})$  which ensures that an unsaturated input will be selected.
Next, we assume that both events  $E^{f}(t)$  and  $E^{f_{t}}(t)$  are true, which allows us to derive an upper bound on  $f_{t}^{[b]}(\pmb{x})$  for all  $\pmb{x} \in S_{t}$  and for all  $b \in [b_{t}]$ :
$$
f _ {t} ^ {[ b ]} (\boldsymbol {x}) \leq f (\boldsymbol {x}) + c _ {t} \sigma_ {t - 1} (\boldsymbol {x}) \leq f (\boldsymbol {x}) + \Delta (\boldsymbol {x}) = f (\boldsymbol {x}) + f \left(\boldsymbol {x} ^ {*}\right) - f (\boldsymbol {x}) = f \left(\boldsymbol {x} ^ {*}\right), \tag {9}
$$
where the first inequality follows from Lemma B.1 and Lemma B.2, and the second inequality results from Definition B.3. Therefore, (9) implies that for every  $b \in [b_t]$ , if both both events  $E^f(t)$  and  $E^{f_t}(t)$  hold, we have that
$$
\mathbb {P} \left(f _ {t} ^ {[ b ]} \left(\boldsymbol {x} ^ {*}\right) > f _ {t} ^ {[ b ]} (\boldsymbol {x}), \forall \boldsymbol {x} \in S _ {t} \mid \mathcal {F} _ {t - 1}\right) \geq \mathbb {P} \left(f _ {t} ^ {[ b ]} \left(\boldsymbol {x} ^ {*}\right) > f \left(\boldsymbol {x} ^ {*}\right) \mid \mathcal {F} _ {t - 1}\right). \tag {10}
$$
Next, conditioning only on the event  $E^{f}(t)$ , for every  $b \in [b_{t}]$ , we can show that
$$
\begin{array}{l} \mathbb {P} \left(\boldsymbol {x} _ {t} ^ {[ b ]} \in \mathcal {X} \backslash S _ {t} | \mathcal {F} _ {t - 1}\right) \geq \mathbb {P} \left(f _ {t} ^ {[ b ]} (\boldsymbol {x} ^ {*}) > f _ {t} ^ {[ b ]} (\boldsymbol {x}), \forall \boldsymbol {x} \in S _ {t} | \mathcal {F} _ {t - 1}\right) \\ \stackrel {(a)} {\geq} \mathbb {P} \left(f _ {t} ^ {[ b ]} \left(\boldsymbol {x} ^ {*}\right) > f \left(\boldsymbol {x} ^ {*}\right) \mid \mathcal {F} _ {t - 1}\right) - \mathbb {P} \left(\overline {{E ^ {f _ {t}} (t)}} \mid \mathcal {F} _ {t - 1}\right) \tag {11} \\ \stackrel {(b)} {\geq} p - 1 / t ^ {2}, \\ \end{array}
$$
which holds for all  $b\in [b_t]$
Next, we use the following Lemma to connect the GP posterior standard deviation given all observations in the first  $t - 1$  iterations (i.e.,  $\sigma_{t-1}(\cdot)$ ) with the conditional information gain from the selected input queries in the  $t^{\text{th}}$  iteration (batch).
Lemma B.6. Define  $C_2 = \frac{2}{\log(1 + \lambda^{-1})}$ . Denote all  $\tau_{t - 1}$  observations from iterations (batches) 1 to  $t - 1$  as  $\mathbf{y}_{1:t - 1}$ , and the  $b_t$  observations in the  $t^{th}$  batch as  $\mathbf{y}_t$ . Then we have that
$$
\sum_ {b = 1} ^ {b _ {t}} \sigma_ {t - 1} (\boldsymbol {x} _ {t} ^ {[ b ]}) \leq e ^ {C} \sqrt {C _ {2} b _ {t} \mathbb {I} (f ; \boldsymbol {y} _ {t} | \boldsymbol {y} _ {1 : t - 1})}.
$$
Proof. Note that as has been described in the main text, the constant  $C$  is chosen such that:
$$
\max  _ {A \subset \mathcal {X}, | A | \leq \mathbb {B}} \mathbb {I} (f; \boldsymbol {y} _ {A} | \boldsymbol {y} _ {1: t - 1}) \leq C, \forall t \geq 1. \tag {12}
$$
Denote by  $\pmb{y}_{t,1:b-1}$  the first  $b-1$  observations within the  $t^{\text{th}}$  batch, then for  $b > 1$ ,
$$
\begin{array}{l} \frac {\sigma_ {t - 1} (\boldsymbol {x})}{\sigma_ {t - 1 , b - 1} (\boldsymbol {x})} = \exp \left(\mathbb {I} (f (\boldsymbol {x}); \boldsymbol {y} _ {t, 1: b - 1} | \boldsymbol {y} _ {1: t - 1})\right) \\ \leq \exp \left(\mathbb {I} (f; \boldsymbol {y} _ {t, 1: b - 1} | \boldsymbol {y} _ {1: t - 1})\right) \\ \leq \exp \left(\max  _ {A \subset \mathcal {X}, | A | \leq b - 1} \mathbb {I} (f; \boldsymbol {y} _ {A} | \boldsymbol {y} _ {1: t - 1})\right) \tag {13} \\ \leq \exp \left(\max  _ {A \subset \mathcal {X}, | A | \leq \mathbb {B}} \mathbb {I} (f; \boldsymbol {y} _ {A} | \boldsymbol {y} _ {1: t - 1})\right) \\ \leq \exp (C). \\ \end{array}
$$
Also note that when  $b = 1$ ,  $\sigma_{t - 1}(\pmb{x}) / \sigma_{t - 1,b - 1}(\pmb{x}) = 1 \leq \exp(C)$ . Therefore, we have that
$$
\begin{array}{l} \sum_ {b = 1} ^ {b _ {t}} \sigma_ {t - 1} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) \leq \sum_ {b = 1} ^ {b _ {t}} e ^ {C} \sigma_ {t - 1, b - 1} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) \leq e ^ {C} \sqrt {b _ {t} \sum_ {b = 1} ^ {b _ {t}} \sigma_ {t - 1 , b - 1} ^ {2} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right)} \\ \leq e ^ {C} \sqrt {b _ {t} \sum_ {b = 1} ^ {b _ {t}} \frac {1}{\log \left(1 + \lambda^ {- 1}\right)} \log \left(1 + \lambda^ {- 1} \sigma_ {t - 1 , b - 1} ^ {2} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right)\right)} \tag {14} \\ = e ^ {C} \sqrt {C _ {2} b _ {t} \frac {1}{2} \sum_ {b = 1} ^ {b _ {t}} \log \left(1 + \lambda^ {- 1} \sigma_ {t - 1 , b - 1} ^ {2} (\boldsymbol {x} _ {t} ^ {[ b ]})\right)} \\ = e ^ {C} \sqrt {C _ {2} b _ {t} \mathbb {I} (f ; \boldsymbol {y} _ {t} | \boldsymbol {y} _ {1 : t - 1})}. \\ \end{array}
$$
The second inequality makes use of the Cauchy-Schwarz inequality, and the last equality follows from the definition of information gain.
The next Lemma gives an upper bound on the expected batch regret in iteration  $t$ :  $\min_{b \in [b_t]} r_t^{[b]} = \min_{b \in [b_t]} (f(\boldsymbol{x}^*) - f(\boldsymbol{x}_t^{[b]})$ .
Lemma B.7. For any filtration  $\mathcal{F}_{t - 1}$ , conditioned on the event  $E^{f}(t)$ , we have that
$$
\mathbb {E} \left[ \min  _ {b \in [ b _ {t} ]} r _ {t} ^ {[ b ]} \Big | \mathcal {F} _ {t - 1} \right] \leq c _ {t} e ^ {C} \left(1 + \frac {2}{p - 1 / t ^ {2}}\right) \mathbb {E} \left[ \sqrt {\frac {1}{b _ {t}} C _ {2} \mathbb {I} (f ; \boldsymbol {y} _ {t} | \boldsymbol {y} _ {1 : t - 1})} \Big | \mathcal {F} _ {t - 1} \right] + \frac {2 B}{t ^ {2}},
$$
in which  $r_t^{[b]} = f(\pmb{x}^*) - f(\pmb{x}_t^{[b]})$ .
Proof. To begin with, we define  $\overline{\boldsymbol{x}}_t$  as the unsaturated input at iteration  $t$  (after the first  $t - 1$  iterations) with the smallest (posterior) standard deviation:
$$
\bar {\boldsymbol {x}} _ {t} \triangleq \arg \min  _ {\boldsymbol {x} \in \mathcal {X} \backslash S _ {t}} \sigma_ {t - 1} (\boldsymbol {x}). \tag {15}
$$
Following this definition, for any  $\mathcal{F}_{t - 1}$  such that  $E^{f}(t)$  is true,  $\forall b\in [b_t]$ , we have that
$$
\begin{array}{l} \mathbb {E} \left[ \sigma_ {t - 1} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) \mid \mathcal {F} _ {t - 1} \right] \geq \mathbb {E} \left[ \sigma_ {t - 1} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) \mid \mathcal {F} _ {t - 1}, \boldsymbol {x} _ {t} ^ {[ b ]} \in \mathcal {X} \backslash S _ {t} \right] \mathbb {P} \left(\boldsymbol {x} _ {t} ^ {[ b ]} \in \mathcal {X} \backslash S _ {t} \mid \mathcal {F} _ {t - 1}\right) \tag {16} \\ \geq \sigma_ {t - 1} (\bar {\boldsymbol {x}} _ {t}) (p - 1 / t ^ {2}), \\ \end{array}
$$
Now we condition on both events  $E^{f}(t)$  and  $E^{f_{t}}(t)$ , and analyze the instantaneous regret as:
$$
\begin{array}{l} \min  _ {b \in [ b _ {t} ]} r _ {t} ^ {[ b ]} \leq \frac {1}{b _ {t}} \sum_ {b = 1} ^ {b _ {t}} r _ {t} ^ {[ b ]} = \frac {1}{b _ {t}} \sum_ {b = 1} ^ {b _ {t}} \Delta \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) = \frac {1}{b _ {t}} \sum_ {b = 1} ^ {b _ {t}} \left[ f \left(\boldsymbol {x} ^ {*}\right) - f (\overline {{\boldsymbol {x}}} _ {t}) + f (\overline {{\boldsymbol {x}}} _ {t}) - f \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) \right] \\ \leq \frac {1}{b _ {t}} \sum_ {b = 1} ^ {b _ {t}} \left[ \Delta \left(\overline {{\boldsymbol {x}}} _ {t}\right) + f _ {t} ^ {[ b ]} \left(\overline {{\boldsymbol {x}}} _ {t}\right) + c _ {t} \sigma_ {t - 1} \left(\overline {{\boldsymbol {x}}} _ {t}\right) - f _ {t} ^ {[ b ]} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) + c _ {t} \sigma_ {t - 1} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right) \right] \tag {17} \\ \leq \frac {1}{b _ {t}} \sum_ {b = 1} ^ {b _ {t}} \left[ c _ {t} \sigma_ {t - 1} (\overline {{\boldsymbol {x}}} _ {t}) + c _ {t} \sigma_ {t - 1} (\overline {{\boldsymbol {x}}} _ {t}) + c _ {t} \sigma_ {t - 1} (\boldsymbol {x} _ {t} ^ {[ b ]}) + f _ {t} ^ {[ b ]} (\overline {{\boldsymbol {x}}} _ {t}) - f _ {t} ^ {[ b ]} (\boldsymbol {x} _ {t} ^ {[ b ]}) \right] \\ \leq \frac {1}{b _ {t}} \sum_ {b = 1} ^ {b _ {t}} \left[ c _ {t} \left(2 \sigma_ {t - 1} (\overline {{\boldsymbol {x}}} _ {t}) + \sigma_ {t - 1} \left(\boldsymbol {x} _ {t} ^ {[ b ]}\right)\right) \right], \\ \end{array}
$$
in which the second inequality results from Lemma B.1 and Lemma B.2, the third inequality follows since  $\overline{\pmb{x}}_t$  is unsaturated, and the last inequality follows from the policy in which  $\pmb{x}_t^{[b]}$  is selected, i.e.,  $\pmb{x}_t^{[b]} = \arg \max_{\pmb{x} \in \mathcal{X}} f_t^{[b]}(\pmb{x})$ .
Now we separately consider the two cases where the event  $E^{f_t}(t)$  is true and false: